Looking Glass

Quickstart

Connect your backend so Looking Glass can evaluate model answers against your ground truth. This page covers architecture, your answers endpoint, and a demo MCP flow.

On this pageArchitectureAnswers EndpointDemo + MCPTroubleshooting

Architecture Overview

1. Create an Answers Endpoint

Expose an HTTP endpoint that returns ground-truth values for a given entity and schema. The backend calls this endpoint with context so you can locate the correct record.

# Request shape (example)
GET https://your-backend/retrieve
Content-Type: application/json

{
  "task_id": "TASK-12345",
  "session_id": "1234",
  "form_schema": {
    "type": "object",
    "properties": {
      "full_name": {
        "type": "string"
      },
      "account_opened": {
        "type": "string",
        "format": "date"
      }
    },
    "required": ["full_name", "account_opened"]
  }
}

Respond with a JSON object keyed by field name containing the authoritative answers. Looking Glass compares these against model predictions during evals.

# Response shape (example)
HTTP/1.1 200 OK
Content-Type: application/json

{
  "full_name": "Jane Doe",
  "account_opened": "2020-04-18"  # ISO-8601 date string
}

Tip: Match the value types to the provided schema (e.g., strings for text fields, ISO dates for date fields).

2. Demo Backend + MCP Flow

Here's a demo backend that queries a Postgres database via an MCP tool.

# app/main.py
from __future__ import annotations

from typing import Any, Dict, List, Optional

from fastapi import FastAPI
from pydantic import BaseModel

from app.agent import get_answers

app = FastAPI(title="Looking Glass Demo Backend", version="0.1.0")

def get_answers(*, task_id: Optional[str], form_schema: Any) -> dict:
    answers: Dict[str, Any] = {}

    # Your MCP server URL (replace with your actual URL)
    mcp_url = "https://0c5dc9c6f332.ngrok-free.app"

    client = OpenAI()

    resp = client.responses.create(
        model="gpt-5-mini",
        tools=[
            {
                "type": "mcp",
                "server_label": "supabase",
                "server_url": f"{mcp_url}/sse",
                "require_approval": "never",
            },
        ],
        input=f"""
        Retrieve the values for the fields using the MCP server.
        
        You are trying to fill out a form with the following schema:
        {form_schema}
        
        The database contains a lot of information with many different users' data. You should only retrieve the records associated with the task id "{task_id}".

        Return the values in the same schema as the form schema.
        """,
        text={
            "format": {
                "type": "json_schema",
                "name": "answers",
                "strict": True,
                "schema": form_schema,
            }
        },
    )

    answers_text = resp.output_text
    answers = json.loads(answers_text)

    return answers


class ForwardPayload(BaseModel):
    page_id: int
    task_id: Optional[str] = None
    form_schema: Any


@app.get("/health")
def health() -> Dict[str, str]:
    return {"status": "ok"}


@app.post("/")
def respond(payload: ForwardPayload) -> Dict[str, Any]:
    answers = get_answers(task_id=payload.task_id, form_schema=payload.form_schema)
    return answers

Run it locally:

cd demo
uv run uvicorn app.main:app --reload --port 8020 --host 0.0.0.0

# Option A: local MCP tool server
uv run postgres-mcp "<POSTGRES_CONNECTION_STRING>" \
--sse-port 8021 --transport sse \
--access-mode restricted

# Option B: dockerized MCP tool server (example)
docker run -p 8021:8000 \
-e DATABASE_URI=postgresql://<user>:<pass>@<host>:<port>/<db> \
crystaldba/postgres-mcp --access-mode=unrestricted \
--transport=sse

# expose your endpoint to the extension
ngrok http 8020

The demo’s POST / endpoint accepts a filing payload and returns candidate answers keyed by field id (if present) otherwise by name/label.

Troubleshooting


Next: Evaluate — run evals and monitor performance.