# Postgres metrics and the runs migration handle large databases

> Agno's PostgresDb and AsyncPostgresDb now refresh metrics for days with tens of thousands of sessions, surface a failed async refresh as an error, and run the sessions-to-runs migration in flat memory.

- Published: 2026-10-20
- Author: Sannya Singal
- Categories: Changelog
- Canonical: https://www.agno.com/blog/postgres-metrics-and-the-runs-migration-handle-large-databases
- Markdown: https://www.agno.com/blog/postgres-metrics-and-the-runs-migration-handle-large-databases.md

Agno's `calculate_metrics()` on `PostgresDb` and `AsyncPostgresDb` now finds runs with `IN (SELECT session_id FROM sessions ...)`, so the query binds the same few parameters for ten sessions or a hundred thousand. Earlier releases bound one parameter per session id. psycopg allows 65,535 parameters per query and asyncpg allows 32,767, so a refresh broke once its window held more than 65,533 sessions on psycopg or 32,765 on asyncpg.

We seeded one day of sessions into an embedded Postgres and called the method on this release and the one before it.

```text
# real output, PostgresDb (psycopg), 65,534 sessions on one day

Before
calculate_metrics() raised OperationalError: (psycopg.OperationalError) sending query and params failed: number of parameters must be between 0 and 65535

Now
calculate_metrics() -> agent_sessions_count=65,534 agent_runs_count=65,534 model_metrics=[{'count': 65534, 'model_id': 'gpt-5.5', 'model_provider': 'OpenAI'}]
```

## Failed async refreshes report the error

`AsyncPostgresDb` used to catch any error while reading sessions for metrics and hand back an empty list. AgentOS then answered `POST /metrics/refresh` with `200` and `[]`, and `agno_metrics` stayed empty with nothing to say why. With 32,766 sessions on asyncpg, the only trace was a log line reading `the number of query arguments cannot exceed 32767`.

That read now raises. A synchronous refresh returns `500` with the database error, and a background refresh marks itself `failed` on `GET /metrics/refresh/status` with the error attached. The first session date also comes from a direct `MIN(created_at)` query, so a failure there surfaces too. The lazy refresh inside `get_metrics()` still logs and moves on, so a broken refresh never breaks a metrics read.

```text
# real output, AgentOS over AsyncPostgresDb, runs table dropped after startup
Before  POST /metrics/refresh -> 200 []
Now     POST /metrics/refresh -> 500 {"detail":"Error refreshing metrics: ... does not exist ..."}
```

## The runs migration streams from Postgres

The migration that moves each session's legacy `runs` column into the runs table now opens a server-side cursor on `PostgresDb` and `AsyncPostgresDb`. Its loop already called `fetchmany(50)`, but the driver had buffered every row before the first batch, so a large `sessions` table could exhaust memory before a single run was copied. Rows now stay in Postgres until the loop asks for the next 50 sessions.

For the migration, a script seeded legacy sessions with 20 runs each, ran `MigrationManager(db).up(table_type="sessions")` in a fresh process, measured peak memory growth, and counted open cursors on the sessions query each time a batch of runs was inserted.

| Legacy runs JSON | Adapter           | Peak memory before | Peak memory now |
| ---------------- | ----------------- | ------------------ | --------------- |
| 303 MB           | `PostgresDb`      | +348 MB            | +24 MB          |
| 303 MB           | `AsyncPostgresDb` | +361 MB            | +18 MB          |
| 1,009 MB         | `PostgresDb`      | +1,097 MB          | +32 MB          |
| 1,009 MB         | `AsyncPostgresDb` | +1,159 MB          | +19 MB          |

Both releases copied all 100,000 runs from the larger table in about eight seconds. Earlier releases showed zero open cursors at every insert, and this release showed one, the sessions query still being read. MySQL and SingleStore keep their existing read for now, and one very large session row still loads whole, since the cursor streams by session.

See the [cookbook](https://github.com/agno-agi/agno/blob/main/cookbook/05_agent_os/13_observability/metrics.py), and learn more about [refreshing AgentOS metrics](https://docs.agno.com/examples/agent-os/observability/metrics) and [migrating your Agno database](https://docs.agno.com/other/v3-migration#migrating-your-agno-db) in the documentation.