--- title: Introspection description: Self-inspection commands that let an agent understand an OMGDB database without guessing — describe, inspect, dump, explain, diagnose, and verify. --- OMGDB is built so an agent can understand a database it has never seen before, without guessing. The introspection surface answers five questions directly: *what collections and fields exist*, *what are the write rules and indexes*, *how many documents are there*, *what is the canonical state*, and *why does a query behave the way it does*. Every command reads through the public store API and produces output meant to be consumed by a human or an agent. `describe` and `inspect` report structure; `dump` exports state; `explain` and `diagnose` explain query behavior; `verify` proves the store is internally consistent. All of them operate on a store directory passed as the first positional argument (the examples use `app.omgdb`). ## describe — a live Markdown manual ```sh omgdb describe app.omgdb ``` `describe` renders a Markdown "manual" of the whole database. The database name is taken from the store directory's file name (falling back to the literal `database` if it can't be resolved). The output contains, in order: - A heading `# Database: ` followed by a blank line. - A line ` collection(s).` - For each collection: a `## ( document(s))` heading, an inferred-schema table, the collection's **indexes**, its **validation spec**, and one sample document. The schema table has three columns — `field`, `types`, and `present` — and one row per top-level field. The `present` column is a `/` ratio showing how many of the sampled documents contained that field. The indexes and validation spec are the collection's **write contract**: what an agent must know *before* inserting — which writes will be rejected (validation rules, unique constraints) and which queries will be fast (indexes). When a collection has at least one document, a `Sample:` line is followed by a fenced ```json``` block containing the first document rendered as canonical JSON. ````markdown # Database: mydb 1 collection(s). ## users (2 document(s)) | field | types | present | |-------|-------|---------| | _id | objectId | 2/2 | | age | long | 1/2 | | email | string | 2/2 | | name | string | 2/2 | Indexes: - `email` — unique Validation spec (enforced on writes): ```json {"required":["email"],"fields":{"email":"string"}} ``` Sample: ```json {"_id":{"$oid":"66f0a1b2c3d4e5f6a7b8c9d0"},"name":"ana","email":"ana@x","age":30} ``` ```` Index lines carry their qualifiers inline: compound indexes list their fields (`` `tenant, email` ``), unique indexes append `— unique`, and partial indexes append their filter (`— partial filter: {...}`). See [schema validation](/docs/schema-validation/) for the spec format and [indexes](/docs/indexes/) for index semantics. > **Note:** Vector embeddings live in ordinary `.__vectors` sidecar collections, so they appear in `describe` and [inspect](#inspect--collections-and-counts) like any other collection — see [vector search](/docs/vector-search/). An empty collection still prints its table header, but with no rows and no sample block. ### describe_json — the machine-readable form The library's `describe_json` returns the same information as canonical JSON — `{"collections":[{name, count, fields:[{name,types,present}], indexes:[{fields,unique,partialFilter?}], validation:{...}}]}` — for callers that want to consume the manual programmatically rather than parse Markdown. (`indexes` and `validation` appear only when the collection has them.) ## Schema inference: types and presence The schema table comes from a scan over the collection. For each document, every **top-level** field is recorded: the distinct value types observed, and a presence counter. The result per field is: | Column | Meaning | |--------|---------| | `field` | The top-level field name. | | `types` | The distinct type-name tokens observed, sorted and de-duplicated. A field seen as both a number and a string lists both. | | `present` | How many sampled documents contained the field, over the total document count. | Type tokens are MongoDB-style names (the same ones used by the [`$type` query operator](/docs/query-operators/)): `null`, `bool`, `long` (an i64 integer), `double` (an f64 float), `string`, `binData` (bytes), `array`, `object`, `objectId`, and `date`. Note that integers report `long`, not `int`. > **Limitation:** Schema inference is top-level only. Nested object fields and array element fields are not descended into, so a field whose value is an embedded document appears as a single `object` type with no breakdown of its inner keys. ## inspect — collections and counts ```sh omgdb inspect app.omgdb omgdb inspect app.omgdb --json ``` `inspect` is the lightweight counterpart to `describe`: it lists every collection and its document count, without sampling fields. By default it prints one text line per collection in the form `: docs` (or `(empty store)` when there are no collections). With `--json` it emits a machine-readable object as canonical JSON: ```json {"collections":[{"name":"users","count":2}]} ``` | Flag | Description | |------|-------------| | `--json` | Emit `{"collections":[{name,count}]}` as canonical JSON instead of text lines. | Because vector embeddings are stored as ordinary collections, a synced field shows up here as a sidecar — for example a `docs` collection synced via `vsync` produces a `docs.__vectors` entry visible in `inspect`. ## dump — deterministic canonical export ```sh omgdb dump app.omgdb ``` `dump` produces a deterministic, line-oriented export of the entire logical state. It writes one line per document, formatted as the collection name, a tab, then the document as canonical JSON: ```text \t ``` Documents are emitted in collection iteration order, then in `_id` order within each collection. The output is stable across runs: two successive `dump` calls on an unchanged store are byte-for-byte identical. This determinism is the basis of invariant I3 — a property that lets you snapshot, diff, and compare database states reliably (see [transactions and durability](/docs/transactions/) for the broader integrity model). ```text c {"_id":1,"v":"a"} c {"_id":2,"v":"b"} ``` ## explain — the query plan ```sh omgdb explain app.omgdb users '{"name":"ana"}' ``` `explain` compiles a MongoDB-style filter and reports, in plain language, how [`find`](/docs/query-operators/) will execute it. The filter argument is a JSON string. The possible plans: | Plan string | When it applies | |-------------|-----------------| | `index scan: equality on \`.\` (secondary index, candidate(s)), then filter` | A top-level equality predicate is served by a single-field index. | | `index range scan: range on \`.\` (secondary index, candidate(s)), then filter` | A top-level range predicate (`$gt`/`$gte`/`$lt`/`$lte`) with scalar bounds is served by a single-field index. | | `compound index scan: equality on \`.()\` (secondary index, candidate(s)), then filter` | Equality predicates cover a compound index's fields. | | `compound index range scan: optional equality prefix and range(s) on \`.()\` (secondary index, candidate(s)), then filter` | An equality prefix plus range predicate(s) fit a compound index. | | `full collection scan of \`\` ( documents), then filter` | No usable index; `N` is the live document count. | Every index plan names the index, reports how many candidate documents the index narrows to, and ends with "then filter" — index scans return a candidate superset that is always re-filtered for exact semantics. When the query falls back to a full scan **and** it contains a top-level equality on a field that is not indexed, the plan appends a self-repair hint: ```text full collection scan of `users` (1000 documents), then filter; no index on `name` — suggest `omgdb create-index users name` ``` The hint names the exact command to create the index that would accelerate the query. For broader, machine-readable recommendations (including sort-aware compound shapes), `omgdb suggest-indexes` analyzes a filter and proposes index definitions the planner can actually use. See [indexes](/docs/indexes/) for which predicates are eligible for index acceleration. ## diagnose — the why-not debugger ```sh omgdb diagnose app.omgdb users '{"age":{"$gte":25}}' ``` `explain` tells you *how* a query runs; `diagnose` tells you *why* it returns what it returns. It is a "why-not" debugger: for each top-level field predicate it counts how many documents satisfy that predicate **alone**, so an agent can immediately see which condition is the limiting (or eliminating) one. The output is canonical JSON with this shape: ```json { "collection": "users", "totalDocuments": 100, "matched": 12, "predicates": [ { "field": "role", "matched": 80 }, { "field": "age", "matched": 12 } ] } ``` The fields are: | Field | Meaning | |-------|---------| | `collection` | The namespace diagnosed. | | `totalDocuments` | Total documents in the collection. | | `matched` | How many documents satisfy the **full** filter. | | `predicates` | One entry per top-level field predicate, with the field name and its standalone match count. | When a single predicate matches **zero** documents, that entry additionally reports the field's observed value range — `observedMin` and `observedMax` under the engine's total value order — so the agent can see how far the predicate's threshold is from any real data: ```json // filter: {"role":"admin","age":{"$gt":90}} { "collection": "users", "totalDocuments": 2, "matched": 0, "predicates": [ { "field": "role", "matched": 2 }, { "field": "age", "matched": 0, "observedMin": 30, "observedMax": 40 } ] } ``` Here the report makes the failure obvious: `role` alone matches both documents, but `age > 90` matches none — and the observed range (30–40) shows the threshold is far above any stored value. > **Tip:** Logical operators are not diagnosed per field. Any top-level key beginning with `$` (such as `$or`, `$and`, `$nor`, `$expr`) is skipped in the `predicates` list, so `diagnose` is most useful on the conjunction of field conditions that makes up the body of a filter. ### Missing collections fail loudly A typo in a collection name can never produce confidently wrong advice. `explain`, `diagnose`, and `suggest-indexes` **error** when the collection does not exist, naming the nearest match and the collections that do: ```text omgdb explain app.omgdb userz '{"name":"ana"}' // error: collection `userz` does not exist — did you mean `users`? (existing: users) ``` `find`, by contrast, keeps MongoDB's empty-result semantics — querying a missing collection prints nothing and succeeds — but it still warns on **stderr** (``warning: collection `userz` does not exist — did you mean `users`?``), so an agent parsing stdout sees clean output while the likely typo is surfaced on the diagnostic stream. ## verify — proving the store is consistent ```sh omgdb verify app.omgdb ``` `verify` is introspection turned on the engine itself: it re-reads the op-log and proves that replaying it reproduces the live state — every record, every document, every collection — and then attests the five derived cache sidecars against the log: the primary `_id`, live-document, secondary-index, and metadata caches each get their contents checked *and* their **checkpoint + tail-replay proof** verified (that the persisted checkpoint plus the log tail beyond it equals a full replay — the boundary recorded by the fifth sidecar, the open-checkpoint proof). At 50,000 documents the whole proof takes about a second. ```text OK: 50000 record(s), 50000 document(s) in 3 collection(s); log reproduces state metadata cache: OK (2 collection spec(s), 3 index definition(s), derived) metadata checkpoint+tail: OK (0 tail record(s), derived) primary _id cache: OK (50000 entry(s), derived) primary _id checkpoint+tail: OK (0 tail record(s), derived) secondary index cache: OK (3 index(es), 41 bucket(s), derived) secondary index checkpoint+tail: OK (0 tail record(s), derived) live document cache: OK (50000 document(s), derived) document checkpoint+tail: OK (0 tail record(s), derived) ``` A stale or malformed cache is a **warning**, not a failure — every sidecar is derived and rebuildable, so the only fatal outcome is `INCONSISTENT: replaying the log does not reproduce the live state`. See [architecture](/docs/architecture/) for the cache design and [transactions](/docs/transactions/) for the invariants verify enforces. ## Structured errors and did-you-mean suggestions Introspection extends into error reporting. When a filter passed to `find`, `explain`, or `diagnose` uses an operator OMGDB does not recognize, the compiler does not fail silently — it returns an `UnknownOperator` error naming the bad token and, when a close match exists, a deterministic "did you mean" suggestion. The suggestion is the closest known operator within a Levenshtein edit distance of 2. ```json {"age":{"$gtee":1}} // error: $gtee (did you mean `$gte`?) ``` This lets an agent self-repair a malformed query in a single step rather than guessing. The known query operators are: `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$in`, `$nin`, `$exists`, `$type`, `$not`, `$and`, `$or`, `$nor`, `$expr`, `$size`, `$all`, `$mod`, `$elemMatch`, and `$regex` — see [query operators](/docs/query-operators/) for their full semantics. The same did-you-mean mechanism covers [update operators](/docs/update-operators/) (`$sett` suggests `$set`), [aggregation](/docs/aggregation/) stages, accumulators, and expression operators, and even `$type`'s type names (`strnig` suggests `string`). ## Putting it together A typical agent workflow uses these commands in sequence: ```sh # 1. Learn the shape of an unfamiliar database — schema, indexes, write rules. omgdb describe app.omgdb # 2. Get a quick count of every collection. omgdb inspect app.omgdb --json # 3. Write a query; check how it will run. omgdb explain app.omgdb users '{"age":{"$gte":25}}' # 4. If it returns nothing unexpected, find out why. omgdb diagnose app.omgdb users '{"age":{"$gte":25}}' # 5. Snapshot the exact state for diffing. omgdb dump app.omgdb # 6. Prove the whole store is consistent. omgdb verify app.omgdb ``` Together these turn a database from an opaque blob into a self-describing artifact: an agent can read its structure and write contract, count its contents, reason about query plans, debug empty result sets, capture a deterministic snapshot, and prove the state — all without prior knowledge of the schema.