CLI reference — wordsworth

wordsworth (alias wordsworthctl) is a dependency-free client for the Wordsworth HTTP API. It lives in src/wordsworth/client.py and uses only the Python 3 standard library, so it runs on any machine with Python 3 — you do not need to install the wordsworth package or its dependencies. This makes it the tool of choice on a machine that only reaches the API over the network (e.g. to ingest a corpus).

Install

Put it on PATH as wordsworth (and wordsworthctl):

scripts/install-cli.sh [--url <api-base-url>] [--bin-dir <dir>]

• --url bakes a default API base URL into the installed copy, so wordsworth health works with zero configuration. The WORDSWORTH_API_URL environment variable still overrides it.
• --bin-dir defaults to ~/.local/bin (ensure it is on your PATH).

No package install is required — the script just copies the single stdlib file. If the wordsworth package is installed (uv sync / pip install), the wordsworth and wordsworthctl console scripts are provided too.

Configuration

Set the API URL once, persistently, so you don't repeat --url:

wordsworth config --url http://100.100.181.23:8000    # also --batch, --timeout
wordsworth config --show                              # print current config

This writes ~/.config/wordsworth/config.yaml (override the path with $WORDSWORTH_CONFIG) — a flat key: value file (url, batch, timeout), parsed with the standard library (so still no dependencies; it is a small YAML subset, not full YAML). install-cli.sh --url writes it for you.

The API base URL is resolved in this order:

1. --url <url> on the command line
2. $WORDSWORTH_API_URL
3. url in the config file
4. built-in default http://localhost:8000

--batch / --timeout resolve as: flag → config file → built-in default.

Commands

Every console script in pyproject.toml is started in its own interpreter by tests/test_entry_points_import.py. That test exists because an import cycle kept wordsworth-ingest (the batch-ingest Job) from starting between 2026-09-22 and 2026-09-24, invisible to a test suite that runs in one process.

Command	Description
wordsworth health	Check the API is up (GET /health).
wordsworth ingest <path>	Upload a PDF file, or every PDF under a directory, to POST /ingest.
wordsworth search <query> [--size N]	Lexical (BM25) search (GET /search).
wordsworth hybrid <query> [--size N]	Hybrid BM25 + vector relevance search (GET /hybrid).
wordsworth ask <query> [--k N]	RAG answer with citations via the local LLM (GET /ask).
wordsworth state <document-id>	Pipeline state of a document (GET /documents/{id}/state).
wordsworth meta <document-id>	Full metadata: duration, PII counts, step trail (GET /documents/{id}).
wordsworth config [--url … --batch … --timeout …] [--show]	Show or set persistent defaults.

ingest

wordsworth ingest <file-or-directory> [--all] [--batch N] [--timeout SECONDS]

• Walks a directory recursively; by default only *.pdf, or --all for every file.
• Uploads in batches of --batch files per request (default 25). On slow (CPU-only) deployments use a smaller batch, e.g. --batch 5 or --batch 1, so each request stays short.
• Prints a per-file line — filename: state, with the processing duration and non-zero PII counts on success (e.g. 2130276.pdf: indexed (1.8s, bsn=1 person=2)), or the error class on failure — and a final X/Y indexed, Z failed summary. A file that fails does not abort the batch. Exit code is non-zero if any file failed.
• The pipeline is PDF-only; non-PDF files come back as error.

wordsworth-ingest

The batch-ingest Job (src/wordsworth/ingest_corpus.py), not the HTTP API.

wordsworth-ingest --corpus-dir ./corpus --dossier "zaak-a"
wordsworth-ingest --corpus-dir ./corpus --dossier "zaak-a" --manifest ./corpus/manifest.jsonl

--manifest is the stricter, provenance-proving path (see docs/reference/ingest.md): an unlisted or sha256-mismatched file aborts the whole run before anything is ingested, each document's provenance is read back from the audit record after it is ingested, check.json next to the manifest must pass, and WORDSWORTH_BACKEND_ENV=homelab is required. Without --manifest, behaviour is unchanged.

wordsworth-dossier-uit-herkomst and wordsworth-dossier-hernoem

Correcting a classification, without guessing.

wordsworth-dossier-uit-herkomst herkomst.jsonl --weg-uit "corpus-2026-09" --dry-run
wordsworth-dossier-hernoem "corpus-2026-09" "Gooise Meren Woo-publicatie 2022"

uit-herkomst puts each document into the dossier its recorded provenance names — scripts/eval/fetch_woo_corpus.py writes one line per document with the source URL and the decision it belongs to. A document with no provenance entry is not assigned and is counted. A classification derived from a date or a text pattern is one nobody can retell afterwards, and that is worse than none.

Both commands update the search index, because the dossiers live there per document: a membership the index does not know about is a document a scoped search cannot reach. They update only that field — writing the whole document back drops the embedding of any caller who forgets vector=, which is exactly what cost 770 documents their vectors on 2026-09-18.

A document that is not in the index at all is reported, not created. It never got through the straat, and indexing it here would bury that under this.

--weg-uit removes them from a dossier they were wrongly placed in, once they are somewhere else. If that leaves documents belonging nowhere at all, the count is reported — such a document is invisible to every scoped search, and the moment it happens is the moment someone can still act on it.

hernoem moves no document: the identity is the dossier, not the word used for it.

The provenance file may contain duplicate lines — the fetcher opens it with append, so running it twice duplicates every entry. Both commands key on the filename, so that is harmless; it is only a reason not to count lines.

wordsworth-access-preflight

Reports the grants that go inert once callers are identities instead of key labels, and changes nothing.

wordsworth-access-preflight

A grant names who may reveal and the caller must be that recipient. A recipient that is a key label stops matching anybody the moment callers are people. Run this before switching, not after — see access identity.

wordsworth-backfill-dossier

Places documents that predate dossiers into one named dossier, so a scoped search can still reach them.

wordsworth-backfill-dossier "corpus-2026-09" --dry-run
wordsworth-backfill-dossier "corpus-2026-09"

A scoped search answers only from the dossiers in scope, so documents belonging to none are invisible — and every document ingested before dossiers existed belongs to none. A scope that makes the existing corpus unfindable is not a migration but a loss.

Name them for where they came from rather than pretending they were a case. They arrived as one corpus, in one go.

It also updates the index, because the index holds the dossiers per document: a membership the index does not know about is a document a scoped search still cannot reach, and then the command would not have done the one thing it exists for. --no-index skips that and says plainly that a reindex stays due.

--dry-run does not touch the index. The database can be rolled back; the index cannot. The first dry run against production wrote 770 documents before this was fixed, which made "dry" a lie.

Run it after deploying the new code, not before: in between, the existing documents are in no dossier while the new code already requires a scope.

wordsworth-dedupe

Retires every document that is a copy of an object another document already is (one-document-per-object). The oldest registered copy of each object survives; the others become superseded, leave the search index and their dossiers (each removal recorded, actor dedupe), and pass on any dossier only they were in.

wordsworth-dedupe                                   # dry run: what it would do
wordsworth-dedupe --apply --expect-copies 173 --expect-memberships 173 \
                  --expect-index-entries 159

--apply refuses to start without the numbers you expect, and stops before changing anything if the database says otherwise: a cleanup that finds more than it was sent for has found something nobody looked at. Index entries can only be counted by removing them, so that number is checked afterwards and a difference sets exit code 1. It commits one object at a time, so it never holds a long transaction on documents. A rerun finds nothing left to do.

Since release 2 the database refuses a second live document for the same bytes (uq_documents_live_object_key), so on a current database a dry run finds nothing. On one that still holds copies, wordsworth-init fails when it tries to create that index: run this command first.

A retired copy is not deleted: its audit trail requires the row. It still answers GET /documents/{id}/state with superseded and superseded_by, and reveal on it is refused with the survivor's id. Actually deleting a document — removing it, not retiring it — is wordsworth-delete, below.

wordsworth-delete

Removes a document's content: the stored object, its text, its pseudonyms, its mappings, its search-index entry and its grants. The documents row and its audit trail stay — the trail is append-only, the same reason a superseded copy keeps its row. There is no undo.

wordsworth-delete <document-id> [<document-id> ...] --actor <who> [--dry-run]

--actor is required and is checked the same way the API checks a caller: only role beheerder (held via an active, unexpired grant naming it) or a label in WORDSWORTH_GRANT_ISSUER_LABELS may delete. Any other actor is refused and nothing is removed.

--dry-run reports the counts it would remove, using the exact same code path, and stops before the first removal — no store, index or database change, and no deleted record. Each named document commits on its own, so a failure partway through a multi-document run leaves only the untouched ones for a rerun to finish; removing what a previous run already removed is a no-op.

--dossier

wordsworth-delete --dossier <dossier-id> --actor <who> [--dry-run]

Deletes a dossier instead of named documents — mutually exclusive with the positional document ids. Removes the dossier's memberships, its topics and the dossier row; a member document is deleted only if it belongs to no other dossier, otherwise only its membership of this one goes. --actor and --dry-run are checked and behave the same way as above; the dry run reports which documents would be deleted, which would only be released, and the number of topics (topics) that would be removed; the output of a real run carries the same topics count.

See deleting documents for what goes, what stays, and DELETE /documents/{id} / DELETE /dossiers/{id} over the API.

wordsworth-backfill-passages

Gives every indexed document a vector per passage (200-word windows every 150 words, wordsworth.passages), so hybrid search can rank a document by its best passage instead of one vector for all of it (change long-documents-rank-fairly).

wordsworth-backfill-passages            # --workers 2 (one per Ollama instance)

New and reprocessed documents get passages when they are indexed; this fills in the ones indexed before. It only sets the passages field (a partial update), never the text or the document vector. It asks the index what is still missing, so a stopped run resumes. A document with no text, or whose embedding fails after the retry budget, is set aside and counted; exit code 1 if any failed.

Switch search over only when it is done: WORDSWORTH_HYBRID_KNN=passage. A document without passages is invisible to the passage kNN half.

wordsworth-backfill-filenames

Gives existing documents back the name their file arrived under, by content hash.

wordsworth-backfill-filenames /data/corpus --dry-run
wordsworth-backfill-filenames /data/corpus

Ingestion has always been content-addressed (documents/<sha256>) — the right identity, and unreadable. Documents ingested before the filename column existed have no name in the database, but the bytes on disk still do, and the hash is the bridge.

It does not guess. A document whose bytes are not in the directory keeps no name, and the console then says naamloos (<first 8 of the hash>) rather than printing a hash where a name belongs. A name already recorded is kept — that one came from the caller at ingest time, which beats a file that happens to sit in a directory today; --overwrite says otherwise.

Reports four numbers: named, kept, documents without a file, files without a document.

wordsworth-measure-combinations

Counts, per declared combination, in how many documents every type occurs. Measure before claiming: without this number, any statement about quasi- identifiers in a corpus is a guess.

wordsworth-measure-combinations profiles/example-wi.json

It accepts a profile (it reads the combinations block) or a bare JSON list of {types, reason} declarations. Output is one line per combination:

    12  BSN + POSTCODE
     0  DATE + GENDER + POSTCODE  [unobservable: DATE, GENDER]

Two things to read carefully.

The count comes from the pseudonyms the pipeline minted per document (document_pseudonyms), not from re-running the detectors over the source text. That makes it a statement about what the pipeline did, on the text it saw.

unobservable names types the corpus carries nowhere, because the pipeline has no detector for them. "Does not occur" and "cannot be seen" are different answers, and a bare zero gives the reassuring one.

Exit code 2 if the file declares no combinations.

Examples

# one-off, explicit URL
wordsworth --url http://100.100.181.23:8000 health

# ingest a corpus directory in small batches
wordsworth --url http://100.100.181.23:8000 ingest /data/corpus --batch 5

# search and inspect
wordsworth search "vergunning" --dossier zaak-a --size 5
wordsworth state 66c86e91-830f-4ed7-99cf-ac4407d262fb

# how often does a declared combination actually occur?
wordsworth-measure-combinations profiles/example-wi.json

See also the live, interactive API docs at /docs (Swagger) and /redoc on a running instance.