v3.0.0 Release Notes¶

Release Date: August 9, 2026 Previous Version: v3.0.0.b4 (February 8, 2026) Status: Stable

Executive Summary¶

v3.0.0 is the first stable release of the v3 line. It closes out a development cycle that began with v3.0.0.a1 in January 2025 and ran through four betas, and it lands roughly 1,400 merged pull requests since v3.0.0.b4 alone.

The headline of v3 is that a corpus is no longer just a bucket of documents. It is a queryable, self-describing, cross-linked body of work:

• [cite] — the v3 surface rebrand. Same MIT-licensed platform, new identity.
• Corpus Intelligence & CAML — corpora describe themselves. A corpus gets an auto-generated Readme.CAML article, a logo, an editorial home page, and live-data embeds (extract grids, reference graphs, Artifact posters).
• Reference enrichment & the Authority system — citations inside documents are extracted, normalized against a jurisdiction's citation grammar, and resolved to authority documents in other corpora. Authority packs ship out-of-repo and install with one command.
• Deep research — long-horizon agent runs with a durable living plan, a memory store, crash recovery, verified citations, and structured finding cards.
• Hybrid search everywhere — PostgreSQL full-text + pgvector HNSW, fused with Reciprocal Rank Fusion, with a global reranker and cross-corpus retrieval.
• Ingestion at scale — off-cluster remote-ingest workers, service accounts with corpus-scoped tokens, resumable chunked uploads, and REST import endpoints.
• A rebuilt foundation — the GraphQL layer moved from graphene to strawberry, authorization consolidated behind a single canonical API and a mechanically-enforced service layer, and mypy landed in pre-commit and CI.

Scale of the change since v3.0.0.b4: 3,541 files changed, 122 new database migrations, ~97 security fixes.

The complete, itemized record lives in CHANGELOG.md under [3.0.0]. This page is the curated tour.

Table of Contents¶

1. Corpus Intelligence and CAML
2. Reference Enrichment and the Authority System
3. Deep Research
4. Search and Retrieval
5. Ingestion at Scale
6. Documents, Formats and Storage
7. Runtime LLM Configuration
8. MCP and Agent Interoperability
9. Extracts and Analyzers
10. Permissioning and the Service Layer
11. Security
12. Platform and Architecture
13. Frontend and Mobile
14. Breaking Changes
15. Upgrade Notes

1. Corpus Intelligence and CAML¶

CAML (the Corpus Article Markup Layer) turns a corpus home page into a written article backed by live data rather than a dashboard of widgets.

• CAML Interactive Article System — articles are documents in the corpus, so they version, permission, and export like any other content.
• Auto-generated corpus identity — creating a corpus generates a Readme.CAML article and a corpus logo. Logos can be regenerated on demand via an agent.
• Corpus home editorial redesign — Collection Overview, a data story section, and shareable Artifact posters.
• Live embeds — extract grids and the reference graph render inline in articles; the extract-grid payload is bounded so a large extract cannot blow up the page.
• One canonical description — corpus description storage consolidated onto the Readme.CAML document body, with auto-maintained read-only caches, one write mechanism, and revision history. The GraphQL surface is unchanged.
• One-click collection-intelligence setup for an existing corpus.
• Citation review tooling — agent tools that walk a CAML article's citations one at a time.

2. Reference Enrichment and the Authority System¶

• Corpus reference enrichment agent — extracts citations from document text, normalizes them, and persists CorpusReference rows with provenance.
• Authority corpora + cross-corpus law linking — a citation in one corpus resolves to the actual authority document in another.
• Citation grammar coverage — federal, state, title-identifier, HTS tariff codes, and a Tier-2a municipal citation grammar with a known-code table, an open-vocabulary shape grammar, and precision guards.
• Authority packs — jurisdiction data (namespaces, aliases, providers, citation vocabulary, SSRF source-host allowlists) ships as a self-contained pack outside the repo. manage.py install_authority_pack installs one; AUTHORITY_PACK_ROOTS points at sideloaded packs; load_authority_pack --check validates without writing.
• Authority Console (/admin/authority) — five phases delivered: namespace management, relationship editing, the discovery queue, the wanted-authorities queue, and a runs tab.
• Agentic web locator + discovery providers — the frontier crawls for missing authorities, gated and rate-capped at the service layer.
• Reference-web graph explorer — a governanceGraph GraphQL query and a graph view; the document References side panel badges in-flight references as "In progress" and re-links reactively as enrichment completes.
• Enrichment runner — per-corpus and install-wide, with live WebSocket job status and provisional persistence so a long run survives interruption.

3. Deep Research¶

• Durable context management — a living plan, a memory store, and resume after crash, so a multi-hour run is not lost to a worker restart.
• Verified citations — finalize checks that each cited passage actually supports the sentence it is attached to, with a polarity guard, and refuses anachronistic citations.
• Finding cards — structured, validated output (including a second card shape for project readiness); unresolved_qualifications cannot be empty.
• Corpus Group targeting — a run can span a CorpusGroup, and group-scoped runs are told the cross-corpus retrieval tool exists.
• Reports open with the takeaway, not the methodology.
• Frontend + chat status — slug resolver, routing, views, notifications, and a chat status tool that reports progress mid-run.

4. Search and Retrieval¶

• Hybrid search — PostgreSQL full-text search fused with pgvector similarity via Reciprocal Rank Fusion.
• HNSW indexes on all embedding vector columns; Python-side materialization eliminated; pgvector extension upgraded to 0.8.0 with PostgreSQL tuning for vector workloads.
• Discover is hybrid across every category, plus a new Documents category and cross-content search.
• Global post-retrieval reranker for vector search.
• Multi-corpus retrieval — CorpusGroup + search_across_corpora, with a user-facing Corpus Groups management GUI.
• Structural context in results — OC_SUBTREE_GROUP relationships are materialized at ingestion and are first-class vector targets, with block-context augmentation on annotation hits and jump-to-relationship deep links.
• Expanded embedding dimensions (including 2048) and an OpenAI embedder pipeline component.

5. Ingestion at Scale¶

• Remote-ingest worker — parsing and enrichment offloaded to off-cluster hosts, including hardware-accelerated auto-detecting parser + embedder images.
• Service accounts — a new Django app with corpus-scoped access tokens, hashed token storage, a DRF authentication backend, and a Worker Account management page.
• REST upload + import endpoints (/api/imports/), a database-backed queue, a batch processor task, multi-queue routing, and pre-computed embedding storage.
• Chunked, resumable uploads for large files, end to end (models, REST endpoints, service layer, periodic cleanup, resilient frontend transport, GC race hardening).
• Bulk import — a bulk PDF import driver, chunked transport in the CLI, Auth0-deployment support via worker tokens, and sidecar imports that carry annotation-to-annotation relationships, link_url/data metadata that survives re-anchoring, and cross-batch document relationships.
• Deferred annotation import — producer annotations re-anchor onto the final PAWLs layer instead of being dropped.
• Gotenberg optional pre-parse conversion stage for non-core formats.
• Ingestion Monitor admin dashboard with Document Ingestion and Import Batches tabs for diagnosing failures.

6. Documents, Formats and Storage¶

• First-class DOCX support via the Docxodus microservice — parser, thumbnail generator, a dedicated DocxAnnotator frontend, and Compose wiring.
• Compact PAWLs v2 (~67% storage reduction) and compact annotation JSON v2 (~75% reduction).
• Document index — a within-document table of contents and an Index tab in the sidebar.
• Clickable links — native PDF link annotations are clickable in the viewer, and OC_URL link annotations anchor hyperlinks to highlighted text.
• Versioning & lineage — document lineage tracking, path history, a corpus versions field, ?v=N deep links, and a DocumentVersionSelector.
• Export — OPEN_CONTRACTS_V2 export format, content_modalities exported, and an export schema bump to V3.
• Pluggable text chunking for TxtParser and a BaseChunkedParser for PDFs, with chord fan-out capped to bound worker-pool pressure.

7. Runtime LLM Configuration¶

• DB-configurable provider credentials and endpoints — no redeploy to change models or keys; secrets are encrypted at rest and never exposed via GraphQL.
• BaseLLMProvider pipeline component with four shipped providers, wired into the standard PipelineComponentRegistry.
• Resolution chain — per-agent override → per-corpus preferred_llm → install default, with created_with_llm recorded for audit.
• Context guardrails — per-model context-window lookup, token estimation, conversation compaction with persisted bookmarks, tool-output truncation, and per-agent compaction configuration.
• In-run history compaction via pydantic-ai history_processors.
• Admin surfaces — per-corpus LLM picker, per-agent LLM picker, agent tool secrets (e.g. tool:web_search), and preferred_enrichers.
• A loud guardrail against the system_prompt= foot-gun (silently dropped when message history is non-empty) with a version-pinning regression test.

8. MCP and Agent Interoperability¶

• Authenticated MCP sessions and a create_thread_message write tool.
• Interactive sign-in for Claude web/desktop and ChatGPT — OAuth discovery endpoints, RFC 9728 path-based protected-resource metadata, and CORS for the MCP endpoints.
• Knowledge tools reworked for low friction — search_corpus returns a unified passage + block feed, list_annotations supports content search, a list_relationships tool was added, and get_document_text does bounded slicing.
• Rate limiting with identity resolution and rate categories, covering both WebSocket and MCP traffic.
• Privacy-preserving telemetry with bounded input validation.
• Agent chat — rich-mention delegation between corpora, an agent memory system, tool-usage badges and popovers, tool results in the timeline, and sub-agent approval propagation.

9. Extracts and Analyzers¶

• Extract iterations & cell-level diff — re-run an extract as a new iteration, then compareExtracts(extractAId, extractBId) for a cell-level diff, with a dedicated frontend.
• Extract & Analyzer agent tools — agents can dispatch configured extracts and analyses.
• Corpus-scoped analyzers, a PII scanner task-based analyzer with an auto-annotate agent tool, and HTS tariff-code analysis.
• Prompt-injection fencing on per-column constraint fields, and full-text injection for short documents.
• Action Library — a CorpusActionTemplate model, five default templates, an addTemplateToCorpus mutation, and a seed_action_templates command.

10. Permissioning and the Service Layer¶

• One canonical authorization API — Manager.user_can(user, obj, perm) and obj.user_can(user, perm), paired with visible_to_user and pinned to agree by an authorization-invariant test suite.
• Two-tier permission caching with request-scoped memoization.
• Service-layer centralization across six phases — every user-context GraphQL call site reaches models through opencontractserver/<app>/services/. Enforced twice: a pytest architecture test and a Django system check (opencontracts.E001) that fails manage.py startup on any inline use, with self-teaching failure messages.
• Scoped admin access — superusers are no longer omniscient over user data by default; the bypass is retained explicitly where it belongs.
• Relationship privacy recursion and privacy gates on aggregate/listing surfaces that previously skipped them.
• Documentation refresh — docs/permissioning/, docs/architecture/query_permission_patterns.md, and docs/development/architecture_invariants.md.

11. Security¶

Roughly 97 security-tagged changes landed in this release. Highlights:

• SSRF hardening in safe_fetch_bytes — DNS-pinned connections closing the DNS-rebind TOCTOU window, a CGNAT gap closed, an IPv4-mapped-IPv6 bypass closed, credentials stripped on cross-host redirects, fail-closed on empty DNS results, and redirect-loop robustness.
• IDOR elimination — an IDOR-safe get_for_user_or_none helper, mutation migration, existence-oracle removal across relationship/metadata-column/datacell mutations, and cross-user regression tests.
• ZIP import bounds — every unguarded member read (data.json, meta.csv, relationships.csv, sidecars) is now bounded through one consolidated implementation, with a per-sidecar size limit and schema validation.
• Auth0 permissioning audit — privilege-escalation, IDOR, and credential-storage hardening, including an allowlist for is_superuser claim sync, a tightened admin-claim cache window, bounded JWKS stale-cache fallback, and hashed Analysis.callback_token at rest.
• WebSocket authentication tokens no longer travel in URL query strings.
• Secret redaction — SECRET-typed parser settings redacted from logs; plaintext secrets rejected by updatePipelineSettings; exception strings no longer echoed from import REST views.
• Privacy — slug-only user policy across the GraphQL surface, extracts never anonymous-visible, and structural annotations no longer leak a private document through their shared structural set.
• Dependencies — outstanding Dependabot/yarn audit alerts resolved and the Django container base image bumped.

12. Platform and Architecture¶

• GraphQL migrated from graphene/graphene-django to strawberry-graphql. The query shape is pinned by a golden SDL contract (config/graphql/schema.graphql) and a parity test that fails on any drift.
• Typing — mypy wired into pre-commit and CI, with the auth, users, notifications, analyzer, shared, agents, badges, worker-uploads, and config.* packages graduated out of the baseline; a frontend any-baseline gate in CI.
• Testing & coverage — Codecov components for backend and frontend, an E2E extract workflow, VCR.py-recorded LLM calls, a traversal A/B benchmark harness, an external-RAG benchmark harness, and broad component-test coverage lifts.
• Docs screenshots — automated capture during Playwright component tests, pushed back to the PR branch on demand.
• Changelog fragments — per-PR files under changelog.d/, collated at release time, so concurrent PRs can no longer conflict on CHANGELOG.md.
• Materialized install statistics (SystemStats) and cached community stats.
• Dead-code removal — the deprecated user_has_permission_for_obj shim, the registry-bypassing SimpleLLMClient, the in-process CrossEncoderReranker, Semantic UI React (fully removed), and a large sweep of dead atoms, reactive vars, styled components, and GraphQL operations.

13. Frontend and Mobile¶

• v3 surface rebrand to [cite] — brand components, an /about page, chrome and meta, PWA and OG assets, and a rebranded social-card worker and discovery endpoints.
• Dedicated mobile layout for DocumentKnowledgeBase below 768px, plus a run of mobile UX fixes: annotations feed auto-switches to the Document tab on row tap, a type-to-launch mobile Ask bar with a separate history affordance, and a tappable Find-sheet results list.
• Desktop layout consolidation — the three scattered bottom-edge floating controls unified, and initial PDF zoom reserves a horizontal margin so fit-to-width never overflows on narrower laptops.
• Reading layer for CAML articles — long-form typography, contrast, and spacing tuned for prose rather than chat bubbles.
• Corpus Chat — message readability, scroll-to-bottom behavior, and the @-triggered agent mention picker in the inline chat bars.
• Runtime-configurable corpus categories with a superuser CRUD GUI.
• Markdown-customizable user profile fields, richer social-media link previews, and Creative Commons license support for corpuses.
• Decomposition — Documents.tsx split into a thin composer plus grid/list/ compact views and shared file utilities.

Breaking Changes¶

Upgrade Notes¶

Run migrations first. This release adds 122 migrations since v3.0.0.b4.

docker compose -f production.yml --profile migrate up migrate
docker compose -f production.yml up

Additional notes:

• pgvector extension 0.8.0 is expected by the vector indexes. The bundled Postgres image ships it; a self-managed database must be upgraded before the HNSW index migrations run.
• Authority packs are no longer in-repo. Installs that relied on the bundled packs should install them explicitly: manage.py install_authority_pack <pack> (or point AUTHORITY_PACK_ROOTS at a sideloaded pack directory).
• LLM configuration moved to the database. Environment-variable model config still boots, but the System Settings LLM registry is the runtime authority. Set provider credentials there after upgrading.
• DOCX support requires the Docxodus service. Add it from the Compose files if you want first-class .docx ingestion.
• Optional Gotenberg service enables pre-parse conversion for non-core file formats.
• See docs/migrations/v3_upgrade_guide.md for the v2 → v3 path.

Full changelog: CHANGELOG.md · Compare: v3.0.0.b4...v3.0.0