OpenContracts v3.0.0.b4 Release Notes¶

Release Date: January 2026 Previous Version: v3.0.0.b3

Executive Summary¶

v3.0.0.b4 introduces corpus-isolated structural annotations and multimodal embedding support. This release prioritizes data isolation and prepares the platform for advanced AI capabilities including image understanding.

Key changes: - Corpus-Isolated Structural Annotations - Each corpus maintains independent structural annotation sets - Multimodal Embeddings - Support for image tokens and visual content in documents - Multi-Embedder Architecture - Each corpus can use different embedding models

Breaking Change: Structural annotations are no longer shared across corpus copies. See Migration Notes below.

Table of Contents¶

1. New Features
2. Corpus-Isolated Structural Annotations
3. Multimodal Embedding Support
4. Architecture Changes
5. Migration Notes
6. Bug Fixes

New Features¶

1. Corpus-Isolated Structural Annotations¶

Structural annotations (headers, sections, paragraphs, tables, figures) are now duplicated per corpus instead of shared across corpus copies.

Why This Change?¶

The previous shared approach created complications for: - Multi-embedder support: Each corpus can use different embedding models, requiring consistent per-corpus vector spaces - Data isolation: Shared data across corpus boundaries complicated permission models and lifecycle management - Vector search: Shared annotations required complex queries to select the right embedding for each corpus's embedder

How It Works Now¶

Document uploaded to Corpus A:
  → Document #1 created
  → StructuralAnnotationSet #1 created (content_hash: "abc123_corpus_a_id")
  → 5000 structural annotations linked to Set #1

Same document added to Corpus B:
  → Document #2 created (corpus-isolated copy)
  → StructuralAnnotationSet #2 created (content_hash: "abc123_corpus_b_id")
  → 5000 structural annotations DUPLICATED to Set #2

Result: Complete isolation between corpuses

Key Benefits¶

Trade-offs¶

• Storage: O(n) storage for structural annotations where n = number of corpus copies
• Processing: Document parsing may occur multiple times for same content

2. Multimodal Embedding Support¶

Documents can now include image tokens for visual content understanding.

What's New¶

• Image Token Model: Unified representation for images in PAWLs format
• Multimodal Embedders: Support for embedding models that process both text and images
• Content Modality Enum: Explicit tracking of TEXT, IMAGE, and MULTIMODAL content types
• Image Retrieval Tools: LLM agents can now retrieve and analyze document images

Image Token Format¶

Images are now represented as special tokens in the PAWLs format:

{
  "page": 0,
  "tokens": [
    {"text": "Introduction", "x": 100, "y": 50, ...},
    {
      "text": "[IMG]",
      "x": 100, "y": 200,
      "width": 400, "height": 300,
      "is_image": true,
      "image_path": "images/page_0_img_1.png"
    }
  ]
}

Supported Embedders¶

Architecture Changes¶

StructuralAnnotationSet Model¶

The content_hash field format changed to support corpus isolation:

Document.source_document Field¶

The source_document field is no longer set when adding documents to a corpus. Each corpus copy is an independent entity with no provenance tracking to the source.

Database Schema Changes¶

New migration applied: - annotations/0056_alter_structuralannotationset_content_hash - Extends content_hash to 128 chars

Migration Notes¶

Automatic Migration¶

The database migration runs automatically and extends the content_hash field. No manual intervention required.

Behavior Change¶

If your application relied on: - Shared structural annotations across corpus copies → They are now duplicated - source_document provenance tracking → This field is no longer set on import - Content-based deduplication → Each upload creates independent documents

Storage Impact¶

Deployments with documents appearing in multiple corpuses will see increased storage usage proportional to the number of corpus copies.

Bug Fixes¶

• Fixed migration using incorrect related_name for document relationships
• Fixed embeddings task signal using wrong filter relationship (documents= → document_paths__document=)
• Fixed annotation queries to use structural_annotation_set.structural_annotations relationship
• Fixed test suite alignment with corpus-isolated annotation behavior

Documentation Updates¶

Updated documentation to reflect corpus-isolated architecture:

• docs/architecture/structural_vs_non_structural_annotations.md - Complete rewrite
• docs/architecture/document_versioning.md - Added multi-embedder rationale
• docs/architecture/data_model/documents_and_annotations.md - Updated StructuralAnnotationSet description
• docs/migrations/v3_upgrade_guide.md - Updated structural annotation section
• docs/architecture/corpus_export_import_v2.md - Updated export format comments

Related Documentation¶

• Structural vs Non-Structural Annotations
• Document Versioning
• Core Data Model