Skip to content

Embeddings Creation & Retrieval

Overview of Creating and Searching Embeddings

Dual Embedding Strategy

OpenContracts uses a dual embedding strategy to enable both global cross-corpus search and corpus-optimized search:

  1. Default Embedder Embedding (Always Created)
  2. Every annotation, document, and note ALWAYS gets an embedding using the platform's default embedder (configured via PipelineSettings.default_embedder in the database)
  3. This creates a common vector space for global search across all corpuses

  4. Enables users to search all their documents regardless of which corpus they belong to

  5. Corpus-Specific Embedding (Created When Different)

  6. If a corpus has a preferred_embedder that differs from the default embedder, annotations also get an embedding using the corpus embedder
  7. Enables optimized search within a corpus using domain-specific models (e.g., legal-bert for legal documents)

Example Scenarios:

Corpus Embedder Embeddings Created
None (default) 1x default embedder
Same as default embedder 1x default embedder
Different embedder 2x (default embedder + corpus embedder)

This strategy is implemented in the embedding tasks (opencontractserver/tasks/embeddings_task.py) which automatically handle the dual embedding creation.

Creating Embeddings

  1. Generate the Embeddings (Text → Vector)
  2. Use the unified utility function generate_embeddings_from_text(text, embedder_path=None) or agenerate_embeddings_from_text(text, embedder_path=None) from opencontractserver/utils/embeddings.py.

  3. These functions:

    • Retrieve any configured Python embedder class for the specified corpus (if corpus_id is provided to get_embedder()).
    • If no embedder is found or it fails, fall back to the configured default embedder (typically our built-in microservice).
    • Return a tuple (embedder_path, vector), allowing you to know which embedder was used and the numeric embedding vector.

  4. Store the Embeddings

  5. Suppose you have a model instance (e.g., Annotation, Document, Note) that uses the HasEmbeddingMixin.
  6. Once you have (embedder_path, vector) from the generation step, you call: 
    instance.add_embedding(embedder_path="my-embedder", vector=[...])
  7. Internally, this uses:
    • The HasEmbeddingMixin.add_embedding(...) method, which contacts the Embedding manager to create or update an embedding record associated with this instance.
    • For multiple vectors (e.g., one instance with many embeddings), use: 
      instance.add_embeddings("my-embedder", [...multiple_vectors...])

  8. This is the "write" step—saving the vectors to your database.

  9. Retrieve Stored Embeddings

  10. To retrieve a stored embedding vector: 
    # Synchronous retrieval
vector = instance.get_embedding(
    embedder_path="openai/text-embedding-ada-002",
    dimension=384
)

# Asynchronous retrieval (uses database_sync_to_async)
vector = await instance.aget_embedding(
    embedder_path="openai/text-embedding-ada-002",
    dimension=384
)
  11. Returns List[float] | None - the embedding vector or None if not found.

Embedding Storage Architecture

The embedding system uses a dedicated Embedding model that supports multiple vector dimensions:

  • Supported Dimensions: 384, 768, 1024, 1536, 2048, 3072, 4096
  • Vector Fields: vector_384, vector_768, vector_1024, vector_1536, vector_2048, vector_3072, vector_4096
  • Reference Fields: document_id, annotation_id, note_id (depending on the model type)
  • Embedder Tracking: embedder_path field stores the identifier of the embedding model used

Embedder Configuration

Embedder settings are managed via the PipelineSettings database singleton (see opencontractserver/documents/models.py):

  • PipelineSettings.preferred_embedders: Map of MIME types to preferred embedder class paths
  • PipelineSettings.default_embedder: Fallback embedder when no preferred embedder is found
  • DEFAULT_EMBEDDING_DIMENSION: Default dimension (768, in config/settings/base.py)

These values are seeded from Django settings on first creation but are managed at runtime via the admin UI or GraphQL mutations. Changes take effect immediately across all processes (cached via Django's cache framework with a 5-minute TTL).

Available embedder implementations in opencontractserver/pipeline/embedders/:

Text-only: - MicroserviceEmbedder: Calls external sentence-transformer microservice (384-dim, default)

Multimodal (text + images): - CLIPMicroserviceEmbedder: CLIP-based multimodal embedder via microservice (768-dim) - QwenMicroserviceEmbedder: Qwen-based multimodal embedder via microservice (1024-dim)

Searching Embeddings

Our search architecture is designed with two layers: a core API that contains our business logic, and framework adapters that provide compatibility with different agent frameworks.

Core Search API

  1. CoreAnnotationVectorStore - Framework-Agnostic Business Logic
  2. Located in opencontractserver/llms/vector_stores/core_vector_stores.py
  3. Contains all the business logic for vector search without dependencies on specific agent frameworks
  4. Key classes: CoreAnnotationVectorStore, VectorSearchQuery, VectorSearchResult

  5. See the source file for initialization and usage examples

  6. Framework-Agnostic Data Structures

  7. VectorSearchQuery: Contains query text/embedding, filters, and search parameters

  8. VectorSearchResult: Contains the annotation and similarity score

  9. Filtering Logic

  10. Corpus Filtering: Annotations are filtered by corpus membership. Structural annotations (structural=True) are always included regardless of corpus, while non-structural annotations must belong to the specified corpus.
  11. Document Filtering: When document_id is provided, only annotations from that document are included.
  12. Visibility Filtering: Annotations are visible if they are structural, public, or created by the requesting user.
  13. Metadata Filtering: Additional filters can be applied on annotation labels and other fields.

The CoreAnnotationVectorStore provides a global_search class method for cross-corpus search:

from opencontractserver.llms.vector_stores.core_vector_stores import (
    CoreAnnotationVectorStore,
    VectorSearchResult,
)

# Search across ALL documents the user has access to
results = CoreAnnotationVectorStore.global_search(
    user_id=my_user_id,
    query_text="contract termination clause",
    top_k=100,
    modalities=["TEXT"],  # Optional: filter by content type
)

# Async version
results = await CoreAnnotationVectorStore.async_global_search(
    user_id=my_user_id,
    query_text="contract termination clause",
    top_k=100,
)

# Process results
for result in results:
    annotation = result.annotation
    similarity = result.similarity_score
    document_title = annotation.document.title if annotation.document else "N/A"
    print(f"Found: {annotation.raw_text[:50]}... (score: {similarity:.3f})")

Key Features: - Uses default embedder embeddings for consistent cross-corpus search - Respects user permissions (only searches documents the user can access) - Supports modality filtering (TEXT, IMAGE, etc.) - Works with both standalone and corpus-bound documents

Vector Search Mixin

Models that store embeddings can use VectorSearchViaEmbeddingMixin from opencontractserver/shared/mixins.py to enable vector similarity searches (the model must also use HasEmbeddingMixin). See the source file for usage examples.

Framework Adapters

Framework adapters are thin wrappers that translate between the core API and specific agent frameworks:

  1. LlamaIndex Adapter - Removed
  2. The LlamaIndex vector store adapter has been removed from the codebase.

  3. If you need LlamaIndex integration, implement your own adapter wrapping CoreAnnotationVectorStore.

  4. PydanticAI Adapter - PydanticAIAnnotationVectorStore

  5. Located in opencontractserver/llms/vector_stores/pydantic_ai_vector_stores.py
  6. Provides async-first API with Pydantic models for type safety

  7. Converts between PydanticAI types and our core types (see source file for usage examples)

  8. PydanticAI Tool Creation

  9. Use create_vector_search_tool() from pydantic_ai_vector_stores.py to create vector search tools for PydanticAI agents.

  10. PydanticAI Response Format

  11. Returns structured PydanticAIVectorSearchResponse with validated data containing annotation details, similarity scores, and location information.
  12. See pydantic_ai_vector_stores.py for the complete response structure.

Async/Sync Compatibility

The embedding system supports both synchronous and asynchronous operations:

Synchronous Methods: - generate_embeddings_from_text() - instance.get_embedding() - instance.add_embedding() - store.search()

Asynchronous Methods: - agenerate_embeddings_from_text() - instance.aget_embedding() (uses database_sync_to_async) - store.async_search()

When working in async contexts (such as Django Channels WebSocket consumers), always use the async variants to avoid blocking operations.

How This Works in Practice

  1. Query Processing
  2. Framework adapter receives a query in framework-specific format
  3. Adapter converts to VectorSearchQuery

  4. Core store processes the query using our business logic

  5. Vector Generation

  6. If query.query_text is provided, core store calls generate_embeddings_from_text(...) or agenerate_embeddings_from_text(...)

  7. If query.query_embedding is provided, uses it directly

  8. Database Search

  9. Core store builds Django queryset with filters (corpus, document, user, metadata)
  10. Applies structural annotation logic (structural annotations bypass corpus filtering)
  11. Uses .search_by_embedding(...) from VectorSearchViaEmbeddingMixin for similarity search

  12. Returns VectorSearchResult objects with annotations and similarity scores

  13. Result Conversion

  14. Framework adapter converts VectorSearchResult objects to framework-specific format
  15. Embedding vectors are retrieved using aget_embedding() for async contexts

Performance Considerations

  1. Database Indexes: The Annotation model includes composite indexes for optimal query performance. See opencontractserver/annotations/models.py for index definitions.

  2. Embedding Retrieval: Use prefetch_related("embeddings") when fetching multiple annotations to avoid N+1 queries.

  3. Vector Dimensions: Choose appropriate embedding dimensions based on your use case:

  4. 384: Fast, good for general similarity
  5. 768: Balanced performance and accuracy
  6. 1536: High accuracy, more computational cost
  7. 3072: Highest accuracy, highest computational cost

Architecture Benefits

This architecture provides: - Reusability: Core logic works with any framework - Maintainability: Business logic changes in one place - Extensibility: Easy to add new framework adapters - Type Safety: Clear interfaces between layers - Testing: Core functionality can be tested independently - Async Support: Full compatibility with async Django applications - Performance: Optimized database queries and indexing strategies

By following this pattern, you can use the same underlying search capabilities across different agent frameworks while maintaining consistency and avoiding code duplication.