Almost every retrieval-augmented generation (RAG) project I’ve worked on ends up with the same hidden cost. Building the first embedding pipeline is easy. Keeping the vectors in sync with the operational data after it changes is hard. Microsoft’s Azure Cosmos DB team has just open-sourced a tool aimed squarely at that problem, called OmniVec, and I think it deserves a look from anyone running AI search on Azure.
Here’s what it is, how it works, and my take on where it fits.
The problem OmniVec solves
A vector index is only useful if it reflects the current state of your data. When a product description changes, a document is updated, or a new record lands, its embedding has to be regenerated and written back. In practice teams build this plumbing themselves: change capture on the source, a pool of workers that call the embedding model, retry and backfill logic, dead-letter handling, and the monitoring to keep it all running. It’s the same code in every project, and it rarely gets the engineering attention it needs.
OmniVec replaces that stack with four configurable pieces. You register a source (where records come from), a model (the embedding model to call), and a destination (the vector store), then create a pipeline that ties them together and says which fields to embed. OmniVec handles the initial backfill, tracks changes, calls the model with batching and retries, and writes the vectors back.
How it works
You deploy OmniVec into your own Azure subscription with the Azure Developer CLI (azd up). The deployment provisions an Azure Kubernetes Service (AKS) cluster that runs the OmniVec services, an Azure Cosmos DB account for pipeline metadata and progress tracking, and an Azure Container Registry for the service images. You manage it through a web UI, a CLI, or a REST API.
Inside the cluster there are a few moving parts:
- A REST API control plane that holds the configuration of sources, models, destinations, and pipelines.
- An ingestion service that watches each source for changes, using the change feed for Azure Cosmos DB, blob events for Azure Blob Storage, and change data capture for SQL Server and PostgreSQL.
- A horizontally scalable worker pool that pulls records off a queue, calls the embedding model, and writes the result to the destination.
- A model routing layer that lets a pipeline call either a hosted model such as Azure OpenAI or a self-hosted model running on a GPU node pool.
At launch it supports Azure Cosmos DB, PostgreSQL, SQL Server, and Azure Blob Storage. The walkthrough in the announcement uses a single Cosmos DB container as both source and destination with an Azure OpenAI text-embedding-3-small model. After the pipeline is resumed, it backfills all existing documents, and from then on every insert or update is re-embedded automatically through the change feed. The CLI even includes a search command, and the web UI has a vector search playground for quick testing.
Security is handled the right way: the OmniVec services authenticate to Cosmos DB with a managed identity rather than connection strings, so you grant it data-plane and control-plane roles instead of passing around keys.
My take: why this matters
What I like most is the separation of concerns. The source, model, and destination are declared independently, so switching embedding models or adding a second vector store becomes a configuration change rather than a rewrite. That matters because embedding models are changing quickly, and re-embedding an entire corpus is exactly the kind of job teams put off because the tooling is fragile.
It also turns embedding freshness into an operational concern with visible metrics (documents embedded, pending, failed) rather than a script someone runs when search results start looking stale. For enterprise RAG and agentic AI solutions, where users quickly lose trust if answers reflect outdated data, that is a real improvement.
Things to weigh before adopting it
- It’s a public preview. Microsoft says to expect changes as it matures, so I’d start with a pilot rather than a production-critical workload.
- It brings an AKS footprint. You’re running and paying for a Kubernetes cluster, a Cosmos DB metadata account, and a container registry. For a small dataset, a simpler scheduled job may still be cheaper. OmniVec earns its keep when you have many sources, frequent changes, or large volumes.
- Check source coverage. The supported sources today are Cosmos DB, PostgreSQL, SQL Server, and Blob Storage. If your data lives elsewhere, you’d need to land it in one of those first.
- Plan your model choice. Hosted Azure OpenAI models keep the cluster light, while self-hosted models need a GPU node pool and the cost that comes with it.
Recommendation
If you’re building RAG, semantic search, or agentic AI solutions on Azure and your source data changes regularly, OmniVec is worth a proof of concept. I’d pick one real dataset, such as a product catalogue in Cosmos DB or a document library in Blob Storage, deploy OmniVec into a sandbox subscription, and measure three things: how long the initial backfill takes, how quickly updates show up in search, and what the running cost looks like. That will tell you quickly whether it can replace the custom embedding plumbing you’re maintaining today.
You can read the full announcement and walkthrough on the Azure Cosmos DB blog: Introducing OmniVec: An Open-Source Embedding Platform for AI Apps on Azure. The code is on GitHub at AzureCosmosDB/OmniVec.
Leave a Reply