Retrieval-augmented generation is the default architecture for grounding an LLM in an organization's own data — embed your documents, store the vectors, retrieve the relevant chunks at query time, feed them to the model as context. It also quietly introduces a new piece of infrastructure with its own security model: the vector store, and the retrieval pipeline around it.
Vector Store Poisoning
If an attacker can insert content into the corpus that gets embedded and indexed — a malicious document uploaded through a feature that feeds the RAG pipeline, a poisoned page in a crawled data source — that content becomes retrievable context for future queries. Unlike training data poisoning, which requires influencing a model retraining cycle, RAG poisoning can take effect on the very next query after ingestion, with a much lower bar to pull off.
Cross-Tenant Retrieval Leakage
Multi-tenant RAG applications need namespace isolation in the vector store itself, not just in the application layer above it. If tenant boundaries aren't enforced at the retrieval query — filtering by tenant ID as part of the vector search, not as a post-filter on results — a crafted query can retrieve chunks embedded from another tenant's documents. This is the RAG-specific version of a broken access control vulnerability, and it's easy to miss because the vector store doesn't look like a database to teams used to auditing SQL access controls.
Indirect Injection via Retrieved Documents
Every document in the corpus is a potential prompt injection vector once it's retrieved and placed in the model's context. A support ticket, a wiki page, a PDF uploaded by a user — any of them can carry hidden instructions that the model treats as part of its context rather than as data it's summarizing. This is the same indirect prompt injection problem covered by OWASP LLM01, specifically scoped to whatever's sitting in your vector store.
Embedding Inversion and Leakage
Embeddings aren't as opaque as they look. Research on embedding inversion shows that, in some conditions, original text can be partially reconstructed from its vector representation — meaning a leaked or improperly secured vector store can leak more of the underlying source content than teams assume when they treat "it's just embeddings" as a reason to relax access controls on the vector database.
Defenses That Actually Scope to This Architecture
- Tenant/namespace isolation enforced at the retrieval query, not as an application-layer afterthought — the vector database needs the same access-control rigor as any other data store holding sensitive content.
- Document provenance and validation before ingestion — scan and sanitize content before it's embedded, the same way you'd validate input before it reaches any other data pipeline.
- Treat retrieved content as untrusted in the prompt — structurally separate "instructions" from "retrieved data" in how context is assembled, and instruct the model (with as much reliability as that affords, which is partial) to treat retrieved chunks as reference material, not directives.
- Access controls on the vector store itself — encryption at rest, restricted query access, and monitoring for anomalous retrieval patterns (a sudden spike in queries touching a specific tenant's namespace, for instance).
- Periodic corpus audits — for ingestion pipelines without strong upstream validation, periodically sample what's actually in the index, not just what was supposed to go in.
The Bottom Line
RAG security review can't stop at "is the LLM jailbroken." The vector store is a real piece of infrastructure with real data in it, accessed by a retrieval query that needs the same access-control discipline as any other data access path — and it's currently one of the most under-reviewed components in production AI systems.
Back to Blog