An enterprise Retrieval-Augmented Generation (RAG) system is only as reliable as the data pipeline behind it. The pipeline must ingest documents, extract useful content, preserve metadata and permissions, create searchable chunks and embeddings, publish updates safely, and remove outdated or unauthorized content quickly.
1. Design an end-to-end RAG data pipeline for an enterprise knowledge assistant.
Answer: Build an event-driven, versioned pipeline that separates source ingestion, document processing, embedding generation, index publication, and query-time authorization. Raw documents remain in durable source storage; chunks, embeddings, and indexes are derived artifacts that can be rebuilt.
- Change-driven ingestion: Use source events, webhooks, incremental APIs, or scheduled reconciliation for creates, updates, moves, and deletions.
- Stable identities: Track source/document IDs, source version, content hash, and processing version.
- Metadata first: Preserve owner, URI, timestamps, classification, tenant, and access rules.
- Idempotency: Reprocessing the same version must not create duplicate chunks or vectors.
- Staged publishing: Validate new chunks and vectors before making them queryable.
- Observability: Track connector lag, parse failures, embedding latency, index freshness, deletion lag, and cost.
Query path: Authenticate the user, derive current permissions, retrieve only authorized content, optionally rerank, and send selected context to the LLM with citations. Never rely on the LLM to enforce permissions.
2. How would you ingest millions of documents from multiple sources?
Answer: Avoid a single serial job. Use source-specific connectors that normalize change events into a common durable queue, then scale stateless processing workers horizontally. Separate discovery and metadata synchronization from expensive extraction and embedding.
- Inventory and checkpoint: Persist continuation tokens, checkpoints, and last successful sync time.
- Incremental discovery: Prefer change feeds, webhooks, modified-time queries, or incremental APIs over downloading every file repeatedly.
- Normalize events: Convert create, update, move, and delete operations to a common event schema.
- Queue work: Buffer bursts and decouple connectors from downstream stages.
- Scale safely: Partition by source/document ID, cap concurrency per source, and apply backpressure.
- Persist status: Track version, hash, retries, errors, and last successful stage.
- Reconcile: Periodically compare source inventory with indexed IDs to find missed events.
| Challenge | Response |
|---|---|
| API throttling | Rate limits, exponential backoff, per-source concurrency caps |
| Large files | Streaming downloads, size limits, isolated processing, timeouts |
| Bursts | Durable queue, autoscaling, queue-age alarms |
| Repeated work | Stable IDs, content hashes, idempotency keys, version-aware upserts |
Metrics: documents discovered/processed per minute, queue age, source lag, parse throughput, embedding tokens per second, retry rate, and cost per thousand indexed documents.
3. How would you choose a chunking strategy?
Answer: Choose chunking based on document structure, retrieval needs, model input limits, and evaluation results—not just a fixed character count. Preserve enough context for useful retrieval without creating chunks so broad that they match irrelevant content.
| Strategy | Good fit | Trade-off |
|---|---|---|
| Fixed token windows | Simple text, predictable processing | Can split ideas or tables awkwardly |
| Structure-aware | Headings, sections, tables, code | Needs format-specific parsing |
| Semantic | Clear topic changes | More complex and potentially costly |
| Parent-child | Precise matching plus broader context | Needs parent-child mapping |
- Parse into meaningful units such as headings, paragraphs, list items, and table rows.
- Respect embedding-model token limits and preserve heading paths or context metadata.
- Start with a sensible chunk size and modest overlap; evaluate alternatives on representative questions.
- Measure retrieval recall, precision, context relevance, answer quality, latency, and index size.
- Preserve table headers with rows and retain page/section references for citations.
Overlap trade-off: Some overlap can preserve boundary context, but too much increases cost and near-duplicate results. Use retrieval evaluation to determine whether it helps.
4. How do you update embeddings when a source document changes?
Answer: Use version-aware incremental reprocessing. Detect the change, compute a content hash or source version, and rerun only the required stages. Do not regenerate the entire corpus when one file changes.
- Receive a change event or discover the update during incremental sync.
- Fetch the current version and calculate a normalized content hash.
- Skip work if content and relevant processing configuration have not changed.
- Re-parse, re-chunk, and re-embed changed content; reuse outputs only when chunk identity and embedding model/configuration remain compatible.
- Write new chunks and vectors under a new version or staging namespace.
- Validate chunk counts, metadata, dimensions, and sample retrieval behavior.
- Promote the new version and retire the previous version safely.
if content_hash == indexed_hash and pipeline_version == indexed_pipeline_version:
skip_reprocessing()
else:
new_version = process_and_embed(document)
validate(new_version)
publish(new_version)
retire_previous_version(document.id)A new embedding model, chunking strategy, or normalization rule may require a broad re-embedding even if source content is unchanged. Track these versions separately.
5. How would you propagate document deletions to the vector index?
Answer: Treat deletion as a first-class event. Quickly prevent the content from being retrieved, then clean up chunks, vectors, caches, and derived copies according to policy.
- Capture deletion: Receive a source tombstone or detect absence during reconciliation.
- Record a tombstone: Persist document ID, source version, event time, and deletion status.
- Suppress retrieval immediately: Mark the document inactive or revoked, even while physical cleanup is asynchronous.
- Delete derived records: Remove every chunk and vector associated with the document and its versions.
- Invalidate caches and replicas: Update search indexes and downstream copies as required.
- Verify: Confirm the deleted document can no longer be retrieved.
- Audit: Retain only minimal deletion metadata allowed by policy.
Make deletion idempotent, handle out-of-order events with source versions, retry partial failures, and reconcile source inventories against indexed IDs.
6. How do you prevent users from retrieving documents they are not authorized to access?
Answer: Enforce authorization during retrieval using trusted identity and current source permissions. Metadata filters narrow candidates, but must not be bypassable, and stale permissions must not expose restricted content.
- Capture source ACLs: Ingest permissions, tenant IDs, groups, classification, and ownership.
- Resolve identity: Derive permitted groups and roles from authenticated identity, not user-supplied filters.
- Filter before LLM context: Apply mandatory tenant and authorization constraints during retrieval and re-check candidates before sending text to the model.
- Handle revocation: Synchronize permission changes promptly; fail closed when current authorization cannot be established.
- Prevent indirect leaks: Exclude unauthorized snippets from reranking, logs, caches, citations, and errors.
- Test isolation: Test cross-tenant queries, revoked access, group changes, and crafted filters.
- Audit access: Log IDs and policy decisions without recording sensitive content.
Example: Store stable document ID, tenant ID, and access metadata with each chunk. At query time, derive permitted scopes from the authenticated user, apply mandatory filters, and verify each candidate remains accessible before passing it to the LLM. For frequently changing ACLs, combine index filters with an authoritative final permission check.
7. How would you handle a failed embedding-generation job?
Answer: Make the stage resumable and idempotent. Persist progress per chunk, retry transient failures with backoff, send persistent failures to a review/dead-letter queue, and never publish a partially updated document as complete.
- Track document version, chunk ID, status, attempt count, error type, and last attempt.
- Classify errors: retry network failures, rate limits, and temporary provider errors; avoid endless retries for invalid input or persistent permission errors.
- Use bounded exponential backoff with jitter and respect provider limits.
- Resume from checkpoints and process only missing/failed chunks.
- Publish only when all required chunks have valid embeddings and metadata.
- Isolate poison records in a review queue.
- Reconcile expected chunk IDs against successfully indexed IDs and repair gaps.
try:
vector = generate_embedding(chunk.text)
upsert_vector(chunk.id, vector, chunk.metadata)
mark_success(chunk.id)
except TransientError:
schedule_retry_with_backoff(chunk.id)
except PermanentError as err:
send_to_review_queue(chunk.id, err)For a large re-index, keep the old index active while building and validating a replacement in staging. Switch the active index only after coverage and quality checks pass.
8. How would you minimize stale or duplicated content in a RAG system?
Answer: Use stable identities, content hashes, version-aware upserts, change detection, tombstones, and periodic reconciliation. Distinguish duplicate content from valid copies that have different owners, permissions, or business context.
Prevent stale content
- Use change feeds and measure synchronization lag.
- Track document version, hash, last-modified, and last-indexed timestamps.
- Retire prior chunks when a newer version is published.
- Use tombstones to suppress deleted content while cleanup runs.
- Set freshness SLAs and alert when sources stop synchronizing.
Prevent duplicates
- Normalize before hashing, while preserving meaningful distinctions such as language and version.
- Use stable source IDs and idempotency keys so retries do not create duplicate chunks.
- Use exact hashes or similarity detection to flag copies; treat similarity as a review signal where needed.
- Preserve provenance and permissions even when text is identical.
- Deduplicate near-identical retrieval results at query time without merging content with different access rights.
Key distinction: Identical text can have different access rights or legal status. Deduplication must not broaden access or erase provenance.
9. How would you scale a RAG ingestion pipeline while controlling costs?
Answer: Scale each stage independently and spend compute only when content or processing configuration requires it. Measure cost per successfully indexed document or token, not only total cloud spend.
| Cost driver | Optimization | Trade-off |
|---|---|---|
| Downloads | Incremental sync, conditional downloads | Missed changes if checkpoints fail |
| Parsing/OCR | Use expensive OCR only when needed; cache extraction | Extraction quality |
| Embeddings | Batch calls; cache by content and model version | Rate limits, invalidation, freshness |
| Chunk volume | Tune size and overlap through evaluation | Retrieval recall |
| Vector storage | Retire old versions; tune index settings | Latency and recall |
| Compute | Autoscale workers; separate CPU/GPU workloads | Queue lag and startup overhead |
- Use durable queues and autoscale download, parsing, chunking, and embedding stages independently.
- Cache embeddings using normalized content hash plus embedding model/version and relevant configuration.
- Prioritize changed/high-value content while enforcing freshness SLAs for required sources.
- Use batch embedding calls where supported and respect rate limits.
- Track tokens, retries, OCR pages, vector count, storage growth, and cost by source or tenant.
- Evaluate retrieval quality before changing chunking, models, or index parameters to reduce spend.
Quick Revision Table
| Question | Key points |
|---|---|
| End-to-end RAG | Connectors, raw storage, parsing, chunking, embeddings, secure retrieval, citations |
| Millions of documents | Incremental sync, queue, horizontal workers, checkpoints, reconciliation |
| Chunking | Structure, token limits, overlap, retrieval evaluation |
| Update embeddings | Hashes, versioned processing, staging, safe promotion |
| Deletion | Tombstones, immediate suppression, cleanup, verification |
| Permissions | Trusted identity, ACL sync, mandatory filters, final check |
| Failed embedding job | Checkpoints, idempotency, retries, dead-letter queue, reconciliation |
| Stale/duplicate content | Stable IDs, hashes, version-aware upserts, provenance |
| Scale and cost | Autoscaling, caching, batching, cost metrics, evaluation |
Scenario-Based Follow-Up Questions
Expand each question to review the core ideas a strong interview answer should cover.
Scenario 1: The index contains only half the chunks for an updated policy.
Scenario 2: A user lost access but can still retrieve a document.
Scenario 3: Embedding costs doubled after a connector change.
Scenario 4: The connector missed a source deletion.
Chapter Summary
- Build RAG ingestion as a durable, observable, versioned pipeline—not a one-off embedding script.
- Use incremental discovery, queues, checkpoints, and horizontal workers for scale.
- Choose chunking based on document structure and retrieval evaluation.
- Use hashes and version-aware processing to update only what changed.
- Propagate deletions quickly with tombstones and verify cleanup.
- Enforce authorization before content reaches the LLM.
- Make jobs resumable and cost-aware with retries, caching, batching, and metrics.
Practice Challenge
Design a RAG ingestion platform for 5 million enterprise documents from three systems. Explain incremental sync, document identity, chunking, embedding retries, permission enforcement, deletion propagation, index promotion, and the metrics used to control freshness and cost.
Comments
Post a Comment