Document and RAG pipelines¶
A retrieval pipeline in this provider is three ordinary tasks. Load documents: DocumentLoaderOperator
parses files (text, CSV, JSON, PDF, DOCX) into a list of {"text", "metadata"} dicts with
no AI framework involved. Embed documents: LlamaIndexEmbeddingOperator chunks those documents and
produces embedding vectors. Retrieve context: LlamaIndexRetrievalOperator pulls the closest chunks back
out for a question, ready to drop into an LLMOperator prompt. Each step
is a task, so indexing can run on a schedule while querying runs on demand.
The LlamaIndex operators read their embedding and language models from a llamaindex
connection. Using LlamaIndex directly: LlamaIndexHook returns those LlamaIndex objects for use in a plain
@task when the operators do not fit.
Ask questions over a growing PDF corpus and Compare companies’ 10-K filings show the whole shape end to end.