Overview

Relevant source files

• .github/workflows/release.yml
• .github/workflows/tests.yml
• Dockerfile
• Dockerfile.deps
• README.md
• README_ar.md
• README_fr.md
• README_id.md
• README_ja.md
• README_ko.md
• README_pt_br.md
• README_tr.md
• README_tzh.md
• README_zh.md
• docker/.env
• docker/README.md
• docker/docker-compose-base.yml
• docker/infinity_conf.toml
• docs/guides/manage_files.md
• docs/quickstart.mdx
• download_deps.py
• helm/values.yaml
• pyproject.toml
• sdk/python/pyproject.toml
• sdk/python/uv.lock
• uv.lock

Purpose and Scope

RAGFlow is an open-source Retrieval-Augmented Generation (RAG) engine that integrates deep document understanding with agentic workflow capabilities. It is engineered to transform complex, unstructured data into high-fidelity, production-ready AI systems by combining intelligent document parsing, embedding, and retrieval with configurable LLM-powered agents and workflows. README.md76-78

Core Characteristics: RAGFlow specializes in parsing complex document formats such as PDF, DOCX, Excel, and PPT using state-of-the-art deep learning models for document layout and table extraction. It segments documents into meaningful chunks using multiple, template-driven chunking strategies, generates contextual embeddings for these chunks, and stores them in a hybrid vector and keyword index. It supports conversational AI applications backed by traceable, grounded citations. Additionally, RAGFlow includes a visual agent workflow system called Canvas for creating multi-step AI agents with persistent memory. README.md112-122 pyproject.toml4

Key Value Statement: "Quality in, quality out" — the system prioritizes deep document understanding to reliably find relevant information even from vast, unbounded token spaces. README.md114-118

Sources: README.md76-122 pyproject.toml4

System Architecture

RAGFlow is built upon a multi-tier microservices architecture, optimizing scalability and decoupling synchronous API operations from intensive background processing tasks. The architecture follows a producer-consumer pattern with Redis Streams managing task queues. docker/.env140-146

High-Level Architecture Diagram

Sources: README.md141-145 docker/.env13-159 docker/docker-compose-base.yml1-230

Three-Tier Architecture

RAGFlow is logically divided into three tiers:

1. Frontend/API Tier:

   • Exposes REST and SDK endpoints via a Quart server running in asynchronous mode allowing concurrent streaming responses. pyproject.toml100-101 pyproject.toml145
   • Supports hybrid deployment with a Go server (cmd/server_main.go) for native components and advanced search handling. docker/.env161-162
   • API proxying is configurable through environment variables (API_PROXY_SCHEME). docker/.env161-162

2. Asynchronous Task Tier:

   • Background workers implement document parsing, embedding, and indexing asynchronously.
   • Tasks are queued in Redis Streams and consumed by the TaskExecutor or the Go-based Ingestor. .github/workflows/tests.yml132-139
   • Supports an orchestrable ingestion pipeline for complex data flows. README.md97

3. Persistence Tier:

   • Utilizes a polyglot persistence approach:
     • Relational Database: MySQL (8.0.39) or PostgreSQL manages metadata, user, tenancy, and settings via the Peewee ORM. docker/docker-compose-base.yml186-188 pyproject.toml133
     • Document Store: Pluggable engines including Elasticsearch, Infinity, OpenSearch, and OceanBase for vector and keyword indexing. docker/.env13-20
     • Object Storage: MinIO for raw files and images. docker/docker-compose-base.yml214-215
     • Cache/Queue: Redis (Valkey) for task orchestration and ephemeral state. docker/.env140-146 pyproject.toml121

Sources: pyproject.toml9-169 docker/.env13-159 docker/docker-compose-base.yml1-230

Core Components and Services

System Entity Mapping Diagram

Bridging natural language descriptions of system functions and their implementation code entities, this diagram relates the major conceptual components to their main code locations:

Sources: .github/workflows/tests.yml132-139 README.md93-101

Application Entry Points

Service/Subsystem	Main File(s)	Description
Python REST API Server	api/ragflow_server.py	Quart-based async API server handling REST endpoints and SDK.
Go Server	cmd/server_main.go	High-performance backend server layer with native components.
Task Executor Worker	rag/svr/task_executor.py	Python background task worker for document ingestion.
Go Ingestor	cmd/ingestor.go	Go-based ingestion worker for high-throughput processing.
MCP Server	mcp/server/server.py	Model Context Protocol server for agent communication.
Admin Server	cmd/admin_server.go	Go-based administration service.

Sources: .github/workflows/tests.yml132-139 docker/.env156-159 pyproject.toml68

Key Features

• Deep Document Understanding: Extracts structured knowledge from complex formats (PDF, DOCX, etc.) using layout recognition and OCR. README.md116-117
• Template-Based Chunking: Offers intelligent chunking strategies (naive, book, paper, laws, QA) to ensure contextually relevant segments. README.md120-124
• Agentic Workflow (Canvas): A visual orchestration system for creating multi-step AI agents with tool-calling capabilities and MCP support. README.md99
• Memory System: Supports persistent 'Memory' for AI agents, including episodic and procedural types. README.md93
• Hybrid Search: Combines vector retrieval with BM25 keyword search, supported by fused re-ranking for precision. README.md137-138
• Data Source Connectors: Supports data synchronization from Confluence, S3, Notion, Discord, and Google Drive. README.md95
• Sandbox Code Executor: gVisor-based sandbox for safe execution of Python/JS code within agent workflows. README.md100 docker/docker-compose-base.yml158-161

Sources: README.md75-140 docker/.env13-20 docker/docker-compose-base.yml158-184

Technology Stack

Backend

• Languages: Python (3.13), Go, C++ (for tokenizer and RAGAnalyzer). pyproject.toml8 .github/workflows/tests.yml132-139
• Frameworks: Quart (Python async), Eino (Go workflow framework). pyproject.toml145
• Database/ORM: Peewee (Python), MySQL 8.0. pyproject.toml133 docker/docker-compose-base.yml188
• AI Integration: LiteLLM (unified LLM access), ONNX Runtime (local inference). pyproject.toml163 pyproject.toml77-78

Infrastructure

• Deployment: Docker Compose, Kubernetes (Helm). README.md153
• OS Base: Ubuntu 24.04. Dockerfile2
• Dependency Management: uv (Python), Go modules. Dockerfile69-83

Sources: pyproject.toml1-180 Dockerfile1-154 docker/.env1-165