Backend — FastAPI server, agents, tools, retrieval & scan

1. Server basics

• Entry point: backend/main.py → uvicorn.run(app, host="127.0.0.1", port=8765)
• Framework: FastAPI, CORS allows http://localhost:5173 and file://
• Startup warmup (background thread, non-blocking): preloads the BGE-M3 embedder, BGE-Reranker, the Ollama LLM (connection check), and the playbook classifier's reference embeddings — so the first user query is fast.
• Logging: backend/legal_retrieval/logging_setup.py (get_logger), level from PLR_LOG_LEVEL (default INFO).

2. HTTP API reference

Chat & files

Workspace (user-picked folder) — REQUIRED for chat/upload

A workspace is mandatory. Chat (/api/chat, /api/chat/stream, /api/chat/resume) and uploads error with "No workspace selected. Select a project folder first." when none is active — there is no default data/ folder fallback anymore. The folder is picked through a native Electron directory dialog (folder:select IPC) so the real absolute path reaches the backend.

Storage layout: with a workspace active, get_raw_dir() returns the project folder itself (uploads land directly in the user's folder, visible as project documents), while processed temp files, the ChromaDB index, the BM25 index and history.json all live under <project>/workspace/ (workspace/processed, workspace/chroma_db, workspace/bm25_index, workspace/history.json). The override is runtime-only (resets on restart / DELETE).

ChatRequest fields: query, mode (rag default | retrieval_only | direct), top_k (default 5), skip_rerank, temperature (default 0.3), conversation_id, history (prior [{role, content}] turns for direct mode), context_hint (raw selected passage from the native right-click menu — the pipeline retrieves the EXACT chunk the selection came from and cites it), and attachment fields attached_file_path(s) / attached_filename(s).

mode behavior:

• rag → intent router (router_intent.run_intent_loop): classify intent, plan, then call tools; answer from documents.
• retrieval_only → no LLM, just search; answer is a formatted list of results.
• direct → LLM only, no retrieval.

GPU manager

Playbook scan

3. Chat flow (end-to-end)

User message + optional attachments
  └─ POST /api/chat/stream (SSE)
       └─ router_intent.run_intent_loop(query, llm, max_iterations, top_k,
                                         skip_rerank, conversation_id,
                                         attached_file_paths, attached_filenames,
                                         context_hint)
            ├─ intent classification event ("intent")
            ├─ planner → writes a 1-2 sentence plan, streams "plan" event,
            │   tool detected from plan
            ├─ executes tools (search_documents, verify_relevance,
            │   scan_document, ingest_file) via _impl functions
            ├─ streams: thinking / tool_call / tool_result / status / node_start
            │   / node_complete / sources / token
            └─ done event: {answer, sources, timing}

SSE event types the frontend consumes (see frontend/src/services/api.ts): meta, intent, status, thinking, tool_call, plan, tool_result, node_start, node_complete, sources, interrupted, resumed, error, token, done. (Workspace ingest has its own event set: ingest_start, ingest_progress, ingest_done, ingest_cancelled, ingest_error.)

4. Agents, tools & modes

4a. Intent router (backend/legal_retrieval/router_intent.py) — default RAG path

A plan-based orchestration. Key nodes (async generators that yield events):

• _run_simple_node — chitchat / clarification / direct-answer intents
• _run_planner_node — the main search path: plans, calls search_documents (exact query, never rephrased), then verify_relevance, then writes the answer from the retrieved chunks.
• _run_scan_node — scan_document for a single attached file
• _run_multi_scan_node — batch scan of multiple attached files
• run_intent_loop / run_intent_resume_loop — entry points
• Uses prompts_intent.get_intent_prompt (a strict "you MUST NOT answer directly; pick a tool" prompt) and detect_tool_from_plan.
• Thread state checkpointer: AsyncSqliteSaver at data/conversations.db (falls back to MemorySaver).

4b. Legacy agentic router (backend/legal_retrieval/router.py)

A LangGraph ReAct agent (ChatOllama + get_all_tools()), used by retrieval_only/direct modes and resume:

• 3-node StateGraph: agent → approval → tools → agent (loop)
• Human-in-the-loop: approval_node calls interrupt(description) before tool execution; the graph pauses; the frontend shows an ApprovalBar; the user approves/rejects; /api/chat/resume streams Command(resume=response). On reject, a message tells the agent to answer without tools.
• should_continue routes: tool calls → approval; rejection message → agent; otherwise END.
• System prompt rules: pass the user's query exactly as-is to search_documents, never rephrase; cite file + page; be honest when documents don't contain the answer.

4c. Tools (backend/legal_retrieval/tools.py)

get_all_tools() / get_tools_for_mode(mode) build the LangChain-compatible tool list.

5. Retrieval pipeline (backend/legal_retrieval/retrieval.py)

query
 ├─ embed (BGE-M3, 1024-dim, L2-normalized)
 ├─ dense search  — ChromaDB (<project>/workspace/chroma_db when a workspace
 │                  is active, otherwise data/chroma_db; collection "legal_chunks",
 │                  cosine/HNSW), top_k_retrieval=50
 ├─ sparse search — BM25 (<project>/workspace/bm25_index or data/bm25_index), top 50
 ├─ merge: hybrid modes (default "hybrid" merged with combine_and_dedup;
 │   rrf and boost_only configurable via SearchConfig.hybrid_merge_strategy)
 ├─ rerank — BGE-Reranker-v2-M3 cross-encoder, top_k_final=5, score >= 0.0
 └─ QueryResponse{query, results[{file_name, page, section, score, text}], latency_ms}

All storage paths resolve through the active workspace (see §2 Workspace): when a user-picked folder is active, VectorStore points at <project>/workspace/chroma_db and SparseRetriever at <project>/workspace/bm25_index, so search and ingestion are isolated per project. RetrievalPipeline.ingest_directory(path) skips the workspace's own storage dirs by default.

RetrievalPipeline.ingest_file(path) → DocumentIngester (Docling parse + OCR) → StructureAwareChunker (480-token chunks, respect_sections=True) → embed → store → returns chunk count. Also builds/updates the BM25 index.

Configuration (backend/legal_retrieval/config.py)

All tunables are dataclasses, overridable with PLR_* env vars:

• ModelConfig: BAAI/bge-m3 (1024-dim), BAAI/bge-reranker-v2-m3; local dirs models/bge-m3, models/bge-reranker-v2-m3; device auto → cuda/mps/cpu.
• ChunkingConfig: chunk_size 480, overlap 0, respect_sections, min 32 tokens.
• StorageConfig: data/chroma_db (or <project>/workspace/chroma_db when a workspace is active), collection legal_chunks, cosine, HNSW.
• SearchConfig: top_k_retrieval 50, top_k_final 5, min_score 0.0, search_mode hybrid, hybrid_merge_strategy combine_and_dedup (default; rrf / boost_only configurable), RRF k=60.
• IngestionConfig: Docling OCR (en), page images, 150 DPI, table structure.

6. Playbook scan (backend/legal_retrieval/playbook/)

A clause-level compliance checker:

document (indexed or just-uploaded)
 └─ scan_document_streaming(document_id, playbook, llm, skip_classification)
      ├─ _fetch_document_chunks → chunks with page/section metadata
      ├─ classify clause type per chunk (embedding classifier, BGE-M3 refs)  [can be skipped]
      ├─ for each rule in playbook: rule check (rule → clause type, severity, keywords/LLM)
      ├─ flag_found events: {flag_id, document_id, chunk_id, section_title,
      │    page_number, rule_id, rule_description, severity, clause_text, reason, confidence}
      └─ scan_done event with result{total_chunks, classified_chunks, total_flags, timing}

• Playbooks: JSON files in backend/legal_retrieval/playbook/playbooks/ (e.g. default.json); loaded by loader.py (load_playbook, list_playbooks).
• Persistence: flags stored in SQLite via persistence.py (save_flags, get_all_flags, update_flag_status); statuses open → resolved / dismissed / escalated.
• Classifier: classifier.py — embedding-based clause-type classification with _get_embedder() singleton and _build_reference_embeddings().

7. LLM integration (backend/legal_retrieval/llm.py)

• OllamaLLM wrapper around Ollama at http://localhost:11434.
• The LangGraph router binds ChatOllama(model, base_url, temperature=0.3).
• RAG prompts live in prompts.py (chat/RAG prompts, extraction, no-context and error templates); intent prompts in prompts_intent.py.

8. Non-streaming chat pipeline (backend/legal_retrieval/chat.py)

create_chat_pipeline() → ChatPipeline.chat(query, mode, top_k) — retrieval → RAG prompt with context → LLM generation → formatted response with citations. Used by /api/chat for retrieval_only/direct modes.

9. RAG graph pipeline (backend/legal_retrieval/pipeline_graph.py)

An alternative node-graph implementation with RetrievalNode, VerificationNode, GenerationNode, PostGenCheckNode, AbstainNode, SuccessNode; build_rag_pipeline(), run_graph_pipeline(), and streaming variants (run_retrieval_only_streaming, run_direct_streaming, run_graph_pipeline_streaming).