What can I edit in the pipeline editor?

A visual canvas with nodes for each retrieval stage — dense, sparse, rerank, ingest, scan — plus toggles and bypass switches. Removing the rerank node sets skip_rerank; dense and sparse presence sets the search mode and fusion weights.

Is the layout persisted?

Yes. Layouts save to localStorage (pipeline-layout-v1), debounced on every change, and the app re-applies them on restart. A default layout is restored if none is saved.

Does the editor affect the backend?

Yes — the editor POSTs the layout to the backend pipeline config endpoint, so changes apply at runtime: top_k overrides, rerank on/off, and search-mode fusion weights all come from the canvas.

📚 Docs / Pipeline Editor — visual pipeline configuration

Pipeline Editor — visual pipeline configuration

The pipeline editor is a ReactFlow canvas (right panel of the app) that visualizes the retrieval/scan pipeline as nodes and edges, lets users strip or restore sub-steps, persists the layout, and supports JSON export/import.

Scope note: this is both a visualization surface and a configuration surface. Since the POST /api/pipeline/config endpoint was added, the exported layout reconfigures the Python backend at runtime — rerank on/off from the rerank node, search mode + dense/sparse fusion weights from the retrieval nodes, and top_k overrides. Config is runtime-only (resets on restart). See PIPELINE_JSON.md §10 for the derivation rules.

1. Where it lives

frontend/src/components/Pipeline/
- PipelineEditor.tsx — canvas, state, persistence, export/import, live status highlighting
- nodes.tsx — node components + the shared buildBypassEdges helper
- defaults.ts — defaultPipelineNodes / defaultPipelineEdges and the particle edge type
- NodePalette.tsx — draggable node types, sub-node toggle switches, legend, reset/export/import buttons
- pipelineStorage.ts — localStorage persistence + JSON serialize/parse
- ParticleEdge.tsx — animated "particle" data-flow edges
Opened via the Pipeline button in the Sources panel header; toggled from the Sidebar's workspace button. It renders in a 560px right panel.

2. Nodes

Node	Type id	Role
Intent Classifier	`intentClassifier`	Classifies greeting vs legal query vs scan
Chitchat	`chitchat`	Fast greeting reply, no retrieval
Planner	`planner`	Writes search plan, dispatches to dense + sparse
Dense Retrieval	`denseRetrieval`	Semantic search over embeddings (ChromaDB)
Sparse Retrieval	`sparseRetrieval`	Keyword / BM25 search
Reranker	`rerank`	Cross-encoder re-scores candidates
Ingest Files	`ingest`	Parse attached files → chunk → embed → store
Compliance Scan	`scan`	Playbook scan + flags
Answer	`answer`	Writes final answer from retrieved documents only
Custom Tool	`tool`	Generic drag-and-drop node

Default flow: intent → planner → dense + sparse → rerank → answer and intent → ingest → scan → answer.

3. Removing a node — auto-bypass

Every node has a ✕ button. Deleting a middle sub-node bypasses it: incoming edges are rewired straight to the outgoing targets. Example: removing the Reranker creates dense → answer and sparse → answer edges, so the flow stays connected.

Implemented by buildBypassEdges(nodeId, edges) (pure helper in nodes.tsx): for each incoming × outgoing pair it builds a bypass edge.
Bypass edges are tagged with ids starting bypass-<nodeId>- and labeled bypass — this tag is what lets re-enabling find and remove them.
The ✕ button (RemoveNodeButton) and the palette toggle share this helper, so both removal paths behave identically.

4. Re-enabling a removed sub-node (palette toggles)

The NodePalette shows the five sub-node types (dense, sparse, rerank, ingest, scan) with an on/off toggle switch (role="switch", brass styling, icon dims to 45% when off):

Off → remove: same auto-bypass as the ✕ button.
On → re-enable: re-adds the node at its default position and re-wires its original edges (planner → dense → rerank), while removing the bypass-<nodeId>-* edges that stood in for it.
Smart restoration: edges whose counterpart node is also absent are skipped, and existing edge ids are never duplicated.

Implementation: PipelineEditor.tsx → toggleSubNode(type) + subNodeStates memo; SUB_NODE_ID_BY_TYPE maps palette type → canonical node id.

5. Persistence (localStorage)

pipelineStorage.ts:

Key: pipeline-layout-v1.
Save: 300ms-debounced on any node/edge change + a beforeunload flush so the last edit survives reload. If the layout equals the shipped defaults, it clears storage instead of saving (so future default updates are never shadowed by a stale copy).
Load: lazy-initialized on mount; falls back to structuredClone of defaults when nothing is saved.
Sanitization: strips runtime-only fields (status, active, selected).

6. Validation on load/import

validateLayout() is shared by localStorage loads and imported files:

Rejects non-objects / missing nodes/edges arrays.
Drops nodes with unknown types (e.g. stale searchDocuments/scanIngest).
Dedupes node ids and edge ids (first occurrence kept).
Drops edges whose source/target nodes don't exist; drops id-less edges.
Coerces unknown edge types to particle.
Accepts a structurally-valid layout with zero nodes — a deliberately emptied canvas round-trips and survives reload.

7. Export / import (JSON)

Export (↓): downloads pipeline-layout-YYYY-MM-DD.json — a pretty-printed {version: 1, nodes, edges} envelope (serializeLayout), sanitized.
Full JSON schema: see PIPELINE_JSON.md — the field-by-field spec (node types, edge rules, named handles, bypass-edge pattern) written so an AI model can author a valid layout file from scratch.
Import (↑): hidden file input (accept=".json,application/json"); reads the file, parses + validates (parsePipelineLayout — accepts the versioned envelope or a bare {nodes, edges}), applies it, and remounts the canvas so the view re-fits.
Reset (↺): clears storage, restores defaults, remounts the canvas.
Transient feedback in the palette header: "Layout imported ✓" / "Invalid layout file ✗" / "Layout exported ✓" (2.5s, timer cleaned up on unmount).

8. Live status highlighting

While a query streams, the canvas highlights the active node and dims/brightens edges:

activeNodeId is derived from the last streaming status keyword (ingest/processing → ingest; scan/playbook/flag → scan; rerank → rerank; dense/embed → dense; sparse/bm25 → sparse; search/retriev → rerank; plan → planner; answer → answer; …).
Nodes get idle / active / done / error status classes; edges animate with particle flows and show dynamic payload previews (query, candidates, results, flags) during streaming.

9. Styling

All pipeline CSS is in index.css under the "Pipeline Editor" section and is theme-variable based (works in light parchment and dark mode): .pipeline-node + status variants, .pipeline-node-remove, .pipeline-node-hint, .pipeline-switch, .pipeline-reset-btn, .pipeline-handle, .pipeline-canvas (ReactFlow overrides incl. minimap and controls), and the particle-edge animations.