Changelog — Lawyer Assistant

Conventions: date groups are newest-first; within a group, commits are listed oldest-first (chronological). Items marked Unreleased (working tree) are implemented but not yet committed. Cross-references go to the knowledge base.

Unreleased (working tree)

Work completed on top of the latest commit (52a822a). Mostly the documentation set and the pipeline-editor feature pack.

Documentation set (this session)

• Full knowledge base added under docs/: README.md (index), QUICKSTART.md, TROUBLESHOOTING.md, ARCHITECTURE.md, BACKEND.md, FRONTEND.md, PIPELINE_EDITOR.md, GLOSSARY.md, AGENT_KNOWLEDGE_BASE.md, and this CHANGELOG.md — every doc fact-checked against the source.
• AGENTS.md at the repo root: read order, topic → source-file table, known traps, and answering etiquette for AI agents.
• Root README.md rewritten as the current non-technical user guide (install + use via the launcher), replacing the old CLI-era content.
• Docs aligned with the unified repo layout — every structure reference in docs/ now matches the current tree: the whole Python side under backend/, the whole frontend + Electron shell under frontend/, plus the launcher/ (setup page), assets/ (screenshots) and docs/. Stale CLI/Flask-era path claims removed.

Pipeline editor feature pack

• Sub-node split: the Search Documents and Scan Document nodes were decomposed into sub-nodes (dense, sparse, rerank, ingest, scan) so users can remove individual stages (e.g. disable reranking).
• Bypass removal: removing a sub-node auto-wires a tagged bypass edge (bypass-<nodeId>-…) so the remaining nodes still connect.
• Per-node toggle re-enable: re-enable a removed sub-node from the palette without clearing the layout; the original edges are rewired automatically.
• Persistence: node/edge layout saved to localStorage (pipeline-layout-v1) with debounced saves, so removals/edges survive reloads; a Reset to default button restores the original flow.
• Export/import: the layout can be exported/imported as a JSON file (versioned format) to share retrieval configurations across machines.
• Zero-node guard: validation prevents committing an empty pipeline.
• Backend runtime config: POST /api/pipeline/config applies the layout JSON to retrieval at runtime — rerank on/off from the rerank node, search_mode (hybrid/dense/sparse) + fusion weights from the dense/sparse nodes, and top_k/top_k_retrieval/weight overrides. Runtime-only (resets on restart); GET reads it, DELETE resets. See PIPELINE_JSON.md §10.

Workspace folders (working tree)

• Select a folder = select the workspace. The "Work in a folder" button now opens a native Electron directory dialog (folder:select IPC, absolute path) instead of only reading the folder name from a webkitdirectory input.
• New backend workspace API: POST/GET/DELETE /api/workspace and POST /api/workspace/ingest. Setting a workspace re-points storage — raw uploads land directly in the folder, and processed/chroma_db/bm25_index live under it — so search & ingestion are isolated per workspace instead of the static project data/.
• Frontend wiring: App.tsx sets the workspace on pick, refreshes the discovered file list, and passes workspace file absolute paths on send (no byte re-upload); WorkspacePanel shows a workspace card with an Ingest all files action. Browser fallback (no Electron bridge) keeps the old webkitdirectory behavior.
• Runtime-only: the override resets on restart / DELETE.

UI fixes (this session)

• Chat input container height reduced ~50%.
• Attachment (+) button wired to the upload backend, and drag-and-drop of files onto the composer added (committed as fe7e13a).

Citation integrity & retrieval warnings (working tree)

• Citation schema validation (sanitize_citations in backend/legal_retrieval/utils.py, applied on both routers' done events and in chat.py/pipeline.py): a citation marker must be an integer in [1, len(sources)]. Malformed markers like [18] (no such source) and [1.9] / [14] / [10.2] (in-document section numbers leaking into citation output) are stripped before reaching the UI.
• Citation content verification: each [N] is checked against the cited chunk's actual text — the chunk must share enough of the claim's distinctive content words (lexical-entailment heuristic). A chunk that does not support the sentence loses its citation marker, so one top-ranked chunk can no longer be cited for five unrelated clause types.
• Answer prompt hardened (prompts_intent.py "answer" + router.py system prompt): explicit rules against leaking section numbers into [N] markers, against reusing one source for unrelated clauses, and a document-scope rule — when results span multiple unrelated companies/entities, section the answer by document instead of blending into one false "this Agreement" narrative, and state the ambiguity.
• Retrieval-quality caveats (derive_retrieval_warning): done events now carry retrieval_warning when the top score is below 70% (weak match) or results span multiple documents; the UI renders it as a visible banner above the answer (client-side fallback for legacy messages).

Answer-transparency & trust fixes (this session)

• Abstain signal shipped end-to-end: the backend now emits answer_found: false on the done event (both routers + /api/chat response) whenever the pipeline abstains. The UI trusts this signal and:
  • hides the confidence percentage bar and the real citation chips on abstained answers,
  • shows a muted "Closest match found (not used)" row instead of a real citation when chunks were retrieved but rejected,
  • shows the three-state ConfidenceBadge (Found / Partial / Not Found) instead of a raw percentage on real answers.
• Step counts moved to a debug view: "Process (27 steps, 0 tools)" and "Pipeline (N steps)" labels no longer appear in the default user-facing view; step/tool counts live inside the collapsible process section only.
• Temp/lock files filtered (~$… Word locks, .~lock…# LibreOffice locks, .tmp/.temp/.bak/.swp/.part): excluded from directory ingestion (ingestion.py, retrieval.py), from VectorStore.list_files (retroactive — already-indexed junk no longer lists), and from the workspace file listing.

Security hardening & CI (this session)

• Content-Security-Policy injected into the renderer HTML by a Vite plugin (contentSecurityPolicy() in frontend/vite.config.ts): production ships script-src 'self' (no inline scripts, no unsafe-eval), development allows 'unsafe-inline' + the HMR websocket — both modes silence Electron's insecure-CSP warning.
• webkitdirectory warning fixed — the folder-picker input passes the Chrome-only attribute as a string with a proper react module type augmentation instead of @ts-ignore.
• CSP regression guard in CI — frontend/scripts/check-csp.mjs fails the build (exit 1) when dist/index.html is missing its CSP meta tag or any policy contains unsafe-eval. Wired into npm run build (frontend self-check) and added as a standalone check:csp script + a GitHub Actions workflow (.github/workflows/ci.yml) that installs, builds, and verifies the CSP on push/PR — the security posture can't regress silently.
• Electron renderer security guard — frontend/electron/scripts/check-electron-security.mjs statically scans the main-process sources (frontend/electron/src/main) and fails the build when any BrowserWindow does not set sandbox: true, contextIsolation: true, and nodeIntegration: false explicitly (or uses an unverifiable spread/variable webPreferences), or when the no-sandbox command-line switch is used anywhere. window.ts now sets sandbox: true explicitly. Wired into npm run build (electron self-check), a check:security entry, and a dedicated electron-security job in the CI workflow — CSP covers what the renderer may load, these flags cover what it can do.

2026-07-31 — light mode & drag-and-drop

2026-07-30 — pipeline editor, chain of thought, retrieval tuning

2026-07-29 — streaming, approval dashboard

2026-07-28 — pipeline loop & centralized prompting

2026-07-27 — benchmarking

2026-07-26 — initial release

How to keep this file accurate

• When a new feature ships, add an entry at the top (Unreleased → dated block on commit).
• If you are an agent answering "what changed recently?", read this file first, then verify against git log and the feature docs linked above.
• Deep dives: PIPELINE_EDITOR.md (editor), BACKEND.md (engine), FRONTEND.md (UI), TROUBLESHOOTING.md (known failures).