Configuration Reference — Complete Environment Variables & Settings

1. Configuration hierarchy

Settings are read in order of precedence (highest wins):

1. Environment variables (PLR_*)     ← highest priority
2. Per-project settings (workspace/history.json)
3. File-based config (api_config.json)
4. Code defaults (config.py)         ← lowest priority

Example:

python# Default in config.py
embedding_batch_size = 32

# Override via environment variable
export PLR_EMBEDDING_BATCH_SIZE=16
# → batch size is now 16

2. Model configuration

Embedding model (BGE-M3)

Example:

bash# Lower batch size for 4GB GPU
export PLR_EMBEDDING_BATCH_SIZE=16

# Force CPU embedding
export PLR_DEVICE=cpu
export PLR_EMBEDDING_USE_FP16=False

Reranker model (BGE-Reranker-v2-M3)

Example:

bash# Smaller batch for low-VRAM GPUs
export PLR_RERANKER_BATCH_SIZE=16

LLM (Ollama)

Why PLR_LLM_NUM_GPU=0 by default:

• Embeddings + reranking are latency-critical for search
• LLM generation is least time-critical
• Keeping LLM on CPU frees entire VRAM budget for embeddings/reranking
• Set to -1 if you prefer faster generation over faster search

Example:

bash# Use LLM on GPU (faster generation, slower search)
export PLR_LLM_NUM_GPU=-1

# Unload LLM immediately after each call (free RAM)
export PLR_LLM_KEEP_ALIVE=0

Device selection

Auto resolution:

1. Try CUDA (NVIDIA GPU)
2. Try MPS (Apple Silicon)
3. Fall back to CPU

Example:

bash# Force CPU (no GPU needed)
export PLR_DEVICE=cpu

3. GPU manager configuration

See GPU_MANAGEMENT.md for GPU manager design.

Residency modes:

• eager — demote always (idle models never occupy VRAM)
• adaptive — keep hot models resident, demote cold (best of both worlds)
• persistent — never demote (all used models stay resident)

Example:

bash# Aggressive GPU freeing (tiny GPU)
export PLR_GPU_RESIDENCY_MODE=eager
export PLR_GPU_MAX_RESIDENT=1

# Persistent (capable GPU)
export PLR_GPU_RESIDENCY_MODE=persistent
export PLR_GPU_MAX_RESIDENT=3

4. Chunking configuration

No env var overrides — chunking config is code-only (edit config.py).

Example (code override):

pythonfrom legal_retrieval.config import get_config, Config, ChunkingConfig
from dataclasses import replace

cfg = get_config()
new_chunking = replace(cfg.chunking, chunk_size=600, overlap=50)
new_cfg = replace(cfg, chunking=new_chunking)

5. Storage configuration

ChromaDB

Other settings (code-only):

• collection_name: legal_chunks
• distance_metric: cosine
• HNSW parameters: m=16, construction_ef=100, search_ef=128

Workspace override:

• When a workspace is active → <workspace>/workspace/chroma_db
• No workspace → data/chroma_db

Paths

All path defaults are in config.py → PathConfig:

Workspace overrides:

• raw_dir → <workspace>/ (uploads land directly in project folder)
• processed_dir → <workspace>/workspace/processed
• chroma_dir → <workspace>/workspace/chroma_db
• bm25_dir → <workspace>/workspace/bm25_index

6. Search & retrieval configuration

No env var overrides — search config is:

1. Code defaults (config.py)
2. Runtime pipeline config (POST /api/pipeline/config)
3. Per-request overrides (ChatRequest.skip_rerank, top_k)

Example (runtime config):

bash# Apply pipeline layout → reconfigures search
curl -X POST http://localhost:8765/api/pipeline/config \
  -H "Content-Type: application/json" \
  -d @pipeline-layout.json

Example (per-request override):

bash# Single query with top_k=10, skip rerank
curl -X POST http://localhost:8765/api/chat \
  -H "Content-Type: application/json" \
  -d '{"query":"What is rescission?","top_k":10,"skip_rerank":true}'

7. Ingestion configuration

No env var overrides — ingestion config is code-only.

Example (disable OCR for faster ingestion):

pythonfrom legal_retrieval.config import get_config, Config, IngestionConfig
from dataclasses import replace

cfg = get_config()
new_ingestion = replace(cfg.ingestion, do_ocr=False, generate_page_images=False)
new_cfg = replace(cfg, ingestion=new_ingestion)

8. Logging configuration

Log files:

• logs/legal_retrieval.log — rotating file (10 MB, 5 backups)
• Console (stderr) — same level as file

Example:

bash# Debug logging
export PLR_LOG_LEVEL=DEBUG
python backend/main.py

Programmatic setup:

pythonfrom legal_retrieval.logging_setup import setup_logging

setup_logging(level="DEBUG", force=True)

9. Agent configuration

What it controls:

• Untagged text after a tool result is treated as chain-of-thought (CoT panel)
• Adaptive window: only the first N short segments count as reasoning
• Longer segments mean the answer body has begun → stop buffering CoT

Example:

bash# More aggressive CoT capture
export PLR_AGENT_POST_TOOL_REASONING_SEGMENTS=5
export PLR_AGENT_POST_TOOL_REASONING_CHARS=500

10. API provider configuration

See API_PROVIDER.md for cloud mode setup.

Configuration file

File: backend/api_config.json (git-ignored)

json{
  "enabled": true,
  "provider": "openai",
  "base_url": "https://api.openai.com/v1",
  "api_key": "sk-...",
  "llm_model": "gpt-4o-mini",
  "embed_model": "text-embedding-3-small",
  "embed_dim": 0,
  "rerank_mode": "auto",
  "temperature": 0.3,
  "max_tokens": 4096,
  "timeout": 120,
  "thinking_budget": 0
}

Environment variable overrides

Precedence: env vars > JSON file > code defaults

Example:

bash# Quick test with OpenAI
export LAWYER_API_ENABLED=1
export LAWYER_PROVIDER=openai
export LAWYER_API_KEY=sk-...
export LAWYER_LLM_MODEL=gpt-4o-mini

python backend/main.py

11. Workspace configuration

Active workspace

Set via API:

bashcurl -X POST http://localhost:8765/api/workspace \
  -H "Content-Type: application/json" \
  -d '{"path":"/absolute/path/to/project"}'

Or via frontend "Work in a folder" button (native directory picker).

Workspace storage

When a workspace is active:

• Raw files → <workspace>/
• ChromaDB → <workspace>/workspace/chroma_db/
• BM25 → <workspace>/workspace/bm25_index/
• Processed → <workspace>/workspace/processed/
• History → <workspace>/workspace/history.json

Runtime-only (resets on restart or DELETE /api/workspace).

Per-project GPU profile

Stored in <workspace>/workspace/history.json under "gpu_profile":

json{
  "gpu_profile": {
    "residency_mode": "adaptive",
    "keep_resident_free_mb": 1024,
    "hot_min_uses": 2,
    "hot_window_seconds": 120,
    "max_resident": 2
  },
  "conversations": [ ... ],
  "working_set": [ ... ]
}

Applied automatically when workspace is selected.

12. Frontend configuration

Build-time configuration

File: frontend/.env.local (git-ignored)

VITE_API_BASE_URL=http://localhost:8765

Usage:

typescriptconst API_BASE = import.meta.env.VITE_API_BASE_URL || 'http://localhost:8765';

Runtime configuration (Electron)

File: frontend/electron/src/main/config.ts

typescriptexport const CONFIG = {
  backend: {
    port: 8765,
    host: '127.0.0.1',
    healthCheckInterval: 500,
    healthCheckTimeout: 30000,
  },
  vite: {
    port: 5173,
  },
};

13. Launcher configuration

File: backend/api_config.json (written by launcher)

See §10 API provider configuration.

Launcher-specific env vars:

14. Configuration files reference

15. Configuration validation

Check current config:

bashcurl http://localhost:8765/api/health

Returns:

json{
  "status": "ok",
  "service": "lawyer-assistant",
  "version": "1.0.0",
  "config": {
    "device": "cuda",
    "embedding_dim": 1024,
    "search_mode": "hybrid",
    "rerank_enabled": true,
    "gpu_manager_enabled": true,
    "workspace_active": true
  }
}

Check GPU config:

bashcurl http://localhost:8765/api/gpu/status

Check API provider:

bashcurl http://localhost:8765/api/providers/status

16. Common configuration scenarios

Scenario 1: Low-VRAM GPU (4GB)

bashexport PLR_GPU_RESIDENCY_MODE=eager
export PLR_GPU_MAX_RESIDENT=1
export PLR_EMBEDDING_BATCH_SIZE=16
export PLR_RERANKER_BATCH_SIZE=16
export PLR_GPU_HEADROOM_MB=512

Scenario 2: High-VRAM GPU (24GB+)

bashexport PLR_GPU_RESIDENCY_MODE=persistent
export PLR_GPU_MAX_RESIDENT=3
export PLR_EMBEDDING_BATCH_SIZE=64
export PLR_RERANKER_BATCH_SIZE=64
export PLR_LLM_NUM_GPU=-1  # use GPU for LLM too

Scenario 3: CPU-only (no GPU)

bashexport PLR_DEVICE=cpu
export PLR_EMBEDDING_USE_FP16=False
export PLR_RERANKER_USE_FP16=False
export PLR_GPU_MANAGER_ENABLED=0
export PLR_EMBEDDING_BATCH_SIZE=4  # smaller for RAM

Scenario 4: Cloud API mode (no local models)

bashexport LAWYER_API_ENABLED=1
export LAWYER_PROVIDER=openai
export LAWYER_API_KEY=sk-...
export LAWYER_LLM_MODEL=gpt-4o-mini
export LAWYER_EMBED_MODEL=text-embedding-3-small

Scenario 5: Debugging

bashexport PLR_LOG_LEVEL=DEBUG
export PLR_GPU_MANAGER_ENABLED=1  # watch GPU logs
python backend/main.py 2>&1 | tee debug.log

17. Configuration troubleshooting

18. Related documentation