| 1 | # Project Architecture & Guidelines (LangGraph + Python + FastAPI) |
| 2 | |
| 3 | ## 1. System Architecture |
| 4 | - **Framework**: FastAPI (Async ASGI with Uvicorn). |
| 5 | - **Agent Orchestration**: LangGraph (`StateGraph`, `MessagesState`) for stateful cyclic multi-agent loops. |
| 6 | - **Data Validation & Tools**: Pydantic v2 (`BaseModel`, `Field`) for type validation and structured LLM tool definitions. |
| 7 | - **Vector Database**: Qdrant / Chroma / pgvector for semantic retrieval augmented generation (RAG). |
| 8 | - **LLM Integration**: LangChain Chat Models (`ChatOpenAI`, `ChatAnthropic`, local Ollama/vLLM endpoints). |
| 9 | |
| 10 | ## 2. Directory & Module Organization |
| 11 | - `app/api/`: FastAPI route handlers (e.g. `POST /api/chat`, `GET /api/runs/{run_id}`). |
| 12 | - `app/agents/`: |
| 13 | - `graph.py`: StateGraph definition, node connections, and compiled runnable workflow. |
| 14 | - `state.py`: TypedDict or Pydantic definitions of agent memory and intermediate scratchpads. |
| 15 | - `nodes.py`: Discrete execution steps (reasoner, retriever, tool_executor, reviewer). |
| 16 | - `app/tools/`: Custom Pydantic-validated tool functions decorated with `@tool`. |
| 17 | - `app/core/`: Configuration, LLM client singleton, and OpenTelemetry / LangSmith tracing. |
| 18 | |
| 19 | ## 3. Agent Graph Guardrails |
| 20 | - Always bound maximum cyclic iterations using `recursion_limit` (e.g. `graph.compile().invoke(..., {"recursion_limit": 25})`). |
| 21 | - Model agent state explicitly: use `Annotated[list[BaseMessage], add_messages]` to ensure message append behavior without overwriting history. |
| 22 | - Implement conditional routing edges (`add_conditional_edges`) checking tool call requests before executing external side effects. |
| 23 | |
| 24 | ## 4. Structured Output & Tool Execution |
| 25 | - Never parse raw LLM strings with regular expressions. Use `model.with_structured_output(PydanticSchema)` or native tool calling. |
| 26 | - Sandbox tool execution with error boundaries: catch tool exceptions and return error messages back to the agent loop to allow self-correction. |
| 27 | - Enforce timeout limits on external API calls executed by agents. |
| 28 | |
| 29 | ## 5. Streaming & Observability |
| 30 | - Expose streaming responses using Server-Sent Events (SSE) via FastAPI `StreamingResponse` using `graph.astream_events(version="v2")`. |
| 31 | - Enable LangSmith or OpenTelemetry tracing via environment variables for complete inspection of prompt tokens, latency, and tool inputs. |
| 32 | |
| 33 | ## 6. Testing Conventions |
| 34 | - Unit test individual graph nodes as pure functions (given state, assert output state) before testing the compiled graph end-to-end. |
| 35 | - Use LangSmith evaluation datasets or `pytest` fixtures with recorded LLM responses (cassettes) to keep tests deterministic and avoid burning API credits on every CI run. |
| 36 | - Test tool-calling contracts by asserting the Pydantic schema rejects malformed arguments, not just that valid ones pass. |
| 37 | - Assert `recursion_limit` is actually enforced with a test that forces a cyclic loop and expects a `GraphRecursionError`. |
| 38 | |
| 39 | ## 7. Git Workflow & PR Conventions |
| 40 | - Conventional Commits (`feat:`, `fix:`, `refactor:`) scoped to the agent/node, e.g. `fix(agents/retriever): handle empty vector store results`. |
| 41 | - Any PR changing a prompt template or tool schema must include before/after LangSmith trace links or eval scores in the description. |
| 42 | - Require `pytest`, `ruff check .`, and `mypy .` green before merge. |
| 43 | - Flag prompt-only changes distinctly (`prompt:` commit prefix) so trace regressions are easy to bisect later. |