Ollama

# MISSION: Scaffold "Ollama" Production Architecture
You are an expert Senior Staff Software Architect and Full-Stack Engineer. Your mission is to scaffold and implement a production-grade, highly reliable, and modular codebase following the proven architecture of **Ollama**.
---
## 1. PROJECT SPECIFICATIONS & BENCHMARK
- **Reference Architecture**: Ollama
- **What It Does**: Ollama is a Go-based local model runner that compiles a C++ llama.cpp inference server per GPU backend (CPU, CUDA, ROCm, Vulkan) plus a dedicated Apple MLX path, fronted by a Gin HTTP API, a CLI, and a React and Vite desktop app.
- **Domain & Category**: Local AI Inference Engine
- **Production Scale**: 1-5 people
- **Development Mode**: AI-AGENT
- **Architectural Rationale**: Inference performance depends on the accelerator, so the C++ llama.cpp server is built as separate CPU, CUDA, ROCm, and Vulkan variants, with native Go code choosing the right one at runtime, while an independent MLX runner handles Apple Silicon instead of one shared code path.
- **Live Website Reference**: https://ollama.com
- **Source Repository**: https://github.com/ollama/ollama
---
## 2. PRODUCTION TECH STACK
- **Full Stack Array**: Go, TypeScript, React, C++, SQLite, Docker, Vite, Tailwind CSS
- **Primary Language(s)**: Go, C, TypeScript
- **License of the reference repo**: MIT
- **Frontend**: React — Implements the desktop app UI under app/ui, including chat and settings screens.; Vite — Builds and serves the desktop app UI.; Tailwind CSS — Styles the desktop app UI.
- **Backend & APIs**: Go — Implements the CLI, HTTP server, model manifest handling, and GPU discovery.; Gin — HTTP web framework used by the Ollama API server.; C++ — llama.cpp-based inference server compiled per GPU backend from the llama/server source tree.
- **Data & persistence**: SQLite — Embedded storage via mattn/go-sqlite3, used for local state such as model metadata.
- **Infrastructure & deploy**: Docker — Multi-stage build compiles CPU, CUDA, ROCm, and Vulkan variants of the llama.cpp server.
---
## 3. KEY ARCHITECTURAL DECISIONS (audited from https://github.com/ollama/ollama @ ec3cc23)
1. **Inference runs through llama.cpp, compiled per accelerator backend**: The Dockerfile builds separate llama-server-cpu, llama-server-cuda_v12, llama-server-cuda_v13, llama-server-rocm_v7_2, and llama-server-vulkan stages from the same llama/server and llama/compat source under cmake presets, publishing each backend as its own scratch-stage artifact under lib/ollama/<variant>.
2. **GPU discovery is implemented natively per platform in Go**: The discover/ package has platform-specific files such as discover/gpu_darwin.go, discover/amd.go, discover/cuda_compat.go, and discover/vulkan.go, so runtime GPU detection for CUDA, ROCm, Vulkan, and Apple hardware is handled in Go rather than shelled out to external tooling.
3. **Apple Silicon gets a dedicated MLX execution path alongside llama.cpp**: The mlx/ and mlxrunner/ directories (mlx/dynamic_darwin.go, mlxrunner/runner.go, mlxrunner/speculate.go) implement a separate inference runner on top of Apple MLX, distinct from the llama.cpp-based llm/ and llama/ packages used for other backends.
4. **OpenAI and Anthropic compatible API surfaces are first-class**: The openai/ package (openai/openai.go, openai/responses.go) and middleware/openai.go and middleware/anthropic.go implement request/response translation layers, letting existing OpenAI- or Anthropic-format clients talk to a local Ollama server without modification.
5. **The desktop app is a native shell around a Vite/React web UI**: app/ui/app/package.json builds a React 19 application with vite build and TanStack Router (@tanstack/react-router), while native platform integration (tray icon, auto-update, OS windowing) lives in sibling Go directories such as app/wintray, app/updater, and app/webview rather than in the web UI itself.
6. **Local model state is stored in embedded SQLite, not an external database**: go.mod depends directly on github.com/mattn/go-sqlite3, and the manifest/ package (manifest/manifest.go, manifest/layer.go) manages on-disk model manifests, keeping Ollama runnable as a single binary with no external database service.
---
## 4. NON-NEGOTIABLE ARCHITECTURAL GUARDRAILS
1. **Monorepo & Modular Separation**:
   - Structure as a Turborepo monorepo with strict package boundaries:
     - `apps/web`: Application UI, routing, layouts, and server endpoints.
     - `packages/ui`: Shared design tokens, CSS variables, and Radix UI primitive components.
     - `packages/db`: Database schemas, client singleton, declarative migrations, and seed scripts.
     - `packages/config`: Shared TypeScript, ESLint, and build configurations.
2. **Strict Type Safety & Zero `any` Policy**:
   - Enable `strict: true`, `noImplicitAny: true`, and `strictNullChecks: true`.
   - Validate ALL external inputs, API request bodies, and query parameters with **Zod** schemas before execution.
3. **Frontend & Rendering Guidelines**:
   - Utilize Vite 6 with TanStack Router for fully type-safe routing. Manage server state and caching via TanStack Query v5 with optimistic UI updates.
4. **Design System & Aesthetics**:
   - Keep every color, radius, shadow and font in a single token file (CSS variables) and consume tokens everywhere; never hardcode hex values in components.
   - Prefer crisp 1px borders and one subtle shadow scale over blurry default shadows. Pair one sans-serif for body/headings with one monospace for tags, badges, metrics, and code.
5. **Data Layer & Reliability**:
   - Write declarative schema definitions with foreign keys, composite indexes on queried filters, and automated timestamp triggers.
   - Use connection pooling and prepared statements for serverless database execution.
---
## 5. STEP-BY-STEP SCAFFOLDING ROADMAP
- **Phase 1: Workspace & Root Config**: Initialize package manager, monorepo configuration (`turbo.json`, `tsconfig.base.json`, `package.json`).
- **Phase 2: Database Schema & Client**: Set up the data layer (SQLite): client, connection pool, models, and migration scripts.
- **Phase 3: Design Tokens & UI Primitives**: Build accessible `Button`, `Input`, `Card`, `Badge`, and layout wrappers inside `packages/ui`.
- **Phase 4: Core Application Routes & Handlers**: Implement primary authentication, user session handling, and application routes.
- **Phase 5: Quality Assurance & Build Verification**: Run `tsc --noEmit`, ESLint, Prettier, and smoke test suites to ensure zero compilation or runtime errors.
---
## 6. EXECUTION INSTRUCTIONS
1. Review all specifications, architectural guardrails, and stack choices above.
2. Present the full monorepo directory tree structure.
3. Systematically generate the complete, production-ready codebase according to the 5-phase roadmap above — starting with the root workspace setup, followed by the database schema, UI design system package, and full-stack application routes until the repository is fully scaffolded and ready to run.