WeKnora

# MISSION: Scaffold "WeKnora" Production Architecture
You are an expert Senior Staff Software Architect and Full-Stack Engineer. Your mission is to scaffold and implement a production-grade, highly reliable, and modular codebase following the proven architecture of **WeKnora**.
---
## 1. PROJECT SPECIFICATIONS & BENCHMARK
- **Reference Architecture**: WeKnora
- **What It Does**: WeKnora is Tencent's open-source LLM knowledge platform. It turns raw documents into a queryable RAG knowledge base with hybrid retrieval, reranking and an agent layer, and exposes it through a web UI, API, CLI, MCP and chat-app integrations.
- **Domain & Category**: LLM Knowledge Base & RAG Platform
- **Production Scale**: 20+ people
- **Development Mode**: CLASSIC
- **Architectural Rationale**: Retrieval, agents and the API sit in one Go service, and parsing is delegated to a Python gRPC service. Storage backends are pluggable, so the same codebase runs as a lite SQLite binary or a full Kubernetes deployment on existing Postgres, Elasticsearch or vector databases.
- **Live Website Reference**: https://weknora.weixin.qq.com
- **Source Repository**: https://github.com/Tencent/WeKnora
---
## 2. PRODUCTION TECH STACK
- **Full Stack Array**: Go, Gin, Vue, Pinia, Vite, Python, gRPC, PostgreSQL, pgvector, MySQL, SQLite, Redis, Elasticsearch, OpenSearch, Qdrant, Milvus, Neo4j, MinIO, Ollama, Model Context Protocol, Docker, Helm, Kubernetes, Swagger
- **Primary Language(s)**: Go, Vue, TypeScript, Python, JavaScript, PLpgSQL
- **License of the reference repo**: NOASSERTION
- **Frontend**: Vue — Vue 3 SPA with TDesign components for knowledge bases, chat, agents and settings; Pinia — Client state management with vue-router and vue-i18n; Vite — Frontend build, served in production by an nginx image that proxies the API
- **Backend & APIs**: Go — Main server (cmd/server, internal/*) plus a Go client SDK (client/) and a Cobra CLI (cli/); Gin — HTTP router and middleware for the REST API documented with Swagger (docs/swagger.yaml); Python — docreader service that parses PDF, Office, HTML and ebook files into chunks; gRPC — Protocol between the Go app and the Python docreader (docreader/proto, connect-go)
- **Data & persistence**: PostgreSQL — Primary database (pgx) with ParadeDB-flavoured migrations for full-text and vector search; pgvector — Vector storage option inside Postgres for embeddings; MySQL — Alternative relational backend with its own migration set (migrations/mysql); SQLite — Lite single-binary mode with sqlite-vec for local vector search (migrations/sqlite); Redis — Cache and Asynq task queue backend for asynchronous ingestion jobs; Elasticsearch — Pluggable keyword/vector retrieval backend (v7 and v8 clients), alongside OpenSearch; Qdrant — Pluggable vector database backend; Milvus — Pluggable vector database backend, with a migration tool in cmd/milvus-migrate; Neo4j — Graph store for knowledge-graph style retrieval; MinIO — Object storage for uploaded files, with S3, Alibaba OSS and KS3 as alternatives
- **Infrastructure & deploy**: Docker — Compose stack of frontend, app, docreader and data services; separate sandbox image; Helm — Kubernetes chart under helm/ with a CI workflow; Swagger — Generated OpenAPI docs with a contract test (docs/swagger_contract_test.go)
- **Tooling, testing & ops**: Ollama — Local model provider (Ollama Go client) next to hosted OpenAI-compatible providers; Model Context Protocol — MCP client and server in internal/mcp, internal/mcpserver and a Python mcp-server package
---
## 3. KEY ARCHITECTURAL DECISIONS (audited from https://github.com/Tencent/WeKnora @ 4364e61)
1. **Go core, Python only where document parsing needs it**: The server, agents, retrieval and API are Go (go.mod, internal/), while file parsing lives in a separate Python docreader service (markitdown, pypdf, python-docx, trafilatura) reached over gRPC, so each side uses its strongest ecosystem.
2. **Every storage layer is pluggable**: go.mod pulls clients for Postgres/pgvector, MySQL, SQLite/sqlite-vec, DuckDB, Elasticsearch 7/8, OpenSearch, Qdrant, Milvus, Neo4j, Redis, MinIO, S3, OSS and KS3, and migrations/ keeps separate mysql, paradedb and sqlite trees, letting deployments pick their existing infrastructure.
3. **A lite single-binary edition next to the full stack**: docs/LITE.md, .env.lite.example, scripts/package-lite.sh, Formula/weknora-lite.rb and deploy/weknora-lite.service package a SQLite-backed build for Homebrew and systemd, while docker-compose.yml and helm/ cover the full multi-service deployment.
4. **Asynchronous ingestion through a Redis-backed queue**: hibiken/asynq on Redis, ants goroutine pools and robfig/cron handle document parsing, chunking and embedding as background jobs so uploads return quickly.
5. **Agents with sandboxed tools and IM integrations**: internal/agent, internal/sandbox and internal/localsandbox run agent tools (with an E2B client and a Docker sandbox image), while Lark, Slack, DingTalk and a WeChat mini program (miniprogram/) expose the knowledge base in chat apps.
---
## 4. NON-NEGOTIABLE ARCHITECTURAL GUARDRAILS
1. **Monorepo & Modular Separation**:
   - Structure as a Turborepo monorepo with strict package boundaries:
     - `apps/web`: Application UI, routing, layouts, and server endpoints.
     - `packages/ui`: Shared design tokens, CSS variables, and Radix UI primitive components.
     - `packages/db`: Database schemas, client singleton, declarative migrations, and seed scripts.
     - `packages/config`: Shared TypeScript, ESLint, and build configurations.
2. **Strict Type Safety & Zero `any` Policy**:
   - Enable `strict: true`, `noImplicitAny: true`, and `strictNullChecks: true`.
   - Validate ALL external inputs, API request bodies, and query parameters with **Zod** schemas before execution.
3. **Frontend & Rendering Guidelines**:
   - Utilize Vite 6 with TanStack Router for fully type-safe routing. Manage server state and caching via TanStack Query v5 with optimistic UI updates.
4. **Design System & Aesthetics**:
   - Keep every color, radius, shadow and font in a single token file (CSS variables) and consume tokens everywhere; never hardcode hex values in components.
   - Prefer crisp 1px borders and one subtle shadow scale over blurry default shadows. Pair one sans-serif for body/headings with one monospace for tags, badges, metrics, and code.
5. **Data Layer & Reliability**:
   - Write declarative schema definitions with foreign keys, composite indexes on queried filters, and automated timestamp triggers.
   - Use connection pooling and prepared statements for serverless database execution.
---
## 5. STEP-BY-STEP SCAFFOLDING ROADMAP
- **Phase 1: Workspace & Root Config**: Initialize package manager, monorepo configuration (`turbo.json`, `tsconfig.base.json`, `package.json`).
- **Phase 2: Database Schema & Client**: Set up the data layer (PostgreSQL, pgvector, MySQL, SQLite, Redis, Elasticsearch, Qdrant, Milvus, Neo4j, MinIO): client, connection pool, models, and migration scripts.
- **Phase 3: Design Tokens & UI Primitives**: Build accessible `Button`, `Input`, `Card`, `Badge`, and layout wrappers inside `packages/ui`.
- **Phase 4: Core Application Routes & Handlers**: Implement primary authentication, user session handling, and application routes.
- **Phase 5: Quality Assurance & Build Verification**: Run `tsc --noEmit`, ESLint, Prettier, and smoke test suites to ensure zero compilation or runtime errors.
---
## 6. EXECUTION INSTRUCTIONS
1. Review all specifications, architectural guardrails, and stack choices above.
2. Present the full monorepo directory tree structure.
3. Systematically generate the complete, production-ready codebase according to the 5-phase roadmap above — starting with the root workspace setup, followed by the database schema, UI design system package, and full-stack application routes until the repository is fully scaffolded and ready to run.