Meilisearch

# MISSION: Scaffold "Meilisearch" Production Architecture
You are an expert Senior Staff Software Architect and Full-Stack Engineer. Your mission is to scaffold and implement a production-grade, highly reliable, and modular codebase following the proven architecture of **Meilisearch**.
---
## 1. PROJECT SPECIFICATIONS & BENCHMARK
- **Reference Architecture**: Meilisearch
- **What It Does**: Meilisearch is a Rust search engine server: an actix-web HTTP API (crates/meilisearch) in front of the milli indexing and ranking engine, which stores each index as its own LMDB environment via heed and supports vector search through arroy, hannoy, and candle embeddings.
- **Domain & Category**: Search Engine
- **Production Scale**: 20+ people
- **Development Mode**: CLASSIC
- **Architectural Rationale**: Keeping the milli search engine as a dependency-free library separate from the actix-web server crate lets ranking and indexing logic be tested and reused independently of HTTP, while per-index LMDB environments make each index independently portable and snapshot-able.
- **Live Website Reference**: https://meilisearch.com
- **Source Repository**: https://github.com/meilisearch/meilisearch
---
## 2. PRODUCTION TECH STACK
- **Full Stack Array**: Rust, Actix-web, LMDB, Tokio, Docker, Prometheus
- **Primary Language(s)**: Rust
- **License of the reference repo**: NOASSERTION
- **Backend & APIs**: Rust — Implements the entire engine across the crates/ Cargo workspace (20+ crates).; Actix-web — HTTP server framework exposing the REST API in the meilisearch crate.; Tokio — Async runtime used by the HTTP server and the index scheduler for background tasks.; milli — Standalone indexing, ranking, tokenization, and filtering engine with no HTTP dependency.; index-scheduler — Owns the task queue and background indexing jobs, decoupled from request handling.; meilisearch-auth — Issues and validates HMAC-signed API keys with scoped permissions.
- **Data & persistence**: LMDB — Each search index is a separate memory-mapped LMDB environment via the heed binding.; S3 — Optional snapshot and backup target for the index scheduler via the rusty-s3 client.; Vector search (arroy/hannoy) — Approximate nearest-neighbor index used for semantic and hybrid search over embeddings.
- **Infrastructure & deploy**: Docker — Multi-stage Rust build produces a minimal Alpine-based runtime image.; Prometheus — Exposes process and server metrics; a sample scrape config ships in assets/.; Grafana — A ready-made dashboard for visualizing the exposed Prometheus metrics.
- **Tooling, testing & ops**: GitHub Actions — Runs the test suite, fuzzers, benchmarks, and publishes Docker/apt/brew releases.; xtask — In-repo cargo xtask crate drives benchmark workloads and release automation.; fuzzers — Dedicated fuzzing crate targets the indexing and filter-parsing code paths.
---
## 3. KEY ARCHITECTURAL DECISIONS (audited from https://github.com/meilisearch/meilisearch @ 1380ada)
1. **Search engine and HTTP server are separate crates**: crates/milli implements indexing, ranking, filtering, and tokenization as a standalone library with no actix-web dependency, while crates/meilisearch wraps it in an HTTP server, so the engine itself could be embedded or tested without a server process.
2. **Each index is its own LMDB environment, not a shared database**: crates/milli/Cargo.toml depends on heed (an LMDB binding) with serde-json and serde-bincode features. Every index gets its own memory-mapped LMDB store, so individual indexes can be moved, backed up, or snapshotted independently of the rest of the instance.
3. **Task scheduling is a dedicated crate, decoupled from request handling**: crates/index-scheduler/Cargo.toml is separate from crates/meilisearch and owns the task queue, background indexing jobs, and optional S3 snapshotting via rusty-s3, so slow indexing work does not block the request-handling code path.
4. **Vector search combines a local ANN index with optional remote embeddings**: crates/milli/Cargo.toml depends on arroy and hannoy for approximate nearest-neighbor search plus candle-core, candle-transformers, tokenizers, and hf-hub for local embedding models, while external-crates/async-openai lets the same pipeline call remote providers such as OpenAI.
5. **Open-core licensing keeps Community and Enterprise Editions in one repo**: The repository ships three license files (LICENSE, LICENSE-MIT, LICENSE-EE), and crates such as meilisearch-types and index-scheduler gate code behind an "enterprise" Cargo feature flag, so Community Edition and Enterprise Edition sources live side by side instead of in separate repositories.
6. **The HTTP API is generated from code with utoipa, not hand-written**: crates/meilisearch and crates/meilisearch-types depend on utoipa (with actix_extras and openapi_extensions features), and a dedicated crates/openapi-generator crate produces the OpenAPI spec from route and type definitions, checked in CI by check-openapi-file.yml.
---
## 4. NON-NEGOTIABLE ARCHITECTURAL GUARDRAILS
1. **Monorepo & Modular Separation**:
   - Structure as a Turborepo monorepo with strict package boundaries:
     - `apps/web`: Application UI, routing, layouts, and server endpoints.
     - `packages/ui`: Shared design tokens, CSS variables, and Radix UI primitive components.
     - `packages/db`: Database schemas, client singleton, declarative migrations, and seed scripts.
     - `packages/config`: Shared TypeScript, ESLint, and build configurations.
2. **Strict Type Safety & Zero `any` Policy**:
   - Enable `strict: true`, `noImplicitAny: true`, and `strictNullChecks: true`.
   - Validate ALL external inputs, API request bodies, and query parameters with **Zod** schemas before execution.
3. **Frontend & Rendering Guidelines**:
   - Isolate interactive UI state to leaf components. Keep core pages lightweight and performant.
4. **Design System & Aesthetics**:
   - Keep every color, radius, shadow and font in a single token file (CSS variables) and consume tokens everywhere; never hardcode hex values in components.
   - Prefer crisp 1px borders and one subtle shadow scale over blurry default shadows. Pair one sans-serif for body/headings with one monospace for tags, badges, metrics, and code.
5. **Data Layer & Reliability**:
   - Write declarative schema definitions with foreign keys, composite indexes on queried filters, and automated timestamp triggers.
   - Use connection pooling and prepared statements for serverless database execution.
---
## 5. STEP-BY-STEP SCAFFOLDING ROADMAP
- **Phase 1: Workspace & Root Config**: Initialize package manager, monorepo configuration (`turbo.json`, `tsconfig.base.json`, `package.json`).
- **Phase 2: Database Schema & Client**: Set up the data layer (LMDB, S3, Vector search (arroy/hannoy)): client, connection pool, models, and migration scripts.
- **Phase 3: Design Tokens & UI Primitives**: Build accessible `Button`, `Input`, `Card`, `Badge`, and layout wrappers inside `packages/ui`.
- **Phase 4: Core Application Routes & Handlers**: Implement primary authentication, user session handling, and application routes.
- **Phase 5: Quality Assurance & Build Verification**: Run `tsc --noEmit`, ESLint, Prettier, and smoke test suites to ensure zero compilation or runtime errors.
---
## 6. EXECUTION INSTRUCTIONS
1. Review all specifications, architectural guardrails, and stack choices above.
2. Present the full monorepo directory tree structure.
3. Systematically generate the complete, production-ready codebase according to the 5-phase roadmap above — starting with the root workspace setup, followed by the database schema, UI design system package, and full-stack application routes until the repository is fully scaffolded and ready to run.