crates.io

# MISSION: Scaffold "crates.io" Production Architecture
You are an expert Senior Staff Software Architect and Full-Stack Engineer. Your mission is to scaffold and implement a production-grade, highly reliable, and modular codebase following the proven architecture of **crates.io**.
---
## 1. PROJECT SPECIFICATIONS & BENCHMARK
- **Reference Architecture**: crates.io
- **What It Does**: crates.io is the official Rust package registry: a Rust backend on Axum, Diesel and PostgreSQL with a custom background worker, crate files in object storage behind a CDN, and a SvelteKit frontend in TypeScript.
- **Domain & Category**: Package Registry
- **Production Scale**: 1M+ MAU
- **Development Mode**: CLASSIC
- **Architectural Rationale**: crates.io runs one pinned-dependency Rust binary as migrate, web and worker processes, splits domains into small internal crates, and generates its TypeScript client from OpenAPI types so the SvelteKit frontend and Axum API share one contract.
- **Live Website Reference**: https://crates.io
- **Source Repository**: https://github.com/rust-lang/crates.io
---
## 2. PRODUCTION TECH STACK
- **Full Stack Array**: Rust, Axum, Diesel, PostgreSQL, Tokio, SvelteKit, Svelte, TypeScript, Vite, AWS, Sentry, OpenTelemetry, Playwright, Storybook
- **Primary Language(s)**: Rust, TypeScript, Svelte, PLpgSQL
- **License of the reference repo**: Apache-2.0
- **Frontend**: SvelteKit — The crates.io web app in svelte/, built with Vite and the static adapter, with Shiki highlighting and Chart.js download graphs.; TypeScript — Frontend language; an OpenAPI-generated API client (openapi-fetch, openapi-typescript) lives in packages/crates-io-api-client.; MSW — Mock Service Worker data layer in packages/crates-io-msw for frontend development and tests without a backend.
- **Backend & APIs**: Rust — Backend language for the crates-io binary and all crates/ members (edition 2024), with every dependency pinned with =.; Axum — HTTP framework for the API (axum 0.8 with macros and matched-path, axum-extra for typed headers and middleware).; utoipa — Generates the OpenAPI description of the API types that the TypeScript client is generated from.; Background worker — Custom job system in crates/crates_io_worker, run as a separate background_worker process.
- **Data & persistence**: PostgreSQL — Primary database, with migrations dating back to 2014 and full-text search through diesel_full_text_search.; Diesel — ORM with diesel-async and a deadpool connection pool; diesel_migrations runs migrations on release.; git index — crates_io_index maintains the package index with git2, alongside crate files in object storage.
- **Infrastructure & deploy**: Heroku — Procfile defines release (migrate), web (server) and background_worker processes; .buildpacks and Aptfile configure the build.; AWS — object_store with the aws feature for crate files, plus CloudFront and SQS SDK clients.; Fastly — CDN API client in crates/crates_io_fastly; the README credits Fastly for CDN services.; Sentry — Error tracking with tower and axum integrations.; OpenTelemetry — Metrics exported over OTLP.; Playwright — End-to-end tests in e2e/ with axe accessibility checks and Percy visual snapshots.
- **Tooling, testing & ops**: AGENTS.md — Structured development documentation for coding agents, referenced from the README; docs/AI-TOOLS.md covers AI tool use.
---
## 3. KEY ARCHITECTURAL DECISIONS (audited from https://github.com/rust-lang/crates.io @ 204522d)
1. **One Rust binary with three process roles**: The Procfile runs the same crates-io binary as release (migrate), web (server) and background_worker, so migrations, HTTP serving and background jobs share code and deploy together while scaling independently on Heroku.
2. **Many small internal crates under crates/**: The workspace (members = ["crates/*"]) splits concerns into crates such as crates_io_database, crates_io_index, crates_io_worker, crates_io_tarball, crates_io_trustpub, crates_io_cdn_logs, crates_io_fastly and crates_io_github_app, each with its own dependency list and tests.
3. **Every dependency pinned to an exact version**: Cargo.toml and every crate manifest pin dependencies with = (for example axum = "=0.8.9", diesel = "=2.3.13"), and .github/renovate.json5 configures Renovate, so upgrades arrive as explicit pull requests instead of drifting through semver ranges.
4. **Typed API contract from Rust to TypeScript**: crates_io_api_types derives OpenAPI schemas with utoipa, and packages/crates-io-api-client regenerates a typed client with openapi-typescript and openapi-fetch, which the SvelteKit app and the MSW mock layer both consume.
5. **Frontend developed against mocks**: packages/crates-io-msw provides an MSW data layer, and svelte/package.json has dev:msw, dev:local and dev:staging scripts plus Storybook with Chromatic, so UI work and Playwright e2e tests run without a full backend.
---
## 4. NON-NEGOTIABLE ARCHITECTURAL GUARDRAILS
1. **Monorepo & Modular Separation**:
   - Structure as a Turborepo monorepo with strict package boundaries:
     - `apps/web`: Application UI, routing, layouts, and server endpoints.
     - `packages/ui`: Shared design tokens, CSS variables, and Radix UI primitive components.
     - `packages/db`: Database schemas, client singleton, declarative migrations, and seed scripts.
     - `packages/config`: Shared TypeScript, ESLint, and build configurations.
2. **Strict Type Safety & Zero `any` Policy**:
   - Enable `strict: true`, `noImplicitAny: true`, and `strictNullChecks: true`.
   - Validate ALL external inputs, API request bodies, and query parameters with **Zod** schemas before execution.
3. **Frontend & Rendering Guidelines**:
   - Utilize Vite 6 with TanStack Router for fully type-safe routing. Manage server state and caching via TanStack Query v5 with optimistic UI updates.
4. **Design System & Aesthetics**:
   - Keep every color, radius, shadow and font in a single token file (CSS variables) and consume tokens everywhere; never hardcode hex values in components.
   - Prefer crisp 1px borders and one subtle shadow scale over blurry default shadows. Pair one sans-serif for body/headings with one monospace for tags, badges, metrics, and code.
5. **Data Layer & Reliability**:
   - Write declarative schema definitions with foreign keys, composite indexes on queried filters, and automated timestamp triggers.
   - Use connection pooling and prepared statements for serverless database execution.
---
## 5. STEP-BY-STEP SCAFFOLDING ROADMAP
- **Phase 1: Workspace & Root Config**: Initialize package manager, monorepo configuration (`turbo.json`, `tsconfig.base.json`, `package.json`).
- **Phase 2: Database Schema & Client**: Set up the data layer (PostgreSQL, Diesel, git index): client, connection pool, models, and migration scripts.
- **Phase 3: Design Tokens & UI Primitives**: Build accessible `Button`, `Input`, `Card`, `Badge`, and layout wrappers inside `packages/ui`.
- **Phase 4: Core Application Routes & Handlers**: Implement primary authentication, user session handling, and application routes.
- **Phase 5: Quality Assurance & Build Verification**: Run `tsc --noEmit`, ESLint, Prettier, and smoke test suites to ensure zero compilation or runtime errors.
---
## 6. EXECUTION INSTRUCTIONS
1. Review all specifications, architectural guardrails, and stack choices above.
2. Present the full monorepo directory tree structure.
3. Systematically generate the complete, production-ready codebase according to the 5-phase roadmap above — starting with the root workspace setup, followed by the database schema, UI design system package, and full-stack application routes until the repository is fully scaffolded and ready to run.