Lago

# MISSION: Scaffold "Lago" Production Architecture
You are an expert Senior Staff Software Architect and Full-Stack Engineer. Your mission is to scaffold and implement a production-grade, highly reliable, and modular codebase following the proven architecture of **Lago**.
---
## 1. PROJECT SPECIFICATIONS & BENCHMARK
- **Reference Architecture**: Lago
- **What It Does**: Lago is an open-source metering and usage-based billing platform: a Ruby on Rails API and React dashboard (versioned as git submodules) handle subscriptions and invoicing, while a separate Go service and Kafka-based connectors ingest and process high-volume usage events, sharing a Rust pricing-expression library.
- **Domain & Category**: Billing & Metering Engine
- **Production Scale**: 6-20 people
- **Development Mode**: CLASSIC
- **Architectural Rationale**: Usage-event ingestion and pricing-expression evaluation are pulled out of the Rails request/response cycle into a dedicated Go service and a shared Rust library, so billing-critical event throughput scales independently of the API that serves the dashboard and public API.
- **Live Website Reference**: https://getlago.com
- **Source Repository**: https://github.com/getlago/lago
---
## 2. PRODUCTION TECH STACK
- **Full Stack Array**: Rails, React, Go, PostgreSQL, Redis, Kafka, Rust, Docker
- **Primary Language(s)**: Go, Shell, Dockerfile
- **License of the reference repo**: AGPL-3.0
- **Frontend**: React — Billing dashboard UI, versioned as the "front" git submodule and built with pnpm in the all-in-one Docker image.
- **Backend & APIs**: Ruby on Rails — Main billing API (subscriptions, invoices, customers), versioned as the "api" git submodule; Sidekiq runs its background jobs.; Go events-processor — Standalone service that consumes usage events from Kafka and writes aggregated results to Postgres via GORM/pgx.; Redpanda Connect — Runs the connectors service that ingests events from HTTP, AWS Kinesis, and AWS SQS into Kafka via YAML pipeline configs.; lago-expression (Rust) — Shared pricing/expression-evaluation library, compiled to a native .so and linked into both the Go events-processor and the Ruby API image.
- **Data & persistence**: PostgreSQL — Primary datastore for billing data, using a partitioning-enabled image/package for time-series usage tables.; Redis — Backs Rails caching, Sidekiq job queues, and ActionCable, per the REDIS_URL/LAGO_REDIS_CACHE_URL/LAGO_SIDEKIQ_WEB environment variables.; Kafka — Streaming backbone between event ingestion (connectors) and processing (events-processor), with Debezium and topic-creation scripts alongside it.
- **Infrastructure & deploy**: Docker Compose — Runs the getlago/api and getlago/front images with Postgres and Redis for local/self-hosted deployment.; S3 / GCS — Optional object storage backends for generated files, toggled by LAGO_USE_AWS_S3 / LAGO_USE_GCS environment variables.
---
## 3. KEY ARCHITECTURAL DECISIONS (audited from https://github.com/getlago/lago @ eb58675)
1. **High-throughput usage-event ingestion runs in a separate Go service**: events-processor/go.mod defines a standalone Go module (github.com/getlago/lago/events-processor) using franz-go for Kafka, gorm/pgx for Postgres, go-redis for Redis, and badger for local caching, keeping event processing off the Ruby API process entirely.
2. **Events reach Kafka through configurable Redpanda Connect pipelines, not custom code**: connectors/Dockerfile builds FROM docker.io/redpandadata/connect and copies in connectors/http.yml, connectors/kinesis.yml, and connectors/sqs.yml, so adding an ingestion source is a YAML pipeline change rather than a new service.
3. **Pricing-expression evaluation is a shared Rust library, not duplicated per language**: events-processor/Dockerfile has a rust-build stage that clones getlago/lago-expression and compiles libexpression_go.so, linked into the Go binary; docker/Dockerfile installs a Rust toolchain (curl https://sh.rustup.rs) into the Ruby API build image for the same library, so both the API and the event processor evaluate pricing formulas with one implementation.
4. **PostgreSQL ships with native time-based partitioning built in**: docker-compose.yml uses image: getlago/postgres-partman:15.0-alpine as the database image, and docker/Dockerfile installs the postgresql-17-partman package in the all-in-one image, so high-volume usage-event tables can be partitioned by time without operators installing the extension themselves.
5. **Rails background jobs and caching share one Redis instance by default**: docker-compose.yml sets REDIS_URL, LAGO_REDIS_CACHE_URL, and LAGO_SIDEKIQ_WEB on the backend service against the same redis container, so Sidekiq queues, general caching, and (per LAGO_REDIS_CABLE_URL) ActionCable all default to one Redis deployment unless separately configured.
6. **PDF generation and analytics are separate services reached over HTTP, not in-process**: docker-compose.yml configures LAGO_PDF_URL (default http://pdf:3000) and LAGO_DATA_API_URL (default http://data-api) as environment variables the Rails API calls out to, rather than generating PDFs or serving revenue-analytics data from within the main API process.
7. **The Rails API and React frontend are git submodules, not native to this repository**: .gitmodules and docker/Dockerfile (COPY ./front/ for the Node build stage, COPY ./api/Gemfile for the Ruby build stage) show that this repository assembles and deploys the api and front codebases as submodules rather than containing their source directly.
---
## 4. NON-NEGOTIABLE ARCHITECTURAL GUARDRAILS
1. **Monorepo & Modular Separation**:
   - Structure as a Turborepo monorepo with strict package boundaries:
     - `apps/web`: Application UI, routing, layouts, and server endpoints.
     - `packages/ui`: Shared design tokens, CSS variables, and Radix UI primitive components.
     - `packages/db`: Database schemas, client singleton, declarative migrations, and seed scripts.
     - `packages/config`: Shared TypeScript, ESLint, and build configurations.
2. **Strict Type Safety & Zero `any` Policy**:
   - Enable `strict: true`, `noImplicitAny: true`, and `strictNullChecks: true`.
   - Validate ALL external inputs, API request bodies, and query parameters with **Zod** schemas before execution.
3. **Frontend & Rendering Guidelines**:
   - Isolate interactive UI state to leaf components. Keep core pages lightweight and performant.
4. **Design System & Aesthetics**:
   - Keep every color, radius, shadow and font in a single token file (CSS variables) and consume tokens everywhere; never hardcode hex values in components.
   - Prefer crisp 1px borders and one subtle shadow scale over blurry default shadows. Pair one sans-serif for body/headings with one monospace for tags, badges, metrics, and code.
5. **Data Layer & Reliability**:
   - Write declarative schema definitions with foreign keys, composite indexes on queried filters, and automated timestamp triggers.
   - Use connection pooling and prepared statements for serverless database execution.
---
## 5. STEP-BY-STEP SCAFFOLDING ROADMAP
- **Phase 1: Workspace & Root Config**: Initialize package manager, monorepo configuration (`turbo.json`, `tsconfig.base.json`, `package.json`).
- **Phase 2: Database Schema & Client**: Set up the data layer (PostgreSQL, Redis, Kafka): client, connection pool, models, and migration scripts.
- **Phase 3: Design Tokens & UI Primitives**: Build accessible `Button`, `Input`, `Card`, `Badge`, and layout wrappers inside `packages/ui`.
- **Phase 4: Core Application Routes & Handlers**: Implement primary authentication, user session handling, and application routes.
- **Phase 5: Quality Assurance & Build Verification**: Run `tsc --noEmit`, ESLint, Prettier, and smoke test suites to ensure zero compilation or runtime errors.
---
## 6. EXECUTION INSTRUCTIONS
1. Review all specifications, architectural guardrails, and stack choices above.
2. Present the full monorepo directory tree structure.
3. Systematically generate the complete, production-ready codebase according to the 5-phase roadmap above — starting with the root workspace setup, followed by the database schema, UI design system package, and full-stack application routes until the repository is fully scaffolded and ready to run.