OpenSEO

# MISSION: Scaffold "OpenSEO" Production Architecture
You are an expert Senior Staff Software Architect and Full-Stack Engineer. Your mission is to scaffold and implement a production-grade, highly reliable, and modular codebase following the proven architecture of **OpenSEO**.
---
## 1. PROJECT SPECIFICATIONS & BENCHMARK
- **Reference Architecture**: OpenSEO
- **What It Does**: OpenSEO is an open-source alternative to Semrush and Ahrefs for keyword research, rank tracking, backlinks, site audits and AI visibility. It runs on Cloudflare Workers with the user's own DataForSEO key and exposes an MCP server for AI agents.
- **Domain & Category**: Open-Source SEO Research Platform
- **Production Scale**: 2-5 people
- **Development Mode**: HYBRID
- **Architectural Rationale**: Running on Cloudflare Workers with D1, Workflows and Durable Objects makes it free-tier friendly to self-host and to scale when hosted. Supporting both D1 and Postgres through Drizzle, plus a Docker path, keeps the project portable beyond Cloudflare.
- **Live Website Reference**: https://openseo.so
- **Source Repository**: https://github.com/every-app/open-seo
---
## 2. PRODUCTION TECH STACK
- **Full Stack Array**: TypeScript, React, TanStack Start, TanStack Query, Tailwind CSS, Cloudflare, Cloudflare Workers, Cloudflare D1, PostgreSQL, Drizzle, Better Auth, Vercel AI SDK, Model Context Protocol, Zod, PostHog, Docker, Vite, Vitest, Playwright
- **Primary Language(s)**: TypeScript, MDX, JavaScript, CSS
- **License of the reference repo**: MIT
- **Frontend**: React — React 19 UI for keyword research, rank tracking, backlinks, audits and reports; TanStack Start — Full-stack React framework with file routes (src/routes, routeTree.gen.ts) and server functions; TanStack Query — Client data fetching and caching, alongside TanStack Form and TanStack Table; Tailwind CSS — Styling with Tailwind 4 plus daisyUI components, lucide icons and Recharts charts
- **Backend & APIs**: Cloudflare Workers — Runtime for the app worker (src/server.ts) and a separate site-audit worker (src/audit-worker.ts); Better Auth — Authentication with API keys, organizations and generated Drizzle schemas for D1 and Postgres; Zod — Validation for server functions, MCP tool inputs and configuration
- **Data & persistence**: Cloudflare D1 — Default SQLite database on Cloudflare, migrated with wrangler d1 from drizzle/; PostgreSQL — Alternative database with its own migration set (drizzle-pg/) and a D1-to-Postgres runbook; Drizzle — ORM and migration generator for both the SQLite (D1) and Postgres dialects
- **Infrastructure & deploy**: Cloudflare — Hosting via wrangler and Alchemy deploys, Workflows for audits/rank checks, cron triggers; Docker — Self-host image (Dockerfile.selfhost) run by compose.yaml on workerd with local bindings; PostHog — Product analytics on client and server, with source map upload in a CI workflow; Playwright — End-to-end tests in e2e/ plus a deliberately broken badseo fixture site
- **Tooling, testing & ops**: Vercel AI SDK — Powers SAM, the in-app SEO agent, via ai/@ai-sdk/react with an OpenRouter provider; Model Context Protocol — MCP server (with Cloudflare OAuth provider) so external agents can query SEO data; Cloudflare Agents — Durable Object chat agent (SamChatAgent) persisting each chat session in DO SQLite
---
## 3. KEY ARCHITECTURAL DECISIONS (audited from https://github.com/every-app/open-seo @ 0ffff93)
1. **Cloudflare Workers as the primary runtime, Docker as a local fallback**: wrangler.jsonc defines the app worker with smart placement, Workflows, Durable Objects and cron triggers; compose.yaml runs the same bundle in a container with CLOUDFLARE_INCLUDE_PROCESS_ENV so bindings come from environment variables for single-machine self-hosting.
2. **Site audits split into a separate worker to avoid memory limits**: Comments in wrangler.jsonc explain that multi-MB Lighthouse payloads OOMed the main worker, so the audit engine moved to open-seo-audit (wrangler.audit.jsonc, src/audit-worker.ts) and is reached through a service binding and a cross-script Workflow binding.
3. **Dual database dialects with parallel Drizzle migrations**: drizzle/ holds D1 (SQLite) migrations and drizzle-pg/ holds Postgres ones, with separate drizzle configs and Better Auth schema generators per dialect; runbooks/ and scripts/migrate-d1-to-postgres.ts cover moving a live install from D1 to Postgres.
4. **Bring-your-own DataForSEO key instead of owning SEO data**: SEO data comes from the DataForSEO API with the user's own key (docs/DATAFORSEO_API_KEY.md). The hosted version meters usage with autumn-js, as described in specs/0002-hosted-dataforseo-metering-with-autumn.md.
5. **Agent-first surface through MCP, skills and an in-app agent**: The MCP server and skills (plugins/openseo, .claude-plugin, .cursor-plugin) let external agents use the data, while SAM runs in-app as a Cloudflare Agents Durable Object built on the Vercel AI SDK with OpenRouter models.
6. **Spec-driven development with a supply-chain delay**: Features start as numbered design docs in specs/ (0001-0014). pnpm-workspace.yaml sets minimumReleaseAge so new package versions wait about eight days before install, and it lists security overrides with a GHSA reference for each.
---
## 4. NON-NEGOTIABLE ARCHITECTURAL GUARDRAILS
1. **Monorepo & Modular Separation**:
   - Structure as a Turborepo monorepo with strict package boundaries:
     - `apps/web`: Application UI, routing, layouts, and server endpoints.
     - `packages/ui`: Shared design tokens, CSS variables, and Radix UI primitive components.
     - `packages/db`: Database schemas, client singleton, declarative migrations, and seed scripts.
     - `packages/config`: Shared TypeScript, ESLint, and build configurations.
2. **Strict Type Safety & Zero `any` Policy**:
   - Enable `strict: true`, `noImplicitAny: true`, and `strictNullChecks: true`.
   - Validate ALL external inputs, API request bodies, and query parameters with **Zod** schemas before execution.
3. **Frontend & Rendering Guidelines**:
   - Utilize Vite 6 with TanStack Router for fully type-safe routing. Manage server state and caching via TanStack Query v5 with optimistic UI updates.
4. **Design System & Aesthetics**:
   - Keep every color, radius, shadow and font in a single token file (CSS variables) and consume tokens everywhere; never hardcode hex values in components.
   - Prefer crisp 1px borders and one subtle shadow scale over blurry default shadows. Pair one sans-serif for body/headings with one monospace for tags, badges, metrics, and code.
5. **Data Layer & Reliability**:
   - Write declarative schema definitions with foreign keys, composite indexes on queried filters, and automated timestamp triggers.
   - Use connection pooling and prepared statements for serverless database execution.
---
## 5. STEP-BY-STEP SCAFFOLDING ROADMAP
- **Phase 1: Workspace & Root Config**: Initialize package manager, monorepo configuration (`turbo.json`, `tsconfig.base.json`, `package.json`).
- **Phase 2: Database Schema & Client**: Set up the data layer (Cloudflare D1, PostgreSQL, Drizzle): client, connection pool, models, and migration scripts.
- **Phase 3: Design Tokens & UI Primitives**: Build accessible `Button`, `Input`, `Card`, `Badge`, and layout wrappers inside `packages/ui`.
- **Phase 4: Core Application Routes & Handlers**: Implement primary authentication, user session handling, and application routes.
- **Phase 5: Quality Assurance & Build Verification**: Run `tsc --noEmit`, ESLint, Prettier, and smoke test suites to ensure zero compilation or runtime errors.
---
## 6. EXECUTION INSTRUCTIONS
1. Review all specifications, architectural guardrails, and stack choices above.
2. Present the full monorepo directory tree structure.
3. Systematically generate the complete, production-ready codebase according to the 5-phase roadmap above — starting with the root workspace setup, followed by the database schema, UI design system package, and full-stack application routes until the repository is fully scaffolded and ready to run.