Data & Storage Infra Tech Stacks
Databases, object storage, data pipelines, and query engines. 14 verified production examples.
AGENTS.md Rules & Agent Skills for Data & Storage Infra
Next.js + ClickHouse + Prisma (Analytics & Observability)
Architecture guidelines for high-volume event ingestion and analytical queries using ClickHouse alongside a PostgreSQL/Prisma control plane.
advancedRust + Axum + PostgreSQL (sqlx & Tokio)
High-concurrency systems architecture for Rust 2024, Axum 0.8, Tokio async runtime, compile-time SQL verification with sqlx, and Tower middleware.
Production Data & Storage Infra Projects
elekcsv
High-performance, headless, locale-aware CSV data ingestion and validation toolkit. Unifies parsing, fuzzy column mapping, and validation into a single pass. Compiles schemas into optimized raw V8 JavaScript functions (new Function()) and utilizes dense bitmasks for O(1) cell error lookups, eliminating Garbage Collector (GC) thrashing on 100k+ row browser imports.
ClickHouse
Blazing-fast analytical database; processes queries orders of magnitude faster than traditional row-based engines by scanning only relevant columns with SIMD vector instructions.
Hasura
Turns databases into high-throughput GraphQL APIs in seconds; compiles GraphQL queries directly into optimized single-query SQL statements in Haskell with enterprise access controls.
MinIO
Standard for private cloud object storage; engineered in Go with SIMD assembly acceleration to deliver multi-gigabyte/sec read/write throughput compatible with the Amazon S3 API.
PostGraphile
Compiles PostgreSQL database schemas, functions, and Row-Level Security (RLS) policies into an instant, high-performance GraphQL API without N+1 query overhead. Generates optimized single-query SQL execution plans directly from incoming GraphQL ASTs.
SeaweedFS
Highly scalable distributed file system optimized for billions of small and large files (Haystack + Ficus design). Separates directory metadata from volume servers with Raft consensus, eliminating single-point metadata bottlenecks and achieving O(1) disk lookups.
Garage (RustFS)
Lightweight, self-hosted geo-distributed S3-compatible object storage designed to run across multiple data centers and low-bandwidth connections. Uses CRDT metadata replication and RocksDB block storage with minimal memory overhead.
QuestDB
High-performance columnar time-series database engineered for financial tick data, IoT metrics, and real-time telemetry. Uses a zero-garbage-collection Java execution engine with C++ SIMD vectorization and lock-free out-of-order data ingestion.
Dragonfly
Next-generation drop-in replacement for Redis and Memcached. Delivers 25x higher throughput and sub-millisecond tail latency by utilizing a modern shared-nothing multi-threaded architecture with Linux io_uring asynchronous I/O and custom memory serialization.
SurrealDB
Unified document, graph, and relational cloud database written in Rust. Replaces complex backend stacks with schema-full/schema-less flexibility, real-time WebSocket live queries, granular row-level permissions, and distributed key-value storage backends (TiKV/RocksDB).
ElectricSQL
Local-first architecture moving the network off the critical user interaction path. Elixir actor processes manage Postgres logical replication streams, broadcasting differential "Shapes" to embedded SQLite on client devices with CRDT-based active-active conflict resolution.
Crowd.dev
Community intelligence platform; event-driven worker architecture tracks open-source developer engagement across GitHub, Discord, and Slack.
Mage AI
Modern data orchestration tool; Python execution engine with interactive React notebook UI for building real-time data pipelines.
Meilisearch
Blazing fast sub-50ms search-as-you-type: custom Rust indexing engine with typo-tolerance out of the box.
Frequently Asked Questions
How many Data & Storage Infra tech stacks are in the directory?
14 verified production Data & Storage Infra projects are currently indexed, each with a full architecture breakdown and copy-paste AI scaffolding prompt.
What AGENTS.md rules or agent skills apply to Data & Storage Infra projects?
Next.js + ClickHouse + Prisma (Analytics & Observability), Rust + Axum + PostgreSQL (sqlx & Tokio) are battle-tested architecture rules for this category, each with a full AGENTS.md, CLAUDE.md, Cursor .mdc file and a Claude Code SKILL.md.
How do I get a custom stack recommendation for a Data & Storage Infra project?
Take the STACK IT FAST architecture survey and select "Data & Storage Infra" as your project type to get a ranked rule recommendation and an instant AI scaffolding prompt tailored to a "Data & Storage Infra" project.