QuestDB is a high-performance open-source time-series database engineered with SQL support, zero-GC off-heap Java, and SIMD-vectorized execution.
- Language
- Java
- Database
- PostgreSQL
- Hosting
- Docker
- License
- Apache-2.0
- Running for
- 4 years
- Team
- 20+ People
Why this architecture
Combining a zero-GC Java engine with C++/Rust vector acceleration gives QuestDB ultra-low latency ingestion and microsecond SQL queries over billions of metrics.
Tech stack by layer
5 technologies · audited Sep 17, 2026- PostgreSQLExposes the PostgreSQL wire protocol allowing standard psql, JDBC, and BI tools to connect.
- DockerProduction Dockerfile packaging the precompiled QuestDB daemon and Web Console.
QuestDB architecture diagram
Open SVGDiagram as text
- SQL clients (psql · JDBC · BI tools) → PG wire server (Postgres protocol): SQL
- Web Console (built-in UI) → PG wire server (Postgres protocol): SQL
- Ingest clients (InfluxDB Line Protocol) → ILP ingestion (out-of-order writes): line protocol
- PG wire server (Postgres protocol) → SQL engine (Java zero-GC · SIMD): queries
- ILP ingestion (out-of-order writes) → Column store (mmap · Parquet): append
- SQL engine (Java zero-GC · SIMD) → Column store (mmap · Parquet): scan
- SQL engine (Java zero-GC · SIMD) → Native kernels (C++ · Rust): aggregate
Key architectural decisions
5 decisions- 01
Zero-GC architecture using off-heap memory management
QuestDB avoids Java Garbage Collection pauses during high-throughput ingestion by allocating data structures directly in off-heap memory via sun.misc.Unsafe, preventing latency spikes.
- 02
Column-oriented, append-only disk storage with memory mapping
Data is partitioned chronologically by timestamp and written in contiguous columnar files using mmap, allowing parallel queries to read only the necessary columns directly from page cache.
- 03
Dual native protocol support: PostgreSQL Wire and InfluxDB Line Protocol
Ingestion supports high-throughput InfluxDB Line Protocol (ILP) over TCP for millions of rows/sec, while queries execute through the PostgreSQL wire protocol or HTTP REST endpoints.
- 04
C++ and Rust SIMD vectorization for numerical aggregation
Native kernels in core/rust and core/CMakeLists.txt leverage AVX-512 and AVX2 CPU extensions to compute mathematical aggregations across millions of data points per millisecond.
- 05
Lock-free out-of-order data ingestion engine
Incoming historical or delayed metrics are committed without global table locks through an out-of-order commit algorithm that reconciles timestamp partitions concurrently.
How QuestDB is built
How QuestDB is structured
QuestDB is organized as a high-performance database repository centered around a modular Maven workspace (pom.xml, core/pom.xml). The architecture couples a custom zero-allocation Java storage engine with native C++ and Rust computational kernels:
core/: Houses the primary database engine, SQL parser, off-heap memory allocators, columnar storage readers, network protocol listeners (PostgreSQL Wire, HTTP, InfluxDB Line Protocol), and the embedded web console.core/rust/: Native Rust components integrated via JNI, providing hardware-accelerated SIMD filters, vectorized operations, and Apache Parquet format exporters (docs/parquet-metadata.md).core/CMakeLists.txt: C++ compilation toolchain providing SIMD math routines (AVX-512, AVX2, NEON) invoked directly from Java through foreign function bindings.benchmarks/: Comprehensive TSBS (Time Series Benchmark Suite) configurations (benchmarks/pom.xml) and profiling harnesses to continuously assert latency SLAs.artifacts/: Integration connectors, including Kafka connectors, Grafana datasources, and DBeaver plugin configurations.
Frontend
QuestDB ships with an embedded Web Console served directly from the database's built-in HTTP server on port 9000 (.github/console.webp). The console allows users to execute SQL queries interactively, visualize time-series metrics on live charts, explore schema layouts, and monitor disk partition allocations without installing desktop database clients.
Backend & APIs
The database server runs on modern Java (JDK 17/21) within core/. Traditional Java database engines frequently suffer from garbage collection (GC) latency spikes when handling millions of incoming records per second. QuestDB solves this by engineering a custom zero-GC architecture:
- Objects are never allocated on the Java heap during the query execution or ingestion hot paths.
- Memory is allocated and managed explicitly in off-heap native memory using low-level pointers and
sun.misc.Unsafe. - Query execution plans operate on primitive column arrays directly, avoiding boxed object wrappers.
QuestDB exposes multiple network ingestion and querying interfaces:
- InfluxDB Line Protocol (ILP): A high-throughput TCP socket server on port 9009 designed for high-frequency telemetry ingestion from Telegraf, Vector, or custom application clients.
- PostgreSQL Wire Protocol: Emulates the Postgres protocol on port 8812, providing immediate compatibility with Postgres drivers (
pg,psql, psycopg, JDBC). - HTTP REST API: Offers JSON and CSV query endpoints on port 9000 alongside bulk
/writeendpoints for web integrations.
Data & persistence
QuestDB implements an append-only, columnar storage format optimized specifically for time-series and financial tick data:
- Chronological Partitioning: Tables with designated timestamp columns are partitioned into discrete directories by hour, day, week, month, or year.
- Columnar Files: Each column is stored in its own pair of files: data (
.d) and index offsets (.k). Writing a record appends fixed-size primitives to individual column files without moving existing data. - Memory-Mapped I/O: Files are read via memory-mapped buffers (
mmap). The operating system kernel manages page caches automatically, enabling O(1) sequential column access during aggregation. - Out-of-Order (O3) Ingestion: If metrics arrive late or out of sequence, QuestDB's lock-free O3 commit algorithm merges out-of-order batches into existing partitions without locking the entire table or blocking concurrent read queries.
Build, test & deploy
The build pipeline bridges Java and native toolchains:
- Build System: Maven coordinates compilation of Java sources (
mvn clean install), while CMake and Cargo build native shared libraries (.so,.dylib,.dll) packaged into the final JAR. - Testing: Automated regression suites in
ci/run unit tests, fuzz testing (ci/test-fuzz.yml), and bytecode size verification (ci/check-bytecode-size.sh). - Containerization: The production
core/Dockerfilebuilds a standalone Debian/Ubuntu image containing the compiled binary and entrypoint script (core/docker-entrypoint.sh).
Self-hosting notes
QuestDB is designed for simple, self-contained single-node or clustered deployments:
- Run the official Docker container:
docker run -p 9000:9000 -p 9009:9009 -p 8812:8812 questdb/questdb. - Mount a host volume at
/var/lib/questdbto ensure persistent storage for database tables and configuration files. - Tuning off-heap memory: In high-throughput production environments, set
JAVA_OPTS="-XX:MaxDirectMemorySize=..."to allocate sufficient off-heap native memory for column buffers and active partition caches.
What to copy (and what not to)
Architectural lessons from QuestDB:
- Off-heap memory management in managed languages: Bypassing garbage collection pauses by allocating query buffers off-heap enables high-throughput data processing in languages like Java or C#.
- Simultaneous support for ingest protocols: Providing both InfluxDB Line Protocol (optimized for write throughput) and the PostgreSQL wire protocol (optimized for query compatibility) gives users the best of both worlds.
- Columnar file partitioning by timestamp: Partitioning directories chronologically enables effortless data retention management—dropping an entire day or month of expired data is as simple as deleting an OS directory.
Avoid attempting zero-GC off-heap architectures unless sub-millisecond p99 latency guarantees are an absolute business requirement, as manual memory management introduces severe complexity and crashes if memory pointers are mishandled.
Sources & repo audit
- Audited
- Sep 17, 2026
- Commit
- fc7a28b
- License
- Apache-2.0
- core/pom.xml (Core engine Maven dependencies and JVM compile configuration)
- core/CMakeLists.txt (Native C++ SIMD compilation and vector acceleration)
- core/rust (Rust native bindings for vectorized processing and Parquet)
- core/Dockerfile (Production container image packaging the QuestDB distribution)
Independent analysis of repository at github.com/questdb/questdb. Spotted an inaccuracy? Use the claim form to request a correction.
Show this stack on your README or website
[](https://stackitfast.com/project/questdb)Paste it into your README. It shows your current stack and links to this page.
Scaffold it with your agent
Paste this prompt into Claude Code, Cursor, Windsurf or AGY to start a project with QuestDB's architecture.
- 1Copy the promptThe full markdown spec, with every layer and decision.
- 2Open your AI toolClaude Code, Cursor, Windsurf or Copilot, in a new repo.
- 3Paste and scaffoldUse it as the first instruction; review before you ship.
# MISSION: Scaffold "QuestDB" Production Architecture
You are an expert Senior Staff Software Architect and Full-Stack Engineer. Your mission is to scaffold and implement a production-grade, highly reliable, and modular codebase following the proven architecture of **QuestDB**.
---
## 1. PROJECT SPECIFICATIONS & BENCHMARK
- **Reference Architecture**: QuestDB
- **What It Does**: QuestDB is a high-performance open-source time-series database engineered with SQL support, zero-GC off-heap Java, and SIMD-vectorized execution.
- **Domain & Category**: Time-Series SQL Database
- **Production Scale**: 20+ People
- **Development Mode**: CLASSIC
- **Architectural Rationale**: Combining a zero-GC Java engine with C++/Rust vector acceleration gives QuestDB ultra-low latency ingestion and microsecond SQL queries over billions of metrics.
- **Live Website Reference**: https://questdb.com
- **Source Repository**: https://github.com/questdb/questdb
---
## 2. PRODUCTION TECH STACK
- **Full Stack Array**: Java, Rust, C++, Docker, PostgreSQL
- **Primary Language(s)**: Java, Rust, C++
- **License of the reference repo**: Apache-2.0
- **Backend & APIs**: Java — Zero-GC off-heap execution engine, SQL query planner, and storage manager in core/pom.xml.; Rust — High-performance native libraries for SIMD acceleration and Parquet metadata in core/rust.; C++ — SIMD-optimized native aggregation kernels compiled via CMake in core/CMakeLists.txt.
- **Data & persistence**: PostgreSQL — Exposes the PostgreSQL wire protocol allowing standard psql, JDBC, and BI tools to connect.
- **Infrastructure & deploy**: Docker — Production Dockerfile packaging the precompiled QuestDB daemon and Web Console.
---
## 3. KEY ARCHITECTURAL DECISIONS (audited from https://github.com/questdb/questdb @ fc7a28b)
1. **Zero-GC architecture using off-heap memory management**: QuestDB avoids Java Garbage Collection pauses during high-throughput ingestion by allocating data structures directly in off-heap memory via sun.misc.Unsafe, preventing latency spikes.
2. **Column-oriented, append-only disk storage with memory mapping**: Data is partitioned chronologically by timestamp and written in contiguous columnar files using mmap, allowing parallel queries to read only the necessary columns directly from page cache.
3. **Dual native protocol support: PostgreSQL Wire and InfluxDB Line Protocol**: Ingestion supports high-throughput InfluxDB Line Protocol (ILP) over TCP for millions of rows/sec, while queries execute through the PostgreSQL wire protocol or HTTP REST endpoints.
4. **C++ and Rust SIMD vectorization for numerical aggregation**: Native kernels in core/rust and core/CMakeLists.txt leverage AVX-512 and AVX2 CPU extensions to compute mathematical aggregations across millions of data points per millisecond.
5. **Lock-free out-of-order data ingestion engine**: Incoming historical or delayed metrics are committed without global table locks through an out-of-order commit algorithm that reconciles timestamp partitions concurrently.
---
## 4. NON-NEGOTIABLE ARCHITECTURAL GUARDRAILS
1. **Monorepo & Modular Separation**:
- Structure as a Turborepo monorepo with strict package boundaries:
- `apps/web`: Application UI, routing, layouts, and server endpoints.
- `packages/ui`: Shared design tokens, CSS variables, and Radix UI primitive components.
- `packages/db`: Database schemas, client singleton, declarative migrations, and seed scripts.
- `packages/config`: Shared TypeScript, ESLint, and build configurations.
2. **Strict Type Safety & Zero `any` Policy**:
- Enable `strict: true`, `noImplicitAny: true`, and `strictNullChecks: true`.
- Validate ALL external inputs, API request bodies, and query parameters with **Zod** schemas before execution.
3. **Frontend & Rendering Guidelines**:
- Isolate interactive UI state to leaf components. Keep core pages lightweight and performant.
4. **Design System & Aesthetics**:
- Keep every color, radius, shadow and font in a single token file (CSS variables) and consume tokens everywhere; never hardcode hex values in components.
- Prefer crisp 1px borders and one subtle shadow scale over blurry default shadows. Pair one sans-serif for body/headings with one monospace for tags, badges, metrics, and code.
5. **Data Layer & Reliability**:
- Write declarative schema definitions with foreign keys, composite indexes on queried filters, and automated timestamp triggers.
- Use connection pooling and prepared statements for serverless database execution.
---
## 5. STEP-BY-STEP SCAFFOLDING ROADMAP
- **Phase 1: Workspace & Root Config**: Initialize package manager, monorepo configuration (`turbo.json`, `tsconfig.base.json`, `package.json`).
- **Phase 2: Database Schema & Client**: Set up the data layer (PostgreSQL): client, connection pool, models, and migration scripts.
- **Phase 3: Design Tokens & UI Primitives**: Build accessible `Button`, `Input`, `Card`, `Badge`, and layout wrappers inside `packages/ui`.
- **Phase 4: Core Application Routes & Handlers**: Implement primary authentication, user session handling, and application routes.
- **Phase 5: Quality Assurance & Build Verification**: Run `tsc --noEmit`, ESLint, Prettier, and smoke test suites to ensure zero compilation or runtime errors.
---
## 6. EXECUTION INSTRUCTIONS
1. Review all specifications, architectural guardrails, and stack choices above.
2. Present the full monorepo directory tree structure.
3. Systematically generate the complete, production-ready codebase according to the 5-phase roadmap above — starting with the root workspace setup, followed by the database schema, UI design system package, and full-stack application routes until the repository is fully scaffolded and ready to run.Frequently asked about QuestDB
What is QuestDB built with?
QuestDB is written in Java utilizing off-heap memory and zero-GC principles, paired with native C++ and Rust SIMD kernels for mathematical vectorization.
How does QuestDB achieve zero Garbage Collection pauses in Java?
QuestDB manages all transient data buffers and columnar structures in off-heap memory using Unsafe APIs, keeping the JVM garbage collector completely out of the critical ingestion path.
Can I query QuestDB with standard SQL tools?
Yes. QuestDB implements the PostgreSQL wire protocol on port 8812, allowing connection from psql, DBeaver, Grafana, and standard JDBC/ODBC drivers.
What is the fastest way to insert data into QuestDB?
The InfluxDB Line Protocol (ILP) over TCP on port 9009 provides the highest ingestion throughput, achieving over one million rows per second on standard hardware.
One email a month: new deep dives and stack trends
New source-audited architectures, head-to-head comparisons and the monthly stack report. No spam, unsubscribe anytime.
Similar architectures
- ElectricSQLClassicLocal-First Sync Engine · 6-20 PeopleShares PostgreSQL · Rust
- ClickHouseClassicColumnar Real-Time Analytical Database (OLAP) · 20+ peopleShares C++ · Rust · Docker
- HasuraClassicInstant GraphQL Engine & Unified Data Access · 20+ peopleShares Rust · PostgreSQL · Docker
- SurrealDBClassicMulti-Model Real-Time Database · 20+ PeopleShares Rust · Docker
- DragonflyClassicMulti-Threaded In-Memory Key-Value Store · 20+ PeopleShares C++ · Docker
- MeilisearchClassicSearch Engine · 20+ peopleShares Rust · Docker