Analytical SQL over Parquet

SQL over Parquet, with the optimizer turned inside out

Point it at a prefix of Parquet files on local disk or in S3-compatible object storage, hand it SQL, and it parses the query, prunes the partitions, columns and row-groups it can prove it doesn't need, and fans the remaining scan across a process pool. There is no load step and no copy of the data.

Athena will tell you it scanned 4.2 MB. It won't tell you which files it skipped or why. This does: every run reports the files it never opened, the row-groups it ruled out on min/max statistics, the columns it left on disk, and the query plan it actually executed.

Open the SQL workspace Engine capabilities

Across the last 200 runs it read 33.3% of the bytes those queries would have needed without pruning — 59.3 MB of compressed data never left the disk.

Queries run 15 15 succeeded, 0 failed

Median latency 30.0ms median of 15 timed runs

Dataset bytes read 33.3% 29.6 MB of 88.9 MB compressed

Ran without error 100.0% across the last 200 runs

Latency

last 200 successful runs

Fastest 8.0 ms · median 30.0 ms · slowest 102.0 ms. Wall-clock, planning included.

Bytes read vs bytes avoided

per run

Each column is the full compressed size of the dataset that run targeted; the blue part is what actually came off disk. Both sides are compressed bytes.

How a query avoids reading data

Details

Prune whole files

Hive-style partition values are parsed out of the directory path, so a predicate on a partition column eliminates entire files before any file is opened. A pruned file costs zero I/O — not a cheap read, no read at all.

Skip row-groups

Every Parquet footer carries min/max statistics per row-group. The engine reads footers only, tests the predicate against those ranges, and skips any row-group it can prove holds no match before decompressing a byte.

Read only the columns asked for

Parquet is columnar, so unreferenced columns are never touched. A COUNT(*) resolves to the narrowest column on disk rather than the first declared one — a 19x difference on the sample dataset.

Recent runs

Full history

Run	SQL	Status	Duration	Read
#15	SELECT region, COUNT(*) AS n, SUM(distance) AS tota…	success	17.76 ms	16.5%
#14	SELECT region, COUNT(*) AS n, AVG(distance) AS avg_…	success	30.01 ms	23.2%
#13	SELECT COUNT(*) AS total FROM trips WHERE trip_id >…	success	34.74 ms	26.9%
#12	SELECT COUNT(*) AS total FROM trips WHERE region = …	success	7.99 ms	0.5%
#11	SELECT * FROM trips LIMIT 20	success	89.94 ms	100.0%
#10	SELECT region, COUNT(*) AS n, SUM(distance) AS tota…	success	13.7 ms	16.5%
#9	SELECT region, COUNT(*) AS n, AVG(distance) AS avg_…	success	28.05 ms	23.2%
#8	SELECT COUNT(*) AS total FROM trips WHERE trip_id >…	success	31.37 ms	26.9%
#7	SELECT COUNT(*) AS total FROM trips WHERE region = …	success	8.12 ms	0.5%
#6	SELECT * FROM trips LIMIT 20	success	101.34 ms	100.0%
#5	SELECT region, COUNT(*) AS n, SUM(distance) AS tota…	success	17.57 ms	16.5%
#4	SELECT region, COUNT(*) AS n, AVG(distance) AS avg_…	success	30.89 ms	23.2%

Datasets

All

trips 15 files · 6.0 MB compressed regiondate

Registering a dataset reads one footer to infer the schema and parses partition keys from the paths.

What it runs

Single-table SELECT with WHERE, GROUP BY, ORDER BY and LIMIT, over Parquet on local disk or in S3-compatible object storage. The depth is in the scan path, not the SQL surface.

Every result is differentially tested against DuckDB reading the same files, so correctness is verified rather than asserted — and the Performance page breaks down the I/O each query actually paid for.