๐ง WAL Cake: Ordered ACKs from Parallel S3 Uploads
How WAL Cake reads the Postgres WAL in order, uploads Parquet files to S3 in parallel, and uses one ring buffer to keep the replication slot ACK in order.
Hi There ๐
I’m Pratik. As a software engineer, I enjoy tackling both technical challenges and everyday problems. My passion lies in finding solutions and making improvements where possible, whether that means optimizing AWS costs, enhancing server response times, or streamlining processes.
Want to talk ?
pg [@at@] backend.how (if you use GPG, here’s my signature)Interested in updates ?
Found a mistake ?
More at about me!
Checkout posts for all ๐
How WAL Cake reads the Postgres WAL in order, uploads Parquet files to S3 in parallel, and uses one ring buffer to keep the replication slot ACK in order.
The outbox pattern needs durable, ordered, resumable delivery, and database replication already provides it. Postgres' pg_logical_emit_message writes events into the replication stream. A line-by-line explanation of factlib and OwlPost.
Every SQL statement a Temporal workflow runs against Postgres, and why each one exists. Then Absurd, a 5-table durable-execution engine in one SQL file, does the same job with about a fifth of the statements and 20ร the throughput.
Napkin math and reproducible benchmarks for a ledger at a billion payments a day: TigerBeetle vs PostgreSQL, traced with eBPF down to the io_uring and fsync calls.
How I went from a 38-minute 4K to a 24:45 5K with one goal at a time and one running metric at a time, measured with a Garmin watch. Compare your cadence, ground contact time, and vertical oscillation with Garmin's percentile bands, and get your heart-rate zones in bpm.
A repost of TigerBeetle's coding guide. It ranks safety, then performance, then developer experience, and gives rules for assertions, limits, static memory, batching, and naming.