๐ง WAL Cake: Ordered ACKs from Parallel S3 Uploads
How WAL Cake reads the Postgres WAL in order, uploads Parquet files to S3 in parallel, and uses one ring buffer to keep the replication slot ACK in order.
How WAL Cake reads the Postgres WAL in order, uploads Parquet files to S3 in parallel, and uses one ring buffer to keep the replication slot ACK in order.
The outbox pattern needs durable, ordered, resumable delivery, and database replication already provides it. Postgres' pg_logical_emit_message writes events into the replication stream. A line-by-line explanation of factlib and OwlPost.
Every SQL statement a Temporal workflow runs against Postgres, and why each one exists. Then Absurd, a 5-table durable-execution engine in one SQL file, does the same job with about a fifth of the statements and 20ร the throughput.
Postgres reads data in 8 KB pages. This post explains why one lookup reads 33,333 pages without an index and 4 with one, when the planner skips an index, how to read EXPLAIN, and what SELECT * and LIMIT save.