I haven't used these extensions, but I like the direction of being able to extend Postgres by tailing its own internals (pg_squeeze approach) than triggers (pg_repack).
We (the SpiceDB developers) took a similar approach to solve a different problem a few years ago: implementing strict serializability in Postgres[0]. By hooking into the transactions layer themselves (pg_xact), we could layer on our own MVCC reusing the real Postgres transactions to compute visibility and total ordering. When you can just tail the Postgres internals, you don't even need to implement a Postgres extension, which can keep it portable across locked-down, managed Postgres offerings.
Do folks know of other clever implementations of behavior by "spying" on the Postgres internals?
They added plan hinting, but sadly not via comments. To use hinting an app needs to issue a `SET` command, which adds round trip latency for each query. Need to use quite unergonomic query batching/pipelining to negate that latency.
Isn't losing MVCC consistency a pretty big drawback? Am I misinterpreting or does that mean occasionally a query might just return nothing for possibly difficult to diagnose reasons?
Trying to recoup space at a slow period, I remember using Repack on New Years Eve on my local computer for a multi TB production DB and waiting hours and hours hoping my connection doesn’t drop. When it dropped, I had to restart all over again.
I thought it would be a simple process but it ended up taking many hours and a few retries. Holy hell it was a miserable experience and it ruined my New Years Eve.
I feel like that whole experience deserves a blog post.
pg_repack, pg_squeeze, pg_reorg, all felt like it was "use as directed" and if you misuse them we're sorry if we corrupt data - that's a little extreme but they were still very advanced tools.
Having this in core and learning from all the previous Postgres paths is the way of Postgres. I'm with you couldn't be more excited to have this in core, to me it was always the flagship feature and I'm just glad it stayed in 19 vs. a revert compared to so many other flagship features.
Before long-running commands, open a screen [1] session. That way, if you get disconnected, the job keeps running, and you can reconnect to the screen session afterwards.
The man page is quite long, use "screen -R <appropriate name" to create the session or reconnect to it, and (IIRC) CTRL-A D to exit screen without closing the session.
I would have loved REPACK CONCURRENTLY when we were generating millions of dead tuples an hour on our trading database …
Sadly it doesn’t support partitions yet, which is typical when reaching sufficient scale that bloat management really matters. People often mention disk space when talking about dead tuples, but the real killer is what’s hidden under - bloat in the ctid & btree leaf linked lists make every lookup feel like table scans…!
Thankfully HOT (Heap-only Tuples) has saved us since. Virtually no dead tulles generated, and they get otherwise collected during regular ops like SELECT, amortizing cost over time, and preventing dangerous build up.
I haven't used these extensions, but I like the direction of being able to extend Postgres by tailing its own internals (pg_squeeze approach) than triggers (pg_repack).
We (the SpiceDB developers) took a similar approach to solve a different problem a few years ago: implementing strict serializability in Postgres[0]. By hooking into the transactions layer themselves (pg_xact), we could layer on our own MVCC reusing the real Postgres transactions to compute visibility and total ordering. When you can just tail the Postgres internals, you don't even need to implement a Postgres extension, which can keep it portable across locked-down, managed Postgres offerings.
Do folks know of other clever implementations of behavior by "spying" on the Postgres internals?
[0]: https://m.youtube.com/watch?v=_f6fHy4DWWM
They added plan hinting, but sadly not via comments. To use hinting an app needs to issue a `SET` command, which adds round trip latency for each query. Need to use quite unergonomic query batching/pipelining to negate that latency.
Isn't losing MVCC consistency a pretty big drawback? Am I misinterpreting or does that mean occasionally a query might just return nothing for possibly difficult to diagnose reasons?
Trying to recoup space at a slow period, I remember using Repack on New Years Eve on my local computer for a multi TB production DB and waiting hours and hours hoping my connection doesn’t drop. When it dropped, I had to restart all over again.
I thought it would be a simple process but it ended up taking many hours and a few retries. Holy hell it was a miserable experience and it ruined my New Years Eve.
I’m so happy this is now built in.
I feel like that whole experience deserves a blog post.
pg_repack, pg_squeeze, pg_reorg, all felt like it was "use as directed" and if you misuse them we're sorry if we corrupt data - that's a little extreme but they were still very advanced tools.
Having this in core and learning from all the previous Postgres paths is the way of Postgres. I'm with you couldn't be more excited to have this in core, to me it was always the flagship feature and I'm just glad it stayed in 19 vs. a revert compared to so many other flagship features.
Before long-running commands, open a screen [1] session. That way, if you get disconnected, the job keeps running, and you can reconnect to the screen session afterwards.
The man page is quite long, use "screen -R <appropriate name" to create the session or reconnect to it, and (IIRC) CTRL-A D to exit screen without closing the session.
[1] https://man7.org/linux/man-pages/man1/screen.1.html
Nowadays screen is considered legacy and it is recommended to use tmux, but the idea is the same.
It was my first time doing it. I didn’t know it’d be such a frustrating experience.
Having worked on clusters where bloat was an ongoing serious problem, this looks really useful! Thanks for doing this work.
I would have loved REPACK CONCURRENTLY when we were generating millions of dead tuples an hour on our trading database …
Sadly it doesn’t support partitions yet, which is typical when reaching sufficient scale that bloat management really matters. People often mention disk space when talking about dead tuples, but the real killer is what’s hidden under - bloat in the ctid & btree leaf linked lists make every lookup feel like table scans…!
Thankfully HOT (Heap-only Tuples) has saved us since. Virtually no dead tulles generated, and they get otherwise collected during regular ops like SELECT, amortizing cost over time, and preventing dangerous build up.