{"id":49128567,"title":"Show HN: Penca – branchable, versioned OLTP+OLAP on one open copy of your data","points":12,"user":"jnbh","time":1785532265,"time_ago":"12 hours ago","type":"link","content":"

Hi HN!

This is an early proof of concept of a branchable, versioned OLTP + OLAP database that runs on a single, open copy of your data in object storage. If you are familiar with Databricks' LTAP (https://www.databricks.com/company/newsroom/press-releases/d...) announcement from June, you can think of this as aspiring to be a fully open source, Apache 2.0 LTAP alternative with additional data versioning/auditability guarantees that enable audit, as_of queries, and (eventually) revert straight out of the box.

How it works:

1. Writes land in vanilla postgres which functions as an ephemeral hot tier

2. A background pass flushes committed rows out to columnar files in an object storage cold tier

3. A DataFusion based query engine merges results across the two tiers (with some fancy caching and indexing)

For a deep dive on the architecture/intended design/vision see: https://penca.io/blog/the-penca-architecture.html.

This project is very, very early on. It is mostly a PoC right now and there are many bugs and shortcomings with many major items still on the roadmap. I hope to get some real performance numbers soon.

A few features I would like to add in no particular order:

- Iceberg export

- Branching (from any branch, not just main)

- Improved SQL support (e.g. multi-statement execution)

- Full-text search and vector indexes

- Configurable isolation levels (currently last write wins)

- A pgwire frontend

Any and all feedback is appreciated. I'd also like help! If any of this appeals to you and the project doesn't seem like the craziest idea of all time, shoot me an email at nico@penca.io.

Disclaimer: before someone comments this, yes, the code is heavily AI generated. I have tried to leave as much of the dev tooling in the repo as possible to demonstrate what the end-to-end development workflow looks like. Also, the majority of the commit history is missing as I had to rename/migrate + scrub the repo prior to open sourcing.","url":"https://github.com/penca-io/penca","domain":"github.com","comments":[],"comments_count":0}