{"id":50249636,"title":"Show HN: Flowfile – Open-source visual ETL that round-trips to Python (Polars)","points":1,"user":"edwardeechoud","time":1793043081,"time_ago":"7 hours ago","type":"link","content":"

Hi HN, I'm Edward, the author of Flowfile.

Flowfile is a visual ETL tool built on Polars. You build pipelines on a canvas, and every flow is also Python code. You can export it or write code with the FlowFrame API (Polars-like) and it shows up as a visual graph (or vice versa - keep reading).

Flowfile is what it is today because I believe we are spending too much time translating non-technical and technical implementations of data problems. Sometimes visual works better, and sometimes code. You should be able to choose whichever is easiest, without fully committing to one way of working. Flowfile solves this by standardizing the backend component without favoring either visual or code.

Which brings me to Flowfile's newest feature: notebook mode. The flow opens as Python cells that stay in sync with the canvas. If you add a filter (df.filter(...)) in a cell, it appears on the canvas. And it also works the other way around: if you add a sort on the canvas, it appears as a .sort(...) in the notebook. So now you can switch between visual and code without making a total mess of it under the hood.

It also has a data catalog, a scheduler, sandboxed Python execution, etc.. It's MIT licensed, and `pip install flowfile` installs all dependencies.

Github: https://github.com/edwardvaneechoud/Flowfile

I'd love feedback, especially on the notebook mode.","url":"https://flowfile.io/","domain":"flowfile.io","comments":[{"id":50249831,"level":0,"user":"brabican","time":1793043830,"time_ago":"7 hours ago","content":"

Definitely going to look into this, looks really promising. Interested in how it stacks up to KNIME overall. Thanks for sharing!","comments":[{"id":50249904,"level":1,"user":"edwardeechoud","time":1793044153,"time_ago":"7 hours ago","content":"

[dead]","dead":true,"comments":[]}]}],"comments_count":2}