5 points | by stayux a day ago ago
2 comments
This looks like an interesting way to scrape a website for AI to use as reference.
Yes, I created this targeting my personal archiving needs, Developers, Design Engineers, ML engineers.
Markdown output has frontmatter for categorising data. HTML for browser consumption is optional.
The goal was to have a simple TUI for quick fetching, but the CLI can be scripted and extendable. All of this with focus on ethical scraping.
Commands are created following the standard from clig.dev.
Thanks for your response.
This looks like an interesting way to scrape a website for AI to use as reference.
Yes, I created this targeting my personal archiving needs, Developers, Design Engineers, ML engineers.
Markdown output has frontmatter for categorising data. HTML for browser consumption is optional.
The goal was to have a simple TUI for quick fetching, but the CLI can be scripted and extendable. All of this with focus on ethical scraping.
Commands are created following the standard from clig.dev.
Thanks for your response.