{"id":47381248,"title":"Show HN: ngrep – grep plus word embeddings (Rust)","points":3,"user":"xnan","time":1773522901,"time_ago":"2 days ago","type":"link","content":"

I got curious about a simple question: regular expressions are purely syntactic, but what happens if you add just a little bit of semantics?

To answer, I ended up building ngrep: a grep-like tool that extends regular expressions with a new operator ~(token) that matches a word by meaning using word2vec-style embeddings (FastText, GloVe, Wikipedia2Vec).

A simple demo: "~(big)+ \\b~(animal;0.35)+\\b" over Moby-Dick can find many ways used to refer to a large animal, surfacing "great whale", "enormous creature", "huge elephant" and so on. Pipe it through sort | uniq -c and the winner is, unsurprisingly, "great whale" :)

Built in Rust on top of the awesome fancy-regex, and ~() composes with all standard operators (negative lookahead, quantifiers, etc.). Currently a PoC with many missing optimizations (e.g: no caching, no compilation to standard regex, etc.), obviously without the guarantees of plain regex and subject to the limits of w2v-style embeddings...but thought it was worth sharing!","url":"https://github.com/0xNaN/ngrep","domain":"github.com","comments":[{"id":47381422,"level":0,"user":"jruohonen","time":1773523880,"time_ago":"2 days ago","content":"

Just a remark about the name; there is already a well-known ngrep.

https://linux.die.net/man/8/ngrep.","comments":[{"id":47381535,"level":1,"user":"xnan","time":1773524523,"time_ago":"2 days ago","content":"

Oops, didn't know, thanks! I just added a section on how to compile and try it without a system installation to avoid any name clash :)","comments":[]}]},{"id":47381309,"level":0,"user":"shablulman","time":1773523291,"time_ago":"2 days ago","content":"

[dead]","dead":true,"comments":[]},{"id":47393693,"level":0,"user":"spranab","time":1773621298,"time_ago":"12 hours ago","content":"

[dead]","dead":true,"comments":[]}],"comments_count":4}