14 points | by MayCXC 3 hours ago ago
6 comments
Paper written by AI
Is this just a theory about an architecture or are there actually some benchmarks/results to substantiate it?
appears it's just a theory, sort of surprised this lifted up so far on hn
not sure w/o context why this is important -- no results/implementation & i believe there are prior combinations of transformers/RNNs. but perhaps I'm missing the relevance/insight
I thought it's about https://en.wikipedia.org/wiki/Autotransformer
I can't believe that actually works? This is far more interesting than the half baked ML idea.
Edit: Thanks to your link I've now learned about the zigzag transformer which is yet another design that somehow feels like cheating reality. https://en.wikipedia.org/wiki/Zigzag_transformer
Paper written by AI
Is this just a theory about an architecture or are there actually some benchmarks/results to substantiate it?
appears it's just a theory, sort of surprised this lifted up so far on hn
not sure w/o context why this is important -- no results/implementation & i believe there are prior combinations of transformers/RNNs. but perhaps I'm missing the relevance/insight
I thought it's about https://en.wikipedia.org/wiki/Autotransformer
I can't believe that actually works? This is far more interesting than the half baked ML idea.
Edit: Thanks to your link I've now learned about the zigzag transformer which is yet another design that somehow feels like cheating reality. https://en.wikipedia.org/wiki/Zigzag_transformer