-
Breaking the O(N²) ceiling
A Deep Dive into Flash Attention with Triton
-
Probability and Information Theory
A chill, easy read on how probability and information theory help us deal with an uncertain world.
-
The Math behind Rotating Attention
A deep dive into how RoPE works in transformers