Technical Blog

Writing FlashAttention in Triton (Part 2): From the Algorithm to a Real Kernel, and Fusing RoPE

Writing FlashAttention in Triton (Part 1): The Memory Wall and the Online Softmax Trick

Transformers (Decoder-Only) (Part 2)

Algorithms (Deep Learning Ops)

Generative Adversarial Networks (GANs)

Transformers (Decoder-Only) (Part 1)

GPU Programming

Understanding Metal and MSL

Deep Learning