Blog
In total, I've written 13 articles on this site. Use the search by title below, or pick a category.
Subscribe to Blog
No spam ever, only raw unfilered throughts and new blog post alerts.
AI Architecture
Explore Xiaomi's MiMo-V2.6-Pro in an interactive 3D animation: how tokens, attention, a mixture of experts and speculative decoding produce an answer.
Take a night drive through Kimi K3 as an interactive 3D city: follow one token through Kimi Delta Attention, MLA, attention residuals and 896 experts.
Take the red pill and follow one token through DeepSeek V4-Pro in interactive 3D: compressed attention, hyper-connections, 384 experts and 61 layers.
Press start and watch a large language model finish a sentence in an interactive visualization: tokens, attention, the KV cache and mixture of experts.
How AI models think
––– views
How a machine that only predicts the next word ends up building world models, planning ahead, and doing things that look like thinking.
Exploring GPT-4: Part 1
––– views
Explain how you understand language and what are your core competencies as an AI model.