Field Notes
Things I haven't figured out yet.
Ideas, experiments, questions and rabbit holes.
02 NOTES · 02 RESEARCH · 00 PERSONAL
QUESTIONS I'M CARRYING
01
COMPUTATION
Can transformer computation be described through stable geometric structures?
02
INTERPRETABILITY
What does a model actually need to represent before it can reason?
03
EFFICIENCY
Can interpretability become an inference mechanism?
04
INTELLIGENCE
Where does artificial intelligence stop resembling biological intelligence?
NOTES / 2025–2026
SEARCH ARCHIVE
FRAGMENTS
03:17 AM
I keep wondering whether attention heads are really "specialized", or whether specialization is just a convenient description of geometry.
Observation
Removing tokens seems much easier than teaching the model to use fewer tokens.
Question
If we can identify the computation, can we intervene in it?