A Mathematical Framework for Transformer Circuits (2021)

(transformer-circuits.pub)

48 points | by Bluestein 3 hours ago

3 comments

myself248 2 hours ago
Nothing about B-H curves or magnetization current, must be that other kind of transformer, in that other kind of circuit.
amelius 2 hours ago
> A previous project, the Distill Circuits thread, has attempted to reverse engineer vision models, but so far there hasn’t been a comparable project for transformers or language models.
How successful was that?
[-]
- Bluestein 2 hours ago
  The entire thing went down:
  - https://distill.pub/2021/distill-hiatus/
  ... and, here we are.-
oofbey 52 minutes ago
I’ve tried reading this many times. But it’s sooooo long. Is it worth it?