Intuitions for Tranformer Circuits

(connorjdavis.com)

58 points | by cjamsonhn 9 hours ago ago

4 comments

  • skyberrys 7 hours ago

    I was glad to discover this is an analogy to electrical circuits further down the page. It's a good topic to understand, the write-up is comprehensive.

  • jopolous 7 hours ago

    > They are the only man-made technology in history that we don’t fully understand from first principles

    …what? What about bicycles? Ice skates? General anesthetics? I feel like there are a ton of commonly-used man-made items we don’t fully understand from a first-principles perspective.

    Am I missing something here?

    • doctoboggan 5 hours ago

      No, I don't think you are missing anything. Only recently have engineers been inventing things from "first principles". I think for the majority of human civilization we've mostly invented and improved through trial and error.

      • willis936 28 minutes ago

        How many computing or mathematical constructs fall into that category? You don't accidentally land on a new algorithm typically.

        The author shouldn't have discussed inventions when the nugget is about "original thought".