Been tinkering on my personal site and wanted to add some AI features without being the one paying for tokens on every visit. So I went looking at how close browser-side inference actually is to practical. Closer than I expected.
The post is less a tutorial and more me walking through what it felt like. Kicking tires on random models, getting one to actually run, then doing the small unglamorous work to turn "demo" into something I'd put in front of a reader.
Been tinkering on my personal site and wanted to add some AI features without being the one paying for tokens on every visit. So I went looking at how close browser-side inference actually is to practical. Closer than I expected.
The post is less a tutorial and more me walking through what it felt like. Kicking tires on random models, getting one to actually run, then doing the small unglamorous work to turn "demo" into something I'd put in front of a reader.