This is very good. Super snappy and gives me what I'm looking for. Great write-up too.
How much more would it cost to use a multilingual tokeniser? I realise this is somewhat pointless for an English blog, but services like Uber Eats have something like that going on (server-side of course, and they may just be translating queries).
It’s gets much larger (there’s the potion-multilingual-128M model, distilled from bge-m3 which has like 100 languages), so it gets to ~120m range for the lookup table (4x smaller if you’d do int8 quantization, so you could get to to 30-35mb).
I think it's good etiquette to ask the user if they want to download the model or any assets larger than the average page size, in case they have limited bandwidth.
This is very good. Super snappy and gives me what I'm looking for. Great write-up too.
How much more would it cost to use a multilingual tokeniser? I realise this is somewhat pointless for an English blog, but services like Uber Eats have something like that going on (server-side of course, and they may just be translating queries).
It’s gets much larger (there’s the potion-multilingual-128M model, distilled from bge-m3 which has like 100 languages), so it gets to ~120m range for the lookup table (4x smaller if you’d do int8 quantization, so you could get to to 30-35mb).
This is a very exciting field to make stuff in! I also experimented with a completely client side wallpaper search xD,a while back.
In case some one is interested here is it : https://web-inky-ten-60.vercel.app/
I think it's good etiquette to ask the user if they want to download the model or any assets larger than the average page size, in case they have limited bandwidth.
Hi ! This was made mostly for me to use. I didnt think much about the UX back then. Will keep it in mind. Thanks for the feedback.
Every browser should have an embedding API to avoid having to worry about the plumbing and data transfer.
Chrome ships with one (https://news.ycombinator.com/item?id=43973502) but AFAICT it’s only used for history, there’s no api for it. There’s also an included LLM? https://www.thatprivacyguy.com/blog/chrome-silent-nano-insta...