Rebuilding my local voice assistant pipeline

(blog.platypush.tech)

29 points | by todsacerdoti 2 hours ago ago

2 comments

  • fragilerock 35 minutes ago

    Interesting read! I have my own local voice assistant I implemented exactly the same question mark trick just the other day.

    The overall pipeline is quite similar to my previous setup, but I found the STT -> LLM -> TTS pipeline just infuriatingly slow. TTS and STT with whisper/piper worked fine and reasonably fast for me locally, but the real bottleneck was always the fact that they're not running concurrently with the LLM call. I ended up forking an open source openai realtime API ESP32 plugin and patching it to my liking. It's a bit more expensive in terms of API cost but I only use it sparingly and it's so much nicer to use now.

  • stavros an hour ago

    This is really useful, thanks! The other day I bought a small screen and didn't know what to do with it so I made an assistant that talks to my bot:

    https://m.youtube.com/shorts/3FseVyAG5uM

    I used microwakeword too, and training my own wakeword was pretty simple (basically just recording a few tens of samples). Highly recommended.

    I also wanted to be able to issue some voice commands for lights and other things in my home with low latency, so I added a short-circuit plugin that runs right after recognition. If the recognized command matches one of the regexes, it's immediately executed, which has a latency of under a second. Otherwise, it gets sent to my assistant, which usually will tweak the regex (or add a new one) to improve the whole system as it fails.