How does it handle large outputs? If I pipe a big config file through, does the full content travel back to local for LLM processing, or is there any streaming/chunking?
It does travel back to local entirely. It's streamed, although buffered at local before making the completion. I also imagine on the fly compression at remote and decompression at local should be possible
How does it handle large outputs? If I pipe a big config file through, does the full content travel back to local for LLM processing, or is there any streaming/chunking?
It does travel back to local entirely. It's streamed, although buffered at local before making the completion. I also imagine on the fly compression at remote and decompression at local should be possible