Speculative pre-positioning: off-path decode for stateful inference sessions

1 comments

• logotype 6 hours ago

  With native support for sessions in an inference engine we can make use of idle GPUs... doing work!