The Htop for LLM Inference

(github.com)

4 points | by helasaoudi 12 hours ago ago

2 comments

  • rbanffy 11 hours ago

    This is super cool. Would be nice if it could be adapted to other architectures besides Nvidia. How is it getting its data?

    BTW, exporting the data to Prometheus would be a killer feature.

  • helasaoudi 12 hours ago

    [flagged]