Keeping Up with the Models: Online Deployment and Routing of LLMs at Scale.

Sun, 01 Jun 2025 01:00:00 +0200

Shaoang Li, Jian Li:
Keeping Up with the Models: Online Deployment and Routing of LLMs at Scale. CoRR abs/2506.17254 (2025)