Stop overpaying for AI responses! Route queries to the right model & cut costs.

By analyzing prompt complexity, an LLM router/gateway smartly chooses the most efficient model for each query. Essentially only serving more capable/expensive models when needed.

Here are a few open-source tools to get started!

LiteLLM Proxy Server (LLM Gateway)

OpenAI Proxy Server (LLM Gateway) to call 100+ LLMs in a unified interface & track spend, set budgets per virtual key/user.

Machine Learning Engineer. 

Learning about Agentic AI and Web3. 

Interested in d/acc - Climate Tech

LiteLLM Proxy Server (LLM Gateway)

OpenAI Proxy Server (LLM Gateway) to call 100+ LLMs in a unified interface & track spend, set budgets per virtual key/user.

https://github.com/BerriAI/litellm

Portkey Gateway

The AI Gateway is designed for fast, reliable & secure routing to 1600+ language, vision, audio, and image models. It is a lightweight, open-source, and enterprise-ready solution that allows you to integrate with any language model in under 2 minutes.

Bonus: RouteLLM

A Python framework for serving and evaluating LLM routers - save LLM costs without compromising quality.

For those wanting to delve deeper and learn more about LLM routing.

Bonus: RouteLLM

A Python framework for serving and evaluating LLM routers - save LLM costs without compromising quality.

For those wanting to delve deeper and learn more about LLM routing.

https://github.com/lm-sys/RouteLLM

Portkey Gateway

The AI Gateway is designed for fast, reliable & secure routing to 1600+ language, vision, audio, and image models. It is a lightweight, open-source, and enterprise-ready solution that allows you to integrate with any language model in under 2 minutes.

https://github.com/portkey-ai/gateway