LLM gateway vs LLM router

"LLM gateway" and "LLM router" are often used interchangeably, but they describe different jobs. Understanding the distinction helps you evaluate tools accurately.

What an LLM gateway does

A gatewayis the single entry point between your app and many providers. It normalizes their APIs (usually into an OpenAI-compatible shape), centralizes auth and billing, and gives you one place for logging and controls. The gateway's job is unification.

What an LLM router does

A router is the decision engine: for each request, it chooses whichmodel and provider to use, based on cost, quality, latency, and policy. The router's job is optimization.

Why you want both

A gateway without routing just proxies to whatever you hardcode. A router without a gateway has nothing consistent to dispatch through. Together, one endpoint reaches every model and sends each request to the best one — with failover and cost controls on top.