# AI gateway vs MCP gateway: which one do you need?

> AI gateway vs MCP gateway: one governs your apps' calls to models, the other which tools each person's AI client may call, and on whose account.

**TL;DR** An AI gateway sits between your applications and model providers: it routes prompts, caches, rate-limits, tracks token spend and logs prompts. An MCP gateway sits between AI clients and the tools they call, and decides which apps each person's agent may reach, on whose account, with a record of each call that runs. Elaichi, a governed MCP control plane and hosted MCP gateway, is not a model router, so it runs beside an AI gateway rather than replacing one.

Your platform team put an AI gateway in front of OpenAI and Anthropic last quarter. Every model call from the company's own apps now passes through it, with a budget per team and a log of each prompt. Then support asks for Claude with access to Zendesk, and finance asks for ChatGPT with access to Xero. The first thought is that the gateway already handles AI traffic. MCP (Model Context Protocol) is the standard way an AI assistant calls tools in other apps.

The AI gateway vs MCP gateway question comes down to which traffic each one sees. An AI gateway sees prompts on their way to a model. An MCP gateway sees tool calls on their way to an app. A company that builds on models and also rolls out AI clients to staff has both kinds of traffic, and each gateway answers a different question about it.

## AI gateway vs MCP gateway: what is the difference?

An AI gateway sits between your applications and the model providers they call. An MCP gateway sits between AI clients, such as Claude, ChatGPT and Cursor, and the tools those clients call in other apps. One governs the request for an answer. The other governs the action taken after it.

An AI gateway, also called an LLM gateway (LLM is short for large language model), answers questions about models. Which model got this prompt, what did it cost, and is the team over budget? An MCP gateway answers questions about people and tools. Which apps may this person's agent reach, whose account does a call run on, and what did each call touch? An agent here is an AI client acting for a person.

## What does an AI gateway control?

An AI gateway controls traffic from your code to model providers: routing, caching, rate limits, spend and logs. Its user is usually a developer whose application sends prompts.

Cloudflare's [AI Gateway overview](https://developers.cloudflare.com/ai-gateway/) (checked October 2026) describes visibility and control over AI apps. It lists caching, rate limiting, request retries and model fallback, and its analytics count requests, tokens and cost. Cloudflare's [logging page](https://developers.cloudflare.com/ai-gateway/observability/logging/) says each log can include the prompt, the response, token usage and cost.

Other products share the shape. The [LiteLLM AI Gateway](https://docs.litellm.ai/docs/simple_proxy) is a self-hosted server that gives applications one OpenAI-compatible endpoint across model providers. Each user, team or project gets a virtual key with its own models, budgets and rate limits, and the gateway records the cost of each request. Portkey's [AI Gateway docs](https://portkey.ai/docs/product/ai-gateway), which carry the name Prisma AIRS AI Gateway as of October 2026, list budget limits on cost or tokens, rate limits on requests or tokens, caching, and fallbacks between providers. Both pages were checked October 2026.

Look at what all of them key on: an application, a key, a model and a token count. The caller is code you wrote, holding a key you issued.

## What does an MCP gateway control?

An MCP gateway controls what an AI client may do in other apps once the model decides to act. It signs each person in, decides which tools that person may reach, attaches the right account's credential, and records the call.

The traffic is different in kind. Under the [MCP specification](https://modelcontextprotocol.io/specification/2026-07-28/server/tools), a client runs a tool with a `tools/call` request, the MCP message that runs one tool, which carries the tool's name and its arguments. There is no model choice to route, no prompt to cache and no token bill to count. What there is to govern is a person, a tool and an account.

That gives an MCP gateway three questions to answer on every call:

1. Which apps and tools may this person's agent reach?
2. Whose credentials does the call run on: the person's own account, or one shared with them?
3. What is the record afterwards: who called which tool, on which account, and did it work?

Products answer those three in different ways, and [the four shapes of MCP gateway](/blog/what-is-an-mcp-gateway/) sorts them by who runs the servers and who writes the connectors.

## What does each gateway see, control and record?

Each gateway sees one kind of request, and its controls and records follow from that. The AI gateway column follows [Cloudflare's AI Gateway docs](https://developers.cloudflare.com/ai-gateway/) and [LiteLLM's docs](https://docs.litellm.ai/docs/simple_proxy), checked October 2026.

| Question | AI gateway (LLM gateway) | MCP gateway |
|---|---|---|
| Sits between | Your applications and model providers | AI clients and the apps they act in |
| What passes through | Prompts and model responses | Tool calls: a tool name and its arguments |
| Who the caller is | Your code, holding a key you issued | A named person, through their AI client |
| Main controls | Routing, fallback, caching, rate limits, budgets | Which apps and tools each person may reach, and on which account |
| What it records | Prompt, response, tokens, cost | Person, tool, account reached, outcome |
| The question it answers | Which model ran this, and what did it cost? | Who did what, in which app, on whose account? |

Read the last row as the test. If the question after an incident is about a model bill, it is an AI gateway question. If it is about a changed record in Salesforce, it is an MCP gateway question.

## Why can an AI gateway not see an employee's tool calls?

Because the tool call does not pass through it. An AI gateway sees the model requests pointed at it. A tool call is a separate request, and it goes to an MCP server, not to a model provider.

Claude is a clear case. Anthropic's [custom connector guide](https://support.claude.com/en/articles/11175166-get-started-with-custom-connectors-using-remote-mcp) (checked October 2026) says Claude connects to a remote MCP server from Anthropic's cloud, on every Claude client, the desktop and mobile apps included. That request leaves Anthropic and lands at the MCP server. A model gateway in front of your own apps is not on that path.

The blind spot runs the other way too. An MCP gateway never sees the person's prompt, because a `tools/call` request does not carry it. Elaichi's MCP endpoint, for example, reads only a tool name and arguments from each call. Prompt logging and spend tracking belong on the model side. Each gateway is blind to the other's traffic, so one does not replace the other.

## Can one product be both an AI gateway and an MCP gateway?

Yes. Several products cover both kinds of traffic, but they still answer two separate sets of questions. Buying one product does not merge the jobs.

Kong's [AI Gateway documentation](https://developer.konghq.com/ai-gateway/) (checked October 2026) describes one control plane for LLM, MCP and agent-to-agent traffic, with a single endpoint you can define for any of the three. The [LiteLLM AI Gateway](https://docs.litellm.ai/docs/simple_proxy) also gives access to MCP tools, with the same keys and spend records as model calls. Portkey documents a separate [MCP Gateway](https://portkey.ai/docs/product/mcp-gateway) that proxies between MCP clients and MCP servers, and handles authentication, access control and logging. Both of those pages were checked October 2026.

A combined product suits a platform team that already runs it for model traffic and wants one system to operate. Before you count it as your MCP gateway, put the three tool questions to it. Who writes and maintains the tools for each SaaS app? Does a call carry a named person, or a key? Which account at the app did the call actually reach? [The Kong and Cloudflare comparison](/blog/kong-cloudflare-mcp-gateway-vs-managed-connectors/) works through those answers for two vendors. For gateways that put MCP on top of existing APIs, read [API gateways with MCP against MCP-native servers](/blog/api-gateway-with-mcp-vs-mcp-native/).

## Where does Elaichi fit: AI gateway or MCP gateway?

Elaichi is an MCP gateway, not an AI gateway. It is a governed MCP control plane and hosted MCP gateway. Every call from an AI client passes through it and is checked against the caller's access, and every call that reaches execution is written to the audit log. It does not sit between your code and a model provider, and its endpoint receives tool calls, never the person's prompt.

Claude, ChatGPT, Cursor or any MCP client signs in to one endpoint, `https://api.elaichi.ai/mcp`. An endpoint is the one address every client points at, and Elaichi's is the same for every organization. Each person approves their own OAuth grant, the sign-in record that lets a client act as one named person. Every call then acts as that person. Behind the address sit 600+ connectors. Elaichi authors and runs most of them, governs vendors' own MCP servers for the rest, and lets a team bring its own remote MCP server under the same rules.

Elaichi's answers to the three tool questions are specific. Restrictions, rules set on a role or on one member, decide which connectors and which individual tools a target may reach. A call runs on one connection: the person's own account, or a shared one, which runs on its owner's account. Credentials sit in a separate credential service, and the AI client holds only an Elaichi token. Each call that reaches execution is written to the [audit log](/blog/what-an-ai-audit-log-must-capture/), the append-only record of calls. The entry names the person, the tool, the connector, the account the call actually reached, the MCP client and the outcome.

Changes in Elaichi land on a known schedule. A role or restriction change takes about two minutes to apply. Removal is faster. In Elaichi, removing or suspending a member revokes every live grant in the same transaction as the membership change, so the person's AI clients fail on the next call.

## When does a company only need an AI gateway?

When AI at the company means your own software calling models, and no one's AI client acts in company apps. In that case an MCP gateway has no traffic to govern.

Picture a product team that ships a summary feature and a support chatbot inside its own app. The code calls OpenAI and Anthropic. The tools the model uses are functions in the same codebase, called by the team's own code. Staff do not connect Claude or ChatGPT to Salesforce or Zendesk. That team needs routing, fallback, budgets and prompt logs. An AI gateway covers all of it, and Elaichi would add nothing.

The answer changes when staff start connecting AI clients to company apps. A second client arrives, two people share one app account, or someone leaves while their AI client still holds access. [When an MCP gateway is premature](/blog/when-you-dont-need-an-mcp-gateway/) lists those signals in full.

## When do you need both, and how do you split them?

You need both when you build on models and also give staff AI clients that act in company apps. Split by traffic, not by team: model calls go through the AI gateway, and tool calls go through the MCP gateway.

Here is a common split. Jake Morgan's platform team keeps an AI gateway in front of the models its product calls, with a budget per team. Emily Carter in IT rolls out Claude and ChatGPT to support and finance through an MCP gateway. Each person signs in with their own grant, and each app call is recorded. Neither gateway needs to know about the other.

Keep each control in one place. Spend caps live on the model side. Which apps a person may reach lives on the tool side. A rule set in both places drifts, and the copy that gets updated is the one somebody remembers. To see which apps the tool side already covers, browse the [connector catalog](/connectors/) or the [team rollouts](/use-cases/).

## FAQ

### What is the difference between an AI gateway and an MCP gateway?

An AI gateway, also called an LLM gateway, sits between applications and model providers. It routes prompts to models, caches responses, rate-limits requests, tracks token spend and logs prompts. An MCP gateway sits between AI clients such as Claude, ChatGPT and Cursor and the apps they act in. It decides which tools each person may reach and which account a call runs on, and it records each tool call. A company that builds on models and also gives staff AI clients needs both.

### Can an AI gateway control which tools an AI agent can use?

Only for traffic that passes through it. An AI gateway sees the model requests pointed at it. When an employee's Claude or ChatGPT calls a tool in Salesforce or Zendesk, that call goes to an MCP server, not to a model provider. Some products, such as Kong AI Gateway (https://developer.konghq.com/ai-gateway/), LiteLLM (https://docs.litellm.ai/docs/simple_proxy) and Portkey (https://portkey.ai/docs/product/mcp-gateway), also handle MCP traffic (vendor docs checked October 2026). Ask each one which person and which app account a tool call runs as.

### Is Elaichi an AI gateway or an MCP gateway?

Elaichi is a governed MCP control plane and hosted MCP gateway, not an AI gateway. Claude, ChatGPT, Cursor or any MCP client signs in to one address, https://api.elaichi.ai/mcp, and each call acts as the person who approved the grant. Elaichi serves 600+ connectors, checks each call against the person's access, and writes each executed call to an audit log that names the account it reached. It receives tool calls, never the person's prompt.

### Do we need an MCP gateway if we already run an AI gateway?

Only if staff connect AI clients to company apps. If AI at your company is your own code calling models, an AI gateway covers routing, budgets and logs, and an MCP gateway has nothing to govern. Once people use Claude, ChatGPT or Cursor to act in apps such as Salesforce or Zendesk, the questions become which tools each person may reach and on whose account. A model gateway does not answer those.

## Read next

- [What is an MCP gateway? The four shapes](/blog/what-is-an-mcp-gateway/) — What is an MCP gateway: one address between AI clients and their tools that signs people in, applies rules and records calls. It comes in four shapes.
- [Kong vs Cloudflare MCP gateway, or managed connectors?](/blog/kong-cloudflare-mcp-gateway-vs-managed-connectors/) — Kong vs Cloudflare MCP gateway: both govern MCP servers you build or bring. A managed connector platform ships the SaaS connectors already written.
- [An API gateway for MCP, or an MCP-native server?](/blog/api-gateway-with-mcp-vs-mcp-native/) — An API gateway for MCP fits when the tools are your own APIs, already behind it. For SaaS accounts your staff sign in to, an MCP-native server fits better.