Call models from your Convex app without setting up provider accounts, managing API keys, or stitching together another billing dashboard.
Suppose you want to add AI chat to your app. Maybe for customer support, product onboarding, or data analysis and reporting.
Before you start writing the code, you need to create an account with the model provider, add an API key, and figure out where to store it. If you want to test a different provider, you might need to go through this whole process again, and potentially update your code to work with the new provider’s SDK.
We're introducing AI Gateway to take that work off your plate. Convex handles provider credentials, includes the usage on your Convex bill, and provides added transparency on usage across your app. AI Gateway is available today to all teams on a paid plan. It gives you access to hundreds of models without needing to juggle API keys or provider accounts.
Here's what that looks like.
Call any model
With the AI SDK, install the Convex provider alongside your existing Convex dependency:
npm install ai @convex-dev/ai-sdk-provider
Then write an action:
// convex/summaries.ts
import { internalAction } from "./_generated/server";
import { v } from "convex/values";
import { generateText } from "ai";
import { convexGateway } from "@convex-dev/ai-sdk-provider";
export const summarize = internalAction({
args: { text: v.string() },
handler: async (_ctx, { text }) => {
const result = await generateText({
model: convexGateway("openai/gpt-5.6-terra"),
system: "Summarize the supplied text in three concise bullet points.",
prompt: text,
});
return result.text;
},
});
Notice there’s no OPENAI_API_KEY to copy into your deployment settings, and you never have to create an OpenAI account.
The provider works with generateText, generateObject,streamText, embed, evaluate (Jev), with image and video coming soon. To swap (or eval) a different model, just change the slug passed to convexGateway.
Your SDK, your rules
AI Gateway isn't a new agent framework: you don't need to move your prompts, tools, or application logic into a Convex-specific API to use it.
If you’re using the OpenAI SDK instead of AI SDK, configure it inside your action like this:
import OpenAI from "openai";
import { getServiceToken } from "convex/server";
const client = new OpenAI({
baseURL: "https://ai-gateway.convex.dev/v1",
apiKey: () => getServiceToken("ai-gateway"),
});
const response = await client.chat.completions.create({
model: "anthropic/claude-sonnet-5",
messages: [{ role: "user", content: "Explain vector search in one sentence." }],
});
The gateway also supports the Convex Agent Component:
import { Agent } from "@convex-dev/agent";
import { convexGateway } from "@convex-dev/ai-sdk-provider";
import { components } from "./_generated/api";
const agent = new Agent(components.agent, {
name: "Support agent",
languageModel: convexGateway("openai/gpt-4o-mini"),
instructions: "You answer questions about our product.",
});
The gateway also supports OpenAI Responses and Anthropic Messages APIs as well through the AI SDK.
Pick the interface your application needs and check the documentation for all the supported options.
Embeddings: a match made in vector space
To answer user questions, your support assistant might need to first find relevant documentation and then ask a language model to answer using those results.
You can generate the embeddings for RAG through the same gateway:
import { embed } from "ai";
import { convexGateway } from "@convex-dev/ai-sdk-provider";
const { embedding } = await embed({
model: convexGateway.embeddingModel("openai/text-embedding-3-small"),
value: "How do I invite a teammate?",
});
Run this inside an action, then you can retrieve the vector with Convex vector search.
The call from inside the house
Typically when you call a model provider, the API key tells the service which account to charge, but it doesn't help you identify which function or deployment made the request.
Fortunately, Convex already has that context. When an action requests a gateway token, Convex issues a short-lived credential scoped to the deployment. The gateway verifies it and records usage attributed to the calling project, deployment, and function.
That means less credential setup, but it also means better answers when you ask, “Where did this spend come from?”
See the cost alongside the app
Open your team's Usage page in the Convex dashboard and select AI Gateway to see daily spend by project and model. In the function breakdown, choose the AI tab to see which functions are responsible.
AI Gateway usage appears as a separate line item on your Convex invoice. We charge the same model rates as OpenRouter, without a gateway markup.
Before opening an AI feature to users, set a team spending limit in Team Settings → Billing. You can configure a warning threshold and a disable threshold. AI Gateway spend counts toward that limit alongside your other Convex usage.
One important detail: the disable threshold disables the team's projects, not just AI Gateway. Choose it with your whole application in mind. The spending limits documentation explains the behavior.
Try it in your next action
AI Gateway is available on paid Convex plans, including Starter. Start with a model call, check its usage in the dashboard, and build from there. It also works with authenticated, project-linked local development.
With the setup required to call models removed, we hope you’ll have more bandwidth to focus on what makes your feature useful: the context you give the model, the tools it can use, and what your app does with the result.
