Skip to main content
Connect your AI

Setting up the AI gateway

The AI gateway is a StackJack address that your people's AI tools call instead of calling the AI vendor directly. The gateway checks each call against your data-loss rules, counts what it costs,…

Written By Christopher Scaminaci

Last updated 9 days ago

The AI gateway is a StackJack address that your people's AI tools call instead of calling the AI vendor directly. The gateway checks each call against your data-loss rules, counts what it costs, applies the budgets you set, and then passes the call on to the vendor with your own vendor key. Your people never see that key. Every call is recorded as a fact in AI Governance, so Activity, Findings and Spend show what went through.

At a glance

  • What it does: puts a StackJack address between your staff's AI tools and OpenAI, Anthropic or Azure OpenAI.
  • Plan: Business and above.
  • Who: an Administrator.
  • Where you set it up: AI Governance → AI traffic → AI gateway.
  • Where you see the data: Activity → Activity explorer, Activity → Findings and Activity → Spend (budgets).

The gateway needs the Business plan or higher. Below it, you can open the page and prepare things, but the switches stay off and the page says why.

The same address also serves remote MCP servers you approved: your people's tools call a proxy address with the same personal key, and StackJack checks each tool call. See Setting up the MCP proxy.

Which vendors and tools it serves

VendorWhat the gateway servesThe base address a tool uses
OpenAIChat completions, the Responses API, embeddings and the model listhttps://ai.stackjack.io/openai/v1 (United States) or https://ai-eu.stackjack.io/openai/v1 (Europe)
AnthropicMessages, token counting and the model listhttps://ai.stackjack.io/anthropic (United States) or https://ai-eu.stackjack.io/anthropic (Europe)
Azure OpenAIThe calls of the Azure OpenAI resource you add as a vendor keyAn address that carries the id of that vendor key. Ask support for it.

Your region decides the address: ai.stackjack.io serves organizations in the United States region and ai-eu.stackjack.io serves organizations in the Europe region. A personal key names its region (sjk_us_ or sjk_eu_), and a key sent to the other region's address is refused. The exact address for your organization is on AI Governance → AI traffic → AI gateway, in the row called AI edge address. (The AI edge is the set of StackJack servers that run the gateway.)

Two limits to know first:

  • Gemini CLI is not covered. It uses a different protocol that the gateway does not serve. Only OpenAI, Anthropic and Azure OpenAI traffic goes through the gateway.
  • Cursor sends its requests from Cursor's own servers, not from your person's computer. Cursor's servers read the prompt before the gateway does, so the gateway cannot hold back what Cursor has already read. That is also why the gateway addresses are public.

Set it up

  1. Open AI Governance → AI traffic → AI gateway. You need the governance administrator role.
  2. Under Lanes, switch AI gateway on and select Save. StackJack creates a source called StackJack AI gateway the first time, at the capture level Scan only. Change the level on All sources if you want text kept (see Capture levels and privacy).
  3. Under Vendor keys, add the keys your people should reach the vendors with: choose the vendor, give the key a label, paste the secret and select Add the key. For Azure OpenAI you also give the address of your Azure resource, which must be an https address on openai.azure.com or services.ai.azure.com, with no path. The secret goes to StackJack's vault and is not shown again. The first key of a vendor becomes its default; you can make another the default, rotate a key (a new secret) or revoke it.
  4. Under Personal keys, give each person a seat (Give a seat, or Give every staff member a seat). A person collects their own key. For an agent (a bot, not a person), name it and select Make the key; the key is shown once, to you. See Your personal AI key.
  5. Point the tools at the gateway:
    • Claude Code and Codex. Use the endpoint settings (Staff computers → Coding tools). When the gateway is on, the settings StackJack generates carry the gateway address and the way to read each person's key. Turn on Require the gateway in settings if the settings should also allow only the gateway as a provider, so a tool cannot go around it. The person runs the one-line install command that their key card shows once.
    • Cursor and VS Code. These are manual and a pushed policy cannot set them. In Cursor, open Settings, then Models, turn on Override OpenAI Base URL, paste the OpenAI address above and paste the personal key as the API key. In VS Code, add a custom model provider in the Copilot chat settings with the same address and key.
  6. Set budgets if you want limits (see Your personal AI key). Open Activity and Findings to see the first calls.

What your people see

A policy refusal is never an HTTP 429 or 5xx, because tools retry those. The tool shows the message below as an error from the AI, in the vendor's own error shape.

What happenedWhat the person sees
A block rule matched"StackJack blocked this request: it matched the data-loss rule 'rule name'." The text that matched is never repeated.
A warn rule matched"This request matched the data-loss rule 'rule name', which asks for a reason before it can continue. Open (a link), give a reason, then retry in about a minute."
A mask rule matchedOn OpenAI and Azure OpenAI traffic, StackJack replaces the matching value in the prompt before the AI sees it and the call goes on. The person sees an ordinary answer.
A mask rule matched on Anthropic trafficThe gateway cannot change a Claude request (Claude Code breaks on a rewritten request). It does what you chose in When a lane cannot mask: Block (the default) refuses the call like a block, Allow lets it through unmasked.
The budget is used up (with a hard stop)"The AI budget for this account is used up, so StackJack stopped this request."
The key is wrong"StackJack does not recognise this personal key." Or: "This personal key was revoked.", "This personal key has expired.", or "This personal key belongs to another StackJack region. Use the address of the region that issued it."
The key may not use the model or the vendor"This personal key is not allowed to use this model." or "...to use this AI vendor."
The lane is not available"This StackJack AI lane is not available for this organization: it needs the Business plan and the lane switched on."

Warn. The link opens a StackJack page that needs no sign-in. The person types a reason of 10 to 500 characters and gets an exception for that one rule that lasts 15 minutes. The link works for 30 minutes and once. The gateway learns of the exception within about a minute, which is why the message says to retry in about a minute.

Answers are checked too. The gateway reads an answer as it streams. A block rule can stop it part-way, and the tool then shows the same kind of message. Text the tool had already shown stays on the screen; it is what StackJack records as the partial answer, at a capture level that keeps text.

When the gateway cannot read the text. The gateway reads prompt and answer text only when the capture level of its source allows scanning (Scan only, Masked text or Full text). At Facts only it still checks the key, the model, the vendor and the budget, and it still records what the call cost, but it does not read the text, so a data-loss rule that needs the text does not act.

When StackJack's copy of your policy is stale

The gateway works from a copy of your settings, your keys, your rules and your budgets that StackJack refreshes about every minute. If the copy cannot be refreshed and becomes older than Stale after (5 to 240 minutes, 15 by default), the gateway does one of two things, depending on the setting of the lane:

  • By default it keeps serving with your last good settings and marks each recorded call as stale, so you can see which calls ran on an old copy.
  • If you switched on Block calls when the AI edge cannot decide in time, it refuses the call with "StackJack could not confirm this organization's current policy, so it stopped this request. Try again in a few minutes."

If the gateway has no copy at all for your organization, it cannot tell whose key it is holding, so it answers "StackJack does not recognise this personal key." It never lets such a call through unchecked. If StackJack has to pause the gateway itself, calls go on to the vendor without checks, so your people's work does not stop because of a StackJack switch.

Good to know

  • Cost. The gateway prices a call from the token counts the vendor reports and a price table. A model that is not in the table is allowed, shown as unpriced, and never trips a hard stop.
  • Vendor errors come back exactly as the vendor sent them. A 502 means the vendor could not be reached.
  • Who can set it up. Only governance administrators open the AI gateway page.
  • If the gateway is down and people cannot work, see Send staff straight to the vendors.

Related: Get started with AI Governance, Data-loss rules, findings and scanning, Setting up the MCP proxy.