Skip to main content
Connect your AI

Vertex AI request-response logs

If your people (or your own software) call models on Google Cloud Vertex AI, Vertex can write every request and every response to a BigQuery table in your project. AI Governance reads that table and…

Written By Christopher Scaminaci

Last updated 13 days ago

If your people (or your own software) call models on Google Cloud Vertex AI, Vertex can write every request and every response to a BigQuery table in your project. AI Governance reads that table and adds the calls to the same Spend and Activity views as the rest of your AI use. This works for Gemini models and for Claude models on Vertex.

At a glance

  • What it does: reads Vertex AI request and response logs from your Google Cloud project.
  • Plan: every plan. Scanning text with rules needs Pro.
  • Who: an Administrator.
  • Where you set it up: AI Governance → Connected accounts → Cloud AI.
  • Where you see the data: Activity → Activity explorer and Activity → Spend.

The source reads a resource in your Google Cloud account, with a service account key you create. It never writes to your account. You set it up on AI Governance → Connected accounts → Cloud AI, and the page writes the exact steps for the names you type.

What it reads, and what it never reads

  • It reads one table: the request-response logging table you name. It runs read-only queries against it, in the billing project you name.
  • It reads the time of each call, the model, the token counts when they are present, and, at a capture level that keeps text, the request and the response.
  • It never reads another table, never reads another dataset, and never writes, changes or deletes anything in your project.

Set it up

You need a Google Cloud project that already uses Vertex AI, and the right to create a dataset, a service account and a key.

  1. Create the dataset and the table. Make a dataset (note its location) and a table partitioned by logging_time. Do this only if your Vertex setup accepts a table you made in advance. If not, let Vertex create the table when you turn logging on, and use the names it gives.
  2. Turn on request-response logging for each model you want StackJack to see, and send every model to the same one table. The Cloud AI page shows the REST call (setPublisherModelConfig). Claude models on Vertex are set with the REST call only. You choose a sampling rate here: 1 means every request.
  3. Create a service account and give it two roles: BigQuery Data Viewer on the dataset, and BigQuery Job User on the project the queries run in (the billing project). Those two roles let StackJack read the table and run queries, and nothing else.
  4. Create a JSON key for the service account. Google Cloud organizations created on or after 2024-05-03 block key creation by default, and you need a policy exception first. Google explains it at https://cloud.google.com/resource-manager/docs/secure-by-default-organizations.
  5. Add the source. Open the Cloud AI page, fill in the project, the billing project, the dataset, the table and the location, and paste the key. StackJack runs one test query with your key before it stores anything. If the test fails, nothing is stored and the page says the key could not read the table. If it succeeds, the key is stored sealed and is never shown again.

To change the key later, select Replace key on the source. The new key is tested against the stored table first, and the old key stays in use until the new one passes.

The cost to you

The queries run in your billing project, so Google bills them to you. StackJack limits each query to a number of bytes (a platform limit you can only lower, in the form), reads only the columns it needs, and stops a slice that the limit refuses. You see the problem on the source's health page and StackJack tries again later. A small table costs very little. A large table with long text costs more.

What you should know

  • Vertex calls are not linked to people. The log has no reliable sign-in name, so StackJack counts these calls as one source and never as a named person's use.
  • A sampling rate is shown, not applied. If your logging samples requests, the rate you typed appears beside the figures on the source. StackJack never multiplies a count by it.
  • Large calls are not logged. Google does not log a call whose logged row would be over 10 MB, so StackJack never sees it.
  • Cost figures can be missing. StackJack has no list price for Vertex models yet, so a Vertex call shows as unpriced. It never shows as free. Token counts are shown when they are in the log.
  • Two views of one call are two facts. A call that Claude Code reports and that Vertex logs is counted once for each source, and the two are never added together.

What we have not confirmed

These points follow Google's published pages and are not confirmed against a live project:

  • That Vertex writes into a table you made in advance and partitioned (step 1 says "if your Vertex setup accepts it").
  • That the token counts sit inside the logged response in the places StackJack looks for them (the usage block of a Gemini response, and the usage block of a Claude response). Google's page shows no sample response. If counts are missing, the call still appears, without tokens.
  • The exact field names of the setPublisherModelConfig body. The Cloud AI page tells you to check them against Google's page about request-response logging.

Where the data goes and what stops it

Rows from a dataset in another region are copied into your StackJack region, and the setup page says so. You can stop the source in three ways: switch it off on All sources; turn logging off on the models in Vertex; or delete the service account key in Google Cloud, which stops StackJack at once.

Tell your staff

Your staff notice must say that AI Governance reads the request and response logs of AI calls made through your Google Cloud project. The exact wording is for your legal adviser to confirm.