Skip to main content
Control and respond

Rule packs, modes and custom rules

Rules decide what StackJack looks for in the text of AI use. A rule pack is a ready-made set of rules for one kind of risk. You turn a pack on to start scanning for it. You can start in Test mode to…

Written By Christopher Scaminaci

Last updated 4 days ago

Rules decide what StackJack looks for in the text of AI use. A rule pack is a ready-made set of rules for one kind of risk. You turn a pack on to start scanning for it. You can start in Test mode to see what a pack finds before it alerts. You can also write custom rules for things only your organization knows about, and replay a rule on past data to see what it would have found.

At a glance

  • What it does: scans the text of AI use and records a finding when a rule matches.
  • Plan: Pro and above. On Free, rules do not scan telemetry.
  • Who: any governance role reads the Rule packs page. An Administrator turns packs on and off and writes custom rules.
  • Where you set it up: AI Governance → Rules → Rule packs.
  • Where you see the result: Activity → Findings and the Alerts inbox.
  • Needs: a source with a capture level that scans text (Scan only, Masked text or Full text).

The rule packs

PackWhat it looks for
Secrets (gitleaks)Credentials and keys of hundreds of vendors.
Cloud and platform secretsConnection strings, database addresses, cloud keys, key files, bearer tokens, and AI and billing keys.
MSP stackStackJack's own secrets, the credentials an MSP pastes for the connectors StackJack offers, and password-manager exports.
Personal dataContact details, national identifiers, bank and crypto identifiers and named entities.
Payment cards (PCI)Card numbers, a card number beside its security code or expiry date, and magnetic-stripe data.
Health (HIPAA)Medical record and health plan numbers, patient details and health terms beside an identifier, and the markers of HL7, FHIR and CDA messages.
GDPR special categoriesHealth, origin, opinion, belief, union membership, genetic, biometric and sex-life terms next to a person, in English, German, French, Spanish, Italian and Dutch.
Defense (CMMC): CUI and export-control markingsControlled unclassified information markings, classified markings, distribution statements and export-control classifications.
Agent safetyPrompt-injection markers in tool results and responses, instructions to send data out, and attempts to switch off logging, monitoring and permission checks.
Client identifiersYour own clients, found through your client names list. A match attributes the AI use to the client. It is not a finding unless you give a rule an action. See Client names.
AI judgments (Jev)Rules that ask an AI model to judge the text. See AI judgments (Jev).

The page shows how many packs are on, for example "3 of 11 on".

Turn a pack on or off

On the Rule packs page, each pack has a button:

  • Turn on. The pack starts with every rule as StackJack ships it.
  • Turn on in Test mode. Every rule of the pack starts in Test mode, so nothing alerts or changes stored text until you make a rule Live.
  • Turn off. The pack's rules stop scanning new data. Findings already recorded stay, and so do your changes to its rules.

The menu on a pack also offers See its rules.

Modes

Each rule has a mode.

ModeWhat it does
LiveFindings count, and the rule's actions apply.
TestFindings are kept. They never alert or change stored text.
OffThe rule does not run. It stays in the list so you can still try it on past data.

Start a new pack in Test mode. Read the findings for a few days. When you trust a rule, make it Live.

The actions a rule can take

A Live rule can ask for one or more of these actions:

  • Alert. Raises an alert.
  • Mask the stored copy. Hides the matched value in the text StackJack stores.
  • Open an incident. Needs the Business plan.
  • Open a PSA ticket.
  • Block the call. Refuses the call.
  • Mask before the AI sees it. Hides the matched value before the AI reads it.
  • Warn the person. Asks the person for a reason first.

Block, mask before the AI sees it, and warn act only where StackJack is in the path of the call. The rule list shows where each rule acts, for example StackJack tools, the AI gateway, the MCP proxy, Claude hooks, the browser extension and the endpoint agent. Where a lane is switched off for your organization, the page says so. Your plan also limits these actions. See Data-loss rules, findings and scanning StackJack tool traffic and Plans and what each includes.

Look at the rules

The Rules table lists every rule your packs and you run. Filter it by Search (name or key), Pack, Origin, Mode and At least this serious. Select Details on a rule to see how it runs, change it if you are an administrator, and see what it has been through.

The panel that opens has four tabs:

  • Overview: what the rule looks for, its mode, and what it does on each lane (one row for each lane). A lane that is switched off for your organization says so in its row.
  • Your changes: change a shipped rule's severity, actions, numbers and where it applies. Each box shows the value StackJack ships, and the day your organization changed it. A box you changed but did not save is outlined and marked Not saved. Select Save changes at the bottom of the panel, or Discard. An administrator only.
  • Try it: replay the rule on past data. See Replay a rule, below.
  • History: every version of your changes, and the rule as it runs.

A change of mode applies at once. Your other changes wait for Save changes. If you close the panel with changes that are not saved, StackJack asks first.

Write a custom rule

A custom rule looks for something only your organization knows about, such as a project code name.

  1. On the Rule packs page, select New custom rule.
  2. Enter a Name and, optionally, What it finds.
  3. Under Looks for, choose one of:
    • A list of words or phrases. One per line. Options: Whole words only and Match capital letters exactly.
    • A pattern (regular expression). Use Check the pattern to test it.
    • Your clients (the client dictionary). Choose which parts of a client count. None ticked means all of them.
  4. Choose How serious a match is.
  5. Under What happens to a Live finding, tick the actions you want.
  6. Optionally narrow it: Tool names, Where the AI was used (Command line, Editor, Browser chat, Direct API, Agent, MCP tool), Which part of the exchange (Prompt, Answer, Tool arguments, Tool results, System instructions) and Which accounts (Company accounts, Personal accounts).
  7. Save.

A new custom rule starts in Test mode. It keeps its findings and never alerts or changes stored text until you make it Live.

A custom rule that is too slow is paused for a short time and resumes by itself. A busy server can pause a healthy rule.

Replay a rule

A replay shows what a rule would have found in the past.

  1. Open a rule's Details and select the Try it tab, then Try it on the last 30 days. Or open the custom rule page after you save it.
  2. Enter Days to look back. The range is 1 to 30.
  3. Start the replay. The page updates every few seconds.

The result shows how many records were scanned and how many hits the rule would have had. Replay reads stored text only, so sources at Facts only or Scan only are not replayed. A replay is part of the Pro plan and above.

What you will see

  • Findings in Activity → Findings, labelled by rule.
  • Alerts in Alerts → Inbox when a Live rule asks for an alert.
  • On the Rule packs page, how many packs are on.

Troubleshooting

What you seeWhat to check
A pack is on but no findings arriveCheck the plan is Pro or above. Check the source's capture level is not Facts only. Check the rule is Live, not Test.
Findings arrive but nothing alertsThe rules are in Test mode. Make them Live.
"The rule is no longer part of what your organization runs. Turn its pack on and try again."The replay needs the pack to be on.
A replay finds nothingThe source keeps no text. Replay reads stored text only.
A custom rule is refusedThe page says why in words, for example a name that needs a letter or a definition that is not valid JSON.

Next