Custom classifiers

A custom classifier teaches Gateway your own definition of “simple” versus “complex” from a set of prompts you label. Gateway turns those labels into a runtime complexity scorer that a Build Your Own Router config uses to weigh how hard each incoming request is. Use it when the built-in complexity signal does not match how your workload actually splits, for example when a short support message can still be a hard reconciliation task.

How it works

You supply labeled {prompt, complexity} examples, where complexity runs from 0 (trivial) to 1 (complex). Gateway builds the classifier from them in one step, with no eval run and no judge model:

1. Split your examples at 0.5 into a simple set (below 0.5) and a complex set (0.5 and above)
2. Embed every prompt, then average each set into a single reference point, called a centroid
3. Store the simple centroid and the complex centroid together as the classifier

At request time, Gateway embeds the incoming prompt and scores it by which centroid it sits closer to. The classifier adds a few milliseconds per request, negligible compared to LLM inference.

A classifier scores complexity only. It does not carry per-model scores (those come from your benchmarks) and it does not detect request categories like code or math (BYOR handles that separately).

Prepare your training data

Each example is one prompt and one complexity score from 0 to 1. Score prompts the way your workload actually experiences them, not by length. A one-line request can be complex, and a long request can be trivial.

Gateway needs at least 3 examples on each side of the 0.5 split to build a usable classifier: 3 or more scored below 0.5, and 3 or more scored at 0.5 or above. Below that floor on either side the centroid is too noisy to trust, and Gateway saves the examples but does not produce a scorer until you add more. More examples per side give a steadier centroid, so aim for a spread of representative prompts rather than the minimum.

Import training data

In the create dialog, paste a JSON array of {prompt, complexity} objects into the scored-prompts field. This is the whole classifier.

1
[
2
  {
3
    "prompt": "A customer's OAuth token refresh fails intermittently across three regions, trace the root cause",
4
    "complexity": 0.95
5
  },
6
  {
7
    "prompt": "Reconcile these two conflicting invoices for customer_abc and explain every discrepancy",
8
    "complexity": 0.9
9
  },
10
  {
11
    "prompt": "Draft a migration plan to move our webhook consumers off the deprecated v1 endpoint",
12
    "complexity": 0.85
13
  },
14
  {
15
    "prompt": "Change the billing email on account customer_abc",
16
    "complexity": 0.15
17
  },
18
  {
19
    "prompt": "Reset the password for [email protected]",
20
    "complexity": 0.1
21
  },
22
  {
23
    "prompt": "What are your support hours?",
24
    "complexity": 0.05
25
  }
26
]

complexity must be a number from 0 to 1, and prompt must be a non-empty string. If a value falls outside that range or an item is missing a field, the dialog reports the 1-based item number so you can fix it before saving.

You can also import from a spreadsheet. Choose Import from CSV and upload a file with one example per row and the complexity score in the last column. A header row is fine, Gateway skips it, and commas inside the prompt are preserved as long as the score is the final value on the line.

prompt,complexity
"A customer's OAuth token refresh fails intermittently across three regions, trace the root cause",0.95
"Reconcile these two conflicting invoices for customer_abc and explain every discrepancy",0.9
Draft a migration plan to move our webhook consumers off the deprecated v1 endpoint,0.85
Change the billing email on account customer_abc,0.15
Reset the password for [email protected],0.1
What are your support hours?,0.05

Create a classifier

1. Go to Configuration → Classifiers in the dashboard and select Add classifier
2. Give it a name (unique within your organization) and an optional description
3. Paste your JSON, or import a CSV
4. Confirm the count hint shows at least 3 simple and 3 complex examples, then select Create

The name must be unique in your organization, so the classifier selector never shows two identically-named classifiers. Creating a classifier requires the Manage routing permission; viewing one requires View routing.

Use a classifier in routing

A custom classifier becomes the complexity scorer for a Build Your Own Router config. BYOR ranks models on the benchmark scores you choose; the classifier adds the complexity signal a score-only benchmark lacks, measured on your own scale.

1. Open your BYOR config in Configuration → Routing in the dashboard
2. In the Classifier selector, choose your custom classifier instead of the default
3. Save the config

Gateway pushes the classifier to that config’s runtime, and from then on it scores each request’s complexity against your simple and complex centroids. Leave the selector on the default to use Gateway’s built-in complexity scorer.

Intelligent routing does not use custom classifiers today. The complexity classifier is part of the BYOR flow.

Edit, retrain, or delete

Editing a classifier’s examples rebuilds its centroid pair from the new labels, and Gateway resyncs any BYOR config that uses it so the change takes effect without re-saving each config. Retrain a classifier whenever your sense of a hard request drifts, for example after you add a new product area, by adding or rescoring examples.

Deleting a classifier detaches it from any BYOR config that referenced it. Those configs fall back to Gateway’s default complexity scorer rather than failing.

Next steps