Custom classifiers

A custom classifier teaches Gateway what “simple” and “complex” mean for your workload, from prompts you label, and replaces the built-in complexity scorer in a Build Your Own Router config. Use one when the built-in signal misreads your traffic, such as a one-line support message that is really a hard reconciliation task.

How it works

You supply {prompt, complexity} examples, 0 (trivial) to 1 (complex). Gateway splits them at 0.5, embeds every prompt, and averages each side into a centroid, with no eval run or judge model. Each request is embedded and scored by how much closer it sits to the complex centroid than the simple one.

The classifier scores complexity only, which BYOR uses to tilt model scores. Per-model scores come from your benchmarks, and categories such as code or math from BYOR’s capability classifier.

Prepare training data

Score prompts by difficulty for your workload, not length. You need at least 3 examples below 0.5 and 3 at 0.5 or above; more per side gives a steadier score.

[
  {"prompt": "Reconcile these two conflicting invoices for customer_abc and explain every discrepancy", "complexity": 0.9},
  {"prompt": "A customer's OAuth token refresh fails intermittently across three regions, trace the root cause", "complexity": 0.95},
  {"prompt": "Draft a migration plan to move our webhook consumers off the deprecated v1 endpoint", "complexity": 0.85},
  {"prompt": "Change the billing email on account customer_abc", "complexity": 0.15},
  {"prompt": "Reset the password for alice@acme.com", "complexity": 0.1},
  {"prompt": "What are your support hours?", "complexity": 0.05}
]

prompt must be a non-empty string and complexity a number from 0 to 1; errors name the 1-based item number. For a spreadsheet, choose Import from CSV: one example per row, score in the last column. A header row is skipped, and commas inside the prompt are kept.

Create a classifier

1. In Routing policies → Prompt classifiers, select Add classifier
2. Enter a name (unique in your organization) and optional description
3. Paste the JSON into Scored prompts (JSON), or import a CSV
4. Select Add classifier. The button stays disabled until each side has 3 examples

Creating or editing needs Manage routing; viewing needs View routing.

Use it in a BYOR config

In the BYOR config editor under Routing policies → Policies, pick your classifier in Classifier and save, or leave Default for the built-in scorer. A classifier edited below 3 examples per side has no scorer and disappears from the selector. Intelligent policies can’t use classifiers.

Edit or delete

Editing examples rebuilds the classifier, and configs using it pick up the change on the next request. Deleting it detaches it, and those configs fall back to the built-in scorer instead of failing.

Next steps