typesafe/docs/api.md

Documentation Index

Fetch the complete documentation index at: https://docs.typesafe.ai/llms.txt Use this file to discover all available pages before exploring further.

API reference

Full HTTP API reference for the TypeSafe evaluation endpoint.

Evaluate a state against a map of typed questions and get back structured answers, one per question. For a guided introduction, start with the primitives.

Evaluation endpoint

POST https://api.typesafe.ai/v1/systemone
Authorization: Bearer <API_KEY>
Content-Type: application/json

Request body

The top-level shape of every request. Each entry in the questions map is a typed question you name.

The content to evaluate. A plain string for text, or structured data (object/array) for things like chat logs, records, or the current state of your application. See [State](/concepts/state) for formats and best practices. The model that handles the request. Use `"jev-latest"`, TypeSafe's flagship model. See [Models](/models) for the available models and aliases. A map of typed [Question](#question-types) objects. You choose each key; answers come back under the same keys. A key you choose. The matching [Answer](#answer-types) is returned under this same id. The key is not sent to the underlying model and is not used in inference.

{
  "state": "Help! My payouts have been failing for 3 days.",
  "model": "jev-latest",
  "questions": {
    "is_urgent": {
      "type": "noul",
      "instructions": "Does this convey urgency?"
    }
  }
}

Question types

A Question is one of three types, set by its type field. All three share type and instructions; each adds its own criteria.

Noul

A yes/no question. Returns the probability the answer is yes.

The yes/no question to evaluate. Optional descriptions of what a yes and a no mean. What a yes (value near 1) means.

<ParamField body="false" type="string">
  What a no (value near 0) means.
</ParamField>

{
  "state": "Help! My payouts have been failing for 3 days.",
  "model": "jev-latest",
  "questions": {
    "is_urgent": {
      "type": "noul",
      "instructions": "Does this convey urgency?",
      "criteria": {
        "true": "Explicitly time-sensitive",
        "false": "No urgency expressed"
      }
    }
  }
}

Choice

Picks one option from a set you define. Returns the chosen option and the full probability distribution.

What the model should decide. A map of option to rubric description; use null when an option needs no extra detail. A key you choose. A description of this option.

{
  "state": "Help! My payouts have been failing for 3 days.",
  "model": "jev-latest",
  "questions": {
    "department": {
      "type": "choice",
      "instructions": "Which team should handle this?",
      "criteria": {
        "billing": "Payments, invoicing, refunds",
        "technical": "Bugs, outages, integrations",
        "sales": "Pricing, upgrades, new accounts"
      }
    }
  }
}

Score

Rates the state along a rubric you define. Returns a probability-weighted value across your levels.

What the model should rate. An ordered array of level descriptions. You must include at least two levels.

{
  "state": "Help! My payouts have been failing for 3 days.",
  "model": "jev-latest",
  "questions": {
    "frustration": {
      "type": "score",
      "instructions": "How frustrated is the customer?",
      "criteria": ["Calm", "Frustrated", "Very angry"]
    }
  }
}

Response body

One answer per question, returned under the same ids you provided.

The model that performed the evaluation. One [Answer](#answer-types) per question, keyed by the same ids you used in questions. The same id you chose in questions. Token usage for the request.

<ResponseField name="output_tokens" type="integer" />

{
  "model": "jev-latest",
  "answers": {
    "is_urgent": {
      "type": "noul",
      "noul": 0.92
    }
  },
  "usage": { "input_tokens": 312, "output_tokens": 48 }
}

Answer types

Every answer carries a type matching its question. Choice and Score answers also carry a confidence between 0 to 1, derived from the answer's probability distribution. See Confidence.

Noul answer

The yes/no answer on a scale from 0 (no) to 1 (yes).

{
  "model": "jev-latest",
  "answers": {
    "is_urgent": {
      "type": "noul",
      "noul": 0.92
    }
  },
  "usage": { "input_tokens": 312, "output_tokens": 48 }
}

Choice answer

The highest-probability option. Every option mapped to its probability (floats that sum to 1). An option you defined in criteria. How certain the model is, derived from probabilities.

{
  "model": "jev-latest",
  "answers": {
    "department": {
      "type": "choice",
      "choice": "technical",
      "probabilities": { "billing": 0.08, "technical": 0.85, "sales": 0.07 },
      "confidence": 0.82
    }
  },
  "usage": { "input_tokens": 312, "output_tokens": 48 }
}

Score answer

The probability-weighted answer across the levels; can land between levels. Each level number mapped back to its description. Each level (string key) mapped to its probability (floats that sum to 1). A level index, as a string key matching legend. How certain the model is, derived from probabilities.

{
  "model": "jev-latest",
  "answers": {
    "frustration": {
      "type": "score",
      "score": 1.6,
      "legend": { "0": "Calm", "1": "Frustrated", "2": "Very angry" },
      "probabilities": { "0": 0.05, "1": 0.3, "2": 0.65 },
      "confidence": 0.78
    }
  },
  "usage": { "input_tokens": 312, "output_tokens": 48 }
}

Errors

Errors use standard HTTP status codes with a JSON body describing what went wrong.

Status	Meaning
401 Unauthorized	Missing or invalid API key. Check the Authorization header.
422 Unprocessable Entity	The request body failed validation — for example a missing required field or a malformed question. The body details the offending field.
429 Too Many Requests	You have exceeded your rate limit. Back off and retry after a short delay.
529 Overloaded	TypeSafe is temporarily overloaded. Retry after a short delay.

Handling rate limits

When you receive a 429 Too Many Requests or 529 Overloaded response, retry the request with exponential backoff instead of retrying immediately. Our client SDKs handle this automatically, so no extra handling is needed if you use one of our SDKs with its default retry policy.