teren de test TypeSafe: docs offline + script de proba
Separat de produsele ROA. docs/ = documentatia oficiala descarcata ca Markdown (111 pagini), reluabila cu update_docs.sh. typesafe_test.py face un apel cu cate o intrebare din fiecare tip (choice/noul/score) pe o linie de factura de furnizor. Cheia API se ia din TYPESAFE_API_KEY, nu se versioneaza. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KHLUSsKP99G6ebv2fFUKQV
This commit is contained in:
91
docs/introduction/machine-learning-primer.md
Normal file
91
docs/introduction/machine-learning-primer.md
Normal file
@@ -0,0 +1,91 @@
|
||||
> ## Documentation Index
|
||||
> Fetch the complete documentation index at: https://docs.typesafe.ai/llms.txt
|
||||
> Use this file to discover all available pages before exploring further.
|
||||
|
||||
# AI primer
|
||||
|
||||
> Why TypeSafe trains decision models with calibrated probabilities instead of optimizing for generated text.
|
||||
|
||||
Most AI products are built around a conversation between a model and a person. TypeSafe starts from a different bet: large-scale automation will be dominated by AI-to-AI and AI-to-software interactions, so the machine interface matters more than the chat interface.
|
||||
|
||||
> **We call this Machine Native Intelligence:**
|
||||
>
|
||||
> AI with software-like properties such as structure, reliability, observability, testability, speed, consistency, and low cost.
|
||||
|
||||
## Building prod, not God
|
||||
|
||||
TypeSafe is not trying to build a model that does everything. It is designed for production systems where code needs a narrow decision it can inspect and act on.
|
||||
|
||||
Our expectation is that large-scale AI automation will be closer to 99% machine-to-machine interactions and 1% human interaction. That shifts the design target from responses that feel good to read toward outputs that behave predictably inside software.
|
||||
|
||||
Read the [TypeSafe manifesto](https://typesafe.ai/manifesto).
|
||||
|
||||
## Three post-training approaches
|
||||
|
||||
Pretrained language models have been adapted in two major ways. TypeSafe adds a third. RLHF and RLVR are shown here for context; TypeSafe's training path is RLCD.
|
||||
|
||||
<Columns cols={3}>
|
||||
<Card title="RLHF" icon="messages-square" type="note">
|
||||
**Reinforcement learning from human feedback** turned pretrained models into chatbots. It trains models to produce responses people prefer.
|
||||
</Card>
|
||||
|
||||
<Card title="RLVR" icon="brain-circuit" type="note">
|
||||
**Reinforcement learning with verifiable rewards** created reasoning models that are strong at tasks such as mathematics, but slower and more expensive.
|
||||
</Card>
|
||||
|
||||
<Card title="RLCD" icon="binary" type="tip">
|
||||
**Reinforcement learning for calibrated decisions** trains TypeSafe to return decisions and calibrated probabilities instead of generated text.
|
||||
</Card>
|
||||
</Columns>
|
||||
|
||||
RLHF was used to train InstructGPT and ChatGPT and was [co-invented by Diogo Almeida](https://scholar.google.com/citations?user=0T4y07QAAAAJ\&hl=en), cofounder of TypeSafe.
|
||||
|
||||
<Frame>
|
||||
<img className="block dark:hidden" src="https://mintcdn.com/ts-docs/aFVnpmCIX68NpsV1/images/ai-primer/training-paths-light.webp?fit=max&auto=format&n=aFVnpmCIX68NpsV1&q=85&s=61898215ac31388d3be15bf583b743ee" alt="Pretrained language models branch into muted RLHF and RLVR paths and an emphasized RLCD decision-model path." width="2048" height="810" data-path="images/ai-primer/training-paths-light.webp" />
|
||||
|
||||
<img className="hidden dark:block" src="https://mintcdn.com/ts-docs/aFVnpmCIX68NpsV1/images/ai-primer/training-paths-dark.webp?fit=max&auto=format&n=aFVnpmCIX68NpsV1&q=85&s=2747633edb0e54fa3f14a8aba830f4fd" alt="Pretrained language models branch into muted RLHF and RLVR paths and an emphasized RLCD decision-model path." width="2048" height="810" data-path="images/ai-primer/training-paths-dark.webp" />
|
||||
</Frame>
|
||||
|
||||
## RLCD and calibrated decisions
|
||||
|
||||
RLCD optimizes for a different output contract:
|
||||
|
||||
* The model does not generate text.
|
||||
* It returns decisions and probabilities.
|
||||
* Higher probability should correspond to a greater chance that the answer is correct.
|
||||
|
||||
Calibration makes uncertainty usable by software. Across many predictions from a well-calibrated model:
|
||||
|
||||
* Outcomes assigned a probability of `0.2` should occur about 20% of the time.
|
||||
* Outcomes assigned a probability of `0.8` should occur about 80% of the time.
|
||||
* Outcomes assigned a probability of `1.0` should occur 100% of the time.
|
||||
|
||||
These rates describe groups of predictions, not a guarantee about any single answer. See [Confidence](/confidence) for guidance on deciding when software should act or escalate.
|
||||
|
||||
## The problems with RLHF
|
||||
|
||||
RLHF teaches a model to say things that people prefer. That objective works well for chatbots, but it can also reward sycophancy and confident-sounding hallucinations.
|
||||
|
||||
Preference optimization also causes **mode dropping**: the model learns to favor a particular style, such as instruction following, while reducing the probability of other possible outputs.
|
||||
|
||||
<Frame>
|
||||
<img className="block dark:hidden" src="https://mintcdn.com/ts-docs/aFVnpmCIX68NpsV1/images/ai-primer/mode-dropping-light.webp?fit=max&auto=format&n=aFVnpmCIX68NpsV1&q=85&s=d51758a6212b526fc243cc9a81572cc7" alt="The probability distribution of a base model compared with a narrowed, mode-dropped distribution after RLHF." width="2048" height="1117" data-path="images/ai-primer/mode-dropping-light.webp" />
|
||||
|
||||
<img className="hidden dark:block" src="https://mintcdn.com/ts-docs/aFVnpmCIX68NpsV1/images/ai-primer/mode-dropping-dark.webp?fit=max&auto=format&n=aFVnpmCIX68NpsV1&q=85&s=4330e6251ca335515a61f61794f21389" alt="The probability distribution of a base model compared with a narrowed, mode-dropped distribution after RLHF." width="2048" height="1117" data-path="images/ai-primer/mode-dropping-dark.webp" />
|
||||
</Frame>
|
||||
|
||||
<Warning>
|
||||
An output can be compelling to a person without being reliable enough for unattended automation. Human preference and machine trustworthiness are different optimization targets.
|
||||
</Warning>
|
||||
|
||||
Mode dropping is a milder version of **mode collapse**. In the classic generative-adversarial-network failure mode, a generator learns to produce the same kind of output repeatedly because that output continues to fool the discriminator.
|
||||
|
||||
<Accordion title="Mode collapse analogy">
|
||||
<Frame>
|
||||
<img className="block dark:hidden" src="https://mintcdn.com/ts-docs/aFVnpmCIX68NpsV1/images/ai-primer/mode-collapse-light.webp?fit=max&auto=format&n=aFVnpmCIX68NpsV1&q=85&s=2896f125ad1a5835b31b088fbc64eff1" alt="Repeated characters illustrate a GAN suffering from mode collapse." width="1084" height="759" data-path="images/ai-primer/mode-collapse-light.webp" />
|
||||
|
||||
<img className="hidden dark:block" src="https://mintcdn.com/ts-docs/aFVnpmCIX68NpsV1/images/ai-primer/mode-collapse-dark.webp?fit=max&auto=format&n=aFVnpmCIX68NpsV1&q=85&s=95645bdefd0bb3fa093edc3dd9308337" alt="Repeated characters illustrate a GAN suffering from mode collapse." width="1084" height="759" data-path="images/ai-primer/mode-collapse-dark.webp" />
|
||||
</Frame>
|
||||
</Accordion>
|
||||
|
||||
RLHF remains a good fit for conversational models. TypeSafe's position is that production automation needs a different training objective—one centered on constrained decisions and calibrated uncertainty.
|
||||
226
docs/introduction/quickstart.md
Normal file
226
docs/introduction/quickstart.md
Normal file
@@ -0,0 +1,226 @@
|
||||
> ## Documentation Index
|
||||
> Fetch the complete documentation index at: https://docs.typesafe.ai/llms.txt
|
||||
> Use this file to discover all available pages before exploring further.
|
||||
|
||||
# Quick start
|
||||
|
||||
> Prefer to just dive in? Here's everything you need to get started immediately.
|
||||
|
||||
## Try it: the Playground
|
||||
|
||||
1. **Open the [Playground](https://console.typesafe.ai/playground)** and log in.
|
||||
2. **Paste any text** as the state.
|
||||
|
||||
```plaintext title="Sample state" theme={null}
|
||||
Hi, I've been trying to connect my Stripe account for 3 days and it keeps failing. I'm losing sales. Please help ASAP.
|
||||
```
|
||||
|
||||
3. **Add a question.** Try a Noul question: `"Does this message express urgency?"`
|
||||
|
||||
```json theme={null}
|
||||
{
|
||||
"urgency": {
|
||||
"type": "noul",
|
||||
"instructions": "Does this message express urgency?"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
4. **Add more questions.** Mix Noul, Choice, and Score in one call and see all results at once.
|
||||
|
||||
## Call it: the API
|
||||
|
||||
1. **Get your API key** from the [dashboard](https://console.typesafe.ai/settings/keys)
|
||||
2. **Make a POST request** to the API endpoint
|
||||
3. **Review the [API Reference](/api)** for all the details.
|
||||
|
||||
```http theme={null}
|
||||
POST https://api.typesafe.ai/v1/systemone
|
||||
Authorization: Bearer <API_KEY>
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
### Sample cURL command
|
||||
|
||||
```bash theme={null}
|
||||
curl -X POST https://api.typesafe.ai/v1/systemone \
|
||||
-H "Authorization: Bearer $TYPESAFE_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d @- <<'EOF'
|
||||
{
|
||||
"state": "Hi, I've been trying to connect my Stripe account for 3 days and it keeps failing. I'm losing sales. Please help ASAP.",
|
||||
"model": "jev-latest",
|
||||
"questions": {
|
||||
"urgency": {
|
||||
"type": "noul",
|
||||
"instructions": "Does this message express urgency?"
|
||||
}
|
||||
}
|
||||
}
|
||||
EOF
|
||||
```
|
||||
|
||||
### Request body
|
||||
|
||||
```json theme={null}
|
||||
{
|
||||
"state": "Hi, I've been trying to connect my Stripe account for 3 days and it keeps failing. I'm losing sales. Please help ASAP.",
|
||||
"model": "jev-latest",
|
||||
"questions": {
|
||||
"department": {
|
||||
"type": "choice",
|
||||
"instructions": "Which team should handle this",
|
||||
"criteria": {
|
||||
"billing": "Payment or subscription issues",
|
||||
"technical": "Bugs or integration problems",
|
||||
"sales": "Pricing or account questions"
|
||||
}
|
||||
},
|
||||
"frustration": {
|
||||
"type": "score",
|
||||
"instructions": "How frustrated the customer appears",
|
||||
"criteria": [
|
||||
"Calm, just stating facts",
|
||||
"Frustrated but civil",
|
||||
"Very angry, strong language"
|
||||
]
|
||||
},
|
||||
"is_urgent": {
|
||||
"type": "noul",
|
||||
"instructions": "The message conveys urgency or time-sensitivity"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Response body
|
||||
|
||||
```json theme={null}
|
||||
{
|
||||
"model": "jev-latest",
|
||||
"answers": {
|
||||
"department": {
|
||||
"type": "choice",
|
||||
"choice": "billing",
|
||||
"probabilities": {
|
||||
"billing": 0.84,
|
||||
"technical": 0.159,
|
||||
"sales": 0.001
|
||||
},
|
||||
"confidence": 0.596
|
||||
},
|
||||
"frustration": {
|
||||
"type": "score",
|
||||
"score": 1.035,
|
||||
"legend": {
|
||||
"0": "Calm, just stating facts",
|
||||
"1": "Frustrated but civil",
|
||||
"2": "Very angry, strong language"
|
||||
},
|
||||
"confidence": 0.842
|
||||
},
|
||||
"is_urgent": {
|
||||
"type": "noul",
|
||||
"noul": 0.999
|
||||
}
|
||||
},
|
||||
"usage": {
|
||||
"input_tokens": 312,
|
||||
"output_tokens": 48
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
See the [API Reference](/api) for all the details.
|
||||
|
||||
## Code it: the Python SDK
|
||||
|
||||
1. **Install the SDK** (requires Python >= 3.10).
|
||||
|
||||
```bash title="With pip" theme={null}
|
||||
pip install typesafe-sdk
|
||||
```
|
||||
|
||||
```bash title="With uv" theme={null}
|
||||
uv add typesafe-sdk
|
||||
```
|
||||
|
||||
2. **Use the SDK.** The client reads `TYPESAFE_API_KEY` from the environment and calls `jev-latest` by default.
|
||||
|
||||
```python theme={null}
|
||||
from typesafe_sdk import Choice, Noul, Score, TypeSafeClient
|
||||
|
||||
client = TypeSafeClient()
|
||||
|
||||
ticket = "Hi, I've been trying to connect my Stripe account for 3 days and it keeps failing. I'm losing sales. Please help ASAP."
|
||||
|
||||
response = client.system_one(
|
||||
state=ticket,
|
||||
questions={
|
||||
"department": Choice(
|
||||
instructions="Which team should handle this",
|
||||
criteria={
|
||||
"billing": "Payment or subscription issues",
|
||||
"technical": "Bugs or integration problems",
|
||||
"sales": "Pricing or account questions",
|
||||
},
|
||||
),
|
||||
"frustration": Score(
|
||||
instructions="How frustrated the customer appears",
|
||||
criteria=[
|
||||
"Calm, just stating facts",
|
||||
"Frustrated but civil",
|
||||
"Very angry, strong language",
|
||||
],
|
||||
),
|
||||
"is_urgent": Noul(
|
||||
instructions="The message conveys urgency or time-sensitivity",
|
||||
),
|
||||
},
|
||||
)
|
||||
|
||||
print(response.answers["department"].choice) # "billing"
|
||||
print(response.answers["frustration"].score) # 1.035
|
||||
print(response.answers["is_urgent"].noul) # 0.999
|
||||
```
|
||||
|
||||
See [client SDKs](/sdk) for installation options and detailed usage.
|
||||
|
||||
## Vibe it: the agent skill
|
||||
|
||||
1. **[Install the TypeSafe skill](/agent-skill#installation)** using the Claude Code plugin or `npx skills add typesafe-ai/skills --skill typesafe-ai`. You can also [read SKILL.md on GitHub](https://github.com/typesafe-ai/skills/blob/main/skills/typesafe-ai/SKILL.md).
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Claude Code">
|
||||
Run these two commands in your terminal:
|
||||
|
||||
```bash theme={null} theme={null} theme={null} theme={null}
|
||||
claude plugin marketplace add typesafe-ai/skills
|
||||
claude plugin install typesafe@typesafe-ai
|
||||
```
|
||||
</Tab>
|
||||
|
||||
<Tab title="Other agents">
|
||||
```bash theme={null} theme={null} theme={null} theme={null}
|
||||
npx skills add typesafe-ai/skills --skill typesafe-ai
|
||||
```
|
||||
|
||||
Choose your agent when prompted. Installation is project-local by default; add `-g` to install globally.
|
||||
</Tab>
|
||||
|
||||
<Tab title="Copy to your agent">
|
||||
Paste this prompt into your coding agent:
|
||||
|
||||
```text wrap theme={null} theme={null} theme={null} theme={null}
|
||||
Install the TypeSafe skill. If you're in Claude Code, run `claude plugin marketplace add typesafe-ai/skills`, then `claude plugin install typesafe@typesafe-ai`. If you're in another agent, run `npx skills add typesafe-ai/skills --skill typesafe-ai` and select your agent. Use one installation method. You can read the skill directly at https://github.com/typesafe-ai/skills/blob/main/skills/typesafe-ai/SKILL.md (raw: https://raw.githubusercontent.com/typesafe-ai/skills/main/skills/typesafe-ai/SKILL.md). Then use the TypeSafe skill when working on this project.
|
||||
```
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
2. **Tell your coding agent** to use the TypeSafe skill as you build!
|
||||
|
||||
```plaintext title="Coding agent prompt" theme={null}
|
||||
Let's build a simple CLI that uses the TypeSafe API to evaluate a set of supplied documents on multiple dimensions. Use the TypeSafe skill to understand how to use the TypeSafe API and how to structure the system. Ask me questions about what kinds of documents I want to evaluate and on what dimensions.
|
||||
```
|
||||
|
||||
See the [Agent Skill](/agent-skill) page for more details.
|
||||
Reference in New Issue
Block a user