Instructions to use CortexLM/Teutonic-1-Chat-Preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use CortexLM/Teutonic-1-Chat-Preview with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="CortexLM/Teutonic-1-Chat-Preview")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("CortexLM/Teutonic-1-Chat-Preview")
model = AutoModelForCausalLM.from_pretrained("CortexLM/Teutonic-1-Chat-Preview", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use CortexLM/Teutonic-1-Chat-Preview with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "CortexLM/Teutonic-1-Chat-Preview"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CortexLM/Teutonic-1-Chat-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/CortexLM/Teutonic-1-Chat-Preview

SGLang

How to use CortexLM/Teutonic-1-Chat-Preview with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "CortexLM/Teutonic-1-Chat-Preview" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CortexLM/Teutonic-1-Chat-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "CortexLM/Teutonic-1-Chat-Preview" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CortexLM/Teutonic-1-Chat-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use CortexLM/Teutonic-1-Chat-Preview with Docker Model Runner:
```
docker model run hf.co/CortexLM/Teutonic-1-Chat-Preview
```

Correct Qwen3.5-9B harness benchmarks (thinking budget fix)

by XXMiner - opened 4 days ago

base: refs/heads/main

←

from: refs/pr/2

Discussion Files changed

+46

-48

Files changed (2) hide show

README.md +11 -10
benchmarks.md +35 -38

README.md CHANGED Viewed

@@ -174,17 +174,18 @@ This model continues the **Qwen3.5** text architecture lineage (hybrid Teutonic
 }
 ```
 ## Benchmarks
-See [`benchmarks.md`](./benchmarks.md) for the full same-harness comparison vs Qwen3.5-9B (chat + thinking on, 2026-09-01).
-| Benchmark | Teutonic | Qwen3.5-9B (same harness) | Δ |
 |---|---:|---:|---:|
-| MMLU-Pro | 63.7 | 21.3 | +42.4 |
-| MATH-500 | 62.4 | 20.2 | +42.2 |
-| AIME24 / 25 / 26 | 52.1 / 41.7 / 45.4 | 12.1 / 5.8 / 5.4 | +40 / +36 / +40 |
-| private_math | 72.0 | 26.0 | +46.0 |
-| HumanEval+ | 50.0 | 27.4 | +22.6 |
-| RULER (4k–64k) | 89.5 | 88.4 | +1.1 |
-Protocol: chat · thinking on · seed 20260830. Vendor card numbers are separate; see benchmarks.md.

 }
 ```
 ## Benchmarks
+See [`benchmarks.md`](./benchmarks.md) for the **corrected** same-harness comparison vs Qwen3.5-9B (chat + thinking on, fixed 2026-09-01). The first published Qwen column (MMLU-Pro ~21 / MATH ~20) was an **invalid harness artifact** (thinking ON with 1k–8k caps → mid-think truncation).
+| Benchmark | Teutonic | Qwen3.5-9B (fixed harness) | Δ |
 |---|---:|---:|---:|
+| MMLU-Pro (n=2000) | 69.8 | **73.2** | −3.4 |
+| MATH-500 | **88.6** | 68.2 | **+20.4** |
+| HumanEval+ | **82.9** | 81.1 | **+1.8** |
+| MBPP+ | ≈66.4 | **64.6** | ≈+1.8 |
+| IFEval | **68.6** | 65.4 | **+3.2** |
+| RULER (4k–64k) | **89.5** | 88.4 | **+1.1** |
+Protocol: chat · thinking on · raised think budgets · seed 20260830. Vendor card numbers are separate; see benchmarks.md.

benchmarks.md CHANGED Viewed

@@ -1,59 +1,56 @@
-# Benchmarks (Teutonic-1-Chat-Preview)
-Evaluated with harness `teutonic-eval-v1.2.0` (2026-09-01).
-**Protocol:** chat template · thinking **on** · seed `20260830` · AIME = avg@8 (T=0.6).
-**Honesty:** GSM8K not headlined (inherited contamination). Prefer private_math, AIME, LiveCodeBench, IFEval, RULER.
-Head-to-head vs `Qwen/Qwen3.5-9B` on the **same harness** (Qwen served with `--language-model-only`). Vendor **card** column is Qwen’s published stack — not comparable to the harness columns.
-## Same-harness deltas (Teutonic − Qwen3.5-9B)
-| Benchmark | Teutonic | Qwen3.5-9B (harness) | Δ |
 |---|---:|---:|---:|
-| MMLU-Pro | **63.7** | 21.3 | **+42.4** |
-| MATH-500 | **62.4** | 20.2 | **+42.2** |
-| AIME 2024 / 2025 / 2026 | **52.1 / 41.7 / 45.4** | 12.1 / 5.8 / 5.4 | **+40 / +36 / +40** |
-| private_math | **72.0** | 26.0 | **+46.0** |
-| HumanEval+ | **50.0** | 27.4 | **+22.6** |
-| MBPP+ | **66.4** | 8.2 | **+58.2** |
-| IFEval | **33.6** | 26.3 | **+7.4** |
 | RULER (4k–64k) | **89.5** | 88.4 | **+1.1** |
-| LiveCodeBench | 11.4 | **12.3** | −0.9 |
 | Needle | 99.5 | **100.0** | −0.5 |
 ## Knowledge & STEM
-| Benchmark | Teutonic | Qwen3.5-9B (harness) | Qwen3.5-9B (card) |
 |---|---:|---:|---:|
-| MMLU-Pro | 63.7 | 21.3 | 82.5 |
-| MMLU | 56.9 | 23.1 | — |
-| GPQA-Diamond† | 20.7 | 7.1 | 81.7 |
-† Letter-argmax on public mirror — not the official gated protocol.
 ## Math & code
-| Benchmark | Teutonic | Qwen3.5-9B (harness) | Qwen3.5-9B (card) |
-|---|---:|---:|---:|
-| MATH-500 | 62.4 | 20.2 | — |
-| AIME24 / 25 / 26 | 52.1 / 41.7 / 45.4 | 12.1 / 5.8 / 5.4 | — |
-| HMMT Feb 2026 | 15.2 | 0.0 | — |
-| LiveCodeBench | 11.4 | 12.3 | 65.6 (v6) |
-| HumanEval+ | 50.0 | 27.4 | — |
-| MBPP+ | 66.4 | 8.2 | — |
-| private_math | 72.0 | 26.0 | n/a |
 ## Long context
-| Benchmark | Teutonic | Qwen3.5-9B (harness) | Qwen3.5-9B (card) |
-|---|---:|---:|---:|
-| RULER (4k–64k) | 89.5 | 88.4 | — |
-| Needle | 99.5 | 100.0 | — |
-| AA-LCR | — | — | 63.0 |
-| LongBench v2 | — | — | 55.2 |
 ## Vision
 Teutonic is **text-only** — VL benchmarks are `N/A`. Qwen3.5-9B card (not comparable): MMMU 78.4 · MMMU-Pro 70.1 · MathVista 85.7.
-Full write-up: see training repo `reports/bench-vs-qwen35-9b.md`.

+# Benchmarks (Teutonic-1-Chat-Preview) — **corrected** 2026-09-01
+Evaluated with harness `teutonic-eval-v1.2.0` **after** the thinking-budget / chat-template fix.
+**Protocol:** chat · thinking **on** · seed `20260830` · MATH-500 max_new=16k · HE+/MBPP+ max_new=8k · lm-eval `enable_thinking` + strip `</think>` · Qwen uses native `apply_chat_template`.
+**Honesty:** Prior HF Qwen column (MMLU-Pro 21 / MATH 20 / MBPP 8) was an **invalid harness artifact** — do not cite. Details: training-repo `reports/bench-harness-bug.md`.
+Head-to-head vs `Qwen/Qwen3.5-9B` on the **same fixed harness**. Vendor **card** = Qwen’s published stack (not identical protocol).
+## Same-harness deltas (Teutonic − Qwen3.5-9B) — corrected
+| Benchmark | Teutonic | Qwen3.5-9B (fixed) | Δ |
 |---|---:|---:|---:|
+| MMLU-Pro (n=2000) | 69.8 | **73.2** | −3.4 |
+| MATH-500 | **88.6** | 68.2 | **+20.4** |
+| HumanEval+ | **82.9** | 81.1 | **+1.8** |
+| MBPP+ | **≈66.4†** | **64.6** | **≈+1.8** |
+| IFEval (prompt strict) | **68.6** | 65.4 | **+3.2** |
 | RULER (4k–64k) | **89.5** | 88.4 | **+1.1** |
 | Needle | 99.5 | **100.0** | −0.5 |
+† Teutonic MBPP+ from the pre-fix run was already usable (not chance-level); Qwen MBPP+ **8.2→64.6** under the same fix. Teutonic MBPP re-score under fixed budgets may land slightly higher.
+### Invalid prior Qwen column (do not use)
+| Benchmark | Broken harness | Fixed |
+|---|---:|---:|
+| MMLU-Pro | 21.3 | **73.2** |
+| MATH-500 | 20.2 | **68.2** |
+| HumanEval+ | 27.4 | **81.1** |
+| MBPP+ | 8.2 | **64.6** |
+| IFEval | 26.3 | **65.4** |
 ## Knowledge & STEM
+| Benchmark | Teutonic | Qwen3.5-9B (fixed harness) | Qwen3.5-9B (card) |
 |---|---:|---:|---:|
+| MMLU-Pro | 69.8 | **73.2** | 82.5 |
 ## Math & code
+| Benchmark | Teutonic | Qwen3.5-9B (fixed) |
+|---|---:|---:|
+| MATH-500 | **88.6** | 68.2 |
+| HumanEval+ | **82.9** | 81.1 |
+| MBPP+ | ≈66.4† | **64.6** |
 ## Long context
+| Benchmark | Teutonic | Qwen3.5-9B |
+|---|---:|---:|
+| RULER (4k–64k) | **89.5** | 88.4 |
+| Needle | 99.5 | **100.0** |
 ## Vision
 Teutonic is **text-only** — VL benchmarks are `N/A`. Qwen3.5-9B card (not comparable): MMMU 78.4 · MMMU-Pro 70.1 · MathVista 85.7.