README.md

---
license: apache-2.0
license_name: apache-2.0
license_link: https://www.apache.org/licenses/LICENSE-2.0
library_name: transformers
pipeline_tag: image-text-to-text
language:
  - en
  - zh
base_model: Qwen/Qwen3.8-27B
base_model_relation: finetune
tags:
  - qwen
  - qwen3.8
  - cortex
  - cortexlm
  - post-training
  - vision-language
  - conversational
  - preview
---

<p align="center">
  <img src="banner.jpg" alt="Cortex" width="100%" />
</p>

# Cortex-Mini-1-Preview

**Cortex-Mini-1-Preview** is a post-trained derivative of [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B), released by [CortexLM](https://huggingface.co/CortexLM).

This repository is a fork of the official Qwen3.8-27B checkpoint. We start from that open-weight native vision-language model and continue with Cortex post-training (Relearn). The goal is to improve the model on held-out tasks **without** collapsing general capability or overfitting public eval splits.

> Preview. Weights, training recipe, and numbers in this card will move. Treat this page as the public home for the Cortex Mini 1 line, not a frozen release.

## Thanks

This work exists because the Qwen team open-sourced [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) under Apache 2.0.

Thank you to the **Qwen Team** and **Alibaba Cloud** for the architecture, the training, the weights, and the docs. Cortex-Mini-1-Preview inherits their model; our contribution is post-training on top. If you use this checkpoint, please also cite Qwen3.8 (see [Citation](#citation)).

## What this is

| | |
|---|---|
| Base | [`Qwen/Qwen3.8-27B`](https://huggingface.co/Qwen/Qwen3.8-27B) |
| Relation | Post-training / continued training (`finetune`) |
| Type | Native vision-language causal LM (text, image, video) |
| Parameters | 27B dense |
| License | Apache License 2.0 (same as the base) |
| Org | [CortexLM](https://huggingface.co/CortexLM) |

Qwen3.8-27B already accepts image and video input. Cortex-Mini-1-Preview does **not** bolt on a separate vision encoder. Miners and researchers post-train this 27B checkpoint itself.

## License

The base model is licensed under **Apache License 2.0**, Copyright 2026 Alibaba Cloud. That license allows commercial use, modification, and redistribution, including this fork.

This repository redistributes Qwen3.8-27B weights and configuration as a derivative work:

- The Apache 2.0 license text ships with the repo (`LICENSE`).
- Attribution to the Qwen Team / Alibaba Cloud is retained (this card + `NOTICE`).
- Files we change are marked as such. The model card, banner, and Cortex post-training artifacts are new.

You must keep the license, attribution, and NOTICE if you redistribute further. "Qwen" remains a trademark of the original authors; this project does not claim that mark.

Full terms: [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0) and the `LICENSE` file in this repo.

## Model overview (inherited from Qwen3.8-27B)

These architectural facts come from the [Qwen3.8-27B model card](https://huggingface.co/Qwen/Qwen3.8-27B). Cortex post-training does not change the network shape.

- **Type:** Causal language model with vision encoder (native VLM)
- **Training stage (base):** Pre-training and post-training by Qwen
- **Training stage (this repo):** Additional Cortex post-training on the Qwen checkpoint
- **Parameters:** 27B
- **Hidden size:** 5120
- **Layers:** 64
- **Context:** 262,144 tokens natively, extensible to 1,000,000 with YaRN
- **Modalities:** Text, image, video

For serving details, sampling defaults, thinking mode, and YaRN flags, follow the Qwen card. Those recipes still apply.

## Intended use

- Research and development on Cortex Relearn (post-train a capable open VLM without wrecking general skills).
- Downstream chat, coding, agent, and vision-language workloads that already run on Qwen3.8-27B.
- Further fine-tuning under Apache 2.0.

**Out of scope**

- Do not present this preview as a drop-in Qwen Cloud / Qwen3.8-Max replacement.
- Do not treat unpublished scores as Qwen's official numbers.
- Do not strip license or attribution.

## How to use

Load the repo id once weights are in place:

```python
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "CortexLM/Cortex-Mini-1-Preview"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
```

For production serving, use vLLM, SGLang, or TokenSpeed with the **local directory** of the downloaded weights, not a guessed path. See Qwen's [vLLM recipe](https://recipes.vllm.ai/Qwen/Qwen3.8-27B) and [SGLang cookbook](https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B).

Thinking mode is on by default in the Qwen3.8 stack (`enable_thinking`, `preserve_thinking`, `reasoning_effort`). Keep those defaults unless you know you want instruct/non-thinking sampling.

## Evaluation

Preview: we do not publish a public leaderboard on this card yet. Cortex scores live training on a **private holdout**. Public splits are informational. General-capability checks run off the visible reward so they are not an overfitting target.

When a numbered release ships, this section will get frozen metrics and the eval recipe.

## Limitations

- Preview checkpoint: APIs, tokenizer extras, and chat template can still change.
- Post-training can move behavior away from the Qwen defaults (style, refusal, tool use). Compare against [`Qwen/Qwen3.8-27B`](https://huggingface.co/Qwen/Qwen3.8-27B) before you swap it in.
- Long-context and hour-scale video settings are inherited from Qwen; we have not re-tuned those knobs here.
- As with any open VLM, outputs can be wrong, biased, or unsafe. You are responsible for downstream filters.

## Files of ours vs Qwen's

| File | Source |
|---|---|
| Weights, tokenizer, `config.json`, processor configs | Forked from Qwen/Qwen3.8-27B |
| `LICENSE` | Apache 2.0 from the base (Copyright 2026 Alibaba Cloud) |
| `README.md`, `banner.jpg`, `NOTICE` | CortexLM |

## Citation

If you use Cortex-Mini-1-Preview, please cite both this work and Qwen3.8:

```bibtex
@misc{cortex-mini-1-preview,
  title  = {Cortex-Mini-1-Preview},
  author = {CortexLM},
  year   = {2026},
  url    = {https://huggingface.co/CortexLM/Cortex-Mini-1-Preview}
}

@misc{qwen38,
  title  = {{Qwen3.8-Max}: A New Bar for Coding and Cowork},
  url    = {https://qwen.ai/blog?id=qwen3.8},
  author = {{Qwen Team}},
  month  = {August},
  year   = {2026}
}
```

## Links

- Base model: [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B)
- Qwen collection: [Qwen3.8 on Hugging Face](https://huggingface.co/collections/Qwen/qwen38)
- CortexLM: [huggingface.co/CortexLM](https://huggingface.co/CortexLM)
- Cortex: [github.com/CortexLM/cortex](https://github.com/CortexLM/cortex)