---
language:
- en
- zh
license: apache-2.0
library_name: transformers
pipeline_tag: image-text-to-text
tags:
- agnes-ai
- reasoning
- multimodal
- long-context
- hybrid-attention
---
# Agnes-3.0-Flash Preview
## Model version clarification
This repository contains an earlier open-weight **Preview checkpoint** of Agnes 3.0 Flash. It is distinct from the newer **production/API checkpoint** listed on [Artificial Analysis](https://artificialanalysis.ai/models/agnes-3-0-flash).
The Preview release has **33B parameters** and a context window of **262,144 tokens**. The production/API model uses a different checkpoint and configuration, with a **1M-token context window**. Its benchmark results should not be attributed to the Preview weights released here.
This repository was initially published as `Agnes-3.0-Flash` without the `Preview` suffix. The model card now explicitly identifies this release as **Agnes-3.0-Flash Preview** to clarify the distinction between the open-weight release and the production/API model.
The specifications and Agnes benchmark results below refer to the **Preview checkpoint**.
Hello! 👋 Today we are introducing **Agnes-3.0-Flash Preview**, an **open-weights multimodal preview model** built for people who want flagship-class reasoning without flagship-class hardware.
Highlights:
- **Competitive across core capabilities.** Agnes-3.0-Flash Preview posts competitive results across reasoning, coding, and instruction-following evaluations.
- **Built for demanding work.** A **262 144-token context window**, adjustable reasoning effort, tool calling, and **text, image and video** understanding.
### Benchmarks
> **Benchmark scope:** The Agnes results in the chart and table below belong to the **Agnes-3.0-Flash Preview open-weight checkpoint** released in this repository. They are not results for the production/API Agnes 3.0 Flash model listed on Artificial Analysis.
The Agnes-3.0-Flash Preview scores in the chart correspond to the open-weight checkpoint released in this repository.
Reference results across contemporary models are shown below. The figures were compiled from different sources, harnesses, and model snapshots and do not constitute a controlled head-to-head comparison.
| Benchmark |
Agnes-3.0-Flash Preview |
Qwen3.6-35B-A3B
35B / 3B active |
Kimi K2.5
1T / 32B active |
Muse Glimmer
30B |
Qwen3.5
27B |
DeepSeek V4 Flash 0731
284B / 13B active |
Qwen3.8
27B |
Gemini 3.5 Flash
undisclosed |
Qwen3.8 Flash Next
125B / 6B active |
MiniMax M3
428B / 23B active |
| IFBench | 74.20 | 64.4 | 43.7 | 77.0 | 75.6 | 75.8 | 79.5 | 76.3 | 81.3 | 82.9 |
| SciCode | 38.08 | 35.8 | 39.6 | 43.6 | 39.5 | 50.3 | 46.6 | 53.1 | 50.6 | 45.4 |
| GPQA Diamond | 85.05 | 84.1 | 78.9 | 83.5 | 85.8 | 90.8 | 90.5 | 92.2 | 92.3 | 92.9 |
| AA-LCR | 68.33 | 66.7 | 59.0 | 80.0 | 72.3 | 79.7 | 82.0 | 81.0 | 79.7 | 74.0 |
| AA-Omniscience Accuracy | 23.00 | 18.8 | 22.9 | 27.0 | 20.7 | 40.4 | 18.4 | 51.4 | 24.5 | 16.7 |
Higher is better for every row. Header parameter figures mix total and active counts, and harnesses and snapshot dates differ across sources, so treat cross-column comparisons as reference values rather than a controlled head-to-head evaluation.
## Architecture
Agnes-3.0-Flash Preview is a hybrid-attention decoder: three of every four layers run a gated delta rule (recurrent, with per-layer state independent of sequence length), and the fourth runs standard global attention. Only 18 of the 72 layers therefore hold a KV cache that grows with context.
| | |
|---|---|
| Context length | **262 144** tokens |
| Decoder layers | 72 = 54 delta-rule recurrent + 18 global attention, alternating 3 : 1 |
| Hidden size | 5120 |
| Global attention | 24 query heads / 4 KV heads (6 : 1 GQA), head dim 256; RMS-norm on q and k, sigmoid-gated output |
| Delta-rule layers | 16 key heads / 48 value heads, head dim 128; causal conv (kernel 4) in front, gated RMS-norm; recurrent state in fp32 |
| Feed-forward | SwiGLU, intermediate size 17408; plus a parallel SwiGLU 2048 branch in every layer |
| Positions | 3-axis rotary (text / height / width), interleaved mrope sections 11 : 11 : 10, base 1e7, applied to the first 25 % of each head dim (64 dims) |
| Vocabulary | 248 320 |
| Vision tower | 27 layers, hidden 1152, patch 16, 2 × 2 spatial merge, projected to 5120 |
## Quickstart
REMOTE CODE REQUIRED
Agnes-3.0-Flash Preview ships its own model implementation. Always load it with trust_remote_code=True.
### Requirements
```bash
pip install "transformers>=5.12" torch torchvision accelerate
```
Tested on transformers 5.12.1. Image and video inputs go through the bundled processor, which needs `torchvision`.
### Transformers
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
path = "Agnes-AI/Agnes-3.0-Flash"
tok = AutoTokenizer.from_pretrained(path)
model = AutoModelForCausalLM.from_pretrained(
path, dtype="bfloat16", device_map="auto", trust_remote_code=True
)
msgs = [{"role": "user", "content": "请用三句话解释什么是人工智能。"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=256)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
```
### Images and video
Image and video inputs go through the bundled processor (also remote code):
```python
from transformers import AutoProcessor
proc = AutoProcessor.from_pretrained(path, trust_remote_code=True)
msgs = [{"role": "user", "content": [{"type": "image", "image": "photo.jpg"},
{"type": "text", "text": "描述这张图。"}]}]
inputs = proc.apply_chat_template(msgs, add_generation_prompt=True, tokenize=True,
return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256)
print(proc.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0])
```
### Reasoning effort
The chat template exposes three reasoning levels — `high` (default), `medium`, `low` — plus a thinking-off switch:
```python
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt",
reasoning_effort="medium") # or enable_thinking=False
```
### Tool calling
The chat template renders tool definitions for you. The model emits calls as ``, and you feed results back as a `tool` role message:
```python
tools = [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Look up current weather for a city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string", "description": "City name"}},
"required": ["city"],
},
},
}]
msgs = [{"role": "user", "content": "What's the weather in Beijing right now?"}]
ids = tok.apply_chat_template(msgs, tools=tools, add_generation_prompt=True,
return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=256)
reply = tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True)
#
#
#
# Beijing
#
#
#
# run the tool, append the result, generate the final answer
msgs += [{"role": "assistant", "content": reply},
{"role": "tool", "content": "Clear, 26°C, light northeasterly wind"}]
```
Over the OpenAI API pass `tools=` the same way. The server returns the text above verbatim by default; to get structured `tool_calls`, configure sglang with a tool-call parser matching this format (likewise a reasoning parser, if you want the thinking span in `reasoning_content`).
### SGLang
`serve.sh` starts a server from a stock public image, overlaying three files onto the image's `sglang` package and nothing else. See `sglang_patch/README.md`.
```bash
docker run --gpus all --shm-size 64g -p 30001:8080 \
-v /path/to/agnes-3.0-flash:/model \
lmsysorg/sglang:nightly-dev-20260908-20ca564b \
bash /agnes-3.0-flash/serve.sh --served-model-name Agnes-3.0-Flash
```
`serve.sh` forwards extra command-line arguments to sglang, which is how `--served-model-name` takes effect; `--tp 2` works the same way. The server listens on port 8080 inside the container:
```python
from openai import OpenAI
client = OpenAI(api_key="EMPTY", base_url="http://localhost:30001/v1")
response = client.chat.completions.create(
model="Agnes-3.0-Flash",
messages=[{"role": "user", "content": "Design a fault-tolerant event processing architecture."}],
temperature=1.0,
max_tokens=2000,
)
print(response.choices[0].message.content)
```
Pass `stream=True` for streaming; `tools=` and `reasoning_effort=` are accepted the same way.
## Hardware Requirements
| Resource | Recommendation |
|---|---|
| GPUs | 1 × NVIDIA H200 141 GB or NVIDIA H100 80 GB (or equivalent) at bf16 |
| Tensor parallel | `--tp 1`; `--tp 2` for maximum context and concurrency |
| Weights on disk | Approximately 66 GB for the bf16 checkpoint |
| Host memory | 128 GB or more recommended |
Actual context length and concurrency depend on KV-cache allocation, runtime overhead, and tensor-parallel configuration; validate the target workload on the intended hardware.
## Recommended Inference Settings
| Setting | Recommended |
|---|---|
| `temperature` | 1.0 |
| `top_p` | 0.95 |
| `top_k` | 20 |
| `reasoning_effort` | `high` for hard reasoning, `low` for latency-sensitive traffic |
| `max_tokens` | 2000 or higher |
These are the checkpoint's own `generation_config.json` defaults.
## Model Capabilities
| Capability | Support |
|---|---|
| Advanced reasoning | Yes, with `high` / `medium` / `low` effort levels |
| Coding and debugging | Yes |
| Long-context analysis | 262 144 tokens |
| Image understanding | Yes |
| Video understanding | Yes |
| Tool calling | Yes (`` / ``) |
| Streaming | Yes |
| OpenAI-compatible APIs | Chat Completions via sglang |
## License
Released under the [Apache License 2.0](LICENSE).
## Citation
```bibtex
@misc{agnes30flash2026,
title = {Agnes-3.0-Flash Preview},
author = {{Agnes AI}},
year = {2026},
month = sep,
howpublished = {Open-weights preview checkpoint},
url = {https://agnes-ai.com/}
}
```