# sglang patch for Agnes 3.0 Flash

Serve with a stock sglang image; `serve.sh` overlays three files onto the image's
`sglang` package and starts the server.

| Variant | Source | Status |
|---|---|---|
| `nightly-dev-20260908-20ca564b/` | `lmsysorg/sglang:nightly-dev-20260908-20ca564b` | served and measured (see below) |
| `v0.5.19/` | `sglang==0.5.19` | generated from the release source, not served-tested |
| other versions | `apply_patch.py <path/to/sglang>` | patched in place by `serve.sh` when no variant matches |

```
docker run --gpus all --shm-size 64g -p 30001:30002 \
    -v /path/to/agnes-3.0-flash:/model \
    lmsysorg/sglang:nightly-dev-20260908-20ca564b \
    bash /model/serve.sh            # extra sglang args may follow, e.g. --tp 2
```

## The three files

| File | Change |
|---|---|
| `sglang/srt/configs/agnes.py` | new. Reads the `model_type: agnes` config and presents it to the server in the terms of its built-in hybrid (delta-rule + global attention) implementation: layer plan from `global_attention_interval`, the parallel FFN width added to `intermediate_size`, the checkpoint directory recorded as a config field for the loader. |
| `sglang/srt/utils/hf_transformers/common.py` | +3 lines at the end: registers `AgnesConfig` for `model_type` `agnes`. |
| `sglang/srt/models/qwen3_5.py` | one generator in front of the weight stream in `load_weights`: `delta_attn.*` → `linear_attn.*`, `global_attn.*` → `self_attn.*`, and each layer's `mlp.parallel_ffn.{gate,up,down}_proj` concatenated onto the main projections (gate/up along the output dim, down along the input dim). Checkpoints without a parallel branch pass through untouched. |

Nothing else in the image is modified. `--trust-remote-code` is required because sglang
resolves the model configuration through transformers first, which reads the
`configuration_agnes.py` shipped with the checkpoint. `serve.sh` also exports
`AGNES_MODEL_PATH` as a fallback for the loader.

## Numerics

Folding the parallel branch into the main MLP changes the reduction length of the down
projection, so served logits are not bit-identical to the transformers implementation.
Measured on the nightly image (TP1, H200, 2144 teacher-forced positions, full 248 320-way
softmax): full-vocabulary KL 5.9e-4 against the same weights served without the branch by the
unpatched engine, below the 6.5e-4 measured between the transformers and sglang
implementations of one and the same checkpoint. Perplexity 17.07 vs 17.05.

`apply_patch.py` anchors on `QWEN3_5_KV_SCALE_MAPPER` in `models/qwen3_5.py` and on the end of
`utils/hf_transformers/common.py`; it is idempotent.