# Per-layer n-gram tables (engram): mix and match

Qwen3.8-Flash-Next has a large per-layer n-gram embedding table (`per_layer_token_embd.weight`, 160 x 320,001,536 = 51.2B values). In a quant that keeps this table in its own shard, you can swap the table for a smaller or a lossless one without re-downloading the rest of the model.

| file | table type | size | relative RMSE vs BF16 | notes |
|---|---|---|---|---|
| `Qwen3.8-Flash-Next-engram-BF16.gguf_file` | BF16 | 102.4 GB | 0 | lossless, exactly as in the checkpoint |
| `Qwen3.8-Flash-Next-engram-Q8_0.gguf_file` | Q8_0 | 54.4 GB | 0.0053 | |
| `Qwen3.8-Flash-Next-engram-Q5_1.gguf_file` | Q5_1 | 38.4 GB | 0.0376 | |
| `Qwen3.8-Flash-Next-engram-IQ4_NL.gguf_file` | IQ4_NL | 28.8 GB | 0.0757 | |

Sizes are GB = 10^9 bytes. Relative RMSE is the root-mean-square error of a dequantized table row divided by the RMS of the same BF16 row, median over 3,000 random rows. The table rows are 160 values wide, and K-quants (Q5_K, Q6_K) need rows that are a multiple of 256, so this table cannot be stored as Q5_K or Q6_K. Q5_1 and Q8_0 are the types llama.cpp itself falls back to for Q5_K and Q6_K on such tensors.

The files end in `.gguf_file` so tools that scan the repo for models do not pick them up; you rename one to your quant's table shard name when swapping (step 3 below).

## Which shard is the table

| quant | table shard | slot / count | bundled table |
|---|---|---|---|
| UD-IQ1_S, UD-IQ1_M, UD-Q2_0, UD-IQ2_XS | `-00003-of-00003` | 3 3 | IQ4_NL |
| UD-Q2_K_XL, UD-IQ3_XXS, UD-Q3_K_XL, UD-IQ4_XS, UD-Q4_K_XL | `-00004-of-00004` | 4 4 | IQ4_NL |
| UD-Q5_K_XL, UD-Q6_K_XL, Q8_0 | `-00003-of-00006` | 3 6 | Q8_0 |
| BF16 | `-00003-of-00008` | 3 8 | BF16 |

## Download a quant without its bundled table

You do not need to download the table a quant ships with if you want a different one. Exclude the quant's table shard and fetch the table you want from this folder instead. Example: UD-Q4_K_XL with the Q8_0 table (saves the 28.8 GB IQ4_NL shard):

```bash
# 1. UD-Q4_K_XL without its table shard (-00004-of-00004)
hf download unsloth/Qwen3.8-Flash-Next-GGUF \
  --include "UD-Q4_K_XL/*" --exclude "UD-Q4_K_XL/*-00004-of-00004.gguf" --local-dir .

# 2. the Q8_0 table + the script
hf download unsloth/Qwen3.8-Flash-Next-GGUF \
  --include "engram/*Q8_0*" "engram/set_split.py" --local-dir .

# 3. stamp it as shard 4 of 4 and put it in the table shard's place
python engram/set_split.py engram/Qwen3.8-Flash-Next-engram-Q8_0.gguf_file 4 4
mv engram/Qwen3.8-Flash-Next-engram-Q8_0.gguf_file \
   UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00004-of-00004.gguf

# 4. run as usual
llama-server -m UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf
```

For another quant, use its table shard and slot / count from the table above.

## How to swap a table you already downloaded (example: UD-Q6_K_XL with the lossless BF16 table)

llama.cpp checks that each shard's `split.no` / `split.count` match its place in the set, so the table file has to be stamped for your quant first. `set_split.py` (in this folder, plain Python, no dependencies) changes those two header values in place in milliseconds; nothing else in the file changes.

```bash
# 1. get the table + the script
hf download unsloth/Qwen3.8-Flash-Next-GGUF \
  --include "engram/*BF16*" "engram/set_split.py" --local-dir .

# 2. stamp it as shard 3 of 6 (UD-Q6_K_XL's table slot)
python engram/set_split.py engram/Qwen3.8-Flash-Next-engram-BF16.gguf_file 3 6

# 3. keep the old table aside, then rename the new one to the table shard's name
mv UD-Q6_K_XL/Qwen3.8-Flash-Next-UD-Q6_K_XL-00003-of-00006.gguf UD-Q6_K_XL/engram-Q8_0.bak
mv engram/Qwen3.8-Flash-Next-engram-BF16.gguf_file \
   UD-Q6_K_XL/Qwen3.8-Flash-Next-UD-Q6_K_XL-00003-of-00006.gguf

# 4. run as usual
llama-server -m UD-Q6_K_XL/Qwen3.8-Flash-Next-UD-Q6_K_XL-00001-of-00006.gguf
```

For another quant only the two numbers and the file name change (table above). The files here are stamped as shard 1 of 1, so using one without `set_split.py` fails to load with `invalid split file idx` instead of loading the wrong thing.