Harshith Kantamneni
The ledger

Every number and its source

What each number on this site means, the conditions it holds under, and where to check it. Where a source isn't public, it says so.

M.S. ECE finished December 2025

Dec 2025

UW-Madison, Professional option

Not public: Degree record and resume

First upstream kernel work

since March 2026

First commit on vLLM #38338 is dated 2026-03-27

vllm #38338

Date PR states were last read

Nov 3, 2026

Checked by hand against each PR page. #1412 and #1319 also confirmed in the repo's git history

TransformerEngine #3591

One decode step over 64K cached keys

4.3 ms, 64K keys, at batch 1

H200, B=1, one query, HQ=32, H=8, D=128, median from do_bench. Op level, not a model-level latency

flash-linear-attention #1412

Grid size against SM count

32 programs on 132 SMs

B x Lq x HQ = 1 x 1 x 32 on H200. "32 of 132 busy" is derived, not counted

flash-linear-attention #1412

Time to read K/V once at rated bandwidth

about 0.06 ms

Not measured. 2 x 8 x 65536 x 128 x 2 bytes = 268 MB at the H200 rated 4.8 TB/s. A floor, not a target

flash-linear-attention #1412

Shared memory per tile config

64×32 tiles, 57-89 KiB, 64×64, 129-193 KiB

D=128, 2 to 4 pipeline stages, per config

flash-linear-attention #1319

Occupancy of the backward kernels

about 12% occupancy

Nsight Compute at B=2, T=4096, H=16, D=128, base clocks. 255 registers per thread, 12.5% theoretical

flash-linear-attention #1319

Time against the original authors

1.04-1.42×

Forward plus backward. H100 SXM, same session, 8 shapes, against the faster of dense and varlen at 7619a3c. Higher means slower. Forward alone is 1.07-1.54x

flash-linear-attention #1319

Planted bugs the new tests catch and the SM80 suite misses

3 injected bugs, 3 of 3, all 411, all 60, 3 bugs, A 4th planted bug, 3 planted bugs

A100, one edit at a time then restored. A 4th bug was a control the old suite caught

cudnn-frontend #1287

Close date

Oct 7, 2026

UTC. Closed by the author, not merged

vllm #38338

Throughput held on one shape

1,014-1,076 TFLOP/s, batch 8192 up

RTX 5090, down shape 4096x14336, M=8192 to 49152, autotuned tiles, hot L2, about 2 hours apart with 0.988-1.046x drift. M=1024 and 2048 run 946 and 889

vllm #38338

First revision at the largest batch

fell to 418

Same shape, M=49152, hot L2, from 852.4 at M=8192

vllm #38338

Ratio against CUTLASS, Llama QKV

1.38-3.04× CUTLASS up to batch 128, 0.77-0.91× above

RTX 5090, N=6144, K=4096, GEMM only, hot L2. Cold L2 at M=1 to 128 is 1.33-1.79x

vllm #38338

Ratio against CUTLASS on B200

about 0.27× across 4 Llama shapes on B200 from batch 1024

Geomean over four Llama-3.1-8B shapes, hot L2, from M=1024 up. Rows at lower M range 0.19-0.33. Not QKV alone, and no profile exists

vllm #38338

K/V copy time against the attention kernel

64/8 heads, 128K KV cache, 7.28 ms, 6.0-6.7 ms, 64 query heads and 8 KV heads, 128K cache

A100, session B, d128, S_q=4, S_kv=131072, b=1. Develop kernel 6.01, PR kernel 6.66. Develop run was bimodal

cudnn-frontend #1223

Batched speedup from removing copies

6 batched rows 2.6-3.3×, 2.6-3.3×

A100, S_q 2 and 8, S_kv=128K, b=32 and 128, one pass per side, 3 configs

cudnn-frontend #1223

Rows that stopped running out of memory

10 more

Develop needs a 64 to 512 GiB expanded K/V workspace. 10 of 16 batched rows

cudnn-frontend #1223

Multi-token rows where the cuDNN backend stayed faster

27 of 31

Same table, PR ms against backend ms

cudnn-frontend #1223

Comment date

Sep 23, 2026

UTC. A comment on an RFC someone else wrote

vllm #46295

Per-workload oracle against the best static table

at most 1.8%, one H200

Qwen3-30B-A3B, one H200, CPU ngram drafts, 2 synthetic workloads, offline batches, 3 repeats. 0.00 / 1.81 / 1.16 / 1.18% at B=1/4/16/64. Prices identifying the workload only. No controller was built, so per-step control is unmeasured

vllm #46295

MFU byte model overcount of expert reads

up to 3×

Worst cell: sonnet, K=0, B=16. About 2x pooled. 1.00, 1.45, 3.05, 2.10x across B=1/4/16/64

vllm #46295

Accuracy at the smaller memory

bert .96, grep .92, full-context .90

98,519-token corpus. Claude Sonnet reader via claude -p, 200K window, 1 run per question, 3-judge majority, synthetic corpus, 50 questions

bert: benchmarks/M1_REPORT.md

Accuracy at the larger memory

bert and grep .90, full-context .08, 0.90bert, both scored 0.90, 0.90

1,261,774-token corpus. bert 45/50, grep 44/49. Paired difference +0.020, CI [-0.08, +0.12], Holm p 1.0. Claude Sonnet reader via claude -p, 200K window, 1 run per question, 3-judge majority, synthetic corpus, 50 questions

bert: benchmarks/M1_REPORT.md

Full context at the larger memory

fell to 0.08

Reader capped at a 120K budget, so it keeps only about the newest 10%. A 1M-context reader would hold about 80% and would not collapse

bert: benchmarks/M1_REPORT.md

Grep tokens over bert tokens per question

median 1.46×

Paired per-question ratio, cache reads counted in full. Means 282K (n=49) and 149K (n=50), or 118K on the shared 49. Grep higher on 86%

bert: README.md

Same 20-question eval before and after the retriever fix

0.10 → 0.85, same 20-question eval from 0.10 to 0.85, same 20-question test from 0.10 to 0.85

n=20, 131K-token httpx and starlette corpus, llama-3.3-70b reader, non-Claude grader. The same questions exposed and rescored the bug. Directional, no significance test

bert: benchmarks/results/B9_RAG_RESULT.md

Snippet cap in the buggy path

240-character

core/retrieval.py before the fix

bert: README.md

Fused against unfused on the best shape

1.73×, M=128, N=11008, K=4096, 1.00×

fp16, fused 76.9 us, baseline 133.2 us. Best of 152 cells. A100-SXM4-80GB. Baseline is cuBLAS GEMM plus separate bias and GeLU, PyTorch eager

triton-vs-cublas-llm-benchmarks: README.md

Median fused against unfused

0.96×, Median 0.96×, Median across 76 shapes is 0.96×

fp16, 76 shapes. A100-SXM4-80GB. Baseline is cuBLAS GEMM plus separate bias and GeLU, PyTorch eager

triton-vs-cublas-llm-benchmarks: README.md

gpt-oss-120b decode speed

58-153 tokens/s, 11-40%

RTX 5090, mean speed, 30 problems, 256 tokens, batch 1. 153 is the faster-memory host. Stock clock is 58-134

moe-speed-of-light: paper/tab_grid.tex

Cache speed as a share of the machine limit

25 consumer machines, 31-54%

gpt-oss C=14. Limit from Belady's MIN and probed read rates, so relative to the probe, not physical. Consumer scope set after the data. 3 server machines reach 12-46%. The machine list drops 5 unstable rentals, all server processors, so the drop doesn't touch the consumer range

moe-speed-of-light: paper/wsg_decomp.tex

Configurations faster than FreeToken

11 of 12

2 RTX 5090 hosts, 6 cells each, 30 problems per cell. The one slower cell is Qwen3 43.75% on host S at 0.97x

moe-speed-of-light: paper/tab_grid.tex

Authors

Team of 4

Ivaturi, Nagendra, Kantamneni, Avvaru

Not public: ECE 759 course report byline

Authors

Team of 3

Lyu, Kantamneni, Ryu. Kantamneni built the gem5 GPUFS environment and HIP benchmark suite. Parameter tuning was a teammate's

Not public: ECE 752 final report, contributions section. Not public

Dense-vs-MoE design

34.7M, 62.99M, 5000 steps, 10.24M tokens, 12 models

One M3 Pro 18 GB laptop, 2.56M tokens per expert at k=2, K=8

Not public: Lab program files, not public

Largest per-pair difference, and the prior on the verdict

0.6314pp, prior of 0.17

18-cell evaluable matrix, against a 4pp decision threshold. O4 pre-registered with prior 0.17

Not public: Lab program files, not public

Timing error of model v3 on one test tile, with predictions frozen before the rental

32×64×64, 0.51-0.69%, 3 models, 11.76%

One rented GH200 per rental, bf16, 700 W limit, 1710 MHz lock, skewed routing, R3's 32x64x64 test tile. Rentals 6 and 7. This is not one of vLLM's own tiles. Byte predictions on OLMoE w1 failed their open test, 11.76% rms, scored apart

moe-kernels: scripts/scoring/rental7/skew.score.txt

Decode blind test, pooled rms, against the bar registered then

11.4%, 5 models unseen in decode, 1.5%

One rented GH200 per rental, bf16, 700 W, 220 cells. Predicted too slow on average, mean +6.0%, worst 39.2%. Per model: DeepSeek-V2-Lite 2.35%, Qwen1.5 5.3%, Phi-3.5 9.5%, Granite-3B 13.4%, JetMoE 19.5%. No model certified. Phi-3.5 was also in the v3 test, so unseen means unseen in decode

moe-kernels: scripts/scoring/decode/blind.score.txt

Multi-token steps with K/V copies removed, against develop

1.15-2.19×

A100, bf16, batch 1, S_q=2, S_kv=128K, 5 shapes. The time saved follows how much the copy writes: 2.19x at 64/8 d128, 1.15x at 8/1 d128, where the expansion is 8x smaller

cudnn-frontend #1223

Weakest shape against CUTLASS

0.53-0.57×

RTX 5090, GEMM only, gate_up, large M, with hot L2

vllm #38338

Per-model error behind the timing model chart

0.51%, 0.69%, 0.54%, 2%, 2.35%, 5.3%, 9.5%, 13.4%, 19.5%, Mixtral-8x7B, Phi-3.5-MoE

One rented GH200 per rental, bf16, 700 W. v3 at 1710 MHz on the 32x64x64 test tile, bar rms at most 2%. Decode blind test, bar rms at most 1.5%

moe-kernels: scripts/scoring/decode/blind.score.txt

KV-cache decode time per token by context length

398.1, 63.1, 63.9, 64.9, 114.6, 209.4, 256, 512, 1024, 2048, 4096, 8192

Jetson Orin Nano, 15 W mode, B=1, H=8, D=64, fp16, single stream. A sweep of kernels, not serving throughput, and no counter data in the repo

edge-llm-gpu-profiling: results/raw_csv/decode_kv_sweep.csv

Accuracy of every memory arm at both measured sizes, with 95% intervals

98,519 tokens, 1.26M tokens, 95% interval, 50 questions, 0.96, 0.92, 0.90, 0.50, 0.10, 0.08, 0.40, 0.04, [0.90, 1.00], [0.84, 0.98], [0.80, 0.98], [0.36, 0.64], [0.02, 0.20], [0.82, 0.98], [0.02, 0.16], [0.26, 0.54], [0.00, 0.10]

Synthetic corpus, 98,519 and 1,261,774 tokens. Claude Sonnet reader through Claude Code (claude -p), 200K window, 1 run per question, 50 questions (grep 49 at the larger size), 3 Llama judges by majority, none of them Claude. Percentile bootstrap intervals. Naive vector is one-shot cosine top-10, while bert and grep are multi-turn, so that comparison is confounded

bert: benchmarks/M1_REPORT.md, bert: benchmarks/M1_REPORT.md, bert: benchmarks/M1_REPORT.md, bert: benchmarks/M1_REPORT.md, bert: benchmarks/M1_REPORT.md, bert: benchmarks/m1_arms.py

Paired comparison of bert and grep at the larger size

+0.020, -0.08 to +0.12, 7 discordant questions, 44/49, 45/50

1,261,774-token synthetic corpus. Exact paired McNemar, Holm corrected, p 1.000. 41 both right, 4 bert only, 3 grep only, 1 neither. n=49, because the grep row for one question is missing

bert: benchmarks/M1_REPORT.md, bert: README.md

Why full context fell at the larger size

120K budget, about the newest 10%, 1M-context reader, about 80%

The reader window is 200K. The harness keeps newest sessions up to 120,000 tokens (chars / 4), since 180K tripped "Prompt is too long". 120,000 of 1,261,774 is 9.5%. The 1M-context figure is the report's arithmetic, 1M of 1.26M. No 1M-context reader was run on this corpus

bert: benchmarks/m1_arms.py, bert: benchmarks/M1_REPORT.md, bert: benchmarks/M1_REPORT.md

Accuracy of every code arm, with 95% intervals

249 questions, 228 are graded by exact value, 21 by the same judges, 95% interval, 0.968, 0.855, 0.663, 0.643, 0.610, 0.510, 0.177, 0.120, [0.944, 0.988], [0.811, 0.900], [0.606, 0.719], [0.582, 0.703], [0.550, 0.671], [0.450, 0.570], [0.133, 0.225], [0.129, 0.225], [0.080, 0.161]

Real open-source repos, 249 frozen code-fact questions. Claude Sonnet reader through Claude Code. 228 graded programmatically against AST-derived values, 21 multi-hop questions by the 3-judge majority. Percentile bootstrap intervals

bert: benchmarks/results/v2/v3_stats.json, bert: benchmarks/results/v2/v3_stats.json, bert: benchmarks/RETRIEVAL_BENCHMARK.md

Quality change from orchestration, with the model held constant

+0.007, on a 0 to 1 scale, n=11, 1.3-5.7×, 1.4-4.5×

Opus in every arm, graded by non-Claude judges. Gain is full orchestration minus bare Opus with the same rubric block. Token and wall-clock ratios are against bare Opus, counted cache-inclusive on both sides. Small n

bert: benchmarks/BENCHMARK_SYNTHESIS.md, bert: README.md

nDCG@10 and mean latency on BEIR scifact, by method

95% interval, 0.745, 0.740, 0.719, 0.658, [0.707, 0.783], [0.696, 0.779], [0.676, 0.759], [0.614, 0.703], 3376 ms, 17 ms, 23 ms, 8 ms, 18 GB

300 queries, bootstrap 95% intervals, M3 Pro with 18 GB, MPS. BM25 is the repo's own implementation. Latency is on that laptop only

bert: benchmarks/results/B2_BEIR_MULTI_RESULT.md, bert: benchmarks/results/B2_BEIR_MULTI_RESULT.md, bert: benchmarks/results/B2_BEIR_MULTI_RESULT.md, bert: benchmarks/results/B2_BEIR_MULTI_RESULT.md

The one question on a changed fact

scored 0.00 on every arm

knowledge_update category at the larger size, n=1. Under-sampled, so this shows nothing either way

bert: benchmarks/M1_REPORT.md

Distinct configs behind the batched row counts

3 configs, 5 configs, 6 batched rows, 10 out-of-memory rows

A config is the head count, d and batch size. A100, S_kv=128K. Each config ran at S_q 2 and 8, which stay within 1% of each other, so each counts as 2 rows

cudnn-frontend #1223

How each timing was taken, and what same-code rows showed

median of 20 samples, median ratio of 1.000

One rented A100-SXM4-80GB, torch 2.9.1, cuDNN 9.26, bf16. torch.profiler device time after 5 warm-ups, with a 256 MiB L2 flush before each sample. Develop and the prototype alternated in one pod. SM clocks not locked. The 1.000 is PR over develop on 88 backend rows and 18 FROST rows that run identical code on both sides

cudnn-frontend #1223

Multi-token steps at batch 1: develop, prototype and backend milliseconds, and the gain

2.19, 1.95, 1.49, 1.37, 1.15, 14.57, 8.23, 9.91, 17.15, 7.59, 6.66, 4.23, 12.55, 6.63, 3.30, 4.09, 9.96, 64/8 d128, 64/8 d64, 32/8 d128, 32/2 d256, 8/1 d128, at batch 1, At batch 1 with a 2-token query (S_q=2), all 5 shapes

A100, bf16, batch 1, S_kv=128K. The table is S_q=2, 5 shapes. All 15 rows across S_q 2, 4 and 8 fall in 1.15-2.19x. Mean of 2 pass medians per side, backend one pass, clocks not locked. 64/8 and 8/1 have the same head ratio, so the ratio alone does not set the gain

cudnn-frontend #1223

Batched multi-token rows: develop, prototype and backend milliseconds

10 of the 16 batched rows, 64-512 GiB, all 10, The other 4, 64/8 d64 at batch 32 and 128, at S_q=8, 64/8 d64, batch 32, 8/1 d128, batch 128, 8/1 d128, batch 32, 64/8 d64, batch 128, 64/8 d128, batch 32, 32/8 d128, batch 32, 32/2 d256, batch 32, 32/2 d256, batch 128, 155.9, 160.3, 42.4, 47.1, 54.3, 16.3, 179.0, 103.8, 54.5, 102.8, 391.2, 3.31×, 2.95×, 2.60×, 53.5, 21.7, 8.5, 212.7, 43.0, 22.1, 93.7, 367.8

A100, bf16, S_kv=128K, batch 32 and 128, S_q 2 and 8, one pass per side, clocks not locked. The table is S_q=8. Develop needs a 64 to 512 GiB expanded K/V workspace on the 10 rows. The prototype is faster than the backend only at 64/8 d64

cudnn-frontend #1223

Size of the K/V expansion the prototype removed

4 GiB per call at 64/8 heads, 8x smaller at 8/1 heads

d128, batch 1, S_kv=131072, 2 bytes per element. 2 x 131072 x 64 x 128 x 2 bytes = 4 GiB, and 0.5 GiB at 8/1. Computed, not counted. The PR states the 4 GiB as the workspace drop

cudnn-frontend #1223

What the merged tests add, and the ratios NVIDIA tested before

80 tests, 66 cases, 4/2 and 16/4 heads, 8/4 and 8/2

fp16 at d128 and d256 for the 66 bitwise cases, on all three schedulers. 2 more cover a short last block and 12 cover 8/4, 8/2 and MQA. Passing on A100, develop 4c0dc9a8, torch 2.8.0, cuDNN 9.26

cudnn-frontend #1287

New tests against NVIDIA SM80 tests, per planted bug

all 411 cases, all 60 d256 cases, The first 3 bugs, 4 of 80, 70 of 80, 37 of 80, All 411 pass, All 60 d256 cases pass, Caught, 30 fail, LPT_L2

A100-SXM4-80GB, develop 4c0dc9a8 plus the patch, torch 2.8.0, cuDNN 9.26. One kernel edit at a time, then restored. The third bug is d256 only, so it ran against 60 d256 cases, not all 411. The fourth is a control

cudnn-frontend #1287

The one shape behind the 7.28 ms

64/8 d128 at S_q=4

A100, batch 1, S_kv=131072, the harness kernel breakdown. Develop samples at this shape ranged 11.96 to 14.57 ms in one run

cudnn-frontend #1223

Pass-to-pass spread on some GQA rows

up to 20%

A100, clocks not locked. Larger than those rows' control bands. Every row outside its band stays outside on either pass alone. Batched rows ran once per side, so they have no spread figure

cudnn-frontend #1223

Base K/V size, and why large batches are not servable

0.5 GiB per layer, batch 32

64/8, d128, 8 KV heads, S_kv=131072, 2 bytes per element, batch 1. 2 x 131072 x 8 x 128 x 2 bytes = 0.5 GiB, so 16 GiB per layer at batch 32 on an 80 GB card. Computed, not measured

cudnn-frontend #1223

Session logs in the audit, how many contain the gate's precondition, and how many show the detector run by the director instead of the gate

763, 270, 6 sessions, 763 session logs

By my count: string counts over 763 private runner logs, named Apr 17 to May 30, 2026. Eligible means the log contains GRACEFUL_CHECKPOINT, a text match that isn't checked against true clean exits. The 6 sessions are a regex match on a shell tool call, column detector_run_by_hand. The public CSV has one row per log with size and SHA-256 and no log text, so its sums can be checked and its rows can't be checked without the logs

agi-lab-framework: audits/gate_audit_sessions.csv, agi-lab-framework: audits/README.md

Audit counts for the detector, the evaluator check and the remediation of the incident

23 of 23, 6 of 10, 32 of 174

By my count over the private archive. Detector: full archive run, false positives in three classes. Evaluator: of 14 reports recording a pass, 10 quote the detector and 6 of those quote a scan of 0 signatures. Receipts: 31 dated before their role's first dispatch by type, and 1 role never dispatched by type. This is a date test, not proof for each receipt

agi-lab-framework: audits/README.md, agi-lab-framework: audits/README.md, agi-lab-framework: audits/README.md

Dispatch receipts tracked in git

0 of 195

By my count over the lab's private tree. The receipts were never added to git, rather than being gitignored. Session logs are gitignored

agi-lab-framework: audits/README.md

Forward plus backward, in ms, against the PR's earlier head and the original authors' kernel

8 shapes, 7619a3c, B=8, T=1024, H=16, D=128, B=4, T=2048, H=16, D=128, B=2, T=4096, H=16, D=128, B=1, T=8192, H=16, D=128, B=4, T=2048, H=32, D=64, Packed 512+2048+512+5120, H=16, D=128, Packed 8×1024, H=32, D=64, Packed 8192, H=16, D=128, 2.795, 1.942, 1.870, 5.157, 3.497, 3.267, 9.986, 6.752, 5.841, 19.582, 13.194, 10.975, 6.283, 4.859, 3.421, 9.363, 6.532, 6.083, 3.460, 2.651, 2.013, 18.827, 13.231, 11.146, 1.04, 1.07, 1.16, 1.19, 1.20, 1.32, 1.42

H100 SXM, torch 2.8.0, Triton 3.4.0, bf16, HQ=H, K=V=D, attend_current=False, median ms from triton.testing.do_bench. Earlier head is 5f378d6, timed in an earlier session on the same GPU type. Authors are shawntan/stickbreaking-attention at 7619a3c on the same H100, the faster of their dense and varlen kernels on dense inputs, varlen on packed. The last column is this PR over the authors, so higher means slower

flash-linear-attention #1319, flash-linear-attention #1319

Forward plus backward against the PR's own earlier head

1.29-1.48×

H100 SXM, 8 shapes, this PR 4c539a5 against earlier head 5f378d6. The two were timed in different sessions on the same GPU type, where parallel_attn matched within 1% on forward plus backward and within 6% on forward. Mostly the backward rewrite

flash-linear-attention #1319

What each change gave on H100: backward rewrite, then retile and launch order

1.27-1.34×, 1.02-1.12×, 1.11-1.82×

H100 SXM, 8 shapes. 1.27-1.34x is the rewrite alone on forward plus backward, against the earlier head, different sessions. 1.02-1.12x and 1.11-1.82x are retile plus launch order on forward plus backward and on forward alone, 617966b against 4c539a5 in one process

flash-linear-attention #1319, Not public: Per-shape split from a bench file that is not public

Forward L2 hit rate before and after the retile and launch order

92%, 78%

H100 SXM, Nsight Compute at base clocks, B=2, T=4096, H=16, D=128. DRAM reads rose from 101 MB to 295 MB, and the kernel time still fell

flash-linear-attention #1319

Nsight Compute kernel time and SM throughput

B=2, T=4096, H=16, D=128, 2174 us, 1262 us, 3808 us, 3580 us, 31%, 49%, 32%, 35%, 85%

H100 SXM, Nsight Compute at base clocks, B=2, T=4096, H=16, D=128. Backward share is (3808+3580)/(1262+3808+3580). All at about 12% occupancy. Tile size, launch order and CTA size were not separated inside Nsight Compute

flash-linear-attention #1319

Backward pipe use and snapshot traffic

22-32%, 14-18%, about 3%, 96%

H100 SXM, Nsight Compute at base clocks, B=2, T=4096, H=16, D=128. ALU and XU across dq and dkv. dq DRAM writes total 0.39 GB over 3.8 ms, and the HBM share divides that traffic by HBM bandwidth

flash-linear-attention #1319

Why occupancy sits at 12.5% and why the retile did not raise it

12.5%, 255 registers, 2 CTAs of 4 warps, 8 of 64 warps, 1 to 2, 8 warps to 4, 65536 registers, 128 threads

H100 SXM, Nsight Compute, Block Limit Registers 2. An SM has 65536 registers, and 128 threads x 255 registers fits 2 CTAs. Before the retile the forward ran 1 CTA of 8 warps, which is the same 12.5%. Achieved occupancy is 12.16-12.49%

flash-linear-attention #1319, Not public: Nsight Compute report, not public

Tensor-core scans: forward gain and failed saturated-logit tests

about 3×, 20 of the 28

H100 SXM, bf16 operands, against the earlier forward, on the same test file. Failures are bf16 saturated-logit tolerance failures. With TF32 the forward is about 2.5x faster and also misses the tolerance

flash-linear-attention #1319, flash-linear-attention #1319

Test results and saturated-logit range for #1319

99/99, 56 of the 99, 15 in fp16, 30 in bf16

H100 SXM at 4c539a5, two runs. Separately the maintainer ran 99/99 on GB200 after merging main. Coverage stops at logit 15 in fp16 because casting score gradients to fp16 can underflow beyond it

flash-linear-attention #1319

One decode step against recomputing the full prefix, op level

T=1024, T=4096, T=16384, T=65536, B=1, T=1024, 72.4, 72.8, 1.0×, B=1, T=4096, 275.0, 929.4, 3.4×, B=1, T=16384, 1082.4, 14190.3, 13.1×, B=1, T=65536, 4301.8, 226529.2, 52.7×, B=8, T=16384, 1272.4, 114637.2, 90.1×, 1024 keys, 18 op-level rows

H200 (141 GB), torch 2.11.0+cu128, Triton 3.6.0, bf16, FLA_CI_ENV=0, commit f5e7b1cc. One new query per sequence, dense, HQ=32, H=8, D=128, median us from do_bench. Recompute is parallel_stickbreaking_attn over all T tokens. Each 4x step in T costs 3.80-3.97x. 90.1x is the best of 18 rows

flash-linear-attention #1412

Cache test against the uncached forward, and H200 pass counts

67-token prompt, 74 tokens

H200 at f5e7b1cc, before the rebase. test_cache sends a 67-token prompt, then steps of 1, 3, 1 and 2 tokens, and compares them with one uncached forward. Test groups passed 135, 85 and 30, with 5 skips that need flash-attn, which was not installed. Rebased commits reran only on the CPU interpreter

flash-linear-attention #1412

Model-level cached step, and the same step against recomputing

about 9 ms, 8.98-9.07 ms, 0.9×, 45.5×, 3 calls, B=1 with 512- and 2048-token prompts

H200, commit f5e7b1cc. Random-init StickBreakingAttentionForCausalLM, hidden 2048, 12 layers, 32/8 heads of dimension 64, bf16. Greedy generate, 64 new tokens, all-ones mask, (t(64) - t(1)) / 63, each t the median of 3 calls after 1 warmup. At B=1 with 512 and 2048 tokens the cached path is 9% and 6% slower than recomputing. 45.5x is B=8 with an 8192-token prompt. Where the 9 ms goes was not profiled

flash-linear-attention #1412

Decode speed and ratios in the 12-cell RTX 5090 grid

2.0-4.0×, all 12, 12 cells, 11 of 12 cells, 95% CI, 2 RTX 5090 hosts, 30 AIME-25 problems, 256 greedy tokens, Ryzen 9 9950X3D, Ryzen 9 9950X, 17001 MHz, 14001 MHz, 0.97 [0.96, 0.99], 1.03 [1.02, 1.04], 1.05 [1.04, 1.06], 1.09 [1.08, 1.11], 1.15 [1.13, 1.17], 1.20 [1.16, 1.22], 1.21 [1.19, 1.22], 1.27 [1.25, 1.29], 1.29 [1.28, 1.31], 103.1, 108.2, 109.2, 11%, 12.5%, 122.2, 132.2, 133.6, 152.5, 16.1, 18.5, 19.5, 2.00, 2.05, 2.08, 2.73, 2.80, 2.86, 2.91, 22.1, 23.8, 25%, 28.4, 28.9, 3.20, 3.34, 3.81, 32.7, 33.5, 33.6, 34.9, 38.7, 4.02, 40%, 40.0, 40.1, 43.75%, 47.7, 48.0, 51.4, 53.9, 54.0, 54.7, 57.9, 63.1, 69.9, 78.7, 85.6, 94.1, 95.8, 98.3

Two RTX 5090 hosts, 2 models, 3 budgets each. Host B has a 9950X3D, card memory at 17001 MHz, a best probed host read of 88 GB/s and a 575 W limit. Host S has a 9950X, stock clock, card memory at 14001 MHz, 71 GB/s and a 400 W limit. Rented hosts, batch-1 decode, equal GPU expert memory, 30 AIME-25 problems, first 256 greedy tokens, each system decodes its own text. Ratios are of mean speeds, intervals are 95% paired bootstrap over problems. FreeToken uses its faster backend per budget, picked on host B. Host B rows come from 3 rentals of one machine. The Qwen3 25% row on host B is the selection launch only

moe-speed-of-light: paper/tab_grid.tex, moe-speed-of-light: prereg/stockclock_089.json, moe-speed-of-light: paper/tab_headline.tex, moe-speed-of-light: paper/tab_headline.tex

The one cell where FreeToken was faster than the cache

0.97×

Qwen3-30B-A3B at 43.75% of experts on the GPU, host S, 30 problems. The interval is entirely below 1. Cause not separated

moe-speed-of-light: paper/tab_grid.tex

LRU in place of the decayed-frequency rule

6-21%, 4 of 6

Host S, all else as in the full system, 6 cells. At the other 2 cells LRU was faster than FreeToken at gpt-oss 25% and its interval includes 1 at gpt-oss 40%. The rule accounts for 77-84% of the difference from FreeToken at gpt-oss 25 and 40%, and more than all of it at gpt-oss 11% and Qwen3 12.5 and 25%

moe-speed-of-light: paper/wsg_numbers2.tex, moe-speed-of-light: paper/app_wsg.tex

Time per token of the cache and its oracles on one machine, against the limit

36%, 20.59, 20.18, 20.11, 15.13, 13.78, 10.02, Ryzen 9 9950X

Host O4, Ryzen 9 9950X, RTX 5090, gpt-oss-120b at 11% of experts on the GPU, job 096, teacher-forced, shuffled order. Milliseconds are 1000 over mean speed. 36% is (13.78 - 10.02) / (20.59 - 10.02), the share of the gap left on this machine. Oracles read the future routing from a recorded trace. Read-ahead is the paced copy path. The limit is computed, the rest measured

moe-speed-of-light: paper/tab_accounting_measured.tex, moe-speed-of-light: paper/app_foresight.tex

The deployed cache's gap to the limit, split by oracles in the engine

0 [-11, 12], 0 [-2, 1], 0 [-2, 2], 0 [-2, 3], 1 [-3, 4], 15 [11, 19], 18 [10, 25], 18 [14, 23], 18 [15, 20], 19 [13, 25], 19 [14, 24], 21 [20, 23], 25 [18, 31], 27 [25, 30], 28 [20, 36], 3 [-1, 8], 3 [-5, 12], 33 [21, 44], 35 [22, 48], 36 [13, 58], 4 [-6, 13], 43 [31, 56], 46 [38, 55], 52 [43, 62], 8 [2, 14], 15 machines, 5 machines, 19%, 43-52%, 42-48%, 95% interval

gpt-oss-120b at 11% of experts on the GPU. Mean share of the gap in percent, 95% t-interval over machines. Panel: 15 machines, exploratory. New: 5 machines rented for a registered test. Copy table off: 5 machines of two registered tests. With the table on, the first row measures MIN's schedule with the table, not its set held. The last three rows are attributed, not measured. Left at 25% is 42-48%

moe-speed-of-light: paper/tab_dm.tex

Timed runs below the limit, and the closest

302 timed runs, 1.33×

gpt-oss-120b at 11% and 25%, 25 launches. The closest is the read-ahead oracle. This is consistent with the limit, not a proof of it

moe-speed-of-light: paper/wsg_dm.tex

Cache speed as a share of the limit, beside the headline range

median 47%, 23-36%, 3 server machines, 12-46%

Median and server range are gpt-oss-120b at 11%. 23-36% is the 25 consumer machines at gpt-oss 25%. Limit from Belady's MIN and probed read rates. The 31% low end is one Threadripper with a 178 GB/s probe. Consumer means Ryzen, Core and Threadripper, server means EPYC, Xeon and engineering samples

moe-speed-of-light: paper/wsg_decomp.tex, moe-speed-of-light: paper/wsg_decomp.tex, moe-speed-of-light: paper/main_body.tex

Tests added with the rebuilt sign-off check, and how many fail on the code before it

171 new tests, 62 of them fail on the old code, 171 tests, 62 of them

tests/test_signoff_gaps.py (62) and tests/test_signoff_receipts.py, run on Nov 4, 2026 with a stub harness, not a live lab. All 62 gap tests failed against the previous commit and pass on the new code. The other 109 are not claimed to fail on the old code

agi-lab-framework: tests/test_signoff_gaps.py

New tests failing in a maintainer's own run with the second planted bug in the kernel

31 of 68, 6 of 12

Posted on the PR by the maintainer who approved and merged it, Oct 2, 2026, on an A100. The existing tests still passed. This reproduces one of the planted bugs, not all of them

cudnn-frontend #1287