From GPU kernels
to autonomous agents.
M.S. ECE. I optimize CUDA & Triton kernels and LLM inference at the bottom of the stack, and build agent infrastructure, retrieval, and evaluation on top.
I work the whole stack.
From hand-written GPU kernels at the bottom to multi-agent systems at the top. Working every layer is how I find the one that owns the latency.
Autonomous systems that build software.
An autonomous ML research lab and HIVE, a multi-agent org, ship from pre-registered plans. I built their orchestration and anti-forgery sign-off governance.
Memory that beats the context wall.
bert gives agents a searchable memory of a whole project. Hybrid retrieval, fully local, built for projects that outgrow the context window.
LLM inference, profiled and tuned.
Attention and KV-cache decode profiled on edge silicon: fused SDPA (FlashAttention backend) vs naive attention. 15.4K tok/s on a single-stream KV-cache decode microbenchmark, Jetson Orin Nano.
Down to the metal.
Hand-written Triton kernels, GEMM fusion, removing HBM round-trips. Up to 1.73× lower latency than the unfused path on memory-bound FFN. The bare autotuned GEMM peaks at 213 TFLOP/s on an A100.
bert
A local MCP server that gives a coding agent searchable memory of a whole project. Once the project outgrows the context window, stuffing everything in fails and truncation drops the answer. bert pulls back only the slice that holds it.
A project's memory outgrows the model's window.
A mature project's decisions, designs, and post-mortems run to millions of tokens, beyond even a 1M window. Stuff it all in and the request fails. Keep only the recent slice and you lose the fact you need. Accuracy falls apart either way.
Hybrid retrieval, fully local.
sqlite-vec stores bge-base-en-v1.5 dense vectors, BM25 covers exact keywords, reciprocal-rank fusion merges the two, and a cross-encoder reranks the top hits. No API keys, no model calls. The host sees only the slice that matters.
Memory holds where stuffing the context collapses.
One reader (Claude Sonnet, 200K window), one variable: how it gets the context. On a 1.26M-token project memory, about 6× that window, full-context drops to 0.08 while bert holds 0.90. bert matches an agent grepping the raw files and spends half the tokens to do it, and it beats a plain vector lookup by 0.50. The honest caveat: on source code, where any file re-reads cheaply, grep wins. bert is built for accumulated memory, not code search.
A silent bug had pinned it at near-random.
A dict-key mismatch zeroed the dense signal, and a 240-character cap chopped the answer spans, so the hybrid path was really keyword-only. I traced it, fixed it against the shipped retriever, and the held-out eval jumped from 0.10 to 0.85.
0.745 nDCG@10 on a public IR benchmark.
Three public BEIR datasets: scifact, nfcorpus, fiqa. bge-base matches its published reference on all three. On scifact the full stack scores 0.745, past the published BM25 baseline of 0.665 and level with its own published mark of 0.741. Public data, public metric, nothing I defined myself.
Triton vs cuBLAS
Where does fusing the epilogue beat the standard unfused path (a cuBLAS GEMM plus separate bias and GeLU kernels)? I built the benchmark to find out, across 76 LLM-shaped GEMMs on an A100.
on small-batch FFN
Small-batch FFN projections are memory-bound.
Linear, then bias, then GeLU: three passes, each writing to HBM and relaunching. On small batches the GPU spends more time moving data than computing.
Fuse the three into one Triton kernel.
linear + bias + GeLU in a single launch, keeping the intermediate in registers. Two HBM round-trips and two kernel launches, gone.
Up to 1.73× lower latency.
On the best small-batch FFN shape, M=128, N=11008, K=4096. The bare autotuned Triton GEMM peaks at 213 TFLOP/s, 68% of the A100's fp16 tensor-core peak.
Where custom kernels pay off.
I profiled all 76 shapes on p50/p90 latency, throughput, bandwidth, and jitter to find the line where a fused kernel beats cuBLAS and where the vendor library already wins. Small, skinny GEMMs favor fusion. Mapping that boundary is the result.
Selected work
About
I keep the results that disagree with me. Writing CUDA and Triton kernels taught me which operations are memory-bound and which are compute-bound. Building the layers on top, LLM inference, retrieval, and multi-agent systems, taught me which of those bottlenecks the real workload actually cares about. I measure all of it end to end.