Making MiniMax-H3 up to 9x faster on H100s

MiniMax-H3 is a 33B open-weight model that generates video and audio together. At full quality it is slow: a 15-second clip takes about seven minutes on four H100s. We spent three weeks making that faster, and the short version is that a 15-second clip now takes 46 seconds in the best configuration we trust, and a 5-second clip takes about 15.

Every number in this post comes from the way a production server actually runs: real requests, varied prompts, and a visual check on the output rather than a single metric. That discipline is the point of our optimization loop, and it is why we can tell you what held up and what did not.

The setup

• Hardware: 4× H100 80GB SXM on RunPod, about $13.16/hour for the pod.
• Serving stack: SGLang multimodal_gen 0.5.17, running with Ulysses-4 sequence parallelism and FSDP. The obvious alternative, TP2 plus Ulysses-2, hangs on this model, so every number here uses the first layout.
• Baseline: the plain model, 50 denoising steps, no LoRA, no cache, no sparse attention. It takes about 79–81 s for a 5-second clip and 419 s for a 15-second clip.
• How we measured: real requests to the running server (POST /v1/videos), not an offline script, then downloaded the files and looked at the frames. Most times are the server’s reported inference time, which leaves out about 5 s of MP4 muxing on a 15 s clip; the Sol-Engine full-step numbers are full-pipeline.

The results

At the pod rate above, the dense baseline costs about $0.29 of GPU time per 5-second clip. Turbo costs $0.05. The 15-second clip goes from about $1.53 to $0.17. The two shipping tiers exist because the techniques behind them do not stack at today’s step counts, which is the first of the surprises below.

What worked

Step distillation, once we fixed two bugs in the loader

Cutting 50 steps to 9 with a distilled LoRA is the single biggest lever: 5.3× to 5.7× on its own. It did not work out of the box on SGLang.

• The 9-step LoRA crashed on every forward pass. MiniMax-H3 merges Q, K and V into one projection using a class built for merging exactly two equal-sized outputs, and SGLang’s LoRA layer dispatch trusted the class name. A small patch to the weight slicing fixed it.
• The 5-step LoRA loaded without complaint and did nothing: it matched 0 of 266 layers because the adapter used different layer names. After we rebuilt it with the right names, a second bug appeared. There was no way to set the LoRA scale for this pipeline, and the default was roughly 16× too strong, which produced visibly corrupted video. We baked the correct scale into the converted weights.

We then excluded the 5-step LoRA anyway. It is the fastest row in the table, and its quality was not good enough to sell.

NVIDIA’s Sol-Engine, on the full 50-step path

Sol-Engine combines three things: fused kernels (exact), a sparse attention kernel (approximate), and a cache that skips a denoising step when the first transformer block barely changed (approximate). Together they gave 3.3× at 15 seconds without touching the step count, so this tier keeps the model’s full 50-step trajectory.

Sparse attention on top of turbo, at 15 seconds only

Layered on the 9-step LoRA, with the cache turned off and only the first denoising step run with full attention, sparse attention took the 15-second clip from 73 s to 46 s. At 5 seconds the same setting is a trap; more on that below.

The benchmark was not the product

NVIDIA reports 3.56× for Sol-Engine on H100. We reproduced 3.49× with their script. Then we put the same code behind the server and got between 2.1× and 2.9×, with a roughly 12 second penalty that kept coming back.

The cause was specific. The sparse attention kernel is compiled just in time and cached per exact tensor shape, and one of the dimensions in that key was the live token count, which changes with every prompt length. Two prompts that differed by two words meant a fresh 12–14 s compile. The offline script runs a warm-up pass with the same prompt it then times, so it never pays this. A real server pays it on almost every request.

We fixed it in two steps: use the padded, 64-token-aligned length that SGLang already computes, so prompts in the same bucket share a compiled kernel, and thread an existing masking path through so the padding contributes exactly zero instead of merely a small amount. Three prompts of different lengths now share one compile. The number to quote for realistic traffic is the mixed-prompt one: 31.0 s at 5 s (2.6×) and 128.3 s at 15 s (3.3×), not the 28 s you get from repeating one prompt.

The same effect shows up in our best result. The 45.7 s figure is a repeated prompt; mixed prompts land near 58 s, because the shape-caching fix has not been ported into that code path yet. Plan capacity around 58.

Things that did not stack

We expected turbo LoRA and Sol-Engine to multiply. They do not, and the reason is useful. Turbo’s weights are calibrated for about nine steps and corrupt the output if you run longer. Sol-Engine’s cache works by finding steps that barely change, and a 9-step schedule has no redundant steps left to find: at that length the cache either reused nothing or corrupted the output. Levers that act on every step equally, like distillation, compose. Levers that need many steps to find something to skip compete with distillation for the same slack.

The same mismatch explains why published ratios from other models do not transfer. A multi-lever result on a different model tells you how much headroom that model had, not how much this one has, and multiplying ratios from different vendors is how bad targets get set.

Dead ends we closed on purpose

We also profiled where the time goes now, to avoid optimizing the wrong thing. For a 15-second clip, denoising is about 88% of inference time and VAE decode is about 8.5% (3.9 s). A paper claiming a 6× faster VAE is therefore worth at most a few seconds here, and we did not chase it.

Verification that catches what the metrics miss

Sol-Attn ships with a numerical correctness gate that compares the sparse output against dense output inside a single attention call. A fast configuration at 5 seconds, with only one dense step, passed it cleanly, yet the video had a faint second rendering of the moving subject in every sampled frame. We did not ship it.

The gate only looks within one forward pass, so it has no way to see an inconsistency that builds up across denoising steps. LPIPS did not separate the broken 5-second clip from the clean 15-second one either. What caught it was putting same-seed frames from the baseline and the accelerated run side by side. That is why the sparse-attention setting is only used at 15 seconds, where three different prompts came out clean.

Custom kernels

We wrote a small tile-lang kernel library and benchmarked it against SGLang’s production ops, not against PyTorch eager. The fused RMSNorm-scale-shift kernel is 2.2–2.5× faster, and QK-norm plus RoPE is 1.15–2.22× faster, shrinking at larger sizes. The residual-gate kernel ties, because SGLang’s was already optimal.

The more ambitious one is a block-sparse attention kernel whose decision about which blocks to skip costs about 0.6 ms, against 800–860 ms per step of overhead in the shipped Sol-Attn. At the real production size (37,712 tokens) it first hit a pipelining bug, which we traced to a data-dependent load inside a software-pipelined loop. The fixed kernel runs at that size and is 1.63× faster than dense attention in isolation.

What is still open

• Integrate the fixed sparse kernel into the live server and measure it with the paired-frame check.
• Port the shape-caching fix into the sparse-attention-only path, which should pull the 58 s mixed-prompt number toward 46 s.
• Find the root cause of the 5-second ghosting, which probably needs inspection at the latent level rather than in output frames.
• Finish the two-stage cascade that denoises at low resolution and refines at high resolution. It is the one path with plausibly large untapped headroom, and it has not yet produced a full end-to-end number.

What it cost

The September 3–5 round alone, covering the sparse-attention retuning, the phase profiling and the kernel work, cost roughly $50–55 of GPU time. Profile, change one thing, verify against the baseline, record the result: that is the loop, and it is cheap enough to run on every model and every chip.