h3-metal：Apple Silicon 原生推理 MiniMax-H3

h3-metal Native MiniMax-H3 inference for Apple Silicon. The project is being built as a sequence of working vertical slices: deterministic host/model metadata first, then portable Metal block parity, prompt encoding, prompt-to-video/audio, and first/last-frame conditioning and then ordered references. Prompt-to-video/audio, first/last-frame conditioning, and ordered Ref2VA image/video/audio references work end to end. The current work is incremental H3-specific Metal performance and memory optimization on M3 Max and M5 Max. ## Tutorial ## 1. Build and inspect the model The examples assume that the Hugging Face snapshot is in ./MiniMax-H3 and that FFmpeg and FFprobe are available on PATH. ``` make -j8 mkdir -p outputs ./h3 --info -d ./MiniMax-H3 ``` --info checks the model layout and prints the selected Metal device without mapping all weights or generating media. Run ./h3 --help for the complete CLI reference. Without -p, the same binary starts an Iris-style interactive session: ``` ./h3 -d ./MiniMax-H3 --width 512 --height 512 --steps 6 ``` Type a prompt to generate a numbered video. The session keeps the exact BF16 prompt conditioning, prepared DiT, and video decoder in memory, so repeating a prompt with another seed avoids loading and encoding them again. Useful commands are !status, !seed random, !seconds 2, !show, !save output.mp4, and !cache. Use !help for the full, short list. First/last-frame conditioning is persistent in the session: ``` h3> !first opening.png h3> !last ending.png h3> The camera moves slowly around the subject. ``` Use !first clear or !last clear to remove an anchor. Generated videos are written to the session directory printed at startup. For a general Ref2VA conditioning image, use !ref-image PATH instead. Images are appended in order and exposed to the model as , , and so on; filenames have no meaning to the model. ``` h3> !ref-image person.png h3> Make the person shown in Picture 1 wave to the camera. ``` !refs lists the current order, !ref-remove N removes one entry, and !refs clear removes them all. Ref2VA references cannot be mixed with !first/!last anchors. ## 2. Make a first fast video Start with the validated balanced preset. It generates 22 frames at 24 fps (about 0.92 seconds), displays the evolving middle-video frame after every denoising transition in a supported graphical terminal, and prints phase timings: ``` ./h3 --profile \ -d ./MiniMax-H3 \ -p "A red fox walks through fresh snow in a pine forest. Medium tracking shot, natural winter light, realistic fur, soft footsteps and wind." \ --width 512 --height 512 \ --frames 22 --steps 20 \ --layers 45 --reuse 2 \ --show \ -o outputs/fox-fast.mp4 ``` This is deliberately not the most aggressive configuration: - --steps 20 performs the default 20 denoising passes. - --reuse 2 computes 11 fresh denoiser velocities instead of all 20 and extrapolates the skipped transitions. - --layers 45 runs 45 of the 50 transformer blocks, reducing both time and unified-memory use. - --show is optional. It supports Kitty/Ghostty and iTerm2/WezTerm/Konsole graphical protocols. It loads a resident preview VAE, displays one representative middle-video frame after every Euler transition, and then displays all final frames. Display dimensions default to 2x so the image has its intended logical size on macOS Retina screens; use --zoom 1 on a non-HiDPI display. This adds preview decode time and roughly 10 GiB of temporary model residency; runs without --show are unchanged. - --profile is optional and does not select a different generation path. The first process invocation also pays model loading and filesystem-cache costs. Compare performance using repeated runs, and alternate variants when the machines are warming up because this workload is sensitive to thermal throttling. For a very short iteration, request four denoising passes directly: ``` ./h3 --profile \ -d ./MiniMax-H3 \ -p "A red fox walks through fresh snow in a pine forest. Medium tracking shot, natural winter light, realistic fur." \ --width 512 --height 512 --frames 22 \ --steps 4 --layers 50 --reuse 1 \ --show \ -o outputs/fox-four-step.mp4 ``` --steps N always means exactly N denoising passes. Four through seven passes use the same schedule that won the low-budget comparison; increasing from 4 to 7 progressively improves detail and motion. Keep --reuse 1 at such small budgets so every requested pass runs the model. --show displays one preview after each pass. Several tail-heavy schedules were evaluated because most visible cleanup happens late in a long run. They preserved too few early composition updates and produced woven texture, weak motion, or clipped colors. The retained mode uses the released linear base grid with one terminal point. On the 512-square, 22-frame fox test, the selected four-pass result had 0.556 full-video SSIM against a 29-pass reference; an independent surfer test measured 0.547. The four-pass denoise took about 3.5 seconds on M5 Max, versus 26.4 seconds for the reference. ## 3. Move toward reference quality Change one control at a time when evaluating quality. First restore all layers, then all denoiser evaluations, and finally raise the default 20-pass schedule to the slower 50-pass reference: ``` ./h3 --profile \ -d ./MiniMax-H3 \ -p "A red fox walks through fresh snow in a pine forest. Medium tracking shot, natural winter light, realistic fur, soft footsteps and wind." \ --width 512 --height 512 \ --frames 22 --steps 50 \ --layers 50 --reuse 1 \ -o outputs/fox-close.mp4 ``` The defaults are --steps 20 --layers 50 --reuse 1; keep --steps 50 explicit for this close path. It performs 50 complete 50-block denoiser forwards and is much more expensive than the default, but is the right oracle when a fast mode changes the subject, anatomy, motion, or composition. Numerical pixel identity with MLX is not expected because the random-number and execution engines differ; the depicted content and motion should agree. ## 4. Choose a speed/quality preset These controls are independent unless noted otherwise: Control | Slow reference | Default | Aggressive | Main impact Denoising passes | --steps 50 | --steps 20 | --steps 4..7 | The number always names actual denoising passes. Whole denoiser reuse | --reuse 1 | --reuse 2 | --reuse 3 | At 20 steps: 20, 11, or 8 fresh DiT evaluations. Active DiT blocks | --layers 50 | --layers 45 | --layers 40 | Fewer blocks reduce compute and resident transformer weights. Core residual reuse | --core-reuse 1 | --core-reuse 4 | --core-reuse 6 | Refreshes patch/head work every step but runs the expensive core less often. Token reduction | off | optional | --token-reduction | Pairs horizontal video tokens inside middle blocks; faster but may change composition. Internal canvas | output size | 384x384 for 512 square output | 320x320 | Runs DiT/VAE smaller, then upscales with vImage. On M5, --use-int8-row-fc2 uses one activation scale per FC2 row and a single full-width TensorOps product. It is optional because it is less numerically conservative than grouped int8. It reduced complete denoiser forwards by about 2.6% in reciprocal tests. Matched four-step fox and surfer videos kept the same subjects, setting, and motion (full-video SSIM 0.919 and 0.828). In the interactive session, use !int8-row-fc2 on. --reuse and --core-reuse are mutually exclusive. Layer thinning can be combined with either one. To make the first command faster while keeping its output resolution, add token reduction: ``` ./h3 --profile \ -d ./MiniMax-H3 \ -p "A surfer riding inside a sharp blue ocean wave, one rider and one white board, realistic spray." \ --width 512 --height 512 --frames 22 --steps 20 \ --layers 45 --reuse 2 --token-reduction \ -o outputs/surfer-fast.mp4 ```