Why Llama 4 took minutes to load

I expected Llama 4 to be slow to load. The model I was testing is a Scout-based llama4:latest build: a 108.6-billion-parameter mixture-of-experts model in a 67.4 GB Q4_K_M file. What I did not expect was for the same model to load in about a minute on one run and then spend four or five minutes appearing to do almost nothing on the next.

This was running through Ollama with ROCm on my AMD Strix Halo system, which has a Radeon 8060S and 128 GB of unified memory. The machine was not running out of memory. Even during the slow runs it stayed below 100 GB in use, with more than 50 GB available at the lowest point I measured.

The confusing part was that the first round of tests looked reasonable. I ran the original model at successively larger context sizes and it passed from 8K through 64K. Load time rose with the context, but it did not explode.

Later, the same 16K test took about 248 seconds and a 32K test took about 277 seconds. That made context size look guilty, but it did not explain why earlier runs at those sizes had been much faster.

Clearing out the obvious explanations

The first serious stall appeared after I tried the model from OpenWebUI, so the frontend was an obvious suspect. I restarted Ollama and its container several times while trying to get back to a clean state. I also rebooted the Halo host twice during the investigation. Those reboots were useful recovery points, but they were not proof that a cache had been cleared or that OpenWebUI was the cause.

After the first reboot, a direct 16K request worked but still took about four minutes. A later direct runner test reproduced the slow load without OpenWebUI in the path. That ruled the frontend out as the root cause.

Llama 4 is multimodal, and the model and its projector were packaged in the same large blob. The amount of disk traffic initially made me wonder whether the projector was causing the file to be scanned twice. A controlled text-only test removed that explanation too: on Ollama 0.30.10 it was still unavailable after more than ten minutes and had read 166.5 GB.

At that point the useful question was no longer “why is generation slow?” It was “what is the loader doing before generation can begin?”

Measuring where the time went

I instrumented an original 32K run and separated server startup from prompt and token evaluation. The complete request took 311.46 seconds. Of that, 309.11 seconds happened while the server was starting.

The host read 144.5 GB from storage for a 67.4 GB model, or a little more than twice the model size. Average read throughput was about 463 MB/s and briefly peaked near 4.2 GB/s. Meanwhile, the GPU was mostly idle and the system showed no thermal, kernel, or out-of-memory error. The bytes were moving; the loader was simply taking a very long path to make them usable.

I updated the container from Ollama 0.30.10 to 0.32.1 before continuing. ROCm still detected the Radeon correctly and the existing model store was intact. The update was worth doing, but it did not by itself answer the question, so I tested the loading modes one at a time.

mmap was the important switch

By default, llama.cpp memory-maps model files. Its server documentation says that mmap is enabled by default and that it can be controlled with LLAMA_ARG_MMAP. It also exposes direct I/O as a separate, disabled-by-default option. That distinction mattered here: mmap and direct I/O are independent loader choices.

With mmap disabled and direct I/O enabled, the 32K Llama 4 runner became ready in about 20 seconds after reading roughly one model's worth of data. I then tested the ordinary buffered path with both mmap and direct I/O disabled. It became ready in 18.36 seconds. The default mmap run, by comparison, was still loading when I stopped it after 144.1 seconds.

This is the result that separated the fix from the workaround. Direct I/O was fast, but it was not necessary. Avoiding mmap was the part that changed Llama 4's startup behavior on this machine.

I checked smaller models so I would not optimize the system for one unusual file while quietly hurting everything else. A 13.8 GB gpt-oss:20b model and a 42.5 GB llama3.3:70b model also started quickly without mmap. Those two results came from a warm filesystem cache, so their tiny physical-read numbers are not fair cold-load benchmarks. The useful point is narrower: disabling mmap did not introduce an obvious startup or generation penalty in those runs. The Llama 4 result was more persuasive because it still caused 52.3 GB of real reads and finished in 18.36 seconds.

What I would keep

For this Halo system, the setting I would carry into the Ollama container is:

environment:
  LLAMA_ARG_MMAP: 'false'

I have not treated that as a universal recommendation. mmap is normally a sensible way to load large models, and the llama.cpp documentation even warns that disabling it can make loading slower on other systems. The result here is specific to this large Llama 4 file, Ollama's llama.cpp-based runner, ROCm, and the unified-memory behavior of Strix Halo. Ollama now documents the Radeon 8060S as a supported gfx1151 GPU, but reports around this platform also show that the software stack is still sensitive to backend and version details. Ollama's GPU documentation is the place I would recheck after a future upgrade.

The larger lesson was about the shape of the delay. The free-memory number told me the system was not memory-bound. The fast token evaluation told me inference was healthy once the model was ready. Reboots, container restarts, context changes, and bypassing OpenWebUI helped remove possibilities, but none located the time. Measuring startup separately and counting physical reads did. Once I did that, the five-minute mystery became a loader comparison, and one setting made the difference visible.