Still some work todo, look for the next blog post where I get more time in programming with Qwen 3.5. Previously it has done some implementation work in swift, it did ok with codex as the agent cli.
Request) Grok on June 8th 2026
Based on our last conversation about local llm’s using ollama on the Mac. Some YouTubers mentioned offloading layers, managing the layers ollama uses for cpu vs gpu, and all around getting higher performance out of models that already performed really well with MLX on the M5 Pro with 32 GB ram. What are some of these command line options that you can dig into and give me some examples. Basically when I’ve run ollama in the past, the context window was to large, I didn’t know, and responses were really slow, or crashed using some of the terminal CLI agents.
ollama launch opencode –model gemma4
ollama launch codex –model qwen3.5:35b-a3b-coding-nvfp4
through trial and error I found this out for swift iPhone programming;
Winner qwen3.5:35b-a3b-coding-nvfp4 USE codex for agent work!
Run it local on the MAC
ollama launch codex –model qwen3.5:35b-a3b-coding-nvfp
Response from Grok)
Ollama on Apple Silicon (especially M5 Pro 32 GB) has evolved significantly. Since around v0.19 (March 2026), Ollama switched its Apple Silicon backend to Apple’s MLX framework. This leverages unified memory much more efficiently than the older llama.cpp + Metal path, delivering big speedups (often 2x+ in decode tokens/sec for models like your Qwen variant). That’s why models that already shine in pure MLX setups also perform well under Ollama now.
Your winner — qwen3.5:35b-a3b-coding-nvfp4 (the NVFP4-quantized coding-specialized variant) — is an excellent choice for Swift/iOS work. The ollama launch codex --model ... (and opencode) integration is perfect because it wires the CLI coding agent directly to your local Ollama model with OpenAI-compatible endpoints.
Why Large Context Windows Caused Slowdowns/Crashes
The main culprit is almost always context size (num_ctx). Larger context inflates the KV cache (key-value cache for attention) dramatically. On a 32 GB unified-memory M5 Pro, a ~35B model (even heavily quantized) already uses a substantial chunk of RAM just for weights + activations. Pushing num_ctx to 64k+ (or letting agents run wild with history) easily leads to memory pressure, swapping, or OOM-style behavior in long-running CLI agents like Codex/OpenCode.
Rule of thumb for your hardware + this model + coding agents:
- Start with 8k–16k tokens for snappy performance.
- 32k is often viable and very useful for multi-file Swift work.
- 64k+ is risky on 32 GB unless the session is very light.
Ollama (with MLX) is smarter about memory than older setups, but physics still applies.
Key Command-Line / Configuration Options
Here are the practical levers YouTubers and power users talk about:
PARAMETER num_gpuin a Modelfile (still the main way to influence layer offloading)
- Even with the MLX backend, this guides how aggressively layers/compute go to the GPU/Neural Engine vs. CPU fallback.
- High values (99 or 999) = “use as much GPU acceleration as possible.”
- Ollama/MLX handles the unified-memory details automatically, but this prevents overly conservative defaults.
PARAMETER num_ctx— Context window size (the biggest lever for your past issues).- Environment variables (set before launching the server or
ollama launchcommands):
OLLAMA_MAX_LOADED_MODELS=1— Prevents multiple models fighting for RAM (highly recommended for 32 GB machines).OLLAMA_NUM_PARALLEL=1(or 2) — Limits concurrent requests per model.OLLAMA_MLX=1— Ensures the MLX backend is active (check logs; it may be default now on recent versions).- Optional:
OLLAMA_FLASH_ATTENTION=1(can help on Apple Silicon). On Mac (persistent for the Ollama app):
launchctl setenv OLLAMA_MAX_LOADED_MODELS 1
launchctl setenv OLLAMA_NUM_PARALLEL 1
launchctl setenv OLLAMA_MLX 1
# Then restart the Ollama app or `ollama serve`
For a one-off terminal session:
export OLLAMA_MAX_LOADED_MODELS=1
export OLLAMA_NUM_PARALLEL=1
export OLLAMA_MLX=1
ollama launch codex --model your-model
- Runtime tweaks (in interactive
ollama run):
/set parameter num_ctx 16384- Then
/save my-optimized-modelto persist it.
Recommended Practical Setup for You (Swift/iOS Coding)
Create a custom optimized variant of your winning model:
Modelfile.qwen-coder (save this file):
FROM qwen3.5:35b-a3b-coding-nvfp4
# Sweet spot for 32 GB M5 Pro + coding agents (Swift, multi-file, tools)
PARAMETER num_ctx 16384
# Push maximum GPU/Neural Engine layer offload (MLX unified memory)
PARAMETER num_gpu 999
# Coding-friendly sampling (more deterministic, less hallucinated code)
PARAMETER temperature 0.2
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1
# Optional but nice for your use case
SYSTEM You are an expert Swift and iOS/SwiftUI developer. Write clean, idiomatic, modern Swift code following Apple guidelines. Explain non-obvious decisions when helpful. Prefer Swift Concurrency, SwiftUI best practices, and clear architecture.
Build it:
ollama create qwen3.5-coder-16k --file Modelfile.qwen-coder
Launch your agents with it:
# Codex CLI (your previous winner setup)
ollama launch codex --model qwen3.5-coder-16k
# Or OpenCode
ollama launch opencode --model qwen3.5-coder-16k
Want to experiment with context?
Change num_ctx to 8192 (safer/faster) or 32768 (more capable for big codebases) and rebuild the custom model. Test with real Swift tasks and watch memory in Activity Monitor.
Monitoring & Debugging
ollama ps— Shows loaded models, context size, and processor split info.- Activity Monitor → Memory + GPU/CPU tabs while the agent is thinking.
- Run the server manually with debug for insight:
OLLAMA_DEBUG=1 ollama serve
Look for MLX mentions and any layer/memory allocation logs.
- If responses feel slow: drop
num_ctxor confirmnum_gpu 999is taking effect.
Quick Notes on MLX vs. Pure llama.cpp Layer Control
With the MLX backend, Ollama is more “set it and forget it” than classic -ngl / n_gpu_layers in raw llama.cpp. The num_gpu parameter still gives you useful influence, and MLX’s unified memory handling is generally superior for Apple Silicon. If you ever want maximum raw speed on a specific prompt (outside the launch codex workflow), some people drop to pure MLX-LM Python scripts, but you lose the seamless Codex/OpenCode integration.
Your current model + ollama launch codex workflow is already one of the best practical setups for local Swift coding on a 32 GB M-series Mac. The tweaks above (especially a sensible num_ctx + num_gpu 999 in a custom model + the env vars) should eliminate the previous slowdown/crash issues while keeping (or improving) the excellent performance you’ve seen.
Try the 16k custom model first and let me know how the memory usage and response times feel on real Swift tasks — we can fine-tune further (e.g., 8k vs 24k context, temperature, or even a lighter quant if needed). Happy coding!
Notes from using the 16k model build by Bill)
I had qwen build a front end using angular latest version.
/status
╭─────────────────────────────────────────────────────────────────────────────╮
│ >_ OpenAI Codex (v0.138.0) │
│ │
│ Model: qwen3.5-coder-16k (reasoning medium, summaries auto) │
│ Model provider: Ollama - http://127.0.0.1:11434/v1 │
│ Directory: ~/source2025/ollama/new-front-end │
│ Permissions: Workspace (Ask for approval) │
│ Agents.md: <none> │
│ Collaboration mode: Default │
│ Session: 019eaaa1-d52f-7aa0-9e79-1c057b5e4906 │
│ │
│ Token usage: 177K total (175K input + 2.35K output) │
│ Context window: 100% left (8.61K used / 122K) │
│ Limits: not available for this account │
╰─────────────────────────────────────────────────────────────────────────────╯
Token usage: total=177,139 input=174,786 output=2,353
To continue this session, run codex resume QWEN and the agent got stuck on “cd my-angular-app && npm install” for 4+ minutes. It consumed context as well. I did stop it eventually did that step manually, and exited the session.