Qwen3.8 Flash Next on my GX10: install, systemd and goodbye Qwen3.6

On the ASUS Ascent GX10 I finally have the setup I wanted: Qwen3.8 Flash Next running locally, with a systemd service that starts at boot and a script to switch models on the fly. This post documents the current installation and closes the Qwen3.6 chapter.

Why a hand-compiled llama.cpp

To exploit the GX10 you need fine-grained levers — precise quantization, very large context, MTP drafts for speculative decoding — and only llama.cpp exposes all of them, without intermediate layers.

git clone https://github.com/ggml-org/llama.cpp ~/llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)
~/llama.cpp/build/bin/llama-server --version
# version: 0.3.0-dev (build 10802, commit d1a92352c) — Linux aarch64

A note for GX10 owners (or any Arm64 machine): you need CUDA for aarch64 and a local build; x86 binaries won’t even execute. The compile takes about twenty minutes but it is a one-off operation.

The model

I picked an UD-IQ4_XS quantization (the classic Unsloth Dynamic quality/size trade-off) plus the MTP draft and the projection for the vision part:

With the GX10’s 128 GB of unified memory the model loads comfortably (at steady state the system sits around ~96 GB used) and there is headroom left for a generous context window.

The launch script

All the configuration lives in ~/llama-server/scripts/start-qwen38:

#!/usr/bin/env bash
set -euo pipefail

MODEL="$HOME/models/Qwen3.8-Flash-Next-UD-IQ4_XS/UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf"
DRAFT="$HOME/models/Qwen3.8-Flash-Next-UD-IQ4_XS/MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf"
MMPROJ="$HOME/models/Qwen3.8-Flash-Next-UD-IQ4_XS/mmproj-F16.gguf"

exec "$HOME/llama.cpp/build/bin/llama-server" \
    --metrics \
    -m "$MODEL" \
    --mmproj "$MMPROJ" \
    -md "$DRAFT" \
    --spec-type draft-mtp \
    --spec-draft-n-max 4 \
    -c 256000 \
    -np 1 \
    --host 0.0.0.0 \
    --port 30000

The parts that matter:

• -c 256000 → a 256k token context. It fits thanks to the GX10’s unified RAM, and it is what makes the difference on long tasks and agents.
• --spec-type draft-mtp --spec-draft-n-max 4 → speculative decoding with the MTP draft: up to 4 proposed tokens per verification step.
• -np 1 → a single inference slot: with an 88 GB model, parallelizing brings more problems than benefits.
• --mmproj + --spec-type draft-mtp → vision and MTP arrived with recent builds; in the earliest versions of the script I had a novision variant.
• --host 0.0.0.0 --port 30000 → OpenAI-compatible API exposed on the LAN, so the laptop can use it too.
• --metrics → metrics ready for Prometheus/Grafana.

Starting at boot: user systemd + linger

The part I cared about most: the model must be up when I power on the machine, without a login and without thinking about it. On a user service two pieces are enough.

~/.config/systemd/user/llama-qwen38.service:

[Unit]
Description=llama.cpp Qwen3.8 Flash Next Server
After=network-online.target
Wants=network-online.target
Conflicts=llama-qwen36.service

[Service]
Type=simple
ExecStart=%h/llama-server/scripts/start-qwen38
Restart=on-failure
RestartSec=10

[Install]
WantedBy=default.target

• Restart=on-failure + RestartSec=10: if the server dies, systemd brings it back after 10 seconds.
• Conflicts=llama-qwen36.service: the two models share port 30000, they cannot both be up at once.
• %h = home directory, so the file stays portable across machines.

The trick to start a user service without a login is linger:

loginctl enable-linger $USER        # verify: Linger=yes
systemctl --user enable --now llama-qwen38.service

Quick check after a reboot:

systemctl --user status llama-qwen38 --no-pager
curl -s http://127.0.0.1:30000/health
curl -s http://127.0.0.1:30000/v1/models | python3 -m json.tool

Switching models on the fly

With two models installed and 128 GB of RAM contested, I wrote ~/bin/model-switch:

model-switch status      # what is running right now
model-switch qwen38      # switch to Qwen3.8
model-switch qwen36      # go back

The logic, briefly: stop both services, wait for the port to free up (up to 60s), start the right service and wait for /health to answer (up to 180s) before declaring success. Without those two waits you end up sending requests to a model that is still loading — with 88 GB to mmap it is not instant.

The predecessor: Qwen3.6-35B-A3B

Before the current setup, the GX10 ran Qwen3.6-35B-A3B (MoE, ~3B active parameters), always with llama.cpp but in a simpler form: a single start script that read a config file and pulled the model straight from Hugging Face with -hf:

# how it was (configs/qwen3.6-35b.env, now removed)
MODEL="unsloth/Qwen3.6-35B-A3B-MTP-GGUF:UD-Q4_K_XL"
# → llama-server -hf "$MODEL" --ctx-size 262144 ...

Along the way I also tried the NVIDIA NVFP4 variant (nvidia/Qwen3.6-35B-A3B-NVFP4), later removed together with the Hugging Face cache to free up space.

Why the switch to 3.8:

• Quality: the 3.6 → 3.8 jump is visible, especially on code and long tasks.
• File management: moving from -hf (an HF cache that grows on its own) to GGUFs downloaded and versioned under ~/models/ gives me control over what occupies the disk.
• Explicit configuration: a script with hand-written flags beats the indirected env-file when there are many flags to balance.

The 35B MoE was not a bad model — it is quick and light on short exchanges, and its 3B of activated parameters still make it valid when you need lightning answers. For my usage (agents, huge contexts, code editing) the 3.8 in IQ4_XS with MTP is another category; which is why model-switch qwen36 stays there, ready.

What is left to do

• Serious benchmarks (tok/s in prefill and decode, with and without MTP) on realistic tasks, instead of impressions
• Energy comparison: GX10 vs the desktop GPU I used before
• Grafana dashboard with llama-server metrics (--metrics is already on, the panel is missing)
• Evaluate whether a smaller context window frees RAM useful to the rest of the system