Turn a gaming PC or a MacBook/Mac Mini into an AI server. Run one install script and start using local AI models: chat, coding, images and video, all switched from a web page.

On a gaming PC with an NVIDIA card you get everything. On an Apple Silicon Mac you get chat, coding and images (no video yet): see On a Mac.

No setup, no config files to learn. RigUp installs the driver, Docker and the engines, then gives you a dashboard. Pick a model and it downloads whatever is missing, stops whatever currently holds the card, and starts the new one. A gaming card holds one big model at a time, so that's how RigUp runs them. The recipes (launch flags, quantization, context length) are tuned for 24 GB cards and come with suggested options for 16 GB and 12 GB cards.

Example: Qwen3.6 REAP serving on an RTX 3090. The tabs at the top switch between the 24, 16 and 12 GB recipe sets.

Ports 8000 and 8188 are never up at the same time, because each one needs the whole GPU.

• Ubuntu 22.04 or newer (other distros may work; the installer is written for apt)
• One NVIDIA GPU with 12 GB VRAM or more
• 32 GB system RAM recommended (the 12 and 16 GB recipes keep part of the model in RAM)
• Disk: about 60 GB for the engines (vLLM image ~28 GB, ComfyUI ~19 GB, llama.cpp ~7 GB, Open WebUI ~7 GB), plus 10 to 45 GB per model you install. Plan for 200 GB or more.
• SSH access with a user that can sudo

Measured on an RTX 3090 (24 GB) with the default 24 GB recipes.

Same four prompts on each model, at 2K. Times are per image with the model already loaded; the first image after a switch adds the model load. The prompts and settings are in docs/samples/generate.py and generate_others.py, so you can rerun them.

	FLUX 3 Klein (16 steps)	Krea 2 Turbo (8 steps)	Z-Image Turbo (8 steps)
Landscape 2048×1152	36 s	48 s	18 s
Portrait 1536×2048	50 s	66 s	28 s
Travel 2048×1360	44 s	59 s	24 s
Desk 2048×1152	36 s	48 s	21 s

Click an image for the full resolution file. A second seed of each is in docs/samples/alternates/. The sample images are free to reuse.

Every number in this README was measured on this PC: a small form factor box that sits on a desk and is used over the network.

Open	Closed

Prices move fast right now: RAM prices roughly tripled during 2026 because of AI demand for memory, SSDs went up too, and used 3090s climbed for the same reason. Check current listings before buying. The GPU is most of the cost; everything else is ordinary gaming PC parts.

Why these parts:

• The GPU sets the limits. VRAM decides which models fit, and memory bandwidth decides how fast they write. The 3090's 24 GB and 936 GB/s match the newer RTX 4090 on capacity and come close on bandwidth, at a fraction of the price used.
• The CPU barely matters. The models run on the GPU. A 6 core chip is plenty.
• 32 GB of RAM is the practical minimum. The 12 and 16 GB card recipes keep part of the model in system RAM, and model files pass through RAM when loading.
• A fast NVMe SSD makes model switches quicker. A 20 GB model reads from disk in a few seconds; most of a switch is the engine starting up.
• 750 W is enough for a 3090 (up to 350 W stock, 420 W on this EVGA card) plus a 65 W CPU. Use a quality unit: 3090s have short power spikes well above their rating.

Key specs:

The SSD's rated speeds need many reads in parallel; a single stream like loading one model file gets about half. That's still a 20 GB model in under 10 seconds.

git clone https://github.com/guscanales/rigup.git
cd rigup
./install.sh

The installer:

1. Installs the NVIDIA driver if nvidia-smi doesn't work yet. A new driver needs a reboot, so it stops and asks you to reboot and run ./install.sh again.
2. Installs Docker and the NVIDIA Container Toolkit, then checks that a container can see the GPU.
3. Asks for a dashboard password (stored hashed in data/config.json).
4. Starts Open WebUI on port 3000.
5. Installs the dashboard as a systemd service (rigup) that starts at boot.

It doesn't download any models. Each model downloads the first time you use it.

Options, set before running:

RIGUP_PORT=9000 ./install.sh          # dashboard on another port
RIGUP_DATA=/mnt/big ./install.sh      # keep models on another disk

RigUp also runs on Apple Silicon Macs (M1 or newer): the LLMs (chat, coding agents and the API) and ComfyUI with three image models. Video isn't on the Mac version; LTX-2.5 needs more memory and speed than a 24 GB Mac has.

ComfyUI running on an M4 Pro Mac with 24 GB, next to the LLMs it can switch to.

git clone https://github.com/guscanales/rigup.git
cd rigup
./install-mac.sh

You need Homebrew. The installer adds llama.cpp and uv from Homebrew, asks for a dashboard password, and installs the dashboard as a login service. The dashboard runs on the Python that comes with macOS. The first time you switch to ComfyUI, it downloads ComfyUI and builds its own Python environment with uv, which takes a few minutes.

What's different from the Linux version:

• Native engines instead of Docker (Docker on a Mac can't reach the GPU): llama.cpp with Metal for LLMs, ComfyUI on PyTorch's Metal backend for images. I measured MLX too: it was about 10% faster, but it has no Anthropic API, so Claude Code couldn't connect. llama.cpp serves both APIs.
• Image models as GGUF. The int8 files the Linux version uses need a matrix operation Apple GPUs don't support yet, so the Mac packs use GGUF versions of the same models (the ComfyUI-GGUF loader comes with the install). ComfyUI's built-in templates expect the int8 files, so open the mac_ workflows from the Workflows sidebar instead.
• Ports: the dashboard on 8090, the LLM API on 8000 and ComfyUI on 8188. There's no Open WebUI on the Mac version; any chat app that takes an OpenAI-compatible URL works, and the coding agent setup below is the same.
• Recipes by unified memory: tabs for 16, 24, 32, 48, 64 and 128 GB Macs. A Mac with an in-between size (36 or 96 GB) uses the tier below it. macOS lets the GPU use roughly three quarters of the memory, so a 24 GB Mac has about 18 GB for models; the dashboard shows the exact figure.
• Memory meter only. macOS doesn't expose GPU load, temperature or power without admin rights, so the dashboard shows the memory the model is using.
• Keep it awake. The server stops while the Mac sleeps. For an always-on box, turn on "Prevent automatic sleeping when the display is off" in System Settings > Energy.

Measured on an M4 Pro with 24 GB (memory bandwidth 273 GB/s, about a third of an RTX 3090):

Reading a long prompt ran at about 460 tokens per second, so an 8K token prompt takes around 18 seconds before the answer starts. That's the main gap to an NVIDIA card for coding agents, which send long prompts.

Images, same Mac, 1024×1024, warm run:

Memory is ComfyUI's peak, text encoder included; Krea nearly fills a 24 GB Mac, so close other apps while it runs. That's roughly ten times slower than the RTX 3090. The GPU is the limit, not the file size: Z-Image ran at the same speed from Q4, Q8 and BF16 files. Klein and Krea share a text encoder, so installing both downloads it once. Klein on the Mac is text to image only; its prompt upsampler and the edit workflows use int8 files.

The models for bigger Macs (32 to 128 GB) are estimates sized from file sizes, including a 122B model for 128 GB Macs. If you have one of those, run Auto-tune and press Share results.

Uninstall on a Mac:

launchctl bootout gui/$(id -u) ~/Library/LaunchAgents/com.rigup.dashboard.plist
rm ~/Library/LaunchAgents/com.rigup.dashboard.plist
rm -rf data/            # deletes every downloaded model
brew uninstall llama.cpp uv   # optional

Open http://<server-ip>:8090, log in with any username and your password.

• Lab card (LLMs): tap a model, then tap again to confirm. If its files aren't on disk yet, the button says Get 16 GB + switch and the download happens first; progress shows in the row and in the Console panel. Once it says Active, the API on :8000 and Open WebUI on :3000 answer with that model.
• ComfyUI card (images and video): install the packs you want with Install, then press Switch to ComfyUI. Each pack tells you which template or workflow to open in ComfyUI. The first ComfyUI start installs ComfyUI itself and takes several minutes.
• Free GPU stops whatever is running.
• Remove deletes a model's files. Files that another installed model shares (for example the same text encoder) are kept.
• Tabs 24 GB / 16 GB / 12 GB show the recipes for each card size. Your own card's tab is marked "yours". Recipes for a bigger card than yours are shown for reference but can't be started.

The recipes were measured on an RTX 3090. Auto-tune (under the Lab list) measures them on your card instead:

1. Checks for new releases. It downloads the newest model list from this GitHub repo. Models added since your copy show up with a new badge, ready to get and run. Then it looks at recent uploads on Hugging Face from the publishers in watch (in recipes.json) and lists the ones that could fit your card but have no recipe yet, with links. Those need a recipe before RigUp can run them.
2. Tunes every installed Lab model. It starts each one with its recipe settings. If the model doesn't fit, it lowers the context length until it starts. On vLLM, if there's memory to spare, it tries a longer context. Then it times a ~400 token answer and reads the VRAM in use.
3. Saves the results for your GPU. Rows show a tuned badge with your own context, tok/s and VRAM, and switching to that model uses the tuned settings from then on. A model that can't fit at all gets no fit.
4. Puts back whatever was running before.

It takes a few minutes per installed model. Downloads aren't part of it: get the models you want first.

Share results opens a GitHub issue on this repo, prefilled with your GPU name, VRAM, driver version and the tuned numbers. Nothing is sent until you press Submit on GitHub. Results from 12 and 16 GB cards are especially welcome, since those recipes are estimates.

FLUX 3 Klein and LTX-2.5 require accepting a license on Hugging Face before download:

1. Sign in at huggingface.co and accept the license on flux-3-image-klein-comfy and LTX-2.5.
2. Create a read token at huggingface.co/settings/tokens.
3. Paste it in the dashboard (Open panel, "Hugging Face token") and press Save.

If you skip this, the download stops with a message telling you which page to accept.

The LLM API on :8000 works with coding agents and other harnesses. It speaks the OpenAI API (/v1/chat/completions), and both engines also speak the Anthropic Messages API (/v1/messages), so Claude Code connects directly with no proxy.

OpenCode with the RigUp models added next to its cloud models.

Three things to know first:

• The model name is the recipe id: qwen3-coder-30b, qwen3.6-reap, qwen3.8-27b, and so on. The dashboard's Open panel shows the one that's running.
• Only the running model answers. If an agent asks for a model that isn't loaded, the request fails. Switch to it in the dashboard first.
• The API key is optional. By default none is checked, and harnesses that insist on one accept any placeholder. To require one, set LLM API key in the dashboard's Open panel; it applies from the next model start, on both the Linux and Mac versions. In the examples below, local stands for your key.

Which model for agents: qwen3-coder-30b is the fastest with solid tool calling. qwen3.6-reap is the best all-rounder, with 256K context and image input, which helps with big repos and screenshots. qwen3.8-27b is the smartest, but its 38K context is too small for agents with large system prompts like Claude Code; use qwen3.8-27b-gguf (256K) for that.

In the examples below, replace <server-ip> with your server's address.

Set these environment variables, either in your shell for one session or in the env block of ~/.claude/settings.json to make it permanent:

export ANTHROPIC_BASE_URL=http://<server-ip>:8000
export ANTHROPIC_AUTH_TOKEN=local
export ANTHROPIC_MODEL=qwen3.6-reap
export ANTHROPIC_DEFAULT_HAIKU_MODEL=qwen3.6-reap   # background tasks use the same local model
claude

Unset them (or remove the env block) to go back to Claude. Claude Code is tuned for Claude models, so expect a local model to need more guidance and to make more tool call mistakes.

Add a provider to ~/.config/opencode/opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "rigup": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "RigUp",
      "options": { "baseURL": "http://<server-ip>:8000/v1", "apiKey": "local" },
      "models": {
        "qwen3-coder-30b": { "name": "Qwen3 Coder 30B", "limit": { "context": 81920, "output": 8192 } },
        "qwen3.6-reap": {
          "name": "Qwen3.6 35B-A3B REAP",
          "limit": { "context": 262144, "output": 16384 },
          "modalities": { "input": ["text", "image"], "output": ["text"] },
          "attachment": true
        },
        "qwen3.8-27b": { "name": "Qwen3.8 27B", "limit": { "context": 38912, "output": 8192 } }
      }
    }
  },
  "model": "rigup/qwen3.6-reap"
}

Then pick the model with /models. Setting limit.context matters: OpenCode uses it to decide when to compact the conversation.

Run hermes model, choose Custom endpoint (self-hosted / VLLM / etc.), enter http://<server-ip>:8000/v1, skip the API key, and enter the model name. Or edit ~/.hermes/config.yaml:

model:
  default: qwen3.6-reap
  provider: custom
  base_url: http://<server-ip>:8000/v1
  context_length: 262144

Set context_length yourself: Hermes otherwise guesses from the server and may pick a smaller window.

Add a provider to ~/.openclaw/openclaw.json and make it the default model:

{
  "models": {
    "providers": {
      "rigup": {
        "baseUrl": "http://<server-ip>:8000/v1",
        "apiKey": "local",
        "api": "openai-completions",
        "timeoutSeconds": 300,
        "models": [
          { "id": "qwen3.6-reap", "name": "Qwen3.6 REAP (RigUp)", "reasoning": true, "input": ["text", "image"],
            "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 },
            "contextWindow": 262144, "maxTokens": 16384 }
        ]
      }
    }
  },
  "agents": { "defaults": { "model": { "primary": "rigup/qwen3.6-reap" } } }
}

Any tool with an "OpenAI compatible" option works the same way: base URL http://<server-ip>:8000/v1, any API key, model name = recipe id. Quick test from the command line:

curl http://<server-ip>:8000/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model": "qwen3.6-reap", "messages": [{"role": "user", "content": "Say hi"}]}'

The status badge on each row says how much to trust the numbers:

• measured: run and timed on that card size (all measured on an RTX 3090, 24 GB)
• runs: run on that card size, not timed yet
• estimate: sized on paper, not run on that card size yet. Expect to adjust the context length if it runs out of memory.

All Lab models are Apache 2.0. The 16 and 12 GB recipes use llama.cpp's --fit on, which sizes the GPU part automatically and keeps the rest in system RAM.

Download sizes count each pack on its own; packs that share files (Klein and Klein Base, the two Qwen-Image packs) download those files once.

Licenses matter. The weights are downloaded from their publishers under their own licenses; this project doesn't redistribute them. Read each license before using outputs commercially.

Each model's original release on Hugging Face, and the repo the dashboard downloads from (a quantized or ComfyUI-packaged build of the original).

Engines: vLLM, llama.cpp, ComfyUI (run with the mmartial/comfyui-nvidia-docker image on Linux, natively with ComfyUI-GGUF on a Mac), Open WebUI.

rigup/
├── install.sh          one-time setup (Ubuntu + NVIDIA)
├── install-mac.sh      one-time setup (Apple Silicon Mac)
├── server.py           dashboard + API (Python standard library only)
├── test_server.py      self-check for the parts that need no GPU: python3 test_server.py
├── recipes.json        every model: files, launch flags, sizes, measured numbers
├── web/                the dashboard page
├── workflows/mac/      ComfyUI workflows for the Mac's GGUF image packs
└── data/               created at install; not in git
    ├── config.json     password hash, Hugging Face token (chmod 600)
    ├── models/         LLM weights, by Hugging Face repo
    ├── comfyui/        ComfyUI install, model packs, workflows, outputs
    ├── open-webui/     chat history and accounts
    ├── cache/          vLLM compile cache, so later starts are faster
    ├── installed/      one marker file per finished download
    ├── tuned.json      auto-tune results for your GPU
    ├── catalog.json    newest model list from GitHub, fetched by auto-tune
    ├── engine.log      Mac only: output of the running llama.cpp or ComfyUI
    └── releases.json   recent Hugging Face models with no recipe yet

Add an entry to recipes.json and restart the service (sudo systemctl restart rigup). Look at a similar existing entry and copy its shape:

• vLLM models list the Hugging Face repo ({"repo": "org/name"}) and the vLLM flags in args.
• llama.cpp models list each GGUF file ({"url": ..., "to": "models/org/name/file.gguf"}) and the llama-server flags.
• ComfyUI packs list each file and the ComfyUI folder it goes in ({"url": ..., "dir": "diffusion_models"}).
• fit says which card sizes it's meant for and how sure you are.

The dashboard can start, stop and delete things, so it always asks for a password. It uses HTTP basic auth without TLS, which is fine on a home network or over a VPN such as Tailscale. Don't expose port 8090 to the internet.

ComfyUI (:8188) has no login of its own, and the LLM API (:8000) only checks a key if you set one in the dashboard (Open panel, LLM API key), so keep the server on a trusted network or behind a firewall. On Linux, after setting a key, enter the same key in Open WebUI under Admin Settings > Connections, or its chat stops getting answers. Open WebUI (:3000) has its own accounts; the first account you create becomes the admin.

Auto-tune is the only feature that contacts anything besides Hugging Face downloads. It reads recipes.json from this GitHub repo and the public Hugging Face model list. New models from the GitHub list run with the launch flags written there, the same as if you had run git pull. Recipes you already have only change when you update your copy.

• A model fails to start: the Console shows the engine's last log lines. Out-of-memory usually means the context is too long for your card; lower -c (llama.cpp) or --max-model-len (vLLM) in recipes.json.
• Service logs: journalctl -u rigup -f
• Something else is using the GPU: nvidia-smi lists the processes. RigUp only stops its own containers (rigup-llm, rigup-comfy).
• Download stopped halfway: press the button again. Downloads resume where they stopped.

sudo systemctl disable --now rigup
sudo rm /etc/systemd/system/rigup.service
docker rm -f rigup-llm rigup-comfy rigup-webui
sudo rm -rf data/       # deletes every downloaded model and chat history (sudo: the engines write some files as root)

Docker, the NVIDIA driver and the container toolkit stay installed.