GLM-5.3-Flash (321 B) and DeepSeek-V4.1-Flash-1017 (748 B) on your own GPU
private and fast · one NVIDIA GPU or up to 16 · AMD and Windows (experimental) · chat, pictures, OpenAI- and Anthropic-compatible API · several requests at once

Created and led by Peasant Smith (@mw00), co-maintained with @needmorevram

_{Project Maya is free. If it runs well on your PC, a coffee keeps the work on it going.}

Models this size usually need a datacenter. Maya runs GLM-5.3-Flash (zai-org, MIT license; 321 B parameters, about 18 B active per token, a context of up to 1 M tokens) and DeepSeek-V4.1-Flash-1017 (DeepSeek, MIT license; 748 B parameters) on the hardware you already have. Each model has an engine built for it. It keeps the experts your conversation uses most on the GPU, the next ones in RAM and the rest on your NVMe SSD, and moves them as you chat. It measures your GPUs, CPU, RAM and SSD and tunes itself to them. Nothing leaves your machine.

• Fast: up to 124 tokens/s on two H100s, 118 on four RTX 4090s, 81 on one RTX PRO 6000, 30+ on a single RTX 5090. Real numbers from real users below.
• Close to the original: Maya's own quants of GLM-5.3-Flash keep 97.7-99.2% of the FP8 model's zero-shot accuracy; Maya-S of DeepSeek-V4.1-Flash-1017 picks the same next token as the original 92.6% of the time.
• Plugs into your tools: OpenAI- and Anthropic-compatible API with tool calls, ready for coding agents.
• Several requests at once: up to 8 answered together - for agents that run several sessions, or several people on one PC.
• Everything in the box: a chat and a live monitor in the browser, pictures, thinking levels, a one-command setup.

Project Maya's own quants of GLM-5.3-Flash, made from Z.ai's FP8 release with statistics from the FP8 model. Each keeps the model's MTP block, which drafts tokens ahead.

The setup asks which one to download and recommends one for your PC. How they're made and measured: MAYA-S, MAYA-S24, MAYA-M, MAYA-L. On Hugging Face.

The setup also offers DeepSeek-V4.1-Flash-1017 (DeepSeek, MIT license; 748 B parameters) on an engine of its own (maya-ds4): ./setup.sh --setup and pick it, or ./setup.sh --model DeepSeek-V4.1-Flash-1017.

The server uses the pinned official prompt encoder, with numeric reasoning budgets and ID-matched parallel tool results. See the prompt format notes for API details and the standalone Jinja limits.

• Any amount of memory: the most-used experts go to the GPUs, the others are read in place from the model's files - from RAM where they fit, from the SSD otherwise. More VRAM and RAM is faster; keep the files on an NVMe SSD.
• The DSpark drafter: with the GGUFs the setup also fetches the model's own drafter (8 GB from Maya's repo: DeepSeek's FP4 experts, its dense weights in 8-bit) and measures whether drafting makes your PC faster; it keeps it only where it does.
• Pictures: with the GGUFs the setup fetches DeepSeek's vision tower (1 GB of its checkpoint); it runs on the first GPU in about 320 MB. --no-vision leaves it out.
• One GPU or several: the layers are split across the GPUs (pairs of equal GPUs also split each layer where the files' formats allow), its thinking levels (Off, Low, High, Max) in the chat, and its own numbers in the monitor.
• Strix Halo / Gorgon Halo on Windows: Windows counts the GPU's share of the memory against its page file, so the setup asks for a page file big enough to keep every expert on the GPU and says the size (docs/AMD_MAYA.md).

Real machines, real numbers, most of them measured with ./maya.sh --bench. A token is about ¾ of a word.

Prefill is an 8K-token prompt. The speed holds as a conversation grows, and the first answers after a start are the slowest while the caches fill with what you use. Share yours: --bench, then --report, in a GitHub issue.

DeepSeek-V4.1-Flash-1017 (maya-ds4):

Several requests at once (--parallel) add up: on one GPU a token of each goes through the model together, so the dense weights are read once for all of them. In tokens per second for all of them together, against the same questions one after another: x1.41 for two and x1.35 for four on a V100 that streams most of Maya-S's experts from RAM, x1.67 for two and x2.02 for four on a Gorgon Halo (Radeon 8065S, 160 GB for the GPU) that holds every expert in GPU memory. The more of the model VRAM holds, the more it gains (how).

The setup checks all of this, prints the exact command for anything missing, and installs nothing system-wide.

git clone https://github.com/mw00/project-maya.git && cd project-maya
./setup.sh

Answer a few questions (Enter takes the recommended choice). The setup shows every download before it starts it, builds everything once (20-40 minutes plus the download) and starts the model. Then open http://127.0.0.1:8080. Ctrl+C stops it, and ./setup.sh starts it again right away.

• Windows: START-MAYA.bat instead of ./setup.sh - see docs/WINDOWS.md.
• AMD: docs/AMD_MAYA.md.
• Updating: the dashboard's About > Update, or git pull then ./setup.sh. The model isn't downloaded again.
• Every option (GPUs, context, a model folder you already have, remote access, ...): docs/SETUP.md.

The start takes a few minutes: the engine pins most of the free RAM (all but about 6 GB) for its expert tier, so other programs get little RAM while Maya runs.

• In the browser: chat (kept in your browser, per-chat instructions, edit and regenerate, HTML previews), a live Monitor of the model and your hardware (down to what fills each GPU's memory), and a context size and port you can change in Settings.
• Your apps and coding agents: an OpenAI-compatible API at http://127.0.0.1:8080/v1, and Anthropic's at http://127.0.0.1:8080/v1/messages (Claude Code: ANTHROPIC_BASE_URL=http://127.0.0.1:8080). Tool calls are streamed as the model writes them, and a long prompt's reading progress too when a request asks for it ("return_progress": true). A web page served from another address can call it too once you allow its address ("cors_origins", docs/SETUP.md).
• Responses API: /v1/responses supports text/image conversations, streaming, function tool calls, schema-validated output, background tasks with cancellation/replay, and optional shared in-memory history (store:true, previous_response_id). Responses are not stored by default. Options and limits: docs/RESPONSES.md.
• Thinking: Off, Low, High (the default) or Max, in Settings or per request.
• Pictures: attach one in the chat, or send image parts through either API.
• From another device: ./maya.sh --setup --host 0.0.0.0 --api-key <secret>. Always set a key.
• Tuning: ./maya.sh --calibrate tunes the engine to your hardware in 10-15 minutes. Every setting: docs/SETTINGS.md.
• DeepSeek on large-RAM Linux hosts: optional locked Engram tables and strict CPU-expert RAM placement. Both are off by default; they need enough physical RAM.
• Several requests at once (GLM-5.3-Flash): ./maya.sh --parallel 4 (Windows: START-MAYA.bat --parallel 4; up to 8) keeps it in the config, and the dashboard shows how many answers are being written. Each extra request keeps its own context in VRAM, so a few fewer experts fit there (how).
• Long contexts (GLM-5.3-Flash): new setups keep each attention layer's cache in RAM with a window of it in VRAM (--kv-streaming on; ./maya.sh --setup adds it to an existing config), so the VRAM holds more experts - at 512K on an RTX 3090, prompts 4-9x faster and decode +5%; 1M context fits one 3090 (how).
• Faster answers with a DFlash2 drafter (GLM-5.3-Flash): ./maya.sh --setup --draft dflash2 (NVIDIA RTX 30 / A100 and newer) downloads a small drafter that drafts up to 7 tokens a round and gives the same answers - Maya-L +44% on nine GPUs and +12% on two (against the MTP block), +19% on one RTX 3090 (how). How much it gains depends on the PC: on one GPU that reads much of the model from RAM it is often slower. So a speed calibration measures it on your PC and keeps it only where it is faster. The drafter's license is non-commercial (CC BY-NC-ND 4.0).

The DeepSeek-V4.1-Flash-1017 engine holds one conversation: a prompt that continues it reuses its state, anything else starts over. With this on, a conversation another request takes the model from - a chat, or an agent's parent and workers - is set aside, and a request that continues it reads only its new tokens: in RAM, or across a restart from the folder. Off by default; the config sets them under the same MAYA_DS4_* names (how):

Run ./maya.sh --report (Windows: START-MAYA.bat --report) right after a slow answer or an error, and attach the maya-report.txt it writes to a GitHub issue. It holds your hardware, your setup and the engine's speed lines; nothing is sent anywhere by itself. The engine's own log is maya-<model>.log in the Maya folder.

• "nvcc ... cannot build for these GPUs": Volta needs CUDA 12.x, Blackwell 12.8 or newer. The setup takes the newest installed toolkit that fits.

• cuda_profiler_api.h or nvtx3/nvToolsExt.h is missing: Maya's CUDA build needs the profiler API and NVTX headers as well as nvcc and the runtime. Install those development components for your toolkit. With a split CUDA 13 toolkit, NVIDIA's official header wheels can also stay in a project-local folder:

  python3 -m pip install --target .cuda-headers \
    nvidia-cuda-profiler-api==13.3.27 nvidia-nvtx==13.3.29
  CPATH="$PWD/.cuda-headers/nvidia/cu13/include${CPATH:+:$CPATH}" ./maya.sh --setup

  Select wheel versions that match your toolkit. This include layout was checked with CUDA 13.3, nvidia-cuda-profiler-api==13.3.27 and nvidia-nvtx==13.3.29; other CUDA major versions can use a different layout. Use the same CPATH when rebuilding after an update. No shared toolkit files need changing.

• "unsupported GNU version": ./maya.sh --setup --host-compiler g++-12 (install g++-12 first).

• Slow, with the SSD busy all the time: not enough free RAM for the experts. Close programs or add RAM.

• Windows: answers suddenly several times slower: the card is too full and Windows moves GPU memory to RAM. Leave more of it free: START-MAYA.bat --setup --vram-reserve-mib 2048 (the expert pool's size).

• Windows on a Strix Halo / Gorgon Halo: other programs run out of memory, or the setup warns about the page file: set the page file to the size the setup prints (docs/AMD_MAYA.md).

• The download stopped: run ./maya.sh again; it continues where it stopped.

• Port 8080 is in use: Maya is already running, or --port 8081 starts another one. Settings > Port moves Maya to another port for good (it starts again there).

Project Maya is free and open source. If it's useful to you, you can support it here - thank you!

Speed reports, bug reports and pull requests are welcome: CONTRIBUTING.md says what helps most.

• People: Project Maya is created and led by Peasant Smith (@mw00, X), who owns the project, its name and the Maya models. @needmorevram (X) is a co-maintainer and contributed major features: DFlash2 drafting, EXL3 support, KV streaming, conversation checkpoints, the layer-split search, the expert profile and the CPU expert kernels, and for maya-ds4 the 3.5-bit EXL3 option, tensor parallel over 4 / 8 GPUs and the threaded expert-cache swaps (#276). Thanks also to everyone who sent code and speed reports (contributors).
• Built on Strata (MIT License, Copyright (c) 2026 Niko1221 and the Strata contributors): Maya's engine started from Strata's and was rewritten for GLM-5.3-Flash (the expert tiers across VRAM, RAM and SSD, the layer split, MTP decoding), and its server and dashboard grew from Strata's. And on ggml / llama.cpp (MIT License, Copyright (c) 2023-2026 The ggml authors): the quantization formats, the CPU dot products and the prompt path's MMQ kernels, built from a pinned commit (third_party/ggml/LICENSE).
• EXL3 support follows exllamav3's format, codebooks, CPU expert kernel and tensor-core decoding of a tile (MIT License, Copyright (c) 2025 Turboderp; third_party/exllamav3/LICENSE). The DFlash2 drafter's forward follows beellama.cpp's (MIT License, Copyright (c) 2023-2026 The ggml authors, Anbeeld; third_party/beellama.cpp/LICENSE).
• The models: GLM-5.3-Flash by zai-org (MIT); Maya's quants and the image encoder file are made from zai-org's released weights and keep its license. DeepSeek-V4.1-Flash-1017 by DeepSeek (MIT); Maya's quant, its Engram tables and the drafter are made from DeepSeek's released weights and keep its license; the vision tower comes from DeepSeek's checkpoint, the GGUF option from unsloth's GGUFs, the EXL3 options are LibertAIDAI's (2.9 and 3.5 bpw). The optional DFlash2 drafter (GLM-5.3-Flash-DFlash2-GGUF by Anbeeld) is CC BY-NC-ND 4.0, non-commercial use only, and is downloaded only when you choose it.
• The dashboard's font: Outfit (SIL Open Font License 1.1, serve/web/fonts/OFL.txt).
• Maya is open source under the MIT License; the notices of Strata, ggml, exllamav3 and beellama.cpp stay with every copy.