Turn an AI video of your own 3D model into an animation of your own rig in Blender.

New here? Start with TUTORIAL.md: every step in order, from your model to an animation in your game, with pictures of what the start picture and a good video look like and how to write the video's prompt.

The tool renders a start picture of your rigged model, an image-to-video generator animates it from that first frame ("he lifts the log onto his shoulder", "the giant staggers back"), and video2rig rebuilds that motion on your skeleton, frame by frame, with no hand-placed points. The result is an ordinary Blender Action you can edit, retime and export (glTF, FBX).

The video generator is your choice: anything that makes a video from a given first frame and keeps it. We made and tested every clip with Seedance (ByteDance; on Dreamina, Topview or the BytePlus API), so the examples here are Seedance's; Kling, Veo, Runway, Hailuo and others with a first-frame mode work the same way in principle.

How: once per creature, the tool trains a small joint reader on renders of your model in random poses through the start picture's own cameras. After that, every clip is read joint by joint, in every frame and every view, and the rig is fitted to those joints (docs/HOW-IT-WORKS.md).

• Your own armature: knees and elbows are found by their names (Mixamo, Rigify, Unreal and common names) or set in creature.json. Tested on a Mixamo person and on custom rigs (a troll, wolves, a centaur, a wyvern, the boss above).
• No skeleton yet? The auto-rig makes one (UniRig + checks on the mesh) and skins the model.
• Props come along: a log, a stone, a weapon is a rigid object tracked together with the character.
• Two cameras in one frame (front | side) when depth matters; one camera works too.
• The tool runs locally on your own computer: no cloud, no account, no API key (only the video generator is online; the reader's backbone and the tracker's weights are downloaded once).

Made for the Can It Code? channel (YouTube). Tested on a giant troll (two-camera clips: stagger, roar, throw, combo, walk, run), wolves (walk, run, bite, howl), the boss of the release video (idle, roar, combo, stagger, gore, death) and a man lifting a log onto his shoulder and a stone to his chest (one camera, with the prop: through the camera both match; from the side the log rests on his shoulder, but the stone ends up beside his hip. Film props held in front of the body with two views).

Install (Windows PowerShell; the same with pip elsewhere):

python -m pip install -r requirements.txt
python -m pip install torch --index-url https://download.pytorch.org/whl/cu128   # PyTorch with CUDA
winget install Gyan.FFmpeg                                                      # or any ffmpeg on PATH

The joint reader runs in the same Python as the tool, so PyTorch must be installed there. Only the older road's tracker can use PyTorch from another Python (a separate venv): $env:V2R_TORCH_PYTHON = "C:\path\to\venv\Scripts\python.exe".

# 0. A folder for the creature with its settings file ('{}' is enough; docs/CONFIG.md)
mkdir my_troll; Set-Content -Encoding ascii my_troll/creature.json '{}'
# 1. Blender: export the rig (armature + skinned meshes + props) in the pose you will start from
blender model.blend --background --python blender/export_rig.py -- --out my_troll/rig
# 2. Blender: render the start picture for the video generator (front | side) and its cameras
blender model.blend --background --python blender/render_start.py -- --out my_troll/start
# 3. Once per creature: train its joint reader on renders of it through those cameras (Blender renders in the
#    background, then the graphics card trains; about 40 minutes on an RTX 4070 with --steps 9000)
python -m video2rig.reader.prepare my_troll --blend model.blend --steps 9000
# 4. The video generator: my_troll/start/start.png as the FIRST FRAME, write the move, save the clip as
#    my_troll/clips/stagger.mp4      (how to write prompts that work: docs/VIDEO.md)
# 5. The tool: one command
python -m video2rig run my_troll stagger
# 6. Blender: the motion onto the rig as an Action
blender model.blend --background --python blender/import_motion.py -- --motion my_troll/out/stagger.npz --save my_troll/stagger.blend

Do not skip step 3. run uses the reader when my_troll/reader/reader.pt exists. Without it, it takes the older road (a point tracker and a whole-clip solve), which is far less accurate: on the bench's wolf gallop 21 degrees per bone against about 3 with the reader. A creature needs step 3 again only when its model, rig or start picture changes; one reader reads every move of it.

Step 5 prints what it does and how long it took (about a minute for a 5 s clip), and writes my_troll/out/stagger.npz plus a quality report. Before it, python -m video2rig check my_troll stagger says in seconds whether the generator drew the same motion in both views (FAIL: generate the clip again). After it, python -m video2rig review my_troll stagger makes a film of the video next to the fit (through the same camera) and from a three-quarter view, to check it before you use it.

Read next:

• TUTORIAL.md: the whole road, in order, with pictures.
• docs/WORKFLOW.md: every step in detail, the project folder, what to check.
• docs/VIDEO.md: start pictures and prompts that give clips the tool can read.
• docs/CONFIG.md: creature.json (joints, limits, smoothing) and the reader fit's settings, when the defaults are not enough.
• docs/HOW-IT-WORKS.md: the method (the joint reader), what it can and cannot see.
• docs/TROUBLESHOOTING.md: graphics card memory, timing, wrong legs, sliding feet.

Copyright (C) 2026 Can It Code. This tool is free software under the GNU Affero General Public License v3.0 (LICENSE). In plain words:

• Use it for anything, commercial games included. The animations you make with it are yours.
• Change it and share it, keeping this licence and the copyright notice.
• Nobody can turn it into a closed paid tool. Whoever shares a changed version, or offers it to other people as a program, an app or a website (paid or free), must publish their complete source code under this same licence, free for everyone to use.

The point tracker in third_party/tapnet and third_party/einshape is Google DeepMind's code under Apache 2.0 (licence files next to it; local changes are listed in third_party/README.md); its weights are Apache 2.0 too. The joint reader is built on Meta's DINOv2-small (Apache 2.0), downloaded from the Hugging Face hub on first use. UniRig (the auto-rig's skeleton model) is VAST-AI-Research's, MIT, installed separately.