jwuthri-cloud/SelfJev

Website   ·   Get started   ·   PyPI   ·   Download the model   ·   Evaluation dataset   ·   API reference   ·   Research notebook

SelfJev turns text and questions into decisions your code can use. Route a request, check an AI response, or apply a policy. Get typed answers and probabilities from a 4B model running on infrastructure you control.

The default engine reads a document once and shares that computation across its questions and answer choices. It scores the choices directly, without generating a written response. A Python SDK and Jev-compatible HTTP API make it straightforward to add to an application.

Four answer types cover the decisions an application needs:

You define the questions, options and criteria. SelfJev evaluates them against your text. Run several questions together to reuse the document computation.

SelfJev is self-hosted: you run the model server on your own GPU, and your application talks to it. There is no SelfJev cloud.

On a Linux machine with an NVIDIA GPU and Python 3.12+:

pip install "selfjevai-sdk[serve,gpu]"

# Choose your own server secret; use the same value in the client.
export SELFJEV_API_KEYS="$(python -c 'import secrets; print(secrets.token_urlsafe(32))')"
selfjev serve --host 0.0.0.0 --port 8000

The first start downloads the selfjev-4b vision adapter (230 MB; text and images) and its pinned Qwen3.5-4B base from Hugging Face. --adapter <dir or Hugging Face repo> serves another adapter, such as your own fine-tune.

A 24 GB NVIDIA GPU, 16–32 GB host RAM and 50 GB free disk is a practical starting configuration, not a measured minimum. Memory use depends on text length, question count and concurrency. 8 GB card (RTX 4060 class): pre-quantized 4-bit checkpoint (3.3 GB): selfjev serve --quantized-model Jwuthrich/selfjev-4b-vision-4bit, ≈ 1 point below bf16 on Decision Bench, texts up to 16K tokens (measurements); the 8-bit build needs ≈ 10 GB for long texts. Needs selfjevai-sdk[serve,gpu,quant] from GitHub master until the next PyPI release. Hardware guide → · Docker, AWS, Runpod and GCP: deployment guide →

Code written for TypeSafe's Python SDK (typesafe-sdk) runs unchanged against your server:

export TYPESAFE_BASE_URL="http://your-selfjev-host:8000"
export TYPESAFE_API_KEY="the key you set in SELFJEV_API_KEYS"

from typesafe_sdk import Choice, Noul, TypeSafeClient

client = TypeSafeClient()  # reads the two variables above
res = client.system_one(
    state="I was charged twice. Please refund the duplicate payment.",
    questions={
        "refund": Noul(instructions="Does the customer want a refund?"),
        "team": Choice(instructions="Which team?", criteria={"billing": "payments and refunds", "support": "technical issues"}),
    },
)

The default model name jev-latest is answered by selfjev-4b; client.models.list(), errors and retries behave as they do against Jev. OpenRouter's decisions path (/api/alpha/decisions) is served too.

It adds Multi (select all that apply) and the fine-tuning API:

pip install selfjevai-sdk

from selfjev import Choice, Multi, Noul, SelfJev

client = SelfJev(
    base_url="http://your-selfjev-host:8000",
    api_key="the key you set in SELFJEV_API_KEYS",
)

result = client.system_one(
    state="I was charged twice. Please refund the duplicate payment.",
    questions={
        "refund": Noul("Does the customer want a refund?"),
        "team": Choice(
            "Which team should handle this?",
            {
                "billing": "payments and refunds",
                "support": "technical issues",
            },
        ),
        "topics": Multi(
            "Which topics are mentioned?",
            {
                "payment": "a payment or charge",
                "refund": "a refund request",
                "login": "an account access problem",
            },
        ),
    },
)

print(result.nouls["refund"].noul)  # Probability of yes
print(result.choices["team"].choice)  # Selected team
print(result.multis["topics"].multi)  # Selected topics

AsyncSelfJev supports the same interface with await; SELFJEV_BASE_URL and SELFJEV_API_KEY work in place of the arguments. Requests, responses and authentication →

SelfJev-4B combines Qwen3.5-4B with a trained rank-64 LoRA adapter. Its shared-prefix tree reuses the document and question computations while keeping answer branches separate. Every choice is judged with the alternatives visible; the resulting scores become the typed answers above.

The default TreeServer runs this tree directly. The vLLM backend uses the full merged checkpoint and its own prefix cache. Their performance depends on the workload. Architecture → · Measured latency and hardware →

Current SelfJev-4B served by TreeServer, compared with Jev on the same questions:

Accuracy means matching the expected answer; select-all questions require the entire set to match. These are project evaluations, not a universal model ranking. The first two suites have AI-authored, AI-checked labels and have informed research decisions. The broader benchmark was reused during development. A fresh independent holdout remains necessary.

The original evaluation engine recorded 95.8% on Text Decisions; the table uses the current serving-engine result. Comparison sources · All experiments and limitations · Findings

Measured 2026-09-30. Eight open "Jev-like" decision models from Hugging Face and SelfJev were each served with their card's recommended setup on the same GPU (one NVIDIA L40S) and asked exactly the requests TypeSafe's Jev receives. A request a model cannot answer counts as wrong. Jev is shown for reference.

SelfJev's test suites. Text Decisions (1,991 questions) and AI Response Review (946), published as selfjev-decision-bench, and photos:

• Every model gets the same Jev-shaped questions; select-all questions become one yes/no per option, which costs SelfJev 1.4–1.8 points against its native evaluation (96.1 and 92.5). Held-out images: the 4 image datasets of the SelfJev image test that none of these models trained on (772 questions). Times: each suite end to end, 4 requests in flight; SelfJev 0.4.2 was measured 2026-10-05 on the same GPU model, and the row for version 0.4.1 is the same weights on the engine before it. ² Refuses inputs over 8,192 tokens (2 % and 1 % of the questions), counted wrong.
• No other model trained on these suites, but they come from the same authors and judges as SelfJev's training data, so they favour SelfJev.
• SelfJev is the most accurate open model at 4.5B parameters or less on both suites (paired tests, p ≤ 0.023) and ties imajev-4b on held-out images; the 27B openjev is higher on Text Decisions. Among the 4B models only jpt-4b is faster on Text Decisions, and kev-4b and jpt-4b on AI Response Review.

Protocol, per-model setups, paired tests and raw reports: selfjev.dev.

Decision Bench contains Text Decisions, AI Response Review, and a smaller Record Reasoning challenge. The 3,471-question development benchmark above is separate. Each release card documents its provenance, intended use and current licensing information.

Adapt the model to your vocabulary, policies and edge cases with your own labeled examples:

uv run selfjev finetune --data my_train.jsonl \
  --out runs/mine --init Jwuthrich/selfjev-4b-vision

selfjev rlcd adds calibration-oriented training. You can also enable fine-tuning jobs over HTTP with selfjev serve --fine-tuning. Fine-tuning guide →

uv sync
uv run pytest -q
uv run pre-commit install

SelfJev is an independent project implementing a Jev-compatible interface. It is not affiliated with TypeSafe or Qwen. Training provenance, including the use of Jev probabilities as a teacher signal, is documented in the model card.