Getting started

Quickstart

Turn an annotated recording into a memory graph and a judge report. Start with an existing recording, or create the sample below.

No recording yet? Install MemUse in step 1, then follow Create a sample recording before step 2.

Before you begin

TaskRequirements
Inspect a recording offlineGit, uv, Python 3.11+, and schema-10 recordings with memory_* annotations.
Run the judgeNode.js 22.19+, npm, and a judge endpoint/key.
Create the optional sampleDocker, an agent model endpoint/key, and a mem0 Platform account/key. The recipe installs memory-bridge and traj-recorder.

Keep the complete recording directory, including its call captures. A plain model transcript cannot reconstruct memory annotations.

Cloning and dependency installation need network access. Graph rendering and evidence preparation make no model calls. Recording a live task and running the judge can incur charges.

1. Install MemUse

Run from the directory where you keep your checkouts:

git clone --branch agentic_search https://github.com/ChangranXU/mem-use.git mem-use
cd mem-use
uv sync

All evaluation commands below run from the mem-use folder.

2. Inspect the memory graph

Replace the example path with your recorder directory, a trajectory.jsonl file, or a directory of recordings:

REC=/absolute/path/to/recording
uv run -m memgraph "$REC" -o memgraph.html --json memgraph.json

Open memgraph.html in a browser. Check the validation notes, then select a memory to trace its creation, updates, recalls, and deletion. Upward triangles mark writes; downward triangles mark recalls.

Choose the right input for a resumed run

When the runner provides manifests, use evaluation.json for checkpoint-credited history. Use evaluation-attempts.json for all attempts, or evaluation-discarded.json for interrupted or uncheckpointed work. Load one view at a time; do not combine it with a recursive scan of the same recordings.

See the input guide for supported views.

3. Check the judge input offline

uv run -m injudge "$REC" --dry-run --limit 3
uv run -m injudge "$REC" --prepare-only -o output/prepared/

--dry-run prints evidence for up to three eligible spans and writes no files. --prepare-only saves episode evidence under output/prepared/<run-stamp>/. Neither needs Node.js or judge credentials.

Review diagnostics before paying for a judgment. Missing episode boundaries or gaps in a delivery’s consuming input can block that episode. The method guide defines episodes, entries, and exposure.

4. Judge a small sample

Install the judge dependencies and copy its configuration template. Keep an existing configured .env file:

npm --prefix agent install
test -f .env || cp .env.example .env

Edit these values in .env:

.env
MODEL="your-judge-model"
API_KEY="your-judge-api-key"
BASE_URL="https://your-endpoint.example"
API="openai-chat"

For an OpenAI-compatible endpoint, use the base URL that serves /chat/completions. For Anthropic, set API="anthropic" and use its API host root. See configuration for budgets and concurrency.

Makes judge model calls
uv run -m injudge "$REC" --limit 3 --html -o output/

Review the sample before removing --limit 3. To select episodes, add --episode TASK_ID with an exact recorded ID; the flag is repeatable.

No reference answer or task score is required. To attach them, use the gold and outcome guide.

5. Inspect one judgment

Each judge run creates a new output/<run-stamp>/ folder:

OutputUse
summary.mdStart with the counts, coverage, diagnostics, and suggested spans to review.
memgraph-verdicts.htmlOpen a recall to inspect its entry signals, verdict, and cited steps.
injudge-report.json / summary.jsonRead the full record or compact statistics programmatically.
evidence/ / agent-runs/Audit the files the judge could read and its recorded conversation.

Choose a non-neutral span, read its analysis, and follow the cited steps. Check unknown values, errors, and unjudged spans before comparing runs. Use verdict rules and entry-share profiles to interpret the result.

Render an existing report again

Use the same recording view and replace RUN with the report folder:

uv run -m memgraph "$REC" \
  --verdicts output/RUN/injudge-report.json -o memgraph-verdicts.html

Create a sample recording

This optional path runs MiniSWE with mem0 Platform on a small SWE-bench sample. It needs Docker and separate credentials for the agent and mem0. For other products or task suites, use the runner catalog.

Install the recording projects

Start from the parent directory containing your mem-use checkout. If your terminal is still in mem-use after step 1, run cd .. first.

mkdir -p extension
git clone --branch v2 https://github.com/ChangranXU/memory-bridge.git
git clone --branch memory https://github.com/ChangranXU/traj-recorder.git extension/traj-recorder
cd memory-bridge
git clone https://github.com/SWE-agent/mini-swe-agent.git mini-swe-agent
git -C mini-swe-agent checkout 04d809ceab9df28f9adaed044884180159172930
uv sync

The runner starts the recorder from the sibling checkout and resolves its dependencies. Keep running the following commands from memory-bridge.

Configure and check

test -f settings/shared/llm.env || cp settings/shared/llm.env.example settings/shared/llm.env
test -f settings/recorder/roles.env || cp settings/recorder/roles.env.example settings/recorder/roles.env
test -f settings/memory/mem0.env || cp settings/memory/mem0.env.example settings/memory/mem0.env

Fill in the agent endpoint, model, and key in settings/shared/llm.env. Set MEM0_OPTION=cloud and MEM0_API_KEY in settings/memory/mem0.env. Check the applicable model card and required settings in the mem0 setup guide. Keep existing settings files if already configured.

uv run python scripts/check_settings.py --provider mem0 --mem0-option cloud

This check starts no task or paid model call. For a credentials-free API exercise, use the offline bridge example.

Run and verify

Runs Docker tasks and calls the agent and memory service
uv run python scripts/swebench/swe-mini-agent/run.py \
  --ids scripts/swebench/swe-mini-agent/ids.smoke.txt \
  --output runs/mem0-smoke --provider mem0 --mem0-option cloud --strict

Use a new output directory. Inspect runs/mem0-smoke/verify.log and summary.json for trace and memory failures. A completed task alone does not verify memory use.

Return to mem-use with cd ../mem-use. Continue at step 2, setting REC=../memory-bridge/runs/mem0-smoke/recorder.

Record without memory-bridge

traj-recorder can record compatible model traffic on its own. Your client must use the proxy endpoint, and a host must supply annotations for memory analysis. Follow the standalone recorder quickstart.

Common problems

The memory graph is empty

Check that you selected the intended recording and that it contains memory_* events. A none baseline or standalone unannotated proxy recording correctly has no memory operations.

A recall shows “created earlier”

The loaded recordings contain a recall but no observed creation for that item. Load earlier runs if you need that history.

Spans are unassessable

No observed agent step can be attributed as the delivery’s consumer. This is missing consumption evidence, not a judgment that the agent ignored the memory.

injudge exits with status 2

Read the console diagnostics. A normal judge run can also report span errors in summary.md. Offline preparation writes diagnostics to its evidence package; dry-run creates no report.