Recall@5
0.0%
Share of the answer’s conversations found in the first 5 results.
LongMemEval
Make0 on LongMemEval.
LongMemEval tests whether a memory can find the right past conversation for a question asked much later. Across all 500 questions, Make0 finds 95.6% of the conversations that hold the answer in its first 5 results, and 96.2% in its first 10. These are Make0’s live numbers. LoCoMo and SWE-ContextBench results are on their own pages.
Recall@10
0.0%
Share of the answer’s conversations found in the first 10 results.
MRR
0.0%
How close to rank 1 the right conversation lands.
500 questions · evaluated 1 Nov 2026
Recall@10
Found in the first 10 results
Share of questions, by type, where the conversation that holds the answer is in Make0’s first 10 results. Higher bars are better.
LongMemEval Recall@10 by question type for Make0
MRR
How close to the top it lands
Mean reciprocal rank by question type. 100% means the right conversation is always result 1; 50% means it is result 2 on average. Higher bars are better.
LongMemEval mean reciprocal rank by question type for Make0
What it means
What each question type tests
- Single-session user
- A fact the user said once, in one conversation.
- 100% in top 10
- Single-session assistant
- Something the assistant said, recalled later.
- 100% in top 10
- Knowledge update
- A fact that changed, where the latest version is the answer.
- 99.4% in top 10
- Single-session preference
- A taste or preference the user implied, never stated as a fact.
- 96.7% in top 10
- Temporal
- When something happened, or what came before or after.
- 93.5% in top 10
- Multi-session
- An answer spread across several separate conversations.
- 93.4% in top 10
Preference questions are the hardest: the user never states the preference outright, so the right conversation is found but often lower in the list (84% MRR).
Stats for nerds
Exact numbers and how they are scored
+
Stats for nerds
Exact numbers and how they are scored
- Questions
- All 500, including the 30 abstention questions, which ask about something close to what was said but never said
- What counts as a hit
- A result from an evidence session, the conversation LongMemEval marks as holding the answer
- Recall@k
- Share of a question’s evidence sessions found in the first k results, averaged over questions
- MRR
- Mean of 1 ÷ rank of the first evidence session; 0 when none is returned
| Type | Questions | Recall@5 | Recall@10 | MRR |
|---|---|---|---|---|
| Single-session user | 70 | — | 100.00% | 92.97% |
| Single-session assistant | 56 | — | 100.00% | 99.11% |
| Knowledge update | 78 | — | 99.36% | 98.08% |
| Single-session preference | 30 | — | 96.67% | 84.00% |
| Temporal | 133 | — | 93.46% | 96.53% |
| Multi-session | 133 | — | 93.44% | 96.58% |
| AbstentionIncluded in the types above | 30 | 88.06% | 88.89% | 89.00% |
| All questions | 500 | 95.58% | 96.21% | 95.82% |
Each type includes its abstention questions, weighted by question count. Recall@5 is not reported per type. This is retrieval of the evidence sessions, not LongMemEval’s answer-accuracy score. Evaluated 1 Nov 2026. Benchmark: LongMemEval.
Start with one Brain
Free to startBuild your Brain in two minutes
Sign up, add a few memories, and connect your first agent. That's the whole setup.
Your BrainClaudeChatGPT
Cursor
Grok BotMake0 Agent