Mnemetric
DocsView results

Research preview · operator-run

Memory benchmarks, with evidence.

Evidence for AI memory. Compare measured results and inspect the traces behind every number.

View resultsSystem landscapeRead the docs

Latest verified run

mnemosyne-local · longmemeval-retrieval

operator-run

2026-10-04-longmemeval-retrieval-b0cdbd89

nDCG@529.7%
Recall@528.1%
Inspect run
Verified results1Signed and digest-checked
Systems measured1Under the public harness
Benchmark families14Tracked in the catalog
Whole-memory modules2024 capabilities mapped

Measured results

Run by our harness and checked against their evidence before this page was built.

Compare runs

longmemeval-retrieval

retrieval · mnemosyne-local

longmemeval-retrieval-v1
0%25%50%75%100%29.7%nDCG@528.1%Recall@5
longmemeval-retrieval longmemeval-retrieval-v1: reported values with intervals
nDCG@529.7% (interval 26.4% to 33.0%)
Recall@528.1% (interval 24.9% to 31.4%)

Bars show the reported value; whiskers show the interval the run supplied. Violet marks the operator entry.

How to read these charts

Finding evidence and answering well are different things, so they never share a chart.

BarThe number the run reported
WhiskersHow much it could vary
VioletMnemosyne, our own system
MissingNot measured, which is not the same as zero
Reading results

Published elsewhere

What other studies reported. Quoted, not measured by us.

Open the landscape

Answer accuracy

LoCoMo · higher is better

Run by Mem0
0%25%50%75%100%72.9%Full context (no memory)68.44%Mem0 graph66.88%Mem065.99%Zep58.1%LangMem52.9%OpenAI memory48.38%A-Memn/aMnemosyne
LoCoMo LLM-judged accuracy from one study authored by Mem0, higher is better
Full context (no memory)72.9%
Mem0 graph68.44%
Mem066.88%
Zep65.99%
LangMem58.1%
OpenAI memory52.9%
A-Mem48.38%
Mnemosyneno measurement

From the Mem0 paper, which Mem0 wrote and which scores its rivals. Zep disputes its number. We skip LoCoMo: its answer key is unreliable.

Speed

Seconds per query · lower is better

Run by Mem0
0s5s10s15s20s0.47 sOpenAI memory0.71 sMem01.09 sMem0 graph1.29 sZep1.41 sA-Mem9.87 sFull context (no memory)18.53 sLangMemn/aMnemosyne
Typical total latency per query, lower is better
OpenAI memory0.47 s
Mem00.71 s
Mem0 graph1.09 s
Zep1.29 s
A-Mem1.41 s
Full context (no memory)9.87 s
LangMem18.53 s
Mnemosyneno measurement

A typical query, search and answer together. We have not timed Mnemosyne yet.

Every published claim

Figures from the leading memory systems, open and closed, each with its source, commit and line.

All results

Every metric we have measured, with its range and evidence.

Download JSON
SystemBenchmarkMetricResultEvidence
mnemosyne-localregistered-operator-retrievallongmemeval-retrievallongmemeval-retrieval-v1recall_at_5retrieval0.2806 ratioInterval: 0.24880000000000002 to 0.31373333333333336View run 2026-10-04-longmemeval-retrieval-b0cdbd89operator-run; not publishableOperator: Mnemosyne project
Run disclosures
Systemmnemosyne-local
Trackregistered-operator-retrieval
Benchmarklongmemeval-retrieval longmemeval-retrieval-v1
Publicationoperator-run; not publishable
OperatorMnemosyne project; disclosed: true

Metrics

  • retrieval: ndcg_at_5 = 0.2967188496001503 ratioInterval: 0.26378166565316336 to 0.33038741454372256
  • retrieval: recall_at_5 = 0.2806 ratioInterval: 0.24880000000000002 to 0.31373333333333336

Immutable artifacts

run_commitb0cdbd8986b94c91a3bf20833ed33f405612025a
build_fingerprintsha256:7ba4789e105be00461f40cdf5c3b84e29785b2234839976e3a20ef8d954b079c
config_digestsha256:3cad9149f630c8c2d610ad87d9203e32713d071df6e96bd918c610291d3492a9
bundle_digestsha256:c8449069cc96e7be7700283fec67d5ed53150c37d6ded0af634535c04e99f175
trace_index_digestsha256:7146385bdc49de2c0353dd91689c08a4a60b74083de39f2bd5a7e802efabf20a
mnemosyne-localregistered-operator-retrievallongmemeval-retrievallongmemeval-retrieval-v1ndcg_at_5retrieval0.2967188496001503 ratioInterval: 0.26378166565316336 to 0.33038741454372256View run 2026-10-04-longmemeval-retrieval-b0cdbd89operator-run; not publishableOperator: Mnemosyne project
Run disclosures
Systemmnemosyne-local
Trackregistered-operator-retrieval
Benchmarklongmemeval-retrieval longmemeval-retrieval-v1
Publicationoperator-run; not publishable
OperatorMnemosyne project; disclosed: true

Metrics

  • retrieval: ndcg_at_5 = 0.2967188496001503 ratioInterval: 0.26378166565316336 to 0.33038741454372256
  • retrieval: recall_at_5 = 0.2806 ratioInterval: 0.24880000000000002 to 0.31373333333333336

Immutable artifacts

run_commitb0cdbd8986b94c91a3bf20833ed33f405612025a
build_fingerprintsha256:7ba4789e105be00461f40cdf5c3b84e29785b2234839976e3a20ef8d954b079c
config_digestsha256:3cad9149f630c8c2d610ad87d9203e32713d071df6e96bd918c610291d3492a9
bundle_digestsha256:c8449069cc96e7be7700283fec67d5ed53150c37d6ded0af634535c04e99f175
trace_index_digestsha256:7146385bdc49de2c0353dd91689c08a4a60b74083de39f2bd5a7e802efabf20a

Benchmarks

14 families, from LongMemEval to BEAM, with honest stage labels.

Coverage map

24 memory capabilities across 20 modules.

Raw evidence

Experiments and failed attempts, with every file downloadable.

Documentation

Methodology, protocols and how to reproduce a run.