A measured ranking of shared-memory agents under utility, access-control, and active-forgetting constraints.
GATEMEM evaluates memory agents in multi-principal environments where the same memory pool must support authorized recall, enforce contextual access boundaries, and honor deletion requests. This page presents the official benchmark ledger. The default view shows the Medical domain; use the domain tabs below to switch domains.
The leaderboard can be sorted by headline score or by individual governance dimensions. Use the domain tabs to switch between Medical, Office, Education, and Household. U denotes effective utility; A denotes access-control violation rate; F denotes active-forgetting failure rate; MGS = U · (1 − A) · (1 − F).
| Rank | System | Backbone | Domain | Status | U ↑ | A ↓ | F ↓ | MGS ↑ |
|---|
The official score is intentionally multiplicative: a system cannot compensate for leakage with high utility, nor compensate for low utility with broad refusal.
Run the official evaluator locally, then upload a complete package containing raw predictions, normalized outputs, deterministic scores, judge labels, and the summary.
Community results use contributor-run evaluation artifacts. Verified results are independently re-scored by the maintainers from the exact same predictions.
Evaluate locally with the official scripts, package the resulting artifacts, and submit them for ranking across utility, access safety, active forgetting, and MGS.
If you use GATEMEM, please cite the accompanying paper and dataset.
@misc{ren2026gatemembenchmarkingmemorygovernance,
title={GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agents},
author={Zhe Ren and Yibo Yang and Yimeng Chen and Zijun Zhao and Benshuo Fu and Zhihao Shu and Bingjie Zhang and Yangyang Xu and Dandan Guo and Shuicheng Yan},
year={2026},
eprint={2606.18829},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2606.18829},
}