README.md

metadata

title: GateMem Submit
emoji: 😻
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 6.19.0
python_version: '3.13'
app_file: app.py
pinned: false
license: mit
short_description: Online result submission and leaderboard update for GateMem.
thumbnail: >-
  https://cdn-uploads.huggingface.co/production/uploads/662df118b0d404635d7a2b46/Zjor79PUC1II3-4Hh7-Yn.jpeg

GateMem Submit

Online submission interface for GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agents.

This Space provides a public interface for submitting predictions.jsonl files to the GateMem benchmark. Submitted results are evaluated with the official GateMem evaluator and can be reviewed for inclusion in the public leaderboard.

Resources

• 📄 Paper: https://arxiv.org/abs/2606.18829
• 🌐 Project Page: https://rzhub.github.io/GateMem/project.html
• 💻 Code: https://github.com/rzhub/GateMem
• 🤗 Dataset: https://huggingface.co/datasets/Ray368/GateMem
• 🏆 Leaderboard: https://rzhub.github.io/GateMem/
• 🚀 Submit Results: https://huggingface.co/spaces/Ray368/GateMem-Submit

What is GateMem?

GateMem is a benchmark for evaluating memory governance in multi-principal shared-memory agents.

Most memory-agent benchmarks ask whether an agent can remember useful information. GateMem asks a harder question:

Can a shared-memory agent remain useful, enforce requester-specific access control, and honor deletion requests?

GateMem evaluates three dimensions:

• Utility: answering legitimate authorized requests
• Access Control: avoiding unauthorized or over-scoped disclosure
• Active Forgetting: avoiding recovery, confirmation, or reconstruction of explicitly deleted information

Submission Format

Your submission should be a JSONL file named:

predictions.jsonl

Each line should correspond to one checkpoint prediction.

For the full prediction format, see:

https://github.com/rzhub/GateMem/blob/main/docs/prediction_format.md

Evaluation and Leaderboard

Submissions are evaluated using the official GateMem evaluation protocol. Results may be reviewed before being added to the public leaderboard.

Leaderboard:

https://rzhub.github.io/GateMem/

Citation

@misc{ren2026gatemembenchmarkingmemorygovernance,
      title={GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agents}, 
      author={Zhe Ren and Yibo Yang and Yimeng Chen and Zijun Zhao and Benshuo Fu and Zhihao Shu and Bingjie Zhang and Yangyang Xu and Dandan Guo and Shuicheng Yan},
      year={2026},
      eprint={2606.18829},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2606.18829}, 
}