Noémi Ligeti-Nagy

Hi, and congratulations on the article and the dataset, great work!
That said, I’d like to clarify that your claim about the lack of Hungarian benchmarks isn’t entirely accurate. :)
We’ve recently introduced HuGME (https://hugme.nytud.hu), a comprehensive evaluation suite for generative and reasoning capabilities in Hungarian. It has just been presented at the GEM Workshop at ACL 2025, and has been actively used in benchmarking for a while now.
While not all parts of the dataset are publicly released – to preserve the integrity of future evaluations – detailed information and example tasks are available on the website. The benchmark is broader in scope and larger in scale than what’s currently described in your work.
Also, a note of caution: making evaluation data fully open can lead to rapid memorization by large models, which undermines the reliability of the benchmark. Something we’ve been careful to avoid.
If you’re interested in collaboration or Hungarian-specific benchmarking tools, feel free to get in touch. Happy to connect!