EXIA GHOST Results: F1 71.99% — First independent evaluation (1 user)

[francisdu53]

Hello HaluMem Team,

We are submitting results for EXIA GHOST V5, a bio-inspired AI memory system with 5 specialized stores (contextual, semantic, episodic, procedural, prospective).

This is the first independent evaluation of HaluMem — all other scores in the paper were evaluated by the HaluMem team.

Results (1 user, GPT-4o judge)

Metric	Score
F1 (Extraction)	71.99%
Precision	92.90%
Recall	58.66%

Metric	Score
F1 (Update)	62.47%
Precision	73.00%
Recall	54.64%

Metric	Score
QA Accuracy	68.07%

Note: This evaluation was run on 1 user (HaluMem-Medium). A full 20-user evaluation is planned.

Verification

• Repository: https://github.com/francisdu53/exia-ghost-benchmarks
• Full results & methodology: halumem/RESULTS.md
• Adapter code: halumem/eval_exiaghost.py
• Judge: GPT-4o (official HaluMem standard)
• Raw results: included in repository (scores.json + eval_results.jsonl)

Author: Francis BABIN (Solo Developer)

Thank you for this important benchmark. We look forward to your feedback.

[hush-cd]

Thank you very much for your interest in HaluMem and for sharing the EXIA GHOST V5 results. We appreciate the detailed breakdown, the transparency of your evaluation setup, and the effort you’ve put into open-sourcing the code and logs.

It’s great to see an independent evaluation, and we especially value that you’ve included results across all three core tasks (Extraction, Updating, and QA), along with verification artifacts.

That said, for inclusion in the official leaderboard, we currently require evaluations to be conducted across the full standard setting of the benchmark. In particular, this includes:

• Multi-user evaluation (typically 20 users) rather than a single-user run
• Consistent reporting across all tasks under the same evaluation scale

The current results are based on a single-user (HaluMem-Medium) setup, which makes them difficult to compare fairly against existing leaderboard entries that follow the full protocol.

Once the planned 20-user evaluation is completed, we would be very happy to review your updated results for potential inclusion. The fact that you’ve already provided code, methodology, and raw logs is excellent and will help streamline the verification process.

Thanks again for your contribution and for engaging with the benchmark—we’re looking forward to seeing the full evaluation!