KDD 2026 · Jeju · Research Feature
Two workshop papers examine what static AI benchmarks miss: agents that change the world they observe, and systems that repeat a known failure after the first correction.
The common failure
Modern AI evaluation is unusually good at asking whether a model can answer once. It is much weaker at asking what happens after the answer changes users, markets, evidence, incentives, or the next state the model will see.
A second gap appears after failure: correcting one response does not establish that the system formed a durable, inspectable defense against recurrence. ReflexBench and Cognitive Immunity make these two gaps measurable without treating workshop acceptance as proof beyond the reported evidence.
Status: both papers were accepted at KDD 2026 workshops and presented in person at ICC Jeju. They are non-archival workshop papers and are not part of the KDD main-conference proceedings.
Paper 01 · Agentic AI evaluation
Oral presentation · KDD Workshop on Evaluation and Trustworthiness of Agentic AI · August 9, 2026
Most benchmarks treat an AI agent as an outside observer of a fixed task. Deployed agents are different: a recommendation changes users, a forecast changes incentives, an intervention changes future data, and a visible benchmark can change the system being benchmarked.
The diagnostic is a depth gap, not another winner-only score: Delta_OD = mean(s_deep) - mean(s_shallow). A large negative gap means surface competence does not survive recursive feedback.
The fixed panel contains 720 scored responses. Every tested model degraded at deeper observer levels; the rounded shallow-to-deep mean gap was about -0.44. Reasoning-specialized models reduced part of the OD-2 burden, but no tested model eliminated the OD-n gap.
Boundary: this is a language-only, stylized audit snapshot. It is not a safety certificate, a full causal-discovery method, a permanent leaderboard, or a replacement for domain simulation and human review. Full confidence intervals, inter-rater reliability, and rubric-sensitivity estimates require the retained row-level table and scorer records.
Paper 02 · Trustworthy LLM agents
Poster presentation · 2nd SeT-LLM Workshop · August 10, 2026
A fluent apology after an error is not evidence that a system learned a durable safety lesson. Cognitive Immunity adds a bounded runtime layer that converts verified failure events into scoped, reviewable rules without retraining the base model.
Each rule carries provenance, scope, strength, decay, review and deletion handles. Every intervention records which source failure matched, why the rule fired, what changed, and what happened afterward.
The protocol contains 20 task templates, five rounds, three seeds, and four strategies. It yields 300 score events per strategy and 1,200 total score events, not 1,200 independent tasks or public raw transcripts. The paired delta has a 95% bootstrap interval of [-0.211, -0.003].
This is a recurrence-reduction result, not all-metric dominance: Reflexion retained the highest WQ point estimate. The mechanism also creates its own attack surface, including memory poisoning, false intervention, overblocking, staleness, cold start, and audit gaps.
Boundary: the recovered package supports score-level recomputation, not raw-response replication or calibrated safety evidence. Cognitive Immunity targets recurrence of known observed failure classes; it does not replace red teaming, access control, sandboxing, or human oversight.
One research program
Together they reject the same shortcut: compressing a dynamic system into a one-shot answer. Reliable agents must represent the causal footprint of their outputs and retain observed failure in a form that remains scoped, reversible, and auditable.
Evidence boundaries
Jeju, August 2026
Exact answers
Yes. ReflexBench was an oral presentation at the KDD Workshop on Evaluation and Trustworthiness of Agentic AI. Cognitive Immunity was a poster presentation at the 2nd SeT-LLM Workshop. Both are non-archival workshop papers.
It measures whether an agent's competence survives when its own output changes users, institutions, evidence, incentives, or other actors, from fixed-world reasoning through recursive observer depth.
Under the documented score-level protocol, bounded runtime failure memory reduced recurrence of known observed severe-threshold failures relative to No Memory. It does not claim general or certified safety.
Both PDFs, citations, hashes, formulas, results, and a JSON facts record are available at mianzhang.org/papers/kdd-2026/.