CONSONANCE.for your information
Monday, 5 October 2026frenvi

Worth reading closely

01 — agents 69 upvotes

Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents

QUESTION — How can language models estimate output confidence more reliably by leveraging accumulated past experiences rather than relying solely on the current inference process?

This work introduces XConf (eXperiential Confidence), a confidence estimation method that stores past graded episodes containing tasks, reflections, stated confidence, outcomes, and lessons. Given a new task, XConf recalls similar past episodes to retrieve historical success rates and prompts the model to identify recurring failure modes and restate its confidence. Operating without logit access or weight updates, XConf beats or matches ten-sample self-consistency in AUROC on 23 of 24 comparisons across nine benchmarks, significantly lowers ECE, and raises delivered success rates on agent tasks by up to 8.7 points when abstaining on the 10% least-confident episodes.

XConf beats or matches ten-sample self-consistency in discrimination (AUROC) on 23 of 24 comparisons.

Abstaining on the 10% least-confident episodes raises the delivered success rate by up to 8.7 points on agent tasks.

caiqizh · 15 Sept 2026 read the original ↗
↑