Papers & Preprints

Preprint · arXiv 2026

Confidently Wrong, Silently So: Auditing Undetectable Failures of a Deployed On-Device Language Model

S. Pandey*, S. Pandey*, S. Raghu

We audit a deployed on-device language model, finding confident failures that user-visible signals cannot reliably detect while a black-box consistency wrapper substantially recovers reliability.

ICML '26 CTB · FAGEN

Proper Scoring Rules for Agentic Uncertainty Quantification

S. Raghu*, S. Pandey*, S. Pandey

A family of strictly proper trajectory-level scoring rules for evaluating uncertainty in LM agents, including a censored-trace extension and negative results for common calibration metrics.

ICML '26 FAGEN

SELFDOUBT: Uncertainty Quantification for Reasoning LLMs via the Hedge-to-Verify Ratio

S. Pandey*, S. Raghu*, S. Pandey

A black-box uncertainty framework that extracts hedge and verify signals from reasoning traces, improving discrimination at lower inference cost.

COLM '26 AIW · SCI-FM

Don't Blink: Evidence Collapse during Multimodal Reasoning

S. Raghu*, S. Pandey*

An analysis of evidence collapse in reasoning VLMs, showing how visual grounding can decay during generation even when model confidence remains high.

Preprint

Repair of Thought: Advancing Automated Program Repair through a Dual-Model Reasoning Framework

S. Pandey, et al.

A function-level automated program repair framework combining dual-model reasoning with verification through AST alignment, control-flow symbolic analysis, and semantic checks.