跳转至

ICML 2026 · 全部收录(566 篇)

仅列出通过预过滤且已打分的论文;分数是「评测方法学价值」,不是会议录用质量。 返回总览

# 论文 轨道 分数 链接
1 Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarks Main 10 页 · PDF
2 Diagnosing the Reliability of LLM-as-a-Judge via Item Response Theory Main 10 页 · PDF
3 A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness Main 9 页 · PDF
4 A Judge-Aware Ranking Framework for Evaluating Large Language Models without Ground Truth Main 9 页 · PDF
5 AgentSuite: Toward More Reliable Agent Evaluation with a Component-Based Benchmark Auditing Pipeline Main 9 页 · PDF
6 AI Cartography: Mapping the Latent Landscape of AI Benchmark Ecosystems Main 9 页 · PDF
7 Are LLM Evaluators Really Narcissists? Sanity Checking Self-Preference Evaluations Main 9 页 · PDF
8 Benchmarking at the Edge of Comprehension Main 9 页 · PDF
9 Benchmarking World-Model Learning with Environment-Level Queries Main 9 页 · PDF
10 CapBencher: Give Your LLM Benchmark a Built-in Alarm for Test-Set Overfitting Main 9 页 · PDF
11 CHB: A Diagnostic Toolkit for Hardness-Aware Clustering Evaluation Main 9 页 · PDF
12 Coverage, Not Averages: Semantic Stratification for Trustworthy Retrieval Evaluation Main 9 页 · PDF
13 Cutting LLM Evaluation Costs with SySRs: A Bandit Algorithm That Provably Exploits Model Similarity Main 9 页 · PDF
14 DEER: A Benchmark for Evaluating Deep Research Agents on Expert Report Generation Main 9 页 · PDF
15 Dependence-Aware Label Aggregation for LLM-as-a-Judge via Ising Models Main 9 页 · PDF
16 DSGym: A Holistic Framework for Evaluating and Training Data Science Agents Main 9 页 · PDF
17 Efficient Inference for Noisy LLM-as-a-Judge Evaluation Main 9 页 · PDF
18 Evaluating LLMs When They Do Not Know the Answer: Statistical Evaluation of Mathematical Reasoning via Comparative Signals Main 9 页 · PDF
19 Expanding the AI Evaluation Toolbox with Statistical Models Main 9 页 · PDF
20 FairJudge : An Adaptive, Debiased, and Consistent LLM-as-a-Judge Main 9 页 · PDF
21 Faults in Our Formal Benchmarking: Dataset Defects and Evaluation Failures in Lean Theorem Proving Main 9 页 · PDF
22 How to Correctly Report LLM-as-a-Judge Evaluations Main 9 页 · PDF
23 Instance-Level Costs for Nuanced Classifier Evaluation Main 9 页 · PDF
24 Is Your Diffusion Sampler Actually Correct? A Sampler-Centric Evaluation of Discrete Diffusion Language Models Main 9 页 · PDF
25 Item Response Scaling Laws: A Measurement Theory Approach for Efficient and Generalizable Neural Scaling Estimation Main 9 页 · PDF
26 Judging What We Cannot Solve: A Consequence-Based Approach for Oracle-Free Evaluation of Research-Level Math Main 9 页 · PDF
27 Leaderboard Incentives: Model Rankings under Strategic Post-Training Main 9 页 · PDF
28 Margin-Adaptive Confidence Ranking for Reliable LLM Judgement Main 9 页 · PDF
29 Mitigating Perceptual Judgment Bias in Multimodal LLM-as-a-Judge via Perceptual Perturbation and Reward Modeling Main 9 页 · PDF
30 MMBench-Live: A Continuously Evolving Benchmark for Multimodal Models Main 9 页 · PDF
31 Needles in the Haystack: Addressing Signal Dilution Improves scRNA-seq Perturbation Response Modeling and Evaluation Main 9 页 · PDF
32 Nonparametric LLM Evaluation from Preference Data Main 9 页 · PDF
33 PerceptionRubrics: Calibrating Multimodal Evaluation to Human Perception Main 9 页 · PDF
34 Pluralistic Leaderboards Main 9 页 · PDF
35 Position: AI Evaluations Should be Grounded on a Theory of Capability Main 9 页 · PDF
36 Position: LLM Benchmark Datasets Should Be Contamination-Resistant Main 9 页 · PDF
37 Pressure Reveals Character: Behavioural Alignment Evaluation at Depth Main 9 页 · PDF
38 QEDBench: Quantifying the Alignment Gap in Automated Evaluation of University-Level Mathematical Proofs Main 9 页 · PDF
39 Quantifying Biases in LLM-as-a-Judge Evaluations Main 9 页 · PDF
40 RLCracker: Evaluating the Worst-Case Vulnerability of LLM Watermarks with Adaptive RL Attacks Main 9 页 · PDF
41 Robust AI Evaluation through Maximal Lotteries Main 9 页 · PDF
42 RubricRobustness: Evaluating the Sensitivity of Rubrics-Based Benchmarks to Simple Perturbations Main 9 页 · PDF
43 SEDRAS: Symbolically Evaluated Deep Research And Science Main 9 页 · PDF
44 STABLEVAL: Disagreement-Aware and Stable Evaluation of AI Systems Main 9 页 · PDF
45 Strategic Candidacy in Generative AI Arenas Main 9 页 · PDF
46 SWE-ABS: Adversarial Benchmark Strengthening Exposes Inflated Success Rates on Test-based Benchmark Main 9 页 · PDF
47 The Generalization Spectrum: A Chromatographic Approach to Evaluating Learning Algorithms Main 9 页 · PDF
48 Towards a Science of AI Agent Reliability Main 9 页 · PDF
49 Turning Bias into Bugs: Bandit-Guided Style Manipulation Attacks on LLM Judges Main 9 页 · PDF
50 What LLMs Explain Is Not What They Believe: Evaluating Explanation Sufficiency Under Models’ Own Input Beliefs Main 9 页 · PDF
51 When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation Main 9 页 · PDF
52 ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models Main 9 页 · PDF
53 \(τ\)-Knowledge: Evaluating Conversational Agents over Unstructured Knowledge Main 8 页 · PDF
54 \(τ^2\)-Bench: Evaluating Conversational Agents in a Dual-Control Environment Main 8 页 · PDF
55 A Diagnostic Study of Multi-Agent LLMs for Real-World Debates Main 8 页 · PDF
56 A\(^2\)RBench: An Automatic Paradigm for Formally Verifiable Abstract Reasoning Benchmark Generation Main 8 页 · PDF
57 ABC-Bench: An Agentic Bio-Capabilities Benchmark for Biosecurity Main 8 页 · PDF
58 Agentic Confidence Calibration Main 8 页 · PDF
59 AlgoVeri: An Aligned Benchmark for Verified Code Generation on Classical Algorithms Main 8 页 · PDF
60 AppWorld-UL: Benchmarking Diverse Agent-User Interactions for Tool-Use Main 8 页 · PDF
61 Are We Overconfident in Models and Results for Semi-Supervised 3D Medical Image Segmentation? Main 8 页 · PDF
62 ASyMOB: Algebraic Symbolic Mathematical Operations Benchmark Main 8 页 · PDF
63 BEAR: Dissecting Embodied Abilities in Multimodal Language Models through Skill-level Evaluation and Diagnosis Main 8 页 · PDF
64 Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks Main 8 页 · PDF
65 Benchmarking Reward Hack Detection in Code Environments via Contrastive Analysis Main 8 页 · PDF
66 Beyond Benchmarks: Toward Causally Faithful Evaluation of Large Language Models Main 8 页 · PDF
67 Beyond Drift: Stabilizing Subjective LLM Evaluation with Information-Theoretic Rubrics Main 8 页 · PDF
68 Beyond Model Ranking: Predictability-Aligned Evaluation for Time Series Forecasting Main 8 页 · PDF
69 Beyond the Final Answer: Evaluating the Reasoning Trajectories of Tool-Augmented Agents Main 8 页 · PDF
70 BFCL Audio: An Audio Function Calling Evaluation for Large Language Models Main 8 页 · PDF
71 BrokenMath: A Benchmark for Sycophancy in Theorem Proving with LLMs Main 8 页 · PDF
72 CARE: Confounder-Aware Aggregation for Reliable LLM Evaluation Main 8 页 · PDF
73 Causally Evaluating the Learnability of Formal Language Tasks Main 8 页 · PDF
74 CausalProfiler: Generating Synthetic Benchmarks for Rigorous and Transparent Evaluation of Causal Machine Learning Main 8 页 · PDF
75 CentaurEval: Benchmarking Human-in-the-Loop Value in Agentic Coding Main 8 页 · PDF
76 CoCoReviewBench: A Completeness- and Correctness-Oriented Benchmark for AI Reviewers Main 8 页 · PDF
77 ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox Main 8 页 · PDF
78 Confidence is Not Universal: Task-Dependent Calibration and Emergent Behavior in LLMs Main 8 页 · PDF
79 Constructing Industrial-Scale Optimization Modeling Benchmark Main 8 页 · PDF
80 Conversation for Non-verifiable Learning: Self-Evolving Large Language Models through Meta-Evaluation Main 8 页 · PDF
81 Copyright-Bench: Agentic Evaluation of Copyright Law Compliance Main 8 页 · PDF
82 CUARewardBench: A Benchmark for Evaluating Reward Models for Computer-Using Agents Main 8 页 · PDF
83 CyberGym-E2E: Scalable Real-World Benchmark for AI Agents’ End-to-End Cybersecurity Capabilities Main 8 页 · PDF
84 DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories Main 8 页 · PDF
85 Distribution-Calibrated Inference Time Compute for Thinking LLM-as-a-Judge Main 8 页 · PDF
86 Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebook Main 8 页 · PDF
87 Do Text Edits Generalize to Visual Generation? Benchmarking Cross-Modal Knowledge Editing in UMMs Main 8 页 · PDF
88 Do Vision and Text Cues Exhibit Evidential Coupling? UFO: A Benchmark for Compositional Multimodal Reasoning in Unified Models Main 8 页 · PDF
89 DRIFT-BENCH: Diagnosing CoopeRative Breakdowns in LLM Agents under Input Faults via Multi-Turn Interaction Main 8 页 · PDF
90 DRPBench: Evaluating LLMs in Concurrent Code Comprehension via Fine-Grained Data Race Prediction Main 8 页 · PDF
91 Dynamic Multimodal Evaluation via Knowledge-Enhanced Benchmark Evolution Main 8 页 · PDF
92 DynaSchedBench: Calibrated Dynamic Scheduling Benchmarks and Observability Paradox in LLM-based Scheduling Agents Main 8 页 · PDF
93 EnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settings Main 8 页 · PDF
94 Evaluating AI Grading on Real-World Handwritten College Mathematics: A Large-Scale Study Toward a Benchmark Main 8 页 · PDF
95 Evaluating and Steering Modality Preferences in Multi-modal LLMs Main 8 页 · PDF
96 Evaluating Language Models in Realistic Conversational Contexts Main 8 页 · PDF
97 Evaluating LLM Uncertainty in Long-Form Generation Using Deterministic Ground Truth Main 8 页 · PDF
98 Evaluating Robustness of Reasoning Models on Parameterized Logical Problems Main 8 页 · PDF
99 Explanations are a Means to an End: Decision Theoretic Explanation Evaluation Main 8 页 · PDF
100 FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific Insights Main 8 页 · PDF
101 Formally Exploring Visual Anomaly Detection Evaluation Metrics Main 8 页 · PDF
102 FormulaCode: Evaluating Agentic Optimization on Large Codebases Main 8 页 · PDF
103 From Weak Cues to Real Identities: Evaluating Inference-Driven De-Anonymization in LLM Agents Main 8 页 · PDF
104 From Winning to Understanding: A Diagnostic Long-Horizon RTS Benchmark for LLMs Main 8 页 · PDF
105 GameDevBench: Evaluating Agentic Capabilities Through Game Development Main 8 页 · PDF
106 Generalized Correctness Models: Learning Calibrated and Cross-Model Correctness Predictors from Historical Patterns Main 8 页 · PDF
107 GICDM: Mitigating Hubness for Reliable Distance-Based Generative Model Evaluation Main 8 页 · PDF
108 Hair-Trigger Alignment: Black-Box Evaluation Cannot Guarantee Post-Update Alignment Main 8 页 · PDF
109 Helpful to a Fault: Measuring Illicit Assistance in Multi-Turn, Multilingual LLM Agents Main 8 页 · PDF
110 HiPhO: How Far Are (M)LLMs from Humans in the Latest High School Physics Olympiad Benchmark? Main 8 页 · PDF
111 Hunt Instead of Wait: Evaluating Deep Data Research on Large Language Models Main 8 页 · PDF
112 HypoSpace: A Diagnostic Benchmark for Set-Valued Hypothesis Generation under Underdetermination and Sublinear Coverage Bounds Main 8 页 · PDF
113 Implicit Intelligence - Evaluating Agents on What Users Don’t Say Main 8 页 · PDF
114 InnoEval: On Research Idea Evaluation as a Knowledge-Grounded, Multi-Perspective Reasoning Problem Main 8 页 · PDF
115 InteractComp: Evaluating Search Agents With Ambiguous Queries Main 8 页 · PDF
116 It’s a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents Main 8 页 · PDF
117 JADE: Expert-Grounded Dynamic Evaluation for Open-Ended Professional Tasks Main 8 页 · PDF
118 Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarking Main 8 页 · PDF
119 LALM-as-a-Judge: Benchmarking Large Audio-Language Models for Safety Evaluation in Multi-Turn Spoken Dialogues Main 8 页 · PDF
120 Learning More from Less: Unlocking Internal Representations for Benchmark Compression Main 8 页 · PDF
121 LitReview Arena: Evaluating Literature Review Agents with Battle-style Peer Review Platform Main 8 页 · PDF
122 LiveNewsBench: Evaluating Web Search Agents with Freshly Curated News Main 8 页 · PDF
123 LOCA-bench: Benchmarking Language Agents Under Controllable and Extreme Context Growth Main 8 页 · PDF
124 LongCoT: Benchmarking Long-Horizon Chain-of-Thought Reasoning Main 8 页 · PDF
125 LOVE: Benchmarking and Evaluating Text-to-Video Generation and Video-to-Text Interpretation Main 8 页 · PDF
126 MADE: Benchmark Environments for Closed-Loop Materials Discovery Main 8 页 · PDF
127 Measuring Agents in Production Main 8 页 · PDF
128 Measuring Meta-Cultural Competency: A Spectral Framework for LLM Knowledge Structures Main 8 页 · PDF
129 MGAL: A Multilingual Granularity-Aware Long-Context Benchmark Main 8 页 · PDF
130 Mitigating Translationese Bias in Multilingual LLM-as-a-Judge via Disentangled Information Bottleneck Main 8 页 · PDF
131 MM-Snowball: Evaluating and Mitigating Hallucination Snowballing in Multimodal Multi-Turn Dialogue Main 8 页 · PDF
132 MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety Main 8 页 · PDF
133 MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Models Main 8 页 · PDF
134 NAVIGATE: Evaluating Visual-Guided Search Decision-Making on the Open Web Main 8 页 · PDF
135 On Stable Long-Form Generation: Benchmarking and Mitigating Length Volatility Main 8 页 · PDF
136 OPT-Engine: Benchmarking the Limits of LLMs in Optimization Modeling via Complexity Scaling Main 8 页 · PDF
137 ORLoopBench: Solver-in-the-Loop Benchmarks for Self-Correction and Behavioral Rationality in Operations Research Main 8 页 · PDF
138 Outrunning LLM Cutoffs: A Live Kernel Crash Resolution Benchmark for All Main 8 页 · PDF
139 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Main 8 页 · PDF
140 Position: Agent Evaluation Should Be Agentified for Openness, Standardization, and Reproducibility Main 8 页 · PDF
141 Position: Benchmarks for Vision–Language Models in Urban Perception Should Be Reliability-Aware and Negotiated Main 8 页 · PDF
142 Position: Evaluating LLMs in Finance Requires Explicit Bias Consideration Main 8 页 · PDF
143 Position: Genomic Model Research Must Move Beyond Anecdotal Evaluation of Interpretability Methods Main 8 页 · PDF
144 Position: LLM-Safety Evaluations Lack Robustness Main 8 页 · PDF
145 Position: Peer Review Should Be Calibrated via LLM Scoring Main 8 页 · PDF
146 Position: Stop Automating Peer Review Without Rigorous Evaluation Main 8 页 · PDF
147 Position: Sycophancy is an Educational Safety Risk: Why LLM Tutors Need Sycophancy Benchmarks Main 8 页 · PDF
148 Position: The Open Benchmark Paradox Must Be Resolved through Sovereign Medical Evaluation Main 8 页 · PDF
149 PPT-Eval: A Benchmark for Computer-Use Agents on PowerPoint Tasks Main 8 页 · PDF
150 ProbeLLM: Automating Principled Diagnosis of LLM Failures Main 8 页 · PDF

评论