跳转至

ACL 2026 · 全部收录(459 篇)

仅列出通过预过滤且已打分的论文;分数是「评测方法学价值」,不是会议录用质量。 返回总览

# 论文 轨道 分数 链接
1 BenchMarker: An Education-Inspired Toolkit for Highlighting Flaws in Multiple-Choice Benchmarks Long 9 页 · PDF
2 Beyond the Leaderboard: Rethinking Medical Benchmarks for Large Language Models Long 9 页 · PDF
3 BrowseComp-Plus: A Fair and Disentangled Evaluation Benchmark for Deep Search Agents Long 9 页 · PDF
4 Controllable Contamination Detection for Reliable LLM Evaluation with Statistical Guarantees Long 9 页 · PDF
5 Decentralized Arena: Towards Democratic and Scalable Automatic Evaluation of Language Models Long 9 页 · PDF
6 DeepFact: Co-Evolving Benchmarks and Agents for Deep Research Factuality Long 9 页 · PDF
7 Evaluating Legal Reasoning Traces with Legal Issue Tree Rubrics Long 9 页 · PDF
8 From Word Sequences to Behavioral Sequences: Adapting Modeling and Evaluation Paradigms for Longitudinal NLP Long 9 页 · PDF
9 GenPT: Beyond Self-Report for Reliable LLM Psychometrics via Generative Projective Testing Long 9 页 · PDF
10 Illusions of the Gold Standard: A Large-scale Analysis of Human Evaluation Protocols for Long-form Text Generation Long 9 页 · PDF
11 Label Effects: Shared Heuristic Reliance in Trust Assessment by Humans and LLM-as-a-Judge Long 9 页 · PDF
12 LLMEval-Fair: A Large-Scale Longitudinal Study on Robust and Fair Evaluation of Large Language Models Long 9 页 · PDF
13 Lost in Simulation: LLM-Simulated Users are Unreliable Proxies for Human Users in Agentic Evaluations Long 9 页 · PDF
14 Macaron: Controlled, Human-Written Benchmark for Multilingual and Multicultural Reasoning via Template-Filling Long 9 页 · PDF
15 MedEinst: Benchmarking the Einstellung Effect in Medical LLMs through Counterfactual Differential Diagnosis Long 9 页 · PDF
16 Mediocrity is the key for LLM as a Judge Anchor Selection Long 9 页 · PDF
17 MQM Re-Annotation: A Technique for Collaborative Evaluation of Machine Translation Long 9 页 · PDF
18 ROSE: An Intent-Centered Evaluation Metric for NL2SQL Long 9 页 · PDF
19 SPENCE: A Syntactic Probe for Detecting Contamination in NL2SQL Benchmarks Long 9 页 · PDF
20 Stress Testing Factual Consistency Metrics for Long-Document Summarization Long 9 页 · PDF
21 Test of Time: Rethinking Temporal Signal of Benchmark Contamination Long 9 页 · PDF
22 The Proxy Presumption: From Semantic Embeddings to Valid Social Measures Long 9 页 · PDF
23 User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios Long 9 页 · PDF
24 VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-Checking Long 9 页 · PDF
25 When Benchmarks Leak: Inference-Time Decontamination for LLMs Long 9 页 · PDF
26 Your Reasoning Benchmark May Not Test Reasoning: Revealing Perception Bottleneck in Abstract Reasoning Benchmarks Long 9 页 · PDF
27 A Functionality-Grounded Benchmark for Evaluating Web Agents in E-commerce Domains Long 8 页 · PDF
28 ACIArena: Toward Unified Evaluation for Agent Cascading Injection Long 8 页 · PDF
29 AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following Long 8 页 · PDF
30 AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments Long 8 页 · PDF
31 Are We Using the Right Benchmark: An Evaluation Framework for Visual Token Compression Methods Long 8 页 · PDF
32 arXiv2Table: Toward Realistic Benchmarking and Evaluation for LLM-Based Literature-Review Table Generation Long 8 页 · PDF
33 AutoJudger: An Agent-Driven Framework for Efficient Benchmarking of MLLMs Long 8 页 · PDF
34 Benchmarking Deflection and Hallucination in Large Vision-Language Models Long 8 页 · PDF
35 Benchmarking Fine-Grained Error Detection in Multimodal Reasoning Long 8 页 · PDF
36 Benchmarking Web Agent Safety under E-commerce Deceptive Interfaces Long 8 页 · PDF
37 Beyond "I Don’t Know": Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty Long 8 页 · PDF
38 Beyond Ranking: Fine-Grained Diagnostics and Self-Improvement for MLLMs Long 8 页 · PDF
39 Beyond Static Benchmarks: Synthesizing Harmful Content via Persona-based Simulation for Robust Evaluation Long 8 页 · PDF
40 Beyond the Last Frame: Process-aware Evaluation for Generative Video Reasoning Long 8 页 · PDF
41 Beyond Value Benchmarks: Measuring Value-Structure Alignment in Large Language Models via Symmetric Q-Sorts Long 8 页 · PDF
42 Bloom-Eval: A Hierarchical Evaluation Benchmark for Automatic Survey Generation Based on Bloom’s Taxonomy Long 8 页 · PDF
43 CAR-bench: Evaluating the Consistency and Limit-Awareness of LLM Agents under Real-World Uncertainty Long 8 页 · PDF
44 Chain-of-Thought as a Lens: Evaluating Structured Reasoning Alignment between Human Preferences and Large Language Models Long 8 页 · PDF
45 ChemReason-Bench: Benchmarking Large Language Models for Procedural Reasoning in Experimental Chemistry Long 8 页 · PDF
46 CIG: Measuring Conversational Information Gain in Deliberative Dialogues with Semantic Memory Dynamics Long 8 页 · PDF
47 CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks Long 8 页 · PDF
48 CogToM: A Comprehensive Theory of Mind Benchmark inspired by Human Cognition for Large Language Models Long 8 页 · PDF
49 CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data Long 8 页 · PDF
50 Cross-Examination Framework: A Task-Agnostic Diagnostic for Information Fidelity in Text-to-Text Generation Long 8 页 · PDF
51 CxMP: A Linguistic Minimal-Pair Benchmark for Evaluating Constructional Understanding in Language Models Long 8 页 · PDF
52 DAC-Bench: A Decision-Aware Benchmark for Compositional Mobile GUI Tasks Long 8 页 · PDF
53 DeepPlanning: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraints Long 8 页 · PDF
54 DR-Arena: an Automated Evaluation Framework for Deep Research Agents Long 8 页 · PDF
55 DREAM: Deep Research Evaluation with Agentic Metrics Long 8 页 · PDF
56 E2EDev: Benchmarking Large Language Models in End-to-End Software Development Task Long 8 页 · PDF
57 EmoS: A High-Fidelity Multimodal Benchmark for Fine-grained Streaming Emotional Understanding Long 8 页 · PDF
58 Evaluating Language Model Pluralism through In-the-wild Crowd Discussions Long 8 页 · PDF
59 Evaluation Pitfalls and Challenges in Multimedia Event Extraction Long 8 页 · PDF
60 EVM-QuestBench: An Execution-Grounded Benchmark for Natural-Language Transaction Code Generation Long 8 页 · PDF
61 FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning Long 8 页 · PDF
62 From Laboratory to Real-World Applications: Benchmarking Agentic Code Reasoning at the Repository Level Long 8 页 · PDF
63 Gap-K%: Measuring Top-1 Prediction Gap for Detecting Pretraining Data Long 8 页 · PDF
64 GeoRC: A Benchmark for Geolocation Reasoning Chains Long 8 页 · PDF
65 Gradient-Guided Multi-Judge Prompt Optimization Long 8 页 · PDF
66 Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Math Long 8 页 · PDF
67 HoWToBench: Holistic Evaluation for LLM’s Capability in Human-level Writing using Tree of Writing Long 8 页 · PDF
68 HSCodeComp: A Realistic and Expert-level Agent Benchmark for Hierarchical Rule Application Long 8 页 · PDF
69 HypoEval: Hypothesis-Guided Evaluation for Natural Language Generation Long 8 页 · PDF
70 IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluation Long 8 页 · PDF
71 IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation Long 8 页 · PDF
72 Inflated Excellence or True Performance? Rethinking Medical Diagnostic Benchmarks with Dynamic Evaluation Long 8 页 · PDF
73 Is EEG-to-Text Feasible in Real-World Scenarios? An In-Depth Analysis Using a Neuropsychology-Inspired Benchmark Long 8 页 · PDF
74 JurisBench: A Deep Benchmark for Assessing Large Language Models in Professional Legal Practice Long 8 页 · PDF
75 Knowledge without Wisdom: Measuring Misalignment between LLMs and Intended Impact Long 8 页 · PDF
76 League of LLMs: A Benchmark-Free Paradigm for Mutual Evaluation of Large Language Models Long 8 页 · PDF
77 LexRel: Benchmarking Legal Relation Extraction for Chinese Civil Cases Long 8 页 · PDF
78 LiveCLKTBench: Towards Reliable Evaluation of Cross-Lingual Knowledge Transfer in Multilingual LLMs Long 8 页 · PDF
79 LiveFact: A Dynamic, Time-Aware Benchmark for LLM-Driven Fake News Detection Long 8 页 · PDF
80 Locomo-Plus: Beyond-Factual Cognitive Memory Evaluation Framework for LLM Agents Long 8 页 · PDF
81 MADE: A Living Benchmark for Multi-Label Text Classification with Uncertainty Quantification of Medical Device Adverse Events Long 8 页 · PDF
82 MathSight: A Benchmark Exploring Have Vision-Language Models Really Seen in University-Level Mathematical Reasoning? Long 8 页 · PDF
83 Measuring Distribution Shift in User Prompts and Its Effects on LLM Performance Long 8 页 · PDF
84 Measuring Social Bias in Vision-Language Models with Face-Only Counterfactuals from Real Photos Long 8 页 · PDF
85 METER: Evaluating Multi-Level Contextual Causal Reasoning in Large Language Models Long 8 页 · PDF
86 MM-JudgeBias: A Benchmark for Evaluating Compositional Biases in MLLM-as-a-Judge Long 8 页 · PDF
87 MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments Long 8 页 · PDF
88 Monotonic Scaffolding as a Diagnostic Lens for Legal Reasoning in LLMs Long 8 页 · PDF
89 MPR-GUI: Benchmarking and Enhancing Multilingual Perception and Reasoning in GUI Agents Long 8 页 · PDF
90 MTR-Suite: A Framework for Evaluating and Synthesizing Conversational Retrieval Benchmarks Long 8 页 · PDF
91 Multi-Agent-as-Judge: Aligning LLM-Agent-Based Automated Evaluation with Multi-Dimensional Human Evaluation Long 8 页 · PDF
92 NaturalGAIA: A Verifiable Benchmark and Hierarchical Framework for Long-Horizon GUI Tasks Long 8 页 · PDF
93 Neo-Classic: A Benchmark for Evaluating Linguistic-Aesthetic Reasoning in Classical Chinese Poetry Long 8 页 · PDF
94 OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding Long 8 页 · PDF
95 Omni-RewardBench: Toward a Comprehensive Evaluation of Generative Reward Models Across Modalities Long 8 页 · PDF
96 PEAR: Pairwise Evaluation for Automatic Relative Scoring in Machine Translation Long 8 页 · PDF
97 Persona-Grounded Safety Evaluation of AI Companions in Multi-Turn Conversations Long 8 页 · PDF
98 PIArena: A Platform for Prompt Injection Evaluation Long 8 页 · PDF
99 PluRule: A Benchmark for Moderating Pluralistic Communities on Social Media Long 8 页 · PDF
100 PRBench: Large-Scale Expert Rubrics for Evaluating High-Stakes Professional Reasoning Long 8 页 · PDF
101 Putting HUMANS first: Efficient LAM Evaluation with Human Preference Alignment Long 8 页 · PDF
102 Quantifying Metric and Model Agreement in Bias Evaluation of Large Language Models Long 8 页 · PDF
103 Quantifying the Impact of Translation Errors on Multilingual LLM Evaluation Long 8 页 · PDF
104 Question Difficulty Estimation for Large Language Models via Answer Plausibility Scoring Long 8 页 · PDF
105 Reliable Evaluation Protocol for Low-Precision Retrieval Short 8 页 · PDF
106 ReproEvalCard: A Reporting Standard for Reproducible Evaluation of LLM Pipelines Short 8 页 · PDF
107 Rethinking Evaluation for LLM Hallucination Detection: A Desiderata, A New RAG-based Benchmark, New Insights Long 8 页 · PDF
108 ReTraceQA: Evaluating Reasoning Traces of Small Language Models in Commonsense Question Answering Long 8 页 · PDF
109 Revisiting Metric Reliability for Fine-grained Evaluation of Machine Translation and Summarization in Indian Languages Long 8 页 · PDF
110 Revisiting the Reliability of Language Models in Instruction-Following Long 8 页 · PDF
111 S2S-Arena: Evaluating Paralinguistic Instruction Following in Speech-to-Speech Models Long 8 页 · PDF
112 SAGE: A Search-AuGmented Evaluation of Large Language Models on Free-Form QA Long 8 页 · PDF
113 SATQuest: A Verifier for Logical Reasoning Evaluation and Reinforcement Fine-Tuning of LLMs Long 8 页 · PDF
114 SecureVibeBench: Benchmarking Secure Vibe Coding of AI Agents via Reconstructing Vulnerability-Introducing Scenarios Long 8 页 · PDF
115 Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent Systems Long 8 页 · PDF
116 SILO-BENCH: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systems Long 8 页 · PDF
117 Simple Agents, Biased Judges: Efficient Multi-Party Dialogue Generation & The Evaluation Gap Long 8 页 · PDF
118 SpeechLLM-as-Judges: Towards General and Interpretable Speech Quality Evaluation Long 8 页 · PDF
119 Still Between Us? Evaluating and Improving Voice Assistant Robustness to Third-Party Interruptions Long 8 页 · PDF
120 Subject-level Inference for Realistic Text Anonymization Evaluation Long 8 页 · PDF
121 SwissGov-RSD: A Human-annotated, Cross-lingual Benchmark for Token-level Recognition of Semantic Differences Between Related Documents Long 8 页 · PDF
122 TabReX: Tabular Referenceless eXplainable Evaluation Long 8 页 · PDF
123 The Visual Iconicity Challenge: Evaluating Vision-Language Models on Sign Language Form–Meaning Mapping Long 8 页 · PDF
124 Toward Robust Evaluation for Multilingual Grammatical Error Correction: Can Large Language Models Replace Human References? Long 8 页 · PDF
125 TRACE: Evaluating Execution Efficiency of LLM-Based Code Translation Long 8 页 · PDF
126 Translation or Recitation? Calibrating Evaluation Scores for Machine Translation of Extremely Low-Resource Languages Short 8 页 · PDF
127 Unlearners Can Lie: Evaluating and Improving Honesty in LLM Unlearning Long 8 页 · PDF
128 UrbanGeoEval: A City-Scale Benchmark for Evaluating Large Language Models in Geospatial Reasoning Long 8 页 · PDF
129 USB: A COMPREHENSIVE AND UNIFIED SAFETY EVALUATION BENCHMARK FOR MULTIMODAL LARGE LANGUAGE MODELS Long 8 页 · PDF
130 VC-Inspector: Advancing Reference-free Evaluation of Video Captions with Factual Analysis Long 8 页 · PDF
131 VisAidMath: Benchmarking Visual-Aided Mathematical Reasoning Long 8 页 · PDF
132 VL-Calibration: Decoupled Confidence Calibration for Large Vision-Language Models Reasoning Long 8 页 · PDF
133 When High Accuracy Hides Poor Calibration: Rethinking Confidence Evaluation in Transformer-Based Text Classification with Balanced Brier Score Long 8 页 · PDF
134 When KV Cache Reuse Fails in Multi-Agent Systems: Cross-Candidate Interaction is Crucial for LLM Judges Long 8 页 · PDF
135 When More Words Say Less: Decoupling Length and Specificity in Image Description Evaluation Short 8 页 · PDF
136 When Vision-Language Models Judge Without Seeing: Exposing Informativeness Bias Long 8 页 · PDF
137 YIELD: A Large-Scale Dataset and Evaluation Framework for Information Elicitation Agents Long 8 页 · PDF
138 "Newspaper Eat" Means "Not Tasty": A Taxonomy and Benchmark for Coded Language in Real-World Chinese Online Reviews Long 7 页 · PDF
139 A Multi-Agent Framework for Feature-Constrained Difficulty Control in Reading Comprehension Item Generation Long 7 页 · PDF
140 A Multilingual Social Bias Benchmark Incorporating Thinking Processes Long 7 页 · PDF
141 ADAPT: Benchmarking Commonsense Planning under Unspecified Affordance Constraints Long 7 页 · PDF
142 AEGIS: A Holistic Benchmark for Evaluating Forensic Analysis of AI-Generated Academic Images Long 7 页 · PDF
143 AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts Long 7 页 · PDF
144 Aligning Agents via Planning: A Benchmark for Trajectory-Level Reward Modeling Long 7 页 · PDF
145 All That Glisters Is Not Gold: A Benchmark for Reference-Free Counterfactual Financial Misinformation Detection Long 7 页 · PDF
146 Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token Optimization Long 7 页 · PDF
147 Ascending the Infinite Ladder: Benchmarking Spatial Deformation Reasoning in Vision-Language Models Long 7 页 · PDF
148 Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models Long 7 页 · PDF
149 Audio MultiChallenge: A Multi-Turn Evaluation of Spoken Dialogue Systems on Natural Human Interaction Long 7 页 · PDF
150 Automated Creativity Evaluation of Language Models Across Open-Ended Tasks Long 7 页 · PDF

评论