跳转至

EMNLP 2025 · 全部收录(315 篇)

仅列出通过预过滤且已打分的论文;分数是「评测方法学价值」,不是会议录用质量。 返回总览

# 论文 轨道 分数 链接
1 Analyzing Uncertainty of LLM-as-a-Judge: Interval Evaluations with Conformal Prediction Main 9 页 · PDF
2 Benchmark Profiling: Mechanistic Diagnosis of LLM Benchmarks Main 9 页 · PDF
3 Benchmarking Large Language Models Under Data Contamination: A Survey from Static to Dynamic Evaluation Main 9 页 · PDF
4 Beyond the Surface: Measuring Self-Preference in LLM Judgments Main 9 页 · PDF
5 Can Large Language Models Outperform Non-Experts in Poetry Evaluation? A Comparative Study Using the Consensual Assessment Technique Main 9 页 · PDF
6 CheckEval: A reliable LLM-as-a-Judge framework for evaluating text generation using checklists Main 9 页 · PDF
7 DCR: Quantifying Data Contamination in LLMs Evaluation Main 9 页 · PDF
8 DyePack: Provably Flagging Test Set Contamination in LLMs Using Backdoors Main 9 页 · PDF
9 Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs Main 9 页 · PDF
10 Languages Still Left Behind: Toward a Better Multilingual Machine Translation Benchmark Main 9 页 · PDF
11 PERSEVAL: A Framework for Perspectivist Classification Evaluation Main 9 页 · PDF
12 Reliable Evaluation and Benchmarks for Statement Autoformalization Main 9 页 · PDF
13 SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants? Main 9 页 · PDF
14 SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction Main 9 页 · PDF
15 The Illusion of Progress: Re-evaluating Hallucination Detection in LLMs Main 9 页 · PDF
16 A Causal Lens for Evaluating Faithfulness Metrics Main 8 页 · PDF
17 A Fully Probabilistic Perspective on Large Language Model Unlearning: Evaluation and Optimization Main 8 页 · PDF
18 A Necessary Step toward Faithfulness: Measuring and Improving Consistency in Free-Text Explanations Main 8 页 · PDF
19 Agent-as-Judge for Factual Summarization of Long Narratives Main 8 页 · PDF
20 Are Checklists Really Useful for Automatic Evaluation of Generative Tasks? Main 8 页 · PDF
21 Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performance Main 8 页 · PDF
22 Arena-lite: Efficient and Reliable Large Language Model Evaluation via Tournament-Based Direct Comparisons Main 8 页 · PDF
23 Benchmarking Debiasing Methods for LLM-based Parameter Estimates Main 8 页 · PDF
24 Beyond the Score: Uncertainty-Calibrated LLMs for Automated Essay Assessment Main 8 页 · PDF
25 Blind Men and the Elephant: Diverse Perspectives on Gender Stereotypes in Benchmark Datasets Main 8 页 · PDF
26 Break the Checkbox: Challenging Closed-Style Evaluations of Cultural Alignment in LLMs Main 8 页 · PDF
27 Can LLMs Help You at Work? A Sandbox for Evaluating LLM Agents in Enterprise Environments Main 8 页 · PDF
28 ClimateViz: A Benchmark for Statistical Reasoning and Fact Verification on Scientific Charts Main 8 页 · PDF
29 Co-Eval: Augmenting LLM-based Evaluation with Machine Metrics Main 8 页 · PDF
30 CREPE: Rapid Chest X-ray Report Evaluation by Predicting Multi-category Error Counts Main 8 页 · PDF
31 Debatable Intelligence: Benchmarking LLM Judges via Debate Speech Evaluation Main 8 页 · PDF
32 Don’t Sweat the Small Stuff: Segment-Level Meta-Evaluation Based on Pairwise Difference Correlation Main 8 页 · PDF
33 ESC-Judge: A Framework for Comparing Emotional Support Conversational Agents Main 8 页 · PDF
34 Evaluating Language Translation Models by Playing Telephone Main 8 页 · PDF
35 Evaluating the Evaluators: Are readability metrics good measures of readability? Main 8 页 · PDF
36 Fooling the LVLM Judges: Visual Biases in LVLM-Based Evaluation Main 8 页 · PDF
37 Forget What You Know about LLMs Evaluations - LLMs are Like a Chameleon Main 8 页 · PDF
38 F²Bench: An Open-ended Fairness Evaluation Benchmark for LLMs with Factuality Considerations Main 8 页 · PDF
39 Graders Should Cheat: Privileged Information Enables Expert-Level Automated Evaluations Main 8 页 · PDF
40 GuessingGame: Measuring the Informativeness of Open-Ended Questions in Large Language Models Main 8 页 · PDF
41 How Far Can LLMs Improve from Experience? Measuring Test-Time Learning Ability in LLMs with Human Comparison Main 8 页 · PDF
42 ImpliRet: Benchmarking the Implicit Fact Retrieval Challenge Main 8 页 · PDF
43 In Benchmarks We Trust ... Or Not? Main 8 页 · PDF
44 Investigating How Pre-training Data Leakage Affects Models’ Reproduction and Detection Capabilities Main 8 页 · PDF
45 Is the Top Still Spinning? Evaluating Subjectivity in Narrative Understanding Main 8 页 · PDF
46 iVISPAR — An Interactive Visual-Spatial Reasoning Benchmark for VLMs Main 8 页 · PDF
47 LiteraryQA: Towards Effective Evaluation of Long-document Narrative QA Main 8 页 · PDF
48 Long-Form Information Alignment Evaluation Beyond Atomic Facts Main 8 页 · PDF
49 MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors Main 8 页 · PDF
50 Measuring Bias or Measuring the Task: Understanding the Brittle Nature of LLM Gender Biases Main 8 页 · PDF
51 Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps Main 8 页 · PDF
52 Measuring scalar constructs in social science with LLMs Main 8 页 · PDF
53 Mechanisms vs. Outcomes: Probing for Syntax Fails to Explain Performance on Targeted Syntactic Evaluations Main 8 页 · PDF
54 Metric Calculating Benchmark: Code-Verifiable Complicate Instruction Following Benchmark for Large Language Models Main 8 页 · PDF
55 Mind the Blind Spots: A Focus-Level Evaluation Framework for LLM Reviews Main 8 页 · PDF
56 MR. Judge: Multimodal Reasoner as a Judge Main 8 页 · PDF
57 Not-Just-Scaling Laws: Towards a Better Understanding of the Downstream Impact of Language Model Design Decisions Main 8 页 · PDF
58 PoSum-Bench: Benchmarking Position Bias in LLM-based Conversational Summarization Main 8 页 · PDF
59 Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance Main 8 页 · PDF
60 ProcWorld: Benchmarking Large Model Planning in Reachability-Constrained Environments Main 8 页 · PDF
61 ReEvalMed: Rethinking Medical Report Evaluation by Aligning Metrics with Real-World Clinical Judgment Main 8 页 · PDF
62 Rethinking Backdoor Detection Evaluation for Language Models Main 8 页 · PDF
63 reWordBench: Benchmarking and Improving the Robustness of Reward Models with Transformed Inputs Main 8 页 · PDF
64 Same evaluation, more tokens: On the effect of input length for machine translation evaluation using Large Language Models Main 8 页 · PDF
65 Seeing is Believing, but How Much? A Comprehensive Analysis of Verbalized Calibration in Vision-Language Models Main 8 页 · PDF
66 seqBench: A Tunable Benchmark to Quantify Sequential Reasoning Limits of LLMs Main 8 页 · PDF
67 SSA-COMET: Do LLMs Outperform Learned Metrics in Evaluating MT for Under-Resourced African Languages? Main 8 页 · PDF
68 SSA: Semantic Contamination of LLM-Driven Fake News Detection Main 8 页 · PDF
69 SYNC: A Synthetic Long-Context Understanding Benchmark for Controlled Comparisons of Model Capabilities Main 8 页 · PDF
70 TCP: a Benchmark for Temporal Constraint-Based Planning Main 8 页 · PDF
71 The LLM Already Knows: Estimating LLM-Perceived Question Difficulty via Hidden Representations Main 8 页 · PDF
72 ToM-SSI: Evaluating Theory of Mind in Situated Social Interactions Main 8 页 · PDF
73 Towards a Holistic and Automated Evaluation Framework for Multi-Level Comprehension of LLMs in Book-Length Contexts Main 8 页 · PDF
74 Towards Optimal Evaluation Efficiency for Large Language Models Main 8 页 · PDF
75 Translationese-index: Using Likelihood Ratios for Graded and Generalizable Measurement of Translationese Main 8 页 · PDF
76 TSVer: A Benchmark for Fact Verification Against Time-Series Evidence Main 8 页 · PDF
77 TTT-Bench: A Benchmark for Evaluating Reasoning Ability with Simple and Novel Tic-Tac-Toe-style Games Main 8 页 · PDF
78 Unlearning vs. Obfuscation: Are We Truly Removing Knowledge? Main 8 页 · PDF
79 VELA: An LLM-Hybrid-as-a-Judge Approach for Evaluating Long Image Captions Main 8 页 · PDF
80 VeriFact: Enhancing Long-Form Factuality Evaluation with Refined Fact Extraction and Reference Facts Main 8 页 · PDF
81 We Need to Measure Data Diversity in NLP — Better and Broader Main 8 页 · PDF
82 What You See is What You Ask: Evaluating Audio Descriptions Main 8 页 · PDF
83 XLQA: A Benchmark for Locale-Aware Multilingual Open-Domain Question Answering Main 8 页 · PDF
84 3MDBench: Medical Multimodal Multi-agent Dialogue Benchmark Main 7 页 · PDF
85 A Culturally-diverse Multilingual Multimodal Video Benchmark & Model Main 7 页 · PDF
86 A Position Paper on the Automatic Generation of Machine Learning Leaderboards Main 7 页 · PDF
87 AbsVis – Benchmarking How Humans and Vision-Language Models “See” Abstract Concepts in Images Main 7 页 · PDF
88 AccessEval: Benchmarking Disability Bias in Large Language Models Main 7 页 · PDF
89 Adapting Bias Evaluation to Domain Contexts using Generative Models Main 7 页 · PDF
90 Advancing Arabic Diacritization: Improved Datasets, Benchmarking, and State-of-the-Art Models Main 7 页 · PDF
91 AesBiasBench: Evaluating Bias and Alignment in Multimodal Language Models for Personalized Image Aesthetic Assessment Main 7 页 · PDF
92 Annotating Training Data for Conditional Semantic Textual Similarity Measurement using Large Language Models Main 7 页 · PDF
93 Are Vision-Language Models Safe in the Wild? A Meme-Based Benchmark Study Main 7 页 · PDF
94 Argument Summarization and its Evaluation in the Era of Large Language Models Main 7 页 · PDF
95 Audio-centric Video Understanding Benchmark without Text Shortcut Main 7 页 · PDF
96 Benchmarking and Mitigating MCQA Selection Bias of Large Vision-Language Models Main 7 页 · PDF
97 Beyond Human Labels: A Multi-Linguistic Auto-Generated Benchmark for Evaluating Large Language Models on Resume Parsing Main 7 页 · PDF
98 Beyond Static Testbeds: An Interaction-Centric Agent Simulation Platform for Dynamic Recommender Systems Main 7 页 · PDF
99 BOUQuET : dataset, Benchmark and Open initiative for Universal Quality Evaluation in Translation Main 7 页 · PDF
100 C3: A Bilingual Benchmark for Spoken Dialogue Models Exploring Challenges in Complex Conversations Main 7 页 · PDF
101 Calibrating LLMs for Text-to-SQL Parsing by Leveraging Sub-clause Frequencies Main 7 页 · PDF
102 Calibration Across Layers: Understanding Calibration Evolution in LLMs Main 7 页 · PDF
103 Can Prompts Rewind Time for LLMs? Evaluating the Effectiveness of Prompted Knowledge Cutoffs Main 7 页 · PDF
104 CausalVLBench: Benchmarking Visual Causal Reasoning in Large Vision-Language Models Main 7 页 · PDF
105 CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Space Main 7 页 · PDF
106 Collab-Overcooked: Benchmarking and Evaluating Large Language Models as Collaborative Agents Main 7 页 · PDF
107 CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward Main 7 页 · PDF
108 CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answering Main 7 页 · PDF
109 Contra4: Evaluating Contrastive Cross-Modal Reasoning in Audio, Video, Image, and 3D Main 7 页 · PDF
110 CourtReasoner: Can LLM Agents Reason Like Judges? Main 7 页 · PDF
111 Deep Associations, High Creativity: A Simple yet Effective Metric for Evaluating Large Language Models Main 7 页 · PDF
112 Direct Judgement Preference Optimization Main 7 页 · PDF
113 DischargeSim: A Simulation Benchmark for Educational Doctor–Patient Communication at Discharge Main 7 页 · PDF
114 Do Large Language Models Truly Grasp Addition? A Rule-Focused Diagnostic Using Two-Integer Arithmetic Main 7 页 · PDF
115 EGOILLUSION: Benchmarking Hallucinations in Egocentric Video Understanding Main 7 页 · PDF
116 EquiBench: Benchmarking Large Language Models’ Reasoning about Program Semantics via Equivalence Checking Main 7 页 · PDF
117 Estimating LLM Consistency: A User Baseline vs Surrogate Metrics Main 7 页 · PDF
118 Evaluating LLM-Generated Diagrams as Graphs Main 7 页 · PDF
119 Evaluating Spatiotemporal Consistency in Automatically Generated Sewing Instructions Main 7 页 · PDF
120 Exploring Response Uncertainty in MLLMs: An Empirical Evaluation under Misleading Scenarios Main 7 页 · PDF
121 Extending Automatic Machine Translation Evaluation to Book-Length Documents Main 7 页 · PDF
122 F2TEval: Human-Aligned Multi-Dimensional Evaluation for Figure-to-Text Task Main 7 页 · PDF
123 Fann or Flop: A Multigenre, Multiera Benchmark for Arabic Poetry Understanding in LLMs Main 7 页 · PDF
124 FB-Bench: A Fine-Grained Multi-Task Benchmark for Evaluating LLMs’ Responsiveness to Human Feedback Main 7 页 · PDF
125 FinRAGBench-V: A Benchmark for Multimodal RAG with Visual Citation in the Financial Domain Main 7 页 · PDF
126 FlashAdventure: A Benchmark for GUI Agents Solving Full Story Arcs in Diverse Adventure Games Main 7 页 · PDF
127 From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge Main 7 页 · PDF
128 From Understanding to Generation: An Efficient Shortcut for Evaluating Language Models Main 7 页 · PDF
129 HESEIA: A community-based dataset for evaluating social biases in large language models, co-designed in real school settings in Latin America Main 7 页 · PDF
130 How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark Main 7 页 · PDF
131 Incorporating Diverse Perspectives in Cultural Alignment: Survey of Evaluation Benchmarks Through A Three-Dimensional Framework Main 7 页 · PDF
132 InfiniBench: A Benchmark for Large Multi-Modal Models in Long-Form Movies and TV Shows Main 7 页 · PDF
133 InMind: Evaluating LLMs in Capturing and Applying Individual Human Reasoning Styles Main 7 页 · PDF
134 Investigating Value-Reasoning Reliability in Small Large Language Models Main 7 页 · PDF
135 JUDGEBERT: Assessing Legal Meaning Preservation Between Sentences Main 7 页 · PDF
136 LaMP-QA: A Benchmark for Personalized Long-form Question Answering Main 7 页 · PDF
137 LASER: An LLM-based ASR Scoring and Evaluation Rubric Main 7 页 · PDF
138 Leveraging Large Models to Evaluate Novel Content: A Case Study on Advertisement Creativity Main 7 页 · PDF
139 LingGym: How Far Are LLMs from Thinking Like Field Linguists? Main 7 页 · PDF
140 LMR-BENCH: Evaluating LLM Agent’s Ability on Reproducing Language Modeling Research Main 7 页 · PDF
141 LORAXBENCH: A Multitask, Multilingual Benchmark Suite for 20 Indonesian Languages Main 7 页 · PDF
142 Mahānāma: A Unique Testbed for Literary Entity Discovery and Linking Main 7 页 · PDF
143 Measuring Risk of Bias in Biomedical Reports: The RoBBR Benchmark Main 7 页 · PDF
144 Measuring the Effect of Disfluency in Multilingual Knowledge Probing Benchmarks Main 7 页 · PDF
145 MEBench: Benchmarking Large Language Models for Cross-Document Multi-Entity Question Answering Main 7 页 · PDF
146 MemeArena: Automating Context-Aware Unbiased Evaluation of Harmfulness Understanding for Multimodal Large Language Models Main 7 页 · PDF
147 MiLQ: Benchmarking IR Models for Bilingual Web Search with Mixed Language Queries Main 7 页 · PDF
148 Mind the Inclusivity Gap: Multilingual Gender-Neutral Translation Evaluation with mGeNTE Main 7 页 · PDF
149 MMDocIR: Benchmarking Multimodal Retrieval for Long Documents Main 7 页 · PDF
150 MolErr2Fix: Benchmarking LLM Trustworthiness in Chemistry via Modular Error Detection, Localization, Explanation, and Correction Main 7 页 · PDF

评论