EMNLP 2025 · 全部收录(315 篇)¶
仅列出通过预过滤且已打分的论文;分数是「评测方法学价值」,不是会议录用质量。 返回总览
| # | 论文 | 轨道 | 分数 | 链接 |
|---|---|---|---|---|
| 1 | Analyzing Uncertainty of LLM-as-a-Judge: Interval Evaluations with Conformal Prediction | Main | 9 | 页 · PDF |
| 2 | Benchmark Profiling: Mechanistic Diagnosis of LLM Benchmarks | Main | 9 | 页 · PDF |
| 3 | Benchmarking Large Language Models Under Data Contamination: A Survey from Static to Dynamic Evaluation | Main | 9 | 页 · PDF |
| 4 | Beyond the Surface: Measuring Self-Preference in LLM Judgments | Main | 9 | 页 · PDF |
| 5 | Can Large Language Models Outperform Non-Experts in Poetry Evaluation? A Comparative Study Using the Consensual Assessment Technique | Main | 9 | 页 · PDF |
| 6 | CheckEval: A reliable LLM-as-a-Judge framework for evaluating text generation using checklists | Main | 9 | 页 · PDF |
| 7 | DCR: Quantifying Data Contamination in LLMs Evaluation | Main | 9 | 页 · PDF |
| 8 | DyePack: Provably Flagging Test Set Contamination in LLMs Using Backdoors | Main | 9 | 页 · PDF |
| 9 | Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs | Main | 9 | 页 · PDF |
| 10 | Languages Still Left Behind: Toward a Better Multilingual Machine Translation Benchmark | Main | 9 | 页 · PDF |
| 11 | PERSEVAL: A Framework for Perspectivist Classification Evaluation | Main | 9 | 页 · PDF |
| 12 | Reliable Evaluation and Benchmarks for Statement Autoformalization | Main | 9 | 页 · PDF |
| 13 | SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants? | Main | 9 | 页 · PDF |
| 14 | SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction | Main | 9 | 页 · PDF |
| 15 | The Illusion of Progress: Re-evaluating Hallucination Detection in LLMs | Main | 9 | 页 · PDF |
| 16 | A Causal Lens for Evaluating Faithfulness Metrics | Main | 8 | 页 · PDF |
| 17 | A Fully Probabilistic Perspective on Large Language Model Unlearning: Evaluation and Optimization | Main | 8 | 页 · PDF |
| 18 | A Necessary Step toward Faithfulness: Measuring and Improving Consistency in Free-Text Explanations | Main | 8 | 页 · PDF |
| 19 | Agent-as-Judge for Factual Summarization of Long Narratives | Main | 8 | 页 · PDF |
| 20 | Are Checklists Really Useful for Automatic Evaluation of Generative Tasks? | Main | 8 | 页 · PDF |
| 21 | Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performance | Main | 8 | 页 · PDF |
| 22 | Arena-lite: Efficient and Reliable Large Language Model Evaluation via Tournament-Based Direct Comparisons | Main | 8 | 页 · PDF |
| 23 | Benchmarking Debiasing Methods for LLM-based Parameter Estimates | Main | 8 | 页 · PDF |
| 24 | Beyond the Score: Uncertainty-Calibrated LLMs for Automated Essay Assessment | Main | 8 | 页 · PDF |
| 25 | Blind Men and the Elephant: Diverse Perspectives on Gender Stereotypes in Benchmark Datasets | Main | 8 | 页 · PDF |
| 26 | Break the Checkbox: Challenging Closed-Style Evaluations of Cultural Alignment in LLMs | Main | 8 | 页 · PDF |
| 27 | Can LLMs Help You at Work? A Sandbox for Evaluating LLM Agents in Enterprise Environments | Main | 8 | 页 · PDF |
| 28 | ClimateViz: A Benchmark for Statistical Reasoning and Fact Verification on Scientific Charts | Main | 8 | 页 · PDF |
| 29 | Co-Eval: Augmenting LLM-based Evaluation with Machine Metrics | Main | 8 | 页 · PDF |
| 30 | CREPE: Rapid Chest X-ray Report Evaluation by Predicting Multi-category Error Counts | Main | 8 | 页 · PDF |
| 31 | Debatable Intelligence: Benchmarking LLM Judges via Debate Speech Evaluation | Main | 8 | 页 · PDF |
| 32 | Don’t Sweat the Small Stuff: Segment-Level Meta-Evaluation Based on Pairwise Difference Correlation | Main | 8 | 页 · PDF |
| 33 | ESC-Judge: A Framework for Comparing Emotional Support Conversational Agents | Main | 8 | 页 · PDF |
| 34 | Evaluating Language Translation Models by Playing Telephone | Main | 8 | 页 · PDF |
| 35 | Evaluating the Evaluators: Are readability metrics good measures of readability? | Main | 8 | 页 · PDF |
| 36 | Fooling the LVLM Judges: Visual Biases in LVLM-Based Evaluation | Main | 8 | 页 · PDF |
| 37 | Forget What You Know about LLMs Evaluations - LLMs are Like a Chameleon | Main | 8 | 页 · PDF |
| 38 | F²Bench: An Open-ended Fairness Evaluation Benchmark for LLMs with Factuality Considerations | Main | 8 | 页 · PDF |
| 39 | Graders Should Cheat: Privileged Information Enables Expert-Level Automated Evaluations | Main | 8 | 页 · PDF |
| 40 | GuessingGame: Measuring the Informativeness of Open-Ended Questions in Large Language Models | Main | 8 | 页 · PDF |
| 41 | How Far Can LLMs Improve from Experience? Measuring Test-Time Learning Ability in LLMs with Human Comparison | Main | 8 | 页 · PDF |
| 42 | ImpliRet: Benchmarking the Implicit Fact Retrieval Challenge | Main | 8 | 页 · PDF |
| 43 | In Benchmarks We Trust ... Or Not? | Main | 8 | 页 · PDF |
| 44 | Investigating How Pre-training Data Leakage Affects Models’ Reproduction and Detection Capabilities | Main | 8 | 页 · PDF |
| 45 | Is the Top Still Spinning? Evaluating Subjectivity in Narrative Understanding | Main | 8 | 页 · PDF |
| 46 | iVISPAR — An Interactive Visual-Spatial Reasoning Benchmark for VLMs | Main | 8 | 页 · PDF |
| 47 | LiteraryQA: Towards Effective Evaluation of Long-document Narrative QA | Main | 8 | 页 · PDF |
| 48 | Long-Form Information Alignment Evaluation Beyond Atomic Facts | Main | 8 | 页 · PDF |
| 49 | MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors | Main | 8 | 页 · PDF |
| 50 | Measuring Bias or Measuring the Task: Understanding the Brittle Nature of LLM Gender Biases | Main | 8 | 页 · PDF |
| 51 | Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps | Main | 8 | 页 · PDF |
| 52 | Measuring scalar constructs in social science with LLMs | Main | 8 | 页 · PDF |
| 53 | Mechanisms vs. Outcomes: Probing for Syntax Fails to Explain Performance on Targeted Syntactic Evaluations | Main | 8 | 页 · PDF |
| 54 | Metric Calculating Benchmark: Code-Verifiable Complicate Instruction Following Benchmark for Large Language Models | Main | 8 | 页 · PDF |
| 55 | Mind the Blind Spots: A Focus-Level Evaluation Framework for LLM Reviews | Main | 8 | 页 · PDF |
| 56 | MR. Judge: Multimodal Reasoner as a Judge | Main | 8 | 页 · PDF |
| 57 | Not-Just-Scaling Laws: Towards a Better Understanding of the Downstream Impact of Language Model Design Decisions | Main | 8 | 页 · PDF |
| 58 | PoSum-Bench: Benchmarking Position Bias in LLM-based Conversational Summarization | Main | 8 | 页 · PDF |
| 59 | Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance | Main | 8 | 页 · PDF |
| 60 | ProcWorld: Benchmarking Large Model Planning in Reachability-Constrained Environments | Main | 8 | 页 · PDF |
| 61 | ReEvalMed: Rethinking Medical Report Evaluation by Aligning Metrics with Real-World Clinical Judgment | Main | 8 | 页 · PDF |
| 62 | Rethinking Backdoor Detection Evaluation for Language Models | Main | 8 | 页 · PDF |
| 63 | reWordBench: Benchmarking and Improving the Robustness of Reward Models with Transformed Inputs | Main | 8 | 页 · PDF |
| 64 | Same evaluation, more tokens: On the effect of input length for machine translation evaluation using Large Language Models | Main | 8 | 页 · PDF |
| 65 | Seeing is Believing, but How Much? A Comprehensive Analysis of Verbalized Calibration in Vision-Language Models | Main | 8 | 页 · PDF |
| 66 | seqBench: A Tunable Benchmark to Quantify Sequential Reasoning Limits of LLMs | Main | 8 | 页 · PDF |
| 67 | SSA-COMET: Do LLMs Outperform Learned Metrics in Evaluating MT for Under-Resourced African Languages? | Main | 8 | 页 · PDF |
| 68 | SSA: Semantic Contamination of LLM-Driven Fake News Detection | Main | 8 | 页 · PDF |
| 69 | SYNC: A Synthetic Long-Context Understanding Benchmark for Controlled Comparisons of Model Capabilities | Main | 8 | 页 · PDF |
| 70 | TCP: a Benchmark for Temporal Constraint-Based Planning | Main | 8 | 页 · PDF |
| 71 | The LLM Already Knows: Estimating LLM-Perceived Question Difficulty via Hidden Representations | Main | 8 | 页 · PDF |
| 72 | ToM-SSI: Evaluating Theory of Mind in Situated Social Interactions | Main | 8 | 页 · PDF |
| 73 | Towards a Holistic and Automated Evaluation Framework for Multi-Level Comprehension of LLMs in Book-Length Contexts | Main | 8 | 页 · PDF |
| 74 | Towards Optimal Evaluation Efficiency for Large Language Models | Main | 8 | 页 · PDF |
| 75 | Translationese-index: Using Likelihood Ratios for Graded and Generalizable Measurement of Translationese | Main | 8 | 页 · PDF |
| 76 | TSVer: A Benchmark for Fact Verification Against Time-Series Evidence | Main | 8 | 页 · PDF |
| 77 | TTT-Bench: A Benchmark for Evaluating Reasoning Ability with Simple and Novel Tic-Tac-Toe-style Games | Main | 8 | 页 · PDF |
| 78 | Unlearning vs. Obfuscation: Are We Truly Removing Knowledge? | Main | 8 | 页 · PDF |
| 79 | VELA: An LLM-Hybrid-as-a-Judge Approach for Evaluating Long Image Captions | Main | 8 | 页 · PDF |
| 80 | VeriFact: Enhancing Long-Form Factuality Evaluation with Refined Fact Extraction and Reference Facts | Main | 8 | 页 · PDF |
| 81 | We Need to Measure Data Diversity in NLP — Better and Broader | Main | 8 | 页 · PDF |
| 82 | What You See is What You Ask: Evaluating Audio Descriptions | Main | 8 | 页 · PDF |
| 83 | XLQA: A Benchmark for Locale-Aware Multilingual Open-Domain Question Answering | Main | 8 | 页 · PDF |
| 84 | 3MDBench: Medical Multimodal Multi-agent Dialogue Benchmark | Main | 7 | 页 · PDF |
| 85 | A Culturally-diverse Multilingual Multimodal Video Benchmark & Model | Main | 7 | 页 · PDF |
| 86 | A Position Paper on the Automatic Generation of Machine Learning Leaderboards | Main | 7 | 页 · PDF |
| 87 | AbsVis – Benchmarking How Humans and Vision-Language Models “See” Abstract Concepts in Images | Main | 7 | 页 · PDF |
| 88 | AccessEval: Benchmarking Disability Bias in Large Language Models | Main | 7 | 页 · PDF |
| 89 | Adapting Bias Evaluation to Domain Contexts using Generative Models | Main | 7 | 页 · PDF |
| 90 | Advancing Arabic Diacritization: Improved Datasets, Benchmarking, and State-of-the-Art Models | Main | 7 | 页 · PDF |
| 91 | AesBiasBench: Evaluating Bias and Alignment in Multimodal Language Models for Personalized Image Aesthetic Assessment | Main | 7 | 页 · PDF |
| 92 | Annotating Training Data for Conditional Semantic Textual Similarity Measurement using Large Language Models | Main | 7 | 页 · PDF |
| 93 | Are Vision-Language Models Safe in the Wild? A Meme-Based Benchmark Study | Main | 7 | 页 · PDF |
| 94 | Argument Summarization and its Evaluation in the Era of Large Language Models | Main | 7 | 页 · PDF |
| 95 | Audio-centric Video Understanding Benchmark without Text Shortcut | Main | 7 | 页 · PDF |
| 96 | Benchmarking and Mitigating MCQA Selection Bias of Large Vision-Language Models | Main | 7 | 页 · PDF |
| 97 | Beyond Human Labels: A Multi-Linguistic Auto-Generated Benchmark for Evaluating Large Language Models on Resume Parsing | Main | 7 | 页 · PDF |
| 98 | Beyond Static Testbeds: An Interaction-Centric Agent Simulation Platform for Dynamic Recommender Systems | Main | 7 | 页 · PDF |
| 99 | BOUQuET : dataset, Benchmark and Open initiative for Universal Quality Evaluation in Translation | Main | 7 | 页 · PDF |
| 100 | C3: A Bilingual Benchmark for Spoken Dialogue Models Exploring Challenges in Complex Conversations | Main | 7 | 页 · PDF |
| 101 | Calibrating LLMs for Text-to-SQL Parsing by Leveraging Sub-clause Frequencies | Main | 7 | 页 · PDF |
| 102 | Calibration Across Layers: Understanding Calibration Evolution in LLMs | Main | 7 | 页 · PDF |
| 103 | Can Prompts Rewind Time for LLMs? Evaluating the Effectiveness of Prompted Knowledge Cutoffs | Main | 7 | 页 · PDF |
| 104 | CausalVLBench: Benchmarking Visual Causal Reasoning in Large Vision-Language Models | Main | 7 | 页 · PDF |
| 105 | CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Space | Main | 7 | 页 · PDF |
| 106 | Collab-Overcooked: Benchmarking and Evaluating Large Language Models as Collaborative Agents | Main | 7 | 页 · PDF |
| 107 | CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward | Main | 7 | 页 · PDF |
| 108 | CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answering | Main | 7 | 页 · PDF |
| 109 | Contra4: Evaluating Contrastive Cross-Modal Reasoning in Audio, Video, Image, and 3D | Main | 7 | 页 · PDF |
| 110 | CourtReasoner: Can LLM Agents Reason Like Judges? | Main | 7 | 页 · PDF |
| 111 | Deep Associations, High Creativity: A Simple yet Effective Metric for Evaluating Large Language Models | Main | 7 | 页 · PDF |
| 112 | Direct Judgement Preference Optimization | Main | 7 | 页 · PDF |
| 113 | DischargeSim: A Simulation Benchmark for Educational Doctor–Patient Communication at Discharge | Main | 7 | 页 · PDF |
| 114 | Do Large Language Models Truly Grasp Addition? A Rule-Focused Diagnostic Using Two-Integer Arithmetic | Main | 7 | 页 · PDF |
| 115 | EGOILLUSION: Benchmarking Hallucinations in Egocentric Video Understanding | Main | 7 | 页 · PDF |
| 116 | EquiBench: Benchmarking Large Language Models’ Reasoning about Program Semantics via Equivalence Checking | Main | 7 | 页 · PDF |
| 117 | Estimating LLM Consistency: A User Baseline vs Surrogate Metrics | Main | 7 | 页 · PDF |
| 118 | Evaluating LLM-Generated Diagrams as Graphs | Main | 7 | 页 · PDF |
| 119 | Evaluating Spatiotemporal Consistency in Automatically Generated Sewing Instructions | Main | 7 | 页 · PDF |
| 120 | Exploring Response Uncertainty in MLLMs: An Empirical Evaluation under Misleading Scenarios | Main | 7 | 页 · PDF |
| 121 | Extending Automatic Machine Translation Evaluation to Book-Length Documents | Main | 7 | 页 · PDF |
| 122 | F2TEval: Human-Aligned Multi-Dimensional Evaluation for Figure-to-Text Task | Main | 7 | 页 · PDF |
| 123 | Fann or Flop: A Multigenre, Multiera Benchmark for Arabic Poetry Understanding in LLMs | Main | 7 | 页 · PDF |
| 124 | FB-Bench: A Fine-Grained Multi-Task Benchmark for Evaluating LLMs’ Responsiveness to Human Feedback | Main | 7 | 页 · PDF |
| 125 | FinRAGBench-V: A Benchmark for Multimodal RAG with Visual Citation in the Financial Domain | Main | 7 | 页 · PDF |
| 126 | FlashAdventure: A Benchmark for GUI Agents Solving Full Story Arcs in Diverse Adventure Games | Main | 7 | 页 · PDF |
| 127 | From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge | Main | 7 | 页 · PDF |
| 128 | From Understanding to Generation: An Efficient Shortcut for Evaluating Language Models | Main | 7 | 页 · PDF |
| 129 | HESEIA: A community-based dataset for evaluating social biases in large language models, co-designed in real school settings in Latin America | Main | 7 | 页 · PDF |
| 130 | How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark | Main | 7 | 页 · PDF |
| 131 | Incorporating Diverse Perspectives in Cultural Alignment: Survey of Evaluation Benchmarks Through A Three-Dimensional Framework | Main | 7 | 页 · PDF |
| 132 | InfiniBench: A Benchmark for Large Multi-Modal Models in Long-Form Movies and TV Shows | Main | 7 | 页 · PDF |
| 133 | InMind: Evaluating LLMs in Capturing and Applying Individual Human Reasoning Styles | Main | 7 | 页 · PDF |
| 134 | Investigating Value-Reasoning Reliability in Small Large Language Models | Main | 7 | 页 · PDF |
| 135 | JUDGEBERT: Assessing Legal Meaning Preservation Between Sentences | Main | 7 | 页 · PDF |
| 136 | LaMP-QA: A Benchmark for Personalized Long-form Question Answering | Main | 7 | 页 · PDF |
| 137 | LASER: An LLM-based ASR Scoring and Evaluation Rubric | Main | 7 | 页 · PDF |
| 138 | Leveraging Large Models to Evaluate Novel Content: A Case Study on Advertisement Creativity | Main | 7 | 页 · PDF |
| 139 | LingGym: How Far Are LLMs from Thinking Like Field Linguists? | Main | 7 | 页 · PDF |
| 140 | LMR-BENCH: Evaluating LLM Agent’s Ability on Reproducing Language Modeling Research | Main | 7 | 页 · PDF |
| 141 | LORAXBENCH: A Multitask, Multilingual Benchmark Suite for 20 Indonesian Languages | Main | 7 | 页 · PDF |
| 142 | Mahānāma: A Unique Testbed for Literary Entity Discovery and Linking | Main | 7 | 页 · PDF |
| 143 | Measuring Risk of Bias in Biomedical Reports: The RoBBR Benchmark | Main | 7 | 页 · PDF |
| 144 | Measuring the Effect of Disfluency in Multilingual Knowledge Probing Benchmarks | Main | 7 | 页 · PDF |
| 145 | MEBench: Benchmarking Large Language Models for Cross-Document Multi-Entity Question Answering | Main | 7 | 页 · PDF |
| 146 | MemeArena: Automating Context-Aware Unbiased Evaluation of Harmfulness Understanding for Multimodal Large Language Models | Main | 7 | 页 · PDF |
| 147 | MiLQ: Benchmarking IR Models for Bilingual Web Search with Mixed Language Queries | Main | 7 | 页 · PDF |
| 148 | Mind the Inclusivity Gap: Multilingual Gender-Neutral Translation Evaluation with mGeNTE | Main | 7 | 页 · PDF |
| 149 | MMDocIR: Benchmarking Multimodal Retrieval for Long Documents | Main | 7 | 页 · PDF |
| 150 | MolErr2Fix: Benchmarking LLM Trustworthiness in Chemistry via Modular Error Detection, Localization, Explanation, and Correction | Main | 7 | 页 · PDF |