JudgmentBench supplies the first public paired rubric and preference annotations from legal experts on the same LLM outputs, showing comparative judgments outperform rubrics in recovering quality orderings.
hub
In: 2018 IEEE 31st Computer Security Foundations Symposium (CSF)
12 Pith papers cite this work, alongside 29 external citations. Polarity classification is still indexing.
12
Pith papers citing it
29
external citations · external index
hub tools
representative citing papers
MM-Eval unifies evaluation of multimodal summaries by integrating factual text quality, cross-modal relevance via MLLM judge, and visual diversity via truncated CLIP entropy, then calibrates their combination on human preferences.
SPeCTrA-Sum uses hierarchical cross-modal fusion via DVP and DPP-distilled image selection via VRP to generate more accurate and visually grounded multimodal summaries.
IRAP quantifies ambiguous performance requirements into mathematical functions via interactive retrieval-augmented preference elicitation and outperforms ten prior methods on four real-world datasets with up to 40x gains in five interaction rounds.
Flipping 1-2 sign bits in DNN parameters, located without data or optimization, drops accuracy to near zero across image classification, detection, segmentation, and language models.
Annotary is a concolic execution system for smart contract vulnerability detection that uses source-code annotations, symbolic EVM execution, and blockchain data resolution to handle inter-contract and inter-transaction flows.
Learning progression-derived rubrics produce AI feedback on student science writing that matches expert rubric quality in key dimensions.
SPEAR applies multi-agent systems with planning, execution, and repair agents using negotiation protocols to smart contract auditing and compares it empirically to centralized and pipeline approaches.
Active learning for optimal experimental design in ML-based building energy system identification yields up to 54% lower RMSE than passive random sampling on the BOPTEST simulator across neural network and Gaussian process models.
Empirical review of 233 real-world vulnerabilities from 34 TON audits produces a specialized checklist for asynchronous message handling, supported by case studies and an 11-person practitioner survey.
A study of 143 students found that rubric-based generative AI feedback helped students revise writing for better organization and style, saved teachers time, but produced inconsistent automated ratings.
LLM graders achieve substantial human agreement on math and science MCAS items but vary on ELA, performing best as sources of formative narrative feedback rather than summative numerical scores.
citing papers explorer
-
JudgmentBench: Comparing Rubric and Preference Evaluation for Quality Assessment
JudgmentBench supplies the first public paired rubric and preference annotations from legal experts on the same LLM outputs, showing comparative judgments outperform rubrics in recovering quality orderings.
-
Measuring What Matters Beyond Text: Evaluating Multimodal Summaries by Quality, Alignment, and Diversity
MM-Eval unifies evaluation of multimodal summaries by integrating factual text quality, cross-modal relevance via MLLM judge, and visual diversity via truncated CLIP entropy, then calibrates their combination on human preferences.
-
Towards Visually Grounded Multimodal Summarization via Cross-Modal Transformer and Gated Attention
SPeCTrA-Sum uses hierarchical cross-modal fusion via DVP and DPP-distilled image selection via VRP to generate more accurate and visually grounded multimodal summaries.
-
Conjecture and Inquiry: Quantifying Software Performance Requirements via Interactive Retrieval-Augmented Preference Elicitation
IRAP quantifies ambiguous performance requirements into mathematical functions via interactive retrieval-augmented preference elicitation and outperforms ten prior methods on four real-world datasets with up to 40x gains in five interaction rounds.
-
Maximal Brain Damage Without Data or Optimization: Disrupting Neural Networks via Sign-Bit Flips
Flipping 1-2 sign bits in DNN parameters, located without data or optimization, drops accuracy to near zero across image classification, detection, segmentation, and language models.
-
Annotary: A Concolic Execution System for Developing Secure Smart Contracts
Annotary is a concolic execution system for smart contract vulnerability detection that uses source-code annotations, symbolic EVM execution, and blockchain data resolution to handle inter-contract and inter-transaction flows.
-
Using Learning Progressions to Guide AI Feedback for Science Learning
Learning progression-derived rubrics produce AI feedback on student science writing that matches expert rubric quality in key dimensions.
-
SPEAR: An Engineering Case Study of Multi-Agent Coordination for Smart Contract Auditing
SPEAR applies multi-agent systems with planning, execution, and repair agents using negotiation protocols to smart contract auditing and compares it empirically to centralized and pipeline approaches.
-
Active Learning for Optimal Experimental Design in Machine Learning-Based Building Energy System Identification
Active learning for optimal experimental design in ML-based building energy system identification yields up to 54% lower RMSE than passive random sampling on the BOPTEST simulator across neural network and Gaussian process models.
-
From Paradigm Shift to Audit Rift: Empirical Analysis and Validation of Security Audit Methodologies for Asynchronous Smart Contract Systems
Empirical review of 233 real-world vulnerabilities from 34 TON audits produces a specialized checklist for asynchronous message handling, supported by case studies and an 11-person practitioner survey.
-
Generative AI Feedback, English Writing and Teacher Rubrics: A Multiple-Case Study of CyberScholar
A study of 143 students found that rubric-based generative AI feedback helped students revise writing for better organization and style, saved teachers time, but produced inconsistent automated ratings.
-
Creating and Evaluating K-12 GenAI Assessment Graders Through Context Engineering
LLM graders achieve substantial human agreement on math and science MCAS items but vary on ELA, performing best as sources of formative narrative feedback rather than summative numerical scores.