Using Schelling's model, the authors find that LLM-generated move/stay advice leads to a similarly segregated society across models and demographic categories, regardless of the model's measured bias.
LangBiTe: A Platform for Testing Bias in Large Language Models
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
The integration of Large Language Models (LLMs) into various software applications raises concerns about their potential biases. Typically, those models are trained on a vast amount of data scrapped from forums, websites, social media and other internet sources, which may instill harmful and discriminating behavior into the model. To address this issue, we present LangBiTe, a testing platform to systematically assess the presence of biases within an LLM. LangBiTe enables development teams to tailor their test scenarios, and automatically generate and execute the test cases according to a set of user-defined ethical requirements. Each test consists of a prompt fed into the LLM and a corresponding test oracle that scrutinizes the LLM's response for the identification of biases. LangBite provides users with the bias evaluation of LLMs, and end-to-end traceability between the initial ethical requirements and the insights obtained.
fields
physics.soc-ph 1years
2024 1verdicts
REJECT 1representative citing papers
citing papers explorer
-
Observing Micromotives and Macrobehavior of Large Language Models
Using Schelling's model, the authors find that LLM-generated move/stay advice leads to a similarly segregated society across models and demographic categories, regardless of the model's measured bias.