{"arxiv_id":"2608.21766","incoming":{"papers":0,"sentences":0,"groups":[],"unlabelled_sentences":0},"outgoing":{"papers":17,"sentences":30,"groups":[{"kind":"method_or_tool","label":"Uses the method or tool of","papers":2,"sentences":2,"quotes":[{"arxiv_id":"2308.10248","title":"Steering Language Models With Activation Engineering","url":"/paper/2308.10248","section":"Methodology","sentence":"To steer models towards or against the direction of evaluation awareness previously extracted, we follow the protocol of Contrastive Activation Addition [ref] .","intent_p":1.0,"contests_p":null},{"arxiv_id":"2510.20487","title":null,"url":"/paper/2510.20487","section":"Methodology","sentence":"We replicate the construction of linear probes from [ref] , using the same $n = 16$ contrastive pairs ${x_i = (x_i^+, x_i^-) |1 ≤i ≤n}$.","intent_p":1.0,"contests_p":null}]},{"kind":"data_used","label":"Uses data or values from","papers":3,"sentences":3,"quotes":[{"arxiv_id":"2110.14168","title":"Training Verifiers to Solve Math Word Problems","url":"/paper/2110.14168","section":"Methodology","sentence":"Instead, to cover a diverse set of evaluations, distinct in scope and style, we source evaluation prompts from three benchmarks: GSM8K [ref] (grade school math word problems), MASK [ref] (a dataset for measuring honesty of language models), and TruthfulQA [ref] (a dataset to measure if language models are truthful in generating answers to questions).","intent_p":0.81,"contests_p":null},{"arxiv_id":"2109.07958","title":"TruthfulQA: Measuring How Models Mimic Human Falsehoods","url":"/paper/2109.07958","section":"Methodology","sentence":"Instead, to cover a diverse set of evaluations, distinct in scope and style, we source evaluation prompts from three benchmarks: GSM8K [ref] (grade school math word problems), MASK [ref] (a dataset for measuring honesty of language models), and TruthfulQA [ref] (a dataset to measure if language models are truthful in generating answers to questions).","intent_p":0.81,"contests_p":null},{"arxiv_id":"2503.03750","title":null,"url":"/paper/2503.03750","section":"Methodology","sentence":"Instead, to cover a diverse set of evaluations, distinct in scope and style, we source evaluation prompts from three benchmarks: GSM8K [ref] (grade school math word problems), MASK [ref] (a dataset for measuring honesty of language models), and TruthfulQA [ref] (a dataset to measure if language models are truthful in generating answers to questions).","intent_p":0.81,"contests_p":null}]},{"kind":"supports_this_paper","label":"Cites as support","papers":6,"sentences":6,"quotes":[{"arxiv_id":"2505.23836","title":"Large Language Models Often Know When They Are Being Evaluated","url":"/paper/2505.23836","section":"Introduction","sentence":"In practice, evaluation awareness has been observed in open-weight models [ref] and in proprietary models alike [ref] , culminating in the International AI Safety Report, which lists it as a key development of the past year [ref] .","intent_p":0.96,"contests_p":null},{"arxiv_id":"2509.00591","title":null,"url":"/paper/2509.00591","section":"Introduction","sentence":"In practice, evaluation awareness has been observed in open-weight models [ref] and in proprietary models alike [ref] , culminating in the International AI Safety Report, which lists it as a key development of the past year [ref] .","intent_p":0.96,"contests_p":null},{"arxiv_id":"2510.20487","title":null,"url":"/paper/2510.20487","section":"Introduction","sentence":"In practice, evaluation awareness has been observed in open-weight models [ref] and in proprietary models alike [ref] , culminating in the International AI Safety Report, which lists it as a key development of the past year [ref] .","intent_p":0.96,"contests_p":null},{"arxiv_id":"2602.21012","title":null,"url":"/paper/2602.21012","section":"Introduction","sentence":"In practice, evaluation awareness has been observed in open-weight models [ref] and in proprietary models alike [ref] , culminating in the International AI Safety Report, which lists it as a key development of the past year [ref] .","intent_p":0.96,"contests_p":null},{"arxiv_id":"2505.14617","title":null,"url":"/paper/2505.14617","section":"Introduction","sentence":"In practice, evaluation awareness has been observed in open-weight models [ref] and in proprietary models alike [ref] , culminating in the International AI Safety Report, which lists it as a key development of the past year [ref] .","intent_p":0.96,"contests_p":null}]},{"kind":"background","label":"Cites as background","papers":8,"sentences":11,"quotes":[{"arxiv_id":"2505.23836","title":"Large Language Models Often Know When They Are Being Evaluated","url":"/paper/2505.23836","section":"Related work","sentence":"The term was introduced by [ref] , who showed that frontier models can distinguish evaluation transcripts from deployment-like interactions.","intent_p":0.96,"contests_p":null},{"arxiv_id":"2604.17465","title":"Language models recognize dropout and Gaussian noise applied to their activations","url":"/paper/2604.17465","section":"Introduction","sentence":"Located within the broader context of “situational awareness” [ref] , evaluation awareness shares concerns with “alignment faking” [ref] , “scheming” [ref] , “deceptive alignment” [ref] , and “training awareness” [ref] .","intent_p":0.99,"contests_p":null},{"arxiv_id":"2412.04984","title":"Frontier Models are Capable of In-context Scheming","url":"/paper/2412.04984","section":"Introduction","sentence":"Located within the broader context of “situational awareness” [ref] , evaluation awareness shares concerns with “alignment faking” [ref] , “scheming” [ref] , “deceptive alignment” [ref] , and “training awareness” [ref] .","intent_p":0.99,"contests_p":null},{"arxiv_id":"2209.00626","title":"The Alignment Problem from a Deep Learning Perspective","url":"/paper/2209.00626","section":"Introduction","sentence":"Located within the broader context of “situational awareness” [ref] , evaluation awareness shares concerns with “alignment faking” [ref] , “scheming” [ref] , “deceptive alignment” [ref] , and “training awareness” [ref] .","intent_p":0.99,"contests_p":null},{"arxiv_id":"2309.00667","title":"Taken out of context: On measuring situational awareness in LLMs","url":"/paper/2309.00667","section":"Introduction","sentence":"Located within the broader context of “situational awareness” [ref] , evaluation awareness shares concerns with “alignment faking” [ref] , “scheming” [ref] , “deceptive alignment” [ref] , and “training awareness” [ref] .","intent_p":0.99,"contests_p":null}]}],"unlabelled_sentences":8},"claims":[],"thresholds":{"intent_p":0.8,"supports_p":0.95,"contests_p":0.7},"method":"Citing sentences come from each paper's LaTeX source; every \\cite marks the sentence and the works it cites. Pith's classifier typed each sentence. A label is shown at intent probability 0.8 or more (0.95 for support), a contest at 0.7 or more. Measured accuracy against two blind model families is at https://pith.science/accuracy. The quoted sentences are the citing authors' own.","held_up":null,"_links":{"html":"https://pith.science/paper/2608.21766"}}