{"arxiv_id":"2311.09325","incoming":{"papers":2,"sentences":3,"groups":[{"kind":"extends","label":"Builds on it","papers":1,"sentences":1,"quotes":[{"arxiv_id":"2311.07484","title":"Psychometric Predictive Power of Large Language Models","url":"/paper/2311.07484","section":"Reproducing previous findings","sentence":"Such an advantage has been reported with GPT-2 models [ref] ; we further show that this generalizes to other LLMs.","intent_p":0.95,"contests_p":null}]}],"unlabelled_sentences":2},"outgoing":{"papers":7,"sentences":9,"groups":[{"kind":"method_or_tool","label":"Uses the method or tool of","papers":1,"sentences":1,"quotes":[{"arxiv_id":"2006.01912","title":"On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior","url":"/paper/2006.01912","section":"Metrics and evaluation","sentence":"We measure the predictive power of surprisal estimates from different language models, which is denoted as the log-likelihood difference per data point between a linear mixed-effects (LME) regression model using lme4 package [ref] with a predictor of surprisal estimates (target model) and a model without surprisal (base model), following [ref] .","intent_p":1.0,"contests_p":null}]},{"kind":"background","label":"Cites as background","papers":1,"sentences":1,"quotes":[{"arxiv_id":"1503.02531","title":"Distilling the Knowledge in a Neural Network","url":"/paper/1503.02531","section":"Temperature-scaled surprisal","sentence":"Temperature scaling has been successfully applied in several applications: In knowledge distillation [ref] , temperature scaling (with $T>1$) is used to “soften” the knowledge (i.e., probability distribution) provided by the teacher model; in text generation, temperature is used to shape the probability distribution to ease certain aspects of the problems of top-k sampling (e.g., choosing an appropriate $k$ value across varying contexts) [ref] .","intent_p":0.88,"contests_p":null}]}],"unlabelled_sentences":7},"claims":[],"thresholds":{"intent_p":0.8,"supports_p":0.95,"contests_p":0.7},"method":"Citing sentences come from each paper's LaTeX source; every \\cite marks the sentence and the works it cites. Pith's classifier typed each sentence. A label is shown at intent probability 0.8 or more (0.95 for support), a contest at 0.7 or more. Measured accuracy against two blind model families is at https://pith.science/accuracy. The quoted sentences are the citing authors' own.","held_up":null,"_links":{"html":"https://pith.science/paper/2311.09325"}}