REVIEW 2 major objections 2 minor 61 references
Private Noise and Public Error in Collective Information Acquisition
T0 review · 2 major / 2 minor · reviewed 2026-06-28 · grok-4.3
Pith's one-line read Production noise creates shared errors that groups lock onto longer than private comprehension noise does.
desk verdict The experiment shows production noise creates more persistent wrong consensus than comprehension noise via correlated perturbations, but the design leaves room for unmeasured shifts in how people weight the private cue versus social info. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The production-versus-comprehension noise distinction, which controls whether perturbations are correlated across receivers and therefore whether social updating treats noise as common evidence.
What would settle it
A follow-up experiment that keeps production noise but forces the stored perturbations to be independent for each receiver should eliminate the elevated persistence on wrong values if the correlation mechanism is the cause.
Extended reading notes
Core claim
Production noise more often generated a wrong common signal and caused that signal to persist across more rounds than comprehension noise because peer influence acted on more correlated perturbations, thereby converting individual errors into stable group consensus on error.
Load-bearing premise
The experimental conditions differ only in the correlation structure of the noise, with no other systematic differences in how participants interpret or weight the thermometer cue versus social information.
Editorial extensions
If this is right
- Collective consensus can stabilize around error when noise is public rather than private even if the average strength of social influence stays constant.
- Comprehension noise can sometimes increase the chance of escaping an incorrect consensus compared with fully accurate social information.
- Models of social learning must track the correlation structure of noise separately from the magnitude of influence to predict when groups will lock onto mistakes.
- Interventions that reduce shared perturbations (for example by randomizing displayed social information) may shorten erroneous consensus without changing how much people trust peers.
Reading between the lines
- In real-world settings such as online platforms, a single noisy source visible to many people may be more damaging to accuracy than equivalent but independent misperceptions across individuals.
- The GPT-agent result suggests the production-noise vulnerability depends on human-specific patterns of uncertainty registration, so purely algorithmic groups might respond differently.
- Extending the design to larger groups or continuous rather than discrete temperature estimates would test whether the correlation effect scales or saturates.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript reports an online experiment with 600 participants in four-person groups estimating room temperature over 25 rounds under faithful, comprehension-noise (independent perturbations), and production-noise (shared perturbations) conditions. It claims production noise more frequently creates and sustains wrong common signals (Fisher's exact p=0.025; permutation p=0.004) than comprehension noise because peer influence operates on correlated perturbations rather than because participants follow peers more strongly, as shown by dynamic update models; comprehension noise can sometimes improve correction relative to faithful information. A GPT-agent experiment is used to probe boundary conditions.
Significance. If the central claim holds, the work demonstrates that noise correlation structure, rather than noise per se, can stabilize collective error in social learning tasks, distinguishing production from comprehension effects and offering a mechanism-based account supported by permutation tests, Fisher's exact test, and fitted dynamic models. The empirical design with independent conditions and the contrast with GPT agents provide concrete, falsifiable evidence on how communication noise shapes group consensus.
major comments (2)
- [Dynamic update models and experimental conditions] The dynamic update models' attribution that peer influence strength is equivalent across conditions (and thus that correlation alone explains the production-noise effect) is load-bearing for the central claim. The task design notes subjective uncertainty in the thermometer cue and a unitless 50-250 range that conflicts with everyday expectations; production noise (identical perturbations visible to multiple receivers) could systematically alter participants' weighting of social versus private information relative to comprehension noise in ways not captured by the model fits.
- [Dynamic update models] The reported group-level permutation test (p=0.016 for rounds tightly clustered around a wrong value) and the mechanism conclusion require explicit confirmation that influence parameters are statistically indistinguishable across conditions after accounting for any condition-specific differences in cue interpretation. Without details on model fitting procedures, parameter constraints, and data exclusion criteria, it is not possible to verify that the models isolate correlation structure as the sole causal factor.
minor comments (2)
- [Abstract and Results] The abstract and results would benefit from a brief statement of how the 25-round structure and group size were chosen to ensure the persistence metric is not sensitive to arbitrary cutoffs.
- [Methods] Clarify in the methods whether the perturbation distributions were identical in variance across the two noise conditions or whether any scaling was applied to equate perceived noise magnitude.
Simulated Author's Rebuttal
We thank the referee for the careful reading and for highlighting the importance of the dynamic update models to our central claim. We address each major comment below and will revise the manuscript accordingly to improve clarity and verifiability.
read point-by-point responses
-
Referee: [Dynamic update models and experimental conditions] The dynamic update models' attribution that peer influence strength is equivalent across conditions (and thus that correlation alone explains the production-noise effect) is load-bearing for the central claim. The task design notes subjective uncertainty in the thermometer cue and a unitless 50-250 range that conflicts with everyday expectations; production noise (identical perturbations visible to multiple receivers) could systematically alter participants' weighting of social versus private information relative to comprehension noise in ways not captured by the model fits.
Authors: The dynamic update models were fitted separately to each condition, permitting the social influence parameter to take different values. Post-fit comparisons showed no statistically significant differences in the estimated peer influence strengths across the three conditions. Because the thermometer cue, its subjective uncertainty, and the unitless 50-250 scale were identical in every condition, any systematic shift in weighting of social versus private information would appear as a difference in the fitted parameters; none was observed. We will add a supplementary table reporting the condition-specific parameter estimates and their confidence intervals to make this equivalence explicit. revision: yes
-
Referee: [Dynamic update models] The reported group-level permutation test (p=0.016 for rounds tightly clustered around a wrong value) and the mechanism conclusion require explicit confirmation that influence parameters are statistically indistinguishable across conditions after accounting for any condition-specific differences in cue interpretation. Without details on model fitting procedures, parameter constraints, and data exclusion criteria, it is not possible to verify that the models isolate correlation structure as the sole causal factor.
Authors: We agree that the current manuscript lacks sufficient detail on these points. The models were estimated via maximum likelihood with identical parameter bounds (influence weights constrained to [0,1]) and the same convergence criteria in every condition; data exclusion followed the pre-registered rule of discarding trials with response times below 500 ms or above 30 s. The group-level permutation test was applied to the clustering metric after these fits. We will expand the Methods section and add a supplementary note that fully documents the fitting procedure, constraints, exclusion criteria, and the statistical test confirming parameter equivalence across conditions. revision: yes
Circularity Check
No significant circularity; empirical experiment with independent statistical tests.
full rationale
The paper reports results from a controlled online experiment with 600 participants across three conditions (faithful, comprehension noise, production noise), using Fisher's exact test, permutation tests, and dynamic update models to compare outcomes. No equations, parameter fits, or derivations are shown that reduce a 'prediction' to the input data by construction. Claims rest on direct comparisons of experimental conditions and p-values rather than self-definitional relations, self-citation chains, or renamed known results. The dynamic models attribute differences to correlation structure but are presented as interpretive tools, not load-bearing derivations that presuppose the target result. This matches the default expectation for an empirical study with falsifiable tests.
Assumptions & free parameters
assumptions (1)
- standard math Standard assumptions underlying permutation tests and Fisher's exact test apply to the group-level data.
Cite this review
Pith. "Pith review of Private Noise and Public Error in Collective Information Acquisition." pith.science (2026). https://pith.science/paper/L3ILIMBX
@misc{pith2026260530522,
author = {Pith},
title = {Pith review of: Private Noise and Public Error in Collective Information Acquisition},
year = {2026},
howpublished = {\url{https://pith.science/paper/L3ILIMBX}},
note = {Machine review of arXiv:2605.30522}
}
abstract
Collective information acquisition requires groups to combine personal evidence with social information while remaining coupled to the external state. Communication noise can affect this process, but the role of noise remains unclear. In an online experiment, 600 participants worked in four-person human groups estimating a room temperature across 25 rounds while receiving either faithful social information, comprehension noise in which each receiver saw independently perturbed social information, or production noise in which perturbations were stored before display and could be seen by multiple receivers. The thermometer cue was objectively veridical, but its reliability was subjectively uncertain and the unitless 50--250 room-temperature range created a task-induced conflict between displayed evidence and everyday temperature expectations. Production-noise groups spent more rounds tightly clustered around a wrong value than comprehension-noise groups (\(p=0.016\), group-level permutation). Production noise more often created a wrong common signal (\(p=0.025\), Fisher's exact test) and made that signal persist across more rounds (\(p=0.004\), permutation). Dynamic update models showed that production noise was not more harmful because people followed peers more strongly, but because the same peer influence acted on more correlated production-noise perturbations. Exploratory human analyses linked the mechanism to psychological patterns while a GPT-agent experiment clarified a boundary condition: GPT agents registered uncertainty through reduced confidence without reproducing human-scale production-noise vulnerability. Overall, noise did not simply degrade collective information acquisition. Comprehension noise could sometimes improve correction relative to the faithful control, whereas production noise could turn perturbations into common evidence and stabilize consensus on error.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Vox populi
Galton, F. Vox populi. Nature 75, 450–451 (1907)
1907
-
[2]
The Wisdom of Crowds
Surowiecki, J. The Wisdom of Crowds. Doubleday, New York (2004)
2004
-
[3]
& Page, S
Hong, L. & Page, S. E. Groups of diverse problem solvers can outperform groups of high-ability problem solvers. Proceedings of the National Academy of Sciences 101, 16385–16389 (2004)
2004
-
[4]
& Kahneman, D
Tversky, A. & Kahneman, D. Judgment under uncertainty: Heuristics and biases. Science 185, 1124–1131 (1974)
1974
-
[5]
& de Lange, F
Summerfield, C. & de Lange, F. P. Expectation in perceptual decision making: Neural and computational mechanisms. Nature Reviews Neuroscience 15, 745–756 (2014)
2014
-
[6]
Asch, S. E. Effects of group pressure upon the modification and distortion of judgments. In Guetzkow, H. (ed.), Groups, Leadership and Men, 177–190. Carnegie Press, Pittsburgh (1951)
1951
-
[7]
& Smith, P
Bond, R. & Smith, P. B. Culture and conformity: A meta-analysis of studies using Asch’s line judgment task. Psychological Bulletin 119, 111–137 (1996)
1996
-
[8]
& Milyavsky, M
Yaniv, I. & Milyavsky, M. Using advice from multiple sources to revise and improve judgments. Organizational Behavior and Human Decision Processes 103, 104–120 (2007)
2007
Show all 61 references
-
[9]
Bahrami, B. et al. Optimally interacting minds. Science 329, 1081–1085 (2010)
2010
-
[10]
& Helbing, D
Lorenz, J., Rauhut, H., Schweitzer, F. & Helbing, D. How social influence can undermine the wisdom of crowd effect. Proceedings of the National Academy of Sciences 108, 9020–9025 (2011)
2011
-
[11]
E., Analytis, P
Moussa¨ ıd, M., K¨ ammer, J. E., Analytis, P. P. & Neth, H. Social influence and the collective dynamics of opinion formation. PLoS ONE 8, e78433 (2013)
2013
-
[12]
Jayles, B. et al. How social information can improve estimation accuracy in human groups. Proceedings of the National Academy of Sciences 114, 12620–12625 (2017)
2017
-
[13]
& Centola, D
Becker, J., Brackbill, D. & Centola, D. Network dynamics of social influence in the wisdom of crowds. Proceedings of the National Academy of Sciences 114, E5070–E5076 (2017)
2017
-
[14]
& Welch, I
Bikhchandani, S., Hirshleifer, D. & Welch, I. A theory of fads, fashion, custom, and cultural change as informational cascades. Journal of Political Economy 100, 992–1026 (1992)
1992
-
[15]
Banerjee, A. V. A simple model of herd behavior. The Quarterly Journal of Economics 107, 797–817 (1992)
1992
-
[16]
Anderson, L. R. & Holt, C. A. Information cascades in the laboratory. American Economic Review 87, 847–862 (1997)
1997
-
[17]
Do we follow others when we should? A simple test of rational expectations
Weizs¨ acker, G. Do we follow others when we should? A simple test of rational expectations. American Economic Review 100, 2340–2360 (2010)
2010
-
[18]
DeGroot, M. H. Reaching a consensus. Journal of the American Statistical Association 69, 118–121 (1974). 36
1974
-
[19]
& Jackson, M
Golub, B. & Jackson, M. O. Naive learning in social networks and the wisdom of crowds. American Economic Journal: Microeconomics 2, 112–149 (2010)
2010
-
[20]
Flache, A. et al. Models of social influence: Towards the next frontiers. Journal of Artificial Societies and Social Simulation 20, 2 (2017)
2017
-
[21]
D., Krause, J., Franks, N
Couzin, I. D., Krause, J., Franks, N. R. & Levin, S. A. Effective leadership and decision-making in animal groups on the move. Nature 433, 513–516 (2005)
2005
-
[22]
Dall, S. R. X., Giraldeau, L. A., Olsson, O., McNamara, J. M. & Stephens, D. W. Information and its use by animals in evolutionary ecology. Trends in Ecology & Evolution 20, 187–193 (2005)
2005
-
[23]
& Roper, T
Conradt, L. & Roper, T. J. Consensus decision making in animals. Trends in Ecology & Evolution 20, 449–456 (2005)
2005
-
[24]
Phase diagram and optimal information use in a collective sensing system
Salahshour, M. Phase diagram and optimal information use in a collective sensing system. Physical Review Letters 123, 068101 (2019)
2019
-
[25]
& Huber, R
Sch¨ obel, M., Rieskamp, J. & Huber, R. Social influences in sequential decision making. PLoS ONE 11, e0146536 (2016)
2016
-
[26]
Li, L., Li, K. K. & Li, J. Private but not social information validity modulates social conformity bias. Human Brain Mapping 40, 2464–2474 (2019)
2019
-
[27]
& Aral, S
Vosoughi, S., Roy, D. & Aral, S. The spread of true and false news online. Science 359, 1146– 1151 (2018)
2018
-
[28]
Lazer, D. M. J. et al. The science of fake news. Science 359, 1094–1096 (2018)
2018
-
[29]
Bak-Coleman, J. B. et al. Stewardship of global collective behavior. Proceedings of the National Academy of Sciences 118, e2025764118 (2021)
2021
-
[30]
Shannon, C. E. A mathematical theory of communication. Bell System Technical Journal 27, 379–423 and 623–656 (1948)
1948
-
[31]
& Roudi, Y
Salahshour, M., Rouhani, S. & Roudi, Y. Phase transitions and asymmetry between signal comprehension and production in biological communication. Scientific Reports 9, 3428 (2019)
2019
-
[32]
& Rouhani, S
Salahshour, M. & Rouhani, S. Collective movement and collective information acquisition with signaling. Frontiers in Physics 9, 668283 (2021)
2021
-
[33]
& Reynolds, L
Shanahan, M., McDonell, K. & Reynolds, L. Role play with large language models. Nature 623, 493–498 (2023)
2023
-
[34]
Wang, L. et al. A survey on large language model based autonomous agents. Frontiers of Computer Science 18, 186345 (2024)
2024
-
[35]
Argyle, L. P. et al. Out of one, many: Using language models to simulate human samples. Political Analysis 31, 337–351 (2023)
2023
-
[36]
V., Arriaga, R
Aher, G. V., Arriaga, R. I. & Kalai, A. T. Using large language models to simulate multiple humans and replicate human subject studies. Proceedings of the 40th International Conference on Machine Learning, 337–371 (2023). 37
2023
-
[37]
Park, J. S. et al. Generative agents: Interactive simulacra of human behavior. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 1–22 (2023)
2023
-
[38]
and Mavor-Parker, A., 2023, July
Griffin, L., Kleinberg, B., Mozes, M., Mai, K., Vau, M.D.M., Caldwell, M. and Mavor-Parker, A., 2023, July. Large language models respond to influence like humans. In Proceedings of the First Workshop on Social Influence in Conversations (SICon 2023) (pp. 15-24)
2023
-
[39]
and Ungar, L., 2025
Cho, Y.M., Guntuku, S.C. and Ungar, L., 2025. Herd behavior: Investigating peer influence in llm-based multi-agent systems. arXiv:2505.21588 (2025)
2025
-
[40]
and Goldstein, D.G., 2025
Alsobay, M., Rothschild, D.M., Hofman, J.M. and Goldstein, D.G., 2025. Bringing everyone to the table: An experimental study of llm-facilitated group decision making. arXiv:2508.08242 (2025)
2025
-
[41]
& Stefl, C
Mehrabian, A. & Stefl, C. A. Basic temperament components of loneliness, shyness, and con- formity. Social Behavior and Personality: An International Journal 23, 253–264 (1995)
1995
-
[42]
and Gelfand, M.J., 1995
Singelis, T.M., Triandis, H.C., Bhawuk, D.P. and Gelfand, M.J., 1995. Horizontal and verti- cal dimensions of individualism and collectivism: A theoretical and measurement refinement. Cross-cultural research, 29(3), pp.240-275
1995
-
[43]
and Gelfand, M.J., 1998
Triandis, H.C. and Gelfand, M.J., 1998. Converging measurement of horizontal and vertical individualism and collectivism. Journal of personality and social psychology, 74(1), p.118
1998
-
[44]
& Hastie, R
Gigone, D. & Hastie, R. The common knowledge effect: Information sharing and group judg- ment. Journal of Personality and Social Psychology 65, 959–974 (1993)
1993
-
[45]
When are two heads better than one and why? Science 336, 360–362 (2012)
Koriat, A. When are two heads better than one and why? Science 336, 360–362 (2012)
2012
-
[46]
Gender differences in the self-assessment of accuracy on cognitive tasks
Pallier, G. Gender differences in the self-assessment of accuracy on cognitive tasks. Sex Roles 48, 265–276 (2003)
2003
-
[47]
Eagly, A. H. & Carli, L. L. Sex of researchers and sex-typed communications as determinants of sex differences in influenceability: A meta-analysis of social influence studies. Psychological Bulletin 90, 1–20 (1981)
1981
-
[48]
& Nilsson, L.-G
Hansson, P., R¨ onnlund, M., Juslin, P. & Nilsson, L.-G. Adult age differences in the real- ism of confidence judgments: Overconfidence, format dependence, and cognitive predictors. Psychology and Aging 23, 531–544 (2008)
2008
-
[49]
Loftus, E. F. & Palmer, J. C. Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behavior 13, 585–589 (1974)
1974
-
[50]
Lewandowsky, S., Ecker, U. K. H., Seifert, C. M., Schwarz, N. & Cook, J. Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest 13, 106–131 (2012). 38 Table 1: Theory predictions and empirical tests. The St...
2012
-
[51]
Direct Eq
Initial estimates should be compressed toward expectation when evidence conflicts with prior belief. Direct Eq. 1: ifλ i <1 andµ i < T 1, high ev- idence is pulled downward toward ex- pectation. Round-1 slopes were below 1 in all regimes: con- trol 0.807, comprehension 0.762, ...
-
[52]
Direct Positive personal-evidence weight, ωi(t)>0, in Eq
Estimates should show evidence pull,T t −x i(t), when personal evidence re- mains usable. Direct Positive personal-evidence weight, ωi(t)>0, in Eq. 2. Evidence-pull coefficients,T t −x i(t), estimating the personal-evidence weight were small after previous estimates and peer p...
-
[53]
Direct Positive social-information weight, σi(t)>0, on peer pull,S i(t)−x i(t), in Eq
Estimates should move toward social information, Si(t). Direct Positive social-information weight, σi(t)>0, on peer pull,S i(t)−x i(t), in Eq. 2. The social-information weight,σ i(t), estimated as the coefficient on peer pull,S i(t)−x i(t), was positive in all regimes: control...
-
[54]
Moderator Eq
Noisy channels can increase the social- information weight,σ i(t), without a production- specific increase. Moderator Eq. 3:σ N allows noisy channels to change the social-information weight, σi(t), whileσ P tests production be- yond comprehension. In a condition-only interacti...
-
[55]
Channel Eqs
Production noise should create more correlated channel-noise components. Channel Eqs. 12–13; comprehension-noise perturbations average out, whereas production-noise perturbations are reused across receivers. Observed group-mean channel variance was 0.841 in comprehension noise...
-
[56]
Channel Production-noise perturbations sur- vive averaging and enter the group- level social signal
Group-level channel- noise components should move the group most clearly under production noise. Channel Production-noise perturbations sur- vive averaging and enter the group- level social signal. Group-level channel-noise components predicted next group-mean movement in prod...
-
[57]
Moderator In Eq
Confidence should re- duce the social-information weight,σ i(t). Moderator In Eq. 3,σ C <0 means confidence reduces the social-information weight. Higher previous confidence reduced the social- information weight,σ i(t), in the pooled full model (bσC =−0.130,p= 3.5×10 −7) and ...
-
[58]
Moderator In Eq
Peer dispersion,D i(t), should reduce the social- information weight,σ i(t), if disagreement marks low so- cial reliability. Moderator In Eq. 3,σ D <0 means peer disper- sion reducesσ i(t). The peer-pull by dispersion interaction, [S i(t)− xi(t)]×D i(t), estimatedbσD =−0.063 i...
-
[59]
Moderator In Eq
Shared and personal anchors may differ in how they change the social- information weight,σ i(t). Moderator In Eq. 3,σ G andσ a test whether the shared group anchor or the personal anchor changes the social-information weight. After both anchor terms were entered, shared group ...
-
[60]
Channel Eqs
Comprehension noise can help when anchors are correctable. Channel Eqs. 8–9; faithful transmission car- ries current group bias, whereas comprehension-noise perturbations weaken the coherence of biased peer pull,S i(t)−x i(t), while evidence pull,T t −x i(t), remains. At a rep...
-
[61]
Confidence state Eq
Confidence should fall when reliability cues worsen. Confidence state Eq. 4: noisy channels, larger error, peer dispersion,D i(t), and stronger personal anchors can reduce confi- dence, while the shared group anchor is estimated as a context term. In the pooled confidence equa...
Reviewed June 28, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.