{"id":"02027b88-5fd6-477d-9560-09dee4e7bb9f","arxiv_id":"2501.05788","paper_version":2,"verdict":"UNVERDICTED","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Scientific norms, including replicability and objectivity, are historically and socially shaped, so protecting physics from a replication crisis requires community reflection and interdisciplinary collaboration rather than technical checklists alone.","lead":"Two early-career physicists argue that the replication crisis gripping other sciences could hit physics too, and that the problem is social and cultural, not just technical. They urge physicists to reflect on how funding, metrics, and publication norms shape research, and to work with historians and philosophers on safeguards.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The essay's central claim rests on the unmeasured premise that physics is vulnerable to a replication crisis; the cited evidence does not establish systematic unreliability, so the urgency of the cultural prescription is not secured.","rationale":"The reader's weakest_assumption is exactly the load-bearing concern: the paper's normative conclusion depends on the empirical claim that physics is vulnerable to a replication crisis, yet that claim is supported only by high-profile retractions and general references, not by measured replication rates. The historical and philosophical portions of the essay (Ziman, Daston and Galison, Longino, Oreskes) are well-supported and internally coherent, and the proposal for interdisciplinary reflection is reasonable. However, the urgency attached to that proposal is calibrated to a crisis whose existence in physics is not demonstrated. I agree with UNVERDICTED because the manuscript is a perspective/argument piece with no new empirical data or testable result; my concern does not require changing the verdict, but it does identify the specific point that would need evidence before the paper could carry scientific weight as a diagnosis rather than an opinion piece. The proposed audit is directly responsive to the quoted assertion that relevance is not limited to a few problem areas and would settle whether the premise lands.","tokens_in":3209,"tokens_out":4034,"duration_ms":41406,"concrete_test":"Conduct a systematic reproduction audit modeled on the Open Science Collaboration protocol but adapted to physics: independently re-analyze the data and code from 100 papers randomly sampled across condensed matter, high-energy, and applied physics (using the subfield coverage of the cited arXiv report 2501.18631) and count how many published findings are reproduced. Compare this rate with the roughly 36% success rate reported in psychology. If the physics reproduction rate is above 90%, the premise of a looming replication crisis in physics fails; if it is comparable to psychology, the essay's urgency is empirically supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The conclusion that 'addressing the replication crisis requires more than enforcing stricter methodologies' and requires cultivating a culture of replication and critique is only compelling if physics actually faces, or credibly threatens, a systematic replication problem. The essay supports this premise with a MIT Technology Review article on a wave of retractions (ref 1), a commentary (ref 3), and an arXiv reproducibility report (ref 4), but it reports no measured reproduction or replication rates across physics subfields. High-profile retractions in high-Tc superconductivity and Majorana fermions are anecdotal; they document fraud or errors in specific high-stakes cases, not prevalence. The authors acknowledge that many physicists regard their fields as self-correcting, but rebut this only rhetorically ('But think again') and with general questions about whether a careful re-analysis would match published claims. The phrase 'relevance is not limited to a few problem areas' is asserted, not quantified. If the reproducibility rate in physics is actually high, then the warning 'if our research becomes plagued by replication crises' remains a hypothetical, and the call for a community-wide cultural overhaul loses its evidentiary basis; it becomes a precautionary preference rather than a diagnosis.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This perspective essay argues that the replication crisis is not merely a technical problem in physics but a symptom of culturally and historically contingent norms. The authors draw on Anderson's concept of emergence, Ziman's sociology of science, Daston and Galison's history of objectivity, Longino's social epistemology, and Oreskes's work to support the view that replicability and objectivity evolve with community practices. They conclude that addressing the replication crisis requires cultivating a culture that values replication, critique, and transparency as much as discovery, and they recommend multidisciplinary collaboration between physicists and humanities/social-science scholars.","tokens_in":3415,"tokens_out":5921,"duration_ms":57105,"significance":"The essay's main contribution is a clear, accessible synthesis of historical and philosophical scholarship applied to the replication debate in physics. Its strength is the explicit connection between Daston and Galison's historical account of objectivity and Longino's social epistemology, which gives physicists a vocabulary for understanding scientific norms as changeable rather than fixed. The paper does not present new empirical data or machine-checked proofs, but it makes a falsifiable and important claim: if physics reproducibility is already high, the urgency of a cultural overhaul diminishes. The proposal for multidisciplinary collaboration is concrete enough to seed productive debate, and the essay productively bridges physics with philosophy and history of science.","major_comments":[{"comment":"The load-bearing premise that physics is at risk of a replication crisis is asserted rather than demonstrated. The sentence '...their relevance is not limited to a few 'problem' areas' is supported only by refs 3 and 4, and no replication or reproducibility rates for physics subfields are reported. The high-profile retractions in high-Tc superconductivity and Majorana physics are anecdotal, documenting fraud or error in specific cases rather than systemic prevalence. Because the paper's conclusion—that addressing the replication crisis requires cultural change—depends on this premise, the essay should either present quantitative evidence (e.g., from the arXiv reproducibility report, ref 4) or explicitly reframe the argument as precautionary rather than diagnostic.","section":"Introduction (first paragraph)"},{"comment":"The analogy between Anderson's physical emergence and Ziman's account of scientific knowledge is asserted rather than argued. The phrase 'In a similar manner, argued Ziman' does not specify in what sense social norms are emergent or how spontaneous symmetry breaking maps onto social epistemology. Since the paper's central claim is that replicability and objectivity are 'emergent and evolving,' this analogy is load-bearing. The authors should add a substantive explanation of the notion of emergence at work and its limits, rather than relying on rhetorical similarity.","section":"Scientific knowledge as an emergent property"},{"comment":"The dismissal of the self-correction view with 'But think again' is a rhetorical move rather than an argument. If the essay intends to challenge the claim that physics is self-correcting, it should engage with the relevant literature, including the von Hippel reference (ref 2), which documents a replication crisis in early 20th-century chemistry and the conditions under which self-correction occurred. Without such engagement, the paper does not justify overriding the common physicist viewpoint it explicitly acknowledges, and the urgency of its prescription rests on an unexamined contrast with an underexplored opposing position.","section":"Introduction (second paragraph)"}],"minor_comments":[{"comment":"The first line of the full text contains 'Evolu3on' and 'Scien3fic' in place of 'Evolution' and 'Scientific'; this appears to be a typographical artifact and should be corrected.","section":"Title/first line"},{"comment":"The sentence 'A step forward in this discussion requires accepting that that the knowledge we produce...' contains a doubled 'that' and should be edited to remove the repetition.","section":"Where do we go from here?"},{"comment":"Reference 9 (Oreskes 2004) is a paper on scientific consensus about climate change; it does not directly support the claim that scientific knowledge is 'inherently social.' A more apt citation to Oreskes's work on trust in science or the social nature of scientific knowledge would strengthen the argument.","section":"References"},{"comment":"The essay uses 'replicability' and 'reproducibility' interchangeably without defining them; standard definitions (e.g., new experiment versus reanalysis of the same data) would sharpen the argument and avoid confusion in the physics audience.","section":"Definitions"},{"comment":"The author list of reference 4 appears incomplete or incorrectly formatted (e.g., 'Quader, B. S. K. F.' with an unclear sequence); please verify the full author list and the arXiv identifier against the original report.","section":"Reference 4"}],"recommendation":"major_revision","confidential_remarks":"This essay is best read as a call to reflection rather than an empirical study. For a journal focused on physics and society, the lack of quantitative evidence might be acceptable if the authors explicitly frame the argument as precautionary. However, the current framing overstates the diagnosis by implying that physics is already 'plagued' by a replication crisis. The editor may wish to ask the authors to soften this claim or to support it with the data from the arXiv reproducibility report cited as ref 4, so that the prescriptive conclusion is better matched to the evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is an essay, not a research paper. It rehearses established ideas from Anderson, Ziman, Daston and Galison, Longino, and Oreskes to argue that objectivity and replicability are historically evolving, community-level norms, and that physics should respond to concerns about replication through cultural change rather than purely technical fixes. As a piece of scholarly synthesis, it is clear, fair to its sources, and genuinely aimed at a physics audience that may not have encountered this literature. That is its real value: it translates philosophy of science into a language working physicists can actually use.\n\nThe soft spot is exactly the one the stress-test flags. The load-bearing premise that physics is genuinely vulnerable to a replication crisis is not demonstrated. The essay cites a wave of retractions and a recent condensed-matter reproducibility report, but it gives no measured reproduction rates across physics subfields. The phrase \"relevance is not limited to a few problem areas\" is asserted, not quantified. High-profile fraud cases in Majorana or high-Tc are anecdotes, not prevalence. The rhetorical move from \"physicists say self-correcting\" to \"but think again\" is an invitation to worry, not an argument. If reproducibility in physics is actually high, the urgency weakens to a precautionary preference.\n\nThat said, I don't think the essay fails on its own terms. It does not claim to be a systematic study; it is a reflection piece. The problem is that the philosophical framing does more work than the meager empirical grounding can hold. The proposal for multidisciplinary collaboration is also not new, since it appears in the cited literature. Still, the authors are honest about the limits, and they don't overstate what their sources say.\n\nFor a philosopher of science or a physics faculty seminar, this is a useful conversation starter. It has no new result, so it should not be treated as a research preprint. A serious editor at a venue that publishes perspectives could send it to a referee to check the historical interpretations and the balance of claims; it would benefit from a supplementary note on what evidence would count as a physics replication crisis. I'd read it myself, but I wouldn't cite it as a source of facts.","headline":"A clear, well-sourced philosophical synthesis on scientific norms, but its central premise that physics needs a cultural fix for a replication crisis is asserted rather than measured; treat it as a useful commentary, not a research result.","tokens_in":3889,"tokens_out":2088,"would_cite":false,"duration_ms":21909,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This essay argues that replicability and objectivity are evolving social norms, so addressing the replication crisis in physics requires cultural change, not just stricter methodology.","keywords":["replication crisis","replicability","scientific norms","objectivity","emergence","physics culture","research integrity","science and society"],"falsifier":"A large-scale, pre-registered audit that attempts to independently reproduce a random sample of recently published physics results across subfields—using the original raw data and analysis code—and finds a high success rate would undercut the claim that physics faces an urgent replication crisis.","tokens_in":79,"feed_emoji":"🔬","tokens_out":7741,"duration_ms":133538,"temperature":0.7,"pith_summary":"This paper argues that the replication crisis in physics is not primarily a technical failure but a cultural one: what counts as replicable and objective science is an evolving social norm, not a fixed standard. The authors point to high-profile retractions and fraud cases, and to pressure from funding bodies, universities, and publication metrics, as forces that shape the norms of physics practice. They contend that standard practices—small bespoke samples, embargoed data, and unavoidable expert judgement—can undermine replicability without being misconduct. Because these norms are created and maintained by scientists collectively, they can be deliberately changed. The paper calls for reflection on these social forces, for making replication and critique valued as much as discovery, and for collaborations between physicists and historians, philosophers, and social scientists to seed that change.","feed_headline":"Replication crisis in physics is cultural, not just technical","feed_subtitle":"Replicability and objectivity evolve with community norms; fixing science means changing culture, not just checklists.","key_machinery":"The conceptual machinery is the application of emergence to scientific knowledge itself: just as the collective behavior of many particles gives rise to properties not predictable from the laws governing individual particles, the collective practices of a scientific community give rise to what counts as valid, reproducible knowledge. The paper combines this with a historical account of objectivity as shifting through distinct ideals—truth-to-nature, mechanical objectivity, and trained judgement—and with a social model of objectivity that locates it in community critical interaction and diversity of perspectives. These concepts do the work of moving the replication crisis from a technical problem of methodology to a cultural problem of norms and values.","core_discovery":"The paper's central claim is that the very notions of replicability and objectivity are emergent and historically changing properties, produced by the collective actions and interactions of scientific communities, rather than timeless criteria that individual scientists either satisfy or violate. It follows that the replication crisis is a sociological and cultural problem: current incentives have created norms that undervalue replication, critique, and transparency. Because scientists themselves create and reproduce these norms through their everyday choices, the norms can be reshaped. The paper does not offer a new technical protocol; instead it argues that a durable response requires redefining what replication and self-correction mean under current pressures, rewarding replication work, reimagining the scientific paper as a living document, and building multidisciplinary spaces for reflection.","pith_inferences":["Editor's inference: The cultural diagnosis yields a measurable prediction—communities, journals, or subfields that explicitly reward replication and critical exchange should show higher independent-success rates, a claim that could be tested with a large replication audit.","Editor's inference: If the argument is right, format changes such as living papers or open data will fail unless they are coupled to credit and incentive reform; otherwise they will be adopted superficially without changing practice.","Editor's inference: The historical account of objectivity suggests that the field may need context-specific replication standards rather than one universal criterion, since exact duplication is often impossible in complex experimental settings.","Editor's inference: The paper's call for collaboration with the humanities is itself a testable proposal: structured exchanges between physicists and historians or philosophers could be evaluated by whether they generate new norms and practices within physics groups."],"forward_implications":["Because replicability is an evolving norm, technical checklists and stricter statistics are necessary but insufficient; without cultural change they will not restore trust in physics.","If norms are shaped by incentives, then changes in funding priorities, hiring criteria, and publication practices are direct levers for improving replicability.","If objectivity depends on community critical interaction, then increasing diversity of viewpoints and protecting dissent within research groups and review should improve reliability.","If trained judgement is unavoidable in data selection and analysis, transparent communication of those judgement calls becomes essential to prevent selective reporting.","If scientific papers are treated as closed, static units of knowledge, then the rise of digital tools will not by itself produce more transparent or revisable science; norms must change as well."],"supporting_citations":[{"why":"Supplies the motivating evidence of recent high-profile retractions and fraud cases in physics.","marker":"1"},{"why":"Provides a recent report on reproducibility in condensed matter physics used to show the issue is not limited to a few problem areas.","marker":"4"},{"why":"Supplies the original emergence concept from condensed matter physics that the paper extends to scientific knowledge.","marker":"5"},{"why":"Extends the emergence idea to scientific norms and practices, grounding the claim that norms are negotiated and changing.","marker":"6"},{"why":"Documents historical shifts in the ideal of objectivity, supporting the claim that scientific norms evolve over time.","marker":"7"},{"why":"Provides a social model of objectivity based on community critical interaction and diversity of perspectives.","marker":"8"},{"why":"Supports the view that scientific knowledge is provisional and shaped by debate, critique, and consensus-building.","marker":"9"},{"why":"Serves as a model of physicist-humanities collaboration cited to support the paper's proposed multidisciplinary approach.","marker":"10"}],"fun_headline_variants":["Replication crisis isn't technical; it's cultural","To fix replication, fix the norms that created it","Physics replication crisis is a culture problem","Scientific norms make or break replication"],"cache_read_input_tokens":6144,"weakest_assumption_plain":"The load-bearing premise is that physics actually has a replication problem serious enough to warrant a cultural overhaul, supported in the paper by high-profile retractions and a recent reproducibility report rather than by measured replication rates across physics subfields.","fun_headline_variants_meta":{"raw":{"variants":["Replication crisis isn't technical; it's cultural","To fix replication, fix the norms that created it","Physics replication crisis is a culture problem","Scientific norms make or break replication"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000994,"raw_usage":{"total_tokens":4122,"prompt_tokens":769,"completion_tokens":3353,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":385,"completion_tokens_details":{"reasoning_tokens":3297}},"tokens_in":385,"tokens_out":3353,"duration_ms":22387,"temperature":1.0,"reasoning_tokens":3297,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:06:30.623225+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A large-scale, pre-registered audit that attempts to independently reproduce a random sample of recently published physics results across subfields—using the original raw data and analysis code—and finds a high success rate would undercut the claim that physics faces an urgent replication crisis.","supporting_citations":[{"cited_title":"A wave of retractions is shaking physics","cited_arxiv_id":null,"evidence_quote":"Supplies the motivating evidence of recent high-profile retractions and fraud cases in physics."},{"cited_title":"D., Devaty, R","cited_arxiv_id":null,"evidence_quote":"Provides a recent report on reproducibility in condensed matter physics used to show the issue is not limited to a few problem areas."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the original emergence concept from condensed matter physics that the paper extends to scientific knowledge."},{"cited_title":"Is science losing its objectivity? Nature 382, 751–754 (1996)","cited_arxiv_id":null,"evidence_quote":"Extends the emergence idea to scientific norms and practices, grounding the claim that norms are negotiated and changing."},{"cited_title":"& Galison, P","cited_arxiv_id":null,"evidence_quote":"Documents historical shifts in the ideal of objectivity, supporting the claim that scientific norms evolve over time."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides a social model of objectivity based on community critical interaction and diversity of perspectives."},{"cited_title":"The Scientific Consensus on Climate Change","cited_arxiv_id":null,"evidence_quote":"Supports the view that scientific knowledge is provisional and shaped by debate, critique, and consensus-building."},{"cited_title":"de, Thresher, A","cited_arxiv_id":null,"evidence_quote":"Serves as a model of physicist-humanities collaboration cited to support the paper's proposed multidisciplinary approach."}],"review_version":1}