Multiagent Systems · arXiv:2611.16315 v1 · submitted 12 Nov 2026 · 17 references
Shared Item-Level Error Limits the Aggregation of Small Language Models in Fermi Estimation
The paper says
“Multi-agent systems of language models are often motivated by an averaging argument: if the agents' errors are unbiased and independent, the error of an aggregate of N answers falls as 1/sqrt(N).”
more of the abstract
Fermi estimation, whose continuous answers are aggregated on the logarithmic scale, permits a direct test. We decompose the log10 error of each answer into the mean error of its model on the item, common to all its draws on the item, and a residual, and estimate both from 2.51M answers given before any communication by 15 open-weight models (3B-72B) on realFP, with replication under a single prompt pair on realFP, Science Olympiad Fermi questions and a guesstimation set. Given the item, teammates' errors are uncorrelated and teams behave as independent draws, but the item-level component accounts for a median 0.906 of the error variance over items. Teams of thirty therefore reduce a single agent's error by a median 1.7% (at most 5.4%), whereas draws that were also unbiased on each item would reduce it by 74-79%. All models compress their estimates toward about 10^6; this trend explains a median 17% of the item-level component, and a cross-validated linear recalibration on labelled items removes the overall bias but not that component. Three rounds of debate reduce within-team dispersion by 93% and leave the item-level component unchanged; revision without peers attains the same small gain. Mixing five models of similar accuracy reduces error by 6.6% relative to their own homogeneous teams, but no mixture improves on the most accurate model. Higher sampling temperature changes team error by about 1% at most, and contrastive activation steering not detectably; per-agent scaffold prompts (not significant after multiplicity correction) and skipping one layer in a single-model screen improve it. The results are conditional on the models, benchmarks, prompts and settings tested.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 13 Nov 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. The paper convincingly shows that on continuous estimation tasks, the 1/sqrt(N) averaging argument fails because item-level error dominates, but the headline rho=0.906 may be slightly inflated by run/prompt variation not controlled in the ANOVA. This line is wrong
This review says the paper's mathematics is wrong. Pith did not check that against the paper's source.
4 major objections · 7 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 17 references say.
The paper says, at §3.1, Eq. (1)–(2); §4.1; Table D.2
The review did not quote this passage.
Pith says: Objection 1 of 4 · major
The one-way ANOVA estimate of ρ pools all round-1 draws of a model, treating every draw as sharing the same item-level component c_j. However, the authors report that a run component (deviation of a specific run from the model's other runs) accounts for up to 11% of error variance in 6 models (Table D.2), and that the main sweep mixes …
more of objection 1
… two prompt pairs unevenly across models, with the revised pair lowering MAE@5 by −0.20 OOM in models that ran both (Table B.2). Because runs and prompt pairs are not crossed with items in the ANOVA, the between-item variance A absorbs run-level and prompt-level variation that is common to all draws of a run but not truly a property of the item. The second corpus uses a single prompt pair and reproduces ρ=0.915 (Table 2), which is close but not identical; the run component remains. If the run and prompt variation were removed, ρ would decrease, and the available reduction from aggregation would increase. The paper acknowledges the run component but does not quantify how much of the between-item variance A is actually due to runs/prompts rather than items, nor does it report a decomposition that controls for run. This is load-bearing because the headline claim that aggregation is nearly useless depends on ρ being near 0.9; a ρ of, say, 0.8 would imply Neff(30)=1.27 instead of 1.10 and a larger potential gain. I request a sensitivity analysis: re-estimate ρ after partialling out run-level means (or using a two-way ANOVA by item and run) for the subset of models with multiple runs, and report the range of ρ under that decomposition.
The paper says, at §3.2; §5; Table 2
The review did not quote this passage.
Pith says: Objection 2 of 4 · major
The 'unbiased given the item' reference (draws with independent random signs) is used to compute the share of the available reduction that real teams realise (median 2.2%). This reference removes µ + c_j entirely, which is the strongest possible counterfactual.
more of objection 2
The paper also reports a 'pooled' reference (draws across items with random signs) in Appendix D, which gives a smaller available reduction (51–63%) and a share realised of at most 10.7%. The choice of reference affects the interpretation of 'how much aggregation could in principle achieve'. The paper should state more prominently that the 74–79% figure is an upper bound that assumes not only independence but also zero item-level error, and that the more realistic reference (across items) still leaves the real teams far short. This is not a flaw but a clarity issue; the current presentation in the abstract and Section 5 emphasises the 74–79% figure without immediately noting that it is an extreme bound.
The paper says, at §7; Table 4; Appendix G
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.81):
“The screen stored only per-item team errors computed with the earlier parser and the median rule, and covers one model and 150 items, so it identifies a candidate rather than an established effect.”
Pith says: Objection 3 of 4 · major
The per-agent scaffold arm improves team MAE@5 by −0.104 OOM (p=0.022 unadjusted, p=0.089 after Holm correction). The paper states that this does not survive correction for multiplicity and is not significant under the median rule. However, the conclusion in the abstract and Section 7 that 'per-agent scaffold prompts (not significant after multiplicity correction)' is accurate but the presentation in the main text (Table 4) shows the unadjusted p-value and the Holm-adjusted p-value, which is appropriate.
more of objection 3
No change required, but I note that the evidence for scaffolds is weak and the paper's cautious language is correct.
The paper says, at §6; Table 3; §8
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.65):
“Heterogeneous teams and misparsed inputs.Rounds 2 and 3 were generated by a harness whose parser misread some answers written as a mantissa times a power of ten, so in 22.5% of the homogeneous and 32.6% of the heterogeneous teams an agent was shown a peer value that differed from the one writ- ten.”
Pith says: Objection 4 of 4 · major
The defective parser in rounds 2 and 3 (22.5% of homogeneous teams, 32.6% of heterogeneous teams) is a serious data-quality issue for the debate results. The authors acknowledge it and state that excluding those teams leaves the homogeneous results unchanged but removes the significance of the SciOly difference and the degradation of the heterogeneous pool.
more of objection 4
They also state 'We therefore draw no conclusion about debate in heterogeneous teams.' This is appropriate. However, the paper still reports debate results for homogeneous teams (Table 2, Table F.1) and uses them to support the claim that debate does not reduce item-level error. Given that the parser error affects what agents were shown, not just the scoring, and that the error rate is substantial, I would like the authors to confirm that the homogeneous-team debate results are robust to excluding teams with misparsed inputs (they state this in Section 6 but only for the comparison with self-revision; please also confirm for the main debate statistics in Table 2 and Table F.1).
7 minor objections
The paper says, at Abstract
Pith says: minor objection 1 of 7
The abstract states 'Teams of thirty therefore reduce a single agent's error by a median 1.7% (at most 5.4%), whereas draws that were also unbiased on each item would reduce it by 74–79%.' The phrase 'draws that were also unbiased on each item' could be misinterpreted as a realistic alternative; consider adding 'under the strong counterfactual that the item-level error is zero' or similar.
The paper says, at §3.1, Eq. (2)
Pith says: minor objection 2 of 7
The definition of Neff(N) = N / (1 + (N-1)ρ) is the Kish design effect. It would be helpful to note that this formula assumes equal cluster sizes and homoscedasticity, which approximately hold here.
show 5 more
The paper says, at §4.1, Table D.2
Pith says: minor objection 3 of 7
The run component is reported as a percentage of error variance for each model. It would be useful to also report the run component as a share of the between-item variance A, to directly assess how much of A might be inflated by run-level variation.
The paper says, at §5, Figure 2
Pith says: minor objection 4 of 7
The right panel of Figure 2 shows the median over models of three references. The kink at N=2 for the 'unbiased given the item' reference is explained in the caption as arising because identical draws with opposite signs cancel exactly. This is correct but the explanation could be expanded slightly for clarity.
The paper says, at §7, Table 4
Pith says: minor objection 5 of 7
The column '∆A(%)' for the calculator-offload arms is marked as not interpretable because winsorization lowers A when an arm inflates the within-item spread. Consider adding a footnote or note in the table caption to this effect, as it is currently only in the text.
The paper says, at Appendix G, Table G.3
Pith says: minor objection 6 of 7
The activation screen uses unwinsorized A and the median rule. The paper notes this. It would be helpful to state explicitly that the screen's results are not directly comparable to the main sweep's winsorized estimates.
The paper says, at Throughout
Pith says: minor objection 7 of 7
The paper uses 'OOM' as an abbreviation for 'orders of magnitude'. This is standard but should be defined at first use (it is defined in the introduction).
What would settle it
Re-estimate ρ using a mixed-effects model that includes random intercepts for item, run, and prompt pair (where known), on the round-1 draws of the main sweep. If the item-level variance component shrinks by more than 0.05 relative to the one-way ANOVA estimate, then the headline ρ is inflated by run/prompt variation and the central claim about the limit of aggregation is weaker than reported.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 13 Nov 2026
- Reader holds up, with conditions
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee major revision: 4 major objections, 7 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Hey,
This paper is worth reading. It does something clean: it decomposes the error of LLM estimates on Fermi problems into an item-level component that every draw of a model shares and a residual that averages out. The headline result—that item-level error accounts for a median 90.6% of variance, so teams of thirty reduce error by only 1.7%—is internally consistent and supported by two corpora, a second estimator via team means, and robustness checks. The paper is unusually honest about its limitations: the prompt-pair mixing, the defective parser in debate rounds, the unregistered analysis plan, the small Guesstimate set. That candor earns trust.
What is new: the quantitative decomposition itself. Prior work reported correlated errors on discrete benchmarks, but here the continuous scale lets them measure the common component in the units of the answer and show that it is the item, not dependence between agents, that kills aggregation. The finding that debate collapses dispersion without changing the item-level error, and that self-revision matches peer revision, is a clean replication of earlier results in a new setting.
The soft spots are real but not fatal. The one-way ANOVA by item pools all round-1 draws of a model, treating every draw as sharing the same item-level component cj. But the authors report that a run component accounts for up to 11% of variance in 6 models, and the main sweep mixes two prompt pairs unevenly across models. Because runs and prompt pairs are not crossed with items in the ANOVA, the between-item variance A absorbs run-level and prompt-level variation that is common to all draws of a run but not truly a property of the item. The second corpus uses a single prompt pair and reproduces rho=0.915, which is close, but the run component remains. If that variation were removed, rho would decrease and the available reduction from aggregation would increase. The paper acknowledges the run component but does not quantify how much of A is actually due to runs/prompts rather than items, nor does it report a decomposition that controls for run. This is load-bearing because the headline claim depends on rho being near 0.9; a rho of 0.8 would imply Neff(30)=1.27 instead of 1.10 and a larger potential gain.
The other soft spot is reproducibility: the paper claims a complete rerun reproduces all numbers exactly, but no code or data is released. Given the size of the dataset and the complexity of the pipeline, that claim cannot be verified. The authors should release the code and scored answers.
Who this is for: anyone working on multi-agent LLM systems, forecast combination, or wisdom of crowds. It deserves a serious referee. I would accept peer review and recommend the same, with the expectation that the authors address the run/prompt confound and release code and data.
Best, [Your name]
The strongest objection
The central claim that item-level error accounts for 90.6% of variance (ρ=0.906) rests on a one-way ANOVA by item that pools all round-1 draws of a model, treating every draw as sharing the same item-level component c_j. However, the authors report that a run component (deviation of a specific run from the model's other runs) accounts for up to 11% of error variance in 6 models (Table D.2), and that the main sweep mixes two prompt pairs unevenly across models, with the revised pair lowering MAE@5 by −0.20 OOM in models that ran both (Table B.2). Because runs and prompt pairs are not crossed with items in the ANOVA, the between-item variance A absorbs run-level and prompt-level variation that is common to all draws of a run but not truly a property of the item. The second corpus uses a single prompt pair and reproduces ρ=0.915 (Table 2), which is close but not identical; the run component remains. If the run and prompt variation were removed, ρ would decrease, and the available reduction from aggregation would increase. The paper acknowledges the run component but does not quantify how much of the between-item variance A is actually due to runs/prompts rather than items, nor does it report a decomposition that controls for run. This is load-bearing because the headline claim that aggregation is nearly useless depends on ρ being near 0.9; a ρ of, say, 0.8 would imply Neff(30)=1.27 instead of 1.10 and a larger potential gain.
The reader's weakest_assumption correctly identifies that the decomposition treats c_j as common to all draws of a model on an item, and notes the run component of up to 11% variance in 6 models. However, the reader does not fully develop the consequence: that the one-way ANOVA by item conflates item-level variation with run-level and prompt-level variation, potentially inflating ρ. The paper's own Table D.2 shows that for some models (e.g., mistral-7b, run component 10.6%; smollm3-3b, 8.1%; llama3.1-8b, 6.7%) the run component is substantial, and the prompt-pair mixing adds further unmeasured variation. The second corpus replication (ρ=0.915) uses a single prompt pair but still includes run variation; it is reassuring but not a full control. The concern is not that the paper is wrong, but that the headline ρ may be an overestimate of the truly item-level share, and the paper's own data provide the means to check this. The concrete test is feasible with the existing data (if released) or by the authors. The verdict remains CONDITIONAL because the paper's own limitations and the second corpus support the qualitative conclusion, but the precise magnitude of ρ is load-bearing for the quantitative claim about the 1.7% reduction. I agree with the reader that the central result is plausible and the paper is honest about limitations; my concern is more specific about the variance decomposition's sensitivity to uncontrolled factors.
The referee's report
If the decomposition and the headline ρ ≈ 0.9 hold, the paper provides a clear statistical explanation for why multi-agent aggregation of language models on continuous estimation tasks yields much smaller gains than the 1/√N averaging argument would predict. The finding that the item-level error is largely irreducible by standard aggregation or communication protocols is practically important for the design of multi-agent systems. The paper is thorough: it includes two corpora, multiple benchmarks, robustness checks (median rule, item-clustered intervals, alternative reference), and explicit limitations. The decomposition itself is a standard one-way ANOVA, but its application to Fermi estimation is novel and the empirical scale of the study (2.51M answers, 15 models) is a strength.
Pith's reading of the paper (a machine's summary, not the authors')
The paper tries to establish that the dominant source of error in language-model-based Fermi estimation is not the variance between draws of the same model on an item, but the shared error that every draw of that model makes on that item. This item-level component accounts for about 90% of the error variance, so aggregating many agents (even 30) reduces error by only a few percent, far less than the 1/√N rate that motivates multi-agent systems. The finding matters because it challenges the premise that simply adding more agents can substitute for a better model, and redirects attention to changing the item-level error itself.
What carries the argument. The error decomposition ejk = μ + cj + ujk, where cj is the item-level component common to all draws of a model on item j, and ρ = A/(A+B) is the intraclass correlation over items. This decomposition isolates the part of the error that aggregation cannot remove (μ+cj) from the part it can (ujk). The design effect Neff(N) = N/(1+(N-1)ρ) then quantifies the effective number of independent agents.
Core claim. The paper decomposes the log10 error of each answer into a model-item mean error (cj) and a residual (ujk). The item-level component cj accounts for a median 90.6% of the error variance over items (ρ=0.906). Given the item, teammates' residuals are uncorrelated, so teams behave as independent draws, but the shared error passes through unchanged under aggregation rules like mean, median, or trimmed mean. Consequently, teams of 30 reduce single-agent error by a median 1.7% (max 5.4%), whereas unbiased independent draws would reduce it by 74–79%. The paper also finds that all models compress estimates toward $10^{6}$, explaining a median 17% of cj, and that communication (debate) reduces within-team dispersion by 93% but leaves cj unchanged.
The premise it rests on. The decomposition ejk = μ + cj + ujk assumes that the item-level component cj is a property of the model-item pair that is common to all draws, and that the benchmark's items are a representative distribution of the quantities of interest.
If the paper is right (Pith's inferences, not the authors')
Multi-agent LLM systems that rely on averaging or voting without addressing shared item-level error will see diminishing returns beyond a few agents.
Benchmarks like realFP and SciOly have item-level error structures that may generalize to other continuous estimation tasks, limiting the wisdom-of-crowds benefit.
Interventions that reduce the item-level component (e.g., model edits, different prompts, or recalibration) are more promising than increasing team size or debate.
The compression effect (estimates regressing toward 10^6) suggests a systematic bias that linear recalibration can partially remove but not fully eliminate.
Temperature increases and activation steering do not meaningfully change team error, so diversity interventions that only increase variance are ineffective.
If the item-level error is mostly due to shared training data or model architecture, then combining models from different families or training paradigms may reduce cj more than combining similar models.
The findings suggest that for Fermi estimation, the practical ceiling on aggregation is low; a single strong model (like gpt-oss-20b) already outperforms any mixture of weaker models.
The result may extend to other tasks where errors are dominated by item-level factors (e.g., factuality, numerical reasoning), implying that multi-agent systems in those domains also face a similar limit.
The paper's decomposition method could be applied to other continuous-output tasks to diagnose whether aggregation is worth pursuing.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Multi-agent systems of language models are often motivated by an averaging argument: if the agents' errors are unbiased and independent, the error of an aggregate of N answers falls as 1/sqrt(N). Fermi estimation, whose continuous answers are aggregated on the logarithmic scale, permits a direct test. We decompose the log10 error of each answer into the mean error of its model on the item, common to all its draws on the item, and a residual, and estimate both from 2.51M answers given before any communication by 15 open-weight models (3B-72B) on realFP, with replication under a single prompt pair on realFP, Science Olympiad Fermi questions and a guesstimation set. Given the item, teammates' errors are uncorrelated and teams behave as independent draws, but the item-level component accounts for a median 0.906 of the error variance over items. Teams of thirty therefore reduce a single agent's error by a median 1.7% (at most 5.4%), whereas draws that were also unbiased on each item would reduce it by 74-79%. All models compress their estimates toward about 10^6; this trend explains a median 17% of the item-level component, and a cross-validated linear recalibration on labelled items removes the overall bias but not that component. Three rounds of debate reduce within-team dispersion by 93% and leave the item-level component unchanged; revision without peers attains the same small gain. Mixing five models of similar accuracy reduces error by 6.6% relative to their own homogeneous teams, but no mixture improves on the most accurate model. Higher sampling temperature changes team error by about 1% at most, and contrastive activation steering not detectably; per-agent scaffold prompts (not significant after multiplicity correction) and skipping one layer in a single-model screen improve it. The results are conditional on the models, benchmarks, prompts and settings tested.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-11-13
The authors' main claim, as the classifier picked it out 0.66
“Teams of thirty therefore reduce a single agent's error by a median 1.7% (at most 5.4%), whereas draws that were also unbiased on each item would reduce it by 74-79%.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.04 · states limitation 0.97 · human participants 0.07 · controlled comparison 0.84 · randomized assignment 0.05 · states quantitative result 0.99 · compares with prior results 0.84 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Multi-agent systems of language models are often motivated by an averaging argument: if the agents' errors are unbiased and independent, the error of an aggregate of N answers falls as 1/sqrt(N). | background 1.00 | 0.06 | 0.03 | asserted 0.78 |
| Fermi estimation, whose continuous answers are aggregated on the logarithmic scale, permits a direct test. | background 0.46 | 0.15 | 0.02 | asserted 0.85 |
| We decompose the log10 error of each answer into the mean error of its model on the item, common to all its draws on the item, and a residual, and estimate both from 2.51M answers given before any communication by 15 open-weight models (3B-72B) on realFP, with replication under a single prompt pair on realFP, Science Olympiad Fermi questions and a guesstimation set. | method 0.98 | 0.97 | 0.04 | asserted 0.96 |
| Given the item, teammates' errors are uncorrelated and teams behave as independent draws, but the item-level component accounts for a median 0.906 of the error variance over items. | result 1.00 | 0.95 | 0.62 | asserted 1.00 |
| Teams of thirty therefore reduce a single agent's error by a median 1.7% (at most 5.4%), whereas draws that were also unbiased on each item would reduce it by 74-79%. | result 0.97 | 0.93 | 0.66 | asserted 1.00 |
| All models compress their estimates toward about 10^6; this trend explains a median 17% of the item-level component, and a cross-validated linear recalibration on labelled items removes the overall bias but not that component. | result 0.94 | 0.95 | 0.07 | asserted 0.99 |
| Three rounds of debate reduce within-team dispersion by 93% and leave the item-level component unchanged; revision without peers attains the same small gain. | result 1.00 | 0.95 | 0.07 | asserted 1.00 |
| Mixing five models of similar accuracy reduces error by 6.6% relative to their own homogeneous teams, but no mixture improves on the most accurate model. | result 1.00 | 0.95 | 0.08 | asserted 1.00 |
| Higher sampling temperature changes team error by about 1% at most, and contrastive activation steering not detectably; per-agent scaffold prompts (not significant after multiplicity correction) and skipping one layer in a single-model screen improve it. | result 1.00 | 0.94 | 0.03 | asserted 0.93 |
| The results are conditional on the models, benchmarks, prompts and settings tested. | limitation 1.00 | 0.41 | 0.03 | asserted 0.88 |
What kind of paper the classifier reports this is: empirical study 1.00 · evidence from observation or experiment 0.87
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-13: 1 request, 7,910 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (2)
Figures are the authors', reproduced from the paper.
References (17)
- [1] Break the quantity into a short chain of factors you can estimate (e.g. population x rate x fraction) on Pith
- [2] Give each factor a rough round value WITH its unit on Pith
- [3] Combine them (multiply/divide) into ONE number. Track units and powers of ten carefully – a dropped or extra factor of 10 is the most common failure. Aim to be within an order of magnitude. Format EXACTLY (answer and confidence FIRST, then reasoning): FINAL: <your single-numbe... on Pith
- [4] Name a quantity of the SAME KIND whose size you are confident about, and state its value on Pith
- [5] Say how the target differs from it – larger or smaller, and by roughly what ratio on Pith
- [6] Apply that ratio to your reference value. Rest the whole estimate on one anchor and one adjustment. Do not enumerate sub-factors. Format EXACTLY (answer and confidence FIRST, then reasoning): FINAL: <your single-number estimate; plain or scientific notation> CONF: <0-100> ANCH... on Pith
- [7] State a value the answer is CERTAINLY ABOVE, and why it cannot be smaller on Pith
- [8] State a value the answer is CERTAINLY BELOW, and why it cannot be larger on Pith
- [9] Tighten whichever bound rests on the weaker argument on Pith
- [10] Report the geometric middle of your final bracket – the square root of the two multiplied. Format EXACTLY (answer and confidence FIRST, then reasoning): FINAL: <your single-number estimate; plain or scientific notation> CONF: <0-100> BRACKET: <lower bound, upper bound, and why... on Pith
- [11] State the units the answer must be in on Pith
- [12] Write an equation whose units reduce to those units – a product or quotient of measurable quantities, SYMBOLS ONLY, no numbers yet on Pith
- [13] Only then assign a magnitude to each symbol on Pith
- [14] Evaluate, and check the units of the result match step 1. Format EXACTLY (answer and confidence FIRST, then reasoning): FINAL: <your single-number estimate; plain or scientific notation> CONF: <0-100> EQUATION: <the symbolic equation, then the magnitudes you substituted> Scaff... on Pith
- [15] Break the quantity into a chain of factors you can estimate (e.g. population x rate x fraction) on Pith
- [16] For EACH factor: give a rough round value WITH its unit and note its power of ten. Check the units cancel to the target unit on Pith
- [17] Do NOT multiply the factors yourself – a dropped or extra factor of ten is the most common 14 failure. List them on the FACTORS line as numbers joined by * and / (a calculator computes it). Aim to be within an order of magnitude. Format EXACTLY: FACTORS: <number> * <number> / ... on Pith
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment