REVIEW 3 major objections 5 minor 2 references
Numerical evaluation of deliberative discussions of the UK food system: stimuli, demographics, and opinion reversion
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper develops a quantitative model of deliberative discussions and uses it to show that workshop-induced shifts in citizens' ratings of food-system responsibility partly erode in the days after each workshop, with the degree of…
desk verdict A useful descriptive study with a broken reversion model: equation (5) as printed contradicts the paper's own interpretation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying object is the reversion function $d_{i,w,m} = -1 + \exp\left(\frac{\alpha_i}{D}\right)\exp\left(\frac{\alpha_i}{\Delta_{w,m}-D}\right)$, which is plugged into the linear predictor $U_{i,s,t} = \sum_w \beta_{s,w}\left(I_w - \rho_i d_{i,w}\right)$. The $\beta_{s,w}$ parameters capture the immediate effect of each workshop on each stakeholder's perceived responsibility; $\rho_i$ captures how much of that effect eventually erodes; $\alpha_i$ controls how quickly the erosion happens; both are allowed to depend on individual characteristics. This sits inside a multivariate ordered logit, so the five stakeholder ratings are estimated jointly, with individual random effects and an error component linking the five equations.
What would settle it
Re-estimate the model replacing the parametric decay function with flexible day-count fixed effects (a dummy for each day since the workshop). If the estimated day-specific effects do not decline monotonically toward the reported floors for the four groups, or if the ordering of rural and urban voters and non-voters reverses, the reversion claim would be refuted. A simpler check: evaluate the printed decay function at $\Delta = 17$; it divides by zero, so any implementation must have altered the function, and the estimates should be shown to be insensitive to that alteration.
Extended reading notes
Core claim
The paper claims that a multivariate ordered logistic model of 0–10 responsibility ratings, with a parametric decay function, can document opinion change and opinion reversion in deliberative events. For the Food Conversation, governments were rated most responsible and farmers least; the workshops significantly raised responsibility ratings for individuals and supermarkets, left the food industry essentially unchanged, and lowered responsibility for farmers. The reversion part of the model estimates, for each person, an overall reversion level $\rho_i$ and a rate parameter $\alpha_i$, and reports reversion parameters for four groups: rural non-voters 0.72, rural voters 0.33, urban non-voters 1.07, and urban voters 0.68, where reversion equal to 1 means all of a workshop effect has waned, 0 means none, and greater than 1 means the effect was amplified over time. The paper interprets this as evidence that opinion change during deliberation can wane within days, and that the persistence of such change is heterogeneous.
Load-bearing premise
The reversion results rely on the assumption that opinion fades smoothly along a specific mathematical curve with a floor, and that printed curve is not defined at the largest day gap in the data; if the real fading is not shaped like that curve, or if outside events rather than time caused the changes, the estimates would be biased.
Editorial extensions
If this is right
- Workshop content has measurable effects: specific workshops raised or lowered responsibility ratings for particular stakeholders, such as workshop 2 increasing government responsibility and decreasing supermarket responsibility.
- The reversion results imply that opinion change from a single deliberative session is not necessarily durable, since part or all of the workshop-induced shift can vanish within days.
- Reversion varies by group, so aggregate statements about deliberative effects can hide important differences between rural and urban residents and between voters and non-voters.
- The quantitative approach can be added to existing qualitative deliberation analyses at low marginal cost, providing a replicable way to evaluate whether deliberative processes change opinion.
- The method transfers beyond food systems to any deliberative setting where participants report ordered ratings at multiple time points.
Reading between the lines
- Beyond the paper's claims: if reversion is as fast as the model suggests, deliberative forums may need booster sessions or repeated engagement to have a sustained influence on policy-relevant opinions.
- The finding that urban non-voters show reversion greater than 1 (workshop effect amplified over time) may reflect external events or group dynamics rather than pure decay; a control group that did not attend workshops could separate these explanations.
- The modelling framework could be extended to link reversion to actual behaviour change, such as voting or purchasing, rather than only to self-reported responsibility ratings.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper develops a quantitative complement to qualitative analysis of a UK-wide deliberative event (the Food Conversation), in which 345 citizens attended five workshops. Participants rated the responsibility of five stakeholders for food-system change on a 0–10 scale at the beginning and end of each workshop. The authors fit multivariate ordered logit models with random effects to estimate workshop-specific effects, demographic differences, and a 'reversion' effect capturing how within-workshop opinion shifts erode between workshops. The descriptive results (e.g., governments seen as most responsible, farmers least; responsibility for individuals increased most) are clear and align with the qualitative reports. The central quantitative contribution, however, is the reversion model summarized by Eq. (5), which the paper uses to claim that reversion was strongest among those who intended to vote rather than abstain.
Significance. If the reversion model were sound, the paper would offer a low-cost, replicable way to quantify opinion dynamics in deliberative processes, and its release of code and data would be a useful methodological resource. The descriptive findings and the multivariate modeling framework are valuable complements to existing qualitative work. However, the paper's headline claim about voting-intention differences in reversion rests entirely on the decay function in Eq. (5), and that function has a mathematical defect at an observed data point. Because the reversion estimates and their demographic contrasts are identified through this functional form, the central quantitative result is not currently defensible. The paper also suffers from substantial sample attrition and missing demographic data that threaten the demographic comparisons, though these are acknowledged in the limitations section.
major comments (3)
- [Methods, Eq. (5); Results, Table 3 and Figure 3]
- [Results, Sample subsection and Table 1; Discussion, Limitations]
- [Methods, Eq. (5) and surrounding text; Results, Reversion paragraph]
minor comments (5)
- [Methods, Eq. (3) and Eq. (5)]
- [Table 3]
- [Results, Figure 3]
- [References]
- [Throughout]
Circularity Check
No significant circularity: the reversion parameters are in-sample empirical estimates reported as findings, not out-of-sample predictions derived from their own inputs.
full rationale
The paper develops a statistical model and estimates its parameters from the Food Conversation data; it does not claim to derive reversion effects from an independent source nor to predict them out-of-sample. The central claim that workshop-induced shifts in perceived responsibility waned over time, and differed by voting intention and rurality, is a direct report of estimated coefficients (Table 3, Figure 3). This is ordinary empirical fitting, not circular reasoning. The descriptive rankings (governments most responsible, farmers least) come from pooled means in Table 2, independent of the model. The quantitative results are compared with qualitative findings only after estimation, as a consistency check, not used as inputs to the model. The paper's citations (Hess 2006; Hess and Palma 2019; Hess and Daly 2024) are to estimation software and choice-modelling overviews, and are not used as load-bearing justification for the target result; there are no self-citations by the present authors carrying the argument. The decay function in Eq. (5) has a mathematical singularity at Delta=D=17, and the paper does not test alternative decay specifications, but this is a correctness/robustness concern, not circularity: the reversion estimates are not equivalent by construction to the data used to fit them. No circular step can be exhibited as a reduction of the output to the input, so the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (12)
- beta_supermarkets_workshop1 =
1.56
- beta_government_workshop2 =
0.94
- beta_farmers_workshop1 =
-1.56
- beta_farmers_workshop3 =
0.89
- beta_individuals_workshop5 =
0.76
- rho_base =
0.33
- rho_rural =
0.35
- rho_did_not_vote =
-0.39
- alpha_base =
119.17
- alpha_non_white =
-112.69
- alpha_higher_education =
83.67
- alpha_rural =
-9.85
assumptions (5)
- domain assumption Ordered logit proportionality: the effect of covariates is constant across rating thresholds within each stakeholder equation.
- domain assumption Random effects are normally distributed and approximated by modified Latin hypercube sampling draws.
- ad hoc to paper The decay function in equation (5) correctly represents the temporal pattern of opinion reversion.
- domain assumption Workshop indicators are cumulative: once a workshop has occurred, its effect persists until modified by reversion.
- domain assumption Missing demographic and outcome data are missing at random conditional on included covariates.
Cite this review
Pith. "Pith review of Numerical evaluation of deliberative discussions of the UK food system: stimuli, demographics, and opinion reversion." pith.science (2026). https://pith.science/paper/5JGP4LIC
@misc{pith2026250614102,
author = {Pith},
title = {Pith review of: Numerical evaluation of deliberative discussions of the UK food system: stimuli, demographics, and opinion reversion},
year = {2026},
howpublished = {\url{https://pith.science/paper/5JGP4LIC}},
note = {Machine review of arXiv:2506.14102}
}
read the original abstract
There is increasing acknowledgement - including from the UK government - of the benefit of employing deliberative processes (deliberative fora, citizens' juries, etc.). Evidence suggests that the qualitative reporting of deliberative fora are often unclear or imprecise. If this is the case, their value to policymakers could be diminished. In this study we develop numerical methods of deliberative processes to document people's preferences, as a complement to qualitative analysis. Data are taken from the Food Conversation, a nationwide public consultation on reformations of the food system comprising 345 members of the general public. Each participant attended 5 workshops, each with differing stimuli covering subtopics of the food system. In each workshop, individuals twice reported responsibility, from 0-10, for changing the food system for 5 stakeholders (governments, the food industry, supermarkets, farmers, individuals). Analyses examined individuals' perceptions of food system change responsibility. Governments were most responsible and farmers least so. We assessed variation by workshop content, and by demographics. Reported responsibility changed most for individuals, and changed least for the food industry. We devise a model to document a reversion effect, where shifts in perceptions on responsibility that occurred during workshops waned over time; this was strongest among those who intended to vote (rather than not to). These results can support qualitative analyses and inform food system policy development. These methods are readily adopted for any such deliberative process, allowing for statistical evaluation of whether they can induce opinion change.
Figures
Reference graph
Works this paper leans on
-
[1]
Can Deliberation Have Lasting Effects?
Boulianne S. Building Faith in Democracy: Deliberative Events, Political Trust and Efficacy. Political Studies. 2018;67(1):4-30. CCC report 2022 Fishkin, J., Deliberative polling: Reflections on an ideal made practical. In: Geissel, Brigitte & Newton, Kenneth. (2012). Evaluating democratic innovations: curing the democratic malaise? Eds Geissel, B. and Ne...
work page 2012
-
[841]
Eight ways to institutionalise deliberative democracy
Mouter N, Koster P, Dekker T. Contrasting the recommendations of participatory value evaluation and cost-benefit analysis in the context of urban mobility investments. Transportation Research Part A: Policy and Practice. 2021;144:54-73. OECD (2021), “Eight ways to institutionalise deliberative democracy”, OECD Public Governance Policy Papers, No. 12, OECD...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.