{"id":"fa16b0ef-e35c-4497-9a77-408b88adb8c9","arxiv_id":"2506.14102","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":12,"one_line_summary":"A multivariate ordered logit model of UK citizens' 0-10 responsibility ratings across five deliberative workshops measures opinion shifts and a reversion effect that varies by voting intention and rurality.","lead":"This paper adds a numerical layer to public deliberation events by asking 213 UK citizens to rate responsibility for food system change before and after each of five workshops. It reports that ratings shifted most for individuals, and that a model-measured reversion effect was stronger among those who intended to vote.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation (5) is undefined at the maximum observed gap, and the headline voting-intention reversion result depends on a decay function whose endpoint behavior is an artifact of the functional form.","rationale":"The reader's weakest assumption identifies Eq. (5) as a mathematical weak point, and that is indeed the most load-bearing concern: the reversion effect, and especially the voting-intention comparison, is identified through this decay function. I agree with the reader that the division by zero at Delta = 17 is a serious internal defect. I add that even away from the singularity, the function's limit is -1, so the terminal behavior is determined by the arbitrary functional form rather than by data. The paper's own limitations section candidly acknowledges selection bias and missing demographics, so I do not treat those as new objections; the equation issue is internal and unaddressed. I flag partial rather than full agreement because the reader emphasizes external-event confounding, whereas I see the primary problem as the specification and interpretability of Eq. (5) itself. A single re-estimation with a corrected or alternate decay function would settle whether the voting-intention result is an artifact; until then, the abstract's reversion claim is not reliable. The descriptive results (governments most responsible, farmers least) are independently supported by Table 2 and align with qualitative reports, so they would survive even if the reversion model is rejected.","tokens_in":10231,"tokens_out":6341,"duration_ms":65516,"concrete_test":"Re-estimate the model on the same 213-person sample with a decay function defined on the full observed range, e.g. d = 1 - exp(-alpha_i * Delta/D) or another monotone exponential-style decay with a floor, and inspect whether the 'did not vote' coefficient in the reversion component retains sign and significance. Additionally, rerun the published specification with the Delta = 17 observations excluded or collapsed; if the voting-intention effect flips or loses significance, the Eq. (5) singularity is load-bearing and the rejection stands.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central reversion claim is identified through Eq. (5): d = -1 + exp(alpha/D) * exp(alpha/(Delta-D)), with D=17 and 1 <= Delta <= 17. At Delta = D, the term exp(alpha/(Delta-D)) has a zero denominator, so the function is undefined at the largest observed gap. For Delta < 17, positive alpha makes d approach -1 as Delta approaches 17, so the final observed interval's behavior is an artifact of the chosen formula rather than a measured decay. The paper neither addresses the singularity nor tests an alternative decay specification. Because the headline result is a comparison of reversion by voting intention, any correction to Eq. (5) could change the group-specific reversion values and the sign or size of the 'did not vote' contrast. The acknowledged selection bias is important but secondary; the equation-level defect alone undermines the central quantitative contribution. The descriptive findings about responsibility rankings are not affected by this concern and appear robust.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops a quantitative complement to qualitative analysis of a UK-wide deliberative event (the Food Conversation), in which 345 citizens attended five workshops. Participants rated the responsibility of five stakeholders for food-system change on a 0–10 scale at the beginning and end of each workshop. The authors fit multivariate ordered logit models with random effects to estimate workshop-specific effects, demographic differences, and a 'reversion' effect capturing how within-workshop opinion shifts erode between workshops. The descriptive results (e.g., governments seen as most responsible, farmers least; responsibility for individuals increased most) are clear and align with the qualitative reports. The central quantitative contribution, however, is the reversion model summarized by Eq. (5), which the paper uses to claim that reversion was strongest among those who intended to vote rather than abstain.","tokens_in":10479,"tokens_out":4774,"duration_ms":46440,"significance":"If the reversion model were sound, the paper would offer a low-cost, replicable way to quantify opinion dynamics in deliberative processes, and its release of code and data would be a useful methodological resource. The descriptive findings and the multivariate modeling framework are valuable complements to existing qualitative work. However, the paper's headline claim about voting-intention differences in reversion rests entirely on the decay function in Eq. (5), and that function has a mathematical defect at an observed data point. Because the reversion estimates and their demographic contrasts are identified through this functional form, the central quantitative result is not currently defensible. The paper also suffers from substantial sample attrition and missing demographic data that threaten the demographic comparisons, though these are acknowledged in the limitations section.","major_comments":[{"comment":"","section":"Methods, Eq. (5); Results, Table 3 and Figure 3"},{"comment":"","section":"Results, Sample subsection and Table 1; Discussion, Limitations"},{"comment":"","section":"Methods, Eq. (5) and surrounding text; Results, Reversion paragraph"}],"minor_comments":[{"comment":"","section":"Methods, Eq. (3) and Eq. (5)"},{"comment":"","section":"Table 3"},{"comment":"","section":"Results, Figure 3"},{"comment":"","section":"References"},{"comment":"","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper's descriptive statistics and the multivariate ordered logit framework are sound and potentially useful, and the authors have made code and data available, which is commendable. However, the central reversion result is built on a decay function that is undefined at an observed data point and behaves in the opposite direction to the intended interpretation. This is not a minor typo; it means the reported reversion parameters and the voting-intention comparison are not meaningful as they stand. The issue is technically fixable (specify a valid decay function, re-estimate, and run robustness checks), so I recommend major revision rather than rejection. The sample-selection concerns are secondary but should be addressed with sensitivity analyses. I would advise the editor to request a revised version that re-estimates the reversion model and explicitly reports the sensitivity of the 'did not vote' contrast to the choice of decay specification."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The descriptive findings are likely solid: government most responsible, farmers least, and the rise in perceived individual responsibility is the biggest movement. The authors also deserve credit for an honest set of limitations (sample selection, missing demographics) and for shipping code. The reversion model, though, has a load-bearing mathematical error. Equation (5) is undefined at Δ=D and, for positive alpha, drives d toward -1 as Δ approaches D, which means the workshop effect is amplified rather than decayed. That is exactly the opposite of what the paper claims. Given the alpha estimates are huge (base ~119), the function behaves pathologically across the observed day gaps. So the headline result—reversion strongest among voters—is not identified. This is not a minor typo; it is the core of the novel contribution.\n\nThe rest of the modeling is workmanlike: a multivariate ordered logit with individual random effects is a reasonable way to handle repeated ratings. The demographic interactions are sparse, and the reversion heterogeneity terms drop out. The selection issue is real (213 of 345, with up to 44.6% missing on some covariates) but at least acknowledged.\n\nOverall: I would not cite or build on this as is, but it deserves a serious referee. The data are valuable, the idea of parameterizing reversion is worth pursuing, and the error is fixable with a corrected decay function and sensitivity analysis. A good referee could turn this into a useful paper. I would send it to review.","headline":"A useful descriptive study with a broken reversion model: equation (5) as printed contradicts the paper's own interpretation.","tokens_in":10965,"tokens_out":6130,"would_cite":false,"duration_ms":59932,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper develops a quantitative model of deliberative discussions and uses it to show that workshop-induced shifts in citizens' ratings of food-system responsibility partly erode in the days after each workshop, with the degree of…","keywords":["deliberative processes","choice modelling","food system perceptions","perceptions reversion","ordered logit","UK Food Conversation","opinion dynamics"],"falsifier":"Re-estimate the model replacing the parametric decay function with flexible day-count fixed effects (a dummy for each day since the workshop). If the estimated day-specific effects do not decline monotonically toward the reported floors for the four groups, or if the ordering of rural and urban voters and non-voters reverses, the reversion claim would be refuted. A simpler check: evaluate the printed decay function at $\\Delta = 17$; it divides by zero, so any implementation must have altered the function, and the estimates should be shown to be insensitive to that alteration.","tokens_in":10002,"feed_emoji":"⏳","tokens_out":7327,"duration_ms":73330,"temperature":0.7,"pith_summary":"The paper sets out a low-cost numerical complement to qualitative deliberation analysis, applied to the UK Food Conversation, where 213 participants rated the responsibility of five stakeholders for changing the food system at the start and end of each of five workshops. The central claim is that the workshops shifted these responsibility ratings, and that the shifts did not persist: a 'reversion effect' pulled ratings back toward their pre-workshop levels over the following days. The model reports reversion parameters for four groups—rural non-voters, rural voters, urban non-voters, urban voters—showing that the fade differs by place and voting intention. A sympathetic reader would care because deliberative forums are increasingly used in policy, and knowing whether they actually change opinion, and for whom, bears directly on their value to policymakers.","feed_headline":"Opinion shifts from UK food workshops fade within days","feed_subtitle":"Model tracks responsibility ratings reverting to pre-workshop levels; voting intent and rurality change the fade.","key_machinery":"The carrying object is the reversion function $d_{i,w,m} = -1 + \\exp\\left(\\frac{\\alpha_i}{D}\\right)\\exp\\left(\\frac{\\alpha_i}{\\Delta_{w,m}-D}\\right)$, which is plugged into the linear predictor $U_{i,s,t} = \\sum_w \\beta_{s,w}\\left(I_w - \\rho_i d_{i,w}\\right)$. The $\\beta_{s,w}$ parameters capture the immediate effect of each workshop on each stakeholder's perceived responsibility; $\\rho_i$ captures how much of that effect eventually erodes; $\\alpha_i$ controls how quickly the erosion happens; both are allowed to depend on individual characteristics. This sits inside a multivariate ordered logit, so the five stakeholder ratings are estimated jointly, with individual random effects and an error component linking the five equations.","core_discovery":"The paper claims that a multivariate ordered logistic model of 0–10 responsibility ratings, with a parametric decay function, can document opinion change and opinion reversion in deliberative events. For the Food Conversation, governments were rated most responsible and farmers least; the workshops significantly raised responsibility ratings for individuals and supermarkets, left the food industry essentially unchanged, and lowered responsibility for farmers. The reversion part of the model estimates, for each person, an overall reversion level $\\rho_i$ and a rate parameter $\\alpha_i$, and reports reversion parameters for four groups: rural non-voters 0.72, rural voters 0.33, urban non-voters 1.07, and urban voters 0.68, where reversion equal to 1 means all of a workshop effect has waned, 0 means none, and greater than 1 means the effect was amplified over time. The paper interprets this as evidence that opinion change during deliberation can wane within days, and that the persistence of such change is heterogeneous.","pith_inferences":["Beyond the paper's claims: if reversion is as fast as the model suggests, deliberative forums may need booster sessions or repeated engagement to have a sustained influence on policy-relevant opinions.","The finding that urban non-voters show reversion greater than 1 (workshop effect amplified over time) may reflect external events or group dynamics rather than pure decay; a control group that did not attend workshops could separate these explanations.","The modelling framework could be extended to link reversion to actual behaviour change, such as voting or purchasing, rather than only to self-reported responsibility ratings."],"forward_implications":["Workshop content has measurable effects: specific workshops raised or lowered responsibility ratings for particular stakeholders, such as workshop 2 increasing government responsibility and decreasing supermarket responsibility.","The reversion results imply that opinion change from a single deliberative session is not necessarily durable, since part or all of the workshop-induced shift can vanish within days.","Reversion varies by group, so aggregate statements about deliberative effects can hide important differences between rural and urban residents and between voters and non-voters.","The quantitative approach can be added to existing qualitative deliberation analyses at low marginal cost, providing a replicable way to evaluate whether deliberative processes change opinion.","The method transfers beyond food systems to any deliberative setting where participants report ordered ratings at multiple time points."],"supporting_citations":[{"why":"Supplies the Food Conversation dataset: 345 UK citizens, five workshops, and responsibility ratings for five stakeholders.","marker":"FFCC, 2024"},{"why":"Source of the qualitative Food Conversation reports used for comparison and for details of the civic lottery recruitment.","marker":"HVM, 2025"},{"why":"Provides the estimation software used for the multivariate ordered logit models.","marker":"Hess and Palma, 2019"},{"why":"Supplies the modified Latin hypercube sampling method used for the 5000 draws in the random-parameter specifications.","marker":"Hess, 2006"},{"why":"Supplies the stated-choice modelling framework being adapted for preference elicitation.","marker":"Hensher et al., 2015"},{"why":"Provides the closest prior analysis of lasting effects of deliberation over time, which the reversion measure is compared against.","marker":"Fishkin et al., 2024"},{"why":"Extended Q-methodology with two time points, the methodological antecedent for measuring opinion change in deliberation.","marker":"Reckers-Droog et al., 2020"},{"why":"Participatory value evaluation approach, contrasted as a static opinion-assessment method.","marker":"Mouter et al., 2021"}],"fun_headline_variants":["Food workshop opinion shifts fade within days, UK study","Urban non-voters revert fastest after food deliberation","Model reveals rapid decay of deliberative opinion change","Rural voters retain food responsibility shifts longer","Voting intent predicts reversion in workshop opinion data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reversion results rely on the assumption that opinion fades smoothly along a specific mathematical curve with a floor, and that printed curve is not defined at the largest day gap in the data; if the real fading is not shaped like that curve, or if outside events rather than time caused the changes, the estimates would be biased.","fun_headline_variants_meta":{"raw":{"variants":["Food workshop opinion shifts fade within days, UK study","Urban non-voters revert fastest after food deliberation","Model reveals rapid decay of deliberative opinion change","Rural voters retain food responsibility shifts longer","Voting intent predicts reversion in workshop opinion data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000278,"raw_usage":{"total_tokens":1693,"prompt_tokens":1024,"completion_tokens":669,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":640,"completion_tokens_details":{"reasoning_tokens":597}},"tokens_in":640,"tokens_out":669,"duration_ms":7083,"temperature":1.0,"reasoning_tokens":597,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:17:30.712999+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-estimate the model replacing the parametric decay function with flexible day-count fixed effects (a dummy for each day since the workshop). If the estimated day-specific effects do not decline monotonically toward the reported floors for the four groups, or if the ordering of rural and urban voters and non-voters reverses, the reversion claim would be refuted. A simpler check: evaluate the printed decay function at $\\Delta = 17$; it divides by zero, so any implementation must have altered the function, and the estimates should be shown to be insensitive to that alteration.","supporting_citations":[],"review_version":1}