Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Counterfactual Voting Adjustment for Quality Assessment and Fairer Voting in Online Platforms with Helpfulness Evaluation

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Helpfulness votes carry hidden rank and crowd biases; a counterfactual adjustment recovers the answer quality underneath.

desk verdict The paper's counterfactual adjustment is rank-invariant under its own equations, so the reported ranking gains come from the added position and herding regressors, not from the causal averaging step; the large-scale descriptive study is the salvageable part. read the letter →

arxiv 2506.21362 v1 pith:BOMSB7DG submitted 2025-06-26 cs.CE

Chang Liu , Yixin Wang , Moontae Lee This is my paper
classification cs.CE
keywords causalinferencehelpfulnessvotingpositionbiasherdingcounterfactualadjustmentbackdoorStackExchangecontentranking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Online helpfulness votes are not a clean signal of content quality: an answer displayed higher attracts more votes, and visible prior votes nudge later voters to conform. The paper proposes the Counterfactual Voting Adjustment (CVA), a causal procedure that asks what a vote would have been if the answer had appeared at a different display rank and with a balanced prior vote count, then averages over those counterfactual settings to estimate quality. On semi-synthetic data where true quality is known, CVA recovers quality rankings more accurately than raw vote-difference rankings and than the earlier Chinese Voting Process model. On 120 real StackExchange communities, reranking answers by CVA quality aligns better with comment sentiment and with GPT-4o helpfulness judgments than the platform's vote-based ranking or non-causal model rerankings. If this holds, platforms could rank content by something closer to underlying quality rather than by early popularity.

What carries the argument

The load-bearing object is the counterfactual vote estimate $\hat{Q}^T_{ij} = \sum_t \int E[V^t_{ij}(\tilde{g}^t_{ij}) \mid B^t_{ij}]\, P(\tilde{m}^t_{ij}, \tilde{d}^t_{ij})\, d\tilde{m}^t_{ij}\, d\tilde{d}^t_{ij}$, computed from a logistic voting model with quality, herding, length, and rank terms. It does the work of a backdoor adjustment: conditioning on pre-reading features $B$ (here only relative length) and then integrating over the treatment variables (existing votes $M$ and display rank $D$) removes their influence from the quality estimate while leaving the response text fixed. The same fitted model also yields a position-bias coefficient $\beta$ and a herding-bias odds ratio, which is how the paper compares voting dynamics across communities.

What would settle it

One decisive test is a randomized field experiment on a Q&A platform: assign display ranks randomly and show or hide prior vote counts, then compare true vote rates with CVA's counterfactual predictions; if CVA systematically mispredicts votes under randomization, its adjustment is misspecified. A cheaper semi-synthetic falsifier is to generate voting data where an unobserved confounder such as author reputation drives both rank and votes; if CVA no longer recovers the predefined quality rankings, the ignorability assumption is load-bearing in exactly the way the paper concedes.

Watch

Extended reading notes

Core claim

The paper's central claim is that position bias and herding bias in helpfulness votes can be jointly undone without randomized experiments, so long as the voting trajectory is observed. CVA models the probability of a positive vote as a Bernoulli draw with $\mathrm{sigmoid}(q^t_{ij} + \lambda R^t_{ij} + \nu_i L^t_{ij} + \beta/(1 + D^t_{ij}))$, where $q$ is the answer's quality, $R$ the perceived positive-vote ratio, $L$ the response's relative length, and $D$ the displayed rank. It then forms the quality estimate by integrating out the existing-vote distribution and display rank, an instance of backdoor adjustment, so the estimate is invariant to presentation context while staying close to the observed voting behavior. The fitted coefficients $\lambda$ and $\beta$ quantify each community's herding and position sensitivity, and the quality estimates rerank answers closer to independent quality proxies than vote counts or the CVP baseline.

Load-bearing premise

The load-bearing premise is that, after conditioning on the answer's relative length, no unmeasured factor such as author reputation or question topic affects both where an answer is displayed and how users vote it; if that premise fails, CVA's counterfactual quality estimates inherit confounding bias.

Editorial extensions

If this is right

  • Platforms that log vote trajectories, not just final vote tallies, can estimate answer quality and re-rank content without needing randomized position experiments.
  • Answers that received votes while ranked low, or votes against a prevailing majority, count more in CVA's quality estimate, so late and less-visible high-quality answers can surface.
  • The community-level coefficients $\beta$ and herding odds provide a quantitative map of how different StackExchange communities respond to rank and social conformity.
  • CVA outperforms CVP especially in communities with strong position bias because it models rank and prior votes jointly rather than separately.
  • The framework transfers to other platforms with binary helpfulness voting and vote history, such as product or service review sites.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors leave implicit a direct policy test: re-estimate CVA under two different platform ranking rules; quality estimates that stay stable across rules would confirm the adjustment is removing bias rather than fitting each rule's quirks.
  • A natural stress test for the ignorability assumption is to inject an unobserved confounder such as author reputation or question topic into the semi-synthetic generator and check whether CVA still recovers predefined quality; this would quantify the cost of the paper's stated limitation.
  • Because GPT-4o serves as the real-data ground truth, a subsample validation against human expert judgments would check whether the LLM proxy itself carries biases that CVA then appears to correct.
  • If platforms act on CVA rankings, they change the very display positions that the model treats as exogenous, so CVA would need to be re-estimated iteratively under the new ranking policy before deployment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes Counterfactual Voting Adjustment (CVA), a causal framework for estimating response quality from binary helpfulness votes in online Q&A platforms. The framework models individual votes with a logistic regression that includes answer quality, herding (positive vote ratio), relative length, and display rank, then produces a quality estimate by integrating the fitted vote probability over a population distribution of vote histories and ranks. The authors evaluate CVA on toy examples, a semi-synthetic dataset generated from the same model, and real StackExchange data using comment sentiment and GPT-4o helpfulness scores as quality proxies, and report that CVA rankings align better with these proxies than vote-difference rankings and CVP. They also provide a cross-community analysis of position-bias and herding-bias sensitivities across 120 communities.

Significance. If the central claim were established, CVA would be a practical tool for debiasing helpfulness votes in large Q&A platforms, and the cross-community analysis of 120 StackExchange communities is a useful descriptive contribution. The paper's strengths include the scale of the real-world evaluation, the interpretable parametric model with explicit bias terms, and the attempt to address both position and herding biases simultaneously. However, these strengths do not compensate for the issues below; the empirical evaluation does not isolate the purported causal adjustment, the semi-synthetic validation is circular, and the key identification assumption is implausible and untested.

major comments (5)
  1. [Section 3 / Appendix A, Eq. (5)] The CVA estimator integrates the fitted vote probability over the marginal population distribution P(m̃,d̃), not over the conditional distribution P(m̃,d̃|B). This is not the backdoor adjustment formula, and the claimed identification of counterfactual votes is not established. Furthermore, if P is the same population distribution for every answer (as suggested by 'across all questions'), then for each question the CVA estimate is a strictly increasing function of q_ij + ν_i L_ij, so the CVA ranking is identical to the ranking of the fitted linear predictor; the counterfactual integration cannot change the ranking. If P is instead answer-position-dependent, the integration can change rankings, but then the target described in Section 3 (fixed top rank, even vote split) is not what is computed. In neither case does the paper demonstrate that the causal adjustment is the active ingredient behind the improvements over voteDiff and CVP; a baseline consisting of the same logistic model without counterfactual averaging is missing.
  2. [Section 5.1 / Appendix D] The semi-synthetic experiment generates votes from exactly the model in Eq. (2) using coefficients learned from real data and then fits the same model. Under this protocol, recovering the predefined q_ij is a self-consistency check rather than independent evidence that CVA removes position or herding biases. A meaningful validation would generate data from a different process—e.g., with unobserved confounders, misspecified functional forms, or different mechanisms for rank and prior-vote assignment—to test the model's robustness.
  3. [Section 3, Eq. (1)] The ignorability condition V_ij^t(g) ⊥ M_ij^t, D_ij^t | B_ij^t is load-bearing for the causal interpretation, yet B_ij^t contains only the relative length of the response, as the paper itself concedes. In StackExchange, factors such as answer author reputation, question topic, and user expertise plausibly affect both the answer's displayed rank/vote history and the individual vote, so the backdoor adjustment is likely biased. The manuscript provides no sensitivity analysis or evidence to support this assumption.
  4. [Appendix A] The claim that the estimator in Eq. (5) is optimal (satisfying counterfactual invariance while minimizing KL divergence to the original votes) is attributed to Theorem 1 of Wang et al. (2023), but the theorem is neither stated nor proved, and the relation of that theorem to Eq. (5) is not demonstrated. As written, the optimality claim is unsupported.
  5. [Section 5.2 / Appendix F] The two real-data quality proxies are not validated for this purpose. Comment sentiment may inherit the same position/herding biases as votes, and GPT-4o helpfulness scores are used as ground truth without any comparison to human judgments on this task; the cited evidence (Kamalloo & Rafiei, 2023) concerns open-domain question answering, not helpfulness evaluation. These proxies are too weak to support the conclusion that CVA rankings better match true quality.
minor comments (6)
  1. [Appendix A] The displayed equation after Eq. (5) uses R_ij^t and D_ij^t inside the sigmoid rather than the counterfactual R̃ and D̃; this is confusing and should be corrected.
  2. [Section 4.1] The sentence 'We trained 4 times using the data before T each time' is unclear and does not explain how the quality estimates in Figure 3 are obtained.
  3. [Table 2] The criteria for the reported community statistics (#Questions, #Answers, #Votes, #Comments) and the significance test used for the p-values are not described; please specify the test and the unit of analysis.
  4. [Section 5] There is a typo 'StackExchagne' that should read 'StackExchange'.
  5. [Figure 1] The caption contains subjective, unsupported interpretations (e.g., 'bitcoin emerges as a kind of new language') that are not corroborated by any quantitative analysis.
  6. [Section 3 / Reproducibility] The code and implementation details for fitting the model and computing the integral in Eq. (5) are not provided, making the experiments difficult to reproduce.

Circularity Check

3 steps flagged · score 6.0 of 10

Semi-synthetic validation is a closed-loop self-consistency check; the estimator's 'optimality' is imported from a co-authored theorem; and the counterfactual averaging is monotone in the fitted logistic predictor, so the causal adjustment is not the tested active ingredient.

  1. fitted input called prediction [Section 5.1 (Semi-synthetic experiment) and Appendix D (Semi-synthetic data generation steps)]
    "We generate a semi-synthetic dataset using the proposed model, with detailed data generation steps provided in Appendix D. Using the dataset's ground truth qualities, we assess whether our model can recover a ranking closely aligned with the ground truth quality-based ranking. — At last, we decide to vote positive or negative using the proposed model and the learned coefficients from real data."

    Section 5.1 evaluates recovery of 'ground truth qualities' that Appendix D plants: true q is sampled from a normal distribution and votes are generated by the proposed model itself ('we decide to vote positive or negative using the proposed model and the learned coefficients from real data'), i.e., by the same logistic form as Eq. (2). Re-fitting Eq. (2) to data produced by Eq. (2) and observing that the fitted q tracks the planted q is a maximum-likelihood self-consistency check; under identifiability the recovery is guaranteed by construction. It provides no evidence that Eq. (2)'s position/herding structure matches real voting, since no alternative bias structure is tested.

  2. uniqueness imported from authors [Section 3 (Interactive voting as causal inference) and Appendix A]
    "It turns out this estimator is the optimal estimator that satisfies counterfactual invariance to existing voting distribution and display rank while minimizing the KL divergence to the original votes under the product distribution. — The optimality of predicted response quality is due to Theorem 1 of (Wang et al., 2023), while following the same distribution what presentation configurations ˜mt_ij, ˜dt_ij the response was assigned to."

    The estimator's optimality is attributed entirely to 'Theorem 1 of (Wang et al., 2023)', a paper co-authored by this paper's co-author Yixin Wang. No proof appears in this manuscript, and no verification that the theorem's conditions hold for the voting setting is given; the KL-divergence claim is asserted in a single sentence and the Appendix sentence is garbled ('while following the same distribution what presentation configurations ...'). The choice of CVA is thereby declared forced by a theorem imported from the authors' own prior work as though it were an external mathematical fact, rather than established here.

1 more flagged steps
  1. self definitional [Section 3 (Eq. 5), Appendix A, and Abstract]
    "We finally produce the response quality estimate: ˆQt_ij ∆= Σ_{t=1}^T ∫ E[V t_ij(˜gt_ij)|Bt_ij] × P ( ˜mt_ij, ˜dt_ij)d ˜mt_ij d ˜dt_ij — ∫ E[V t_ij(˜gt_ij)|Bt_ij]P ( ˜mt_ij, ˜dt_ij)d ˜mt_ij d ˜dt_ij = E_{R̃t_ij, L̃t_i, D̃t_ij}[sigmoid(qt_ij + λRt_ij + νiLt_ij + β( 1/(1 +Dt_ij)))]"

    The CVA score is Σ_t E_{R̃,D̃}[sigmoid(c_ij + λR̃ + β/(1+D̃))] with c_ij = q_ij + ν_i L_ij, an expectation of the Eq. (2) logistic probability under the population measure P(m̃,d̃). Since sigmoid is strictly increasing and P is a fixed population distribution across answers (Appendix A), every summand — hence the total score — is strictly increasing in the fitted linear predictor c_ij. The counterfactual averaging therefore cannot reorder answers within a question: the CVA ranking equals the ranking of the fitted logistic model's predictor (up to unanalyzed j-dependence of P). No baseline of the same logistic model without the averaging is reported, so the abstract's claim that CVA 'outperform[s] ...

full rationale

The derivation chain is: Eq. (1) ignorability (V ⊥ M,D | B) with B containing only relative length L; Eq. (2), a logistic voting model 'following Lee et al. (2016)' (Moontae Lee co-authors both papers), transparently posited rather than derived; the CVA estimator in Eq. (5)/Appendix A, defined as a backdoor-style integral of the Eq. (2) probability under a population distribution; and validation via toy examples, a semi-synthetic set, and real StackExchange data with GPT-4o and comment-sentiment proxies. Three steps reduce to inputs by construction. (1) The semi-synthetic experiment is closed-loop: votes are generated by the proposed model with planted quality q, so recovering q is maximum-likelihood inversion of the paper's own generative process, not independent evidence that the model removes real position/herding biases. (2) The 'optimal estimator' status is imported from Theorem 1 of Wang et al. (2023), co-authored by Yixin Wang, with no proof or condition check here; the paper declares its estimator forced by that theorem. (3) The CVA score is a sum of expectations of the strictly increasing sigmoid under a fixed population measure, hence a strictly increasing function of the fitted linear predictor q_ij + ν_i L_ij; the counterfactual averaging is order-preserving and cannot itself change rankings (up to the unanalyzed j-dependence of the integration measure). Reported gains over voteDiff and CVP therefore conflate adding position/herding/length regressors to the vote model with the causal adjustment, and no same-model-without-integration baseline is reported. What is not circular: the real-data evaluation against comment sentiment and GPT-4o is external (those proxies are not derived from Eq. (2)), and CVA rankings align with them better than voteDiff and CVP in most of the 120 communities; the paper also states its limitations honestly (ignorability 'is one of our limitations' and 'all relevant confounders are included' is assumed); and the toy experiments merely confirm the fitted model behaves as its functional form prescribes. Because a genuine external benchmark remains, the circularity is partial: the headline recovery claim reduces by construction, but the paper retains independent real-data content, yielding a score of 6.

Assumptions & free parameters 5 free parameters · 9 assumptions · 0 invented entities

No new physical or structural entities are introduced. The latent perceived quality Q in Figure 2 is the target of inference, not an independent postulated entity. The main ledger items are the per-answer quality intercepts and bias coefficients fitted to the vote data, plus the ignorability and model-form assumptions that make the counterfactual estimates identified. The most consequential entries are the identical generative model in the semi-synthetic validation (circularity) and the common integration distribution that makes the adjustment rank-preserving.

free parameters (5)
  • Per-answer quality intercept q_ij = One value per answer; individual values are not reported.
    The central target: the answer-specific intercept in Eq. (2), fit to observed votes. In semi-synthetic experiments this is the predefined quality to recover.
  • Community-level herding coefficient lambda = One value per community; numerical values are not reported in the main text.
    Coefficient of the positive vote ratio R in Eq. (2); captures herding bias.
  • Community-level position bias coefficient beta = One value per community; reported as coordinates in Figure 1.
    Coefficient of 1/(1+D) in Eq. (2); captures position bias.
  • Question-level length coefficient nu_i = One value per question; not reported.
    Coefficient of relative length L in Eq. (2).
  • L2 regularization weight = Not reported; the objective uses an unspecified multiplier.
    The objective includes -1/2 ||theta||_2^2 but the multiplier is not reported; chosen by hand.
assumptions (9)
  • domain assumption The ignorability condition in Eq. (1) holds, and the feature set B contains only the relative length of the response.
    Required for backdoor adjustment; the paper states 'all relevant confounders are included' as a limitation in the Conclusion.
  • standard math Standard causal inference assumptions (positivity and SUTVA) hold for the voting data.
    Invoked in Section 3 as standard assumptions for causal inference, but not verified.
  • ad hoc to paper The true voting probability follows the Bernoulli-sigmoid model in Eq. (2) with additive quality, herding, length, and position terms.
    The functional form is posited following Lee et al. (2016) and not derived; if misspecified, quality estimates are biased.
  • domain assumption The causal graph in Figure 2 is correct, and conditioning on B blocks all backdoor paths between treatments and outcome.
    The graph justifies ignorability; the paper says 'conditioning on B will block all backdoor paths', which is an assumption.
  • domain assumption A user who selects a response always reads and votes on it, so there are no read-without-vote events.
    Stated in Section 3 and acknowledged as a limitation in the Conclusion; unread answers are excluded.
  • ad hoc to paper The first vote on each answer is arbitrary and can be excluded from training.
    Section 4.1: 'we dropped the first vote of each answer from the training data as it is an arbitrary vote given no context in our setting.'
  • domain assumption The population distribution of prior votes and ranks in Eq. (5) is common across answers and is estimated from the observed data.
    Used in Eq. (5); because it is common, the counterfactual adjustment is a monotone transform applied equally to all answers, preserving ranking.
  • domain assumption GPT-4o provides a reliable proxy for human judgments of helpfulness and comment sentiment.
    Section 5.2 treats GPT-4o scores as ground truth, supported only by a QA benchmark citation, not by helpfulness-label validation.
  • ad hoc to paper The semi-synthetic data is generated by the same model that is later fit, so recovery of simulated quality is a self-consistency check.
    Appendix D generates votes using the proposed model and learned coefficients from real data, making the recovery circular.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Counterfactual Voting Adjustment for Quality Assessment and Fairer Voting in Online Platforms with Helpfulness Evaluation." pith.science (2026). https://pith.science/paper/BOMSB7DG

@misc{pith2026250621362,
  author       = {Pith},
  title        = {Pith review of: Counterfactual Voting Adjustment for Quality Assessment and Fairer Voting in Online Platforms with Helpfulness Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BOMSB7DG}},
  note         = {Machine review of arXiv:2506.21362}
}
read the original abstract

Efficient access to high-quality information is vital for online platforms. To promote more useful information, users not only create new content but also evaluate existing content, often through helpfulness voting. Although aggregated votes help service providers rank their user content, these votes are often biased by disparate accessibility per position and the cascaded influence of prior votes. For a fairer assessment of information quality, we propose the Counterfactual Voting Adjustment (CVA), a causal framework that accounts for the context in which individual votes are cast. Through preliminary and semi-synthetic experiments, we show that CVA effectively models the position and herding biases, accurately recovering the predefined content quality. In a real experiment, we demonstrate that reranking content based on the learned quality by CVA exhibits stronger alignment with both user sentiment and quality evaluation assessed by GPT-4o, outperforming system rankings based on aggregated votes and model-based rerankings without causal inference. Beyond the individual quality inference, our embeddings offer comparative insights into the behavioral dynamics of expert user groups across 120 major StackExchange communities.

Figures

Figures reproduced from arXiv: 2506.21362 by the authors.

Figure 1
Figure 1. Analyses of different voting dynamics across the 120 largest communities on StackExchange. First, math-oriented [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. A causal graph for interactive voting that meets [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Preliminary Experiment Results: Subfigures (a) and (b) demonstrate that CVA mitigates position bias by upweighting the votes received at a lower-ranked position. By time step 6, the learned quality of answer B is more positive than that of answer A in (a), and answer B’s learned quality is more negative than answer A’s in (b). Subfigure (c) illustrates that CVA mitigates Herding bias by upweighting votes that go aga… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Validation on semi-synthetic data (Politics community): Dots represent answers. X-axis: true quality rank; Y-axis: rank by vote difference score (left), CVP learned quality (middle), or CVA learned quality (right). Closer alignment to the diagonal indicates better perf…
Figure 5
Figure 5. Figure 5: Validation on real data (Politics community): Dots represent answers. X-axis (first row): comment sentiment as true quality; X-axis (second row): GPT estimated helpfulness score as true quality; Y-axis: similar to [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Predict positive vote probabilities given different [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Voting Biases in Decentralized Autonomous Organization (DAO) Governance

    cs.CY 2026-07 conditional novelty 6.0 of 10

    Author-selected Snapshot choices show a 58.8 pp higher voting-power share than non-author choices, exceeding approval (27.1 pp) and first-list (7.7 pp) associations.

Reference graph

Works this paper leans on

40 extracted references · 38 canonical work pages · cited by 1 Pith paper

  1. [1]

    https://stackexchange.com/sites

    A ll S ites - S tack E xchange --- stackexchange.com. https://stackexchange.com/sites. [Accessed 03-06-2025]

    work page 2025

  2. [2]

    Is review visibility fostering helpful votes? the role of review rank and review characteristics in the adoption of information

    Alzate Barricarte, M., Arce Urriza, M., and Cebollada Calvo, J. Is review visibility fostering helpful votes? the role of review rank and review characteristics in the adoption of information. Computers in Human Behavior, 2024

    work page 2024

  3. [3]

    Babi \'c Rosario, A., Sotgiu, F., De Valck, K., and Bijmolt, T. H. The effect of electronic word of mouth on sales: A meta-analytic review of platform, product, and metric factors. Journal of Marketing Research, 53 0 (3): 0 297--318, 2016

    work page 2016

  4. [4]

    and Lee, H.-y

    Chiang, C.-H. and Lee, H.-y. Can large language models be an alternative to human evaluations? arXiv preprint arXiv:2305.01937, 2023

    arXiv 2023

  5. [5]

    Vote or not? how various information cues affect helpfulness voting of online reviews

    Deng, W., Yi, M., and Lu, Y. Vote or not? how various information cues affect helpfulness voting of online reviews. Online Information Review, 44 0 (4): 0 787--803, 2020

    work page 2020

  6. [6]

    Quantifying voter biases in online platforms: An instrumental variable approach

    Dev, H., Karahalios, K., and Sundaram, H. Quantifying voter biases in online platforms: An instrumental variable approach. ACM on Human-Computer Interaction, 3 0 (CSCW): 0 1--27, 2019

    work page 2019

  7. [7]

    Neighbor-aware review helpfulness prediction

    Du, J., Rong, J., Wang, H., and Zhang, Y. Neighbor-aware review helpfulness prediction. Decision Support Systems, 148: 0 113581, 2021

    work page 2021

  8. [8]

    Conformity feedback in an online review helpfulness evaluation task leads to less negative feedback-related negativity amplitudes and more positive p300 amplitudes

    Guo, D., Zhao, Y., Zhang, L., Wen, X., and Yin, C. Conformity feedback in an online review helpfulness evaluation task leads to less negative feedback-related negativity amplitudes and more positive p300 amplitudes. Journal of Neuroscience, Psychology, and Economics, 12 0 (2): 0 73, 2019

    work page 2019

Show all 40 references
  1. [9]

    Imbens, G. W. and Rubin, D. B. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press, 2015

    2015

  2. [10]

    Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search

    Joachims, T., Granka, L., Pan, B., Hembrooke, H., Radlinski, F., and Gay, G. Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search. ACM Transactions on Information Systems (TOIS), 25 0 (2): 0 7--es, 2007

    2007

  3. [11]

    and Rafiei

    Kamalloo, Dziri, C. and Rafiei. Evaluating open-domain question answering in the era of large language models. Association for Computational Linguistics, 1 0 (Long Papers): 0 5591--5606, 2023

    2023

  4. [12]

    M., and Imran, A

    Kastrati, Z., Fatima, S., Kurti, A., Daudpota, S. M., and Imran, A. S. Analyzing and predicting the helpfulness of reviews in moocs context using deep learning. Procedia Computer Science, 246: 0 772--781, 2024

    2024

  5. [13]

    Breaking feedback loops in recommender systems with causal inference

    Krauth, K., Wang, Y., and Jordan, M. Breaking feedback loops in recommender systems with causal inference. ACM Transactions on Recommender Systems, 2022

    2022

  6. [14]

    K., Hui, K.-L., Prasarnphanich, P., and Lai, H.-Y

    Kuan, K. K., Hui, K.-L., Prasarnphanich, P., and Lai, H.-Y. What makes a review voted? an empirical investigation of review voting in online review systems. Journal of the Association for Information Systems, 16 0 (1): 0 1, 2015

    2015

  7. [15]

    H., and Mimno, D

    Lee, M., Jin, S. H., and Mimno, D. Beyond exchangeability: The chinese voting process. Advances in Neural Information Processing Systems, 29: 0 4934--4942, 2016

    2016

  8. [16]

    Artificial intelligence for hospitality big data analytics: developing a prediction model of restaurant review helpfulness for customer decision-making

    Lee, M., Kwon, W., and Back, K.-J. Artificial intelligence for hospitality big data analytics: developing a prediction model of restaurant review helpfulness for customer decision-making. International Journal of Contemporary Hospitality Management, 33 0 (6): 0 2117--2136, 2021

    2021

  9. [17]

    Liang, D., Charlin, L., and Blei, D. M. Causal inference for recommendation. In Causation: Foundation to Application, Workshop at UAI. AUAI, 2016 a

    2016

  10. [18]

    Liang, D., Charlin, L., McInerney, J., and Blei, D. M. Modeling user exposure in recommendation. In International Conference on World Wide Web, pp.\ 951--961, 2016 b

    2016

  11. [19]

    Low-quality product review detection in opinion summarization

    Liu, J., Cao, Y., Lin, C.-Y., Huang, Y., and Zhou, M. Low-quality product review detection in opinion summarization. In Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pp.\ 334--342, 2007

    2007

  12. [20]

    Anticipating performativity by predicting from predictions

    Mendler-D \"u nner, C., Ding, F., and Wang, Y. Anticipating performativity by predicting from predictions. Advances in Neural Information Processing Systems, 35: 0 31171--31185, 2022

    2022

  13. [21]

    and Jenamani, M

    Mitra, S. and Jenamani, M. Helpfulness of online consumer reviews: A multi-perspective approach. Information Processing & Management, 58 0 (3): 0 102538, 2021

    2021

  14. [22]

    Local consumer review survey 2017, Jun 2017

    Murphy, R. Local consumer review survey 2017, Jun 2017. URL https://www.brightlocal.com/learn/local-consumer-review-survey-2017/

    2017

  15. [23]

    Comparative analysis of deep learning models for predicting online review helpfulness

    Palahan, S. Comparative analysis of deep learning models for predicting online review helpfulness. In Asia Conference on Computer Vision, Image Processing and Pattern Recognition, pp.\ 1--5, 2023

    2023

  16. [24]

    Causal inference

    Pearl, J. Causal inference. Causality: objectives and assessment, pp.\ 39--58, 2010

    2010

  17. [25]

    The impact of social influence on the perceived helpfulness of online consumer reviews

    Risselada, H., de Vries, L., and Verstappen, M. The impact of social influence on the perceived helpfulness of online consumer reviews. European Journal of Marketing, 2018

    2018

  18. [26]

    Recommendations as treatments: Debiasing learning and evaluation

    Schnabel, T., Swaminathan, A., Singh, A., Chandak, N., and Joachims, T. Recommendations as treatments: Debiasing learning and evaluation. In International Conference on Machine Learning, pp.\ 1670--1679. PMLR, 2016

    2016

  19. [27]

    P., Singh, L., and Tiwari, R

    Sharma, S. P., Singh, L., and Tiwari, R. Prediction of customer review's helpfulness based on feature engineering driven deep learning model. International Journal of Software Innovation (IJSI), 11 0 (1): 0 1--16, 2023

    2023

  20. [28]

    Was this review helpful to you? it depends! context and voting patterns in online content

    Sipos, R., Ghosh, A., and Joachims, T. Was this review helpful to you? it depends! context and voting patterns in online content. In International Conference on World Wide Web, pp.\ 337--348, 2014

    2014

  21. [29]

    Beyond mean rating: Probabilistic aggregation of star ratings based on helpfulness

    Tay, W., Zhang, X., and Karimi, S. Beyond mean rating: Probabilistic aggregation of star ratings based on helpfulness. Journal of the Association for Information Science and Technology, 71 0 (7): 0 784--799, 2020

    2020

  22. [30]

    How product review voting is influenced by existing votes, consumer involvement, review valence, and review diagnosticity

    Tseng, S.-L., Lu, S., Weathers, D., and Grover, V. How product review voting is influenced by existing votes, consumer involvement, review valence, and review diagnosticity. Decision Support Systems, 172: 0 113981, 2023

    2023

  23. [31]

    The matthew effect in social commerce: the case of online review helpfulness

    Wan, Y. The matthew effect in social commerce: the case of online review helpfulness. Electronic Markets, 25: 0 313--324, 2015

    2015

  24. [32]

    Wang, G., Gill, K., Mohanlal, M., Zheng, H., and Zhao, B. Y. Wisdom in the social crowd: an analysis of quora. In International Conference on World Wide Web, pp.\ 1341--1352, 2013

    2013

  25. [33]

    Can online user reviews be more helpful? evaluating and improving ranking approaches

    Wang, J.-N., Du, J., and Chiu, Y.-L. Can online user reviews be more helpful? evaluating and improving ranking approaches. Information & Management, 57 0 (8): 0 103281, 2020 a

    2020

  26. [34]

    Wang, Y., Liang, D., Charlin, L., and Blei, D. M. Causal inference for recommender systems. In ACM Conference on Recommender Systems, pp.\ 426--431, 2020 b

    2020

  27. [35]

    Adjusting machine learning decisions for equal opportunity and counterfactual fairness

    Wang, Y., Sridhar, D., and Blei, D. Adjusting machine learning decisions for equal opportunity and counterfactual fairness. Transactions on Machine Learning Research, 2023

    2023

  28. [36]

    Explaining and predicting helpfulness and funniness of online reviews on the steam platform

    Wang, Z., Chang, V., and Horvath, G. Explaining and predicting helpfulness and funniness of online reviews on the steam platform. Journal of Global Information Management (JGIM), 29 0 (6): 0 1--23, 2021

    2021

  29. [37]

    The effect of content depth and deviation on online review helpfulness: Evidence from double-hurdle model

    Wu, C., Mai, F., and Li, X. The effect of content depth and deviation on online review helpfulness: Evidence from double-hurdle model. Information & Management, 58 0 (2): 0 103408, 2021

    2021

  30. [38]

    Deep semi-supervised learning with weight map for review helpfulness prediction

    Yin, H., Hu, Z., Peng, Y., Wang, Z., Xu, G., and Xu, Y. Deep semi-supervised learning with weight map for review helpfulness prediction. Computer Science and Information Systems, 18 0 (4): 0 1159--1174, 2021

    2021

  31. [39]

    and Guo, B

    Zhou, S. and Guo, B. The order effect on online review helpfulness: A social influence perspective. Decision Support Systems, 93: 0 77--87, 2017

    2017

  32. [40]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.