{"id":"6ae03cdf-df49-4aa6-822b-1b8037f41039","arxiv_id":"1908.08298","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A reaction-weighted, topic-sensitive PageRank method identifies influential users for word-of-mouth marketing in Facebook groups, and a reinforced scheme selects multiple authorities from each subgroup.","lead":"This paper uses reactions inside Facebook groups to rank members by influence, so businesses can pick a few trusted people to promote products. It also adds a 'reinforced' option where several influential members from the same subgroup promote together.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation is partly circular: the §5 graph and the §8.2 votes metric are both computed from the same reaction counts, so PageRank's correlation is not independent evidence of marketing influence; the §8.4 manual labels are external but measure perceived influence, not persuasion or purchase…","rationale":"The reader's weakest assumption is essentially the one I find load-bearing: marketing influence is operationalized as weighted reactions, and the same operationalization appears on both sides of the evaluation. I agree with the reader's CONDITIONAL judgment. The PageRank-versus-baseline comparison is internally coherent, and the manual labels in §8.4 are a real external signal, so this is not a reason to reject outright; it is a reason to demand an outcome-based or at least temporally separated validation before accepting the marketing claim. No internal inconsistency was found; the concern is about external validity and evidence independence. Therefore the verdict should remain CONDITIONAL.","tokens_in":9996,"tokens_out":7051,"duration_ms":81276,"concrete_test":"Run a randomized field experiment in one technical Facebook group: select 20 top-PageRank users and 20 control users matched on prior activity, have both groups publish the same promotional post with unique tracking links or promotion codes in the same week, and compare click-through or conversion rates. If the top-PageRank group does not significantly outperform controls, the reaction-based authority measure fails as a marketing-influence test.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that PageRank on a reaction-weighted graph identifies users who are influential for word-of-mouth marketing. The weakest point is not internal inconsistency but the independence of the evidence. In §5, the graph edge weight is the product of a reaction-type weight (like-on-comment 1, like 2, comment 4, share 8) and boosted topical relevance. In §8.2, the 'votes' and 'topical votes' baseline metrics are weighted sums of the same reactions received by a user, with the same reaction-type weights. Hence the validation signal used in §8.3 is almost the same quantity that constructs the graph; PageRank correlates with votes partly because votes are a near-linear function of the graph's edge weights (weighted in-degree). This does not test whether those users can persuade anyone or generate sales. The manual labels in §8.4 are genuinely independent and give some support, but they are judgments by five students of 'influential' users made from the same posts and reaction counts, so they can inherit the same visibility bias and do not measure actual word-of-mouth effectiveness. The paper should therefore be read as evidence about predicting reaction-weighted popularity, not yet about marketing outcomes.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a method for word-of-mouth marketing in Facebook groups. It constructs a topic-sensitive social interaction graph from user posts and reactions, assigns edge weights based on reaction type and boosted topical relevance, and ranks users with PageRank. The method is compared against HITS, Z-score, eigenvector centrality, betweenness, and closeness using Pearson correlation with votes and topical votes, and with MAP/NDCG against manual labels from five students. The paper reports that PageRank outperforms the other measures, and also sketches a reinforced marketing strategy based on weakly connected components, analyzes influence across topics and time, and recommends marketing periods. The main claimed contribution is that PageRank identifies influential users suitable for word-of-mouth marketing in online social groups.","tokens_in":1735,"tokens_out":1891,"duration_ms":51716,"significance":"If the central claim holds, the paper offers a practical and scalable way to identify potential word-of-mouth marketers in focused Facebook groups using only publicly observable reactions, which would be useful for marketing practitioners and social-media analysts. Strengths include the large dataset (100 groups, 0.3 million posts, 10 million reactions), a clearly described topic-sensitivity mechanism, and a systematic comparison across six ranking algorithms. The manual-label evaluation in Section 8.4 is genuinely independent of the graph construction and gives some support for the plausibility of PageRank. However, as detailed below, the primary validation evidence is partly circular because the same reaction data is used to build the graph and to define the popularity baselines, and the external validation is too small and underspecified to fully carry the word-of-mouth effectiveness claim. The paper is best read as evidence about predicting reaction-weighted popularity rather than about persuasion or purchase influence.","major_comments":[{"comment":"The main evaluation is partly circular. In Section 5.1, graph edge weights are computed as the product of a reaction-type weight (like on comment=1, like=2, comment=4, share=8) and the boosted topical relevance. In Section 8.2, the baseline 'votes' and 'topical votes' are weighted sums of the same reaction types over the same posts and comments, using the same reaction weights. Thus votes are a near-linear function of a user's weighted indegree, and PageRank is computed on a graph whose edge weights are built from the same quantities. The high correlations in Figures 4 and 5 therefore partly measure self-consistency rather than external marketing influence. To support the claim, the authors should compare against a direct weighted-indegree baseline or total-votes baseline, and should clearly state that the evidence concerns reaction-weighted popularity, not persuasion or purchase behavior.","section":"§5.1 vs. §8.2–8.3"},{"comment":"The only non-circular validation is the manual labeling experiment, but it is too thin to carry the central claim. Five students from the authors' own research lab judged whether users are influential, and they were given the posts and reactions of those users. The labels can therefore inherit the same visibility and popularity bias that the reaction-based metrics already encode, and they measure perceived influence rather than actual word-of-mouth effectiveness. No error bars, confidence intervals, or significance tests are reported for the MAP/NDCG values in Table 1, and the 92% agreement figure is not accompanied by a chance-adjusted measure such as Cohen's kappa. The authors should report per-group variability, inter-annotator agreement statistics, and a precise definition of the judgment task.","section":"§8.4"},{"comment":"The reinforced marketing contribution is described but never empirically evaluated. The algorithm depends on the subgroup threshold th and the reinforcement parameter r, but no values, experiments, or sensitivity analyses are provided, and no results show that selecting r top users per weakly connected component improves influence or marketing outcomes. Since reinforced marketing is presented as a key contribution in the abstract and in Problem 3, the authors should either provide an evaluation or clearly state that this part is a proposal rather than a validated result.","section":"§7"},{"comment":"The method has several free parameters whose robustness is not examined: the boost factor alpha (set to 20), the reaction-type weights 1, 2, 4, 8, the PageRank damping factor and iteration count, and the subgroup threshold th. The reported superiority of PageRank in Table 1 and Figures 4–5 could depend on these choices. A sensitivity analysis, or at least a justification for the chosen values, is needed before the claim that PageRank is generally the best authority measure for this task can be accepted.","section":"§5.1, §6, §8.1"}],"minor_comments":[{"comment":"The abstract says '0.3 million messages' while Section 8.1 says '0.3 million posts'; please make the terminology consistent.","section":"Abstract and §8.1"},{"comment":"The description says Pearson correlation is used on ranks, but Pearson is not a rank correlation unless explicitly applied to ranks; please clarify whether raw scores or rank-transformed scores were correlated, and consider reporting Spearman correlation.","section":"§8.3"},{"comment":"Figures 4 and 5 lack axis labels, error bars, and information about the number of groups used for averaging; adding these would make the comparisons interpretable.","section":"§8.2–8.3"},{"comment":"The Semantic Link system is cited only by a URL; please provide a version, a description of its training data, and a statement about how mutual-information scores are normalized across words and posts.","section":"§5.1"},{"comment":"The manual labeling procedure is described too briefly: the paper should state how the students were instructed to define 'influential', how many users each student labeled per group, and whether the labels were collected independently of the ranking algorithms.","section":"§8.4"},{"comment":"The correlations in Table 2 are reported without sample sizes or significance levels, so the claim that high top-20 correlation indicates multi-topic influence is not statistically supported.","section":"§8.5"},{"comment":"The statistics '6.5% users generate 85% of content' and 'less than 2% influence 80% of the population' are asserted without a clear definition of 'influence' or a description of how the 80% figure is computed; please clarify.","section":"§2 and §8.6"},{"comment":"No data or code availability statement is provided. Releasing the dataset (at least in anonymized form) and the ranking implementation would substantially improve reproducibility and allow independent verification of the reported results.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The central evaluation issue is the circularity between graph construction and the popularity baselines; the manual-label experiment provides some independent evidence but is too small and potentially biased. The paper's contributions are clear and the topic is suitable for the journal, but the authors should be asked to reframe their claims as predicting reaction-weighted popularity or to add outcome-based validation, and to strengthen the manual evaluation. A major revision addressing the validation and sensitivity issues is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: this is a straightforward application of known authority-ranking algorithms to Facebook groups, with a topical-relevance twist and a reinforced-marketing selection rule. PageRank ends up looking best, but the headline evidence for that is partly circular, and the only independent check is too weak to carry the marketing claim. Still, there is real work here worth engaging with.\n\nWhat is actually new: the topic-boosted edge weighting (relevance score times reaction type) and the reinforced-marketing component-selection rule are reasonable, simple extensions of standard ideas. The authors also provide a useful descriptive analysis of Facebook group structure—bow-tie components, degree distribution, posting and reaction dynamics over time—on a large dataset (0.3M posts, 10M reactions). The comparison of PageRank against five other measures on this type of graph is useful, even if the evaluation is flawed.\n\nThe main soft spot is the circularity. In Section 5 the graph is built from likes, comments, shares, and likes-on-comments, weighted 1, 2, 4, 8. In Section 8.2 the \"votes\" and \"topical votes\" baselines are weighted sums of those same reactions received by a user, using the same weights. So the correlation in Figures 4 and 5 is largely a self-consistency check: PageRank on a reaction-weighted graph correlates with weighted in-degree. That does not test whether any of these users can persuade anyone or generate sales. The manual labels in Section 8.4 are genuinely independent, but they come from five lab students judging \"influential\" from the same posts and reaction counts, so they can inherit the same visibility bias and say nothing about actual word-of-mouth effectiveness.\n\nThe other issues are more minor but real: no code or data released, hand-set parameters (alpha, reaction weights, subgroup threshold, PageRank damping) with no sensitivity analysis, and no significance tests or error bars on the MAP/NDCG numbers. The semantic relevance step relies on an external tool without any evaluation of the relevance scores themselves.\n\nWhat the paper does well: the framing of marketing as needing topic-specific authorities is sensible, the reinforced-marketing idea, while simple, is a useful practical notion, and the descriptive findings about concentration of content production and activity peaks are interesting.\n\nWho is this for? Readers working on influence-maximization applications for social media platforms, especially practitioners who want a cheap way to pick targets in focused groups. It is not a rigorous scientific demonstration of influence.\n\nBottom line: the paper deserves a serious referee, but it should not be accepted in its current form. The authors need to either validate against an external outcome (purchases, clicks, follow-through) or at least show that the reaction-based \"votes\" correlate with something independent of the graph construction. As is, I would treat it as a conditional paper, not a reject.","headline":"Plausible but under-validated application of PageRank to Facebook groups; the main evidence is partly circular and the independent check is thin, but the dataset and descriptive analysis give it enough substance to deserve a conditional referee.","tokens_in":10745,"tokens_out":1854,"would_cite":false,"duration_ms":21576,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A topic-sensitive PageRank on a reaction-weighted social interaction graph identifies the most influential Facebook group members for word-of-mouth marketing, beating HITS, Z-score, eigenvector, betweenness, and closeness.","keywords":["word-of-mouth marketing","influential users","PageRank","Facebook groups","topic-sensitive ranking","social interaction graph","reinforced marketing","centrality comparison"],"falsifier":"Give matched groups of Facebook members the same product recommendation from a top-PageRank user and from a randomly selected low-rank user and measure actual sign-ups or purchases; if the low-rank user converts as well as the top-ranked one, the claim that PageRank identifies influential marketers for word-of-mouth is falsified.","tokens_in":9837,"feed_emoji":"📣","tokens_out":8227,"duration_ms":72789,"temperature":0.7,"pith_summary":"Marketing on social media often fails because users distrust ads; word-of-mouth from a trusted peer works better. This paper argues that in focused Facebook groups, the right people to recruit for word-of-mouth marketing are the authorities who already attract most reactions, and that a topic-sensitive PageRank on a graph of user reactions finds them reliably. The authors build a social interaction graph whose edge weights combine the type of reaction (like on comment, like, comment, share) with how semantically relevant the post is to the advertised topic, then rank members by PageRank. On 100 groups with 0.3 million posts and 10 million reactions, this outranks HITS, Z-score, eigenvector, betweenness, and closeness in matching human judgments (MAP 0.91, NDCG 0.83). They also propose reinforced marketing, where several authorities in the same subgroup promote a product together, and identify the best months to launch.","feed_headline":"PageRank beats five rivals at finding Facebook influencers","feed_subtitle":"Topic-sensitive ranking on reaction-weighted social graphs matches human judgment for word-of-mouth targets.","key_machinery":"The central object is the topic-sensitive social interaction graph: vertices are group members, and each directed edge from one user to another records that the first user reacted to content created by the second. Edge weight is the product of a reaction-type weight (1, 2, 4, 8 for like on comment, like, comment, share) and a boosted relevance that grows logarithmically with the summed mutual-information similarity between topic words and words in the post. PageRank runs on this weighted graph and propagates authority through the reaction structure, so a user ranks highly not merely by receiving many reactions but by receiving reactions from users who are themselves highly ranked. The same graph supplies sub-group detection via weakly connected components for reinforced marketing.","core_discovery":"The paper's central claim is that PageRank, applied to a topic-sensitive social interaction graph whose edges are weighted by reaction type and topical relevance, identifies the most influential users for word-of-mouth marketing in question-answer style Facebook groups. The authors treat a reaction from user A to user B's post or comment as an edge from A to B with weight equal to the reaction's assigned importance (1 for like on comment, 2 for like, 4 for comment, 8 for share) multiplied by a relevance boost derived from semantic similarity between the post and the advertised topic. Ranking group members by PageRank on this graph yields the top authorities, and the paper reports that this ranking agrees with human expert judgments better than HITS, Z-score, eigenvector centrality, betweenness, and closeness (MAP 0.91, NDCG 0.83). The same framework supports reinforced marketing, in which each selected authority is backed by at least r minus 1 other authorities in the same weakly connected subgroup, so repeated trusted messages reinforce the product.","pith_inferences":["A testable extension: the same graph construction could be applied to other question-answer communities where reactions are observable, and the ranking validated against sign-ups or purchases rather than reactions.","The fixed reaction weights (1,2,4,8) could be treated as free parameters and fit to outcome data; the ranking's sensitivity to those weights is not examined in the paper.","If the mechanism generalizes, marketers could combine reinforced marketing with temporal analysis to schedule launches in periods when top users are already most active."],"forward_implications":["An advertiser can rank the top-k potential promoters in a large topic-focused Facebook group from public reaction data alone, without surveying members.","Reinforced marketing gives a selection rule: for each weakly connected subgroup that is large enough, pick its top-r authorities so that every chosen promoter is backed by at least r-1 other trusted voices within that subgroup.","Because top users rank highly across related topics, the same authorities can promote a range of products in their group, not just a single topic.","Ranking quality degrades when the advertised topic is weakly related to the group's shared interest, so marketers should choose groups whose core topic matches the product.","Posting and reaction peaks in March, April, and October in technical groups suggest those months are the best launch windows, at least in the examined population."],"supporting_citations":[{"why":"Supplies the PageRank algorithm used to rank users on the reaction-weighted graph.","marker":"[16]"},{"why":"Provides the reaction-type weights (1, 2, 4, 8) used to weight edges.","marker":"[2]"},{"why":"Defines HITS, the main baseline authority measure that PageRank outperforms.","marker":"[13]"},{"why":"Defines the Z-score expertise measure used as another baseline.","marker":"[23]"},{"why":"Defines betweenness centrality, one of the centrality baselines compared.","marker":"[8]"},{"why":"Defines closeness centrality, one of the centrality baselines compared.","marker":"[7]"},{"why":"Defines eigenvector centrality, one of the centrality baselines compared.","marker":"[17]"},{"why":"Provides the NDCG metric used to evaluate ranking quality against human labels.","marker":"[11]"}],"fun_headline_variants":["PageRank tops 5 centrality methods for Facebook WoM targets","Reaction-weighted PageRank finds top Facebook influencers","Topic-sensitive PageRank wins influencer hunt on Facebook","PageRank beats HITS, betweenness, closeness on Facebook data","Reinforced marketing: PageRank picks multi-backer influencers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework assumes that the reactions a user receives—likes, comments, shares, and likes on comments—measure that user's real ability to persuade others to buy, because both the PageRank graph and the votes used to validate it are built from the same reactions.","fun_headline_variants_meta":{"raw":{"variants":["PageRank tops 5 centrality methods for Facebook WoM targets","Reaction-weighted PageRank finds top Facebook influencers","Topic-sensitive PageRank wins influencer hunt on Facebook","PageRank beats HITS, betweenness, closeness on Facebook data","Reinforced marketing: PageRank picks multi-backer influencers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000117,"raw_usage":{"total_tokens":1049,"prompt_tokens":885,"completion_tokens":164,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":501,"completion_tokens_details":{"reasoning_tokens":83}},"tokens_in":501,"tokens_out":164,"duration_ms":2323,"temperature":1.0,"reasoning_tokens":83,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:43:14.599388+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Give matched groups of Facebook members the same product recommendation from a top-PageRank user and from a randomly selected low-rank user and measure actual sign-ups or purchases; if the low-rank user converts as well as the top-ranked one, the claim that PageRank identifies influential marketers for word-of-mouth is falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the PageRank algorithm used to rank users on the reaction-weighted graph."},{"cited_title":"new media & society 14(7) (2012)","cited_arxiv_id":null,"evidence_quote":"Provides the reaction-type weights (1, 2, 4, 8) used to weight edges."},{"cited_title":"In: International Computing and Combinatorics Conference","cited_arxiv_id":null,"evidence_quote":"Defines HITS, the main baseline authority measure that PageRank outperforms."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Z-score expertise measure used as another baseline."},{"cited_title":"Sociometry (1977)","cited_arxiv_id":null,"evidence_quote":"Defines betweenness centrality, one of the centrality baselines compared."},{"cited_title":"Social net- works (1978)","cited_arxiv_id":null,"evidence_quote":"Defines closeness centrality, one of the centrality baselines compared."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines eigenvector centrality, one of the centrality baselines compared."},{"cited_title":"ACM Transactions on Information Systems (TOIS) 20(4) (2002)","cited_arxiv_id":null,"evidence_quote":"Provides the NDCG metric used to evaluate ranking quality against human labels."}],"review_version":1}