{"id":"71a1d527-fe05-45f4-ac98-cbaf9fa1b86a","arxiv_id":"2507.07919","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A recommender-system adaptation of LiCE generates counterfactual explanations that remove an item from the top-k list, with plausibility scored by category-level sum-product networks.","lead":"The authors adapt an existing method, LiCE, to generate counterfactual explanations for recommender systems, using mixed-integer optimization and sum-product networks. A user study and numerical tests on three public datasets assess how plausible the explanations are.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The plausibility claim rests on an unvalidated category-aggregated SPN proxy: generation and evaluation both use p(z), yet counterfactuals are item-level, so high category-level likelihood may not imply plausible item-level changes.","rationale":"The reader's weakest assumption is exactly this: category-aggregated SPN likelihood is used as a proxy for item-level plausibility, with no independent check. This is the most load-bearing concern because both the generation objective and the plausibility evaluation are driven by the same aggregated model; if the proxy is poor, the paper's main contribution—plausible counterfactuals—collapses, leaving only a valid-CE generator with a user study that does not favor the SPN. The concern is concrete and testable: train an item-level SPN and compare. Other issues (no baselines, only EASE, post hoc setup selection, small user study) are real but secondary; they affect generalizability and the 'state of the art' claim, while the aggregation issue affects the core definition of plausibility. The reader's CONDITIONAL verdict remains appropriate: the paper is coherent and reproducible, but this missing validation should be supplied before the plausibility claim is accepted.","tokens_in":13058,"tokens_out":3779,"duration_ms":42903,"concrete_test":"On a binarized public dataset (e.g., Yelp), train two SPNs: one on item-level vectors x (on a pruned item set for tractability) and one on the paper's category-aggregated vectors z. Run the category-SPN-based CE generation for 50 users. For each returned CE, compute both p_cat(z') and p_item(x'). Then: (i) measure the rank correlation between p_cat and p_item across CEs; (ii) compare the average p_item of generated CEs with the average p_item of equally close valid CEs obtained by random item-level mutations or by an item-level-SPN-guided search. If category-SPN CEs are not significantly more plausible at item level than random valid CEs of the same distance, the aggregation proxy fails and the plausibility claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that generated CEs are 'highly plausible with respect to the data distribution' depends on the SPN likelihood used in the MIO objective. However, Section IV-A defines the SPN on category-aggregated vectors z_i (sum, mean, or disjunction, Eqs. 7-11), not on the item-level interaction vectors x_i that the counterfactual actually changes. The aggregation is many-to-one: for disjunction, flipping any one item in a category yields the same z_i; for sum, replacing a popular item by an obscure item within the same category is invisible to the SPN. Consequently, the optimizer can select item-level changes that are indistinguishable at category level but differ greatly in item-level plausibility. The reported plausibility metric in Table II is the log-likelihood under this same aggregated SPN, so the evaluation is self-referential: it never checks whether p(z) tracks p(x). The user study indirectly aggravates the concern, since the no-SPN variant C is rated as good as or better than SPN variants. If p(z) does not track p(x), the method's core novelty over plain LiCE is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an extension of the LiCE method for generating counterfactual explanations (CEs) in recommender systems. The authors formulate CE generation as a mixed-integer optimization problem with a k-th rank constraint (Eqs. 5-6), use sum-product networks (SPNs) to model plausibility, and propose three category-level aggregation functions (sum, mean, disjunction) to make SPN training tractable. They compare three ways of using the SPN (optimize likelihood with weight alpha=0.1, constrain likelihood to be above a threshold, and no SPN) across three public datasets and one private dataset, report run-time and plausibility/similarity trade-offs (Tables I-II), and supplement the numerical evaluation with a user study on merchant recommendations (Section VI). The central claims are that the method reliably finds close, faithful counterfactuals without exhaustive search and that the generated CEs have high likelihood with respect to the data distribution.","tokens_in":13281,"tokens_out":5079,"duration_ms":64239,"significance":"If the main claims held, the paper would make a useful contribution: it provides an openly available, MIO-based CE generator tailored to recommender rankings, introduces a k-th rank validity constraint that is more faithful to top-k recommendation than simple score thresholds, and reports a user study with statistical tests. The open-source implementation and the explicit treatment of the ranking constraint are commendable. However, the central plausibility claim currently rests on a self-referential evaluation: the SPN used to measure plausibility is the same category-aggregated SPN that the Optimize variant directly optimizes, and the user study does not test that Optimize variant. These gaps are load-bearing for the paper's main contribution and require additional experiments or careful re-scoping before the claims can be accepted.","major_comments":[{"comment":"The plausibility metric in Table II is the log-likelihood under the same category-aggregated SPN that the Optimize variant is explicitly trained to maximize in the objective. Therefore, the reported improvement in plausibility for the SPN variants is partly a consequence of optimizing the evaluation metric itself, not an independent measurement of plausibility. The manuscript should validate the SPN externally, for example with a held-out SPN, an item-level density model, or human ratings that include the Optimize variant.","section":"§IV-A, §V-A, Table II"},{"comment":"The SPN is defined on category-aggregated vectors z_i, while the counterfactual changes the item-level vector x_i. The aggregation is many-to-one: with the disjunction aggregator, replacing any item in a category by another item in the same category leaves z_i unchanged, and with the sum aggregator, exchanging a popular item for an obscure item within a category is invisible to the SPN. Consequently, high category-level likelihood does not imply that the item-level counterfactual is plausible. The paper should provide evidence that p(z) tracks p(x), for instance by comparing category-level likelihoods with item-level likelihoods on a validation set.","section":"§IV-A, Eqs. (7)-(11)"},{"comment":"The user study does not include the Optimize variant (the alpha=0.1 likelihood term in the objective), which is the main numerical contribution of the paper. The best user-rated approach, C, uses no SPN at all, and the best SPN-based rank-guarantee approach, A, is statistically equivalent to C rather than superior. Thus the user study does not provide external support for the claim that SPN-based optimization improves perceived plausibility; the authors should either rerun the study with the Optimize variant or clearly limit their claims to the configurations actually tested.","section":"§VI, Table IV and Fig. 2"},{"comment":"All numerical experiments use EASE as the recommender, although Section III-b claims the approach improves on the state of the art for 'most recommender models' and Section IV discusses formulations for NCF and VASP. The general claim is not supported by experiments with a single recommender model; testing at least one additional model, such as NCF or a VAE-based recommender, is needed to substantiate the claimed generality.","section":"§V, Tables I-II"},{"comment":"The paper selects the disjunction aggregator after inspecting Table I and then reports only that configuration in Table II. This post-hoc selection, made without a separate validation split, means the reported plausibility and similarity comparisons are not a clean test of the method as specified. The authors should report results for all aggregators or justify the selection on a priori grounds.","section":"§V-A, Tables I-II"}],"minor_comments":[{"comment":"The text around Eq. (5) says 'the merchant j ranks above the merchant i,' but i already denotes the user; the explained item should be c, not i.","section":"§IV, Eq. (5)"},{"comment":"The sentence 'In the continuous case, one can introduce [13], [14] auxiliary variable' is missing an article; it should read 'an auxiliary variable.'","section":"§IV"},{"comment":"There are typos in Table IV and the surrounding text: '5 positions dorp' should be '5 positions drop,' and 'recommendatoin' should be 'recommendation.'","section":"§VI, Table IV"},{"comment":"The text contains the typo 'fied of business' for 'field of business.'","section":"§VI"},{"comment":"Several numeric entries in Table II have stray spacing (e.g., '1 .16±0.49'); the formatting should be made consistent.","section":"Table II"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThis paper is a useful engineering extension of LiCE to recommender systems. The top-k removal constraint in Eqs. 5-6 is a genuine improvement over score-thresholding; it targets the rank directly and the big-M handling looks correct. The code is public, which in this area is real evidence. The category-level SPN aggregation is a sensible answer to sparse item-level data. I agree with the reader's conditional verdict, but I want to underline the stress-test note: the plausibility claim rests on an unvalidated proxy.\n\nThe central issue is circularity. The objective minimizes a likelihood term from an SPN, and the evaluation metric is the log-likelihood under that same SPN. On top of that, the SPN is built on category-aggregated vectors z, while the counterfactual actually changes item-level vectors x. The aggregation is many-to-one: with disjunction, flipping any single item in a category leaves z unchanged, and with sum, swapping a popular item for an obscure one within a category is invisible to the SPN. So high p(z) does not imply high p(x). The numerical results in Table II therefore cannot support the 'highly plausible' claim—they mostly show that the optimizer is good at maximizing its own surrogate. The user study is a more independent check, but it does not test the optimize variant, and the no-SPN variant C does at least as well as the SPN variants. That pattern is consistent with the concern.\n\nSmaller problems: the disjunction aggregator is selected after seeing Table I, which is post hoc selection on the test data, and the experiment uses only EASE with no comparison to the recommender CE methods cited in the paper. These are fixable.\n\nI do think the paper deserves a serious referee. The formulation is novel and correct, the code is public, and the question matters. The right outcome is a major revision focused on validation: add an item-level plausibility check that does not use the SPN as both optimizer and metric, include baseline methods, and run the user study on the optimize variant. If the results survive that, the paper will be a real contribution.\n\nFor now, the contribution is real but the central claim is not established.","headline":"Solid MIO formulation and public code, but the plausibility claim is undercut by a self-referential SPN metric and a user study that omits the optimize variant.","tokens_in":13795,"tokens_out":4178,"would_cite":false,"duration_ms":41164,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Recommender counterfactuals can be found in seconds and are judged plausible by users.","keywords":["counterfactual explanations","recommender systems","sum-product networks","mixed-integer optimization","plausibility","top-k recommendation","explainability","user study"],"falsifier":"Collect paired counterfactuals for the same user and item that have identical category-aggregated SPN likelihood but differ in which specific item within a category is removed (for example, removing a frequently rated versus a rarely rated merchant in the same category); if users do not judge the two as equally plausible, the category-level likelihood is not measuring item-level plausibility and the method's central promise fails.","tokens_in":12901,"feed_emoji":"💡","tokens_out":6337,"duration_ms":67126,"temperature":0.7,"pith_summary":"Recommender systems often explain a suggestion by counterfactual: \"you were recommended X because you interacted with Y; if you had not, X would drop off the list.\" This paper argues that such explanations should not only be close to the user's current behavior but also plausible, meaning likely under the data distribution. It adapts the LiCE method to recommenders by encoding the top-k ranking condition as mixed-integer optimization and using a sum-product network to estimate counterfactual likelihood, testing several ways of aggregating items into categories. On binarized interactions with disjunction-aggregated features, it reliably finds counterfactuals that drop the item from the top-k list, with a median solve time of 11 seconds, and a 20-person user study found the resulting explanations sensible and helpful for understanding the recommender.","feed_headline":"Plausible recommender counterfactuals found in seconds","feed_subtitle":"A mixed-integer optimization reliably drops an item from the top-k list while keeping the change likely.","key_machinery":"The machinery is a mixed-integer optimization model whose objective is the $\\ell^1$ distance from the user's current interaction vector, subject to constraints that encode the recommender's scoring function and a rank-based \"drop out of top-k\" condition built with binary ranking variables and a big-M constant. A sum-product network (SPN), optionally trained on category-aggregated features using sum, mean, or disjunction aggregation, supplies the likelihood term that makes counterfactuals plausible. The rank constraints ensure the explanation actually removes the item from the recommendation list, while the SPN term pushes the search toward changes that look like the training data.","core_discovery":"The author's central claim is that counterfactual explanation generation for top-k recommenders can be formulated as a mixed-integer optimization problem, with a sum-product network scoring the likelihood of candidate counterfactuals, and that this reliably produces high-likelihood, model-faithful counterfactuals without exhaustive search. The rank-based constraint guarantees the explained item actually leaves the top-k list, which a score-threshold constraint alone does not ensure. The paper demonstrates this with EASE as the recommender across three public datasets and a private transaction dataset, and reports that users rate the resulting explanations as sensible.","pith_inferences":["If category-level likelihood does not track item-level plausibility, the method's plausibility guarantees are weaker than they appear; a direct test would be to hold the aggregated SPN likelihood fixed while swapping a frequently-rated item for a rarely-rated one in the same category and asking users which counterfactual is more sensible.","The user study's preference for the fastest, sparsest setup without an SPN hints that users may weigh conciseness over distributional likelihood; an approach that minimizes $\\ell^1$ distance first and then re-ranks the few best candidates by SPN likelihood could capture both.","The same MIO-plus-SPN recipe could apply to other decision problems with discrete inputs and tractable scoring functions, such as loan approval, admission, or hiring, where \"what would change the decision?\" questions have the same top-k or threshold structure.","Because the method only allows interaction values to decrease, it is tailored to \"you were recommended X because of Y\" explanations; allowing increases would require additional monotonicity assumptions in the recommender score."],"forward_implications":["For EASE-style linear recommenders, a counterfactual that drops an item from the top-k list can be computed by an open-source solver in a median of 11 seconds, making the approach usable when explanations are generated on request.","Guaranteeing the drop by a rank constraint matters: with only a score-threshold constraint, roughly 30% of the generated counterfactuals still leave the item inside the top-k.","On binarized interaction data, aggregating item features by disjunction makes the optimization easiest to solve and the learned SPN least restrictive, while rating-based data with mean aggregation often makes the problem infeasible.","Using the SPN likelihood as an objective term (with weight $\\alpha = 0.1$) avoids the frequent infeasibility of the hard median-likelihood threshold variant.","In the user study, respondents rated all tested explanation setups above the midpoint of the sensible scale, and the majority agreed that the explanations made the recommender's decisions more transparent and understandable."],"supporting_citations":[{"why":"Base method being adapted; contributes the SPN-likelihood plus MIO recipe for likely counterfactuals.","marker":"[13]"},{"why":"Defines sum-product networks, the tractable probabilistic model used to score counterfactual likelihood.","marker":"[16]"},{"why":"EASE recommender whose linear score function is encoded in the MIO; used in all experiments.","marker":"[34]"},{"why":"Integer programming background, including big-M formulations used in the rank constraints.","marker":"[14]"},{"why":"Open-source solver that finds the MIO solutions within the stated time limits.","marker":"[45]"},{"why":"Pyomo modeling library used to express the optimization problems.","marker":"[38]"},{"why":"Yelp Open Dataset, one of three public evaluation datasets.","marker":"[41]"},{"why":"Amazon book reviews dataset, one of the public evaluation datasets.","marker":"[42]"},{"why":"Netflix competition data, one of the public evaluation datasets.","marker":"[43]"}],"fun_headline_variants":["Fast plausible counterfactuals for top-k recommender lists","Optimized counterfactuals that users find sensible","Drop an item from top-k with a likely alternative","Sum-product scoring makes counterfactuals credible","Counterfactual explanations that leave the top-k list"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The SPN trained on category-aggregated feature vectors correctly estimates how plausible a change to the original item-level interactions looks, even though the paper does not independently verify that the two agree.","fun_headline_variants_meta":{"raw":{"variants":["Fast plausible counterfactuals for top-k recommender lists","Optimized counterfactuals that users find sensible","Drop an item from top-k with a likely alternative","Sum-product scoring makes counterfactuals credible","Counterfactual explanations that leave the top-k list"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000642,"raw_usage":{"total_tokens":2832,"prompt_tokens":705,"completion_tokens":2127,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":321,"completion_tokens_details":{"reasoning_tokens":2053}},"tokens_in":321,"tokens_out":2127,"duration_ms":16593,"temperature":1.0,"reasoning_tokens":2053,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:28:57.982054+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect paired counterfactuals for the same user and item that have identical category-aggregated SPN likelihood but differ in which specific item within a category is removed (for example, removing a frequently rated versus a rarely rated merchant in the same category); if users do not judge the two as equally plausible, the category-level likelihood is not measuring item-level plausibility and the method's central promise fails.","supporting_citations":[{"cited_title":"Generating likely counterfactuals using sum-product networks,","cited_arxiv_id":null,"evidence_quote":"Base method being adapted; contributes the SPN-likelihood plus MIO recipe for likely counterfactuals."},{"cited_title":"Sum-product networks: A new deep architecture,","cited_arxiv_id":null,"evidence_quote":"Defines sum-product networks, the tractable probabilistic model used to score counterfactual likelihood."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Integer programming background, including big-M formulations used in the rank constraints."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Pyomo modeling library used to express the optimization problems."},{"cited_title":"Yelp Open Dataset","cited_arxiv_id":null,"evidence_quote":"Yelp Open Dataset, one of three public evaluation datasets."},{"cited_title":"The Netflix Prize,","cited_arxiv_id":null,"evidence_quote":"Netflix competition data, one of the public evaluation datasets."}],"review_version":1}