{"id":"925bfed4-5849-45ab-8b97-de14ea110a45","arxiv_id":"1909.02322","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A condense-then-abstract model for opinion summarization uses all input reviews and supports zero-shot aspect- and sentiment-controlled summary generation.","lead":"This paper introduces a two-stage framework that condenses all available reviews into dense vectors before generating an opinion summary, avoiding an extractive pre-selection step. It also enables users to request summaries focused on a specific aspect or sentiment at test time using only background review examples.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Best-performing CONDA SUM owes its SOTA margin to the salience-biased extractive module, not to condensing all reviews; the no-extract variant falls below an EA baseline on key metrics.","rationale":"The reader's verdict is CONDITIONAL and notes in passing that the best model still relies on an extractive step, conflicting with the 'no pre-selection' framing. However, the reader's designated weakest assumption concerns the zero-shot customization background reviews and query vector validity, which is a secondary contribution. The more load-bearing issue is that the paper's headline improvement over SOTA is not attributable to its core novelty (condensing all reviews) because the salience-biased extractive module—a component the framework claims to avoid—accounts for the decisive gains. The ablation data in Table 6 already show that the no-extract CONDA SUM underperforms an EA baseline on ROUGE-L and ROUGE-2, so the causal claim 'using all reviews helps' is not internally supported. This does not invalidate the framework or the customization results, but it means the paper should be accepted only if the authors either demonstrate the condense-all-reviews mechanism's independent contribution or substantially reframe the claim. Hence I keep the verdict CONDITIONAL but with a stronger, more specific condition than the reader's customization-based one.","tokens_in":15648,"tokens_out":5023,"duration_ms":49780,"concrete_test":"Using the released code (or a faithful reimplementation), train the full CONDA SUM and a variant with the salience-biased extractive module removed, with identical hyperparameters. Compare both against BERTCENT+PTGEN on the Rotten Tomatoes test set. If the no-extract variant does not exceed BERTCENT+PTGEN on ROUGE-L and ROUGE-2 (as Table 6 suggests), the claimed advantage of condensing all reviews is unsupported. Additionally, ablate the condenser itself by feeding the same fusion/copy decoder with raw BERT review encodings instead of autoencoder encodings; if performance is unchanged, the condense-all-reviews mechanism is not the driver of the reported gains.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is that the CONDENSE-ABSTRACT framework outperforms extract-abstract models because it leverages all reviews rather than a pre-selected subset. The paper's own ablation (Table 6) undermines this attribution. The best configuration, CONDA SUM, includes a salience-biased extractive step (BERTCENT) that the framework is framed to eliminate. Removing this module drops ROUGE-L from 18.47 to 16.44 and ROUGE-2 from 7.65 to 6.17, while the EA baseline BERTCENT+PTGEN scores 17.25 ROUGE-L and 7.43 ROUGE-2. Thus, without the extractive module, the CA model is worse than an EA baseline on two of five metrics, and the reported 'large margin' largely comes from the extractive component. This means the paper never demonstrates that conditioning on all condense encodings by itself improves informativeness over pre-selected extracts. Furthermore, the customizable variant—the one without the extractive step—is not the SOTA variant, so the twin claims of informativeness and controllability are not shown in a single system. The paper acknowledges that the extractive module hurts customizability, but it does not disclose that the same module is the source of the general-purpose gains, creating an internal tension with the framework's motivation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CONDENSE-ABSTRACT (CA), an alternative to the extract-abstract (EA) two-stage framework for opinion summarization. Instead of pre-selecting a subset of reviews, the CA framework condenses every input review into a dense vector and feeds all such encodings to an abstractive decoder. The authors instantiate this as CONDA SUM, using an LSTM autoencoder for condensing and a decoder with attention and copy mechanisms, optionally augmented with a salience-biased BERTCENT extractive step. They also propose a zero-shot customization technique where background reviews representing a user need (e.g., acting, plot, positive/negative sentiment) are averaged and used as the fusion query vector. Experiments on the Rotten Tomatoes dataset report automatic metric gains over several EA and extractive baselines, human Best-Worst Scaling results on 50 movies, and a customization study in which the variant without the extractive step produces summaries that crowdworkers perceive as more aspect- or sentiment-focused. The paper claims that the CA framework produces more informative and more controllable summaries than existing approaches.","tokens_in":15904,"tokens_out":3938,"duration_ms":41893,"significance":"If the central claim were fully supported, the CA framework would be a meaningful step for multi-document opinion summarization: it offers a way to avoid a hard selection bottleneck, and the zero-shot customization idea is practically attractive because it does not require need-specific training data. The paper's strengths include a clean two-stage formulation, a concrete instantiation, careful ablation experiments, human evaluation via Best-Worst Scaling, and reproducible experimental details. However, the reported evidence is in tension with the framework's motivating premise: the best-performing general-purpose system relies on an extractive BERTCENT module, while the variant that enables customization falls below an EA baseline on two automatic metrics. The paper therefore does not currently demonstrate that conditioning on all condense encodings by itself improves informativeness over pre-selected extracts, and the twin claims of informativeness and controllability are not shown in a single system. This is a load-bearing issue that should be addressed before publication.","major_comments":[{"comment":"The central attribution claim is undermined by the ablation results. The best configuration, CONDA SUM, includes the salience-biased BERTCENT extractive step. Removing this module drops ROUGE-L from 18.47 to 16.44 and ROUGE-2 from 7.65 to 6.17 (Table 6), which is below the EA baseline BERTCENT+PTGEN at 17.25 ROUGE-L and 7.43 ROUGE-2 (Table 2). Thus, when the model actually uses all condense encodings without extractive pre-selection, it does not beat an EA baseline on two of the five reported metrics. The abstract and introduction frame the contribution as eliminating reliance on pre-selected content, but the reported state-of-the-art margin comes largely from a module that reintroduces exactly that pre-selection. Please either reframe the paper's claim as a framework that offers a trade-off between a general-purpose extractive-augmented mode and a customizable non-extractive mode, or provide evidence that the non-extractive variant outperforms EA baselines, for example with a stronger condense model or a different fusion design.","section":"§5, Table 6 vs. Table 2"},{"comment":"The zero-shot customization technique assumes that the average of CONDENSE encodings of a set of background reviews C_x is a valid query vector for the target movie's reviews. The paper uses |C_x|=1,000 from Ficler and Goldberg (2017), whose aspect labels are heuristic, but it reports no analysis of how the size or composition of C_x affects customization quality. It also does not test robustness to noisy or mismatched background reviews. Since this assumption is load-bearing for the controllability claim, please add sensitivity experiments varying the size of C_x, the source of the background reviews, and the quality of the aspect/sentiment labels, and report the resulting proportion of aspect/sentiment mentions in the customized summaries.","section":"§3.3, Customization evaluation in §5"},{"comment":"The customization evaluation shows that the without-extracts variant produces summaries that are more aspect- or sentiment-focused than its own non-customized variant, but it does not assess whether customization degrades other summary qualities. The query-vector replacement in Section 3.3 could, in principle, make customized summaries less informative or factually correct even while increasing aspect focus. Please report automatic metrics or human judgments for the customized outputs against the non-customized CONDA SUM and against the state-of-the-art general-purpose system, so that readers can see whether controllability comes at a measurable cost in summary quality.","section":"§5, Table 5"},{"comment":"All experiments are on a single dataset, Rotten Tomatoes, which is a movie-review domain with short, relatively homogeneous reviews. Opinion summarization is commonly evaluated on product reviews (e.g., Amazon or Yelp), where aspects such as price, comfort, or battery life matter and where the zero-shot customization method would be more clearly motivated. Please add at least one additional domain or explicitly justify why conclusions from movie reviews should transfer. This is important because the paper's broader claims are about opinion summarization generally, not about movie reviews specifically.","section":"§4 and §5"}],"minor_comments":[{"comment":"Typo: \"Morevoer\" should be \"Moreover\" in the paragraph following Table 3.","section":"§5"},{"comment":"The terminology is inconsistent: the subsection is titled \"Salience-biased Extracts\" but later text uses \"salient-biased extracts\". Please choose one term and use it consistently.","section":"§3.2 and §5"},{"comment":"BERTCENT is first described as selecting k=1 review nearest to the centroid, but later the text says \"For all extractive steps, we set k=5\". Clarify whether the BERTCENT baseline uses k=1 or k=5, since this affects the comparison.","section":"§4, Comparison Systems"},{"comment":"The notation in Equation (7) is unclear: W_p is described as a tensor in R^{D_d × D_d × D_d}, but the expression d_i^T W_p \\bar{d} as written suggests a matrix-vector product. Please specify the exact tensor contraction or replace W_p with a matrix.","section":"Eq. (7)"},{"comment":"The code URL is given as a placeholder (\"xxx.yyy.zzz\"). Please provide the actual repository URL in the final version.","section":"§4, Training Configuration"},{"comment":"The rows for REGRESS+S2S have dashes for ROUGE-1/2/L. This is presumably because those metrics were not reported in Wang and Ling (2016); please state this explicitly in the table caption or footnote.","section":"Table 2"},{"comment":"The caption says \"The latter produces more informative and factual summaries\" but the comparison in the example is between an EA system and CONDA SUM; please make clear that the example is illustrative and not from a controlled experiment.","section":"Figure 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is well written and the experiments are largely sound, but the central narrative overstates what the evidence shows. The best general-purpose system depends on the extractive module that the framework is supposed to eliminate, and the customizable variant loses to an EA baseline on two metrics. I would not reject the paper, because the framework and the customization technique are interesting and partially supported; however, the authors need to either reframe their claims as a trade-off between informativeness and controllability or provide new experiments showing that the non-extractive CA model can match or exceed EA baselines. I would also ask for at least a second dataset before accepting such broad claims about opinion summarization."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: the CONDENSE-ABSTRACT framing is genuinely new, and the zero-shot customization trick is clever. But the paper's central claim—that condensing all reviews beats extracting a subset—is not actually supported by the experiments. The best configuration, CONDA SUM, includes a salience-biased extractive step (BERTCENT), and the ablation in Table 6 shows that removing that step drops ROUGE-L from 18.47 to 16.44 and ROUGE-2 from 7.65 to 6.17. Meanwhile the EA baseline BERTCENT+PTGEN scores 17.25 ROUGE-L and 7.43 ROUGE-2. So the no-extract variant is worse than a 2016-style EA baseline on two of five metrics. The paper is transparent about the extractive module's role in Section 3.2, but the abstract and introduction frame the contribution as eliminating reliance on pre-selected content. That framing is misleading.\n\nWhat the paper does well: the CA framework is a sensible way to scale multi-document summarization beyond extract-then-abstract, the attentive pooling over autoencoder encodings is neat, and the zero-shot customization via background review query vectors is a practical and testable idea. The evaluation is thorough for a single dataset: automatic metrics, human Best-Worst Scaling, ablations, and customization judgments. The examples in the appendix are useful and the writing is clear.\n\nSoft spots beyond the attribution issue: the customization experiments only measure whether the desired aspect or sentiment appears more often in the output, not whether the customized summary is actually better or more useful. The technique's sensitivity to the size and choice of background reviews is not analyzed. The paper mentions recent unsupervised baselines (MeanSum, CopyCat) but does not compare against them, weakening the \"state of the art\" claim. And no code is released—the link is a placeholder. These are addressable, but they matter.\n\nMy take: this deserves a serious referee, not a desk reject. The framework is worth discussing, and the main weaknesses are fixable with a sharper framing, additional baselines, and an analysis of the customization assumption. A good referee would push the authors to either weaken the conceptual claims or show that the no-extract variant is competitive on its own. I'd bring it to a reading group to argue about exactly that.","headline":"A genuinely novel framework that is oversold: the SOTA margin comes mostly from the extractive module that the paper claims to move beyond.","tokens_in":16415,"tokens_out":4157,"would_cite":true,"duration_ms":43773,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Opinion summarization improves when the extractive pre-selection step is replaced by condensing all reviews into dense vectors, and user needs can be injected at test time by a zero-shot query built from background reviews.","keywords":["opinion summarization","condense-abstract framework","extract-abstract framework","zero-shot customization","controllable generation","multi-document summarization","Rotten Tomatoes dataset","autoencoder"],"falsifier":"Compare the summaries produced with background reviews for 'acting' against those produced with random reviews of the same size; if the acting query does not yield a significantly higher fraction of acting-related content than the random query, the semantic steering claimed by the zero-shot technique is not actually occurring. A second check is to run the customization with a deliberately mismatched background (e.g., plot reviews labeled as acting); the technique should degrade predictably if it truly uses aspect semantics.","tokens_in":15453,"feed_emoji":"🎬","tokens_out":3964,"duration_ms":34009,"temperature":0.7,"pith_summary":"The paper argues that the standard two-stage approach to opinion summarization, which first extracts a few salient reviews and then abstracts over them, throws away information that could be useful for tailoring a summary to a particular user. It proposes a CONDENSE-ABSTRACT framework in which every review is first compressed into a dense vector and the abstractive model attends over all of these encodings. The instantiation, CONDA SUM, is shown to produce summaries that are more informative and more faithful than extract-then-abstract baselines on the Rotten Tomatoes movie review dataset. The paper also introduces a zero-shot technique that lets a user steer the summary toward an aspect or sentiment by supplying background reviews as a query, with no need for customized training examples.","feed_headline":"Condense all reviews, don't extract: better opinion summaries","feed_subtitle":"The CONDENSE-ABSTRACT framework beats extract-then-abstract baselines and lets users steer summaries by example reviews at test time.","key_machinery":"The central object is the pair of dense encodings produced by the CONDENSE model: a document-level encoding $d$ and word-level encodings $h_i$. The ABSTRACT model fuses all $N$ review encodings via attentive pooling whose query is a mean vector; replacing that query with the mean encoding of background reviews $C_x$ is the zero-shot customization mechanism. A second mechanism is a fusion hinge loss that pushes the fused encoding $d'$ close to the encoding of the gold summary while pushing it away from negative summaries.","core_discovery":"The central claim is that replacing the extractive pre-selection step with a condensation step over all reviews improves both informativeness and controllability of opinion summaries. CONDA SUM uses an LSTM autoencoder to produce a review encoding and word-level encodings for each review, then fuses all review encodings with an attention pooling, averages word encodings across the corpus, and decodes with attention and copying. At test time the query vector for fusion can be replaced by the average encoding of background reviews about a desired aspect (e.g., acting or plot) or sentiment, generating a summary focused on that need. Experiments on Rotten Tomatoes report gains in ROUGE and METEOR over extract-abstract baselines, and human judgments rate CONDA SUM summaries as more informative and correct; separate elicitation shows the customization technique shifts the proportion of summaries perceived to mention acting, plot, or positive/negative sentiment, but only when the salience-biased extractive module is removed.","pith_inferences":["The zero-shot query-vector technique suggests a continuum of control: rather than binary customization, one could interpolate between the general mean query and a need-specific query to trade salience against user focus; the paper does not test this.","Because customization depends on the average encoding of background reviews, the method inherits the biases of that background corpus; weighting background reviews by relevance to the target domain is a natural extension.","The framework could be combined with contrastive or denoising objectives to make condensed encodings carry more disentangled aspect information, potentially sharpening customization without needing to disable the extractive module.","A testable prediction is that the information gain over extract-then-abstract widens as the number of reviews per target grows and as the extractor becomes more imperfect, since condensation loses less information than pre-selection."],"forward_implications":["If the CA framework is right, summaries can be generated from arbitrarily large review sets without a pre-selection bottleneck, since memory scales with per-review encoding rather than the full corpus.","Customizing summaries no longer requires training data of customized summaries; any aspect or sentiment expressible in background reviews can be steered at test time.","The extractive module, while helpful for general-purpose salience, limits customization; the paper's results imply that controllability and extractive bias trade off.","The approach transfers to other multi-document summarization tasks where inputs number in the hundreds, such as product or service reviews."],"supporting_citations":[{"why":"Provides the Rotten Tomatoes dataset and the REGRESS+S2S extract-abstract baseline, the primary comparison and evaluation setting.","marker":"Wang and Ling (2016)"},{"why":"Supplies the pointer-generator copy mechanism used in the ABSTRACT decoder.","marker":"See et al. (2017)"},{"why":"Supplies the attention mechanism used in the ABSTRACT decoder.","marker":"Bahdanau et al. (2014)"},{"why":"Provides BERT representations used by BERTCENT for the salience-biased extracts.","marker":"Devlin et al. (2019)"},{"why":"Defines the centroid-based extraction method that BERTCENT instantiates.","marker":"Radev et al. (2000)"},{"why":"Provides the large corpus of reviews with aspect and sentiment labels used to select background reviews for customization.","marker":"Ficler and Goldberg (2017)"},{"why":"Provides SUMMA RUNNER, the extractive baseline used both as a comparison system and as an alternative salience-biased extractor in ablations.","marker":"Nallapati et al. (2017)"}],"fun_headline_variants":["Condense, don't extract: opinion summaries that adapt","All reviews condensed, not extracted: steerable summaries","Replace extraction with condensation for controllable summaries","Zero-shot customization via condensing all reviews"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The customization technique assumes that the average autoencoder encoding of a handful of background reviews about an aspect (e.g., acting) is a reliable query for what a user means by that aspect for any target movie; if the background reviews are not representative or the encodings do not separate aspects, the generated 'customized' summaries may not actually reflect the requested need.","fun_headline_variants_meta":{"raw":{"variants":["Condense, don't extract: opinion summaries that adapt","All reviews condensed, not extracted: steerable summaries","Replace extraction with condensation for controllable summaries","Zero-shot customization via condensing all reviews"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000216,"raw_usage":{"total_tokens":1423,"prompt_tokens":929,"completion_tokens":494,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":545,"completion_tokens_details":{"reasoning_tokens":435}},"tokens_in":545,"tokens_out":494,"duration_ms":5151,"temperature":1.0,"reasoning_tokens":435,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:52:41.627087+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the summaries produced with background reviews for 'acting' against those produced with random reviews of the same size; if the acting query does not yield a significantly higher fraction of acting-related content than the random query, the semantic steering claimed by the zero-shot technique is not actually occurring. A second check is to run the customization with a deliberately mismatched background (e.g., plot reviews labeled as acting); the technique should degrade predictably if it truly uses aspect semantics.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the attention mechanism used in the ABSTRACT decoder."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides SUMMA RUNNER, the extractive baseline used both as a comparison system and as an alternative salience-biased extractor in ablations."}],"review_version":1}