Pith. sign in

REVIEW 4 major objections 5 minor 63 references

Mobile Application Review Summarization using Chain of Density Prompting

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An adapted Chain of Density prompt generates denser, more informative app review summaries than vanilla prompting or extractive baselines, without hurting readability.

desk verdict Adapting Chain of Density to app reviews looks like a real niche win, but the lack of any faithfulness check makes the headline density result conditional. read the letter →

arxiv 2506.14192 v1 pith:OA6H5IBI submitted 2025-06-17 cs.SE

Shristi Shrestha , Anas Mahmoud This is my paper · ORCID
classification cs.SE
keywords mobileappreviewsreviewsummarizationchainofdensitypromptinglargelanguagemodelsGPT-4semanticreadabilityentityrecall
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the Chain of Density (CoD) prompting strategy, originally designed for news articles, can be adapted to mobile app reviews by redefining an entity as any functional or non-functional app feature users say helps or harms their experience. The adapted prompt, called CoDr, is shown to produce 120-word abstractive summaries that pack in more app features per sentence than the original CoD prompt, a vanilla GPT-4 prompt, or the extractive Hybrid TF.IDF baseline. The paper also reports that human readers find the densest summaries at least as readable as lighter ones, so the added information does not come at the cost of comprehension. If the results hold, app stores could display such summaries above review feeds to help users grasp thousands of reviews at a glance.

What carries the argument

The machinery is the Chain of Density prompt itself, an iterative two-step procedure repeated five times: identify one to three 'missing' entities from the source text and rewrite the whole summary at the same word count to fuse them in, using compression, fusion, and removal of filler. The adaptation that carries this paper's argument is a domain-specific entity definition: an entity is any functional or non-functional app feature that users perceive to harm or enhance their experience. That definition, plus the instruction to avoid other app names and personal or location details, directs the model toward breadth across app features rather than depth on a single dominant topic. The same CoDr prompt generates the summaries whose density is measured at each iteration, and the 120-word ceiling is the fixed-length constraint the summaries must respect.

What would settle it

Take the summaries and search the source review sample for each named entity and its claimed valence; for example, check whether any Calm review supports 'payment disputes' and whether any Robinhood review supports 'market manipulation.' If a substantial share of entities (say more than 5 percent) have no supporting review, the claim that CoDr summaries faithfully represent user feedback would be falsified.

Watch

Extended reading notes

Core claim

The central claim is that an adapted Chain of Density prompt significantly improves the semantic density and entity recall of mobile app review summaries. Across eight apps in four domains, the CoDr prompt out-recalled the original CoD prompt and vanilla prompting, reaching roughly 81% recall of manually identified entities by the third iteration versus 64% for vanilla, and it captured more entities than Hybrid TF.IDF. A readability study with 48 participants found no statistically significant evidence that readability declines as density increases; the 5th-iteration summaries were rated easiest to read. The paper further claims the prompt transfers to other LLMs, though Gemini-1.5-Flash sacrifices sentiment context for extreme density and Llama-3.1-70B-Instruct relaxes the length constraint.

Load-bearing premise

The entire comparison is built on recall of known themes, not on verification that the generated summaries faithfully reflect the source reviews; if the language models invent or misattribute app features, the denser summaries would be fluent text about things users never said.

Editorial extensions

If this is right

  • If the central claim is right, the CoDr prompt can turn a stratified sample of roughly 350 app reviews into a 120-word abstractive summary that a prospective user can read in seconds.
  • The 81% entity recall at the third iteration implies that rare but salient concerns, such as transphobia and harassment on Tinder, survive summarization that frequency-based extractive methods miss.
  • Readability results suggest user-facing placement of dense summaries is viable: readers did not penalize the 5th-iteration summaries, and automated readability ratings favored CoDr over both baselines.
  • Because the prompt transfers across LLMs, the approach is not tied to GPT-4; the practical choice becomes a trade-off between Gemini's extreme density, Llama's length, and GPT-4's balance, along with per-summary cost and latency differences.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the entity definition is the likely driver of the density gain; a clean ablation that swaps the original news-entity definition back into the CoDr scaffolding would isolate how much of the improvement comes from the definition versus the iterative loop.
  • Editorial inference: since precision and hallucination were not measured, a natural extension is to fact-check each summary entity against the source review sample; this would determine whether 'denser' also means 'truer'.
  • Editorial inference: the same iterative-fusion mechanism could apply to other short, noisy user-generated text such as product or game reviews, where entity definitions would be product features rather than app features.
  • Editorial inference: the integration sketch suggests a randomized controlled trial of app-store mock-ups with and without summaries, measuring whether users choose differently and whether anchoring and negativity biases shrink; the authors list this as future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper adapts the Chain of Density (CoD) prompt for mobile app review summarization. The adapted prompt, CoDr, defines an entity as any functional or non-functional app feature that helps or harms user experience, and iteratively fuses such entities into a fixed-length summary. The authors compare CoDr against the original CoD prompt, a vanilla prompt, and Hybrid TF-IDF in terms of entity density and entity recall, evaluate readability through an automated GPT-4o rating and a 48-participant human study, and test the prompt across GPT-4, Gemini-1.5-Flash, and Llama-3.1-70B-Instruct. The central claims are that CoDr significantly improves semantic density and entity recall over the baselines (RQ1) and that readability does not decline as density increases (RQ2).

Significance. If the central claims hold, the paper offers a practical, end-user-oriented approach to app review summarization, with a clear prompt-design contribution and a plausible integration proposal for app stores. The authors share their data and replication scripts, which is a concrete strength. The use of manual annotations, paired t-tests, a human readability study, and a cross-LLM evaluation also goes beyond many prompt-engineering papers. However, the validity of the headline RQ1 result depends on measurement choices that are currently not fully justified, especially the lack of any faithfulness or precision check and the tight coupling between the prompt's entity definition and the evaluation metric. These are fixable within the manuscript's scope, so the work is worth revising rather than rejecting.

major comments (4)
  1. [Section 4.3.1 and Fig. 6, with Section 6] The RQ1 comparison rests entirely on entity counts extracted from the generated summaries, with no verification that each counted entity is actually present in or attributable to the source reviews. The prompt instructs the model that missing entities must be 'faithful (present in the reviews)', but instruction compliance is assumed rather than measured. Section 6 states that hallucinations are 'highly unlikely to occur' given the prompt's sophistication, but this is an assertion, not a test. Because the Hybrid TF-IDF baseline is extractive and therefore has perfect precision by construction, a model that invents fluent but unsupported entities could inflate both density and recall and appear to outperform the extractive baseline. Please add a source-support check for each counted entity, either through manual verification or a retrieval-based alignment step, and report precision and hallucination rates for all systems.
  2. [Section 4.3.1 versus Fig. 3] The evaluation metric uses the same entity definition that was written into the CoDr prompt: an entity is 'any functional or non-functional feature of the app that users mention in their reviews and perceive to either harm or enhance their overall experience.' This creates a partial circularity: the metric rewards exactly the behavior the prompt was engineered to produce, so the high density scores partly measure prompt compliance rather than independently assessed summary quality. This does not invalidate the comparison with CoD, vanilla prompting, or Hybrid TF-IDF, but it narrows the claim to 'density under the authors' feature-based definition.' The paper should either use an independently defined annotation scheme or explicitly reframe RQ1 and the abstract to avoid implying a general measure of semantic density.
  3. [Section 4.3.1 and Fig. 6] The recall analysis is based on entities manually identified in only 30 reviews per app, but the paper reports only mean recall curves in Fig. 6, without confidence intervals, per-app values, or the exact number of entities in each ground-truth set. The claim that CoDr reaches about 81% recall at the third iteration while the original CoD settles near 62% would be more convincing with error bars or a per-app breakdown, especially because the sample size of 30 reviews per app is small and the entity counts in Table 3 vary widely across apps. Please report the underlying data and a variability measure.
  4. [Section 4.3.1 and Section 6] All density and recall results depend on manual entity annotations by three judges, but the paper reports only a 'small conflict rate of ~5%' with no inter-annotator agreement coefficient. Since paired t-tests are run on density values derived from these annotations, the reliability of the annotation process is load-bearing. Please report Cohen's kappa or an equivalent agreement measure for the final entity annotations, not just the pilot session.
minor comments (5)
  1. [Fig. 7] The y-axis label 'readabiliy' is misspelled and should read 'readability'.
  2. [Section 3.1 and Fig. 2] The text and figure refer to 'Adam et al.' but reference [14] is authored by Adams et al.; please correct the name consistently.
  3. [Section 4.1] The phrase 'Natural Langauge Toolkit' contains a typo and should be 'Natural Language Toolkit'.
  4. [Section 4.3.2] The automated readability evaluation cites G-eval as evidence that LLM-based readability scoring correlates with human judgment, but no validation is provided for the particular 1-5 readability prompt on app review summaries; a small validation set would strengthen this preliminary analysis.
  5. [Fig. 8] The stacked bar chart would be easier to interpret if it reported percentages or counts per condition in the figure itself, and the caption's description of the y-axis is ambiguous.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the entity-definition overlap is deliberate construct operationalization, not a reduction of the result to its inputs.

full rationale

The paper defines an entity in the CoDr prompt as "any functional or non-functional feature of the app that users mention in their reviews and perceive to either harm or enhance their overall experience," and the evaluation metric uses a nearly identical definition. This alignment is not circular, however: entity density and entity recall are measured by independent human annotators who inspect each generated summary and the source reviews, and the same definition is applied uniformly to all baselines. The prompt's entity definition is the experimental treatment, not a fitted parameter or a precomputed outcome. The headline RQ1 result is a comparison of measured entity counts and recall percentages across systems, and the RQ2 readability result comes from a separate human readability study. The paper's self-citations, including prior work by the same authors on app review summarization and TF-IDF, support background claims and baseline choices rather than the central comparison, so they are not load-bearing in a circular way. The absence of a precision or hallucination measurement is a legitimate validity and correctness concern, but it does not make the derivation equivalent to its inputs by construction. No specific equation or fitted value is shown to force a stated conclusion, so there is no circular step to report.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central evaluation rests on four unproven premises: LLMs follow the prompt as intended, generated summaries are hallucination-free, the feature-based entity definition is a valid quality measure, and the stratified 350-review samples represent each app's review stream. The paper states or implies all four but verifies none of them directly.

free parameters (4)
  • Summary length limit = 120 words
    Chosen because average review length is about 105 +/- 15 words and because review helpfulness saturates around 144 words (Huang et al. [24]); it is a hand-picked design choice, not derived.
  • Sampling temperature = 0.5
    Set to 0.5 to balance consistency and creativity in GPT-4 outputs; it affects all generated summaries and is chosen by hand.
  • Top-p sampling = 0.5
    Set to 0.5 to limit irrelevant tokens; hand-chosen hyperparameter.
  • Frequency and presence penalties = 0.1
    Set near the lower end of OpenAI's recommended range to reduce repetition; hand-chosen.
assumptions (4)
  • domain assumption GPT-4 and the other tested LLMs follow the CoD and CoDr prompt instructions reliably, including the length constraint and the entity definition.
    The entire evaluation assumes the models generate summaries consistent with the prompt; Section 4.1 and Section 6.
  • domain assumption LLM-generated summaries contain few or no hallucinations.
    Section 6 states hallucinations are 'highly unlikely to occur' but no precision check is performed; recall is measured but not precision.
  • ad hoc to paper The manual entity annotation, using the same feature-based entity definition as the prompt, is a valid measure of summary quality.
    The entity definition in Section 4.3.1 matches the entity definition written into the CoDr prompt, so the metric rewards the exact behavior the prompt was engineered to produce.
  • domain assumption Stratified samples of about 350 reviews per app are representative of each app's review population.
    Section 4.1 selects 350 reviews to reach 95% confidence at 5% margin of error; representativeness is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mobile Application Review Summarization using Chain of Density Prompting." pith.science (2026). https://pith.science/paper/OA6H5IBI

@misc{pith2026250614192,
  author       = {Pith},
  title        = {Pith review of: Mobile Application Review Summarization using Chain of Density Prompting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OA6H5IBI}},
  note         = {Machine review of arXiv:2506.14192}
}
read the original abstract

Mobile app users commonly rely on app store ratings and reviews to find apps that suit their needs. However, the sheer volume of reviews available on app stores can lead to information overload, thus impeding users' ability to make informed app selection decisions. To address this challenge, we leverage Large Language Models (LLMs) to summarize mobile app reviews. In particular, we use the Chain of Density (CoD) prompt to guide OpenAI GPT-4 to generate abstractive, semantically dense, and easily interpretable summaries of mobile app reviews. The CoD prompt is engineered to iteratively extract salient entities from the source text and fuse them into a fixed-length summary. We evaluate the performance of our approach using a large dataset of mobile app reviews. We further conduct an empirical evaluation with 48 study participants to assess the readability of the generated summaries. Our results demonstrate that adapting the CoD prompt to focus on app features improves its ability to extract key themes from user reviews and generate natural language summaries tailored for end-user consumption. The prompt also manages to maintain the readability of the generated summaries while increasing their semantic density. Our work in this paper aims to improve mobile app users' experience by providing an effective mechanism for summarizing important user feedback in the review stream.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 57 canonical work pages

  1. [1]

    In: International Conference on Interaction Sciences, pp

    Siegfried, N., Koch, O.F., Benlian, A.: Drivers of app installation likelihood - a conjoint analysis of quality signals in mobile ecosystems. In: International Conference on Interaction Sciences, pp. 45–53 (2015)

    work page 2015

  2. [2]

    Mobile Media & Communication 3(1), 125–144 (2015)

    Dogruel, L., Joeckel, S., Bowman, N.D.: Choosing the right app: An exploratory perspective on heuristic decision processes for smartphone app selection. Mobile Media & Communication 3(1), 125–144 (2015)

    work page 2015

  3. [3]

    Empirical Software Engineering (23), 3734–3767 (2018)

    Jha, N., Mahmoud, A.: Using frame semantics for classifying and summarizing application store reviews. Empirical Software Engineering (23), 3734–3767 (2018)

    work page 2018

  4. [4]

    Management Science 14(4), 147–156 (1967) 1https://figshare.com/s/e7a9127f7acb8bba9439 25

    Ackoff, R.L.: Management misinformation systems. Management Science 14(4), 147–156 (1967) 1https://figshare.com/s/e7a9127f7acb8bba9439 25

    work page 1967

  5. [5]

    Journal of Information Science 35(2), 180–191 (2009)

    Bawden, D., Robinson, L.: The dark side of information: Overload, anxiety and other paradoxes and pathologies. Journal of Information Science 35(2), 180–191 (2009)

    work page 2009

  6. [6]

    Electronic Commerce Research and Applications 7(4), 386–398 (2008)

    Park, D.-H., Lee, J.: eWOM overload and its effect on consumer behavioral inten- tion depending on consumer involvement. Electronic Commerce Research and Applications 7(4), 386–398 (2008)

    work page 2008

  7. [7]

    Journal of Service Theory and Practice 26(6), 788–810 (2016)

    Furner, C.P., Zinko, R., Zhu, Z.: Electronic word-of-mouth and information over- load in an experiential service industry. Journal of Service Theory and Practice 26(6), 788–810 (2016)

    work page 2016

  8. [8]

    In: The ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp

    Lappas, T., Crovella, M., Terzi, E.: Selecting a characteristic set of reviews. In: The ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 832–840 (2012)

    work page 2012

Show all 63 references
  1. [9]

    Journal of Business Research 100, 27–37 (2019)

    Hu, H., Krishen, A.: When is enough, enough? investigating product reviews and information overload from a consumer empowerment perspective. Journal of Business Research 100, 27–37 (2019)

    2019

  2. [10]

    In: International Conference on Automated Software Engineering, pp

    Ebrahimi, F., Mahmoud, A.: Unsupervised summarization of privacy concerns in mobile application reviews. In: International Conference on Automated Software Engineering, pp. 1–12 (2022)

    2022

  3. [11]

    Information and Software Technology 122, 106290 (2020)

    Tao, C., Guo, H., Huang, Z.: Identifying security issues for mobile applications based on user review summarization. Information and Software Technology 122, 106290 (2020)

    2020

  4. [12]

    In: International Conference on Software Engineering, pp

    Chen, N., Lin, J., Hoi, S., Xiao, X., Zhang, B.: AR-MINER: Mining informative reviews for developers from mobile app marketplace. In: International Conference on Software Engineering, pp. 767–778 (2014)

    2014

  5. [13]

    In: ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp

    Di Sorbo, A., Panichella, S., Alexandru, C.V., Shimagaki, J., Visaggio, C.A., Canfora, G., Gall, H.C.: What would users change in my app? summarizing app reviews for recommending software changes. In: ACM SIGSOFT International Symposium on Foundations of Software Engineering, ...

    2016

  6. [14]

    In: New Frontiers in Summarization Workshop, pp

    Adams, G., Fabbri, A., Ladhak, F., Lehman, E., Elhadad, N.: From sparse to dense: GPT-4 summarization with chain of density prompting. In: New Frontiers in Summarization Workshop, pp. 68–74 (2023)

    2023

  7. [15]

    In: Annual Meeting of the Association for Computational Linguistics, pp

    Liu, Y., Fabbri, A., Liu, P., Zhao, Y., Nan, L., Han, R., Han, S., Joty, S., Wu, C.- S., Xiong, C., Radev, D.: Revisiting the gold standard: Grounding summarization evaluation with robust human evaluation. In: Annual Meeting of the Association for Computational Linguistics, pp...

    2023

  8. [16]

    arXiv:2310.10508 (2023)

    Shin, J., Tang, C., Mohati, T., Nayebi, M., Wang, S., Hemmati, H.: Prompt 26 engineering or fine tuning: An empirical assessment of large language models in automated software engineering tasks. arXiv:2310.10508 (2023)

    2023 arXiv

  9. [17]

    In: International Advancement of Artificial Intelligence Conference on Weblogs and Social Media, pp

    Llewellyn, C., Grover, C., Oberlander, J.: Summarizing newspaper comments. In: International Advancement of Artificial Intelligence Conference on Weblogs and Social Media, pp. 599–602 (2014)

    2014

  10. [18]

    In: International Advancement of Artificial Intelligence Conference on Weblogs and Social Media (2011)

    Khabiri, E., Caverlee, J., Hsu, C.-F.: Summarizing user-contributed comments. In: International Advancement of Artificial Intelligence Conference on Weblogs and Social Media (2011)

    2011

  11. [19]

    Computer 33(11), 29–36 (2000)

    Hahn, U., Mani, I.: The challenges of automatic summarization. Computer 33(11), 29–36 (2000)

    2000

  12. [20]

    IEEE Transactions on Knowledge and Data Engineering (2024)

    Jin, H., Zhang, Y., Meng, D., Wang, J., Tan, J.: A comprehensive survey on process-oriented automatic text summarization with exploration of LLM-based methods. IEEE Transactions on Knowledge and Data Engineering (2024)

    2024

  13. [21]

    Information Processing and Management 43(6), 1606–1618 (2007)

    Vanderwende, L., Suzuki, H., Brockett, C., Nenkova, A.: Beyond SumBasic: Task-focused summarization with sentence simplification and lexical expansion. Information Processing and Management 43(6), 1606–1618 (2007)

    2007

  14. [22]

    Journal of Artificial Intelligence Research 22(1), 457–479 (2004)

    Erkan, G., Radev, D.R.: Lexrank: Graph-based lexical centrality as salience in text summarization. Journal of Artificial Intelligence Research 22(1), 457–479 (2004)

    2004

  15. [23]

    arXiv:2209.12356 (2023)

    Goyal, T., Li, J.J., Durrett, G.: News summarization and evaluation in the era of GPT-3. arXiv:2209.12356 (2023)

    2023 arXiv

  16. [24]

    Computers in Human Behavior 48, 17–27 (2015)

    Huang, A.H., Chen, K., Yen, D.C., Tran, T.P.: A study of factors that contribute to online review helpfulness. Computers in Human Behavior 48, 17–27 (2015)

    2015

  17. [25]

    ACM Computing Surveys 55(8), 1–35 (2022)

    Koh, H.Y., Ju, J., Liu, M., Pan, S.: An empirical survey on long document sum- marization: Datasets, models, and metrics. ACM Computing Surveys 55(8), 1–35 (2022)

    2022

  18. [26]

    In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp

    Pilault, J., Li, R., Subramanian, S., Pal, C.: On extractive and abstractive neural document summarization with transformer language models. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 9308– 9319 (2020)

    2020

  19. [27]

    In: North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp

    Dou, Z.-Y., Liu, P., Hayashi, H., Jiang, Z., Neubig, G.: GSum: A general frame- work for guided neural abstractive summarization. In: North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4830–4842 (2021)

    2021

  20. [28]

    No Starch Press (2020)

    Vasiliev, Y.: Natural language processing with python and spacy: A practical 27 introduction. No Starch Press (2020)

    2020

  21. [29]

    In: COLING/ACL on Interactive Presentation Sessions, pp

    Loper, E., Bird, S.: NLTK: The natural language toolkit. In: COLING/ACL on Interactive Presentation Sessions, pp. 69–72 (2002)

    2002

  22. [30]

    In: International Requirements Engineering Conference, pp

    Williams, G., Mahmoud, A.: Mining Twitter feeds for software user requirements. In: International Requirements Engineering Conference, pp. 1–10 (2017)

    2017

  23. [31]

    In: IEEE/ACM Working Conference on Mining Software Repositories, pp

    Martin, W., Harman, M., Jia, Y., Sarro, F., Zhang, Y.: The app sampling problem for app store mining. In: IEEE/ACM Working Conference on Mining Software Repositories, pp. 123–133 (2015)

    2015

  24. [32]

    In: IEEE International Requirements Engineering Conference, pp

    Maalej, W., Nabil, H.: Bug report, feature request, or simply praise? on automat- ically classifying app reviews. In: IEEE International Requirements Engineering Conference, pp. 116–125 (2015)

    2015

  25. [33]

    In: International Conference on Software Engineering, pp

    Tushev, M., Ebrahimi, F., Mahmoud, A.: Domain-specific analysis of mobile app reviews using keyword-assisted topic models. In: International Conference on Software Engineering, pp. 762–773 (2022)

    2022

  26. [34]

    Khalid, H., Shihab, E., Nagappan, M., Hassan, A.: What do mobile app users complain about? IEEE Software 32(3), 70–77 (2015)

    2015

  27. [35]

    Israel, G.D.: Determining sample size (2009)

    2009

  28. [36]

    In: International Conference on Program Comprehension, pp

    Poch´ e, E., Jha, N., Williams, G., Staten, J., Vesper, M., Mahmoud, A.: Analyzing user comments on YouTube coding tutorial videos. In: International Conference on Program Comprehension, pp. 196–206 (2017)

    2017

  29. [37]

    In: Conference on Empirical Methods in Natural Language Processing, pp

    Pennington, J., Socher, R., Manning, C.: Glove: Global vectors for word repre- sentation. In: Conference on Empirical Methods in Natural Language Processing, pp. 1532–1543 (2014)

    2014

  30. [38]

    In: Advances in Neural Information Processing Systems, pp

    Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed represen- tations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)

    2013

  31. [39]

    In: International Conference on Learning Representations (2016)

    Arora, S., Liang, Y., Ma, T.: A simple but tough-to-beat baseline for sentence embeddings. In: International Conference on Learning Representations (2016)

    2016

  32. [40]

    Transactions of the Association for Computational Linguistics 9, 391–409 (2021)

    Fabbri, A.R., Kry` sci` nski, W., McCann, B., Xiong, C., Socher, R., Radev;, D.: Summeval: Re-evaluating summarization evaluation. Transactions of the Association for Computational Linguistics 9, 391–409 (2021)

    2021

  33. [41]

    Communications of the ACM 1(42), 31–37 (1999)

    Mylopoulos, J., Chung, L., Yu, E.: From object-oriented to goal-oriented require- ments analysis. Communications of the ACM 1(42), 31–37 (1999)

    1999

  34. [42]

    University of Chicago Law Review Online 28 82(1) (2015)

    Rogers, B.: The social costs of Uber. University of Chicago Law Review Online 28 82(1) (2015)

    2015

  35. [43]

    In: ACM Conference on Economics and Computation, p

    Fradkin, A., Grewal, E., Holtz, D., Pearson, M.: Bias and reciprocity in online reviews: Evidence from field experiments on Airbnb. In: ACM Conference on Economics and Computation, p. 641 (2015)

    2015

  36. [44]

    Longyear, R., Kushlev, K.: Can mental health apps be effective for depression, anxiety, and stress during a pandemic? Practice Innovations6(2), 131–137 (2021)

    2021

  37. [45]

    In: Empirical Methods in Natural Language Processing, pp

    Liu, Y., Iter, D., Xu, Y., Wang, S., Xu, R., Zhu, C.: G-eval: NLG evaluation using GPT-4 with better human alignment. In: Empirical Methods in Natural Language Processing, pp. 2511–2522 (2023)

    2023

  38. [46]

    Fu, J., Ng, S.-K., Jiang, Z., Liu, P.: GPTScore: evaluate as you desire. ArXiv. https://arxiv.org/abs/2302.04166 (2023)

    2023 arXiv

  39. [47]

    In: Neural Information Processing Systems, pp

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C.L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., Lowe, R.: Training language models to follo...

    2022

  40. [48]

    Topics in Language Disorders 23(3), 204–224 (2003)

    Meyer, B.: Text coherence and readability. Topics in Language Disorders 23(3), 204–224 (2003)

    2003

  41. [49]

    The Journal of Experimental Education 84(1), 75–97 (2016)

    Reed, D.K., Kershaw-Herrera, S.: An examination of text complexity as char- acterized by readability and cohesion. The Journal of Experimental Education 84(1), 75–97 (2016)

    2016

  42. [50]

    Phi Delta Kappan 99(2), 8–14 (2017)

    Rosen, L.D.: The distracted student mind — enhancing its focus and attention. Phi Delta Kappan 99(2), 8–14 (2017)

    2017

  43. [51]

    International Journal of Higher Education 5(4), 144–156 (2016)

    Kumcagiz, H., Gunduz, Y.: Relationship between psychological wellbeing and smartphone addiction of university students. International Journal of Higher Education 5(4), 144–156 (2016)

    2016

  44. [52]

    In: CHI Conference on Human Factors in Computing Systems, pp

    Miniukovich, A., Scaltritti, M., Sulpizio, S., De Angeli, A.: Guideline-based eval- uation of web readability. In: CHI Conference on Human Factors in Computing Systems, pp. 1–12 (2019)

    2019

  45. [53]

    Biochemia medica 23(2), 143–149 (2013)

    McHugh, M.L.: The chi-square test of independence. Biochemia medica 23(2), 143–149 (2013)

    2013

  46. [54]

    arXiv preprint arXiv:2403.05530 (2024) 29

    Reid, M., Savinov, N., Teplyashin, D., Lepikhin, D., Lillicrap, T., Alayrac, J.- b., Soricut, R., Lazaridou, A., Firat, O., Schrittwieser, J., et al.: Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530 (2024) 29

    2024 arXiv

  47. [55]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, pp

    Ji, B., Liu, H., Du, M., Ng, S.-K.: Chain-of-thought improves text generation with citations in large language models. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 18345–18353 (2024)

    2024

  48. [56]

    arXiv preprint arXiv:2406.01574 (2024)

    Wang, Y., Ma, X., Zhang, G., Ni, Y., Chandra, A., Guo, S., Ren, W., Arulraj, A., He, X., Jiang, Z., et al.: Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. arXiv preprint arXiv:2406.01574 (2024)

    2024 arXiv

  49. [57]

    In: IEEE/ACM 11th International Conference on Mobile Software Engineering and Systems, pp

    Shrestha, S., Mahmoud, A.: Generating rate features for mobile applications. In: IEEE/ACM 11th International Conference on Mobile Software Engineering and Systems, pp. 54–64 (2024)

    2024

  50. [58]

    Science 185(4157), 1124–1131 (1974)

    Tversky, A., Kahneman, D.: Judgment under uncertainty: Heuristics and biases. Science 185(4157), 1124–1131 (1974)

    1974

  51. [59]

    Journal of Business Research 153, 235–250 (2022)

    Colmekcioglu, N., Marvi, R., Foroudi, P., Okumus, F.: l. Journal of Business Research 153, 235–250 (2022)

    2022

  52. [60]

    Journal of Marketing Research 43(3), 345–354 (2006)

    Chevalier, J.A., Mayzlin, D.: The effect of word of mouth on sales: Online book reviews. Journal of Marketing Research 43(3), 345–354 (2006)

    2006

  53. [61]

    arXiv preprint arXiv:2310.03533 (2023)

    Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., Zhang, J.M.: Large language models for software engineering: Survey and open problems. arXiv preprint arXiv:2310.03533 (2023)

    2023 arXiv

  54. [62]

    International Conference on Software Engineering (2024)

    Sallou, J., Durieux, T., Panichella, A.: Breaking the silence: The threats of using LLMs in software engineering. International Conference on Software Engineering (2024)

    2024

  55. [63]

    In: CHI Conference on Human Factors in Computing Systems, pp

    Jasim, M., Collins, C., Sarvghad, A., Mahyar, N.: Supporting serendipitous dis- covery and balanced analysis of online product reviews with interaction-driven metrics and bias-mitigating suggestions. In: CHI Conference on Human Factors in Computing Systems, pp. 1–24 (2022) 30

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.