{"id":"dcd27b7c-414b-4615-aaa4-d20f78176afd","arxiv_id":"2508.16779","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Seven days of passively collected smartphone app data predicts students' cumulative GPA to within 0.36 on average, with books and productivity use linked to higher grades and video use linked to lower grades.","lead":"Researchers built an Android tool that pulls the past 7 days of app usage in under a second and compared it with the grades of 124 Bangladeshi students. They found that more app sessions and video apps track with lower grades, while books and productivity apps track with higher grades, and a model predicts GPA within about 0.36 points.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"P-value adjustment may be missing; raw p-values cannot distinguish 0.024 from 0.036, so the FDR claim needs verification.","rationale":"The reader's weakest_assumption focused on the 7-day window versus cumulative CGPA mismatch, which is real but empirically addressable and not easily falsified from the paper. My review finds a more precisely checkable threat: the p-values reported in Tables 1-4 appear to be raw, despite Section 4.3.1 stating FDR adjustment was applied for multiple comparisons. The paper never states whether the displayed p-values are raw or corrected. If raw, the category-level findings (which the abstract elevates to headline claims) are likely false positives under any reasonable multiple-comparison correction, because 27 categories × 5 metrics yields >100 tests, with only a handful of p-values between 0.01 and 0.045. If corrected, the values are implausibly uniform near 0.05. This is distinct from the reader's 7-day concern and is directly testable using the open-source data. I agree with the reader's CONDITIONAL verdict, but for a different, sharper reason; hence partial agreement.","tokens_in":32556,"tokens_out":1295,"duration_ms":11559,"concrete_test":"Obtain the full correlation matrix (27 categories × 5 metrics plus session measures) from the OSF repository and apply the Benjamini-Hochberg FDR procedure cited as [42] at q=0.05 to all reported correlation tests. If the Table 3 entries for Books (duration rs=0.24, p=0.026), Productivity (duration rs=0.21, p=0.018), and Video (duration rs=-0.21, p=0.023) no longer fall below the adjusted threshold, the central category-level significance claims are not supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central findings—significant negative association of app usage sessions and Video category with CGPA, positive association of Productivity and Books—rest on correlation p-values reported in Tables 1-4 (e.g., rs=-0.24, p=0.009; rs=-0.21, p=0.023; rs=0.21, p=0.018). Section 4.3.1 says p-values were adjusted using the false discovery rate approach for multiple comparisons, but the p-values shown appear to be unadjusted raw values. For example, the aggregated session analysis in Table 1 involves 9 comparisons, and even a single significant correlation in 27 categories times 5 usage metrics would be expected by chance. If the reported p-values are raw, the headline significance claims would not survive FDR correction; if they are already FDR-adjusted, the values are suspiciously close to the 0.05 threshold. Either way, the paper does not report which p-values were adjusted or whether the tables show raw or corrected values, leaving the central category-level claims empirically underdetermined.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents an Android data-collection tool that retrieves the past seven days of app-usage events, and uses 124 Bangladeshi students' objective usage data and university-recorded CGPA to (i) correlate app-usage sessions and 27 app categories with CGPA, (ii) compare high/low CGPA holders and high/low users, and (iii) train regression models to predict CGPA. The headline claims are a significant negative association of session counts with CGPA, significant positive associations for Books and Productivity and negative for Video, no significant Social Media relation, and a KNN model whose test mean absolute error is 0.36 CGPA points on a held-out 30% of the data.","tokens_in":32724,"tokens_out":5993,"duration_ms":66320,"significance":"If the inferential claims survive re-analysis, the paper would be a useful contribution: objective 7-day app logs from a low-resource setting, direct category-level associations, and a lightweight predictor that does not require background sensing. Strengths include real usage events rather than self-report, actual CGPA from university records, open code and data-processing scripts, and a careful attempt to define sessions and diurnal patterns. However, two statistical issues—unreported or unverifiable multiple-comparison correction and outcome-based feature selection outside cross-validation—bear directly on the central claims and must be fixed before the results can be interpreted.","major_comments":[{"comment":"The manuscript states that p-values were adjusted using the false discovery rate approach, but every p-value shown in Tables 1–4 is a raw per-comparison p-value. For example, Table 1 has 9 comparisons; applying Benjamini–Hochberg to the smallest raw p=0.009 gives an adjusted p of about 0.081, which is not below 0.05. The session-level, Books, Productivity, and Video claims therefore cannot be evaluated as reported. Please state explicitly which analyses were FDR-adjusted, report the adjusted p-values or q-values, and show how many of the headline results survive correction.","section":"§4.3.1, Tables 1–4"},{"comment":"The 44 features used by the final models were selected by testing correlation with CGPA on the full dataset of 124 participants before the train/test split. This uses test-set outcome information to choose features, so the reported test MAE of 0.36 and correlation of rs=0.44 are optimistically biased. Feature selection must be nested inside the cross-validation loop or performed only on the training portion of each fold before test evaluation. Without this, the abstract's 'predicts CGPA within 0.36' claim is not supported.","section":"§4.3.3, Table 9"},{"comment":"The outcome is cumulative CGPA from university records, while the predictors are seven days of usage collected just before the Spring-2019 final exam. A single pre-exam week may not represent the behavior that produced a multi-semester CGPA, for example because students cram, change phone-use patterns, or experience network or device problems during that week. The paper does not address this temporal mismatch or provide robustness checks, so both the correlational and predictive claims may be confounded by the timing of measurement. Please justify the proxy or report sensitivity analyses that address this concern.","section":"§4.1–§4.2"},{"comment":"No baseline predictor is reported, so MAE=0.36 is hard to interpret. A model that always predicts the sample mean CGPA would have an MAE roughly equal to the mean absolute deviation of CGPA; reporting this baseline, along with R² or RMSE and the distribution of CGPA, would establish whether the app-usage features add predictive value beyond the marginal distribution.","section":"Table 9"}],"minor_comments":[{"comment":"'Within ±0.36' should be explicitly described as a mean absolute error on a 37-participant test set, not as a bound that holds for every individual student.","section":"Abstract and §5.5"},{"comment":"Several cells have very small sample sizes (e.g., Art & Design with N=3) and report p-values of exactly 1.0 or 0; consider suppressing categories below a pre-specified number of users or reporting exact coefficients with a clear caution.","section":"Table 3"},{"comment":"The Social Media rows report p=0.98 for four different usage metrics; please verify these values and report the actual test statistics and effect sizes, since identical p-values across different metrics are surprising and are likely an artifact of the test procedure.","section":"Table 6 and §5.3.2"},{"comment":"The Z-score outlier rule with a threshold of 3 is stated, but the paper does not report how many observations were removed in each analysis; this should be quantified for reproducibility.","section":"§4.3.1"},{"comment":"The 'light grey' shading for 'close to significant' results is not defined; please state the explicit p-value range or criterion used for this visual category.","section":"Tables 4 and 6"}],"recommendation":"major_revision","confidential_remarks":"The paper has real strengths in data collection and reproducibility, but the two main statistical issues affect the abstract's headline claims: the FDR adjustment is asserted but not visible in the reported p-values, and the feature-selection procedure leaks outcome information into the test evaluation. I recommend major revision rather than rejection because both problems are fixable within the scope of the manuscript, and the corrected analyses may still support the central associations."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a real empirical study with a working tool, open code, and a genuinely new data point, but the headline significance claims rest on raw-looking p-values and the prediction evaluation may be optimistic from feature selection. Both are fixable, but they are load-bearing.\n\nWhat it does well. The tool is minimal and fits the low-resource framing: it queries UsageStatsManager for the last 7 days in ~0.3 seconds, does not need a background service, and the authors validated it against manual calculation and third-party apps. Code and data are on OSF. The session-type analysis (micro/review/engage) is a reasonable behavioral dimension, and the non-result for total duration versus the significant session count is a genuinely useful nuance. The paper is also careful about consent, privacy, and situating itself against prior self-report work.\n\nThe soft spots. First, the stress-test note is right: the FDR statement appears in the comparative-analysis paragraph, not in the correlation analysis, and Tables 1-4 contain what look like raw p-values. With 27 categories, five metrics, and four time periods, several nominally significant correlations are expected by chance. The Books, Productivity, and Video claims need re-analysis with proper adjustment before being reported as significant. Second, feature selection uses correlations with CGPA before the train/test split, and the paper never says the selection is nested inside cross-validation. If the 44 features were chosen using all 124 labels, the test MAE of 0.36 is optimistic. Third, there is no baseline: predicting the mean CGPA for everyone would also produce an MAE, and the paper does not report it. Fourth, the 7-day window before Spring-2019 finals versus cumulative CGPA is a real but nonfatal threat; it deserves a limitations paragraph.\n\nBottom line: this paper is for HCI and learning-analytics researchers interested in low-cost, objective sensing in low-resource settings. It deserves a serious referee, but the referee should send it back for major revision: correct the multiple-testing story, make the feature-selection protocol explicit, add a baseline, and discuss the time-window mismatch. The raw material is honest and useful; the statistical wrapper is not yet trustworthy.","headline":"Solid low-cost app-usage study with open code, but the headline correlations and prediction claims need statistical cleanup before they can be trusted.","tokens_in":33331,"tokens_out":4143,"would_cite":true,"duration_ms":44668,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 7-day snapshot of Android app usage, retrieved in under a second, predicts students' CGPA within ±0.36, and reveals that app-usage sessions, Productivity/Books use, and Video use relate to grades in distinct directions.","keywords":["app usage","academic performance","CGPA prediction","Android usage logs","passive sensing","app usage sessions","app categories","machine learning"],"falsifier":"Collect the same seven-day app-usage snapshot and CGPA data from a new cohort in a different term, and check whether the session-count and Video correlations replicate; the central claim fails if a week without classes or during vacation removes the negative session association, or if a week-by-week longitudinal study shows that the seven-day features do not track semester GPA.","tokens_in":32306,"feed_emoji":"📱","tokens_out":4376,"duration_ms":43553,"temperature":0.7,"pith_summary":"This paper argues that a minimal, unobtrusive data grab—seven days of real Android app-usage events pulled in about a third of a second—is enough to both characterize how phone behavior relates to grades and predict a student's cumulative grade point average. Analyzing 124 Bangladeshi undergraduates, it finds that the number of app-usage sessions, especially short micro and review sessions, carries a significant negative association with CGPA, while time in Productivity and Books & Reference apps is positively associated and time in Video apps is negatively associated. Social Media shows no significant association. If the paper is right, a single week of passively logged app activity can support early academic intervention without long-term sensing or self-reports.","feed_headline":"7 days of app logs predict CGPA within ±0.36","feed_subtitle":"A one-second phone scan ties short app sessions and Video use to lower grades, Productivity and Books to higher.","key_machinery":"The load-bearing mechanism is the Android UsageStatsManager queryEvents API used to reconstruct raw foreground and background events for the past seven days without background sensing, combined with a session definition that joins app events separated by at most 45 seconds and splits sessions into micro (up to 15 seconds), review (15 to 60 seconds), and engage (over 60 seconds) types. Events are aggregated into 27 app categories and five time periods, producing 720 behavioral features; the authors select 44 features by significant correlation with CGPA and feed them into regression models, with KNN performing best. The key behavioral signal is session count: duration and launch frequency do not correlate with grades, but session counts do.","core_discovery":"The claimed discovery is that cumulative grade point average is readable from just the last seven days of app-usage events, and that the relationship is not one-dimensional. Aggregated measures such as total duration and launch count show no significant correlation with CGPA; instead, the total number of app-usage sessions, micro sessions, and review sessions does (Spearman coefficients around -0.20 to -0.24, p<0.05), with afternoon and evening sessions driving the effect. At the category level, Books & Reference and Productivity usage are significantly positively correlated with CGPA, while Video Players & Editors usage is significantly negatively correlated; Social Media is not. High and low CGPA holders differ in Productivity and Video behavior, and a K-nearest-neighbor model built from 44 correlation-selected features predicts CGPA with a mean absolute error of 0.36 on held-out data.","pith_inferences":["Editorial inference: because CGPA is cumulative, a fairer test of the seven-day signal would be predicting semester GPA from the same week's logs; the ±0.36 error against CGPA may understate how tightly behavior tracks recent performance.","Editorial inference: the session-count result suggests a testable design—if students are nudged to consolidate phone use into fewer, longer sessions, and if the association is causal, grades should improve even without reducing total screen time.","Editorial inference: the morning-Video negative correlation implies that timing-aware interventions, such as deferring video watching to after class, could be more effective than blanket video blocks; this is an extension the paper does not test.","Editorial inference: the authors mention a second dataset collected during the COVID-19 pandemic; comparing category associations across pandemic and non-pandemic periods would test whether the Video and Books effects are stable or context-dependent."],"forward_implications":["A student's phone can be read once, in under a second, and yield a grade forecast within ±0.36 of the actual CGPA, enabling early intervention before a semester's grades are final.","Interventions should target the number of app-usage sessions rather than total screen time; reducing micro and review sessions in the afternoon and evening may matter more than cutting overall duration.","Encouraging Productivity and Books & Reference use, and discouraging Video use—particularly morning video—should correlate with better outcomes if the associations reflect a modifiable pathway.","High and low CGPA students can be distinguished by category-level usage patterns, supporting personalized feedback and intervention designs."],"supporting_citations":[{"why":"Provides the predictive baseline the paper contrasts against: a semester-long sensing study that predicted CGPA within ±0.17, motivating the claim that seven days is faster and minimalistic.","marker":"[15]"},{"why":"Supplies the median-break and session-threshold approach for defining app-usage sessions and the prior finding that Facebook-checking context relates to academic performance.","marker":"[1]"},{"why":"Supplies the engage-session concept and the classification of session types that the paper adapts into micro, review, and engage sessions.","marker":"[67]"},{"why":"Provides the evidence that a 30-second threshold is less accurate and supports the paper's choice of a 45-second gap to define a single session.","marker":"[66]"},{"why":"Supplies experimental evidence that cell phone usage, especially during class time, relates negatively to academic performance, motivating the objective-data approach.","marker":"[5]"},{"why":"Provides the Bangladesh-specific prior finding that high smartphone users have lower CGPA, which the paper extends with statistical evidence from actual usage data.","marker":"[6]"},{"why":"Supplies the prior result that time spent on Facebook is significantly negatively related to academic performance, a benchmark for the Social Media category analysis.","marker":"[27]"},{"why":"Provides evidence that entertaining apps and media multitasking relate negatively to GPA, supporting the Video category hypothesis and the multitasking interpretation.","marker":"[13]"},{"why":"Documents the Android UsageStatsManager queryEvents function that the paper's data-collection tool relies on to retrieve raw app-usage events.","marker":"[53]"}],"fun_headline_variants":["7-day app session data predicts CGPA within 0.36","Video apps linked to lower CGPA, productivity and books to higher","Just 7 days of app logs enough to forecast grades","App sessions, not screen time, correlate with grades","High and low CGPA students show distinct app behavior"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that one week of app usage, collected just before the Spring-2019 final exam, represents the long-term behavior that produced a cumulative CGPA; if that week is atypical, the correlations and predictions measure a transient state against a lifetime average.","fun_headline_variants_meta":{"raw":{"variants":["7-day app session data predicts CGPA within 0.36","Video apps linked to lower CGPA, productivity and books to higher","Just 7 days of app logs enough to forecast grades","App sessions, not screen time, correlate with grades","High and low CGPA students show distinct app behavior"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001281,"raw_usage":{"total_tokens":5214,"prompt_tokens":901,"completion_tokens":4313,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":517,"completion_tokens_details":{"reasoning_tokens":4230}},"tokens_in":517,"tokens_out":4313,"duration_ms":34275,"temperature":1.0,"reasoning_tokens":4230,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:10:10.682308+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect the same seven-day app-usage snapshot and CGPA data from a new cohort in a different term, and check whether the session-count and Video correlations replicate; the central claim fails if a week without classes or during vacation removes the negative session association, or if a week-by-week longitudinal study shows that the seven-day features do not track semester GPA.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies experimental evidence that cell phone usage, especially during class time, relates negatively to academic performance, motivating the objective-data approach."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the prior result that time spent on Facebook is significantly negatively related to academic performance, a benchmark for the Social Media category analysis."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides evidence that entertaining apps and media multitasking relate negatively to GPA, supporting the Video category hypothesis and the multitasking interpretation."}],"review_version":1}