{"id":"a7e11885-1a12-4d3c-9f28-adb3bf671253","arxiv_id":"2504.14927","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"The paper claims that a lightweight fusion of gesture, audio, and slide features can predict segment access frequency in lecture archives, but the experiments supporting these numbers are not included in this preprint.","lead":"This paper describes a multimodal machine learning system that predicts how often students replay one-minute segments of recorded lectures, using only non-semantic cues such as teacher movement, voice spectrograms, and slide changes. The authors claim a Pearson correlation of 0.5143 and 69.32% classification accuracy, but the preprint's experimental section is empty.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central performance claim is unsupported: Section 4 is explicitly under development, so the abstract's PCC=0.5143 and 69.32% accuracy are promissory numbers with no inspectable results.","rationale":"The reader's REJECT verdict is appropriate: the abstract reports empirical results that the manuscript does not present. In stress-test terms, the strongest claim is empirical and must be evidenced by Section 4; since Section 4 is explicitly empty, the claim fails under the claim-without-derivation test. My emphasis differs slightly from the reader's weakest_assumption: the reader stresses the small number of valid viewers per lecture, whereas I stress the missing experimental section and the ambiguity about whether the headline metrics were computed on raw or smoothed signals. Both are real, but the smoothing ambiguity is the one most likely to survive even after the experiments are added, because the paper already describes two independent smoothing steps without specifying which signal enters the headline statistics. The concrete test is therefore to require per-fold results and a raw-label evaluation. No code, data, or formal verification is provided, so there is no independent support to offset the missing evidence. I mark agreement as partial because I endorse the verdict while routing the concern through a different technical hinge.","tokens_in":9207,"tokens_out":6285,"duration_ms":63859,"concrete_test":"Obtain from the authors the completed Section 4 plus the exact evaluation script and per-fold predictions, then recompute PCC and three-class accuracy from raw, unsmoothed per-segment labels on held-out lessons. If the abstract's numbers cannot be reproduced, or are reproduced only after smoothing both labels and predictions, the central claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is the pair of numbers in the abstract (PCC=0.5143, 69.32% accuracy). The only experimental section, Section 4, states: \"This section is currently under development and will be completed in a subsequent version.\" Sections 5 and 6 are \"To be completed.\" No table, figure, per-fold breakdown, or evaluation script appears anywhere. The claim therefore has no inspectable support. The problem is not just missing prose: the metric's definition is also ambiguous. Section 3.1.2 smooths the label sequence with a centered five-segment moving average and normalizes within each lecture; Section 3.2.4 applies post-hoc moving-average, Savitzky-Golay, or Kalman smoothing to predictions. The abstract does not state whether PCC and accuracy are computed on raw or smoothed sequences. Smoothing both sides can inflate correlation and accuracy relative to true per-segment prediction, so the headline number could overstate real predictive skill. Consequently, the central feasibility claim is a promissory note rather than a reported result.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a multimodal neural-network framework to predict per-minute segment access frequency in lecture archives using deliberately non-semantic features: instructor action maps (OpenPose plus optical flow), audio spectrograms, and slide-progression features. The authors describe label generation from JAIST LMS viewing logs for seven lectures, yielding 665 one-minute segments after trimming/padding, with access frequencies smoothed and normalized per lecture. They outline two fusion strategies (feature stacking into RGB images and model stacking), a set of backbone comparisons (VGG, ResNet, ViT, CNN+LSTM), and a 7-fold lesson-level cross-validation protocol. The abstract reports a best Pearson correlation of 0.5143 and 69.32% three-class accuracy. However, Section 4, which is supposed to report the experiments, consists only of the sentence \"This section is currently under development and will be completed in a subsequent version,\" and Sections 5 and 6 are marked \"To be completed.\" Consequently, none of the claimed numerical results are inspectable in the manuscript.","tokens_in":9398,"tokens_out":6184,"duration_ms":55941,"significance":"If substantiated, the contribution would be practically valuable: a language-independent, hardware-free method for predicting engagement-relevant segments in unedited, low-resource lecture archives would address a real need in institutions that record face-to-face lectures without transcripts or eye-tracking. The feature choices are well grounded in prior literature, the automatic label-generation pipeline is a sensible way to avoid manual annotation, and the paper honestly acknowledges the very small dataset. The main difficulty is that the central empirical claims appear only in the abstract and have no supporting experimental content in the body; until the results are actually reported, the significance of the work cannot be assessed.","major_comments":[{"comment":"The entire empirical basis for the paper's central claim is missing: Section 4 consists only of \"This section is currently under development and will be completed in a subsequent version,\" and Sections 5 and 6 are \"To be completed.\" No table, figure, per-fold result, error bar, ablation result, or evaluation detail is provided. The abstract's Pearson correlation of 0.5143 and 69.32% accuracy are therefore unsupported assertions rather than reported findings. This is load-bearing because the paper's feasibility claim (RQ1) and the fusion and backbone comparisons (RQ2 and RQ3) depend entirely on experiments that are not present.","section":"Section 4; Abstract"},{"comment":"The metric definitions are ambiguous with respect to smoothing. Labels are smoothed with a centered five-segment moving average and normalized within each lecture, while predictions are post-processed with moving-average, Savitzky-Golay, or Kalman smoothing before thresholding. The abstract does not state whether the reported PCC and three-class accuracy are computed on raw or smoothed sequences. Since smoothing on both sides can inflate correlation and threshold agreement, the headline numbers may overstate true per-segment predictive skill. The authors should report raw and smoothed metrics separately.","section":"§3.1.2, §3.1.3, §3.2.4"},{"comment":"The target variable is derived from an average of only 8.71 valid viewers per lecture after applying filters that exclude viewing records shorter than one minute and students with less than five minutes of total viewing. With so few contributing viewers, segment access frequency is highly sensitive to individual review behavior, and no stability or agreement analysis is provided. The paper itself acknowledges the small dataset, but the validity of the engagement proxy is load-bearing: if these few viewers are not representative, both the reported prediction performance and its interpretation as engagement prediction are called into question. The authors should report per-segment viewer counts or a leave-one-viewer-out label-stability analysis.","section":"Table 1; §3.1.2"},{"comment":"The text states that \"After confirming the superiority of feature-level fusion over model-level fusion in earlier experiments,\" all subsequent evaluations use feature-level fusion, but no such earlier experiments are reported anywhere in the manuscript; Section 4 is empty. This unsupported assertion determines the experimental design for RQ2 and RQ3 and must either be backed by reported experiments or removed. As written, this is a circular reference to results that do not appear.","section":"§3.3.2"},{"comment":"The three-class thresholds (high > 0.5, low < 0.2, medium in between) are arbitrary and are applied after per-lecture normalization and temporal smoothing, so the class balance and the 69.32% accuracy figure depend on these choices. No sensitivity analysis or class-distribution table is provided. Because this classification accuracy is one of only two headline results, the threshold dependence should be quantified.","section":"§3.1.3"}],"minor_comments":[{"comment":"The reference to Kim et al. is rendered as \"[ ?]\" and should be completed.","section":"§2.1"},{"comment":"In the research question list, \"RQ2: RQ2:\" is duplicated and should be corrected.","section":"Introduction"},{"comment":"The optical flow citation [19] (Burton and Radford, 1978) appears mismatched; the Lucas-Kanade method [21] is the relevant citation. The relation between masking the students' seating area and suppressing slide corner points should also be clarified.","section":"§3.2.1"},{"comment":"The caption \"Table 1 shows the statistics of valid viewers and total valid viewing time per lecture\" repeats the surrounding text; a simpler descriptive caption would be clearer.","section":"Table 1"},{"comment":"The definition of Praw[i] = max{xi} - max{xi-1} is not fully specified: the set over which the maximum is taken should be stated explicitly (e.g., all slide-page values within a five-minute segment). Without this, the feature extraction is not reproducible.","section":"§3.2.3"}],"recommendation":"reject","confidential_remarks":"The manuscript is an incomplete draft: Section 4 is empty and Sections 5 and 6 are marked \"To be completed.\" The abstract's quantitative claims have no in-paper support, so the paper cannot be evaluated as a research contribution. I would not invite a revision until the experiments are actually reported; if the authors complete the experimental section, the label-stability and smoothing issues should be addressed in that revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou should know this is an incomplete submission: Section 4, the experiment section, is literally 'under development', and Discussion and Conclusion are 'To be completed'. The only reported results are two numbers in the abstract (PCC 0.5143, 69.32% accuracy), with no table, figure, per-fold breakdown, or code anywhere to inspect. So as a research report, it is not done. The reader's REJECT verdict is the defensible call.\n\nThat said, the paper is not empty of substance. The setup is a real, under-served problem: unedited lecture archives from real classrooms, no usable transcripts, small enrollments. The label definition (segment access frequency from LMS playback logs) and the use of non-semantic signals—OpenPose/optical flow on the instructor, audio spectrograms, slide progression—are a sensible and relatively new combination. The method section describes preprocessing, fusion designs, and backbones in enough detail that the pipeline could be re-implemented. If the promised results materialize and the design works, it would be a useful applied contribution.\n\nThe soft spots are significant. The most obvious is missing evidence. There are also measurement choices that could flatter performance: labels are smoothed with a centered five-segment moving average and normalized per lecture, and predictions get post-hoc moving-average, Savitzky-Golay, or Kalman smoothing. The abstract does not say whether reported PCC and accuracy were computed on smoothed sequences—if both sides are smoothed, correlation can be inflated. That ambiguity needs to be resolved in the full version. The label population is another concern: on average only 8.71 valid viewers per lecture after filtering. The authors acknowledge this, but they do not yet show whether the target variable is stable or whether predictions generalize beyond one course at one institution. There is also a dangling citation marker in Section 2.1, and Section 3.3.2 refers to 'earlier experiments' that do not appear anywhere. These are minor production issues by comparison, but they corroborate the incompleteness.\n\nThe citation pattern looks relevant and honest. No code or data are released, which for a paper whose central claims are two numbers is a real gap.\n\nWho gets value? Someone working on engagement prediction for low-resource lecture settings might read the method section as a design proposal. No one should treat the abstract numbers as evidence yet. My recommendation: this should not go to peer review in its current form. Ask the authors to complete the experiment, discussion, and conclusion first. If they return with real results, a serious referee should look at it, with particular attention to the smoothing question and the small-label problem.","headline":"An incomplete preprint with a genuinely plausible setup and no supporting experiments; the headline correlation and accuracy are promissory numbers.","tokens_in":9935,"tokens_out":3443,"would_cite":false,"duration_ms":30553,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Non-semantic teacher motion, audio, and slide cues predict which lecture segments students revisit, reaching a Pearson correlation of 0.5143.","keywords":["lecture archives","segment access frequency","multimodal fusion","non-semantic features","student engagement","action features","audio spectrogram","slide progression"],"falsifier":"Compute the same features and labels for a lecture with at least fifty valid viewers, train the identical early-fusion model, and measure Pearson correlation on held-out segments; if the correlation falls to near zero or fails to beat a baseline that predicts the per-lecture mean, the reported result is an artifact of the tiny, self-selected viewer pool rather than a robust engagement signal.","tokens_in":8978,"feed_emoji":"🎓","tokens_out":7924,"duration_ms":69553,"temperature":0.7,"pith_summary":"This paper tries to show that segment-level student engagement in unedited lecture archives can be predicted from non-semantic signals alone: what the teacher does, how the voice sounds, and when slides change. It defines engagement as segment access frequency, the number of times each one-minute segment is played back by students in a small real classroom archive. The reported best model, built by stacking action, voice, and slide features into the RGB channels of a single image and running a residual convolutional network, reaches a Pearson correlation of 0.5143 in seven-fold lesson-level cross-validation and 69.32% accuracy in a three-class attention-zone task. If these numbers hold, lecture platforms could highlight likely high-engagement segments automatically, without transcripts, manual annotation, or extra hardware.","feed_headline":"Non-semantic cues predict which lecture segments students revisit","feed_subtitle":"Fusing teacher motion, audio, and slide cues reaches 0.5143 correlation and 69.32% accuracy on replay prediction.","key_machinery":"The load-bearing mechanism is feature-level fusion by channel stacking: the action map (motion trajectories of the instructor's pose), the slide-progression map, and the audio spectrogram are each resized and packed into the three RGB channels of one 480×320 image, then processed by a single convolutional network trained with MSE loss. This design lets the model reuse well-established image backbones, keeps training cheap, and deliberately avoids semantics, so the pipeline is language-independent. Labels are produced in the same automated pipeline: one-minute access counts from aggregated playback logs, filtered, smoothed, and normalized per lecture.","core_discovery":"The central claim is that the fine-grained pattern of which one-minute segments students revisit in a real lecture archive is predictable from how the instructor moves, how the audio spectrum evolves, and how the slide progresses—none of which requires understanding the lecture's content. The authors ground this in an automatically generated label: segment access frequency, computed from playback logs after filtering to valid viewers and smoothed with a five-segment centered moving average, then normalized per lecture to $[0,1]$. Their best configuration, early feature fusion through a residual convolutional backbone, achieves 0.5143 Pearson correlation and 69.32% three-class accuracy, and ablation results point to the action modality as the main driver.","pith_inferences":["The manuscript's own experiment, discussion, and conclusion sections are marked as to be completed, so the headline numbers come from the abstract and methodology rather than a fully written empirical section; those quantitative claims should be treated as provisional until the full write-up appears.","Because the labels are averaged over only about 8.71 valid viewers per lecture, the reported 0.5143 correlation is probably an upper bound for what the features can achieve with cleaner, larger-cohort labels.","The five-minute moving-average smoothing of labels means the model is trained to predict broad engagement zones, not exact moments; finer-grained logs could reveal whether the features align with conceptual boundaries such as new definitions or examples.","A practical follow-up is to use the predicted engagement curve to power automatic skip-and-seek recommendations, an application the paper states as motivation but does not evaluate."],"forward_implications":["Lecture platforms could auto-generate engagement heatmaps or 'most replayed' markers from raw recordings, with no transcripts or manual annotation.","The non-semantic design should carry across languages and lecture domains, as long as teacher motion, audio, and slide changes are present.","Early fusion is enough: the single-stream RGB-stacking model is reported to beat parallel per-modality networks, so deployment remains lightweight.","Instructors could use segment-level replay predictions as feedback about which parts of a lecture students found worth revisiting."],"supporting_citations":[{"why":"Supplies the population-level engagement-score labeling concept that the paper adapts from whole-video to segment-level access frequency.","marker":"[13]"},{"why":"Shows repeated viewing and interaction peaks signal focused engagement, motivating the use of playback-derived access frequencies as meaningful labels.","marker":"[11]"},{"why":"Eye-tracking study showing instructor body language and auditory cues affect students' visual attention, motivating the action and voice modalities.","marker":"[18]"},{"why":"Establishes viewing duration as an engagement proxy and shows production choices affect engagement, the baseline idea this work extends.","marker":"[12]"},{"why":"Provides the pose-estimation method used to extract instructor body-structure features for the action modality.","marker":"[20]"},{"why":"Provides the optical-flow algorithm used to compute instructor motion from the pose keypoints.","marker":"[21]"},{"why":"Describes the institutional lecture-archive system that supplied the seven recorded lectures used in the experiments.","marker":"[28]"}],"fun_headline_variants":["Motion, audio, slides predict lecture replay hotspots","Non-semantic cues forecast lecture segment views","Teacher motion and audio predict student revisits","Predicting lecture rewatches from multimodal cues","Lecture replay prediction without content understanding"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The prediction target itself—segment access frequency averaged over roughly nine valid viewers per lecture—must faithfully represent true student engagement; if those few viewers are atypical, then even a high correlation only means the model learned the habits of a handful of students.","fun_headline_variants_meta":{"raw":{"variants":["Motion, audio, slides predict lecture replay hotspots","Non-semantic cues forecast lecture segment views","Teacher motion and audio predict student revisits","Predicting lecture rewatches from multimodal cues","Lecture replay prediction without content understanding"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000523,"raw_usage":{"total_tokens":2513,"prompt_tokens":912,"completion_tokens":1601,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":1535}},"tokens_in":528,"tokens_out":1601,"duration_ms":10671,"temperature":1.0,"reasoning_tokens":1535,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:36:36.872891+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the same features and labels for a lecture with at least fifty valid viewers, train the identical early-fusion model, and measure Pearson correlation on held-out segments; if the correlation falls to near zero or fails to beat a baseline that predicts the per-lecture mean, the reported result is an artifact of the tiny, self-selected viewer pool rather than a robust engagement signal.","supporting_citations":[{"cited_title":"Understanding in-video dropouts and interaction peaks in online lecture videos","cited_arxiv_id":null,"evidence_quote":"Shows repeated viewing and interaction peaks signal focused engagement, motivating the use of playback-derived access frequencies as meaningful labels."},{"cited_title":"Effects of video instructor’s body language on students’ visual attention: An eye-tracking study","cited_arxiv_id":null,"evidence_quote":"Eye-tracking study showing instructor body language and auditory cues affect students' visual attention, motivating the action and voice modalities."},{"cited_title":"How video production affects student engagement: An em- pirical study of MOOC videos","cited_arxiv_id":null,"evidence_quote":"Establishes viewing duration as an engagement proxy and shows production choices affect engagement, the baseline idea this work extends."},{"cited_title":"OpenPose: Real-time multi-person 2D pose estimation using Part Affinity Fields","cited_arxiv_id":null,"evidence_quote":"Provides the pose-estimation method used to extract instructor body-structure features for the action modality."},{"cited_title":"An iterative image registration technique with an application to stereo vision","cited_arxiv_id":null,"evidence_quote":"Provides the optical-flow algorithm used to compute instructor motion from the pose keypoints."},{"cited_title":"Case studies for self- directed learning using lecture archives","cited_arxiv_id":null,"evidence_quote":"Describes the institutional lecture-archive system that supplied the seven recorded lectures used in the experiments."}],"review_version":1}