{"id":"134b16c8-91e2-4abb-8157-86f46efe1eba","arxiv_id":"2504.20677","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A pipeline combining gaze classification, driver identification, and a new occlusion detection module, trained on the Driver Monitoring Dataset, with RGB models outperforming IR models.","lead":"This paper builds a driver monitoring system that tracks where the driver is looking, identifies the driver, and detects when the driver's face is blocked from the camera. The system uses separate RGB and infrared camera models and is designed to alert the car when monitoring is unreliable.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Occlusion classifier is trained on MTCNN face crops but deployed on full frames when no face is detected, so the reported 99.43% recall does not validate the integrated pipeline.","rationale":"The reader's concern about undefined DMD occlusion labels is genuine but external; the stress test found a more internal, textually checkable gap. Section IV chooses the crop-trained RGB occlusion model for its higher recall, while Section VI routes no-face frames to the occlusion detector, so the deployed model sees full images at exactly the time it is used. This is directly load-bearing for the central claim of a functioning occlusion-aware DMS: if the 99.43% recall does not transfer, the system can fail in precisely the cases it is designed to catch. The fix is a straightforward evaluation change, so the reader's CONDITIONAL verdict remains appropriate rather than moving to REJECT: the system design is plausible and the reported numbers are honest, but the headline metric is measured under a different operating condition than the pipeline uses. Agreement with the reader is partial because both concerns are about evaluation validity, but the reader centers on label semantics while this review centers on the training/deployment input distribution mismatch.","tokens_in":8454,"tokens_out":13067,"duration_ms":123481,"concrete_test":"Re-run the final RGB occlusion model on the withheld occlusion test set exactly as the pipeline operates: for each test frame where MTCNN detects no face, feed the full uncropped frame to the occlusion model and recompute occlusion recall. If this recall is materially below the reported 99.43%, the integrated system's occlusion-detection performance is unvalidated, and the paper must either adopt the full-image model or retrain and re-report metrics.","verdict_should_be":"UNCHANGED","load_bearing_attack":"For the occlusion-aware DMS claim to hold, the RGB occlusion classifier must be reliable at the exact point where the pipeline invokes it. Section VI states that if the MTCNN face detector finds no face in the RGB image, the image goes to the occlusion detector. However, the RGB occlusion model selected in Section IV was trained on MTCNN-cropped face images, and such a crop exists only when face detection succeeds. At inference in the no-face branch, the model receives a full frame, a different input distribution from its training data. The reported 99.43% occlusion recall was measured on cropped test images; the earlier full-image RGB model had a lower recall of 95.34% and is the one that would match full-frame deployment, yet the paper does not say it is the integrated model. The real-life tests in Section VI are descriptive and unquantified, so they do not establish that the crop-trained model transfers to full frames. Even granting that the DMD occlusion labels used in Section II.B encode genuine visual occlusions, the headline recall does not support the integrated system claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an occlusion-aware driver monitoring system (DMS) built on the Driver Monitoring Dataset (DMD). The system combines three components: gaze estimation into nine regions, driver identification via face embeddings, and face occlusion detection, each implemented for both RGB and IR images. The authors report test accuracies of 86.29% for RGB gaze classification, 76.76% for IR gaze classification, 99.43% occlusion recall for the RGB crop-trained model, 90.24% for the IR occlusion model, and 99.38% identification accuracy with low FAR/FRR. The pipeline prioritizes RGB processing, falls back to IR in low-light conditions, and invokes the occlusion detector when no face is detected, alerting the driver if occlusion persists. The paper claims to be the first academic effort to introduce occlusion detection from images for driver monitoring, aligned with EuroNCAP recommendations.","tokens_in":8657,"tokens_out":3541,"duration_ms":36381,"significance":"If the occlusion detection functionality works as claimed, it addresses a genuine safety gap in DMS: alerting when the system cannot see the driver, as recommended by EuroNCAP. The paper's strengths include use of a public dataset, evaluation of multiple architectures, explicit reporting of recall (which is appropriate for imbalanced occlusion data), and a described end-to-end pipeline with both RGB and IR branches. The quantitative results on the DMD are plausible, and the subject-disjoint train/test splits for gaze data reduce the risk of identity leakage. However, the central claim is undermined by two methodological gaps: the occlusion label definition is never specified, and the best-performing RGB occlusion model is trained on face crops but deployed on full frames in the no-face branch. These issues are fixable but require additional work, so the paper is not ready for acceptance in its current form.","major_comments":[{"comment":"The occlusion labels are never defined. The text states that images were taken from the distraction-annotated part of the DMD because it contained 'a higher number of occluded images,' but it does not specify how occlusion was labeled, who labeled it, what annotation protocol was followed, or whether the labels correspond to visual obstructions of the face/eyes versus other distraction states such as hands on the wheel or looking away. Since the central claim of the paper is that occlusion detection ensures proper DMS functioning, the semantics of the occlusion label are load-bearing. If the labels encode a different concept, the reported 99.43% recall does not measure the claimed capability. Please describe the DMD occlusion annotation protocol, provide examples of positive and negative images, and ideally report a small human-verification study of the labels.","section":"Section II.B"},{"comment":"There is a train/deploy distribution mismatch for the RGB occlusion detector. The model with 99.43% occlusion recall was trained on MTCNN-cropped face images, but the pipeline described in Section VI sends the full RGB frame to the occlusion detector exactly when no face is detected. A crop exists only when face detection succeeds, so the no-face branch at inference receives a different input distribution than the training data. The earlier full-frame RGB model had a lower recall of 95.34%, and the paper does not state that this full-frame model is the one used in the integrated no-face branch. The headline 99.43% recall therefore does not validate the integrated pipeline. Please either deploy the full-frame model in the no-face branch and report its performance, or retrain/evaluate the crop-trained model on full-frame no-face images, or provide evidence that the distribution shift is negligible.","section":"Section IV and Section VI"},{"comment":"The IR gaze classifier was retrained after removing images with mean brightness above 235 or below 20, and the reported 76.76% test accuracy is measured on this filtered test set. The paper acknowledges that the unfiltered model had 84.70% accuracy but poor real-world performance, and that the filtered model was chosen because it was 'more representative of real-world conditions.' However, this filtering is an ad hoc, post-hoc exclusion of extreme lighting cases, and no evaluation is reported on the excluded images. Since the paper claims the system functions in challenging low-light scenarios, please quantify how the deployed IR model behaves on the removed extreme-brightness images, or specify the conditions under which the pipeline can be expected to operate.","section":"Section III (IR gaze classifier)"},{"comment":"The real-life tests are described only qualitatively. Statements such as 'the RGB occlusion classifier made accurate predictions in all scenarios we tested' and 'the IR gaze classifier did not perform as well' are not supported by quantitative data. There is no mention of the number of subjects, number of trials per scenario, or any metric such as accuracy or recall. Please provide a tabulated evaluation of the real-life tests, or weaken the conclusive statements about real-world reliability.","section":"Section VI (real-life tests)"}],"minor_comments":[{"comment":"Hyphenation is inconsistent: 'occlusion -aware' and 'gaze estimation by regions' appear with stray spaces in the abstract and introduction.","section":"Abstract and Section I"},{"comment":"Equation (1) is malformed in the manuscript; the fractions 4/5 and 1/5 are not legible due to formatting. Please ensure the equation renders correctly.","section":"Equation (1)"},{"comment":"The phrase 'a three-layer classification layer (instead of one)' is unclear; please specify the exact architecture change.","section":"Section IV"},{"comment":"The initial MobileNet_v3_small gaze model underfitted with 59.12% accuracy, but no training details (learning rate, epochs, batch size) are given for either the underfitting or successful models. Providing a table of hyperparameters would improve reproducibility.","section":"Section III"},{"comment":"Several references are incomplete or have formatting issues, e.g., reference [14] lacks a venue, and reference [6] has a typo in the page range ('4318-432').","section":"References"},{"comment":"The captions for Figures 8 and 9 could be more explicit about which model produced each prediction and what the color coding or region labels mean.","section":"Figures 8 and 9"}],"recommendation":"major_revision","confidential_remarks":"The paper's novelty claim—that this is the first academic occlusion detection method for DMS—is strong and should be verified against a broader literature search beyond the cited references. The two main technical gaps (label semantics and train/deploy distribution shift) are addressable and do not, in my view, require rejection, but they are load-bearing for the central claim. I would also encourage the editor to ask the authors to make the DMD occlusion annotation protocol explicit, possibly by consulting the DMD documentation, since the paper's credibility depends on it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First impression: this is a straightforward integration paper with one genuinely useful result — the RGB/IR performance comparison — and one claim that outruns its evidence, the 99.43% occlusion recall. The stress-test note is right: the selected RGB occlusion model was trained on MTCNN face crops, but in the integrated pipeline the occlusion detector is invoked on a full frame whenever the face detector finds no face. A crop exists only because a face was detected. So the reported recall was measured on cropped test images; the full-frame model achieved 95.34% and is not what the paper says is integrated. The real-life tests are descriptive and unquantified, so they don't bridge that gap. This is a load-bearing flaw for the integrated system claim.\n\nWhat the paper does well: it builds a coherent DMS pipeline from standard components (EfficientNet, MobileNet, MTCNN, FaceNet), evaluates on the public DMD, and is honest about the numbers. The RGB-versus-IR finding is useful and aligns with the fact that IR models are harder to pretrain. The explicit occlusion-aware alert is a sensible EuroNCAP-aligned feature. The paper also notes its own limitations, for example that IR real-world performance was poor and that they excluded extreme lighting cases after inspection.\n\nThe soft spots beyond the train/deploy mismatch are real. The occlusion labels are taken from the distraction-annotated part of the DMD, but the paper never says how they were created or verified. If those labels reflect hand-on-wheel or other distraction states rather than visual obstruction of the face, the reported occlusion recall does not measure the claimed capability. The data cleaning (manual removal of gaze mismatches, brightness cutoffs) is post hoc and not fully specified. The cosine thresholds and MTCNN confidence are tuned without a validation procedure. And the 'first attempt' novelty claim is not supported by a literature survey; there is prior work on occlusion-robust face recognition and DMS robustness.\n\nNone of these are fatal to the paper's value as an engineering report. The integration is clean, the empirical results are plausible, and the RGB/IR comparison is worth reporting. But the paper should be revised before it is a reliable reference for occlusion-aware DMS. At minimum, the authors need to specify the occlusion label generation, evaluate the no-face branch on full frames, and soften the novelty claim.\n\nMy recommendation for peer review: send it out. It needs major revision, but the reviewer time would be well spent — the issues are identifiable and fixable, and the topic is of practical importance.","headline":"Useful integration paper with a load-bearing occlusion-evaluation mismatch: crop-trained model deployed on full frames, plus unspecified occlusion labels.","tokens_in":9203,"tokens_out":2481,"would_cite":false,"duration_ms":23611,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A driver-monitoring pipeline that flags when the camera cannot see the driver's face, keeping gaze and identification honest.","keywords":["driver monitoring system","occlusion detection","gaze estimation by regions","driver identification","RGB and infrared imaging","deep learning","low-light driving"],"falsifier":"Take a random sample of distraction-annotated frames from the public multi-modal driver monitoring dataset and have independent annotators decide whether each 'occlusion' label shows a visible face obstruction. If many labels correspond to hand-on-wheel or other distraction states, the reported 99.43 percent occlusion recall is not a measure of the claimed capability. Alternatively, run the RGB occlusion classifier on a controlled set of partially occluded faces; if recall drops sharply when the face is only partly covered, the practical robustness is narrower than the headline figure.","tokens_in":8255,"feed_emoji":"🚗","tokens_out":8586,"duration_ms":83859,"temperature":0.7,"pith_summary":"This paper claims that a driver monitoring system can be made reliable under occlusion by adding a dedicated face-occlusion classifier that gates the other tasks. The proposed pipeline performs driver identification, gaze-region estimation, and occlusion detection on RGB and infrared images, switching to the IR channel when RGB face detection fails and warning the driver when no channel can see the face. The headline numbers are 86.29 percent RGB accuracy on nine gaze regions, 99.43 percent recall for occluded faces after face cropping, and 99.38 percent identification accuracy with a 0.81 percent false-accept rate. The authors argue that image-based occlusion detection is new for driver monitoring and that it matters because a system that knows when it cannot see is safer than one that guesses. If these results transfer to the car, an in-vehicle system could keep monitoring through darkness, glare, and partial face obstruction.","feed_headline":"Driver monitoring flags occluded faces before gaze or ID goes wrong","feed_subtitle":"A three-task RGB/IR pipeline alerts when the camera cannot see the driver, then falls back or warns.","key_machinery":"The mechanism carrying the argument is an RGB-first pipeline with an occlusion-gated fallback to infrared. In each frame, an MTCNN face detector processes the RGB image; if a face is found, the crop is sent to the gaze classifier and the identity embedder, while the IR frame is held in reserve. If no face is found, a binary occlusion classifier decides whether an obstruction is present; only when no occlusion is detected may the system reuse the previous frame's bounding box, expanded by 20 percent. Persistent occlusion or repeated face-detection failure triggers a switch to the IR pipeline, and persistent occlusion there produces a driver alert and stops gaze and identification. The individual algorithms are EfficientNet-b0 for gaze regions, a MobileNet-v3-based binary classifier for occlusion, and FaceNet embeddings for identity.","core_discovery":"On the paper's own terms, the discovery is that a modular, calibration-free pipeline can combine gaze classification, driver identification, and occlusion detection, and that occlusion detection is what holds the other two together. The RGB gaze classifier uses an EfficientNet-b0 backbone on MTCNN-cropped faces and reaches 86.29 percent test accuracy across nine gaze regions; the RGB occlusion classifier reaches 99.43 percent recall on the occlusion class when given cropped faces, and the IR version reaches 90.24 percent; the identification module uses FaceNet embeddings with a cosine threshold and reports 99.38 percent accuracy on fifteen registered and ten unregistered individuals. The authors state that the RGB models consistently outperform the IR models, which they attribute to the lack of pretrained infrared models and to sensor-dependent IR image quality. They further claim this is the first academic attempt at image-based occlusion detection for driver monitoring, with the purpose of alerting the driver when the system cannot function correctly.","pith_inferences":["Because the occlusion training images came from a distraction-annotated partition, the same data could support a joint label that distinguishes 'driver not looking' from 'camera cannot see'; the current binary occlusion output leaves that distinction implicit.","A natural controlled experiment would be to run the pipeline with the RGB camera artificially covered and the IR channel dimmed, measuring how often gaze or identity still returns a wrong answer instead of an occlusion alert.","A temporal filter over consecutive occlusion predictions would likely remove the false alarms the authors observed when a hand approaches the face in IR; the paper's per-frame design does not exploit this.","If face cropping is what lifts occlusion recall from 95.34 percent to 99.43 percent, then improving the face detector's bounding-box accuracy could further improve occlusion detection without new occlusion labels."],"forward_implications":["A system that detects occlusion before running gaze or identification can alert the driver that monitoring is temporarily unavailable, rather than silently returning unreliable outputs.","The permanent RGB-to-IR switch after repeated face-detection failure gives a graceful degrade path for tunnels, nighttime, and direct sunlight glare.","Face-cropped preprocessing should be adopted in production because it raised occlusion recall from 95.34 percent to 99.43 percent and gaze accuracy from 79.12 percent to 86.29 percent.","The RGB-over-IR accuracy gap predicts that future gains will come from pretraining on infrared data or improving IR sensor image quality, not from architecture changes alone.","Because the pipeline is modular, adding distraction or drowsiness modules later should require only training a new head on the same image stream."],"supporting_citations":[{"why":"Supplies all training and test images, with the gaze-region and distraction annotations that define the three tasks.","marker":"[18]"},{"why":"MobileNet-v3 is the backbone of the occlusion classifier whose RGB recall reaches 99.43 percent after cropping.","marker":"[20]"},{"why":"EfficientNet-b0 is the backbone of the final RGB gaze-region classifier reported at 86.29 percent test accuracy.","marker":"[21]"},{"why":"MTCNN is the face detector whose cropping step raises gaze accuracy and occlusion recall.","marker":"[22]"},{"why":"FaceNet provides the face embeddings used for the 99.38 percent driver identification accuracy.","marker":"[25]"},{"why":"AugMix augmentation is part of the final RGB gaze classifier training setup.","marker":"[23]"},{"why":"TrivialAugment is used in both gaze and occlusion classifier training.","marker":"[24]"},{"why":"CLAHE preprocessing on the IR image is applied when RGB face detection fails in the identification module.","marker":"[26]"},{"why":"The vehicle-safety protocol whose recommendation motivates alerting the driver when the DMS cannot operate correctly.","marker":"[2]"}],"fun_headline_variants":["Occlusion-aware DMS beats gaze and ID by flagging hidden faces first","Driver monitoring adds occlusion check: 99% recall on hidden faces","RGB beats IR in driver monitoring; occlusion detection adds trust","First occlusion-aware driver monitoring alerts when camera can't see"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the dataset's 'occlusion' labels actually mark visual obstructions of the face, so the high recall figures measure occlusion detection rather than some other driving state.","fun_headline_variants_meta":{"raw":{"variants":["Occlusion-aware DMS beats gaze and ID by flagging hidden faces first","Driver monitoring adds occlusion check: 99% recall on hidden faces","RGB beats IR in driver monitoring; occlusion detection adds trust","First occlusion-aware driver monitoring alerts when camera can't see"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001303,"raw_usage":{"total_tokens":5280,"prompt_tokens":874,"completion_tokens":4406,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":490,"completion_tokens_details":{"reasoning_tokens":4344}},"tokens_in":490,"tokens_out":4406,"duration_ms":29089,"temperature":1.0,"reasoning_tokens":4344,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:22:48.567722+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random sample of distraction-annotated frames from the public multi-modal driver monitoring dataset and have independent annotators decide whether each 'occlusion' label shows a visible face obstruction. If many labels correspond to hand-on-wheel or other distraction states, the reported 99.43 percent occlusion recall is not a measure of the claimed capability. Alternatively, run the RGB occlusion classifier on a controlled set of partially occluded faces; if recall drops sharply when the face is only partly covered, the practical robustness is narrower than the headline figure.","supporting_citations":[{"cited_title":"Dmd: A large -scale multi -modal driver monitoring dataset for attention and alertness analysis,","cited_arxiv_id":null,"evidence_quote":"Supplies all training and test images, with the gaze-region and distraction annotations that define the three tasks."},{"cited_title":"Searching for MobileNetV3,","cited_arxiv_id":null,"evidence_quote":"MobileNet-v3 is the backbone of the occlusion classifier whose RGB recall reaches 99.43 percent after cropping."},{"cited_title":"EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,","cited_arxiv_id":null,"evidence_quote":"EfficientNet-b0 is the backbone of the final RGB gaze-region classifier reported at 86.29 percent test accuracy."},{"cited_title":"Joint face detection and alignment using multitask cascaded convolutional networks,","cited_arxiv_id":null,"evidence_quote":"MTCNN is the face detector whose cropping step raises gaze accuracy and occlusion recall."},{"cited_title":"FaceNet: A unified embedding for face recognition and clustering,","cited_arxiv_id":null,"evidence_quote":"FaceNet provides the face embeddings used for the 99.38 percent driver identification accuracy."},{"cited_title":"AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty,","cited_arxiv_id":null,"evidence_quote":"AugMix augmentation is part of the final RGB gaze classifier training setup."},{"cited_title":"Trivialaugment: Tuning-free yet state-of-the- art data augmentation,","cited_arxiv_id":null,"evidence_quote":"TrivialAugment is used in both gaze and occlusion classifier training."},{"cited_title":"Contrast limited adaptive histogram equalization based enhancement for real time video system,","cited_arxiv_id":null,"evidence_quote":"CLAHE preprocessing on the IR image is applied when RGB face detection fails in the identification module."},{"cited_title":"Assessment Protocol – Safety Assist Safe Driving,","cited_arxiv_id":null,"evidence_quote":"The vehicle-safety protocol whose recommendation motivates alerting the driver when the DMS cannot operate correctly."}],"review_version":1}