Real-Time Sleepiness Detection for Driver State Monitoring System

Deepak Ghimire; Juhwan Choi; Sanghyun Park; Sunghwan Jeong; Sunhong Yoon

REVIEW 5 major objections 5 minor 20 references

Real-Time Sleepiness Detection for Driver State Monitoring System

T0 review · 5 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A driver-monitoring system can detect drowsiness in real time by tracking eyes with dynamic template matching and classifying each eye as open or closed with a support vector machine.

desk verdict A competent engineering write-up of a standard eye-tracking pipeline that omits every quantitative result needed to support its real-time accuracy claim. read the letter →

arxiv 2504.14807 v1 pith:H4DW3PMP submitted 2025-04-21 cs.CV cs.HCcs.LG

Deepak Ghimire , Sunghwan Jeong , Sunhong Yoon , Sanghyun Park , Juhwan Choi This is my paper · ORCID

classification cs.CVcs.HCcs.LG

keywords driverdrowsinessdetectioneyetrackingdynamictemplatematchingKalmanfiltersupportvectormachinehistogramoforientedgradientsfacemonitoringsystem

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a driver-drowsiness system can be built from a small set of classical computer-vision components: face detection, normalized cross-correlation template matching for eye tracking, a Kalman filter for smoothing, and a linear support vector machine on HOG features to label each eye as open or closed. The system is designed to run in real time on modest hardware because only the eye regions are processed, and it is meant to keep working when the driver's face is at an angle or only one eye is visible. If the eye state is reported closed for a fixed time interval, an alarm is generated, which is the mechanism for waking a drowsy driver. The paper asserts this design detects eye state accurately even when the driver wears eyeglasses, and that HOG features outperform LBP features for this classification. If correct, a low-cost camera plus this pipeline could alert drivers before they fall asleep, addressing a large share of fatigue-related crashes.

What carries the argument

The load-bearing mechanism is the pairing of dynamic template matching with an SVM classifier. When tracking starts, several eye templates, typically 10 to 20, are stored; on each new frame, normalized cross-correlation measures how well each template matches the search region, the best match locations are averaged, and a Kalman filter smooths the result to produce the tracked eye position. If the correlation falls below a threshold, the system re-detects the face and eyes and refreshes its templates. The tracked eye region is preprocessed with gamma correction, difference-of-Gaussian filtering, and contrast equalization, then described by a 540-dimensional HOG feature vector and scored by the SVM as open or closed. A temporal rule, closed for a specified period, converts those frame-level labels into a drowsiness alarm.

What would settle it

Run the system in a car with the camera mounted off to the driver's left, as the paper describes, over varied lighting and with and without eyeglasses, and record eye-tracking success and open/closed classification accuracy on a labeled frame set. If accuracy drops materially below the reported HOG ROC curve, or if the tracker re-initialization rate rises sharply during ordinary head motion, the claimed real-world accuracy does not hold.

Watch

Extended reading notes

Core claim

The central claim is that normalized cross-correlation-based online dynamic template matching, combined with Kalman filtering, can hold onto eye positions across frames well enough that a support vector machine with histogram of oriented gradients features can then classify each eye as open or closed; a temporal persistence rule, closed for a specified amount of time, then declares sleepiness and sounds an alarm. Face detection runs first using a fast LBP cascade, eyes are found inside the face with a Haar cascade, and eye templates are continuously refreshed from recent detections so the tracker adapts to changing appearance. The SVM is trained offline on 9,763 labeled eye images, of which 4,180 are closed and 5,583 are open, and the experiments compare LBP and HOG features, with HOG giving the better ROC curve. The paper reports the full system runs in real time with minimal computational complexity, can decide from a single visible eye, and detects eye state accurately for drivers wearing eyeglasses.

Load-bearing premise

The SVM is trained offline on a fixed set of eye images, and the blanket accuracy claims assume those images represent real in-vehicle conditions, including off-angle views, changing lighting, and eyeglasses, with no reported domain adaptation or per-driver calibration.

Editorial extensions

If this is right

The pipeline can run on embedded hardware because the only expensive step is face and eye detection, which is re-run only when tracking is lost.
Because single-eye monitoring is allowed, the system remains usable when the driver's face is turned and only one eye is visible to the camera.
Refreshing templates after re-detection lets the tracker recover from brief tracking failures and adapt to slow lighting or pose changes.
Frame-by-frame open/closed labels plus a temporal decision window give a simple, tunable alarm policy: adjusting the required closure period trades false alarms against missed drowsiness.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension not explored in the paper is lifting the same tracking-plus-classification skeleton to other driver state cues, such as head pose, yawning, or gaze direction, by replacing only the SVM label set.
Because no per-driver calibration is reported, a practical deployment would likely need a short personalization step or a much larger multi-driver training set to close the gap between offline training data and real in-vehicle conditions.
A directly testable extension is measuring how tracking success and classification accuracy degrade as camera-driver geometry, lighting, or glasses type changes; the claim of accurate detection with eyeglasses would be stronger if reported separately for glasses and no-glasses subsets.
The decision rule only looks at eye closure, so a driver who is awake but distracted, for example looking down at a phone, would not be flagged; adding a head-pose cue would extend coverage beyond the paper's stated scope.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Quick take on arXiv:2504.14807. The paper is a straightforward integration of known components: Viola-Jones face detection, Haar eye detection, dynamic template matching with multiple templates and Kalman filtering, HOG features with an SVM for open/closed classification, and a temporal drowsiness rule. The authors are honest about the components and cite the relevant prior work. The system description is clear enough that someone could reimplement it, aside from the garbled correlation equations (1) and (2), which are not reproducible as printed.

What the paper does well: it makes a sensible engineering choice to track each eye independently and fuse with face geometry knowledge, and it reports training data counts (9763 eye images). The LBP vs HOG comparison is a legitimate question, and the ROC curves in Fig. 5 are at least directionally informative.

The soft spot is not small: there are no numbers anywhere. No AUC, no accuracy on the 5000-frame test video, no false positive rate, no tracking loss count, no frame rate or per-frame processing time, no baseline comparison against any of the cited systems. The claim that the system works in real time with minimal computational complexity is asserted in the conclusions, not measured. "All detections are correct" in Fig. 6 is a caption, not an evaluation. The training set is heterogeneous, and the test environment (camera offset to the left, possible profile views, eyeglasses) is described but never paired with ground truth or a held-out evaluation.

The citation pattern looks fine; the single self-citation [16] is for color face detection and is not load-bearing.

My verdict matches the stress-test: the missing evaluation is load-bearing. Even if the train/test distribution mismatch were resolved, the paper still would not support its central claim because it never reports a deployment-oriented measurement. This is an incomplete paper as submitted, not a fundamentally unsound one.

Who is this for? Someone working on embedded driver monitoring might find the system architecture interesting as a starting point, but they cannot compare it to alternatives. It deserves peer review only if the venue expects to ask for substantial additional experiments; as a short paper it would need heavy revision. I would not cite it in its current form.

Recommendation: if I were the editor, I would not desk reject it, but I would send it to a rigorous reviewer who will explicitly demand quantitative results for the 5000-frame sequence, a frame-rate measurement, and a comparison to at least one prior system. Without those, it cannot be accepted.

Referee Report

5 major / 5 minor

Summary. The paper proposes a real-time driver drowsiness detection pipeline. The face is detected with an LBP cascade, eyes are localized with a Haar cascade, and eye positions are tracked in subsequent frames by normalized cross-correlation template matching combined with Kalman filtering. A linear SVM with HOG features classifies each eye ROI as open or closed, and a drowsiness alarm is triggered if the eyes remain closed for a specified duration. The authors report qualitative results on in-vehicle and lab videos and compare LBP versus HOG features via ROC curves (Fig. 5), concluding that the system operates in real time with minimal computational complexity and is robust to eyeglasses and profile views.

Significance. If substantiated, the claimed contribution—a low-complexity eye-tracking and eye-state classification pipeline robust to partial occlusion and eyeglasses—would be of practical interest for embedded driver monitoring. The paper's strength is its simple, modular design that reuses well-established components (Viola-Jones detection, Kalman filtering, SVM classification), and the choice of HOG over LBP is a sensible, testable design decision. However, the empirical core of the paper is not reported: there are no numeric accuracy, ROC-AUC, latency, or tracking-failure measurements, and no reproducible evaluation protocol. The central claims of real-time operation, accuracy under eyeglasses, and robustness to off-angle views therefore remain unsupported. The manuscript does not ship code, datasets, or machine-checked derivations; the only quantitative statements are the training set size (9763 eye images) and the length of a sample tracking video (5000 frames).

major comments (5)

[Section 3, Fig. 5] The ROC curves for LBP and HOG are presented without any numeric values: no AUC, no operating point, no false-positive rate, and no accuracy or error rate. The claim that "HOG feature outperformed LBP feature" (Section 3) cannot be assessed, and no statistically meaningful comparison is possible. Please report the full confusion matrix, AUC with confidence intervals, and a chosen operating threshold with the corresponding true-positive and false-positive rates.
[Section 3, Fig. 6] The caption states "All detections are correct," but this is anecdotal; no ground-truth labeling procedure, no frame count, no per-class accuracy, and no stratified results (e.g., with/without eyeglasses, near-frontal/profile, lab/vehicle) are given. The claim that the system "will be detected accurately even if the driver is wearing eyeglass" (Section 1) is a central contribution and must be supported by quantitative results on an eyeglass-annotated test set.
[Section 3, tracking video] The paper describes a 2m59s, 5000-frame tracking video but reports no tracking success rate, no number of lost tracks, no re-initialization frequency, and no localization error relative to ground truth. Since the tracking step feeds the eye-state classifier, the absence of any tracking accuracy measure makes it impossible to evaluate the end-to-end pipeline. Please report per-frame eye-center error or tracking success/failure counts, and specify the ground-truth annotation method.
[Section 3, real-time claim] The paper claims the system "works in real time with minimal computational complexity" (Section 4 and Introduction) but reports no frame rate, per-frame processing time, hardware platform, or resolution. This is a load-bearing claim for a driver monitoring system and must be quantified; please provide measured throughput on the target embedded platform and a per-stage timing breakdown (face detection, eye detection/tracking, feature extraction, SVM classification).
[Section 3, training/test distribution] The SVM is trained on the RIP ISL Eye Dataset plus a personal collection (9763 images), but the paper never characterizes the test distribution (in-vehicle lighting, camera angle, eyeglasses, partial occlusion) or establishes that the training data cover it. No cross-validation, subject independence, or domain-adaptation procedure is described. Because the paper makes deployment claims, the evaluation must use a held-out set that is stratified by the challenging conditions named in the Introduction; otherwise the claimed accuracy may not transfer to actual driving environments.

minor comments (5)

[Section 2.2, Eq. (1)-(2)] The correlation formula contains notation that is not fully defined; in particular, the summation ranges and the relationship between T', I', and the image coordinates are unclear. Please rewrite the equation with explicit definitions of all variables and bounds.
[Section 2.2, text] "The final result of eye detection and tracking is subjected to Kamlan tracking" appears to be a typo for "Kalman tracking". Please correct.
[Section 2.2, text] The description of template update says templates are replaced by the most recent eye templates obtained via Viola-Jones detection, but the trigger condition for re-detection is only "matching score is less than specified threshold"; the threshold value is never given. Please state the threshold and how it was chosen.
[Section 3, Fig. 4] The sample tracking frames are shown without any quantitative measure of tracking accuracy or a description of how the bounding boxes were verified. Please add a caption explaining the video content, frame indices, and what the boxes represent.
[References] Reference [17] misspells "Gobb's" (should be "Dr. Dobb's Journal"), and reference [18] incorrectly lists the first author as "Ojha" (the correct spelling is "Ojala"). Please check all references against the original sources.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the pipeline is an empirical system; the sole self-citation is a non-load-bearing literature example, and no predictive claim reduces by construction to its inputs.

full rationale

The paper's derivation chain is algorithmic and empirical: face detection via the Viola-Jones LBP cascade, dynamic NCC template matching with Kalman filtering for eye tracking, and SVM classification on HOG features. None of these components is defined in terms of the drowsiness output, and no parameter is fitted to the exact quantity being predicted in a way the text exhibits. The SVM is trained on RIP ISL and personal eye images; the reported ROC curves compare LBP and HOG, but the absence of an explicit train/test split is an evaluation-reporting and correctness-risk flaw, not a circular step, because the paper never states that the ROC is computed on the training set. The only self-citation is [16], offered as an example of color-based face detection while the implemented system uses Viola-Jones [15]; hence it is not load-bearing. Claims such as real-time operation and robustness to eyeglasses are unsupported by numeric results, but lack of evidence is not circularity. No equation or reported result is shown to be equivalent to its own input by construction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The system is a pipeline of standard components, each with hand-chosen parameters. The central claim depends on untested thresholds and an assumed match between training data and deployment conditions.

free parameters (5)

Eye closure duration threshold = not specified
The alarm is triggered after eyes remain closed for a specified period, but the period is never quantified.
Template matching correlation threshold = not specified
Tracking is re-initialized when matching score falls below a threshold (Section 2.2); value not reported.
Number of templates (10 to 20) = 10-20
Chosen range, not optimized or justified.
SVM hyperparameters (C, kernel) = not specified
Linear SVM used, but no hyperparameters, training procedure, or cross-validation described (Section 2.3).
Preprocessing parameters (gamma, DOG, contrast equalization) = not specified
Preprocessing steps listed but parameters omitted (Section 2.3).

assumptions (4)

domain assumption Viola-Jones face and eye detectors provide reliable initial detections
The whole pipeline depends on correct face/eye initialization from OpenCV cascades; no detection accuracy is reported for this deployment.
domain assumption RIP ISL Eye Dataset labels are correct
The SVM is trained on this external dataset plus personal collection; no manual verification or cleaning is described.
standard math Normalized cross-correlation template matching is a valid tracking objective
Equation (1) is garbled, but the standard NCC formulation is assumed.
domain assumption Eye geometry constraints reliably disambiguate left/right eyes
The system uses face geometry to verify left/right eye positions, but no failure analysis is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Real-Time Sleepiness Detection for Driver State Monitoring System." pith.science (2026). https://pith.science/paper/H4DW3PMP

@misc{pith2026250414807,
  author       = {Pith},
  title        = {Pith review of: Real-Time Sleepiness Detection for Driver State Monitoring System},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H4DW3PMP}},
  note         = {Machine review of arXiv:2504.14807}
}

read the original abstract

A driver face monitoring system can detect driver fatigue, which is a significant factor in many accidents, using computer vision techniques. In this paper, we present a real-time technique for driver eye state detection. First, the face is detected, and the eyes are located within the face region for tracking. A normalized cross-correlation-based online dynamic template matching technique, combined with Kalman filter tracking, is proposed to track the detected eye positions in subsequent image frames. A support vector machine with histogram of oriented gradients (HOG) features is used to classify the state of the eyes as open or closed. If the eyes remain closed for a specified period, the driver is considered to be asleep, and an alarm is triggered.

Figures

Figures reproduced from arXiv: 2504.14807 by the authors.

**Figure 1.** Flowchart of the proposed driver’s drowsiness detection system 2.1 Face and Eye Detection Face detection is the primary step in driver drowsiness detection system. Several methods are proposed in the literature for face detection in gray scale images (ex. [15]), as well as in color images (ex. [16]). In our system we used the popular face detection method for gray scale images presented by Viola and Jones [15]. But … view at source ↗

**Figure 2.** Example of eye state detection using SVM classification with HOG features 3 Experimental Results The proposed driver state detection algorithm was tested on vehicle driving by various persons as well as in lab environment. In our system the camera is not placed directly in front of the driver but in little left position. The main reason was not to distract the driver so that he/she will not feel the camera is monito… view at source ↗

**Figure 4.** Sample image frames from a video sequence showing tracking result. ROC Curve True Positive True Positive 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 LBP Feature 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 HOG Feature ROC Curve False Positive False Positive (a) (b) [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: ROC curve of the eye state classification using SVM with (a) LBP and (b) HOG feature descriptor. LBP and HOG features are studied in order to discriminate open and closed eye with support vector machine classification. Small region around detected eye position as shown…

**Figure 6.** Figure 6: Example results of eye state detection in vehicle as well as in lab environment. All detections are correct. 4 Conclusions In this paper, a new adaptive method for driver drowsiness detection was proposed by monitoring the state of eyes in video sequence. The correlati…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 20 canonical work pages

[16]

Journal of Information Processing System 9(1), 141--156 (2013)

Ghimire, D., Lee, J.: A robust face detection method based on skin color and edges. Journal of Information Processing System 9(1), 141--156 (2013)

work page 2013
[1]

Sigari, M.H., Fathy, M., Soryani, M.: A Driver Face Monitoring Systems for Fatigue and Distraction Detection Int. J. of Vehicular Technology, 2013, 1--11 (2013)

work page 2013
[2]

Drivers Beware Getting Enough Sleep Can Save Your Life This Memorial Day; National Sleep Foundation (NSF): Arlington, VA, USA, 2010

work page 2010
[3]

Available online: http://www.fraunhofer.de/en/press/research-news/2010/10/eye-tracker-driver- drowsiness.html (last accessed on 17 Sept 2015)

Husar, P.: Eyetracker Warns against Momentary Driver Drowsiness. Available online: http://www.fraunhofer.de/en/press/research-news/2010/10/eye-tracker-driver- drowsiness.html (last accessed on 17 Sept 2015)

work page 2010
[4]

Traffic Accident Causation in Europe (TRACE), 2007

Bayly, M., Fildes, B., Regan, M., Young, K.: Review of Crash Effectiveness of Intelligent Transport System. Traffic Accident Causation in Europe (TRACE), 2007

work page 2007
[5]

Wang, Q., Yang, Jingyu., Ren, M., Zheng, Y.: Driver Fatigue Detection: A Survey. Proc. of the 6th World Congress on Intelligent Control and Automation, June 21-23, 2006, China

work page 2006
[6]

Sensors 12, 16937--16953 (2012)

Sahayadhas, A., Sundaraj, K., Murugappan, M.: Detecting Driver Drowsiness Based on Sensors: A Review. Sensors 12, 16937--16953 (2012)

work page 2012
[7]

2006 6th Int

Zhang, Z., Zhang, J.: A New Real-Time Eye Tracking for Driver Fatigue Detection. 2006 6th Int. Conf. on ITS Telecommunication Proceedings, June, 2006, Chengdu, China

work page 2006

Show all 20 references

[8]

First Int

Devi M.S., Bajaj, P.R.: Driver Fatigue Detection Based on Eye Tracking. First Int. Conf. on Emerging, Trends in Engineering and Technology, July 16-18, 2008, Nagpur, India

2008
[9]

Awais, M., Badruddin, N., Drieberg, M.: Automated Eye Blink Detection and Tracking Using Template Matching, Dec 16-17, 2013, Putrajaya, Malaysia

2013
[10]

Khan, M.I., Mansoor, A.B.: Real Time Eyes Tracking and Classification for Driver Fatigue Detection, LNCS 5112, 729--738 (2008)

2008
[11]

Singh, H., Bhatia, J.S., Kaur, J.: Eye Tracking based Driver Fatigue Monitoring and Warning System, 2010 India Int. Conf. on Power Electronics, Jan 28-30, 2011, India

2010
[12]

Real Time Imaging 8(5), 357--377 (2002)

Ji, Q., Yang, X.: Real-time eye, gaze, and face pose tracking for monitoring driver vigilance. Real Time Imaging 8(5), 357--377 (2002)

2002
[13]

of the 10th Int

Batista, J.: A drowsiness and point of attention monitoring system for driver vigilance, Proc. of the 10th Int. IEEE Conf. on Intelligent Transportation System, Sept 30 – Oct 3, 2007, Seattle, Wash, USA

2007
[14]

IEEE Trans

Bergasa, L.M., Nuevo, J., Sotelo, M.A., Barea R., Lopez, M.E.: Real-time system for monitoring driver vigilance. IEEE Trans. on Intell Trans Sys 7(1), 357--377 (2006)

2006
[15]

International Journal of Computer Vision 57(2), 137--154 (2004)

Viola, P., Jones M.J.: Robust real-time face detection. International Journal of Computer Vision 57(2), 137--154 (2004)

2004
[17]

Bradski, G.: The OpenCV library. Dr. Gobb’s Journal of Software Tools, 2000

2000
[18]

IEEE Trans

Ojha, T., Pietikainen, M., Maenpaa, A.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. PAMI 24(7), 971--987 (2002)

2002
[19]

IEEE Conf

Dalal, N., Triggs B.: Histograms of orientation gradients for human detection. IEEE Conf. on CVPR, June 20-25, 2005, San Diego, CA, USA

2005
[20]

IEEE Int

Wang, P., Ji, Q.: Learning discriminant features for multi-view face and eye detection. IEEE Int. Conf. on CVPR, June 20-25, 2005, San Diego, CA, USA 8

2005

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [16]

Journal of Information Processing System 9(1), 141--156 (2013)

Ghimire, D., Lee, J.: A robust face detection method based on skin color and edges. Journal of Information Processing System 9(1), 141--156 (2013)

work page 2013

[2] [1]

Sigari, M.H., Fathy, M., Soryani, M.: A Driver Face Monitoring Systems for Fatigue and Distraction Detection Int. J. of Vehicular Technology, 2013, 1--11 (2013)

work page 2013

[3] [2]

Drivers Beware Getting Enough Sleep Can Save Your Life This Memorial Day; National Sleep Foundation (NSF): Arlington, VA, USA, 2010

work page 2010

[4] [3]

Available online: http://www.fraunhofer.de/en/press/research-news/2010/10/eye-tracker-driver- drowsiness.html (last accessed on 17 Sept 2015)

Husar, P.: Eyetracker Warns against Momentary Driver Drowsiness. Available online: http://www.fraunhofer.de/en/press/research-news/2010/10/eye-tracker-driver- drowsiness.html (last accessed on 17 Sept 2015)

work page 2010

[5] [4]

Traffic Accident Causation in Europe (TRACE), 2007

Bayly, M., Fildes, B., Regan, M., Young, K.: Review of Crash Effectiveness of Intelligent Transport System. Traffic Accident Causation in Europe (TRACE), 2007

work page 2007

[6] [5]

Wang, Q., Yang, Jingyu., Ren, M., Zheng, Y.: Driver Fatigue Detection: A Survey. Proc. of the 6th World Congress on Intelligent Control and Automation, June 21-23, 2006, China

work page 2006

[7] [6]

Sensors 12, 16937--16953 (2012)

Sahayadhas, A., Sundaraj, K., Murugappan, M.: Detecting Driver Drowsiness Based on Sensors: A Review. Sensors 12, 16937--16953 (2012)

work page 2012

[8] [7]

2006 6th Int

Zhang, Z., Zhang, J.: A New Real-Time Eye Tracking for Driver Fatigue Detection. 2006 6th Int. Conf. on ITS Telecommunication Proceedings, June, 2006, Chengdu, China

work page 2006