{"id":"6cec0629-ff0a-400f-888b-315860ce841f","arxiv_id":"2611.16515","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"PiLLVD provides a paired low-light video dataset with dynamic scenes, an alignment pipeline, and zero-shot benchmarks that expose limitations of current enhancement methods.","lead":"The paper introduces PiLLVD, a real-world paired low-light video dataset captured with an accessible Raspberry Pi and beam splitter setup. The dataset focuses on dynamic indoor and outdoor scenes with uncontrolled human and object motion, and benchmarks show existing pre-trained enhancement models do not generalize uniformly across its sequences.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"No significant objection identified: the acknowledged alignment limitations are handled transparently and do not undermine the central claim.","rationale":"The stress-test confirms the reader's assessment. The paper's strongest claim—that the dataset challenges existing models—is supported by quantitative and qualitative evidence. The weakest assumption, alignment quality, is explicitly acknowledged and backed by reasonable metrics. The concrete test would further validate robustness, but the reader's verdict (ACCEPT) remains appropriate given the current evidence and transparency. No load-bearing flaw was identified; the dataset and benchmark serve their stated purpose.","tokens_in":18731,"tokens_out":2583,"duration_ms":26969,"concrete_test":"Select 200 frames with highest estimated alignment error (e.g., top 10% by feature displacement) across diverse sequences. Manually verify and, if needed, correct alignment via supervised homography refinement. Recompute PSNR, SSIM, LPIPS, and mTWE on this cleaned subset. If the relative ranking of methods (especially the best overall performer) does not change compared to the original evaluation, the alignment concern is not load-bearing. If rankings change, the evaluation protocol must be revised, but this is unlikely given the large inter-method variance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that PiLLVD provides a valuable benchmark exposing generalization failures of existing low-light video enhancement models—is well-supported. The weakest link is the reliance on warped normal-light frames as ground truth for reference-based evaluation, as the reader noted. The authors acknowledge this limitation (Section V: 'perfect pixel-level alignment remains difficult in challenging frames') and provide quantitative alignment metrics (Table III: mean displacement 0.56 px, inlier ratio 0.931). These metrics suggest that residual misalignment is small on average, but the impact on evaluation—especially around fast motion and low-texture regions—could affect metric accuracy. However, the benchmark results (Tables IV and V) show large performance gaps between methods (e.g., PSNR differences of 5–10 dB), so small alignment errors are unlikely to reverse the overall conclusion that no single method generalizes uniformly. The authors' transparency and the robustness of the observed differences make this a non-fatal concern. No evidence of circularity, inconsistency, or overclaiming was found.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"This paper presents PiLLVD, a real-world paired low-light video dataset captured using a low-cost Raspberry Pi-based dual-camera system with a beam splitter. The dataset contains 24 indoor and outdoor sequences (4,531 frames) with dynamic human and object motion under uncontrolled illumination. The authors describe the acquisition pipeline, a post-capture alignment procedure (evaluated with feature-based metrics), and a zero-shot benchmark of five pre-trained low-light video enhancement models. The results show that no existing method generalises uniformly across all sequences, highlighting the dataset's value for exposing limitations in fidelity, perceptual quality, and temporal stability.","tokens_in":19040,"tokens_out":1752,"duration_ms":37216,"significance":"If the dataset is used by the community, PiLLVD addresses a clear gap: the lack of publicly available, real-world paired low-light video data with dynamic scenes. Existing datasets are often static, synthetic, or not publicly released. The benchmark demonstrates that current pre-trained methods fail to generalise, confirming the need for more robust video-level enhancement approaches. The low-cost, reproducible capture system also lowers the barrier for future data collection. The release of aligned reference frames and a transparent evaluation protocol strengthens the contribution.","major_comments":[{"comment":"The benchmark evaluates only 7 of the 24 sequences (V4, V5, V14, V17 indoor; V2, V5, V7 outdoor). The authors state these were 'selected to cover a broad range of the dataset variation,' but the selection criteria are not formalised. Since the dataset is small (24 sequences) and the benchmark is central to the paper's claim, the authors should either justify why these sequences are representative (e.g., by showing that they span the range of illuminance, motion type, and scene structure) or report results on all sequences. At minimum, explain what metric was used to ensure coverage.","section":"§IV-C Evaluation Protocol","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Table III reports mean matched feature displacement reduced from 18.79 to 0.56 px, which is excellent on average. However, the authors acknowledge (Section V) that perfect alignment remains difficult around fast-moving objects and low-texture areas. The paper should provide per-video or per-frame alignment statistics (e.g., standard deviation, worst-case displacement) to quantify the variation. Without this, the reader cannot assess how residual misalignment might affect metric scores in challenging frames.","section":"§III-A4 Alignment accuracy","authors_acknowledge":"Very fast motion also introduces physical motion blur, making it difficult to establish accurate matches between the paired low-light and normal-light frames.","authors_acknowledge_p":0.64}],"minor_comments":[{"comment":"The contributions list states 'post-capture alignment and evaluation pipeline to improve spatial correspondence.' The evaluation in Table III is clear, but the paper would benefit from a brief description of which specific features (e.g., SIFT, ORB) were used for the feature-based metrics.","section":"§I Introduction","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The beam splitter is listed as 70/30 (R/T). Clarify which camera (low-light or normal-light) receives the transmitted vs. reflected path. From the system description it appears the low-light camera gets 30% transmission, but this should be stated explicitly.","section":"§III-A1 Hardware","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The 'Motion diversity' column for existing datasets is marked 'Unknown' or 'Low'. While plausible, this is a qualitative assessment. Consider providing a quantitative measure (e.g., average optical flow magnitude or variance) to support the claim that PiLLVD has higher motion diversity.","section":"Table I","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The mTWE metric is adapted from [48] by replacing FlowNet2 with RAFT. The authors should state whether this change was validated against the original metric (e.g., on a subset) or note any potential differences in the temporal consistency scores.","section":"§IV-D Evaluation Metrics","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The qualitative figures are printed at small scale. In the final version, consider providing zoomed-in crops of the most informative regions (e.g., the moving person in the outdoor sequence) to make visual differences more discernible.","section":"Figure 6 & 7","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The sentence 'very fast motion also introduces physical motion blur' is accurate, but the paper does not report motion-blur statistics across the dataset. A simple categorisation (e.g., number of frames with estimated blur > threshold) would help users understand the dataset's difficulty.","section":"§V Discussion & Limitations","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The reference list includes several entries marked as 'arXiv preprint' or 'to appear'. Some of these (e.g., [29] from 2025) should be checked for final publication venues in the camera-ready version.","section":"General","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"minor_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"This is a straightforward dataset paper that does what it says. The acquisition system is cheap and reproducible, the alignment pipeline is clearly described, and the benchmark results are presented without overclaim. The central claim—that existing LLVE methods generalise poorly to uncontrolled dynamic scenes—is supported by the data.\n\nThe authors are transparent about limitations: alignment imperfections, small size, indoor bias. They quantify alignment quality with feature-based metrics (Table III) rather than glossing over it. That is good practice.\n\nSoft spots: The dataset is small (24 sequences, 4531 frames), and the ground truth is warped normal-light frames, not perfectly aligned. The authors acknowledge this and argue that residual errors are small relative to the performance gaps observed. I find that argument convincing for the benchmark's main conclusion, but users should be aware when using PiLLVD for fine-tuning or pixel-level supervision. The outdoor coverage is thin—only 3 sequences—but the paper states plans to expand.\n\nOne minor concern: the benchmark only includes 5 methods, all of which are reasonably known. It would be stronger to include a wider set, but given the paper's scope (dataset introduction rather than method comparison), this is acceptable.\n\nThis paper is for the low-light video community. It will be cited because it fills a specific gap (real dynamic paired data, publicly available). It deserves a serious referee and should be accepted after minor revisions (perhaps suggesting additional outdoor sequences or a note about ground truth limitations for fine-tuning).","headline":"PiLLVD provides a needed dynamic paired low-light video dataset with honest evaluation; deserves acceptance.","tokens_in":19525,"tokens_out":1168,"duration_ms":24056},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"A new paired low-light video dataset challenges existing enhancement models on dynamic scenes, revealing no single method performs well across fidelity, perceptual quality, and temporal stability.","keywords":["low-light video enhancement","paired video dataset","dynamic scenes","Raspberry Pi","beam splitter","temporal consistency","zero-shot evaluation","benchmark"],"falsifier":"If a carefully designed controlled experiment shows that the residual alignment errors (mean 0.56 px displacement) systematically distort PSNR and SSIM by more than 0.5 dB or 0.02 respectively in challenging frames, then the metric differences between models may be partly an artefact of alignment quality rather than enhancement performance. A cross-check using synthetic dynamic sequences with known ground truth could settle this.","tokens_in":20257,"feed_emoji":"📹","tokens_out":2675,"duration_ms":43978,"temperature":0.7,"pith_summary":"The paper introduces PiLLVD, a real-world paired low-light video dataset captured with a low-cost Raspberry Pi-based dual-camera system and a beam splitter. Unlike existing datasets that are static, synthetic, or constrained by controlled motion, PiLLVD focuses on dynamic indoor and outdoor scenes with diverse uncontrolled human and object motion under varying illumination. The authors benchmark five pre-trained low-light video enhancement models in a zero-shot setting, showing that no model consistently outperforms others across PSNR, SSIM, LPIPS, and temporal consistency (mTWE). Some models improve brightness but introduce colour shifts or ghosting; others preserve temporal stability but under-enhance the image. The dataset exposes that existing methods do not generalise uniformly to real-world dynamic low-light conditions, making PiLLVD a challenging benchmark for developing more robust, temporally consistent low-light video enhancement algorithms.","feed_headline":"Benchmark shows no low-light video model handles dynamic real-world scenes","feed_subtitle":"PiLLVD dataset uses a beam-splitter camera rig to capture paired indoor/outdoor sequences with uncontrolled motion, exposing trade-offs…","key_machinery":"The PiLLVD dataset itself is the central object: a collection of spatially and temporally aligned low-light and normal-light video pairs captured by a synchronised dual-camera system using a 70/30 beam splitter. The acquisition hardware (three Raspberry Pis, two Arducam IMX519 cameras) is low-cost and reproducible. The post-capture alignment pipeline uses histogram matching on the low-light frames, TV-L1 optical flow between the enhanced low-light and normal-light frames, and warps the normal-light frames to the low-light viewpoint, avoiding interpolation on the input. The zero-shot benchmark protocol evaluates five pre-trained models on the same sequences with full-reference (PSNR, SSIM, LPIPS) and temporal (masked temporal warping error, mTWE) metrics, exposing trade-offs between fidelity, perceptual quality, and stability.","core_discovery":"The paper's central contribution is PiLLVD, a paired low-light and normal-light video dataset of 24 sequences (4,531 frames) featuring uncontrolled human and object motion, spanning 77–401 lux. The acquisition system uses a Raspberry Pi master–slave architecture with beam-splitter optics to synchronise two identical cameras with sub-millisecond temporal offset. A post-capture alignment pipeline reduces mean matched feature displacement from 18.79 px to 0.56 px. Benchmarking on four indoor and three outdoor sequences shows that LLVE-STCD leads in indoor SSIM and LPIPS, while DP3DF yields the best temporal consistency (lowest mTWE), and SDSDNet or StableLLVE lead on outdoor LPIPS in different sequences. No method dominates across all metrics and scenes, confirming that dynamic real-world low-light video remains an open challenge.","pith_inferences":["The observed failure modes (colour shift, ghosting, over-enhancement) suggest that current models lack joint optimization for motion compensation and illumination correction; future work may need to integrate optical-flow or transformer-based temporal modules within the enhancement head.","Because PiLLVD focuses on static-camera dynamic scenes, it does not test camera motion; a natural extension would be to record handheld or moving platform sequences, which would stress alignment and temporal consistency further.","The alignment pipeline's reliance on histogram matching and optical flow could be replaced by a learned alignment model trained on synthetic paired data, potentially reducing residual misalignment in low-texture regions.","The limited number of sequences (24) and frame count (4,531) means that dataset size may restrict statistical power; future work should verify whether the observed model rankings hold on an expanded version with more outdoor scenes and faster motion."],"forward_implications":["Researchers can use PiLLVD as a standard benchmark to evaluate temporal stability and motion-aware enhancement of low-light video models.","The dataset's diverse motion patterns (walking, hand movements, object manipulation) enable analysis of how enhancement methods handle non-rigid deformation and occlusion.","The zero-shot results provide baselines for future supervised or fine-tuned models trained on PiLLVD, which may achieve better generalisation than models trained on static or synthetic data.","The low-cost camera rig design can be replicated and extended by other groups, supporting community-driven collection of additional dynamic paired videos."],"supporting_citations":[{"why":"DRV dataset provides paired low-light/normal-light raw videos of static scenes; PiLLVD extends to dynamic scenes.","marker":"[18]"},{"why":"SDSD dataset captures dynamic scenes with controlled camera motion on a rail; PiLLVD offers uncontrolled object/human motion.","marker":"[19]"},{"why":"BVI-RLV is a recent paired low-light video dataset with motorised dolly motion; PiLLVD uses free human/object motion.","marker":"[30]"},{"why":"TV-L1 optical flow method used in the alignment pipeline to warp normal-light frames to low-light frames.","marker":"[40]"},{"why":"Masked temporal warping error (mTWE) metric used to evaluate temporal consistency across consecutive enhanced frames.","marker":"[48]"},{"why":"StableLLVE is one of the benchmarked models, focusing on temporal consistency from single-image training.","marker":"[12]"},{"why":"DP3DF (Deep Parametric 3D Filters) is benchmarked and achieves best temporal stability on several sequences.","marker":"[44]"},{"why":"LLVE-STCD (Spatial-Temporal Consistent Decomposition) is benchmarked and leads on indoor SSIM and LPIPS.","marker":"[45]"},{"why":"SGLLIE/SGZ (Semantic-Guided Zero-Shot) is benchmarked as a zero-reference method for comparison with supervised models.","marker":"[36]"}],"fun_headline_variants":[],"cache_read_input_tokens":0,"weakest_assumption_plain":"The aligned normal-light frames serve as reliable ground truth despite residual misalignment from the optical-flow-based warping pipeline, which the authors acknowledge becomes difficult around fast motion, motion blur, and low-texture regions.","fun_headline_variants_meta":{"raw":{"variants":[]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":9.7e-05,"raw_usage":{"cost":9.7097e-05,"is_byok":false,"cost_details":{"upstream_inference_cost":9.7097e-05,"upstream_inference_prompt_cost":9.6733e-05,"upstream_inference_completions_cost":3.64e-07},"total_tokens":1065,"prompt_tokens":1063,"completion_tokens":2,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":0,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":0}},"tokens_in":1063,"tokens_out":2,"cost_source":"openrouter_usage","duration_ms":12028,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":0,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-11-13T14:42:10.743458+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"If a carefully designed controlled experiment shows that the residual alignment errors (mean 0.56 px displacement) systematically distort PSNR and SSIM by more than 0.5 dB or 0.02 respectively in challenging frames, then the metric differences between models may be partly an artefact of alignment quality rather than enhancement performance. A cross-check using synthetic dynamic sequences with known ground truth could settle this.","supporting_citations":[{"cited_title":"Seeing motion in the dark,","cited_arxiv_id":null,"evidence_quote":"DRV dataset provides paired low-light/normal-light raw videos of static scenes; PiLLVD extends to dynamic scenes."},{"cited_title":"Seeing dynamic scene in the dark: A high-quality video dataset with mechatronic alignment,","cited_arxiv_id":null,"evidence_quote":"SDSD dataset captures dynamic scenes with controlled camera motion on a rail; PiLLVD offers uncontrolled object/human motion."},{"cited_title":"BVI-RLV: A Fully Registered Dataset for Low-Light Video Enhancement","cited_arxiv_id":"2407.03535","evidence_quote":"BVI-RLV is a recent paired low-light video dataset with motorised dolly motion; PiLLVD uses free human/object motion."},{"cited_title":"TV-L1 Optical Flow Estimation,","cited_arxiv_id":null,"evidence_quote":"TV-L1 optical flow method used in the alignment pipeline to warp normal-light frames to low-light frames."},{"cited_title":"Learning blind video temporal consistency,","cited_arxiv_id":null,"evidence_quote":"Masked temporal warping error (mTWE) metric used to evaluate temporal consistency across consecutive enhanced frames."},{"cited_title":"Learning temporal consistency for low light video enhancement from single images,","cited_arxiv_id":null,"evidence_quote":"StableLLVE is one of the benchmarked models, focusing on temporal consistency from single-image training."},{"cited_title":"Deep parametric 3d filters for joint video denoising and illumination enhancement in video super resolution,","cited_arxiv_id":null,"evidence_quote":"DP3DF (Deep Parametric 3D Filters) is benchmarked and achieves best temporal stability on several sequences."},{"cited_title":"Low- light video enhancement via spatial-temporal consistent decomposition,","cited_arxiv_id":null,"evidence_quote":"LLVE-STCD (Spatial-Temporal Consistent Decomposition) is benchmarked and leads on indoor SSIM and LPIPS."},{"cited_title":"Semantic-guided zero-shot learning for low- light image/video enhancement,","cited_arxiv_id":null,"evidence_quote":"SGLLIE/SGZ (Semantic-Guided Zero-Shot) is benchmarked as a zero-reference method for comparison with supervised models."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}