{"id":"259ba735-eae0-4743-b79d-d1550a04b8d0","arxiv_id":"2505.07381","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A few-shot, sketch-based semantic coding method reconstructs surveillance video from a reference frame and six training clips per scene, outperforming video synthesis baselines on KID, LPIPS, PSNR, and SSIM.","lead":"The authors propose a video surveillance system that sends a simple sketch of each frame plus one reference image, then reconstructs the full video with a network trained on only a few example clips per camera. The approach targets the growing cost of storing and transmitting surveillance footage, a practical problem in need of bandwidth-efficient solutions.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Compression claim rests on an incomplete rate comparison: side information (reference frame and first/last sketches) is omitted, so lower storage/transmission is not established.","rationale":"The reader's weakest_assumption focused on the reference-frame/style assumption. That is a real boundary condition, but the paper's own experiments on 64 stationary cameras already operate in the regime where that assumption holds, so it limits external validity rather than undermining the reported numbers. The omitted side information, by contrast, directly threatens the reported compression advantage and the conclusion drawn from it. The system in Eqs. (3)-(6) requires reference material that is free only in the synthesis experiments; if the added cost is included, the advantage at low bitrates where surveillance systems operate may vanish. This is fixable by an end-to-end rate-distortion evaluation, so the manuscript remains conditional rather than rejected. I also note the ambiguity in Eq. (4), where max(s_t,1) with binary sketches collapses the masked sketch to the foreground mask; but the primary load-bearing gap is the incomplete bitrate accounting.","tokens_in":7784,"tokens_out":11099,"duration_ms":108804,"concrete_test":"Re-run the Section III-C comparison end to end: encode the complete proposed bitstream (masked sketch video plus compressed x1, s1, and sT) with H.264/HEVC at QP 30, 40, and 50, and compare total bytes and reconstruction quality (PSNR, LPIPS) against H.264-encoded raw video at the same QPs. If total bytes with side information are not below raw video at every QP, or if the rate-distortion operating point is worse at matched bitrate, the 'lower storing and transmitting consumption' claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim has two halves: better reconstruction and lower storage/transmission. The reconstruction half is supported by Table I, but the storage/transmission half is not. Section III-C reports sizes of H.264-encoded raw videos and masked sketch videos, but the proposed system must additionally transmit the first frame x1 (used as the style reference throughout) and, for the background reconstruction in Eqs. (3) and (6), the first and last sketches s1 and sT. None of these are counted in Table 4. At QP50 the raw video is 53±26 KB; the masked sketch is 49±42 KB, only 8% smaller before side information. Adding any compressed version of x1, a natural image, will likely make the total larger. The conclusion 'lower storing and transmitting consumption' is therefore not quantitatively demonstrated by the reported experiment. This is not a peripheral detail: it is the basis for half of the headline claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a few-shot sketch-based semantic encoding and decoding pipeline for stationary surveillance video. The encoder extracts edge sketches, detects foreground instances with a video segmentation network, and compresses the sketch video by masking out the static background using temporal instance masks. The decoder reconstructs the sketch and then uses a vid2vid-like network augmented with an image translation module that takes the first frame as a style reference, so that only six training videos per scene are needed. Experiments on a self-collected dataset of 64 surveillance scenes compare reconstruction quality (KID, LPIPS, PSNR, SSIM) against vid2vid, few-shot vid2vid, and SGA, reporting significant improvements for the proposed video-level method. The paper also reports H.264 sizes of raw and masked sketch videos to support a sketch-compression benefit.","tokens_in":7957,"tokens_out":8369,"duration_ms":76813,"significance":"If the central claims are fully established, the method is a useful practical contribution to surveillance-oriented semantic communication: it demonstrates that a few-shot-trained sketch-to-video pipeline can reconstruct stationary-camera surveillance footage at higher perceptual quality than established baselines, and that static-background masking can reduce sketch bitrate. The paper's strengths include evaluation on external metrics against independent baselines, a newly collected surveillance dataset, explicit statistical testing, and an ablation separating the image translation module from the full video decoder. The statistical reporting and compression accounting, however, need improvement before the headline claims can be accepted.","major_comments":[{"comment":"The end-to-end bitrate comparison omits side information required by the method, so the central claim of lower storing/transmitting consumption is not demonstrated. The decoding network D in Eq. (7) takes the first frame x1 as reference, and the static-background reconstruction in Eq. (3) uses the first and last sketches s1 and sT. Table 4, however, compares only H.264 sizes of raw videos and masked sketch videos. At QP50 the masked sketch (49±42 KB) is only about 8% smaller than the raw video (53±26 KB), and adding a compressed natural-image reference frame x1, plus s1/sT, will likely make the total larger than the raw video. Section III-B even states that 'the sketch and the first frame of each video need to be transmitted,' which the compression experiment does not count. Please report the total bitrate including all side information, or explicitly limit the claim to the sketch bitrate.","section":"III-C and IV"},{"comment":"Equation (4) defines the masked sketch as mst = mF_t max(st,1), but for the binary sketch described in the paper, max(st,1)=1, so the masked sketch would contain only the foreground mask and lose the sketch structure. This is likely a typo for mst = mF_t * st. The mistake is load-bearing because Eqs. (5)-(6) recover the sketch from mst and the compression statistics in Section III-C depend on the masked sketch content. Please correct the formula and verify whether the reported bitrate and reconstruction results change.","section":"II-A, Eq. (4)"},{"comment":"The statistical claims of superior reconstruction are not fully verifiable from the reported numbers. The paper gives t statistics and p-values but no degrees of freedom, effect sizes, or confidence intervals, and Table I shows only point estimates without error bars. For example, on KID the proposed image translation network (0.2711) and few-shot vid2vid (0.2713) differ in the third decimal; without variance information or a df value the paired t-test cannot be checked. Please report df, confidence intervals, and per-scene error bars for all metrics.","section":"III-B, Table I"},{"comment":"The compression experiment is referenced as 'Tab. 4' and 'Table 4', but the manuscript contains only Table I and Table II, so the sizes of raw videos, sketch videos, static-background sketch videos, and masked sketch videos at the three QP values are not actually presented. Additionally, the sentence 'The matrices of the video reconstructed ... are shown in Tab. 4' appears to refer to Table II. Please correct the table numbering and include the full compressed-size table with means and standard deviations.","section":"III-C"}],"minor_comments":[{"comment":"The term ˆmF_1 s_T (1−ˆmF_1∩ˆmF_t) mixes set-difference and arithmetic notation; it should be written as the set difference ˆmF_1 minus (ˆmF_1 intersect ˆmF_t) multiplied by s_T to be parseable.","section":"II-A, Eq. (6)"},{"comment":"The temporal IoU in Eq. (1) uses intersection and union over the entire video; please define the mask value for frames in which an instance is absent, since instances may enter or leave the field of view.","section":"II-A, Eq. (1)"},{"comment":"The window length appears as both τ and L: the text says 'past L frames' but the equations use τ. Please use one symbol and define it consistently.","section":"II-B, Eqs. (7)-(11)"},{"comment":"The text says 'The matrices of the video reconstructed ...' where 'matrices' should be 'metrics'.","section":"III-C"},{"comment":"There are numerous typographical issues (e.g., 'INTRUDUCTION', 'ANOV A', 'p¡0.01'); a careful proofread is needed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know: this is a plausible few-shot semantic video coding method for surveillance, and the reconstruction half of the claim is supported by the experiments. The compression half is not, because the bitrate comparison omits side information that the system must transmit.\n\nWhat's new: the reference-guided image translation decoder (sketch + reference frame) for few-shot per-scene training, and a static-background sketch compression that replaces background edges with the first/last frames. Neither is a big leap on its own, but the combination and the application to surveillance video with six training samples is not in the cited prior work. The experiments are a real strength: 64 scenes, four metrics, t-tests uniformly favorable. The reconstruction gains over vid2vid, few-shot vid2vid, and SGA are consistent, and the masked sketch costs only a small quality drop.\n\nWhere it falls short: the headline claim of lower storage/transmission is not demonstrated. Table 4 compares H.264 sizes of raw video and masked sketch video, but the proposed system must also send the first frame x1 (the style reference) and, for background reconstruction, the first and last sketches s1 and sT. At QP50, the masked sketch is 49±42 KB vs 53±26 KB for raw – only 8% smaller – and adding a compressed natural image as side information would very likely erase that gap. The authors even acknowledge in Section III-B that \"the sketch and the first frame need to be transmitted\" but then defer that to future work. That leaves the second half of the conclusion unsupported.\n\nOther issues: Eq. (4) is a typo (max(st,1) would collapse the sketch; should be element-wise multiplication). Error bars and degrees of freedom are missing from the t-tests. There is no many-shot baseline to show the few-shot benefit, and no code or data. These are fixable.\n\nThe static-background assumption is acknowledged and plausible for fixed cameras, but lighting/weather drift would hurt; that's a limitation to state more explicitly.\n\nWho would get something from this: someone working on semantic communication or learned video compression for surveillance; it's a useful incremental step. The reconstruction results justify referee time, but the compression claim needs a corrected rate comparison or a revised conclusion.\n\nMy recommendation: accept for peer review, but the authors should be asked to either include side information in the bitrate accounting or soften the claim to 'compression of the sketch stream' rather than lower total storage/transmission.","headline":"The few-shot reconstruction results are credible, but the paper's second headline claim—lower storage/transmission—rests on a bitrate comparison that omits the side information the system must send.","tokens_in":8503,"tokens_out":2594,"would_cite":false,"duration_ms":23113,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A sketch-based semantic codec can reconstruct surveillance video from only six training samples per scene, beating video-to-video baselines on every reported quality metric.","keywords":["semantic communication","video surveillance","few-shot learning","sketch-based coding","image translation","static background compression","video reconstruction","optical flow"],"falsifier":"Record a stationary-camera surveillance scene across an hour that spans dusk, when lighting changes slowly and outdoor shadows move; encode it with the masked-sketch pipeline and compare reconstruction PSNR and SSIM against the paper's reported 0.8% PSNR gap between masked and original sketch. If the gap becomes large, or if the instance-segmentation network marks a permanently parked car as static background and the car disappears from reconstruction, the static-background premise is falsified for that regime.","tokens_in":7584,"feed_emoji":"📹","tokens_out":7957,"duration_ms":68467,"temperature":0.7,"pith_summary":"Surveillance video is mostly static background, and the authors argue that the meaningful signal is the moving foreground, which can be represented by edge sketches. The paper tries to establish that such a representation, combined with one reference frame and a network trained on just six videos per camera scene, supports video reconstruction that is both cheaper to store and better on quality metrics than standard video-to-video synthesis baselines. This would matter because semantic communication promises to escape the bit-rate bottleneck of conventional video coding, but previous semantic decoders needed many training samples per scene. The reported results show the few-shot approach beating vid2vid, few-shot vid2vid, and SGA on KID, LPIPS, PSNR, and SSIM, and the sketch compression cutting bitrate with small quality loss.","feed_headline":"Six training clips per scene are enough to rebuild surveillance video","feed_subtitle":"Sketch-plus-reference-frame codec beats vid2vid on every quality metric using only six training videos per camera.","key_machinery":"The load-bearing mechanism is a reference-conditioned decoding network built on a video-to-video synthesis backbone, with its image generator replaced by a translation network. Given the current sketch, the translation network computes an attention map between sketch features and reference-image features, warps the reference features to match the sketch structure, and injects the reference's global style through adaptive instance normalization. The surrounding video network predicts optical flow between the previous generated frame and the current one, and a mask network decides which pixels are copied from the warped previous frame and which are freshly generated. On the encoding side, an instance-segmentation network finds moving-object masks; the static background assumption lets the sketch compressor drop background edges and recover them from the first and last frame sketches at decode time.","core_discovery":"On its own terms, the paper's discovery is that a surveillance scene can be learned from six training videos and then all subsequent frames reconstructed from compressed sketches plus the first frame of the test video. The decoding network does this by mixing three signals: the new frame is partly produced by warping the previous generated frame with predicted optical flow, and partly by a sketch-to-image translation network that pulls style from the reference frame; a mask network decides the per-pixel blend. The encoding side deletes the static background from the sketch, keeping only foreground edge pixels, and fills the background back in from the first and last frame sketches. In the authors' measurements this yields KID 0.1818 versus 0.2398 for the best baseline, LPIPS 0.3164 versus 0.3858, PSNR 21.18 versus 18.07, and SSIM 0.699 versus 0.550, while the compressed masked sketch costs only 2.9% more KID and 0.1% less SSIM than transmitting the full sketch.","pith_inferences":["If the static-background assumption is relaxed to a periodically refreshed background model, the same compressor could serve scenes with slow illumination changes or weather drift; the paper does not test that, but the fill-in equations are compatible with replacing the first and last frame sketches with updated background sketches.","The bitrate comparison is purely a source-coding comparison on H.264-encoded sketches; a full semantic communication system would still need a channel code, so the headline storage-and-transmission claim is only an upper-bound savings estimate, not an end-to-end system result.","A natural stress test is crowd and occlusion density: the encoder depends on video instance segmentation, so scenes with heavy occlusion or tiny objects would likely degrade both the mask quality and the reconstruction, a regime worth measuring before deployment.","The same sketch-plus-reference translation mechanism could be reused for other camera types, such as thermal or fisheye, where edge structure carries the geometry and the reference frame supplies the style."],"forward_implications":["A new surveillance camera can be put into service after collecting only six short training videos; the reference first frame then drives reconstruction of all later clips from sketches.","Transmitting a masked sketch stream plus a reference frame consumes less bandwidth than the raw video, with the largest savings at high bitrate: under H.264 QP 30 the masked sketch is about 43% smaller than the raw video, while at QP 50 the difference almost disappears.","When a foreground object never moves across the whole clip, its instance mask has high intersection over union and is treated as background, so it is pruned from the sketch; the object then disappears from the reconstruction until it moves.","The image translation network alone beats SGA on perceptual metrics (KID, LPIPS) but loses on PSNR and SSIM; the optical-flow and mask modules are what give the full method its pixel-level advantage.","Statistical significance claims (ANOVA and paired t-tests, all p<0.01 on the headline comparisons) mean the differences are not plausibly sampling noise within this dataset."],"supporting_citations":[{"why":"Supplies the deep edge-detection network that produces the sketch stream used as semantic information.","marker":"[14]"},{"why":"Supplies the video instance segmentation network that yields the foreground masks on which sketch compression is built.","marker":"[15]"},{"why":"Provides the masked-and-adaptive transformer backbone for the feature-alignment stage of the image translation network.","marker":"[16]"},{"why":"Provides adaptive instance normalization, the mechanism that injects the reference image's global style into the decoder.","marker":"[17]"},{"why":"Supplies the vid2vid backbone architecture on which the decoding network is built and is the strongest baseline on KID.","marker":"[12]"},{"why":"Defines the few-shot vid2vid baseline, which the proposed method outperforms on LPIPS, PSNR, and SSIM.","marker":"[18]"},{"why":"Defines the SGA baseline, which the proposed image translation network beats on KID and LPIPS but not on PSNR and SSIM.","marker":"[19]"}],"fun_headline_variants":["Six clips per camera rebuild surveillance video","Sketch-based codec needs just six training clips","Few-shot sketch codec beats vid2vid for surveillance","Surveillance video from six clips + sketch","Semantic codec: sketch + six clips rebuild frames"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the scene background is static and visible in the first and last frames of every clip, so any pixel not covered by a moving foreground can be filled from those two frame sketches; if lighting, weather, or camera position drifts, or a foreground object stays on screen the whole clip, the fill is wrong and both quality and bitrate savings degrade.","fun_headline_variants_meta":{"raw":{"variants":["Six clips per camera rebuild surveillance video","Sketch-based codec needs just six training clips","Few-shot sketch codec beats vid2vid for surveillance","Surveillance video from six clips + sketch","Semantic codec: sketch + six clips rebuild frames"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000196,"raw_usage":{"total_tokens":1373,"prompt_tokens":969,"completion_tokens":404,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":585,"completion_tokens_details":{"reasoning_tokens":330}},"tokens_in":585,"tokens_out":404,"duration_ms":4363,"temperature":1.0,"reasoning_tokens":330,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:17:57.543488+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Record a stationary-camera surveillance scene across an hour that spans dusk, when lighting changes slowly and outdoor shadows move; encode it with the masked-sketch pipeline and compare reconstruction PSNR and SSIM against the paper's reported 0.8% PSNR gap between masked and original sketch. If the gap becomes large, or if the instance-segmentation network marks a permanently parked car as static background and the car disappears from reconstruction, the static-background premise is falsified for that regime.","supporting_citations":[{"cited_title":"Soria, Y","cited_arxiv_id":null,"evidence_quote":"Supplies the deep edge-detection network that produces the sketch stream used as semantic information."},{"cited_title":"Jiang, F","cited_arxiv_id":null,"evidence_quote":"Provides the masked-and-adaptive transformer backbone for the feature-alignment stage of the image translation network."},{"cited_title":"Huang, and S","cited_arxiv_id":null,"evidence_quote":"Provides adaptive instance normalization, the mechanism that injects the reference image's global style into the decoder."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the SGA baseline, which the proposed image translation network beats on KID and LPIPS but not on PSNR and SSIM."}],"review_version":1}