{"id":"d72c2f1a-2e5b-4005-9ce0-2cccd2ef0a57","arxiv_id":"2411.10773","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A single end-to-end neural network performs RAW-to-RGB conversion and image compression jointly, reporting rate-distortion gains over separate ISP-plus-codec baselines.","lead":"RealCamNet is a neural network that takes raw camera sensor data and outputs a compressed, displayable image in one step, instead of the usual separate processing stages. The authors report better image quality at lower bitrates and faster decoding than several standard pipelines, based on a new real-world dataset.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1 does not establish the 'best rate-distortion' claim: the multi-stage baselines are not shown to be retrained on the authors' RAW-RGB data, and the closest end-to-end competitor RAWtoBit is cited but never compared.","rationale":"The reader's weakest assumption was the private dataset's alignment and ground-truth validity. That is a real concern, but the more immediately load-bearing issue for the abstract's 'best rate-distortion' claim is that Table 1 lacks the experimental controls needed to attribute the reported gains to end-to-end joint optimization rather than to unequal training conditions or domain mismatch. If the baselines were never trained on RealCamNet's data, the comparison is not a test of the architecture; it is a test of cross-dataset generalization. The missing RAWtoBit comparison compounds this because the one existing fully end-to-end method is the most direct alternative to the paper's central claim. The ablation study cannot rescue the point because it never varies the training regime. These weaknesses are all addressable with additional experiments, so the reader's CONDITIONAL verdict remains appropriate; no change to the verdict is needed, but the acceptance conditions should explicitly require the controlled comparison. I did not find a deeper internal inconsistency in the method itself: the CADR/CIMC architecture is coherent, and the use of absolute coordinates is a reasonable fix for position-dependent distortion; the concern is entirely about the evidence supporting the headline empirical claim.","tokens_in":14825,"tokens_out":7386,"duration_ms":77580,"concrete_test":"Retrain the two strongest ISP baselines, LiteISPNet and MwISPNet, from scratch on the same 4057 RealCamNet training pairs using their original losses and schedules, keep the codec fixed (e.g., MLIC and VTM), evaluate at matched bitrates on the same 450 test pairs, and recompute BD-PSNR and BD-Rate. If the margin over the best baseline shrinks to within ~0.5 dB or ~10% BD-Rate, the headline 'best rate-distortion' claim is not supported; if the ~2.9 dB / ~39% margin persists after retraining, the claim survives. Running RAWtoBit under the same protocol would additionally test the strongest end-to-end alternative.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Abstract; §4.3) is that RealCamNet achieves the best rate-distortion performance. That claim rests entirely on Table 1, but Table 1 is not a controlled comparison. The paper never states that PyNet, MwISPNet, or LiteISPNet were retrained or fine-tuned on the 4057 training pairs described in §4.1; the appendix that would document baseline training and dataset construction is referenced (§4.1, §4.8) but absent from the arXiv version. If published weights are used, the reported 2.96 dB BD-PSNR and -39.08% BD-Rate versus PyNet+VTM largely measure cross-domain and sensor mismatch rather than the benefit of end-to-end joint optimization. The one true end-to-end competitor, RAWtoBit [21], is discussed in §2.2 but missing from Table 1, so 'best' is not tested against the strongest alternative architecture. The ablation in Table 2 also does not isolate joint training: it adds CADR, CSA, GFT, and LFT, but never trains the same modules with stage-wise ISP and compression losses. The internal numeric inconsistencies (text reports 2.26 dB and 9.53%, while Table 1 reports 2.9603 dB and -39.0842%) further weaken confidence, but the missing control is the decisive issue.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents RealCamNet, an end-to-end neural camera imaging pipeline that maps RAW captures to compressed RGB through a single jointly optimized encoder-entropy-decoder network. The encoder combines a Coordinate-Aware Distortion Restoration (CADR) module that uses absolute pixel coordinates with a Coordinate-Independent Mapping Compression (CIMC) module that performs tone mapping, denoising, demosaicing, and feature compression via channel-spatial attention and global/local feature transforms. The authors introduce a self-collected RAW-RGB dataset of 4507 image pairs and report rate-distortion comparisons against PyNet, LiteISPNet, and MwISPNet combined with VTM, TCM, and MLIC, claiming superior BD-PSNR, BD-MS-SSIM, BD-ΔE, BD-LPIPS, and BD-Rate with lower inference time.","tokens_in":15070,"tokens_out":4534,"duration_ms":41044,"significance":"If the central claim held, the paper would provide evidence that a single jointly optimized network can outperform multi-stage ISP-then-codec pipelines on real-world captures, and the CADR absolute-coordinate embedding is a simple, plausible mechanism for correcting fixed-pattern optical distortions. The manuscript also supplies useful complexity measurements and a receptive-field analysis. However, the evaluation is not presently reproducible or controlled: the companion dataset details and appendix are absent, the closest end-to-end competitor is not compared, and internal numeric inconsistencies prevent the reader from accepting the quantitative claims at face value. With the missing controls supplied, the work could be a useful contribution to learned ISP-plus-compression systems.","major_comments":[{"comment":"The text reports improvements of 2.26 dB in PSNR, 0.71 dB in MS-SSIM, 0.01 in LPIPS, 0.9187 in ΔE, and a 9.53% BD-Rate improvement relative to PyNet+VTM, while Table 1 lists BD-PSNR of 2.9603 dB, BD-MSSSIM of 1.6392 dB, BD-LPIPS of -0.0162, BD-ΔE of -1.1709, and BD-Rate of -39.0842%. These are not rounding differences; the paper must reconcile the text and table before the central rate-distortion claim can be evaluated.","section":"§4.3 / Table 1"},{"comment":"RAWtoBit [21], identified in §2.2 as a fully end-to-end camera ISP network, is absent from Table 1 and Fig. 6(a). Because the abstract claims 'best rate-distortion performance,' the closest architectural competitor should be included in the comparison, or the omission should be explicitly justified.","section":"§2.2 / Table 1 / Fig. 6(a)"},{"comment":"The manuscript never states whether PyNet, LiteISPNet, and MwISPNet were retrained or fine-tuned on the authors' private RAW-RGB training pairs or used with published weights. If published weights were used, the reported gains conflate end-to-end joint optimization with cross-dataset and sensor mismatch; the baseline training protocol must be documented.","section":"§4.3 / Table 1"},{"comment":"Dataset construction, alignment, ground-truth generation, and visual results are deferred to an appendix that is absent from arXiv v1. The entire evaluation depends on this private dataset, so without those details—and preferably a public release—the RAW-RGB reference pairs cannot be checked for alignment, tone-curve consistency, or independence from the training targets.","section":"§4.1 / §3.4 / §4.8 / Appendix"},{"comment":"The ablation in Table 2 does not isolate end-to-end joint training. It adds CADR, CSA, GFT, and LFT one by one, but never trains the same set of modules with stage-wise ISP and compression losses. Consequently, the claimed 'performance improvements from end-to-end joint optimization' are not supported by the presented experiments; a stage-wise trained control is needed.","section":"Table 2 / §4.4-4.5"},{"comment":"No error bars, confidence intervals, or significance tests are reported for any BD metric, despite the metrics being computed on a private 450-image test set. Given the small test set and the sensitivity of BD-PSNR and BD-Rate to the fitted RD curves, the authors should report variance across multiple training runs or bootstrap estimates.","section":"Table 1 / Fig. 6"}],"minor_comments":[{"comment":"The header contains 'BD-LPPHS,' which should presumably be 'BD-LPIPS.'","section":"Table 1"},{"comment":"The reference to 'Table6(a)' should be 'Fig. 6(a)' or a properly labeled table.","section":"§4.6"},{"comment":"The ΔE sign convention should be defined explicitly; in Table 1 negative BD-ΔE is favorable, but §4.3 reports a positive value without a sign convention.","section":"§4.2 / Table 1"},{"comment":"The conversion of MS-SSIM to −10 log10(1−MS-SSIM) should be clarified: state whether the BD-MSSSIM values in Table 1 are in these converted dB units and identify the anchor curve used for the Bjøntegaard calculation.","section":"§4.2"},{"comment":"Several labels in Fig. 3 contain typographical artifacts ('restorate', inconsistent CIMC formatting), and the text in Fig. 6(b) is small and hard to read; higher-quality figures would improve the presentation.","section":"Fig. 3 and Fig. 6"}],"recommendation":"major_revision","confidential_remarks":"The paper has no code or dataset link in the arXiv version, and the missing appendix contains exactly the information needed to judge the dataset and the visual comparisons. I would advise the editor to require the appendix, the RAWtoBit comparison, and a clear baseline-training statement before a second round, since the current version is not independently verifiable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a real architecture paper, not a toy, but treat Table 1 as a claim, not a result. The contribution is the specific combination: CADR with absolute coordinates feeding a learned entropy-coded bottleneck, plus CPE global/local color priors, all jointly trained RAW->bitstream->RGB. RAWtoBit is the closest prior, and the absolute-coordinate embedding is a clean fix to the crop-coordinate ambiguity in earlier work. I buy that the modules do what the text says; the math is straightforward and there is no internal contradiction.\n\nWhat the paper does well: it identifies a concrete sensor problem (vignetting/dark shading) and designs a minimal mechanism (coordinate embedding multiplied into features) that is easy to test; the ablation shows monotone gains from CADR, CSA, GFT, LFT; and the reported FLOPs/latency are specific and favorable. If the numbers are honest, this would be useful for mobile imaging.\n\nSoft spots, in order of severity.\n\nFirst, the evaluation is not controlled. The central claim is \"best RD against state-of-the-art multi-stage pipelines\", but Table 1 never says whether PyNet/LiteISP/MwISP were retrained or fine-tuned on the authors' 4057 training pairs. If they used published weights, the large BD gains mostly measure sensor/domain mismatch, not joint optimization. The text even reports numbers that don't match Table 1: 2.26 dB / 9.53% vs 2.9603 dB / -39.08%. That is the kind of inconsistency I need resolved before trusting the rest.\n\nSecond, the one method that would actually test the end-to-end idea, RAWtoBit, is cited in §2.2 and then never appears in Table 1. Without that comparison, the paper does not establish that joint optimization beats a strong end-to-end baseline.\n\nThird, the dataset and appendix are missing from the arXiv version. Alignment and ground-truth generation are load-bearing: the CADR module's whole premise is that the target RGB is physically meaningful and coordinate-aligned. As it stands I can't verify that premise, nor the CPE details, nor the RawPy comparison in §4.8.\n\nMinor: no error bars, one test set, and the ablation doesn't include the staged-training control (same modules, separate losses), so \"joint training\" itself is not isolated.\n\nBottom line: the architecture is worth a serious look, and the idea of absolute coordinates for distortion restoration is worth remembering. But the paper as posted is a conditional reject, not because the method is flawed in principle but because the evidence for the headline claim is missing. I'd send it to review only after the authors release the dataset/code, add RAWtoBit, document baseline training, and fix the numbers.\n\nRecommendation: worth a serious referee if the authors are given a revision cycle; not desk-rejectable on novelty grounds. I would not cite it in its current form.","headline":"RealCamNet is a sensible end-to-end RAW-to-compressed-RGB design with one genuinely useful trick (absolute-coordinate distortion restoration), but the evaluation does not support 'best rate-distortion' because the only real end-to-end competitor is missing and the baselines are uncontrolled.","tokens_in":15661,"tokens_out":2347,"would_cite":false,"duration_ms":22686,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RealCamNet is a single jointly trained network that turns RAW sensor data into a compressed RGB image, doing the work of a staged ISP plus codec with better quality and speed.","keywords":["camera imaging pipeline","end-to-end learning","RAW-to-RGB conversion","learned image compression","coordinate-aware distortion restoration","lens shading correction","rate-distortion optimization","real-world imaging dataset"],"falsifier":"Test the same network on a fresh, independently validated RAW-RGB set from a different camera with its own vignetting and shading; if the reported BD-PSNR advantage over the best separated ISP-plus-codec pipeline shrinks to zero or reverses, the central claim fails. A cheaper control is to retrain with relative coordinates in place of absolute coordinates in CADR; if BD-PSNR does not drop, the coordinate-awareness mechanism is not what is carrying the gain.","tokens_in":14562,"feed_emoji":"📷","tokens_out":11642,"duration_ms":109538,"temperature":0.7,"pith_summary":"RealCamNet is a camera pipeline that runs the entire chain from RAW sensor data to a compressed RGB image in one jointly trained network. The authors argue that the traditional separation of image signal processing from compression forces each stage to be optimized independently, accumulating errors and wasting compute. Their key design move is to divide the pipeline's work into coordinate-dependent corrections, handled by a module that knows each pixel's true position in the full image, and coordinate-independent mapping and compression, handled by a module that performs tone mapping, denoising, demosaicing, and feature compression together. To train and test this system they assembled a real-world RAW-RGB dataset of 4,507 image pairs. On that dataset they report better rate-distortion performance than combining any of three learned ISP networks with any of three codecs, with lower inference latency and fewer operations.","feed_headline":"One joint network beats staged camera pipelines on quality and speed","feed_subtitle":"RealCamNet folds RAW-to-RGB conversion and compression into one model, reporting better rate-distortion at lower inference cost","key_machinery":"The load-bearing mechanism is the pair of modules CADR and CIMC inside the encoder. CADR computes the absolute coordinates $c^a_{i,j}=((i+m)/H,(j+n)/W)$ of every pixel relative to the full RAW image rather than the cropped patch, passes these coordinates through a small convolution-ReLU embedding, and multiplies the embedding into the feature map so the network can learn fixed position-dependent artifacts. CIMC interleaves Channel-Spatial Attention with Local and Global Feature Transformations, both of the form $y=\\alpha x+\\beta$, using color priors from a downsampled full image and from the local crop; this performs tone mapping, denoising, demosaicing, and compression in one coordinate-independent path. A Color Prior Extraction module supplies the global and local priors, and the decoder uses the same attention building blocks with an entropy-coded latent to reconstruct the RGB image.","core_discovery":"On the paper's own terms, the central discovery is that an end-to-end network that jointly optimizes RAW-to-RGB conversion and compression can outperform the current practice of chaining a learned ISP network to a separate codec. The mechanism is a deliberate split: the Coordinate-Aware Distortion Restoration (CADR) module feeds absolute pixel coordinates into the encoder so that fixed position-dependent optical and sensor defects such as vignetting and dark shading can be learned and removed, while the Coordinate-Independent Mapping Compression (CIMC) module combines global and local tone mapping, denoising, demosaicing, and compact latent compression in a single flow. Trained with a rate-distortion loss and evaluated against combinations of three learned ISP baselines with three codecs, RealCamNet is reported to gain 2.26 dB in BD-PSNR over the strongest separated pipeline, to reduce BD-Rate by 9.53 percent, and to decode 1024x1024 images at 16.8 frames per second while using fewer parameters and FLOPs.","pith_inferences":["The paper does not explore it, but the same absolute-coordinate embedding could also handle other fixed-pattern defects such as column noise, stuck pixels, or sensor-specific nonuniformity, effectively teaching the network the camera's spatial fingerprint.","The paper does not run this control, but a jointly trained version of a separated architecture with the same parameter budget would isolate how much of the reported gain is due to joint optimization itself rather than the CADR and CIMC modules.","The paper's dataset argument implies that a shared, carefully aligned RAW-RGB benchmark from real cameras could shift how ISP and compression are evaluated together; the authors' dataset, if released, would be the natural starting point."],"forward_implications":["A camera system built around one trained network could skip the separate ISP and codec stages, saving both computation and tuning effort while improving image quality at a given bitrate.","The absolute-coordinate embedding gives the network a way to correct lens shading and dark shading without explicit calibration data, because the network can memorize the spatial pattern of the sensor.","Because global color priors are extracted from the full image, tone mapping can remain consistent across large scenes while local feature transforms refine detail; the paper reports this improves color fidelity as measured by $\\Delta E$.","Joint optimization transfers quality gains across the whole pipeline: the ablation attributes a BD-PSNR improvement of over 3 dB to the full configuration relative to the base network.","At deployment the model decodes 1024x1024 images at 16.8 fps on a single consumer GPU, suggesting that the end-to-end approach is not only a quality claim but also a practical speed claim."],"supporting_citations":[{"why":"supplies one of the learned ISP baselines that RealCamNet must beat in the rate-distortion comparison","marker":"[19]"},{"why":"supplies another learned ISP baseline in the multi-stage comparison","marker":"[18]"},{"why":"supplies a third learned ISP baseline, which in combination with codecs forms the strongest separated competitor","marker":"[49]"},{"why":"supplies the learned compression module used as one of the codec baselines in the comparison","marker":"[31]"},{"why":"supplies the learned compression module used as the other learning-based codec baseline","marker":"[22]"},{"why":"supplies the traditional video coding standard used as the classic-codec baseline","marker":"[1]"},{"why":"supplies the relative-coordinate embedding method that CADR replaces with absolute coordinates","marker":"[50]"},{"why":"supplies the rate-distortion loss used to train RealCamNet","marker":"[7]"}],"fun_headline_variants":["End-to-end pipeline beats separate ISP and codec on quality and speed","One neural network for RAW to RGB and compression cuts latency","Joint optimization of camera imaging and compression outperforms chained systems","RealCamNet folds ISP and codec into one model, boosting BD-PSNR","End-to-end camera pipeline: better rate-distortion, fewer FLOPs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's evaluation rests on its privately collected RAW-RGB pairs being correctly aligned and representative; the construction, alignment, and ground-truth generation of this dataset are deferred to an appendix, and if those pairs are biased or misaligned the reported quality gains may not transfer to other cameras or scenes.","fun_headline_variants_meta":{"raw":{"variants":["End-to-end pipeline beats separate ISP and codec on quality and speed","One neural network for RAW to RGB and compression cuts latency","Joint optimization of camera imaging and compression outperforms chained systems","RealCamNet folds ISP and codec into one model, boosting BD-PSNR","End-to-end camera pipeline: better rate-distortion, fewer FLOPs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000345,"raw_usage":{"total_tokens":1903,"prompt_tokens":966,"completion_tokens":937,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":582,"completion_tokens_details":{"reasoning_tokens":842}},"tokens_in":582,"tokens_out":937,"duration_ms":9397,"temperature":1.0,"reasoning_tokens":842,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:19:12.739007+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Test the same network on a fresh, independently validated RAW-RGB set from a different camera with its own vignetting and shading; if the reported BD-PSNR advantage over the best separated ISP-plus-codec pipeline shrinks to zero or reverses, the central claim fails. A cheaper control is to retrain with relative coordinates in place of absolute coordinates in CADR; if BD-PSNR does not drop, the coordinate-awareness mechanism is not what is carrying the gain.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies another learned ISP baseline in the multi-stage comparison"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the learned compression module used as one of the codec baselines in the comparison"},{"cited_title":"Versatile Video Coding (H.266/VVC)","cited_arxiv_id":null,"evidence_quote":"supplies the traditional video coding standard used as the classic-codec baseline"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the relative-coordinate embedding method that CADR replaces with absolute coordinates"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the rate-distortion loss used to train RealCamNet"}],"review_version":1}