{"id":"72e4610f-d567-4590-8e12-b05ef8215122","arxiv_id":"2411.11717","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"RAWMamba unifies image and video sRGB-to-RAW de-rendering with a metadata embedding module and a local tone-aware Mamba backbone, reporting 2.26 dB to 3.37 dB PSNR gains over prior task-specific models.","lead":"RAWMamba is a neural network that reconstructs camera RAW data from ordinary sRGB images and videos using a small amount of reference RAW information. It combines a unified metadata embedding module with a Mamba-based sequence model, reporting large PSNR gains on benchmark image and video de-rendering tasks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Video SSIM drops below RVD baseline despite SSIM loss in training, undermining the unqualified state-of-the-art claim.","rationale":"The reader's verdict is CONDITIONAL, which is appropriate. The reader's formal weakest_assumption focuses on the UME transfer claim (Section 3.3), but the most load-bearing issue for the central claim is the empirical inconsistency between the reported video PSNR gain and the SSIM regression relative to RVD. The reader did mention the SSIM drop in the rationale, but did not single it out as the weakest point. I believe the SSIM regression more directly threatens the 'state-of-the-art' and 'high-quality reconstruction' claims because it is an observed metric contradiction, not a speculative assumption about module behavior. The transfer assumption is plausible and could be corrected by learning; the SSIM gap is an unexplained result in the paper's own evaluation. Therefore, I partially agree with the reader: the same set of concerns is identified, but I would emphasize the metric inconsistency over the transfer assumption. The proposed test—re-running both methods under identical evaluation—would settle whether the SSIM gap is real or an artifact of protocol differences, and would determine whether the SOTA claim needs qualification. I do not see grounds to reject the paper; the architecture and results are promising, but the evaluation needs tightening, which matches the CONDITIONAL verdict.","tokens_in":13636,"tokens_out":4384,"duration_ms":41120,"concrete_test":"Obtain or re-implement RVD's official evaluation code and run both RVD and RAWMamba on the same RVD-Part2 test frames with identical normalization, crop, and border handling, computing per-frame PSNR and SSIM. If RAWMamba's average SSIM remains below RVD's 0.9983 under this controlled comparison, the paper must either explain the trade-off (e.g., SSIM computed differently) or qualify the state-of-the-art claim. Additionally, inspect whether the SSIM drop is concentrated in high-motion frames, which would point to a failure of the UME alignment rather than a global metric difference.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is state-of-the-art sRGB-to-RAW de-rendering on both image and video benchmarks. On RVD-Part2 (Table 2), RAWMamba reports PSNR 51.97 dB vs RVD's 49.71 dB (+2.26 dB) but SSIM 0.9965 vs RVD's 0.9983 (−0.0018). This is not a negligible gap, and the paper's training loss explicitly includes SSIM loss with λ=0.5 (Eq. 18), so a lower SSIM than the baseline is surprising and unexplained. The paper highlights only the PSNR gain ('2.26 dB improvement in PSNR', Section 4.3) and never discusses the SSIM regression. Since SSIM is a recognized perceptual quality metric, a method that trades PSNR for lower SSIM has not unambiguously achieved 'high-quality RAW data reconstruction' as claimed in the abstract. Moreover, the comparison relies on numbers 'reported in the paper' for RVD rather than re-runs under an identical evaluation protocol, so it is unclear whether the SSIM gap reflects a genuine trade-off or a difference in metric implementation (e.g., SSIM on 4-channel RAW vs. luminance, or different border handling). This directly affects the empirical support for the strongest claim; the UME transfer assumption, while conceptually interesting, is a mechanism-level concern that end-to-end training could compensate for, whereas the SSIM drop is an observed inconsistency with the headline result.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces RAWMamba, a unified architecture for sRGB-to-RAW de-rendering that handles both images and videos within a single network design. The method comprises two modules: the Unified Metadata Embedding (UME) module, which aligns sampled pixel metadata (images) or first-frame metadata (videos) with the sRGB input through a global affinity matrix and a local deformable attention mechanism, and the Local Tone-Aware Mamba (LTA-Mamba) module, which uses local (3D Hilbert) and global (raster) scans to propagate the sparse metadata across the whole frame or sequence. The authors report state-of-the-art results on the CAM image benchmark (53.13 dB PSNR vs. 49.76 dB for CAM) and on the RVD-Part2 video benchmark (51.97 dB PSNR vs. 49.71 dB for RVD), along with an ablation of the proposed modules.","tokens_in":13989,"tokens_out":8000,"duration_ms":67699,"significance":"The motivation for a single architecture covering image and video de-rendering is practical and the proposed design is technically sound. The use of a state-space model for long-range metadata propagation is a reasonable extension of recent Mamba-based vision models, and the reported ablations show clear contributions from each module. If the empirical results withstand a careful re-evaluation under a strictly fair protocol, the gains over task-specific baselines are substantial and would make this a strong candidate for publication. However, the video SSIM regression and the reliance on externally reported baseline numbers are concerns that must be addressed before the claims can be fully trusted.","major_comments":[{"comment":"RAWMamba reports a PSNR of 51.97 dB on RVD-Part2, exceeding RVD by 2.26 dB, but its SSIM (0.9965) is lower than RVD's (0.9983). Since the training loss in Eq. (18) includes an SSIM term with weight 0.5, this regression is surprising and is not discussed in the paper. Please provide an analysis of this trade-off, verify that the SSIM computation is identical to that used for the RVD baseline (e.g., same handling of 4-channel RAW, same borders), or report additional perceptual metrics that support the claim of higher reconstruction quality.","section":"Section 4.3, Table 2"},{"comment":"The comparisons against RIR, SAM, INF, and RVD rely on numbers reproduced or reported by other papers, not on re-runs with the authors' own code. Given the observed SSIM discrepancy, the evaluation protocol could differ between methods. Please either re-run these baselines under the exact same evaluation settings, or provide a detailed description of the protocol (SSIM computation, data pre-processing, border handling, etc.) and ensure it is consistent with the baseline papers.","section":"Sections 4.2 and 4.3"},{"comment":"The paper claims a 'unified' model for image and video, but the experiments train image and video models separately with different hyperparameters (batch size, epochs, input size). It is not demonstrated that a single set of weights can handle both tasks. Please clarify whether the architecture is unified but the models are task-specific, or provide a joint-training experiment and report the resulting performance.","section":"Sections 1 and 4.1"},{"comment":"The UME module relies on the assumption that a mapping learned between sRGB data and the sRGB component of metadata is 'inherently applicable to the RAW domain.' This is a strong claim that is not empirically supported. Please provide evidence (e.g., an ablation that uses mismatched metadata, a visualization of the affinity matrix, or a discussion of why this transfer is justified) or temper the statement to reflect that the mapping is learned jointly in an end-to-end framework.","section":"Section 3.3"}],"minor_comments":[{"comment":"The title contains an unintended space: 'RA WMamba' should be 'RAWMamba'.","section":"Title"},{"comment":"The phrase 'We preserves the original splits' should be 'We preserve the original splits'.","section":"Section 4.1"},{"comment":"The sentence 'to encode the raw and sRGB images into a shared latent feature space' is ambiguous because both x_srgb and x'_srgb are sRGB for the input and metadata; consider revising to 'to encode the inputs and metadata into a shared latent feature space'.","section":"Section 3.3"},{"comment":"The notation for the depth-wise convolution is inconsistent: Eq. (15) uses 'DWC' while Figure 3 describes 'DWConv'.","section":"Section 3.4.1"},{"comment":"References [1] and [2] are identical (Brooks et al.) and should be merged or renumbered.","section":"References"},{"comment":"The paper does not report model size, FLOPs, or runtime, despite the efficiency motivation for Mamba; please add these numbers to support the efficiency claims.","section":"Section 4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses an important problem and the proposed architecture is interesting, but the experimental validation has unresolved issues: the video SSIM regression contradicts the state-of-the-art claim, and the baseline comparisons are not fully controlled. These concerns are addressable through additional experiments and clarification, so I recommend major revision rather than rejection. The authors should also carefully re-examine the 'unified' claim, as the current experiments do not demonstrate a single model for both image and video."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is the first Mamba-based unified sRGB-to-RAW de-rendering network for both images and videos, and it reports big PSNR jumps—53.13 dB vs. 49.76 dB on CAM, 51.97 dB vs. 49.71 dB on RVD-Part2. The core idea is sensible and the ablations are honest: removing GEB, LEB, or the local/global Mamba blocks consistently drops performance. The UME module treating all metadata as a unified (sRGB, RAW) pair is a neat way to sidestep the task-specific architectural split, and the design rationale (local tone mapping needs local scanning, global consistency needs global scanning) is reasonable.\n\nThe main soft spot is the video result. The paper reports 0.9965 SSIM on RVD-Part2 while RVD gets 0.9983—despite the training loss explicitly including SSIM loss with λ=0.5 (Eq. 18). The paper only advertises the +2.26 dB PSNR and never mentions this regression. That is an observed inconsistency with the headline claim, not a nitpick. It could be a metric-implementation difference (e.g., SSIM on 4-channel RAW vs. luminance or different borders), but the authors need to explain it rather than ignore it.\n\nThe other issue is the comparison protocol. Image baselines for RIR and SAM come from CAM's reproduction, and video baselines are taken from RVD's paper. No code, no error bars, no re-runs under a unified protocol. That's normal for this subfield but it means the absolute numbers should be read with caution. The gains are large enough that I doubt they'll vanish on re-evaluation, but the SSIM gap specifically could be an artifact.\n\nThe UME transfer assumption—that the sRGB-to-sRGB mapping transfers to the RAW domain—is a heuristic, and end-to-end training could compensate even if the strict mapping is imperfect. Not a fatal flaw.\n\nWho this is for: anyone working on computational photography, camera pipelines, or RAW-based vision. This is a solid, well-motivated paper that deserves referee time, but not desk-reject or automatic accept. I'd want the authors to address the SSIM discrepancy, release code, and ideally re-run or clearly justify the baselines.\n\nRecommendation: send to peer review. The architecture is new and the problem is real. But the current write-up overstates the result, and a careful reviewer should insist on the missing evidence.","headline":"A genuinely unified Mamba-based sRGB-to-RAW de-rendering architecture with large reported PSNR gains, but the unqualified state-of-the-art claim is undercut by an unexplained video SSIM regression and third-party baselines.","tokens_in":14494,"tokens_out":1722,"would_cite":true,"duration_ms":16899,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single state-space model unifies sRGB-to-RAW reconstruction for photos and video.","keywords":["sRGB-to-RAW de-rendering","RAW reconstruction","metadata-guided reconstruction","state space model","Mamba","unified image-video model","ISP inversion","local tone mapping"],"falsifier":"Train RAWMamba on the three CAM cameras and then evaluate it on a fourth camera of the same dataset (or on a camera with a materially different ISP) without fine-tuning, comparing its PSNR against a per-camera fine-tuned variant; if the cross-modal transfer claim is correct, the single offline model should stay within a small margin, while a large gap would contradict it. A more direct test is to apply the sRGB-derived affinity matrix alone to warp the RAW metadata toward the target RAW and check whether the warped RAW approximates the ground truth before any learned refinement, which would isolate whether the transfer is doing the work.","tokens_in":13415,"feed_emoji":"📷","tokens_out":5149,"duration_ms":46619,"temperature":0.7,"pith_summary":"This paper argues that sRGB-to-RAW de-rendering—recovering the camera's original linear sensor readings from a processed sRGB image—can be accomplished with one network for both still photos and video. Previous methods relied on task-specific architectures: image de-rendering uses a few sampled RAW pixels as guidance, while video de-rendering uses the first frame of an sRGB-RAW pair as guidance. RAWMamba unifies these by treating every form of guidance as an (sRGB, RAW) pair and learning a mapping from the sRGB side that is then applied to the RAW side. The paper reports that this single framework improves PSNR by 3.37 dB on the CAM image benchmark and by 2.26 dB on the RVD-Part2 video benchmark over the previous best task-specific models, without any online fine-tuning. The authors interpret this as evidence that a metadata-agnostic, state-space-model-based pipeline can replace incompatible image-specific and video-specific de-rendering systems.","feed_headline":"One model reconstructs RAW camera data from sRGB photos and video","feed_subtitle":"Beats task-specific rivals by 3.37 dB on images and 2.26 dB on video using one unified architecture.","key_machinery":"The central object is the Unified Metadata Embedding (UME) module, which converts any metadata type—sampled RAW pixels for images or a first-frame sRGB-RAW pair for video—into a common (sRGB, RAW) pair. UME encodes the target sRGB, the metadata sRGB, and the metadata RAW into a shared feature space, forms a soft affinity matrix $A_1 = \\mathrm{Softmax}(F_{\\mathrm{srgb}}^T F'_{\\mathrm{srgb}})$, and multiplies it by the RAW metadata features to produce a global embedding. A second branch adds position encodings (fixed for images, optical-flow-warped for video) and a deformable transformer offset to produce a local embedding; the two are summed into the final metadata embedding. The other component is LTA-Mamba, a bidirectional state space model with two scanning orders—3D Hilbert for local tone-mapping context and raster line-by-line for global consistency—combined with channel attention to reduce feature redundancy. Together these components turn sparse metadata into dense, globally consistent reconstruction cues that drive the final RAW reconstruction.","core_discovery":"The central claim is that the alignment mapping between the de-rendering sRGB image and the sRGB part of any metadata (sampled pixels or a first frame) transfers to the RAW domain, because sRGB and RAW are two modalities of the same underlying scene information. Using this mapping, the Unified Metadata Embedding (UME) module computes a global affinity matrix for ISP-wide operations like white balance and gamma correction, plus a locally corrected attention map with optical-flow-adjusted position encodings for local tone mapping, then uses both to pull aligned reference information out of the RAW part of the metadata. The reconstruction side adds a Local Tone-Aware Mamba (LTA-Mamba) module that scans features along a 3D Hilbert curve to capture local spatio-temporal context, then scans line-by-line for global consistency, propagating the sparse metadata cues over the entire image or video. The paper reports state-of-the-art results: 53.13 dB PSNR on the CAM image benchmark and 51.97 dB on the RVD-Part2 video benchmark, outperforming task-specific models without any online learning steps.","pith_inferences":["The transfer premise yields a testable prediction: UME should work for metadata types the paper did not try, such as a low-resolution RAW thumbnail or an ISP parameter vector, as long as an (sRGB, RAW) pair can be assembled; treating any partial information as such a pair would be a natural extension.","The use of a 3D Hilbert curve for local scanning suggests a spatial-locality prior, so ablating by substituting other space-filling curves (such as serpentine or Morton order) would reveal whether topological locality alone drives the gain or whether the specific curve matters.","If the sRGB-to-RAW mapping is truly modality-independent, the same affinity-based alignment could be exported to other paired modalities (e.g., depth-plus-RGB or multispectral pairs), turning UME into a general cross-modal alignment module rather than a de-rendering-specific one.","The reported numbers cover three cameras and one video subset, so a broader stress test across many camera ISPs would clarify whether the 'inherently applicable' claim generalizes beyond the training distribution."],"forward_implications":["One trained model can serve both still and video capture on the same camera, eliminating the need for separate image and video de-rendering deployments.","The 2.26 dB video gain over the previous chained model suggests that first-frame metadata can be propagated through long video sequences more effectively with linear-complexity Mamba scanning than with sequential CNN architectures.","If the cross-modal UME transfer holds, different metadata formats (sampling rate, first-frame, or other partial RAW cues) can be handled without redesigning the network, simplifying camera-side metadata selection.","The local-then-global scan design indicates that explicitly modeling local tone mapping helps invert the ISP even when the available RAW information is extremely sparse, which is directly relevant to low-bandwidth capture scenarios.","Because the image benchmark gain of 3.37 dB is obtained by the same unified model that also handles video, the paper's results imply that joint training across the two modalities does not degrade image-side reconstruction quality."],"supporting_citations":[{"why":"Supplies the CAM dataset, the content-aware metadata sampling baseline, and the reproduced numbers for RIR and SAM; it is the primary image de-rendering comparison target.","marker":"[33]"},{"why":"Introduces the RVD video de-rendering task with first-frame metadata; it is the central video baseline and dataset on which the 2.26 dB gain is measured.","marker":"[54]"},{"why":"Provides the INF baseline that reconstructs RAW from metadata via implicit neural functions, used as a video-domain comparison point.","marker":"[25]"},{"why":"Presents the spatially aware metadata approach that UME generalizes and that serves as an image-domain predecessor.","marker":"[36]"},{"why":"Supplies the selective state space model whose linear-complexity long-range modeling LTA-Mamba builds upon.","marker":"[12]"},{"why":"Provides the bidirectional state space scan strategy used inside the LTA-Mamba blocks.","marker":"[56]"},{"why":"Supplies the deformable transformer block used in the local embedding branch to correct alignment errors.","marker":"[57]"},{"why":"Provides the ConvNeXt backbone that encodes sRGB and metadata inputs into the shared feature space.","marker":"[30]"}],"fun_headline_variants":["Single Mamba model rebuilds RAW from sRGB in photos and video","One Mamba architecture unifies sRGB-to-RAW for images and video","RAWMamba: one state-space model for sRGB to RAW in images and video","Unified Mamba de-renders sRGB to RAW for both photos and video"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the alignment mapping learned between the sRGB image and the sRGB part of the metadata also holds for the RAW part of the metadata, because sRGB and RAW are treated as two views of the same scene information; if that transfer fails for cameras or ISP pipelines unlike those in the training set, the metadata embedding would be misaligned and the reported gains would not generalize.","fun_headline_variants_meta":{"raw":{"variants":["Single Mamba model rebuilds RAW from sRGB in photos and video","One Mamba architecture unifies sRGB-to-RAW for images and video","RAWMamba: one state-space model for sRGB to RAW in images and video","Unified Mamba de-renders sRGB to RAW for both photos and video"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000705,"raw_usage":{"total_tokens":3191,"prompt_tokens":968,"completion_tokens":2223,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":2137}},"tokens_in":584,"tokens_out":2223,"duration_ms":16727,"temperature":1.0,"reasoning_tokens":2137,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:12:35.876957+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train RAWMamba on the three CAM cameras and then evaluate it on a fourth camera of the same dataset (or on a camera with a materially different ISP) without fine-tuning, comparing its PSNR against a per-camera fine-tuned variant; if the cross-modal transfer claim is correct, the single offline model should stay within a small margin, while a large gap would contradict it. A more direct test is to apply the sRGB-derived affinity matrix alone to warp the RAW metadata toward the target RAW and check whether the warped RAW approximates the ground truth before any learned refinement, which would isolate whether the transfer is doing the work.","supporting_citations":[{"cited_title":"Learning srgb-to-raw-rgb de-rendering with content-aware metadata","cited_arxiv_id":null,"evidence_quote":"Supplies the CAM dataset, the content-aware metadata sampling baseline, and the reproduced numbers for RIR and SAM; it is the primary image de-rendering comparison target."},{"cited_title":"Leveraging frame affin- ity for srgb-to-raw video de-rendering","cited_arxiv_id":null,"evidence_quote":"Introduces the RVD video de-rendering task with first-frame metadata; it is the central video baseline and dataset on which the 2.26 dB gain is measured."},{"cited_title":"Metadata- based raw reconstruction via implicit neural functions","cited_arxiv_id":null,"evidence_quote":"Provides the INF baseline that reconstructs RAW from metadata via implicit neural functions, used as a video-domain comparison point."},{"cited_title":"Spatially aware metadata for raw reconstruction","cited_arxiv_id":null,"evidence_quote":"Presents the spatially aware metadata approach that UME generalizes and that serves as an image-domain predecessor."},{"cited_title":"Vision mamba: Efficient visual representation learning with bidirectional state space model","cited_arxiv_id":null,"evidence_quote":"Provides the bidirectional state space scan strategy used inside the LTA-Mamba blocks."}],"review_version":1}