{"id":"a61a38e2-30e1-4925-a610-07723f4afbc1","arxiv_id":"2507.01588","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A multi-exposure HDR network using an overlapped, exposure-specific VQ codebook and a dual-decoder design outperforms prior HDR methods on the Kalantari and Hu benchmarks.","lead":"This paper proposes a new way to pretrain an image codebook for multi-exposure HDR reconstruction: code vectors are divided into overlapping groups for short, mid, and long exposures, while HDR images use all vectors. The authors add a dual-decoder network with pre-trained VQ features and report small but consistent gains over prior HDR methods on two benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation 4 with α=K/4 defines three disjoint K/4-element partial codebooks, contradicting the claimed K/4 overlap; the central OLC mechanism is absent from the printed math and only a corrected offset or released code can resolve it.","rationale":"The paper presents a coherent empirical pipeline with ablations supporting each added component, and the reported gains over strong baselines are plausible. The reader's CONDITIONAL verdict is therefore appropriate. The single most load-bearing issue is the internal inconsistency in Eq. 4: the printed definition of the partial codebooks makes them disjoint, which directly contradicts the proposed 'Overlapped' mechanism and the explicit example in the text. This is not a matter of disagreeing with the community consensus; it is a correctness risk in the paper's central construction. If the authors intended the offset (i−1)α, the equations become consistent and the mechanism is well-defined; if they intended Eq. 4 literally, the method does not implement what it claims. Because the discrepancy is plausibly a typo and could be resolved by a small correction or by releasing the training code, the appropriate outcome remains conditional acceptance pending that verification. I agree with the reader's weakest_assumption, which identifies exactly this equation-level inconsistency. No additional independent concern is needed to justify the conditional status.","tokens_in":12351,"tokens_out":3986,"duration_ms":45392,"concrete_test":"Recompute Eq. 4 with K=1024 and the paper's intended overlap: compute whether |Z_i ∩ Z_{i+1}| = K/4 for the printed indices; it will be 0. Then request the authors' training code or a corrected equation; if the intended masks are Z_i={z_{(i−1)α+1},...,z_{(i+1)α}}, rerun the K=1024 vanilla-vs-OLC comparison in Table 2 to confirm that the 0.32 dB PSNR-µ gain is actually produced by the overlapping masks, not by a different implementation choice.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that OLC's shared codebook structure improves HDR representation learning—rests on the partial codebooks Z_i actually overlapping. As printed, Eq. 4 defines Z_i = {z_{iα+1}, ..., z_{(i+1)α}} with α=K/4 and i∈{1,2,3}. This yields three consecutive, disjoint blocks of size K/4: Z_1={z_{K/4+1},...,z_{K/2}}, Z_2={z_{K/2+1},...,z_{3K/4}}, Z_3={z_{3K/4+1},...,z_K}. Pairwise intersections are empty, so the 'K/4 shared code vectors' described two sentences later cannot exist under this definition. The text also calls Z_i a (K/2)×n_z codebook, but Eq. 4 gives only K/4 elements. The example in the same paragraph—Z_1 and Z_2 sharing {z_{α+1},...,z_{2α}}—matches the corrected offset Z_i={z_{(i−1)α+1},...,z_{(i+1)α}}, not the printed one. Because the quantization in Eq. 5 conditions on Z' = Z_η, every LDR code assignment in the pretraining stage depends on these masks; if the implementation follows Eq. 4 literally, OLC is not overlapped and the attribution of Table 2's gains to overlapping codebooks is unsupported. No code or pretrained checkpoints are released, so the discrepancy cannot be resolved from the paper alone.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a two-step method for multi-exposure HDR imaging. In the first step, a VQGAN is trained with a proposed Overlapped Codebook (OLC), in which LDR frames are quantized with exposure-dependent subsets of a shared codebook while HDR images use the full codebook, with the aim of learning implicit HDR representations aligned with the exposure bracketing process. In the second step, a dual-decoder HDR network combines a pretrained VQ decoder with a fidelity decoder, using a parallel alignment module (PA), a frame-selective merging module (FSM), and a residual fusing module (RF). The method is evaluated on the Kalantari, Hu, and Tursun datasets, reporting improved PSNR, SSIM, HDR-VDP-2, and PU21 scores over prior methods, with ablations attributing gains to the OLC and the proposed modules.","tokens_in":12725,"tokens_out":2759,"duration_ms":30631,"significance":"If the proposed OLC mechanism works as described, the idea of structuring a VQ codebook to match the exposure fusion process is a novel and potentially valuable contribution to learning-based HDR imaging. The reported quantitative gains are consistent and replicated across several metrics and two public benchmarks, and the ablation study gives a plausible attribution of the gains to the OLC and the proposed modules. The paper also includes extensive qualitative comparisons and a code-index distribution analysis. However, the absolute improvements over the strongest prior methods are small (e.g., 44.89 vs. 44.64 PSNR-μ on Kalantari; 48.73 vs. 48.46 on Hu), no statistical significance or error bars are reported, and no code or pretrained models are released, which limits the strength of the central claim.","major_comments":[{"comment":"The printed definition of the partial codebook contradicts the described overlap and the visualization. With α=K/4, Eq. (4) defines Z_i = {z_{iα+1}, ..., z_{(i+1)α}} for i∈{1,2,3}, giving three disjoint blocks of size K/4: Z_1={z_{K/4+1},...,z_{K/2}}, Z_2={z_{K/2+1},...,z_{3K/4}}, Z_3={z_{3K/4+1},...,z_K}. These sets have empty pairwise intersections, so the claimed K/4 shared code vectors between adjacent LDR partial codebooks cannot exist under this definition. The text also calls Z_i a (K/2)×n_z codebook, whereas Eq. (4) yields only K/4 elements. The example in the same paragraph (Z_1 and Z_2 sharing {z_{α+1},...,z_{2α}}) matches the corrected indexing Z_i={z_{(i−1)α+1},...,z_{(i+1)α}}, not the printed one. Since every quantization step in Eq. (5) conditions on Z', this indexing error is load-bearing: if the implementation follows Eq. (4) literally, the OLC is not overlapped at all, and the Table 2 gains cannot be attributed to overlapping codebooks. The authors must correct the formula, state the actual indexing used in the implementation, and ideally release code or checkpoints to resolve the ambiguity.","section":"Sec. 3.1, Eq. (4) and Fig. 1(b)"},{"comment":"The loss weights in the two final training objectives are not reported. Eq. (10) defines L_OLC = λ_rec L_rec + λ_per L_per + λ_vq L_vq + λ_adv L_adv, and Eq. (16) defines L_HDR = L_rec + λ_per L_per + λ_map L_map, but no values or schedules are given for λ_rec, λ_per, λ_vq, λ_adv, or λ_map. Without these weights, the reported results are not reproducible, and the relative contribution of the VQ/OLC losses to the learned representation cannot be assessed. Please provide the exact values and any annealing schedules, as is standard in VQGAN-based training.","section":"Sec. 3.1, Eq. (10); Sec. 3.2, Eq. (16)"},{"comment":"The quantitative comparison reports only point estimates on test sets of 15 samples (Kalantari) and a similarly small synthetic set (Hu). The reported margins over the best prior methods are small (0.25 dB and 0.27 dB PSNR-μ, respectively), and no error bars, multiple-seed variances, or paired significance tests are given. Given the small test sets and the sensitivity of HDR metrics to a few saturated regions, the authors should report standard deviations over multiple training runs or at least a paired test across the test samples to support the claim of consistent improvement.","section":"Sec. 4.3, Table 1"},{"comment":"The notation in the mapping loss is confusing and potentially inconsistent. The text says L_map calculates the distance between the extracted feature z_gt in the HDR network and the ground-truth VQ representation z_gt = Q(E(H), Z), but both quantities are named z_gt and the equation as printed is L_map = ||z_vq − z_gt||_2^2. Please rename the extracted feature (e.g., z_vq) and the ground-truth quantized feature (e.g., z_gt), clarify the dimensions, and specify whether H is the ground-truth HDR image provided during training only. This is important because the mapping loss uses ground-truth latents as supervision, which is legitimate but must be precisely defined.","section":"Sec. 3.2, Eq. (15)"}],"minor_comments":[{"comment":"The caption states 'H.V-2 is HDR-VDP-2 metric,' which contains a typo; it should read 'HDR-VDP-2.'","section":"Sec. 4.3, Table 1 caption"},{"comment":"The adversarial loss definition L_adv = −E_Xhat[D(Xhat))] contains a mismatched parenthesis; the equation should read L_adv = −E_Xhat[D(Xhat)].","section":"Sec. 3.1, Eq. (10)"},{"comment":"The phrase 'horizon/vertical flipping' should be 'horizontal/vertical flipping.'","section":"Sec. 4.2"},{"comment":"The comparison on the Tursun dataset is qualitative only because no ground-truth HDR images exist; this limitation should be stated explicitly in the main text, not only in the figure caption, and the claim of 'more realistic HDR images' should be framed as a perceptual visual assessment rather than a quantitative superiority.","section":"Sec. 4.3 and Fig. 6(b)"},{"comment":"Several references use abbreviated author lists such as 'Grosch, T. & Others' and 'Van Den Oord, A., Vinyals, O. & Others.' Please use the complete author lists or the standard citation style of the venue to improve reproducibility of the bibliography.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core idea is interesting and the experimental work is substantial, but the Eq. (4) inconsistency is a serious correctness issue for the central mechanism, and the missing loss weights and lack of significance testing weaken reproducibility. I would like to see the corrected formula with a clear statement of the implemented indexing, and ideally a code release or at least a detailed implementation description, before the paper can be accepted. The paper also cites several recent Transformer-based HDR methods but does not provide code or checkpoints; this may affect its impact in the community."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a real incremental contribution — structuring a VQGAN codebook by exposure bracket, with overlapping segments, is new for HDR — and the ablations suggest it works. But there is a load-bearing inconsistency in the math, and the paper currently does not let you verify which version was actually trained.\n\nWhat's good. The OLC idea is sensible: tie the short/mid/long LDR frames to partially shared codebook segments and let the HDR image use the full codebook. That mirrors the classic triangle-weighting merge, and it's the first time I've seen the exposure-bracket structure baked into VQ representation learning. The dual-decoder design (pre-trained VQ decoder plus fidelity decoder with the RF affine fusion) is also a reasonable way to inject learned priors without giving up fidelity. The experiments are mostly well done: Table 2 compares OLC against vanilla VQ under identical settings, and OLC wins by a consistent margin; the module ablation in Table 3 is clean, with each addition helping. The visual results on saturated regions are plausible. This looks like honest engineering on top of known components.\n\nWhere it gets soft. Eq. 4 is wrong as written. With α=K/4 and i∈{1,2,3}, the sets are Z1={z_{K/4+1}..z_{K/2}}, Z2={z_{K/2+1}..z_{3K/4}}, Z3={z_{3K/4+1}..z_K} — pairwise disjoint. No K/4 overlap exists, despite the text two sentences later saying Z1 and Z2 share {z_{α+1}..z_{2α}}. The equation also claims a (K/2)×n_z matrix but lists only K/4 elements. This is not a cosmetic typo; the overlap is the whole point of OLC. If the implementation follows Eq. 4, Table 2's gains are from a partitioned codebook, not an overlapped one. If it follows the text and Fig. 1b, the equation needs an offset like (i−1)α+1. No code or checkpoints are provided, so a referee cannot tell which is true. That must be fixed before the central claim can be trusted.\n\nAlso: the loss weights in Eqs. 10 and 16 are never reported, and the headline margins (~0.2–0.3 dB PSNR-µ) come without error bars or significance testing. For a GAN-based pipeline that's a real gap; a lucky seed could explain part of the gap. Some prior methods are marked † and missing from several metrics, so the comparison table is not fully complete.\n\nWho it's for. Researchers in multi-exposure HDR and VQ-based restoration. They will want to see the corrected OLC definition and ideally code.\n\nRecommendation: send to peer review, conditional. The idea is worth airing and the experiments are salvageable, but Eq. 4 and the missing weights must be addressed. I would not cite it in my own work until the overlap definition is confirmed or code appears.","headline":"A genuinely new HDR codebook idea with solid ablations, but the central 'overlapped' mechanism is contradicted by Eq. 4 as printed; fix that and release code before trusting the headline claims.","tokens_in":13266,"tokens_out":3367,"would_cite":false,"duration_ms":39726,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that modeling exposure bracketing inside a vector-quantized codebook—via overlapping partial codebooks for short, mid, and long exposures—improves implicit HDR representation learning, and that the resulting dual-decoder…","keywords":["high dynamic range imaging","multi-exposure HDR","vector quantization","VQGAN","overlapped codebook","image representation learning","deghosting","exposure bracketing"],"falsifier":"Inspect the indexing in the released training or inference code (or, if no code is released, directly compute the intersection of the code vector sets used for the short and mid exposure frames in a trained OLC). If the intersection is empty for all codebook updates, the OLC's shared-representation mechanism is absent; the method would then reduce to a per-exposure codebook conditioning scheme, and the ablation against the vanilla codebook would need re-interpretation.","tokens_in":12172,"feed_emoji":"📷","tokens_out":7340,"duration_ms":75874,"temperature":0.7,"pith_summary":"The paper proposes a two-step approach to multi-exposure high dynamic range (HDR) imaging. First, it trains a VQGAN with an Overlapped Codebook (OLC) in which short, mid, and long exposures are quantized with partially shared code vectors, while HDR images use the full codebook; this mirrors the triangular-weight fusion of exposure bracketing and is intended to make the codebook learn HDR representations as combinations of LDR representations. Second, it builds an HDR reconstruction network with two decoders: a frozen pre-trained VQ decoder that injects these learned HDR priors, and a fidelity decoder augmented with parallel alignment, frame-selective merging, and residual fusing modules. On the Kalantari and Hu benchmarks, the proposed method reports the best PSNR and HDR-VDP-2 scores, with the largest gains in saturated and motion-affected regions. If correct, the work suggests that structuring a codebook to match the physics of exposure fusion is a useful inductive bias for learning image representations.","feed_headline":"Overlapped codebook pushes HDR reconstruction past prior best","feed_subtitle":"A VQGAN whose code vectors mirror exposure bracketing reports top PSNR on Kalantari and Hu benchmarks.","key_machinery":"The key machinery is the Overlapped Codebook (OLC): a master codebook $\\mathcal{Z}=\\{z_k\\}_{k=1}^K$ partitioned into overlapping segments so that the quantization of an input image of type $\\eta$ (short, mid, long, or HDR) uses a conditioned partial codebook $\\mathcal{Z}'$: $\\mathcal{Z}_\\eta$ for LDR frames and the full $\\mathcal{Z}$ for HDR. During training, an input type $\\eta \\sim \\mathrm{Unif}[1,4]$ is sampled, the encoder produces features $\\bar{z}=E(X)$, and quantization $\\hat{z}_j = \\arg\\min_{z_k \\in \\mathcal{Z}'}\\|\\bar{z}_j - z_k\\|_2$ maps each feature to the nearest code vector, with straight-through gradient copying and a commitment loss. The pre-trained VQ decoder is then used in the HDR network, where a Parallel Alignment module aligns non-reference frames, a Frame-Selective Merging module aggregates valid frame contexts, and a Residual Fusing module combines VQ decoder features with context via an affine transformation and residual connection.","core_discovery":"The central discovery is that the representation space of a VQGAN can be organized to reflect the exposure-bracketing process, so that HDR content is represented by the union of code vectors used for the LDR frames rather than by an unrelated set. Concretely, the OLC uses three partial codebooks $\\mathcal{Z}_1,\\mathcal{Z}_2,\\mathcal{Z}_3$, each meant to contain $K/2$ of the total $K$ code vectors and to overlap with its neighbors by $K/4$ vectors, while the HDR image is quantized against the full codebook. A quantization-conditioning scheme selects the partial or full codebook based on whether the input is an LDR frame or an HDR image. The authors show that this scheme trains a VQ decoder whose latent features, when fused into a dual-decoder HDR network, recover saturated regions and fine details better than the same network with a vanilla codebook, and better than prior CNN- and Transformer-based HDR methods on two public benchmarks; for example, PSNR-$\\mu$ reaches 44.89 versus 44.64 on Kalantari and 48.73 versus 48.46 on Hu.","pith_inferences":["The OLC idea could be applied to other multi-frame fusion tasks—e.g., low-light burst photography or multi-exposure super-resolution—where the input frames share content but differ by a known physical parameter such as exposure, gain, or focus.","Because the printed equation (4) defines disjoint partial codebooks, the reported behavior depends on the implementation actually using overlapping indexing; verifying the code would determine whether the gains come from the overlap or from the conditioning scheme alone.","The shared code vectors between neighboring exposures could be inspected to see if they encode exposure-invariant content such as edges and textures, while the exclusive vectors encode exposure-specific radiance; if so, the codebook becomes an interpretable exposure dictionary.","The mapping loss $\\mathcal{L}_{\\mathrm{map}}$ that ties the HDR network's latent features to the pre-trained codebook suggests a general recipe: any restoration network can be regularized by a fixed VQ representation, which may help in domains beyond HDR."],"forward_implications":["The OLC improves HDR reconstruction without adding inference-time computation, since the extra cost is paid only during VQGAN training.","The frozen VQ decoder provides reusable HDR priors: the same pre-trained OLC and decoder are plugged into the HDR network without fine-tuning.","On Kalantari's dataset the method reports PSNR-$\\mu$ 44.89 versus 44.64 for the previous best, and on Hu's dataset 48.73 versus 48.46, with similar gains in HDR-VDP-2.","Larger codebooks benefit more from the overlap structure (K=1024 gives a larger improvement over vanilla than K=512), so OLC is a scalable way to exploit big codebooks.","Qualitative comparisons on the Tursun dataset, which has no ground truth, show the method recovering saturated regions that prior attention- and transformer-based methods wash out."],"supporting_citations":[{"why":"This paper introduces VQGAN, the framework that the OLC extends by adding the overlapped codebook structure.","marker":"[5]"},{"why":"This paper introduces the vector quantization mechanism and the straight-through gradient estimator that the OLC training relies on.","marker":"[4]"},{"why":"This paper supplies the Kalantari dataset used for training and evaluation, and the triangular exposure-fusion formulation (Eq. 1-2) that the OLC mirrors.","marker":"[7]"},{"why":"This paper supplies the Hu synthetic dataset used as a second benchmark for quantitative comparison.","marker":"[24]"},{"why":"This paper presents AHDRNet, an attention-based baseline that motivates implicit feature-level alignment and is included in the comparisons.","marker":"[9]"},{"why":"This paper presents CA-ViT, a transformer-based baseline that the proposed method compares against and outperforms.","marker":"[16]"},{"why":"This paper presents SCTNet, a semantics-consistent transformer baseline used for comparison in the experiments.","marker":"[36]"}],"fun_headline_variants":["Overlapping codebooks mirror exposure bracketing for sharper HDR","Shared codebook structure lifts HDR quality past state of the art","VQGAN codebook organized by exposure yields best HDR yet","Exposure-bracketed codebook improves multi-exposure HDR fusion","OLC scheme helps VQGAN learn true HDR representations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claimed benefit depends on the partial codebooks used for adjacent LDR frames actually sharing code vectors during training and inference; if the implementation uses disjoint segments, the overlap mechanism described as the core of OLC does not exist, and the reported gains must come from a different component.","fun_headline_variants_meta":{"raw":{"variants":["Overlapping codebooks mirror exposure bracketing for sharper HDR","Shared codebook structure lifts HDR quality past state of the art","VQGAN codebook organized by exposure yields best HDR yet","Exposure-bracketed codebook improves multi-exposure HDR fusion","OLC scheme helps VQGAN learn true HDR representations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000223,"raw_usage":{"total_tokens":1461,"prompt_tokens":956,"completion_tokens":505,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":572,"completion_tokens_details":{"reasoning_tokens":415}},"tokens_in":572,"tokens_out":505,"duration_ms":6198,"temperature":1.0,"reasoning_tokens":415,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:48:04.982621+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the indexing in the released training or inference code (or, if no code is released, directly compute the intersection of the code vector sets used for the short and mid exposure frames in a trained OLC). If the intersection is empty for all codebook updates, the OLC's shared-representation mechanism is absent; the method would then reduce to a per-exposure codebook conditioning scheme, and the ablation against the vanilla codebook would need re-interpretation.","supporting_citations":[{"cited_title":"& Ommer, B","cited_arxiv_id":null,"evidence_quote":"This paper introduces VQGAN, the framework that the OLC extends by adding the overlapped codebook structure."},{"cited_title":"& Others Neural discrete representation learning","cited_arxiv_id":null,"evidence_quote":"This paper introduces the vector quantization mechanism and the straight-through gradient estimator that the OLC training relies on."},{"cited_title":"& Others Deep high dynamic range imaging of dynamic scenes..ACM Trans","cited_arxiv_id":null,"evidence_quote":"This paper supplies the Kalantari dataset used for training and evaluation, and the triangular exposure-fusion formulation (Eq. 1-2) that the OLC mirrors."},{"cited_title":"& Polley, M","cited_arxiv_id":null,"evidence_quote":"This paper supplies the Hu synthetic dataset used as a second benchmark for quantitative comparison."},{"cited_title":"& Zhang, Y","cited_arxiv_id":null,"evidence_quote":"This paper presents AHDRNet, an attention-based baseline that motivates implicit feature-level alignment and is included in the comparisons."},{"cited_title":"& Liu, S","cited_arxiv_id":null,"evidence_quote":"This paper presents CA-ViT, a transformer-based baseline that the proposed method compares against and outperforms."},{"cited_title":"& Gin- hac, D","cited_arxiv_id":null,"evidence_quote":"This paper presents SCTNet, a semantics-consistent transformer baseline used for comparison in the experiments."}],"review_version":1}