{"id":"e77fc601-d03c-4b1a-b921-11c00951bbf8","arxiv_id":"2412.05452","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"PSR J1022+1001's pulse profile variability survives rigorous MEM and METM polarization calibration and is consistent across frequency, favoring an intrinsic pulsar origin.","lead":"Pulse profiles of the millisecond pulsar J1022+1001 change shape from one observation to the next in the NANOGrav 15-year data set, even after sophisticated polarization calibration. The paper argues the variability is not caused by telescope calibration or interstellar scintillation, pointing to processes inside the pulsar itself, a result that matters for how pulsar timing arrays model noise.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim that scintillation is ruled out rests on visual inspection of one example subband plot (Fig. 15); without a quantitative across-subband consistency metric, the intrinsic-pulsar conclusion is under-supported.","rationale":"The paper's core demonstration—that J1022+1001's profiles vary and that replacing IFA with MEM+METM does not stabilize them—is credible: it uses public NANOGrav data, a clean null KS comparison (p≥0.54), and consistent peak-ratio scatter across three receivers. The residual vulnerability is the attribution step. The abstract's 'cannot be explained by interstellar scintillation' is a strong modality claim, and the only evidence offered is a visual impression of one subband figure. This is not merely a presentation weakness; it is the statistical foundation for excluding the main astrophysical alternative. If the across-band consistency is not quantified, a referee cannot distinguish 'consistent across the band' from 'consistent within the noise,' and the conclusion should be conditional on that measurement. I agree with the reader's weakest-assumption identification and recommend no change to the conditional verdict.","tokens_in":19622,"tokens_out":5589,"duration_ms":61902,"concrete_test":"For all 1.4-GHz observations passing the paper's S/N cuts, compute the peak-ratio deviation (from the per-subband median) in each 25-MHz subband using the peak-optimizer of §4.2. Test whether deviations are coherent across subbands by comparing, per observation, the observed scatter of subband peak ratios to the quadrature-summed per-subband uncertainties, and by computing a Spearman correlation of the deviation vectors between all subband pairs (or a likelihood-ratio test of a common-amplitude vs independent-amplitude model). Report the fraction of observations with significant common-mode variability. If that fraction is small or the common-amplitude model is not preferred, the scintillation+profile-evolution alternative is not ruled out.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2's exclusion of interstellar scintillation is the load-bearing step for the abstract's claim that the variability 'cannot be explained by interstellar scintillation in combination with profile evolution with frequency' and hence is intrinsic. The evidence is one example: 'We found (see Figure 15 for an example) these subbanded profiles exhibited the same variability across the band within each observation.' No quantitative metric, uncertainty, or count of how many observations show this pattern is given, and the scintillation bandwidth (40–100 MHz) and timescale (~20 min) are 'rough estimates based on visual inspection of our dynamic spectra.' If the cross-subband consistency is not statistically established, a scintillation model with a frequency-dependent profile remains viable, and the central claim loses its main support. The authors themselves flag that the short-timescale variability 'has not been ruled out' as scintillation, so the paper is internally aware that the scintillation exclusion is only as strong as the subband evidence. A quantitative demonstration across the full epoch sample is required before 'cannot be explained' is warranted.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper analyzes 84 Arecibo observations of PSR J1022+1001 from the NANOGrav 15-yr data set and shows substantial epoch-to-epoch pulse profile variability at 430 MHz, 1.4 GHz, and 2 GHz. It tests whether this variability can be explained by the ideal feed assumption (IFA) used in NANOGrav's standard polarization calibration by re-calibrating the data with a combination of Measurement Equation Modeling and Measurement Equation Template Matching. The recalibrated profiles are found to be no less variable than the IFA-calibrated ones, and the METM-based timing solution is not improved. The paper then argues that the variability cannot be explained by interstellar scintillation combined with frequency-dependent profile evolution, concluding that intrinsic pulsar phenomena are the likely cause.","tokens_in":19750,"tokens_out":4497,"duration_ms":47341,"significance":"If the conclusions hold, this is a valuable result for pulsar timing array data quality, since pulse profile stability is a central assumption of standard timing methods, and it substantially extends the long-standing debate about the cause of PSR J1022+1001's profile variability. The manuscript's strengths include the use of independent template pulsars for the METM corrections, multiple variability metrics based on difference profiles and peak fitting, a Kolmogorov-Smirnov comparison between calibration schemes, and a full timing comparison between the IFA and METM data sets. The main vulnerability is the scintillation ruling-out, which currently rests on visual inspection of a single example observation and rough estimates of scintillation parameters; this needs quantitative support before the abstract's 'cannot be explained' claim is warranted.","major_comments":[{"comment":"The abstract's central claim that the subbanded profiles 'cannot be explained by interstellar scintillation in combination with profile evolution with frequency' rests on the statement in §4.2 that 'these subbanded profiles exhibited the same variability across the band within each observation,' but this is supported only by visual inspection of one example observation. Please provide a quantitative across-subband consistency metric (for example, a correlation or reduced chi-square between difference profiles in adjacent 25-MHz subbands, with uncertainties propagated from the data), report how many of the 84 observations show the same pattern, and demonstrate that the metric has power to detect the expected scintillation-induced decorrelation given the quoted 40-100 MHz scintillation bandwidths. Without this, the exclusion of scintillation is not established.","section":"§4.2, Figure 15"},{"comment":"The manuscript itself states that the scintillation bandwidths and timescales are 'rough estimates based on visual inspection' and that the short-timescale intra-observation variability 'in particular' has 'not been ruled out' as scintillation. These caveats are in direct tension with the unqualified 'cannot be explained' phrasing in the abstract and in §5. Either add a quantitative scintillation model that folds in the 25-MHz subband response, the measured scintillation timescales, and the observed frequency-dependent profile evolution, or soften the conclusion to 'unlikely' or 'not favored' so that the conclusion is consistent with the evidence presented.","section":"§4.2, final paragraph"},{"comment":"The difference-profile method is an appropriate way to visualize variability, but the binned standard deviations are presented without an estimate of the noise contribution to the difference profiles. Without propagating the off-pulse noise through the normalization and subtraction steps, the reader cannot tell which phase bins or epochs show variability in excess of measurement noise. Please add a noise estimate or significance threshold so that the variability measurement itself can be assessed quantitatively.","section":"§4.2, Figures 7-9"}],"minor_comments":[{"comment":"There are two typographical errors in this section: 'reciever' should be 'receiver' and 'concontiguous bands' should be 'contiguous bands'.","section":"§2"},{"comment":"The FD parameter inclusion criterion is described only qualitatively ('if doing so did not cause a large change ... in the DMX average'); please specify the numerical threshold used and report how many FD parameters were included in each of the IFA and METM timing solutions.","section":"§3.2"},{"comment":"Please clarify whether the parabola fitting procedure used the same fitting window in phase bins for all three receivers, and report the number of bins used, since this affects the quoted peak-ratio uncertainties.","section":"§4.2, Figure 10"},{"comment":"The proper motion in ecliptic latitude is reported as -2(1) x 10^2 mas/yr, which appears implausibly large compared with the proper motion in longitude of -15.9(1) mas/yr; please check the units or the entry for a typographical error.","section":"Table 3"},{"comment":"The error bars are described as the 1-sigma uncertainties from the least-squares parabola fit, but they do not include uncertainties from profile normalization; please state this limitation explicitly in the captions or text.","section":"Figures 11-13"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and the NANOGrav collaboration's membership does not present a novelty concern. The central issue is that the scintillation exclusion, which is load-bearing for the abstract's main claim, is supported only by visual inspection and rough estimates. This is fixable within the manuscript's scope by adding a quantitative consistency analysis, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper's central claim—that J1022+1001's pulse profile variability is real and not an artifact of polarization calibration—is well supported. I'd send it to review. But the abstract's stronger statement that scintillation is ruled out is ahead of the evidence as written.\n\nWhat's new: this is the first time MEM+METM calibration has been applied to NANOGrav data for this pulsar, and the comparison with standard IFA calibration is a clean null result (KS p-values ≥ 0.54). The variability itself is demonstrated with difference profiles, peak-ratio fits, and binned standard deviations, all consistent across 430 MHz, 1.4 GHz, and 2 GHz. That part is solid.\n\nThe soft spot is the scintillation section. The across-band consistency that rules out scintillation rests on visual inspection of a single example observation (Figure 15). No quantitative metric, no uncertainties, no count of how many epochs show the same pattern. The scintillation bandwidth and timescale are rough visual estimates. The paper even says the short-timescale variability 'has not been ruled out' as scintillation. Given that, the abstract's 'cannot be explained by interstellar scintillation' is stronger than the evidence warrants. This is fixable but it needs doing before publication.\n\nThe FD parameter inclusion criterion (avoid a large change in DMX average) is a bit ad hoc, but it's a minor point and the timing comparison doesn't hinge on it.\n\nBottom line: for pulsar timing array folks and anyone working on TOA generation, this is a useful, well-executed study. It deserves a serious referee. I'd recommend sending it out with a request for a quantitative subband consistency analysis across the full sample.","headline":"Solid observational case that J1022+1001's profile variability is real and calibration-independent, but the scintillation argument needs quantitative support before the abstract's claim holds.","tokens_in":20645,"tokens_out":2074,"would_cite":true,"duration_ms":19626,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PSR J1022+1001's pulse shape changes are real, intrinsic pulsar behavior, not calibration artifacts or interstellar effects.","keywords":["pulsar timing","pulse profile variability","millisecond pulsars","polarization calibration","interstellar scintillation","PSR J1022+1001","NANOGrav"],"falsifier":"Compute, for every 1.4-GHz epoch in the data set, the correlation of the difference profiles across 25-MHz subbands; if the shape deviations are not coherent across the full band, or the coherence pattern follows the expected scintillation structure, the paper's central claim fails.","tokens_in":19378,"feed_emoji":"📡","tokens_out":6995,"duration_ms":63532,"temperature":0.7,"pith_summary":"This paper asks why the radio pulse profile of the millisecond pulsar PSR J1022+1001 changes shape from observation to observation in the NANOGrav 15-yr data set. Because pulsar timing arrays assume profiles are stable, such variability could masquerade as timing noise or even affect gravitational-wave background searches. The authors test the leading instrumental suspect, polarization calibration error, by replacing the standard 'ideal feed' calibration with a full model of the telescope feed's polarimetric response; the profiles are no more stable. They then rule out interstellar scintillation coupled with frequency-dependent profile shape. Their conclusion is that the variability is intrinsic to the pulsar's emission, a claim that matters for how this pulsar and similar ones are timed.","feed_headline":"Pulsar J1022+1001's shape changes are intrinsic, study finds","feed_subtitle":"Better polarization calibration and scintillation both fail to explain the shifts, so the millisecond pulsar itself is to blame.","key_machinery":"The load-bearing object is the polarimetric response (PR) of the telescope feed, the Mueller-matrix transformation that mixes the Stokes parameters $I$, $Q$, $U$, and $V$ on the way from source to recorded signal. The paper compares the standard ideal feed assumption (IFA) calibration against a two-stage scheme using Measurement Equation Modeling (MEM) and Measurement Equation Template Matching (METM), and then uses 25-MHz subbanded profiles to test whether shape deviations are uniform across the band. That subband comparison is what separates a scintillation origin from an intrinsic origin.","core_discovery":"The paper asserts that in the NANOGrav 15-yr data set the integrated pulse profile of PSR J1022+1001 changes shape from observation to observation at 430 MHz, 1.4 GHz, and 2 GHz, and that this variability is not a calibration artifact. Profiles calibrated with a full model of the feed's polarimetric response are no more stable than profiles calibrated under the ideal feed assumption, and the deviations appear consistently across 25-MHz subbands, which the authors argue interstellar scintillation combined with frequency-dependent profile evolution cannot produce. The authors therefore conclude that the variability is intrinsic to the pulsar's emission.","pith_inferences":["If the intrinsic-variability claim survives a quantitative subband test, similar calibration comparisons on other highly polarized millisecond pulsars could reveal that unstable profiles are more common in pulsar timing array data than currently assumed.","A direct extension of the paper's Figure 15 argument would be to compute a cross-band correlation statistic for all epochs, turning a visual ruling-out into a quantitative one.","The few observations that show shape changes within a single observation, over roughly 10-15 minutes, suggest that whatever the mechanism is, it operates on timescales much shorter than the month-to-year variations, which would help narrow the candidates."],"forward_implications":["Pulsar timing arrays that use a fixed template to measure arrival times for PSR J1022+1001 will carry this variability as extra noise in the residuals.","Polarization calibration improvements of the kind tested here will not remove the effect, since the more complete calibration leaves the variability unchanged.","The scintillation-plus-frequency-evolution explanation for this pulsar's changing profile is disfavored, so searches for the mechanism should focus on the pulsar itself.","A TOA-generation method that lets the pulse shape vary could recover some of the timing precision lost to the variability."],"supporting_citations":[{"why":"Provides the NANOGrav 15-yr data set and the observing and timing procedures that supply all the profiles analyzed here.","marker":"Agazie et al. (2023b)"},{"why":"Defines the Measurement Equation Modeling approach used to measure the receiver's polarimetric response.","marker":"van Straten (2004)"},{"why":"Defines the Measurement Equation Template Matching technique used to derive epoch-by-epoch corrections to the receiver solution.","marker":"van Straten (2013)"},{"why":"First established the long-standing profile variability of this pulsar and proposed polarization miscalibration as a candidate cause.","marker":"Kramer et al. (1999)"},{"why":"Earlier study that reached a different conclusion about stability; its variability visualization style is adopted here.","marker":"Hotan et al. (2004)"},{"why":"Supplies the peak-optimizer method used to quantify peak heights and phase differences, and a prior finding of variability.","marker":"Padmanabh et al. (2021)"},{"why":"Proposed the scintillation plus frequency-dependent profile evolution explanation that the paper's subband analysis aims to rule out.","marker":"Shao & You (2016)"}],"fun_headline_variants":["Pulsar J1022+1001's shape-shifting isn't a telescope glitch","Intrinsic pulse variability blamed for pulsar's changing shape","Pulsar's own emission causes its profile to morph, study says","No calibration fix for J1022+1001's variable pulse shape","Pulsar variability traced to the pulsar, not the instrument"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The conclusion rests on visual inspection of a single example observation's 25-MHz subband profiles, so if that apparent across-band consistency is not real or not representative, the ruling-out of the scintillation explanation falls apart.","fun_headline_variants_meta":{"raw":{"variants":["Pulsar J1022+1001's shape-shifting isn't a telescope glitch","Intrinsic pulse variability blamed for pulsar's changing shape","Pulsar's own emission causes its profile to morph, study says","No calibration fix for J1022+1001's variable pulse shape","Pulsar variability traced to the pulsar, not the instrument"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000639,"raw_usage":{"total_tokens":2954,"prompt_tokens":968,"completion_tokens":1986,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":1889}},"tokens_in":584,"tokens_out":1986,"duration_ms":13681,"temperature":1.0,"reasoning_tokens":1889,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:43:13.195813+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute, for every 1.4-GHz epoch in the data set, the correlation of the difference profiles across 25-MHz subbands; if the shape deviations are not coherent across the full band, or the coherence pattern follows the expected scintillation structure, the paper's central claim fails.","supporting_citations":[{"cited_title":"M., Camilo, F., et al","cited_arxiv_id":null,"evidence_quote":"First established the long-standing profile variability of this pulsar and proposed polarization miscalibration as a candidate cause."},{"cited_title":"V., Barr, E","cited_arxiv_id":null,"evidence_quote":"Supplies the peak-optimizer method used to quantify peak heights and phase differences, and a prior finding of variability."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Proposed the scintillation plus frequency-dependent profile evolution explanation that the paper's subband analysis aims to rule out."}],"review_version":1}