{"id":"853627c1-534d-4b46-b9c6-30b62984ba59","arxiv_id":"2508.10075","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A LLaMA-3.1-8B model fine-tuned with LoRA on digit-serialized SDSS spectra predicts redshifts with MAE 0.043 and retains 85% of its astronomy QA performance.","lead":"Fine-tuning a standard large language model on digit-encoded galaxy spectra lets it predict redshifts with a mean absolute error of 0.04 while keeping most of its language ability. The work is a low-cost proof that generic AI models can be adapted to scientific data using simple APIs and little compute.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline MAE rests on validation-set selection and under-specified preprocessing; no held-out test set supports the 0.04 redshift accuracy.","rationale":"The paper's core architectural claim is plausible: a frozen LLaMA-3.1-8B with LoRA adapters can learn to map serialized SDSS spectra to redshifts while retaining most language-benchmark performance. The compute cost, 16 A100 GPU-hours for 3,000 galaxies and rank-8 LoRA, is credible and not internally contradicted. The reader's weakest_assumption correctly identifies the validation-set reuse and under-specified preprocessing as the main threats to the headline accuracy. I partially agree: I also see the validation-set-size inconsistency (§2 vs §4 vs Figure 1) as a concrete sign that the evaluation protocol is not carefully pinned down, and I would add that the absence of any non-LLM baseline on the same data makes the word 'competitive' impossible to assess. However, the paper itself concedes that specialized methods achieve sub-0.01 MAE, so the contribution is explicitly not about surpassing dedicated redshift estimators; it is about cheaply augmenting an LLM. That framing lowers the stakes of the missing baseline but does not remove the need for a held-out test set. The conditional verdict is appropriate: the method may work, but the quantitative headline should not be taken at face value until an independent evaluation and a reproducible preprocessing description are provided. No change to the reader's verdict is needed.","tokens_in":7328,"tokens_out":7076,"duration_ms":73668,"concrete_test":"Hold out 1,000–2,000 SDSS spectra that are never used for model selection. After fixing the fiducial hyperparameters (lr=1e-4, rank=8, epochs=2), retrain on the same 3,000 training galaxies and evaluate MAE on this untouched test set. Also run a standard spectral redshift estimator (e.g., SDSS pipeline or template cross-correlation) on the identical test set. If the held-out LLM MAE is substantially above 0.043, or if the LLM underperforms the simple baseline by a large margin, the central quantitative claim should be weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's quantitative claim—MAE=0.04 with 16 GPU-hours and 0.04% parameters—depends on a number that is not an unbiased estimate of generalization. Section 4 states that learning rate, LoRA rank, and epochs are analyzed and the fiducial configuration (lr=1e-4, rank=8, epochs=2) is chosen from those results, and Table 1 reports validation MAE for all configurations. The same validation set is therefore used for both model selection and final evaluation, so 0.043 is the best of the explored grid, not an independent accuracy estimate. The validation set size is also internally inconsistent: §2 reserves 1,000 galaxies for validation, §4 says the validation set is 20% of galaxy spectra, and Figure 1's caption says it contains 2,000 spectra. Preprocessing is under-specified: wavelength-grid resampling, normalization, binning, and the tokenization parameters (base, precision) that determine the ~500-token-per-spectrum input are not described, making it impossible to separate the reported accuracy from preprocessing artifacts or to reproduce the result. Finally, no non-LLM baseline is run on the same data; the paper concedes specialized estimators reach sub-0.01 MAE, so 'competitive' is unquantified. These issues do not invalidate the qualitative demonstration that a LoRA-adapted LLM can process serialized spectra, but they do undermine the precise headline number.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper demonstrates that LLaMA-3.1-8B-Instruct can be adapted with LoRA to predict galaxy redshifts from SDSS DR16 spectra, after serializing normalized flux values into digit tokens. The authors report a validation-set MAE of 0.043 for their fiducial configuration (learning rate 1e-4, rank 8, 2 epochs), using 16 A100 GPU-hours and adapting 0.04% of model parameters, while retaining 85.2% of AstroBench and 89.4% of general QA benchmark performance. They argue this parameter-efficient route lowers barriers for domain scientists and enables integrated agentic workflows where one model handles both spectral analysis and language reasoning. The central claim is that a generic pre-trained LLM can process a non-textual scientific modality with minimal adaptation while preserving language capabilities.","tokens_in":7724,"tokens_out":3338,"duration_ms":40423,"significance":"If the quantitative claims held, the paper would be a useful demonstration that a frozen-base LLM with LoRA can serve as a general scientific modality processor, without custom tokenizers or architectures. The qualitative result — that digit-serialized spectra can be fed to an instruction-tuned LLM and produce physically plausible redshift predictions — is interesting and plausible, and the paper's low-compute, open-toolchain approach is a strength. The paper also benefits from evaluating language retention with established benchmarks (AstroBench, eval-harness), a clear statement of the parameter count, and a public-data setting. However, the headline MAE is not an unbiased generalization estimate: hyperparameters are selected on the same validation set used to report the number, no held-out test set is described, the validation set size is internally inconsistent, and the spectral preprocessing is under-specified. These issues do not invalidate the proof-of-concept, but they prevent the reader from trusting the precise quantitative claims or reproducing the experiment.","major_comments":[{"comment":"The reported MAE=0.043 is the minimum over a hyperparameter grid evaluated on the validation set. Section 4 and Table 1 use validation MAE to select the fiducial learning rate, LoRA rank, and number of epochs; no separate test set is described. The abstract and §4 present 0.043 as an accuracy statement, but it is a selected best-of-grid number, not an unbiased estimate of generalization. In addition, the validation set size is inconsistent: §2 says 1,000 galaxies, §4 says 20% of galaxy spectra (which would be 2,000), and the Fig. 1 caption says 2,000 spectra. The authors should fix the size inconsistency, hold out a truly independent test set, and report the MAE on that test set for the final configuration (and ideally for all configurations in Table 1).","section":"§4, Table 1; §2; Fig. 1"},{"comment":"The spectral preprocessing is specified too loosely to support the reported accuracy. The text says only that after 'converting from logarithmic to linear wavelength scales and normalizing fluxes' the authors obtain 10,000 samples, and that 'equal-frequency binning' is used to sample 3,000 training galaxies. The wavelength range, the number of wavelength bins, the resampling method, the normalization scheme, the definition of equal-frequency binning, and the final tokenization base/precision are not given. The example in §3.1 uses base=10 and prec=2, but the actual configuration is never stated, so the statement that each spectrum occupies less than 7% of the 8K context cannot be checked. Because the serialization itself is the feature representation, the reported MAE may depend strongly on these choices; the authors should specify the complete preprocessing pipeline and tokenization par","section":"§2, §3.1"},{"comment":"No non-LLM baseline is run on the same validation data. The paper concedes that specialized spectroscopic redshift estimators reach sub-0.01 MAE, but then claims the approach is 'competitive' without quantifying the comparison on an identical sample. A simple baseline (e.g., linear regression or a random forest on binned, normalized fluxes) would establish whether the LLM adaptation is competitive with standard machine learning or merely functional. Without such a baseline, the central scientific claim—that an LLM can be efficiently repurposed for spectroscopy—is demonstrated only in absolute terms, not relative to existing tools.","section":"§4, Conclusion"}],"minor_comments":[{"comment":"Typo: '0 < z <0.50 < z <0.5' appears to be a duplicated interval; it should be '0 < z < 0.5'.","section":"§2"},{"comment":"The example's digit serialization is ambiguous: the value 4.56 becomes '4|5|6', which omits the decimal point and could be parsed as 456 or 4.56 depending on the implied precision. The authors should explicitly state how the decimal location is recovered at prediction time.","section":"§3.1"},{"comment":"The model's response in the sample dialogue says 'Photometric redshift: 0.315', but the task is redshift prediction from a spectrum, not photometric redshift estimation. This should say 'Redshift' or 'Spectroscopic redshift' to avoid conflating the two.","section":"§4, example conversation"},{"comment":"The validation-set inconsistency between 1,000 and 2,000 spectra also affects the stated '20%' figure. Please harmonize these numbers and state the exact number of validation galaxies used for each row of Table 1.","section":"Table 1, Fig. 1"},{"comment":"The abstract reports 'MAE = 0.04' but the results report 0.043. Rounding is fine, but the paper should be consistent about which number is being cited.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is a workshop-style proof of concept with a promising qualitative demonstration, but the evaluation protocol is not yet suitable for a journal. The validation-set circularity and inconsistent validation size are fixable, and the preprocessing details are reportable; no baseline is a larger gap. I would support resubmission after a careful revision that separates model selection from final evaluation and adds a standard baseline."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful proof of concept. The authors LoRA-fine-tune LLaMA-3.1-8B on digit-serialized SDSS spectra and get redshift MAE 0.043 on the validation set while retaining ~85-89% of benchmark QA performance. That specific combination is new in astronomy, and the compute cost (16 GPU-hours, 0.04% of parameters) is genuinely low. The paper is honest that specialized estimators reach sub-0.01 MAE and frames itself as feasibility demo, not SOTA.\n\nWhat it does well: the ablations over learning rate, LoRA rank, and epochs give a concrete picture of the trade-off between spectroscopic accuracy and language retention. The retained-capability numbers are concrete and plausible. The serialization scheme is simple and, in principle, reproducible. The paper also acknowledges the tokenization inefficiency and the existence of better specialized methods.\n\nThe soft spots are mostly about the evaluation protocol. The headline MAE is not an unbiased generalization estimate: hyperparameters are tuned on the same validation set used to report the number, so 0.043 is the best of the explored grid. There is no separate test set. Validation set size is internally inconsistent: Section 2 says 1,000 galaxies, Section 4 says 20% of spectra, and the Figure 1 caption says 2,000. Preprocessing is under-specified: wavelength-grid resampling, normalization, and the tokenization base/precision are not described, which makes reproduction harder than it should be. There is also no baseline method run on the same data, so 'competitive' is unquantified—though the authors do concede the gap to specialized estimators.\n\nNone of this kills the qualitative claim. A general LLM can be adapted to a non-textual scientific modality with modest compute while retaining language ability, and that is the paper's actual contribution. The flaws are fixable and the tone is appropriately limited.\n\nVerdict: worth refereeing, but it needs a proper train/validation/test split, consistent numbers, and at least one non-LLM baseline on the same data. The 0.04 number should be presented as validation-selected, or replaced with a true test-set value. I'd bring it to a reading group as an example of cheap adaptation, and I'd cite it as a proof of concept if I were working on LLMs for astronomical data.","headline":"LoRA-adapted LLaMA can handle serialized galaxy spectra at trivial cost; the 0.04 MAE needs a proper held-out test before it is cited.","tokens_in":8127,"tokens_out":2736,"would_cite":true,"duration_ms":28074,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A pre-trained LLM, adapted with only 0.04% of its parameters, can predict galaxy redshifts from raw spectra while keeping most of its language skills.","keywords":["galaxy redshifts","large language models","Low-Rank Adaptation","spectral tokenization","parameter-efficient fine-tuning","astronomical spectra","foundation models","LLM fine-tuning"],"falsifier":"Take the fine-tuned model and run it on spectra from a different survey or from galaxies at higher redshift than the 0-0.5 training range; if MAE stays near 0.043 that supports generalization, while a sharp increase indicates the model memorized the training distribution. Alternatively, shuffle the wavelength order of each spectrum while keeping the same flux-value string and check whether predictions change; if they barely change, the model is learning from digit statistics, not spectral features.","tokens_in":7278,"feed_emoji":"🔭","tokens_out":6608,"duration_ms":65097,"temperature":0.7,"pith_summary":"This paper claims that a general-purpose large language model, LLaMA-3.1-8B, can be turned into a galaxy-redshift estimator purely by serializing spectra as digit strings and applying Low-Rank Adaptation (LoRA) to a tiny fraction of its weights. The authors report a mean absolute error of 0.043 on SDSS galaxy redshifts after only 16 GPU-hours of training, with less than 15% degradation on astronomy science-QA and about 10% on general QA. The point is not that this beats specialized redshift codes, which still do better; it is that a single model can both read raw spectra and reason about their physical meaning without custom architectures or expensive from-scratch training. If this holds, it opens a cheap route for astronomers to repurpose any new LLM for their own data modalities.","feed_headline":"LLM learns galaxy redshifts in 16 GPU-hours","feed_subtitle":"Retuning 0.04% of an LLM gives redshift error 0.043 and keeps most QA skills.","key_machinery":"The method's two load-bearing pieces are digit-serialization tokenization, where each normalized flux value is written out as a string of sign, digits, and separators (e.g., 4.56 becomes \"4|5|6\") with delimiters between values so the existing BPE tokenizer can ingest spectra without architectural changes, and Low-Rank Adaptation, where weight updates are restricted to low-rank matrices B and A (rank 8, about 3.4M parameters) while the base model stays frozen. Together they make multimodal adaptation as cheap as a standard fine-tuning API call.","core_discovery":"The central claim is that a frozen pre-trained LLM, augmented by low-rank adapters, can acquire a non-textual scientific skill—estimating galaxy redshifts from optical spectra—without losing its language abilities. The authors achieve this by encoding each flux value as a sign, digits, and a separator into plain text, feeding roughly 3,000 spectra as token strings, and fine-tuning rank-8 LoRA adapters for two epochs. In the fiducial configuration this yields a validation MAE of 0.043, retaining 85.2% of AstroBench scientific QA and 89.4% of general QA performance. The authors frame the result as evidence that text-pretrained transformers harbour transferable sequential-processing strategies,","pith_inferences":["A natural next test is whether the learned adapter actually attends to physical spectral features (e.g., the 4000 Å break or emission lines) or merely to coarse flux statistics; running the same serialized inputs with shuffled wavelength order would separate those cases.","The same recipe likely extends beyond redshifts to other spectral regression tasks—chemical abundances, stellar parameters, or even classification—since the tokenization is task-agnostic.","One could try more compact digit encodings (e.g., base-64 or floating-point compression) to shorten token sequences; the current ASCII-digit serialization uses roughly 1.6M tokens for 3,000 spectra and may be a bottleneck for larger datasets.","If the QA degradation is mostly from a few benchmarks rather than uniform, a hybrid adapter or layer-freezing strategy may push retention above 95% without sacrificing redshift accuracy."],"forward_implications":["If the claim holds, researchers can adapt new LLMs to arbitrary spectral or sequential scientific data with zero architectural changes and a few GPU-hours.","A single model can do end-to-end analysis: ingest raw spectra, output physical quantities, and then discuss the result in natural language, removing custom interfaces in agentic scientific workflows.","The 0.043 MAE is competitive in a practical sense while retaining 85-89% of language QA, meaning instrument teams could deploy one model for both data reduction and interpretation.","Because only 0.04% of parameters are trained, the approach directly transfers to proprietary fine-tuning APIs, giving astronomers access to LLM services for niche scientific tasks.","If the result generalizes across surveys and resolutions, it turns pretrained LLMs into quickly re-tunable science adapters for other domains such as stellar spectra or light curves."],"supporting_citations":[{"why":"Supplies the LoRA low-rank adaptation method that the fine-tuning is built on.","marker":"Hu et al., 2021"},{"why":"Source of the SDSS DR16 galaxies, spectra, and reference redshifts used in training and validation.","marker":"Ahumada et al., 2020"},{"why":"Provides the eval-harness benchmark used to measure general QA retention after fine-tuning.","marker":"Gao et al., 2023"},{"why":"Defines the AstroBench astronomy QA benchmark used to measure scientific reasoning retention.","marker":"Ting et al., 2025"},{"why":"The specialized SDSS/BOSS redshift pipeline whose sub-0.01 MAE serves as the state-of-the-art comparison.","marker":"Bolton et al., 2012"},{"why":"A custom astronomy transformer trained from scratch, used as the contrast case for computational cost and specialization.","marker":"Różański et al., 2025"}],"fun_headline_variants":["LLM masters redshift prediction with minimal tuning","Spectroscopy meets LLM: 16-hour training yields high accuracy","LLM learns galaxy distances, keeps language skills","0.04% of LLM parameters unlock redshift prediction","LLM yields redshift error 0.04 after 16 GPU-hours"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The reported MAE of 0.043 is computed on a validation set that was also used to choose the learning rate, rank, and epochs, and the paper does not specify the spectral wavelength grid or normalization; the number may not reflect performance on truly unseen spectra.","fun_headline_variants_meta":{"raw":{"variants":["LLM masters redshift prediction with minimal tuning","Spectroscopy meets LLM: 16-hour training yields high accuracy","LLM learns galaxy distances, keeps language skills","0.04% of LLM parameters unlock redshift prediction","LLM yields redshift error 0.04 after 16 GPU-hours"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000197,"raw_usage":{"total_tokens":1171,"prompt_tokens":683,"completion_tokens":488,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":427,"completion_tokens_details":{"reasoning_tokens":407}},"tokens_in":427,"tokens_out":488,"duration_ms":5459,"temperature":1.0,"reasoning_tokens":407,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T20:42:17.830382+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the fine-tuned model and run it on spectra from a different survey or from galaxies at higher redshift than the 0-0.5 training range; if MAE stays near 0.043 that supports generalization, while a sharp increase indicates the model memorized the training distribution. Alternatively, shuffle the wavelength order of each spectrum while keeping the same flux-value string and check whether predictions change; if they barely change, the model is learning from digit statistics, not spectral features.","supporting_citations":[],"review_version":1}