{"id":"62197b0a-6b9d-4c43-9733-b289bae3d8f0","arxiv_id":"2502.00782","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A benchmark comparing full fine-tuning, lightweight fine-tuning, and LoRA for transferring PINNs across boundary conditions, materials, and geometries, finding full fine-tuning and LoRA help while last-layer fine-tuning hurts.","lead":"The paper tests three ways to reuse a trained physics-informed neural network (full fine-tuning, last-layer fine-tuning, and low-rank adaptation) when the boundary condition, material, or geometry changes. The main finding is that reusing a related solution's weights speeds up training, but the paper does not count the cost of first training that related solution.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Speed claim counts only fine-tuning iterations; including source pretraining cost, the reported convergence advantage may disappear.","rationale":"The reader's weakest_assumption identifies the same concern I consider most load-bearing: the speed comparison ignores pretraining compute. I found no more severe internal flaw. The accuracy inconsistencies in Table 3 (e.g., Circle-to-Elipse displacement: no-transfer 0.0021302 vs full fine-tuning 0.0023966 and LoRA r=4 0.0026557) and the LoRA trainable-parameter counts (Table 1: 900/2700/60300 are not consistent with any simple r(d+m) sum over the stated architecture) are real but secondary; they affect the 'slight enhancement' qualifier and reproducibility, not the core mechanism. The source-pretraining issue is the one that, if it lands, invalidates the headline speed claim. The proposed test requires no new experiments, only re-plotting existing data with total compute and reporting time-to-threshold. Since the reader already conditioned acceptance on this issue, I keep the verdict unchanged.","tokens_in":21964,"tokens_out":12320,"duration_ms":113154,"concrete_test":"Recompute the speed comparison as end-to-end time-to-threshold. For each target, choose the no-transfer final L2 error at the reported evaluation epoch (Tables 1–3) as the threshold, and record the wall-clock time at which transfer (source training plus fine-tuning) and no-transfer first reach that threshold. Use the source training epochs actually reported (e.g., 100,000 in Section 4.2; 60,000 in Section 4.3) and report both single-target and amortized-over-all-targets accounting. If transfer's total time is not lower in a majority of cases, the convergence-speed claim in the abstract is unsupported and should be weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—full fine-tuning and LoRA 'significantly improve convergence speed'—is supported only by target-domain epoch counts. Tables 1–3 and Figures 9–20 compare transfer and no-transfer at identical target iteration counts, but every transfer run consumes additional source training: Section 4.2 states 100,000 source epochs, Section 4.3 states 60,000, and Section 4.1 uses at least 80,000 epochs in the displayed curves. Per-epoch wall-clock time is nearly identical across methods (Table 1: 14.6–14.9 s per 1000 epochs), so adding source time to the transfer budget means transfer spends roughly 2x the no-transfer compute (or about 1.5x if the source is amortized over the two targets per source in Section 4.1). The paper never reports time-to-threshold including total compute. Section 5.2 shows the authors can do total-iteration accounting for accuracy (100k source + 100k target vs 200k no-transfer), but this is not applied to the speed claim. If the source is not assumed free, the headline speed advantage may vanish.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript reports a systematic comparison of transfer learning strategies—full fine-tuning, lightweight fine-tuning, and Low-Rank Adaptation (LoRA)—applied to two flavors of physics-informed neural networks: the strong-form PINN for the Taylor-Green vortex with varying frequency parameter (boundary/initial conditions) and the energy-form Deep Energy Method for functionally graded beams (material property variation) and square plates with holes (geometry variation). The main reported findings are that full fine-tuning and LoRA substantially reduce the number of target-domain training epochs needed to reach a given accuracy and often yield slightly lower relative L2 and H1 errors, whereas lightweight fine-tuning performs poorly. The paper also discusses LoRA rank selection based on source-target similarity and proposes a scenario-fusion idea for improving PINN accuracy under a fixed total iteration budget.","tokens_in":22135,"tokens_out":5674,"duration_ms":52061,"significance":"The paper addresses a practically relevant question: whether pretraining on a related source PDE and adapting via full fine-tuning or LoRA can replace from-scratch training when boundary conditions, material fields, or geometries change. Its scope is broader than most prior studies because it covers both strong-form and energy-form formulations and three adaptation strategies. The positive features include a transparent experimental setup (architectures, optimizers, loss weights, and integration rules), per-epoch timing analysis in Tables 1–3, and an explicit total-iteration comparison in Section 5.2 for accuracy. If the speed claims survive a total-compute analysis, the paper would be a useful reference for practitioners considering transfer learning in PINNs. However, as written, the central speed claim is not yet supported by the reported data because source pretraining cost is excluded, and the absence of repeated-run statistics leaves the conclusions vulnerable to stochastic variation.","major_comments":[{"comment":"The headline claim that full fine-tuning and LoRA can significantly improve convergence speed is measured only by target-domain epochs or iterations. Transfer runs consume extra source pretraining: Section 4.2 states 100,000 source epochs, Section 4.3 states 60,000 source epochs, and Section 4.1 shows source training to 80,000 epochs, while Table 1 reports nearly identical per-epoch wall-clock times across methods (14.6–14.9 s per 1,000 epochs for the Navier-Stokes case). Under total-compute accounting (source pretraining plus target fine-tuning versus target-only training), the reported speed advantage can shrink or disappear. The paper should report time-to-threshold including source training, or an explicit amortization argument when one source is reused for multiple targets, before claiming 'significantly improve convergence speed'.","section":"§4.1–4.3, Tables 1–3"},{"comment":"All accuracy and convergence comparisons are single-run values with no error bars, seed counts, or significance tests. Statements such as 'significantly improve convergence speed' and the rank comparison in Figure 21 need repeated-run statistics (at least 3–5 independent runs) to separate systematic effects from initialization and sampling noise.","section":"§4, Tables 1–3, Figures 9–20"},{"comment":"The LoRA configuration is internally inconsistent. The text says LoRA 'only trains the layers [100, 100, 100, 100]' (four 100-dimensional layers), but the reported trainable-parameter counts (900, 2,700, and 60,300 for r=1, 4, and 100) correspond to three square layers including biases (3 × [r(100+100)+100]). This inconsistency affects the efficiency comparison and the claim that LoRA's parameter count is much smaller than full fine-tuning; please state exactly which weight matrices and biases LoRA modifies and recompute the parameter counts.","section":"§4.1 and Table 1"}],"minor_comments":[{"comment":"The sentence 'Full finetuning refers to fully fine-tuning the pre-trained parameters' is duplicated.","section":"§4.1"},{"comment":"The header 'Epoches' should be 'Epochs'.","section":"Tables 2 and 3"},{"comment":"The claim that the optimal LoRA rank correlates with source-target similarity is presented without a quantitative similarity measure; consider marking this as an observation rather than a concluded design rule.","section":"§5.1 and Figure 21"},{"comment":"The text around Eq. (25) refers to 'solving Eq. (8)' when the pressure Poisson equation is meant, and the pressure variable is not included in the loss function Eq. (27); please clarify which equations are actually being solved.","section":"§2.2 and §4.1"},{"comment":"The code is promised only 'after accepted'; for a reproducibility-focused empirical study, make the code available with the submission or provide a permanent repository link.","section":"Appendix D"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the journal's scope and the experiments are competently designed, but the central speed claim needs total-compute accounting and repeated-run statistics before publication. The paper also leans heavily on the authors' own prior Deep Energy Method and related works; this is common in a niche area, but the reference list should not crowd out independent baselines. I would support publication after the major revisions above."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuinely useful benchmark, but the headline speed claim ignores the cost of pretraining the source model, and the LoRA parameter counts in the main tables don't match the stated implementation. Both are fixable, but they need fixing before the numbers can be trusted.\n\nThe novel contribution is the systematic side-by-side comparison of full fine-tuning, lightweight fine-tuning, and LoRA in both strong-form PINNs and energy-form DEM, across boundary conditions, materials, and geometries. The negative result on lightweight fine-tuning is worth having: freezing early layers and training only the last layer works poorly for PINNs, and the authors explain why via the AD computation graph. The appendices on why lightweight FT doesn't save wall-clock time and why LoRA can be slower than full FT are useful details most papers would skip. The observation that the optimal LoRA rank increases with source-target discrepancy is a reasonable starting point for a practical selection heuristic. The citation pattern leans on the authors' own prior work, but the comparative measurements stand independently.\n\nThe soft spots are real. The speed comparison is target-iteration-only. Source pretraining runs 60,000–100,000 epochs in Sections 4.2 and 4.3, and at least 80,000 in the displayed curves of 4.1. Since per-epoch wall-clock time is nearly identical across methods, including source cost roughly doubles the transfer compute budget. The paper never reports time-to-threshold end-to-end, so the claim that full FT and LoRA 'significantly improve convergence speed' is not supported as stated. Section 5.2 shows accuracy at equal total iterations, but that accounting is not applied to the speed claim. There are no repeated runs or error bars. The LoRA trainable-parameter counts are off: for four 100×100 weight matrices with rank r, the count should be 800r; Table 1 shows 900, 2700, and 60300 for r=1, 4, 100, suggesting a different implementation than described. Table 3 shows no-transfer beating transfer in several cells, so the accuracy gain is not uniform. And the boundary-condition construction for the plate-with-hole problem (Eq. 36) is under-specified; the essential BCs should be stated explicitly.\n\nWho gets value: practitioners deciding whether to warm-start a PINN or use LoRA for parametric studies. Not a breakthrough, but a useful empirical map. It deserves a serious referee; the issues are addressable. I'd ask for a major revision: end-to-end compute accounting, multiple seeds, corrected parameter counts, and clarified BCs.","headline":"Useful systematic benchmark of transfer learning in PINNs, but the speed claim ignores source pretraining cost and the LoRA parameter counts don't add up; deserves a serious referee after major revision.","tokens_in":22740,"tokens_out":5543,"would_cite":false,"duration_ms":44496,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a PINN trained on one problem can be adapted to a related problem—different boundary conditions, materials, or geometries—through full fine-tuning or Low-Rank Adaptation, cutting convergence time and slightly…","keywords":["physics-informed neural networks","transfer learning","low-rank adaptation","deep energy method","AI for PDEs","computational mechanics","fine-tuning"],"falsifier":"Run both pipelines end-to-end on identical hardware for one transfer pair (say Taylor-Green $\\pi \\to 2\\pi$): train the source model to its reported stopping point, fine-tune to a fixed target error, and compare total wall time against training the target from scratch to the same error. If the transferred run is not faster, the paper's central convergence-speed claim is falsified.","tokens_in":21728,"feed_emoji":"🧠","tokens_out":7352,"duration_ms":69290,"temperature":0.7,"pith_summary":"This paper asks whether a PINN trained on one PDE instance can be reused for a nearby instance, so that changes in boundary conditions, material laws, or geometry do not force training from scratch. It compares three parameter-transfer schemes—full fine-tuning, lightweight fine-tuning that freezes all but the last layer, and Low-Rank Adaptation (LoRA)—on strong-form PINNs for the Navier-Stokes Taylor-Green vortex and on the energy-form Deep Energy Method for a graded beam and a holed plate. The claim is that full fine-tuning and LoRA cut target convergence iterations substantially and slightly improve accuracy in most tested cases, while lightweight fine-tuning helps little because PINN networks lack the hierarchical features that make layer freezing effective. A secondary claim is that the optimal LoRA rank grows with the dissimilarity of source and target problems, and that training through an intermediate source problem can beat direct training at equal total iterations.","feed_headline":"Full fine-tuning and LoRA accelerate PINN transfer","feed_subtitle":"Reusing a solved problem's network weights cuts target-domain iterations across boundary conditions, materials, and geometries.","key_machinery":"The central machinery is parameter-based transfer learning: a network pretrained on a source problem provides the initial weights for a target problem, and adaptation follows one of three schemes. The named identity is LoRA, the low-rank update $W^* = W + \\alpha AB$, which keeps pretrained weights fixed and trains a product of two low-rank matrices $A \\in \\mathbb{R}^{d \\times r}$ and $B \\in \\mathbb{R}^{r \\times m}$; rank $r$ acts as a dial interpolating between lightweight and full fine-tuning. The paper's efficiency analysis then shows why the expected savings are muted: since PINN losses require derivatives of the network output with respect to inputs, automatic differentiation rebuilds a computation graph of size growing with the PDE's derivative order, so freezing parameters or decomposing weight updates adds little wall-time benefit per epoch.","core_discovery":"In the paper's own terms, the discovery is that parameter-based transfer learning generalizes across problem instances in both formulations of PINNs: initializing the target network with the source-domain weights and then applying either full fine-tuning or a LoRA low-rank update \"significantly improve convergence speed while providing a slight enhancement in accuracy\" across most scenarios. Lightweight fine-tuning—freezing the early layers and training only the last layer—performs worst in all three test groups, which the authors attribute to fully connected PINN networks not possessing the hierarchical feature extraction of CNNs. They also report that transferring from a high-frequency to a low-frequency Taylor-Green vortex works better than the reverse, and that LoRA's optimal rank increases as the source-target similarity decreases; with rank equal to the full matrix dimension, LoRA reduces to full fine-tuning.","pith_inferences":[],"forward_implications":["Full fine-tuning should be the default transfer baseline for PINNs; in the tested cases it consistently reached a given target error in fewer iterations than training from scratch, and usually matched or beat the no-transfer error.","LoRA with a moderate rank (around 4) matches or exceeds full fine-tuning on the energy-form problems while training far fewer parameters, making it attractive when per-task storage or parameter efficiency matters.","Lightweight fine-tuning that freezes early layers is not a viable transfer strategy for PINNs; its measured error is often worse than no transfer at all.","The LoRA rank should be set according to source-target similarity: the paper's rank sweep suggests larger gaps call for larger ranks (up to around 56 when transferring from $\\pi$ to $3\\pi$), which a proposed cosine-similarity rule could eventually automate.","Training sequentially through source problems before the target—termed \"scenario fusion\"—can beat direct training at equal total iteration counts, implying that intermediate tasks act as useful warm starts.","The high-to-low frequency transfer asymmetry hints at a practical scheduling rule the authors leave implicit: when facing a family of problems ordered by difficulty, training on the hardest instance first and transferring down may compress total effort; a direct test would compare curricula ordered by increasing versus decreasing frequency.","The scenario-fusion result effectively describes a curriculum over tasks, and the paper's proposed cosine-similarity rank rule could be tested immediately by encoding a few hand-chosen problem features and comparing the predicted rank to the empirically optimal rank found in the paper's rank sweep.","The claim that transfer learning 'significantly improve convergence speed' has not been established on a total-compute basis; a fair end-to-end comparison would need to include the source pretraining epochs."],"supporting_citations":[{"why":"Supplies the Deep Energy Method (energy-form PINN) used for the material and geometry transfer experiments.","marker":"[1]"},{"why":"Supplies the strong-form PINN formulation used for the Navier-Stokes boundary-condition transfer experiments.","marker":"[6]"},{"why":"Supplies the taxonomy of transfer learning (parameter-based versus other categories) that structures the study.","marker":"[14]"},{"why":"Supplies the lightweight fine-tuning approach (freeze early layers, train the last layer) applied in the comparisons.","marker":"[20]"},{"why":"Supplies a second precedent for lightweight fine-tuning in multi-fidelity PINN training.","marker":"[21]"},{"why":"Introduces LoRA, the low-rank adaptation method that the paper evaluates and compares against full fine-tuning.","marker":"[22]"},{"why":"Provides prior application of LoRA to PDE-based networks, giving the paper a direct baseline for LoRA in PINNs.","marker":"[23]"},{"why":"Supplies the isogeometric-analysis reference solution for the functionally graded beam benchmark used to measure transfer accuracy.","marker":"[36]"}],"fun_headline_variants":["LoRA and full fine-tuning accelerate PINN transfer","Lightweight fine-tuning fails for PINN transfer","Transfer learning speeds PINN retraining across conditions","Skip lightweight fine-tuning in PINNs, use LoRA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the comparison should count only target-domain fine-tuning iterations and treat the source pretraining as free; if pretraining epochs are included in the total cost, the claimed speed advantage may narrow or disappear.","fun_headline_variants_meta":{"raw":{"variants":["LoRA and full fine-tuning accelerate PINN transfer","Lightweight fine-tuning fails for PINN transfer","Transfer learning speeds PINN retraining across conditions","Skip lightweight fine-tuning in PINNs, use LoRA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000958,"raw_usage":{"total_tokens":4016,"prompt_tokens":810,"completion_tokens":3206,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":426,"completion_tokens_details":{"reasoning_tokens":3156}},"tokens_in":426,"tokens_out":3206,"duration_ms":23623,"temperature":1.0,"reasoning_tokens":3156,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T17:45:35.908582+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run both pipelines end-to-end on identical hardware for one transfer pair (say Taylor-Green $\\pi \\to 2\\pi$): train the source model to its reported stopping point, fine-tune to a fixed target error, and compare total wall time against training the target from scratch to the same error. If the transferred run is not faster, the paper's central convergence-speed claim is falsified.","supporting_citations":[{"cited_title":"Samaniego, C","cited_arxiv_id":null,"evidence_quote":"Supplies the Deep Energy Method (energy-form PINN) used for the material and geometry transfer experiments."},{"cited_title":"Zhuang, Z","cited_arxiv_id":null,"evidence_quote":"Supplies the taxonomy of transfer learning (parameter-based versus other categories) that structures the study."},{"cited_title":"Chakraborty, Transfer learning based multi-fidelity physics informed deep neural network, Journal of Computational Physics 426 (2021) 109942","cited_arxiv_id":null,"evidence_quote":"Supplies a second precedent for lightweight fine-tuning in multi-fidelity PINN training."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the isogeometric-analysis reference solution for the functionally graded beam benchmark used to measure transfer accuracy."}],"review_version":1}