REVIEW 3 major objections 5 minor 15 references
Guiding Large Language Models with Genetic Programming-Evolved Heuristic Knowledge for Dynamic Multi-Mode Project Scheduling
T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read GP-evolved scheduling rules can guide a zero-shot LLM to better dynamic project-scheduling decisions, but the form of that guidance determines its value.
desk verdict Useful reverse GP-to-LLM knowledge-transfer study with clean experiments; the generalization claim needs to be scaled back to the tested 30-activity networks. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is reverse knowledge transfer: GP evolution is run first, and its output is distilled into two knowledge forms. Feature-level knowledge is obtained by counting feature occurrences in a Pareto-based elite repository of generation-best rules and retaining the most frequent features up to 80% cumulative occurrence; this knowledge is injected either by restricting the candidate table (Feature Selection) or by annotating the full table (Feature Hint). Rule-level knowledge is a single compact, high-quality GP priority rule from the same repository, injected either as advisory (Rule Reference) or prescriptive (Rule Follow) guidance. The LLM then acts as an online decision make
What would settle it
Run the guided and unguided LLM variants on held-out project instances with different precedence networks (e.g., 60 activities, or order strengths outside {0.25, 0.5, 0.75}) using the same GP-derived guidance; if Feature Selection and Rule Follow no longer beat the Basic LLM in normalized makespan, the transfer claim fails. A simpler check: test whether the selected feature sets and the rule chosen on validation still rank among the best on new topologies.
Extended reading notes
Core claim
The paper's central claim is that GP-to-LLM knowledge transfer works: an LLM making zero-shot dynamic scheduling decisions can be made more effective by knowledge extracted from evolved GP priority rules. The claim is qualified: the way knowledge is represented determines the benefit. Restricting input to features that appear frequently in elite GP rules (Feature Selection) improves makespan and sharply cuts prompt tokens; supplying a compact GP rule as prescriptive guidance (Rule Follow) matches the GP rule in the most constrained scenario; supplying it as soft advice (Rule Reference) performs comparably but uses more thinking tokens; highlighting important features (Feature Hint) is the le
Load-bearing premise
The evolved rules and feature sets are selected on validation data from the same 30-activity project instances used at test time; if they are specialized to those network topologies, the guided LLM's advantage over the unguided LLM may not carry over to projects with different sizes or precedence structures.
Editorial extensions
If this is right
- GP-evolved rules are a usable knowledge source for zero-shot LLM scheduling, and the LLM does not need fine-tuning or few-shot examples.
- Feature Selection offers the best effectiveness-per-token trade-off, making it suitable for applications with tight inference budgets.
- Rule Follow can approach the GP rule's makespan, indicating explicit symbolic logic compensates for the LLM's weak numerical rule execution.
- Guidance reduces decision variability on repeated identical queries, a practical concern for using LLMs in online control.
- Guidance re-focuses the LLM's rationales on GP-rule features, potentially improving interpretability of LLM decisions.
Reading between the lines
- Because training, validation, and test sets share the same 30-activity project topologies and differ only in duration realizations, the measured gains may overestimate transfer to projects with different precedence structures or sizes; testing on unseen topologies would clarify whether the guidance generalizes.
- The failure of Feature Hint despite raising feature mentions suggests that attention-level prompting is too weak: effective LLM guidance must change the information set or the decision procedure, not just emphasize cues.
- The same reverse-transfer recipe might extend to other online combinatorial decision problems where GP hyper-heuristics already produce readable rules, such as job-shop dispatching or resource-constrained project scheduling variants.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a GP-to-LLM knowledge-transfer framework for dynamic multi-mode resource-constrained project scheduling (DMRCPSP). Genetic programming first evolves priority rules offline; the authors then extract feature-level knowledge (a selected feature set) and rule-level knowledge (a compact GP rule) and inject it into zero-shot LLM decision-making through four mechanisms: Feature Selection, Feature Hint, Rule Reference, and Rule Follow. These are compared with an unguided Basic LLM and with the GP rule itself on 30-activity instances under three order-strength scenarios, evaluating normalized makespan, token consumption, decision stability, and the feature focus of generated rationales. The central claim is that GP-derived guidance generally improves the unguided LLM, that representation matters, and that Feature Selection is the most token-efficient while Rule Follow gives the strongest performance at higher token cost.
Significance. If the results hold, the paper makes a useful contribution by reversing the usual LLM-to-GP knowledge flow and empirically demonstrating that heuristic knowledge extracted from GP-evolved rules can guide LLM scheduling decisions. The study is systematic in comparing four guidance mechanisms, uses repeated runs and pairwise Wilcoxon tests, and reports multiple metrics. The design also reduces circularity since GP evolution is completed before the LLM experiments and no LLM outputs are used to derive the guiding rules. The token-consumption and stability analyses add practical value. However, the breadth of the central claim is constrained by the experimental design, as discussed below.
major comments (3)
- [Section 4.1] The training, validation, and test sets share the same underlying 30-activity network topologies and differ only in duration realizations. Consequently, the feature sets and GP rules selected on validation may be specialized to these particular precedence structures. The observed improvements of Feature Selection and Rule Follow over Basic—and the claim that GP-derived guidance 'generally improves' the unguided LLM—are not established for different network topologies, sizes, or resource configurations. Please either add experiments on held-out network structures or explicitly restrict the conclusion to the tested instances.
- [Table 3 and Table 4] The pairwise Wilcoxon tests appear to be performed within each row over the 10 repeated runs of the same instance, i.e., the same activity durations and the same project state sequence. Such tests capture per-instance stochastic variation of the LLM rather than variation across problem instances. Aggregating per-instance significant results into counts of wins/ties/losses (Table 4) without correcting for multiple comparisons does not provide a statistically valid basis for the 'generally improves' claim. Please report a paired analysis over the 15 instances (e.g., using per-instance means) and address the multiple-testing issue, or justify why the per-instance comparison is sufficient.
- [Figure 5 and Section 4.2] The trade-off conclusion that 'Feature Selection is preferable when token efficiency is important, whereas Rule Follow is a better choice when scheduling performance is the main priority' is based on mean token counts and makespan across the three scenarios. However, the variance across instances is not shown in the trade-off plot, and the token counts are not statistically compared. Some scenarios (e.g., 0.25) show smaller or non-significant differences between Feature Selection and Rule Follow. Please clarify whether the ordering in Figure 5 is supported by statistical tests or is merely descriptive.
minor comments (5)
- [Section 3.3 / Supplementary] The full prompt templates, GP configuration, feature definitions, and the feature-selection cumulative threshold are promised in the supplementary material. If the supplementary material is not included with the arXiv submission, the reproducibility of the experiments is limited. Please ensure these are available.
- [Figure 6] The semantic matching of LLM rationales to feature definitions is not described. How are feature mentions detected? What consistency checks were performed? This matters because the feature-focus analysis is one of the four main metrics.
- [Table 3] The GP Rule column is reported without variance and without significance markers. Since the GP rule is deterministic in these experiments, it is not comparable via the same Wilcoxon tests. State explicitly that the GP rule is a deterministic reference and that no statistical tests are applied to it.
- [Section 4.2 / Stability] The stability analysis samples 25 decision situations per scenario from solved instances. Please specify how the situations were sampled, what 'sampled decision situation' means, and how the entropy-based stability score is computed (e.g., over the 10 queries per situation).
- [Abstract / Conclusions] The abstract says 'GP-derived guidance generally improves the unguided LLM,' but Feature Hint shows only 4 significant wins, 7 ties, and 4 losses against Basic (Table 4). Consider wording that more accurately reflects the instance-dependence, e.g., 'most forms of guidance' or 'guidance often improves but the effect depends on the representation.'
Circularity Check
No significant circularity; GP rules and feature sets are extracted before the LLM experiments and the test comparisons are empirical.
full rationale
The paper's derivation chain is not circular. GP rules are evolved offline on training duration realizations, then evaluated on a validation set; feature-level and rule-level knowledge is extracted from those validation results. The LLM variants are then evaluated on test duration realizations. Critically, the paper states that 'Heuristic knowledge extraction and rule selection were conducted based on the training and validation results only, without using the LLM test results' (Section 4.1). This severs any feedback loop from the LLM outcomes back into the guidance construction. The guided LLM variants do not define or fit the GP rules; they receive externally evolved feature sets or priority expressions. Rule Follow, for example, tests how faithfully the LLM executes a supplied rule, which is an empirical intervention rather than a derivation in which the conclusion is assumed by construction. Comparisons against the unguided Basic variant are statistically evaluated with Wilcoxon tests. The only self-citations (e.g., Ref. [12] for the GP framework and Ref. [10] for GP interpretability) are background methodology and are not load-bearing for the central claim that GP-derived guidance improves unguided LLM decisions. The shared underlying project networks across training, validation, and test sets raise a legitimate generalization concern, but that is a scope limitation, not circularity: the feature/rule selection was not based on the test-set outcomes. Therefore no circular step can be identified.
Assumptions & free parameters
free parameters (3)
- Feature-selection cumulative coverage threshold =
80% of cumulative feature occurrences
- GP evolutionary configuration =
Not reported in this version (supplementary material)
- Elite-repository and compact-rule selection criterion =
Not specified
assumptions (5)
- domain assumption DMRCPSP model with precedence constraints, renewable resources, mode-dependent uncertain durations, and makespan objective
- domain assumption GP priority rules trained on training duration realizations transfer to different realizations of the same project instances
- domain assumption LLM decisions at temperature 0 with thinking mode are repeatable enough that 10 repeated runs measure decision variability
- standard math Pairwise Wilcoxon signed-rank tests on 10 paired runs are a valid significance criterion
- ad hoc to paper Semantic matching of LLM rationales to feature definitions (Figure 6) accurately measures feature focus
Cite this review
Pith. "Pith review of Guiding Large Language Models with Genetic Programming-Evolved Heuristic Knowledge for Dynamic Multi-Mode Project Scheduling." pith.science (2026). https://pith.science/paper/IMFCSCFW
@misc{pith2026260727698,
author = {Pith},
title = {Pith review of: Guiding Large Language Models with Genetic Programming-Evolved Heuristic Knowledge for Dynamic Multi-Mode Project Scheduling},
year = {2026},
howpublished = {\url{https://pith.science/paper/IMFCSCFW}},
note = {Machine review of arXiv:2607.27698}
}
read the original abstract
In dynamic multi-mode project scheduling, activities have alternative execution modes and uncertain durations, while precedence relations and limited resources constrain their execution. Heuristic priority rules support fast online decisions, but their design requires substantial domain expertise. Genetic programming (GP) hyper-heuristics can automatically evolve such rules. Large language models (LLMs), meanwhile, provide a flexible interface for interpreting scheduling information and explaining decisions. However, zero-shot LLM decisions may lack domain knowledge, consume many tokens, and vary across repeated queries. GP-evolved rules therefore provide a potential source of scheduling knowledge for guiding LLM decisions. Unlike existing LLM--GP hybrids that use LLMs to support heuristic evolution, we transfer knowledge in the reverse direction, using knowledge extracted from high-quality GP rules to guide an online LLM decision maker. We extract knowledge from high-quality GP rules and inject it through Feature Selection, Feature Hint, Rule Reference, and Rule Follow. These mechanisms are evaluated in terms of scheduling performance, token consumption, decision stability, and the feature focus expressed in generated rationales. GP-derived guidance generally improves the unguided LLM, but its representation matters. Simplifying the decision context or supplying explicit decision logic is more effective than highlighting important features. Feature Selection offers the best token efficiency, whereas Rule Follow achieves strong performance at greater token cost. Guidance also improves decision stability and changes the features expressed in generated rationales.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Abgaryan, H., Cazenave, T., Harutyunyan, A.: Starjob: Dataset for LLM-Driven Job Shop Scheduling (Mar 2025), arXiv:2503.01877 [cs.LG]
arXiv 2025
-
[2]
Abgaryan, H., Harutyunyan, A., Cazenave, T.: LLMs can Schedule (Aug 2024), arXiv:2408.06993 [cs.AI]
arXiv 2024
-
[3]
Cao, S., Yuan, Y.: ReflecSched: Solving Dynamic Flexible Job-Shop Scheduling via LLM-Powered Hierarchical Reflection (Jan 2026), arXiv:2508.01724 [cs.AI]
arXiv 2026
-
[4]
Journal of Scheduling6(1), 17–38 (2003)
Demeulemeester, E., Vanhoucke, M., Herroelen, W.: RanGen: A Random Network Generator for Activity-on-the-Node Networks. Journal of Scheduling6(1), 17–38 (2003)
2003
-
[5]
In: AI 2024: Advances in Artificial Intelligence
Fang, Z., Ma, H., Chen, G., Hartmann, S., Wang, C.: Leveraging LLM in Genetic Programming Hyper-heuristics for Dynamic Microservice Deployment. In: AI 2024: Advances in Artificial Intelligence. pp. 86–97. Springer Nature, Singapore (2025)
2024
-
[6]
European Journal of Operational Research297(1), 1–14 (Feb 2022)
Hartmann, S., Briskorn, D.: An updated survey of variants and extensions of the resource-constrained project scheduling problem. European Journal of Operational Research297(1), 1–14 (Feb 2022)
2022
-
[7]
IEEE Transactions on Fuzzy Systems34(3), 896–908 (Mar 2026)
Huang, J., Liu, Q., Li, X., Gao, L., Teng, Y.: Automatic Programming via Large Language Models With Population Self-Evolution for Dynamic Fuzzy Job Shop Scheduling Problem. IEEE Transactions on Fuzzy Systems34(3), 896–908 (Mar 2026)
2026
-
[8]
International Journal of Production Research pp
Huang, J., Zhang, H., Liu, Q., Li, X., Gao, L.: Automated design of scheduling heuristics for dynamic flexible job shop scheduling with evolutionary textual gra- dients. International Journal of Production Research pp. 1–22 (Jul 2026)
2026
Show all 15 references
-
[9]
In: Proceedings of the 41st International Conference on Machine Learning
Liu,F.,Tong,X.,Yuan,M.,Lin,X.,Luo,F.,Wang,Z.,Lu,Z.,Zhang,Q.:Evolution of heuristics: towards efficient automatic algorithm design using large language model. In: Proceedings of the 41st International Conference on Machine Learning. ICML’24, vol. 235, pp. 32201–32223. JMLR.org,...
2024
-
[10]
IEEE Transactions on Evolutionary Computation27(3), 621–641 (2023)
Mei, Y., Chen, Q., Lensen, A., Xue, B., Zhang, M.: Explainable artificial intel- ligence by genetic programming: A survey. IEEE Transactions on Evolutionary Computation27(3), 621–641 (2023)
2023
-
[11]
Qiu,J.,Zhuang,H.,Liu,F.,Liu,J.,Zhang,Q.:EvoDR:EvolvingDispatchingRules via Large Language Model for Dynamic Flexible Assembly Flow Shop Scheduling (Mar 2026), arXiv:2601.15738 [cs.NE]
2026
-
[12]
In: 2024 IEEE Congress on Evolutionary Computation (CEC)
Tian, Y., Mei, Y., Zhang, M.: Learning Heuristics via Genetic Programming for Multi-mode Resource-constrained Project Scheduling. In: 2024 IEEE Congress on Evolutionary Computation (CEC). pp. 1–8. Yokohama, Japan (Jun 2024)
2024
-
[13]
Advanced Engineering Informatics76, 104946 (2026)
Xiang,W.,Zhang,J.,Zhang,P.,Wang,M.,Li,J.,Li,C.,Feng,L.,Pan,Z.:Asurvey on large language models for scheduling: From problem modeling to optimization algorithm generation. Advanced Engineering Informatics76, 104946 (2026)
2026
-
[14]
IEEE Transactions on Evolutionary Com- putation (2026), Early Access
Xu, M., Liu, J., Ong, Y.S.: Evospeak: Large language models for interpretable ge- netic programming-evolved heuristics. IEEE Transactions on Evolutionary Com- putation (2026), Early Access
2026
-
[15]
IEEE Guiding LLM with GP-Evolved Heuristic Knowledge for DMRCPSP 13 Transactions on Evolutionary Computation28(1), 147–167 (2024)
Zhang, F., Mei, Y., Nguyen, S., Zhang, M.: Survey on genetic programming and machine learning techniques for heuristic design in job shop scheduling. IEEE Guiding LLM with GP-Evolved Heuristic Knowledge for DMRCPSP 13 Transactions on Evolutionary Computation28(1), 147–167 (2024)
2024
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.