pith. sign in

arxiv: 2604.21807 · v1 · submitted 2026-04-23 · ❄️ cond-mat.mtrl-sci

Modeling High Entropy Alloys' Mechanical Property through Natural Language-Derived Descriptors

Li-Cheng Hsiao , Zi-Kui Liu , Wesley Reinhart This is my paper

Pith reviewed 2026-05-09 21:19 UTC · model grok-4.3

classification ❄️ cond-mat.mtrl-sci
keywords high-entropy alloysmachine learningprocessing treatmentstransformer embeddingshardness predictionnatural language descriptors
0
0 comments X

The pith

Transformer embeddings of processing treatment text improve high-entropy alloy hardness predictions by 20%.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows that natural language descriptions of alloy processing steps contain information that machine learning models can use to predict mechanical properties more accurately. Transformer embeddings of synthesized annealing text reconstruct the original processing parameters with R2 greater than 0.99. When these vector representations are added as descriptors to composition-based models, hardness prediction for high-entropy alloys improves by 20%.

Core claim

Vector representations derived from natural language descriptions of processing treatments serve as effective descriptors that improve machine learning predictions of high-entropy alloy hardness.

What carries the argument

Transformer embeddings of synthesized annealing processing treatment text, used as vector descriptors that encode semantic processing information.

Load-bearing premise

The observed improvement in hardness prediction comes from the semantic content of the processing embeddings rather than from simply adding more input dimensions.

What would settle it

Replace the learned embeddings with random vectors of identical dimension and retrain the hardness model to check whether the 20% gain disappears.

Figures

Figures reproduced from arXiv: 2604.21807 by Li-Cheng Hsiao, Wesley Reinhart, Zi-Kui Liu.

Figure 1
Figure 1. Figure 1: The first (a), the second (b) and the third (c) principal component values of the embeddings of synthesized text with respect to annealing time for annealing description text with different phrasings In this section, we will demonstrate that the embeddings are robust and effective features to incorporate HEAs’ processing informa￾tion into machine learning models. That is, the embedding features should be p… view at source ↗
Figure 2
Figure 2. Figure 2: Feature importance analysis of annealing time (a, c, e) and temperature (b, d, f) models under different regularization schemes: Linear (a, b), L1 (c, d) and L2 (e, f) Feature importance test is conducted to investigate the representation of HEAs’ processing treatment information through their embedding vectors. The permutation importance of mundane linear regression models and linear regression models wit… view at source ↗
Figure 3
Figure 3. Figure 3: (a) The distribution of elements of alloys in the dataset. (b) The distribution of the processing treatments of alloys in the dataset, including: "HT" (heat-treatment), "PMP" (powder metallurgical processes), "AC" (As-cast alloys without further treatments), "Q" (Quenched) and "0" (Processes that cannot be classified as any other processes) To demonstrate the general effectiveness of contextual embeddings … view at source ↗
Figure 4
Figure 4. Figure 4: The R2 and MSE of RF, RF-S and RF-E models We then proceed to compare the performance of different classic machine learning regressor models on different datasets, which are raw dataset with only composition and temperature, the dataset with processing symbol added on top of the raw dataset and the dataset with embeddings instead of processing symbol to investigate how the embeddings are utilized by differ… view at source ↗
Figure 5
Figure 5. Figure 5: The performance of different classic machine learning models trained on the dataset. The baseline dataset contain compositional and temperature information, the categorecal dataset adds one-hot encoded symbols for processing treatment descriptions, and the NLP dataset adds embedding of alloy processing information description on top of baseline dataset [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The performance of models trained with HEA processing treatment information represented with different natural language processing techniques In conclusion, we applied natural language processing techniques to acquire vector representations of HEA processing 6/8 [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
read the original abstract

Processing treatments of alloys, despite being influential to alloy properties, are often neglected in machine-learning aided alloy designs due to the difficulties in expressing this information. We investigated the expressiveness of transformer embeddings through synthesized annealing processing treatment text and verified that embeddings could be utilized to reconstruct the processing parameters of alloys effectively with an R2>0.99. We then utilized the vector representations of alloys' processing treatment descriptions as descriptors to model high-entropy alloys' hardness and achieved a 20% improvement in prediction, verifying that natural language-derived descriptors of processing treatment information could be utilized to improve prediction of alloy properties.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript proposes using transformer embeddings derived from synthesized natural language descriptions of annealing processing treatments as descriptors for high-entropy alloy (HEA) mechanical properties. It reports that these embeddings reconstruct processing parameters with R² > 0.99 and, when concatenated to composition features, yield a 20% improvement in hardness prediction models.

Significance. If the reported improvement can be attributed specifically to the semantic content of the embeddings rather than added dimensionality or model capacity, the approach would offer a practical route to incorporate often-neglected processing information into ML-based alloy design. The high R² reconstruction accuracy is a clear technical strength that demonstrates the embeddings encode relevant processing details.

major comments (2)
  1. [Abstract / hardness prediction results] Abstract and hardness modeling results: the claimed 20% improvement in hardness prediction is presented without any baseline model specification, cross-validation details, feature importance rankings, or ablation studies that control for dimensionality. This makes it impossible to determine whether the gain stems from the natural-language-derived descriptors or from simply increasing the feature space.
  2. [Hardness modeling results] Hardness modeling section: no control experiments (e.g., replacement of embeddings by random vectors of identical dimension or shuffled processing labels) are reported to isolate the contribution of semantic content from spurious correlations or increased model capacity. Without such tests the central claim that the descriptors improve prediction specifically because they are natural-language-derived cannot be evaluated.
minor comments (1)
  1. [Abstract] The abstract would be strengthened by stating the size of the HEA dataset, the exact transformer architecture and embedding dimension (768 is mentioned in the skeptic note but not here), and the regression algorithm used for both reconstruction and hardness prediction.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive and detailed comments, which have helped us strengthen the manuscript. We address each major comment below and have revised the manuscript accordingly to provide the requested details, baselines, and controls.

read point-by-point responses

  1. Referee: [Abstract / hardness prediction results] Abstract and hardness modeling results: the claimed 20% improvement in hardness prediction is presented without any baseline model specification, cross-validation details, feature importance rankings, or ablation studies that control for dimensionality. This makes it impossible to determine whether the gain stems from the natural-language-derived descriptors or from simply increasing the feature space.

    Authors: We agree that the original presentation lacked sufficient specification of the baseline and validation procedures. In the revised manuscript, we have explicitly defined the baseline as a model using only compositional features with the identical machine-learning algorithm and hyperparameter settings. We now report 5-fold cross-validation results with details on data splitting and performance metrics. Feature importance rankings are included, highlighting the relative contribution of the embedding dimensions. We have also added an ablation study that incrementally increases feature dimensionality with non-semantic features and shows that performance gains require the specific content of the natural-language embeddings rather than dimensionality alone. revision: yes

  2. Referee: [Hardness modeling results] Hardness modeling section: no control experiments (e.g., replacement of embeddings by random vectors of identical dimension or shuffled processing labels) are reported to isolate the contribution of semantic content from spurious correlations or increased model capacity. Without such tests the central claim that the descriptors improve prediction specifically because they are natural-language-derived cannot be evaluated.

    Authors: We concur that control experiments are essential to attribute the improvement to semantic content. The revised hardness modeling section now includes two explicit controls: (1) substitution of the transformer embeddings with random vectors of the same dimensionality, and (2) use of embeddings derived from shuffled processing labels. In both controls, the hardness prediction accuracy remains statistically indistinguishable from the composition-only baseline, whereas the original natural-language-derived embeddings produce the reported improvement. These results are presented with statistical significance tests and are discussed in the context of isolating semantic information from model capacity effects. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical ML feature addition with reported performance gain

full rationale

The paper's chain consists of (1) generating text descriptions of annealing treatments, (2) obtaining transformer embeddings, (3) verifying that those embeddings can reconstruct the original processing parameters (R²>0.99), and (4) concatenating the embeddings to composition features to train a hardness regressor that shows a 20% error reduction. None of these steps reduces to a self-definition, a fitted parameter renamed as a prediction, or a self-citation that carries the central claim. The 20% improvement is an empirical outcome of a standard supervised-learning experiment; it is not forced by construction from the reconstruction task or from any prior result by the same authors. No equations, uniqueness theorems, or ansatzes are invoked that would make the result tautological. The absence of an ablation (random vectors vs. semantic embeddings) is a validity concern, not a circularity concern.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The work rests on standard transformer embedding assumptions and the premise that processing text contains recoverable quantitative information; no new free parameters or invented entities are introduced beyond the embedding model itself.

axioms (1)
  • domain assumption Transformer embeddings preserve semantically meaningful information about annealing parameters
    Invoked when claiming R²>0.99 reconstruction and downstream utility for property modeling.

pith-pipeline@v0.9.0 · 5400 in / 1081 out tokens · 17021 ms · 2026-05-09T21:19:45.542160+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

19 extracted references · 2 canonical work pages · 1 internal anchor

  1. [1]

    engineering materials6, 299–303 (2004)

    Yeh, J.-W.et al.Nanostructured high-entropy alloys with multiple principal elements: novel alloy design concepts and outcomes.Adv. engineering materials6, 299–303 (2004)

    2004

  2. [2]

    T., Knight, P

    Cantor, B., Chang, I. T., Knight, P. & Vincent, A. Microstructural development in equiatomic multicomponent alloys. Mater. Sci. Eng. A375, 213–218 (2004)

    2004

  3. [3]

    & Yang, Y

    Ye, Y ., Wang, Q., Lu, J., Liu, C. & Yang, Y . High-entropy alloy: challenges and prospects.Mater. Today19, 349–362 (2016). 4.Yeh, J.-W. Alloy design strategies and future trends in high-entropy alloys.Jom65, 1759–1771 (2013)

    2016

  4. [4]

    & Chen, Q

    Chen, H.-L., Mao, H. & Chen, Q. Database development and calphad calculations for high entropy alloys: Challenges, strategies, and tips.Mater. Chem. Phys.210, 279–290 (2018)

    2018

  5. [5]

    Li, T.et al.Calphad-aided design for superior thermal stability and mechanical behavior in a tizrhfnb refractory high-entropy alloy.Acta Materialia246, 118728 (2023)

    2023

  6. [6]

    & Zhong, Y

    Yang, S., Lu, J., Xing, F., Zhang, L. & Zhong, Y . Revisit the vec rule in high entropy alloys (heas) with high-throughput calphad approach and its applications for material design-a case study with al–co–cr–fe–ni system.Acta Materialia192, 11–19 (2020)

    2020

  7. [7]

    & Körmann, F

    Ikeda, Y ., Grabowski, B. & Körmann, F. Ab initio phase stabilities and mechanical properties of multicomponent alloys: A comprehensive review for high entropy alloys and compositionally complex alloys.Mater. Charact.147, 464–511 (2019)

    2019

  8. [8]

    & Irving, D

    Zaddach, A., Niu, C., Koch, C. & Irving, D. Mechanical properties and stacking fault energies of nifecrcomn high-entropy alloy.Jom65, 1780–1789 (2013)

    2013

  9. [9]

    Wen, C.et al.Machine learning assisted design of high entropy alloys with desired property.Acta Materialia170, 109–117 (2019)

    2019

  10. [10]

    & Zhuang, H

    Huang, W., Martin, P. & Zhuang, H. L. Machine-learning phase prediction of high-entropy alloys.Acta Materialia169, 225–236 (2019). 12.Rao, Z.et al.Machine learning–enabled high-entropy alloy discovery.Science378, 78–85 (2022)

    2019

  11. [11]

    & Pei, Z

    Liu, X., Zhang, J. & Pei, Z. Machine learning for high-entropy alloys: Progress, challenges and opportunities.Prog. Mater. Sci.131, 101018 (2023)

    2023

  12. [12]

    & Guo, W

    Li, Y . & Guo, W. Machine-learning model for predicting phase formations of high-entropy alloys.Phys. Rev. Mater.3, 095005 (2019)

    2019

  13. [13]

    & Zhuang, H

    Islam, N., Huang, W. & Zhuang, H. L. Machine learning for phase selection in multi-principal element alloys.Comput. Mater. Sci.150, 230–235 (2018)

    2018

  14. [14]

    A., Raush, J., Montemore, M

    Sulley, G. A., Raush, J., Montemore, M. M. & Hamm, J. Accelerating high-entropy alloy discovery: efficient exploration via active learning.Scripta Materialia249, 116180 (2024)

    2024

  15. [15]

    & Des.223, 111186 (2022)

    Li, H.et al.Towards high entropy alloy with enhanced strength and ductility using domain knowledge constrained active learning.Mater. & Des.223, 111186 (2022). 18.Wang, H.et al.Scientific discovery in the age of artificial intelligence.Nature620, 47–60 (2023). 19.Vaswani, A.et al.Attention is all you need.Adv. neural information processing systems30(2017)

    2022

  16. [16]

    & Cooper, S

    Lei, G., Docherty, R. & Cooper, S. J. Materials science in the era of large language models: a perspective.Digit. Discov.3, 1257–1272 (2024)

    2024

  17. [17]

    Tshitoyan, V .et al.Unsupervised word embeddings capture latent knowledge from materials science literature.Nature 571, 95–98 (2019)

    2019

  18. [18]

    Gemini: A Family of Highly Capable Multimodal Models

    Jablonka, K. M., Schwaller, P., Ortega-Guerrero, A. & Smit, B. Leveraging large language models for predictive chemistry. Nat. Mach. Intell.6, 161–169 (2024). 23.Gao, S.et al.Empowering biomedical discovery with ai agents.Cell187, 6125–6151 (2024). 24.Team, G.et al.Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805(2023). 7/8

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  19. [19]

    Krajewski, A.et al.Ultrahigh temperature refractory alloys (ultera) database of high entropy alloys.Zenodo. Descr. in: https://phaseslab. com/ultera. Available from: https://doi. org/10.5281/zenodo7566416(2023). 8/8

    work page doi:10.5281/zenodo7566416(2023 2023