pith. sign in

arxiv: 2506.09749 · v3 · submitted 2025-06-11 · 💻 cs.CE · cs.AI

Large Language Models for Combinatorial Optimization of Design Structure Matrix

Shuo Jiang , Min Xie , Jianxi Luo This is my paper

Pith reviewed 2026-05-19 10:11 UTC · model grok-4.3

classification 💻 cs.CE cs.AI
keywords large language modelscombinatorial optimizationdesign structure matrixDSM sequencingengineering designfeedback minimization
0
0 comments X

The pith

Large language models optimize design structure matrix sequencing by combining network topology with domain knowledge to reduce feedback loops.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper shows how large language models can tackle the combinatorial problem of reordering elements in a Design Structure Matrix to minimize feedback loops in complex engineering systems. Traditional mathematical heuristics often overlook contextual details in dependency networks, while LLMs can draw on both the matrix topology and domain knowledge to propose iterative improvements. The proposed framework applies LLMs in repeated reordering steps and tests the results on multiple DSM cases. Experiments indicate faster convergence and better final solutions than standard stochastic and deterministic optimizers. Domain knowledge boosts results across different LLM backbones, pointing to a way for semantic reasoning to aid engineering optimization tasks.

Core claim

The authors introduce an LLM-based framework that integrates network topology with contextual domain knowledge for iterative optimization of DSM sequencing. Experiments on various DSM cases show the method reaches faster convergence and higher solution quality than stochastic and deterministic baselines, with domain knowledge improving performance independently of the LLM chosen.

What carries the argument

LLM-based iterative reordering framework that translates network topology and domain knowledge into successive DSM element permutations.

If this is right

  • LLMs can outperform pure mathematical heuristics on DSM sequencing by incorporating contextual domain knowledge.
  • Performance gains from domain knowledge hold across different LLM backbones.
  • The approach applies to DSM cases of varying sizes and complexity in engineering design.
  • Semantic and mathematical reasoning can be combined in a single iterative loop for combinatorial engineering problems.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same LLM integration pattern could be tested on other dependency-network optimization tasks such as supply-chain sequencing or software module ordering.
  • Hybrid systems that let LLMs propose moves while a deterministic checker validates them might reduce hallucination risks on larger instances.
  • Scaling tests on DSMs with hundreds of elements would reveal whether convergence speed advantages persist as problem size grows.

Load-bearing premise

The method assumes large language models can reliably generate valid reordering steps that improve the DSM without producing combinatorial errors or hallucinations.

What would settle it

Apply the LLM method to a DSM instance whose optimal ordering is known from exhaustive enumeration and check whether the achieved feedback loop count or modularity score matches or beats that optimum.

Figures

Figures reproduced from arXiv: 2506.09749 by Jianxi Luo, Min Xie, Shuo Jiang.

Figure 1
Figure 1. Figure 1: Illustration of a Design Activity DSM: (A) Pre-Sequencing; (B) Post-Sequencing [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the Proposed LLM-based Approach [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: B (Microfilm Cartridge), where both variants of the LLM-based method reach the optimal solution in the first step. As a result, the two lines overlap entirely, reflecting identical performance in this case. This suggests that, for some DSMs with relatively less complexity or clearer structure, even minimal attempts may suffice for the LLM to infer optimal solutions [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Optimization trajectory on the Heat Exchanger DSM case using the proposed method [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: During the engineering design process, systems or tasks can often be formulated as iterative optimization [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
read the original abstract

In complex engineering systems, the dependencies among components or development activities are often modeled and analyzed using Design Structure Matrix (DSM). Reorganizing elements within a DSM to minimize feedback loops and enhance modularity or process efficiency constitutes a challenging combinatorial optimization (CO) problem in engineering design and operations. As problem sizes increase and dependency networks become more intricate, traditional optimization methods that rely solely on mathematical heuristics often fail to capture the contextual nuances and struggle to deliver effective solutions. In this study, we explore the potential of Large Language Models (LLMs) to address such CO problems by leveraging their capabilities for advanced reasoning and contextual understanding. We propose a novel LLM-based framework that integrates network topology with contextual domain knowledge for iterative optimization of DSM sequencing-a common CO problem. Experiments on various DSM cases demonstrate that our method consistently achieves faster convergence and superior solution quality compared to both stochastic and deterministic baselines. Notably, incorporating contextual domain knowledge significantly enhances optimization performance regardless of the chosen LLM backbone. These findings highlight the potential of LLMs to solve complex engineering CO problems by combining semantic and mathematical reasoning. This approach paves the way towards a new paradigm in LLM-based engineering design optimization.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 3 minor

Summary. The manuscript proposes a novel LLM-based framework for combinatorial optimization of Design Structure Matrix (DSM) sequencing. It integrates network topology with contextual domain knowledge to iteratively generate reordering steps, claiming faster convergence and superior solution quality over stochastic and deterministic baselines on various DSM cases, with domain knowledge providing consistent gains independent of the LLM backbone.

Significance. If the reported empirical results hold under the described protocol, the work offers a promising demonstration of LLMs combining semantic and mathematical reasoning for engineering CO problems. Strengths include explicit validity checks on LLM outputs via parsing and rejection sampling, multiple random seeds, direct objective comparisons on identical instances, and measurable correlation between domain-knowledge injection and performance gains. This could support new paradigms in LLM-assisted design optimization.

minor comments (3)
  1. [Abstract] Abstract: The central claim of superior performance and faster convergence would be strengthened by including at least one quantitative metric (e.g., average objective improvement or convergence iterations) even in the abstract.
  2. [Results] Results section: Convergence curves and solution-quality tables should report error bars or standard deviations across the multiple random seeds mentioned in the methods to allow readers to assess variability.
  3. [Methods] Methods: The exact format of the domain-knowledge injection into prompts and the rejection-sampling procedure for enforcing permutation constraints could be described with a short pseudocode snippet or example prompt for reproducibility.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the constructive and positive review, including the recognition of our framework's integration of network topology with domain knowledge, the validity checks, and the observed performance gains. The recommendation for minor revision is noted, and we will prepare a revised manuscript accordingly. As no specific major comments were provided in the report, we have no individual points to address point-by-point below.

Circularity Check

0 steps flagged

No significant circularity in LLM-based DSM optimization

full rationale

The paper's core contribution is an empirical LLM-driven iterative reordering framework for DSM combinatorial optimization, with performance claims resting on direct comparisons to external stochastic and deterministic baselines across multiple DSM instances. Explicit validity enforcement via parsing and rejection sampling, multi-seed runs, and objective-value metrics ensure the reported convergence and quality gains are measured independently rather than defined into existence. No equations, fitted parameters, self-definitional loops, or load-bearing self-citations reduce any prediction or result to the authors' own inputs by construction; the approach remains self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that current LLMs can perform reliable iterative combinatorial search when supplied with topology and domain knowledge; no free parameters or invented entities are stated in the abstract.

axioms (1)
  • domain assumption Large language models possess advanced reasoning and contextual understanding that can be leveraged for combinatorial optimization tasks.
    Invoked when proposing the LLM-based framework in the abstract.

pith-pipeline@v0.9.0 · 5730 in / 1090 out tokens · 24351 ms · 2026-05-19T10:11:15.567857+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

61 extracted references · 61 canonical work pages · 4 internal anchors

  1. [1]

    Design structure matrix methods and applications

    Steven D Eppinger and Tyson R Browning. Design structure matrix methods and applications . MIT press, 2012

    work page 2012

  2. [2]

    The design structure system: A method for managing the design of complex systems

    Donald V Steward. The design structure system: A method for managing the design of complex systems. IEEE transactions on Engineering Management, pages 71–74, 1981

    work page 1981

  3. [3]

    Effective scheduling of user input during the design process

    Young Mi Choi et al. Effective scheduling of user input during the design process. In DS 68-3: Proceedings of the 18th International Conference on Engineering Design (ICED 11), Impacting Society through Engineering Design, V ol. 3: Design Organisation and Management, Lyngby/Copenhagen, Denmark, 15.-19.08. 2011, pages 116–122, 2011

    work page 2011

  4. [4]

    Matching design tasks to knowledge-based software tools: When intuition does not suffice

    Rafael Amen, Ingvar Rask, and Staffan Sunnersjö. Matching design tasks to knowledge-based software tools: When intuition does not suffice. In International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, volume 19715, pages 1165–1174, 1999

    work page 1999

  5. [5]

    A model-based method for organizing tasks in product development

    Steven D Eppinger, Daniel E Whitney, Robert P Smith, and David A Gebala. A model-based method for organizing tasks in product development. Research in engineering design, 6:1–13, 1994

    work page 1994

  6. [6]

    A novel approach to dsm-based activity sequencing problem

    Yanjun Qian, Jun Lin, Thong Ngee Goh, and Min Xie. A novel approach to dsm-based activity sequencing problem. IEEE Transactions on Engineering Management, 58:688–705, 2011

    work page 2011

  7. [7]

    A survey on evaluation of large language models.ACM transactions on intelligent systems and technology, 15:1–45, 2024

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. A survey on evaluation of large language models.ACM transactions on intelligent systems and technology, 15:1–45, 2024

    work page 2024

  8. [8]

    Emergent abilities of large language models.Transactions on Machine Learning Research, 2022

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models.Transactions on Machine Learning Research, 2022

    work page 2022

  9. [9]

    Evolution of heuristics: Towards efficient automatic algorithm design using large language model

    Fei Liu, Tong Xialiang, Mingxuan Yuan, Xi Lin, Fu Luo, Zhenkun Wang, Zhichao Lu, and Qingfu Zhang. Evolution of heuristics: Towards efficient automatic algorithm design using large language model. InInternational Conference on Machine Learning, pages 32201–32223, 2024

    work page 2024

  10. [10]

    Large language models as evolutionary optimizers

    Shengcai Liu, Caishun Chen, Xinghua Qu, Ke Tang, and Yew-Soon Ong. Large language models as evolutionary optimizers. In 2024 IEEE Congress on Evolutionary Computation (CEC) , pages 1–8, 2024

    work page 2024

  11. [11]

    Mathematical discoveries from program search with large language models

    Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco J R Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. Mathematical discoveries from program search with large language models. Nature, 625:468–475, 2024

    work page 2024

  12. [12]

    Large language models as optimizers

    Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. In The Twelfth International Conference on Learning Representations , 2024

    work page 2024

  13. [13]

    Generative op- timization: A perspective on ai-enhanced problem solving in engineering

    Cyril Picard, Lyle Regenwetter, Amin Heyrani Nobari, Akash Srivastava, and Faez Ahmed. Generative op- timization: A perspective on ai-enhanced problem solving in engineering. arXiv preprint arXiv:2412.13281 , 2024

    work page arXiv 2024

  14. [14]

    Autotriz: Automating engineering innovation with triz and large language models

    Shuo Jiang, Weifeng Li, Yuping Qian, Yangjun Zhang, and Jianxi Luo. Autotriz: Automating engineering innovation with triz and large language models. Advanced Engineering Informatics, 65:103312, 2025

    work page 2025

  15. [15]

    Large language models for design and manufacturing

    Liane Makatura, Michael Foshey, Bohan Wang, Felix Hähnlein, Pingchuan Ma, Bolei Deng, Megan Tjandrasuwita, Andrew Spielberg, Crystal Elaine Owens, Peter Yichen Chen, Allan Zhao, Amy Zhu, Wil J Norton, Edward Gu, Joshua Jacob, Yifei Li, Adriana Schulz, and Wojciech Matusik. Large language models for design and manufacturing. An MIT Exploration of Generativ...

    work page 2024

  16. [16]

    Replanvlm: Replanning robotic tasks with visual language models

    Aoran Mei, Guo-Niu Zhu, Huaxiang Zhang, and Zhongxue Gan. Replanvlm: Replanning robotic tasks with visual language models. IEEE Robotics and Automation Letters , 2024

    work page 2024

  17. [17]

    Generative transformers for design concept generation

    Qihao Zhu and Jianxi Luo. Generative transformers for design concept generation. Journal of Computing and Information Science in Engineering , 23:41003, 2023

    work page 2023

  18. [18]

    Use of the design structure matrix in the improvement of an automobile development process

    Michele Wanda Sequeira. Use of the design structure matrix in the improvement of an automobile development process. PhD thesis, Massachusetts Institute of Technology, 1991

    work page 1991

  19. [19]

    A predictive model of sequential iteration in engineering design

    Robert P Smith and Steven D Eppinger. A predictive model of sequential iteration in engineering design. Management Science, 43:1104–1120, 1997

    work page 1997

  20. [20]

    Product development cycle time characterization through modeling of process iteration

    Sean M Osborne. Product development cycle time characterization through modeling of process iteration . PhD thesis, Massachusetts Institute of Technology, 1993

    work page 1993

  21. [21]

    ‘signposting’, a parameter-driven task-based model of the design process

    Peter John Clarkson and James Robert Hamilton. ‘signposting’, a parameter-driven task-based model of the design process. Research in Engineering Design, 12:18–38, 2000

    work page 2000

  22. [22]

    Document viewpoint on one-of-a-kind delivery process

    A-P Hameri. Document viewpoint on one-of-a-kind delivery process. International Journal of Production Research, 37:1319–1336, 1999

    work page 1999

  23. [23]

    From dsm-based planning to design process simulation: a review of process scheme logic verification issues

    Arie Karniel and Yoram Reich. From dsm-based planning to design process simulation: a review of process scheme logic verification issues. IEEE Transactions on Engineering Management, 56:636–649, 2009

    work page 2009

  24. [24]

    Hierarchy in industry architecture: Transaction strategy under technological constraints

    Jianxi Luo. Hierarchy in industry architecture: Transaction strategy under technological constraints . PhD thesis, Massachusetts Institute of Technology, 2010

    work page 2010

  25. [25]

    Resource optimization of product development projects with time-varying dependency structure

    Masaki Ogura, Junichi Harada, Masako Kishida, and Ali Yassine. Resource optimization of product development projects with time-varying dependency structure. Research in Engineering Design, 30:435–452, 2019

    work page 2019

  26. [26]

    Multilayer network model for analysis and management of change propagation

    Michael C Pasqual and Olivier L de Weck. Multilayer network model for analysis and management of change propagation. Research in Engineering Design, 23:305–328, 2012

    work page 2012

  27. [27]

    M-arm: An automated systematic approach for generating new variant design options from an existing product family

    Foo Shing Wong and David C Wynn. M-arm: An automated systematic approach for generating new variant design options from an existing product family. Research in Engineering Design, 35:389–408, 2024

    work page 2024

  28. [28]

    Applying the design structure matrix to system decomposition and integration problems: a review and new directions

    Tyson R Browning. Applying the design structure matrix to system decomposition and integration problems: a review and new directions. IEEE Transactions on Engineering management, 48:292–306, 2001

    work page 2001

  29. [29]

    Structuring product development processes

    Reza Ahmadi, Thomas A Roemer, and Robert H Wang. Structuring product development processes. European Journal of Operational Research, 130:539–558, 2001

    work page 2001

  30. [30]

    A comprehensive mathematical model for sequencing interrelated activities in complex product development projects

    Milad Attari-Shendi, Mohammad Saidi-Mehrabad, and Jafar Gheidar-Kheljani. A comprehensive mathematical model for sequencing interrelated activities in complex product development projects. IEEE Transactions on Engineering Management, 69:2619–2633, 2019

    work page 2019

  31. [31]

    Efficient organizing of design activities.The International Journal Of Production Research, 31:753–769, 1993

    Andrew Kusiak and Juite Wang. Efficient organizing of design activities.The International Journal Of Production Research, 31:753–769, 1993

    work page 1993

  32. [32]

    A fuzzy approach for sequencing interrelated activities in a dsm

    Jun Lin, Yanjun Qian, Ali A Yassine, and Wentian Cui. A fuzzy approach for sequencing interrelated activities in a dsm. International journal of production research, 50:7012–7025, 2012

    work page 2012

  33. [33]

    Springer, 1996

    James L Rogers, Collin M McCulley, and Christina L Bloebaum.Integrating a genetic algorithm into a knowledge- based system for ordering complex design processes . Springer, 1996

    work page 1996

  34. [34]

    Combinatorial optimization, volume 1

    Bernhard H Korte, Jens Vygen, B Korte, and J Vygen. Combinatorial optimization, volume 1. Springer, 2011

    work page 2011

  35. [35]

    Application of combinatorial optimization strategies in synthetic biology

    Gita Naseri and Mattheos A G Koffas. Application of combinatorial optimization strategies in synthetic biology. Nature communications, 11:2446, 2020

    work page 2020

  36. [36]

    Energy efficient motion design and task scheduling for an autonomous vehicle

    Elias Xidias and Philip Azariadis. Energy efficient motion design and task scheduling for an autonomous vehicle. In Proceedings of the Design Society: International Conference on Engineering Design , volume 1, pages 2853–2862, 2019

    work page 2019

  37. [37]

    Scheduling, volume 29

    Michael L Pinedo. Scheduling, volume 29. Springer, 2012

    work page 2012

  38. [38]

    Introducing computer use, a new claude 3.5 sonnet, and claude 3.5 haiku, 2024

    Anthropic. Introducing computer use, a new claude 3.5 sonnet, and claude 3.5 haiku, 2024. Accessed Oct 2024, https://www.anthropic.com/news/3-5-models-and-computer-use

    work page 2024

  39. [39]

    DeepSeek-V3 Technical Report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  40. [40]

    Introducing meta llama 3: The most capable openly available llm to date, 2024

    Meta. Introducing meta llama 3: The most capable openly available llm to date, 2024. https://ai.meta.com/blog/meta-llama-3/. 14 LARGE LANGUAGE MODELS FOR DESIGN STRUCTURE MATRIX OPTIMIZATION

    work page 2024

  41. [41]

    Gpt-4 turbo, 2024

    OpenAI. Gpt-4 turbo, 2024. https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo

    work page 2024

  42. [42]

    Auto-dsm: Using a large language model to generate a design structure matrix.Natural Language Processing Journal, 9:100103, 2024

    Edwin C Y Koh. Auto-dsm: Using a large language model to generate a design structure matrix.Natural Language Processing Journal, 9:100103, 2024

    work page 2024

  43. [43]

    What is generative in generative artificial intelligence? a design-based perspective

    Antoine Bordas, Pascal Le Masson, Maxime Thomas, and Benoit Weil. What is generative in generative artificial intelligence? a design-based perspective. Research in Engineering Design, 35:427–443, 2024

    work page 2024

  44. [44]

    A Survey of Large Language Models

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint arXiv:2303.18223, 1, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  45. [45]

    Modeling and analyzing cost, schedule, and performance in complex system product development

    Tyson R Browning. Modeling and analyzing cost, schedule, and performance in complex system product development. PhD thesis, Massachusetts Institute of Technology, Sloan School of Management . . . , 1998

    work page 1998

  46. [46]

    Product design and development

    Steven D Eppinger and Karl Ulrich. Product design and development. McGraw-Hill New York, 2016

    work page 2016

  47. [47]

    A method for systems design using precedence relationships: An application to automotive brake systems

    Thomas Andrew Black, Charles H Fine, Emanuel M Sachs, et al. A method for systems design using precedence relationships: An application to automotive brake systems. 1990

    work page 1990

  48. [48]

    Genetic algorithms: Theory, genetic operators, solutions, and applications

    Bushra Alhijawi and Arafat Awajan. Genetic algorithms: Theory, genetic operators, solutions, and applications. Evolutionary Intelligence, 17:1245–1256, 2024

    work page 2024

  49. [49]

    Deap: Evolutionary algorithms made easy

    Félix-Antoine Fortin, François-Michel De Rainville, Marc-André Gardner Gardner, Marc Parizeau, and Christian Gagné. Deap: Evolutionary algorithms made easy. The Journal of Machine Learning Research , 13:2171–2175, 2012

    work page 2012

  50. [50]

    Googling the brain: Discovering hierarchical and asymmetric network structures, with applications in neuroscience

    Jonathan J Crofts and Desmond J Higham. Googling the brain: Discovering hierarchical and asymmetric network structures, with applications in neuroscience. Internet Mathematics, 7:233–254, 2011

    work page 2011

  51. [51]

    The measurement of interindustry linkages: Key sectors in the netherlands

    Erik Dietzenbacher. The measurement of interindustry linkages: Key sectors in the netherlands. Economic Modelling, 9:419–437, 1992

    work page 1992

  52. [52]

    Subgraph centrality in complex networks

    Ernesto Estrada and Juan A Rodriguez-Velazquez. Subgraph centrality in complex networks. Physical Review E—Statistical, Nonlinear , and Soft Matter Physics , 71:56103, 2005

    work page 2005

  53. [53]

    Network properties revealed through matrix functions

    Ernesto Estrada and Desmond J Higham. Network properties revealed through matrix functions. SIAM review, 52: 696–714, 2010

    work page 2010

  54. [54]

    mirroring

    Alan MacCormack, Carliss Baldwin, and John Rusnak. Exploring the duality between product and organizational architectures: A test of the “mirroring” hypothesis. Research policy, 41:1309–1324, 2012

    work page 2012

  55. [55]

    Mixtral of Experts

    Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Deven- dra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  56. [56]

    RouterBench: A Benchmark for Multi-LLM Routing System

    Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, and Shriyash Kaustubh Upadhyay. Routerbench: A benchmark for multi-llm routing system. arXiv preprint arXiv:2403.12031, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  57. [57]

    Ai-artifacts in engineering change management–a systematic literature review

    Peter Burggräf, Johannes Wagner, Till Saßmannshausen, Tim Weißer, and Ognjen Radisic-Aberger. Ai-artifacts in engineering change management–a systematic literature review. Research in Engineering Design, 35:215–237, 2024

    work page 2024

  58. [58]

    Using engineering change forecast to prioritise component modularisation

    Edwin C Y Koh, Armin Förg, Matthias Kreimeyer, and Markus Lienkamp. Using engineering change forecast to prioritise component modularisation. Research in Engineering Design, 26:337–353, 2015

    work page 2015

  59. [59]

    Pareto-optimization of complex system architecture for structural complexity and modularity

    Kaushik Sinha and Eun Suk Suh. Pareto-optimization of complex system architecture for structural complexity and modularity. Research in Engineering Design, 29:123–141, 2018

    work page 2018

  60. [60]

    P., Galley, M., Caruana, R., and Gao, J

    Chandan Singh, Jeevana Priya Inala, Michel Galley, Rich Caruana, and Jianfeng Gao. Rethinking interpretability in the era of large language models. arXiv preprint arXiv:2402.01761, 2024

    work page arXiv 2024

  61. [61]

    Seed-bench: Benchmarking multimodal large language models

    Bohao Li, Yuying Ge, Yixiao Ge, Guangzhi Wang, Rui Wang, Ruimao Zhang, and Ying Shan. Seed-bench: Benchmarking multimodal large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13299–13308, 2024. 15 LARGE LANGUAGE MODELS FOR DESIGN STRUCTURE MATRIX OPTIMIZATION Appendix 1. Full prompts Prompt for...

    work page 2024