Modeling and Calibration of Supplier Selection Problem in Freight Agent-Based Simulations
Pith reviewed 2026-05-17 07:13 UTC · model grok-4.3
The pith
A calibrated agent-based model reproduces observed freight commodity flows and shipping distances by integrating trade links, costs, and supplier ratings.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Minimizing the discrepancy between modeled and observed commodity flows while matching regional shipping distance distributions produces a behaviorally accurate representation of real-world supplier selection decisions in freight networks.
What carries the argument
The supplier selection mechanism that integrates inter-sector trade data, transportation costs, and an adapted supplier rating model inside a probabilistic heuristic for assigning shipments.
If this is right
- The framework supports evaluation of targeted policy interventions that affect freight flows.
- Planners can simulate effects of infrastructure investments on supply chain performance.
- The approach highlights spatial variations in commodity trade assignments across regions.
- Consistent reproduction of national shipping distance trends allows comparison of local patterns to broader data.
Where Pith is reading between the lines
- The calibration process could be repeated in additional cities using local freight data to check whether the same objective function remains effective.
- The model structure might be adapted to test scenarios involving sudden disruptions such as port closures or fuel price spikes.
- Similar minimization of flow and distance discrepancies could be tried in related domains like passenger travel or inventory routing.
Load-bearing premise
Real supplier selection decisions can be captured accurately by minimizing differences between simulated and observed commodity flows while also matching shipping distance distributions.
What would settle it
Applying the same calibration procedure to data from an additional city or a later time period and obtaining large mismatches in both commodity flows and distance distributions would show the model does not represent the underlying choices.
Figures
read the original abstract
Freight transportation modeling often struggles with data limitations, especially in accurately representing complex supplier selection processes and their impact on network flows. This research addresses this critical gap by developing a large-scale, calibrated agent-based model for supplier selection, complemented by a probabilistic heuristic for international shipments. Our approach integrates trade relationships between industry sectors, transportation costs, and supplier rating model adapted from existing literature. The model's core objective is to minimize the discrepancy between modeled and observed commodity flows while ensuring a close match to regional shipping distance distributions. Implemented and tested across four major U.S. metropolitan areas, Atlanta, Chicago, Dallas-Fort Worth, and Los Angeles, the model demonstrates high fidelity in replicating observed freight patterns. Key findings reveal consistent alignment with national shipping distance trends and highlight significant spatial variations in commodity trade assignments and demand across the study regions. This behaviorally informed and transport-sensitive framework is designed to approximate real-world decision-making, providing a robust tool for policymakers and planners to evaluate targeted interventions, assess infrastructure investments, and enhance supply chain resilience in the face of disruptions.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper develops a large-scale agent-based model (ABM) for supplier selection in freight transportation. It integrates inter-industry trade relationships, transportation costs, and an adapted supplier rating model from the literature, along with a probabilistic heuristic for international shipments. The core calibration objective is to minimize discrepancy between modeled and observed commodity flows while matching regional shipping distance distributions. The model is implemented and tested in four U.S. metropolitan areas (Atlanta, Chicago, Dallas-Fort Worth, Los Angeles), with the central claim being high-fidelity replication of observed freight patterns and identification of spatial variations in trade assignments.
Significance. If the calibrated parameters can be shown to reflect actual supplier decision processes rather than aggregate fitting alone, the framework could provide a useful tool for assessing infrastructure investments and supply-chain resilience. The combination of ABM with an adapted supplier rating model addresses a recognized data gap in freight modeling; however, the absence of disaggregate validation or out-of-sample tests limits the strength of the behavioral claims.
major comments (3)
- [Abstract] Abstract: The assertion that the model 'demonstrates high fidelity in replicating observed freight patterns' is unsupported by any quantitative metrics (e.g., MAPE, RMSE, or Kolmogorov-Smirnov statistics for flows and distance distributions), error bars, or explicit validation procedures, rendering the central replication claim unevaluable from the given information.
- [Calibration procedure] Calibration procedure (as described): The explicit objective of minimizing discrepancy with observed commodity flows while matching distance distributions creates circularity; the outputs are constructed to reproduce the calibration targets, so apparent 'predictions' reduce to fitted reproductions rather than independent tests of the supplier rating or trade-relationship logic.
- [Model description] Model description: No direct test is reported that the calibrated supplier rating parameters align with disaggregate choice data or observed individual supplier decisions; aggregate flow matching alone does not establish that the behavioral mechanism is correctly specified.
minor comments (1)
- [Abstract] The abstract would benefit from a brief statement of the number of agents, time steps, or computational scale used in the four-city implementations.
Simulated Author's Rebuttal
We thank the referee for the thoughtful and constructive comments on our manuscript. We address each of the major comments in detail below, indicating where revisions will be made to improve the clarity and rigor of the paper.
read point-by-point responses
-
Referee: [Abstract] The assertion that the model 'demonstrates high fidelity in replicating observed freight patterns' is unsupported by any quantitative metrics (e.g., MAPE, RMSE, or Kolmogorov-Smirnov statistics for flows and distance distributions), error bars, or explicit validation procedures, rendering the central replication claim unevaluable from the given information.
Authors: We agree that the abstract's claim would benefit from supporting quantitative evidence. In the revised manuscript, we will include specific metrics such as Mean Absolute Percentage Error (MAPE) and Root Mean Square Error (RMSE) for the commodity flows, as well as Kolmogorov-Smirnov statistics to compare the modeled and observed distance distributions. We will also incorporate error bars or confidence intervals where appropriate and provide a more detailed description of the validation procedures in the methods and results sections. revision: yes
-
Referee: [Calibration procedure] The explicit objective of minimizing discrepancy with observed commodity flows while matching distance distributions creates circularity; the outputs are constructed to reproduce the calibration targets, so apparent 'predictions' reduce to fitted reproductions rather than independent tests of the supplier rating or trade-relationship logic.
Authors: The referee raises a valid point regarding the nature of calibration. Our model is calibrated to reproduce observed flows and distance distributions to ensure behavioral plausibility. However, the underlying logic based on the supplier rating model and inter-industry trade relationships allows the framework to be used for counterfactual analyses beyond the calibration data. We will revise the manuscript to explicitly discuss this distinction, clarify that the results represent calibrated reproductions suitable for scenario modeling rather than independent predictions, and acknowledge the limitations of in-sample calibration. Additionally, we will explore opportunities for out-of-sample testing in future work. revision: partial
-
Referee: [Model description] No direct test is reported that the calibrated supplier rating parameters align with disaggregate choice data or observed individual supplier decisions; aggregate flow matching alone does not establish that the behavioral mechanism is correctly specified.
Authors: We acknowledge that our validation relies on aggregate data matching, which is a limitation given the scarcity of disaggregate freight supplier choice data. The supplier rating model is adapted from established literature on supplier selection, and the calibration process ensures that the aggregate outcomes align with observed commodity flows. We will add a dedicated subsection in the discussion to address this limitation, explain the rationale for using aggregate validation, and suggest directions for future research involving disaggregate data collection or surveys. revision: yes
Circularity Check
Replication of observed freight patterns reduces to calibration fit by construction
specific steps
-
fitted input called prediction
[Abstract]
"The model's core objective is to minimize the discrepancy between modeled and observed commodity flows while ensuring a close match to regional shipping distance distributions. ... Implemented and tested across four major U.S. metropolitan areas, Atlanta, Chicago, Dallas-Fort Worth, and Los Angeles, the model demonstrates high fidelity in replicating observed freight patterns."
The objective function is defined to drive modeled flows toward observed flows and distance distributions; the subsequent claim of 'high fidelity in replicating observed freight patterns' is therefore the direct numerical outcome of that minimization rather than an emergent or predictive result.
full rationale
The paper's central claim of high-fidelity replication across four metro areas rests on a calibration objective that explicitly minimizes discrepancy with observed commodity flows and matches distance distributions. This makes the reported alignment a direct consequence of parameter adjustment to the calibration targets rather than an independent test of the supplier selection mechanism. No disaggregate choice data or out-of-sample behavioral validation is described to break the loop. The derivation chain therefore contains a fitted-input-called-prediction step at its core.
Axiom & Free-Parameter Ledger
free parameters (3)
- calibration parameters for commodity flow matching
- parameters in the adapted supplier rating model
- parameters for probabilistic international shipment heuristic
axioms (1)
- domain assumption Supplier selection can be adequately represented by integrating trade relationships between sectors, transportation costs, and supplier ratings
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
minimize weighted cost of unmet demand, transportation cost, supplier-receiver rating factor, percentage gap in demand tonnage in distance bins, and gap in inter-zonal commodity tonnage flow (objective (1))
-
IndisputableMonolith/Foundation/AbsoluteFloorClosure.leanreality_from_one_distinction unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
decompose into supplier selection then commodity assignment; calibrate to CFS/FAF distance distributions and zonal flows
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
Department of Transportation, Bureau of Transportation Statistics
U.S. Department of Transportation, Bureau of Transportation Statistics. Freight Facts and Figures. Available at https://data.bts.gov/stories/s/Moving-Goods-in-the-United-S tates/bcyt-rqmu accessed on July 9, 2025, 2024
work page 2025
-
[2]
Holgu´ ın-Veras, J., T. Encarnaci´ on, C. A. Gonz´ alez-Calder´ on, J. Winebrake, C. Wang, S. Kyle, N. Herazo-Padilla, L. Kalahasthi, W. Adarme, V. Cantillo, H. Yoshizaki, and R. Gar- rido. Direct impacts of off-hour deliveries on urban freight emissions. Transportation Re- search Part D: Transport and Environment , Vol. 61, 2018, pp. 84–103. doi:https://...
-
[3]
Winebrake, J. J., J. J. Corbett, A. Falzarano, J. S. Hawker, K. Korfmacher, S. Ketha, and S. Zilora. Assessing Energy, Environmental, and Economic Tradeoffs in Intermodal Freight Transportation. Journal of the Air & Waste Management Association, Vol. 58, No. 8, 2008, pp. 1004–1013. doi:10.3155/1047-3289.58.8.1004. URL https://doi.org/10.3155/1047-3289. 58...
-
[4]
Freight Facts and Figures: Moving Goods in the United States
Bureau of Transportation Statistics. Freight Facts and Figures: Moving Goods in the United States. https://data.bts.gov/stories/s/Moving-Goods-in-the-United-States/bcyt-r qmu/, 2024. Accessed: 2025-11-21
work page 2024
-
[5]
LaRocco, L. A. Suez Canal blockage is delaying an estimated $400 million an hour in goods. Available at https://www.cnbc.com/2021/03/25/suez-canal-blockage-is-delay ing-an-estimated-400-million-an-hour-in-goods.html accessed on July 9, 2025, 2021
work page 2021
-
[6]
Tran, N. K., H. Haralambides, T. Notteboom, and K. Cullinane. The costs of maritime supply chain disruptions: The case of the Suez Canal blockage by the “Ever Given” megaship. International Journal of Production Economics, Vol. 279, No. 0925-5273, 2024, p. 109464. doi: https://doi.org/10.1016/j.ijpe.2024.109464. URL https://www.sciencedirect.com/scienc e/...
-
[7]
Swanson, A. and S. Romero. For First Time in Two Decades, U.S. Buys More From Mexico Than China, 2024. URL https://www.nytimes.com/2024/02/07/business/economy/unite d-states-china-mexico-trade.html
work page 2024
-
[8]
Key Ports for Logistics from China to the USA
Tonlexing. Key Ports for Logistics from China to the USA. Available at https://www.tonl exing.com/key-ports-for-logistics-from-china-to-the-usa/ accessed on July 9, 2025, 2025
work page 2025
-
[9]
Freight Analysis Framework (FAF)
Bureau of Transportation Statistics. Freight Analysis Framework (FAF). Available at https: //faf.ornl.gov/faf5/Default.aspx accessed on July 9, 2025, 2017
work page 2025
-
[10]
Pourabdollahi, Z., B. Karimi, K. Mohammadian, and K. Kawamura. A hybrid agent-based computational economics and optimization approach for supplier selection problem. Inter- national Journal of Transportation Science and Technology , Vol. 6, No. 4, 2017, p. 344–355. doi:https://doi.org/10.1016/j.ijtst.2017.09.004. 19
-
[11]
Sarkar, A. and P. K. Mohapatra. Evaluation of supplier capability and performance: A method for supply base reduction. Journal of Purchasing and Supply Management, Vol. 12, No. 3, 2006, p. 148–163. doi:https://doi.org/10.1016/j.pursup.2006.08.003
-
[12]
Thanaraksakul, W. and B. Phruksaphanrat. Supplier Evaluation Framework Based on Bal- anced Scorecard with Integrated Corporate Social Responsibility Perspective. In Proceedings of the International MultiConference of Engineers and Computer Scientists . 2009, pp. 1–6. URL https://www.iaeng.org/publication/IMECS2009/IMECS2009_pp1929-1934.pdf
work page 2009
-
[13]
Chan, F. T. and N. Kumar. Global supplier development considering risk factors using fuzzy extended AHP-based approach. Omega, Vol. 35, No. 4, 2007, p. 417–431. doi:https://doi.org/ 10.1016/j.omega.2005.08.004
-
[14]
Watt, D., B. Kayis, and K. Willey. The relative importance of tender evaluation and contractor selection criteria. International Journal of Project Management, Vol. 28, No. 1, 2010, p. 51–60. doi:https://doi.org/10.1016/j.ijproman.2009.04.003
-
[15]
Ishizaka, A., C. Pearman, and P. Nemery. AHPSort: an AHP-based method for sorting problems. International Journal of Production Research , Vol. 50, No. 17, 2012, p. 4767–4784. doi:https://doi.org/10.1080/00207543.2012.657966
- [16]
-
[17]
Crispim, J. A. and J. P. de Sousa. Partner selection in virtual enterprises. International Journal of Production Research, Vol. 48, No. 3, 2008, p. 683–707. doi:https://doi.org/10.1080/ 00207540802425369
work page 2008
-
[18]
¨On¨ ut, S., S. S. Kara, and E. I¸ sik. Long term supplier selection using a combined fuzzy MCDM approach: A case study for a telecommunication company. Expert Systems with Applications , Vol. 36, No. 2, 2009, p. 3887–3895. doi:https://doi.org/10.1016/j.eswa.2008.02.045
-
[19]
Azadeh, A. and S. Alem. A flexible deterministic, stochastic and fuzzy Data Envelopment Analysis approach for supply chain risk and vendor selection problem: Simulation analysis. Expert Systems with Applications, Vol. 37, No. 12, 2010, p. 7438–7448. doi:https://doi.org/10. 1016/j.eswa.2010.04.022
work page 2010
-
[20]
Aktar Demirtas, E. and O. Ustun. Analytic network process and multi-period goal program- ming integration in purchasing decisions. Computers & Industrial Engineering , Vol. 56, No. 2, 2009, p. 677–690. doi:https://doi.org/10.1016/j.cie.2006.12.006
-
[21]
Farzipoor Saen, R. Developing a new data envelopment analysis methodology for supplier selection in the presence of both undesirable outputs and imprecise data. The International Journal of Advanced Manufacturing Technology , Vol. 51, No. 9-12, 2010, p. 1243–1250. doi: https://doi.org/10.1007/s00170-010-2694-3
-
[22]
Razmi, J. and H. Rafiei. An integrated analytic network process with mixed-integer non- linear programming to supplier selection and order allocation. The International Journal of Advanced Manufacturing Technology , Vol. 49, No. 9-12, 2009, p. 1195–1208. doi:https: //doi.org/10.1007/s00170-009-2445-5. 20
-
[23]
de Boer, L., E. Labro, and P. Morlacchi. A review of methods supporting supplier selection. European Journal of Purchasing & Supply Management , Vol. 7, No. 2, 2001, p. 75–89
work page 2001
-
[24]
Chai, J., J. N. Liu, and E. W. Ngai. Application of decision-making techniques in supplier selection: A systematic review of literature. Expert Systems with Applications, Vol. 40, No. 10, 2013, p. 3872–3885. doi:https://doi.org/10.1016/j.eswa.2012.12.040. URL https://www.scie ncedirect.com/science/article/pii/S095741741201281X
-
[25]
Taherdoost, H. and A. Brard. Analyzing the Process of Supplier Selection Criteria and Meth- ods. Procedia Manufacturing, Vol. 32, No. 32, 2019, p. 1024–1034. doi:https://doi.org/10. 1016/j.promfg.2019.02.317. URL https://www.sciencedirect.com/science/article/pii/ S2351978919303555
work page 2019
-
[26]
Samimi, A., A. Mohammadian, K. Kawamura, and Z. Pourabdollahi. An activity-based freight mode choice microsimulation model. Transportation Letters, Vol. 6, No. 3, 2014, p. 142–151. doi:https://doi.org/10.1179/1942787514y.0000000021
-
[27]
de Bok, M. and L. Tavasszy. An empirical agent-based simulation system for urban goods transport (MASS-GT). Procedia Computer Science , Vol. 130, 2018, p. 126–133. doi:https: //doi.org/10.1016/j.procs.2018.04.021
-
[28]
de Bok, M., L. Tavasszy, S. Thoen, L. Eggers, and I. Kourounioti. MASS-GT: An empirical model for the simulation of freight policies. Simulation Modelling Practice and Theory , Vol. 142, 2025, p. 103140. doi:https://doi.org/10.1016/j.simpat.2025.103140
-
[29]
Stinson, M. and A. K. Mohammadian. Introducing CRISTAL: A model of collaborative, informed, strategic trade agents with logistics. Transportation Research Interdisciplinary Per- spectives, Vol. 13, 2022, p. 100539. doi:https://doi.org/10.1016/j.trip.2022.100539
-
[30]
Spurlock, A., M. A. Bouzaghrane, A. Brooker, J. Caicedo, J. Gonder, J. Holden, K. Jeong, L. Jin, H. Laarabi, Z. Needell, C. Poliziani, S. Sharda, B. Sun, P. Waddell, Y. Wang, R. Waraich, T. Wenzel, and X. Xu. Behavior, Energy, Autonomy & Mobility Comprehensive Regional Evaluator: Overview, calibration and validation summary of an agent-based integrated re...
work page 2024
-
[31]
Auld, J., M. Hope, H. Ley, V. Sokolov, B. Xu, and K. Zhang. POLARIS: Agent-based modeling framework development and implementation for integrated travel demand and network and operations simulations. Transportation Research Part C: Emerging Technologies, Vol. 64, 2016, p. 101–116. doi:https://doi.org/10.1016/j.trc.2015.07.017
-
[32]
Zuniga-Garcia, N., A. Ismael, and M. Stinson. A freight asset choice model for agent-based simulation models. Procedia Computer Science, Vol. 220, 2023, p. 704–709. doi:https://doi.or g/10.1016/j.procs.2023.03.092
-
[33]
Ismael, A., N. Zuniga-Garcia, H.-S. Uhm, H. Shen, O. Sahin, J. Auld, and A. K. Mohamma- dian. Evaluating truck-to-rail mode shift for freight decarbonization in major U.S. transporta- tion hubs with varying urban forms. Procedia Computer Science, Vol. 257, 2025, p. 1008–1013. doi:https://doi.org/10.1016/j.procs.2025.03.130. 21
-
[34]
A Working Demonstration of a Mesoscale Model: Final Report and User’s Guide, 2011
Cambridge Systematics. A Working Demonstration of a Mesoscale Model: Final Report and User’s Guide, 2011
work page 2011
-
[35]
An Agent-based Freight Transportation Modeling Framework - ProQuest
Pourabdollahi, Z. An Agent-based Freight Transportation Modeling Framework - ProQuest . Dissertation, University of Illinois at Chicago, UIC, 2015. URL https://www.proquest.com /openview/42ba116c54e750281b883b73a7d48d58/1?cbl=18750&pq-origsite=gscholar
work page 2015
-
[36]
Bureau of Economic Analysis. Input-Output Accounts Data. Available at https://www.bea. gov/industry/input-output-accounts-data accessed on July 9, 2025, 2025
work page 2025
-
[37]
National Transportation Atlas Database
Bureau of Transportation Statistics. National Transportation Atlas Database. Available at https://geodata.bts.gov/ accessed on July 9, 2025, 2025
work page 2025
-
[38]
Holgu´ ın-Veras, J., L. K. Kalahasthi, A. Ismael, W. F. Yushimito, M. Herrera-Dappe, and M. S. Hoque. Integrated data collection and modeling with freight origin–destination synthesis: Application to Bangladesh. Case Studies on Transport Policy , Vol. 20, 2025, p. 101456. doi: https://doi.org/10.1016/j.cstp.2025.101456
-
[39]
Gurobi Optimizer Reference Manual
Gurobi Optimization, LLC. Gurobi Optimizer Reference Manual. Available at https://docs .gurobi.com/current/ accessed on July 9, 2025, 2025. The submitted manuscript has been created by UChicago Argonne, LLC, Operator of Argonne National Laboratory (“Argonne”). Argonne, a U.S. Department of Energy Office of Science laboratory, is operated under Contract No...
work page 2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.