{"id":"4fb952ad-2556-477b-ad67-897ae050507c","arxiv_id":"2501.18113","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":2,"one_line_summary":"Added and validated MFMA/MCE support in gem5 for AMD MI200 and MI300 GPUs, achieving 1.5% and 1.3% MAPE against real hardware.","lead":"The authors added Matrix Fused Multiply Add (MFMA) instruction support to the gem5 GPU simulator for AMD MI200 and MI300 GPUs. This lets researchers simulate modern machine learning workloads and test hypothetical matrix engine performance changes in a widely used open-source simulator.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation 1's fixed 44-cycle overhead may bias the measured MFMA latencies, leaving the 1.5%/1.3% MAPE validation potentially unsubstantiated.","rationale":"The reader's verdict is CONDITIONAL, largely because the validation rests on Equation 1's ability to extract clean per-instruction latencies from a serialized dependency chain. I agree that this is the weakest point, and I narrow it to the fixed overhead assumption. The paper's own admission that some tests needed cache-line padding is a red flag: it means the measurement window contains variable instruction-fetch effects that the fixed 44-cycle subtraction cannot capture. The low MAPE may be partly an artifact of the formula rather than evidence that gem5's MCE model is faithful to real hardware. This does not mean the contribution is worthless; adding MFMA support to gem5 is a useful engineering effort, and the ISA-manual cycle counts are likely correct. But the validation as presented does not convincingly establish the 1-2% fidelity claim. The proposed test is inexpensive: it uses the already-generated T_total data (or a small rerun) to check whether the overhead is in fact constant, and it uses gem5 itself as a ground-truth harness for Equation 1. If the overhead is constant, the concern is resolved. If not, the authors would need to either refine the formula or present a different validation (e.g., using hardware performance counters or independent MFMA throughput measurements). The verdict should remain CONDITIONAL, with the added condition that the measurement overhead be validated.","tokens_in":14806,"tokens_out":15476,"duration_ms":134580,"concrete_test":"For each microbenchmark, vary N_MFMA from 2 to 5 and fit T_total = A*(N-1) + B. Check whether B equals 44 for every instruction, especially those that required padding; if B differs, recompute T_MFMA using the empirical B and compare the resulting MAPE. Separately, run the identical kernel in gem5 with a known MFMA latency L and verify that the simulated T_total equals (N-1)*L + 44; if the simulation's own timing does not satisfy this identity, Equation 1 is not a faithful inversion of the measured quantity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The numerical claim of 1.5% and 1.3% MAPE rests on Equation 1, which computes T_MFMA = (T_total - T_memtime - T_inst)/(N_MFMA - 1) with T_memtime = 40 and T_inst = 4. The derivation assumes that the final MFMA's contribution to T_total is exactly T_memtime + T_inst, because the end s_memtime runs in the scalar pipeline and does not wait for the final MFMA (Section IV-C). This is a strong model of the measurement harness, not of the MCE. If the true overhead (time from the start s_memtime to the first MFMA issue, plus from the last MFMA issue to the end s_memtime's completion) deviates from 44 cycles, the extracted per-MFMA latency is systematically biased. The paper itself shows the overhead is not constant across instructions: several microbenchmarks (blue rows in Tables II-IV) required padding with s_nop instructions to prevent instruction cache misses inside the timed region (Sections V and VI). A single pair of constants from prior work is therefore unlikely to hold for all tests. If the real-hardware latencies are biased, the reported MAPE does not demonstrate that gem5's timing model is accurate; it only shows that gem5 and the biased measurement agree. The central claim of high-fidelity timing is thus not fully secured.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper adds Matrix Core Engine (MCE) support for AMD MI200 and MI300 GPUs to the gem5 simulator, including functional and timing models for MFMA instructions. The authors validate their implementation with handwritten assembly microbenchmarks that interleave back-to-back, dependent MFMA instructions with s_memtime timing instructions, running the same kernels on real GPUs and in gem5. They compute per-MFMA latency using Equation (1), which subtracts a fixed 44-cycle overhead (Tmemtime=40, Tinst=4) and divides by N_MFMA−1. The reported results show 1.5% MAPE for MI200 and 1.3% MAPE for MI300 relative to real hardware, with gem5 values also matching the 'Expected' latencies from the ISA manuals. The paper also introduces a --mfma-scale configuration parameter for what-if analysis of MFMA latency. The manuscript claims that these changes enable running state-of-the-art ML workloads in gem5, but the validation is limited to microbenchmarks.","tokens_in":15050,"tokens_out":6404,"duration_ms":62470,"significance":"The work fills a real gap in gem5's GPU support: MCEs are a central feature of modern AMD accelerators, and their absence prevented faithful simulation of ML workloads. The paper's strengths are its validation against real MI200 and MI300 hardware, its integration into gem5's mainline public codebase, and its introduction of a configurable latency scaling knob. The reported accuracy (1.5%/1.3% MAPE) is compelling for the tested instruction subset, and the alignment of real-hardware measurements with the ISA manual's expected latencies lends credibility. However, the paper's central motivational claim—that this support 'enables running state-of-the-art ML workloads'—is not backed by any end-to-end workload experiment, and the validation methodology's dependence on fixed overhead constants is not discussed. These issues temper the otherwise solid engineering contribution.","major_comments":[{"comment":"The abstract and introduction state that the changes 'enable running state-of-the-art ML workloads in gem5,' and Section IV-B asserts 'our MFMA support also works for these larger workloads,' yet no ML workload (e.g., PyTorch, TensorFlow, or any real DNN) is run or validated in the paper. The validation is entirely microbenchmark-based. Since the ability to run ML workloads is a primary stated motivation and a load-bearing claim of the contribution, the authors should either add an end-to-end demonstration (even a single representative model) or explicitly soften these claims to indicate that such support is expected but not yet validated.","section":"Abstract, Section I, Section IV-B"},{"comment":"The validation hinges on Equation (1), which assumes a fixed 44-cycle overhead (Tmemtime=40, Tinst=4) taken from prior work. The paper does not analyze how sensitive the extracted latencies are to these constants, nor does it justify that the overhead is identical for padded and unpadded instruction sequences (the blue rows in Tables II-V required extra s_nop padding to avoid I-cache misses). The observed agreement of real-hardware measurements with the 'Expected' columns in Tables II and IV partially mitigates this concern, because incorrect overhead constants would systematically shift the extracted values away from the manual's expected latencies. Nevertheless, a brief sensitivity analysis (e.g., varying Tmemtime/Tinst over a plausible range and reporting the resulting MAPE) or a direct per-test justification of the overhead would substantially strengthen the central accuracy claim.","section":"Section IV-C, Eq. (1), Tables II-V"}],"minor_comments":[{"comment":"The code snippet in Listing 1 appears to have formatting issues (e.g., incomplete operands in the asm lines). Please ensure the listing is complete and compilable as presented.","section":"Listing 1"},{"comment":"Table I uses 'LI Instruction Cache' and 'LI Scalar Cache'; the 'LI' is likely intended to be 'L1' for consistency with the other row labels.","section":"Table I"},{"comment":"The statement 'we provide accurate timing models' should be qualified as 'for the tested MFMA instructions,' since the validation does not cover all MFMA variants (e.g., those using s_set_gpr_idx are explicitly unsupported).","section":"Section V-A"},{"comment":"The paper says the gem5 changes are in 'mainline public support' but does not provide a commit hash, patch, or artifact link. Adding a specific revision or repository URL would improve reproducibility for researchers who want to use or extend the MFMA support.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"This is a useful systems/architecture contribution that will be of interest to the gem5 and GPU architecture community. The microbenchmark validation is solid for the tested instructions, but the paper currently overclaims the ML-workload capability without demonstration. The fixed-overhead issue in Eq. (1) is a real methodological risk, though the agreement with manual 'Expected' values gives some confidence. I recommend major revision; the authors should either add an end-to-end ML evaluation or clearly delimit the claim, and they should add a sensitivity discussion for Eq. (1)."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper is a genuine, useful engineering contribution: it adds MFMA/MCE support to gem5 for MI200 and MI300 GPU models, and it validates the timing against real hardware with microbenchmarks. The reported 1.5% and 1.3% MAPE for MI200/MI300 are believable for the tested instruction subset, and the implementation details (scoreboard-based MCE availability, per-instruction cycle tables from the ISA manual) are clearly described. The authors also deserve credit for an unusually honest limitations section: they explicitly note that MFMA instructions from the same wavefront are not pipelined in their model, that some MFMA classes are unsupported due to the s_set_gpr_idx addressing mode, and that --mfma-scale will not produce linear speedups in real workloads without compiler changes. That transparency raises my confidence in the parts they do claim.\n\nThe soft spots are real but not fatal. The measurement equation (Equation 1) subtracts a fixed 44-cycle overhead (Tmemtime+Tinst) inherited from prior work, and the stress-test worry that this could bias per-instruction latency is legitimate. However, the extracted real-hardware latencies match the ISA manual's expected values almost exactly, so any systematic bias in the measurement harness is either small or cancels between gem5 and hardware. The need for instruction-cache-line padding in several microbenchmarks does suggest the 44-cycle constant is not universal, but the impact on the MAPE numbers is likely minor. The bigger gap is scope: only six or seven MFMA instructions per GPU are validated, and despite the abstract's claim that \"state-of-the-art ML workloads\" can now run, no end-to-end ML results are presented. The what-if analysis is also limited to microbenchmarks, where the authors have full control over the instruction stream; the paper itself says real-workload scaling requires compiler changes. So the central timing model is well-supported, but the broader significance claims are not yet demonstrated.\n\nThis is a paper for the gem5 community and GPU architects who need MCE timing models for simulation-based studies. It is not a theoretical advance, but it is a reproducible, checkable contribution that fills a real gap. I would bring it to a reading group if anyone in our group does gem5 GPU work, and I would cite it if I needed MFMA timing parameters for MI-class GPUs.\n\nFor peer review: yes, send it to a serious venue. It deserves referee time. The right outcome is likely acceptance after the authors either add a real ML workload case study or explicitly restrict the claims to microbenchmark-validated instruction timing.","headline":"A solid, narrowly-scoped gem5 MFMA support paper with credible microbenchmark validation; the broad ML-workload claims outrun the evidence, but the core contribution deserves review.","tokens_in":15610,"tokens_out":2827,"would_cite":true,"duration_ms":30412,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper reports that adding Matrix Core Engine support to gem5 times AMD MI200 and MI300 MFMA instructions within 1.5% and 1.3% of real hardware, making cycle-level ML workload simulation practical.","keywords":["gem5","MFMA","Matrix Core Engine","GPU simulation","AMD MI200","AMD MI300","cycle-level simulation","machine learning workloads"],"falsifier":"Run the identical dependent-MFMA timing kernels on an MI200 or MI300 while replacing the s_memtime overhead constants with directly measured values, for example by timing a single s_memtime around a known number of s_nop instructions, and compare the resulting per-MFMA latencies; a systematic shift larger than the reported MAPE would indicate Equation 1's constants are wrong. Separately, launch two independent MFMA streams from different wavefronts to the same MCE and measure whether throughput exceeds one MFMA per latency, which would falsify the non-pipelining assumption.","tokens_in":14568,"feed_emoji":"🧮","tokens_out":8864,"duration_ms":79257,"temperature":0.7,"pith_summary":"This paper adds Matrix Core Engine (MCE) support to the gem5 simulator for AMD MI200 and MI300 GPUs, so that Matrix Fused Multiply Add (MFMA) instructions, the matrix-multiply workhorses of modern ML libraries, are timed at cycle level rather than treated as ordinary vector operations. The authors validate the new timing model with hand-written assembly microbenchmarks on real MI200 and MI300 hardware and report mean absolute percentage errors of 1.5% and 1.3%, respectively, across a range of precisions and block sizes. If the validation holds, gem5 can simulate modern ML workloads that rely on MFMA instructions with timing within about one to two percent of real hardware, and researchers can perturb MCE latency through a new configuration flag to explore future GPU designs.","feed_headline":"gem5 simulator now times AMD matrix math within 1.5 percent","feed_subtitle":"New Matrix Core Engine support lets ML workloads run in cycle-level simulation with 1.3% error on MI300.","key_machinery":"The load-bearing mechanism is a scoreboard-based MFMA issue rule in gem5's compute-unit timing logic, backed by a per-instruction latency lookup table (mfma_cycles) derived from the MI300 ISA manual and validated by Equation 1: T_MFMA = (T_total - T_memtime - T_inst)/(N_MFMA - 1), where T_memtime = 40 and T_inst = 4 are overhead constants from prior calibration. The scoreboard's NRDY_MATRIX_CORE field per SIMD unit decides when the MCE is free; since the scalar pipeline that runs s_memtime is independent of the MCE pipeline, dependent MFMA chains are needed to make the timing instruction wait for matrix completion.","core_discovery":"On the paper's own terms: gem5's GPU model now includes MCEs as separate functional units, one per SIMD unit (four per compute unit), and uses the NRDY_MATRIX_CORE scoreboard field to prevent more than one MFMA on the same SIMD unit at a time. This design matches AMD's reported MCE throughput and mirrors the compiler's apparent assumption that MFMA instructions from a given wavefront are not pipelined in the MCE. The validation method times back-to-back dependent MFMA instructions with s_memtime and subtracts measured overheads (T_memtime = 40 cycles, T_inst = 4 cycles) to recover per-instruction latency; across the tested instructions, gem5 matches the ISA manual's expected cycle counts and real hardware within 1.5% (MI200) and 1.3% (MI300) mean absolute percentage error. The paper also adds a --mfma-scale parameter that scales MFMA latency to support what-if analysis of future MCE designs.","pith_inferences":["Because the non-pipelining assumption is inferred from compiler-inserted NOPs rather than measured hardware behavior, the gem5 model may need revision if AMD's compiler or future hardware starts pipelining MFMA instructions from the same wavefront; a direct microbenchmark that varies the amount of independent work between dependent MFMAs would settle this.","The same interleaved s_memtime harness could serve as a portable benchmark for measuring MFMA latencies on future AMD GPUs and refreshing the mfma_cycles table, since it depends only on the ISA, not on gem5 internals.","The paper's limitation note implies that --mfma-scale results likely overstate how a real compiled workload would respond to faster or slower MCEs, because the compiler inserts NOPs and independent work based on the original latency; scaling studies therefore need compiler co-design to be realistic.","A similar scoreboard-plus-latency-table structure could be adapted to model NVIDIA TensorCore timing in gem5, although the validation harness would need different timing instructions than s_memtime."],"forward_implications":["Modern ML workloads that call MFMA-based libraries can now be run in gem5 with cycle-level timing rather than only functional simulation.","Researchers can model MCE improvements, faster or slower, by setting --mfma-scale, which multiplies the per-instruction MFMA latency and exposes how application runtime responds.","MI200 and MI300 differences, including new instructions, removed instructions, and changed latencies, are captured in the gem5 model, so cross-generation comparisons can be simulated.","Validation error decreases as more MFMA instructions are timed back-to-back, from 2.3% MAPE at 2 MFMAs to 0.4% at 5 on MI200, suggesting the timing methodology is limited by transient effects rather than by the core model."],"supporting_citations":[{"why":"The gem5 simulator reference; supplies the cycle-level simulation engine that the MFMA and MCE support extends.","marker":"[6]"},{"why":"Documents GPU wavefront scheduler behavior stalling on true data dependencies; the validation relies on this to make s_memtime wait for MFMA completion.","marker":"[8]"},{"why":"AMD's reported MCE operations per clock; used to set one MCE per SIMD unit and four per CU in the simulated MI200 and MI300 models.","marker":"[17]"},{"why":"AMD CDNA 3 architecture reference; establishes that MCEs are separate functional units that can run concurrently with other CU work.","marker":"[29]"},{"why":"AMD MI300 ISA manual; provides the expected MFMA cycle counts used as the timing target and the source for the mfma_cycles lookup table.","marker":"[30]"},{"why":"Prior gem5 GPU validation work; supplies the T_memtime = 40 and T_inst = 4 overhead constants used in Equation 1.","marker":"[35]"},{"why":"Prior gem5 GPU accuracy work; together with [37], used to tune the simulated GPU parameters to real hardware.","marker":"[36]"},{"why":"Further gem5 GPU accuracy calibration; underpins the baseline parameters in Table I used for validation.","marker":"[37]"},{"why":"AMD lab notes; the source of the microbenchmark structure that the paper's timing kernels are based on.","marker":"[40]"}],"fun_headline_variants":["gem5 now models AMD matrix cores within 1.5%","MFMA in gem5: 1.5% accuracy for AMD matrix math","gem5 adds AMD MFMA support with 1.5% timing error","Simulating ML on AMD GPUs: gem5 adds MFMA within 1.5%","gem5 gets Matrix Core Engines, times MFMA to 1.5% error"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the fixed timing overheads (T_memtime = 40 cycles, T_inst = 4 cycles) and the assumption that a wavefront's MFMA instructions never pipeline in the MCE correctly describe real MI200 and MI300 hardware; if either is wrong, the recovered per-instruction latencies, and therefore the claimed 1.5% and 1.3% accuracy, would be biased.","fun_headline_variants_meta":{"raw":{"variants":["gem5 now models AMD matrix cores within 1.5%","MFMA in gem5: 1.5% accuracy for AMD matrix math","gem5 adds AMD MFMA support with 1.5% timing error","Simulating ML on AMD GPUs: gem5 adds MFMA within 1.5%","gem5 gets Matrix Core Engines, times MFMA to 1.5% error"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000745,"raw_usage":{"total_tokens":3260,"prompt_tokens":821,"completion_tokens":2439,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":437,"completion_tokens_details":{"reasoning_tokens":2332}},"tokens_in":437,"tokens_out":2439,"duration_ms":16790,"temperature":1.0,"reasoning_tokens":2332,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T00:36:15.675980+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical dependent-MFMA timing kernels on an MI200 or MI300 while replacing the s_memtime overhead constants with directly measured values, for example by timing a single s_memtime around a known number of s_nop instructions, and compare the resulting per-MFMA latencies; a systematic shift larger than the reported MAPE would indicate Equation 1's constants are wrong. Separately, launch two independent MFMA streams from different wavefronts to the same MCE and measure whether throughput exceeds one MFMA per latency, which would falsify the non-pipelining assumption.","supporting_citations":[{"cited_title":"The gem5 simul ator,","cited_arxiv_id":null,"evidence_quote":"The gem5 simulator reference; supplies the cycle-level simulation engine that the MFMA and MCE support extends."},{"cited_title":"L ost in Abstraction: Pitfalls of Analyzing GPUs at the Intermediat e Language Level,","cited_arxiv_id":null,"evidence_quote":"Documents GPU wavefront scheduler behavior stalling on true data dependencies; the validation relies on this to make s_memtime wait for MFMA completion."},{"cited_title":"AMD CDNA™ 3 Architecture,","cited_arxiv_id":null,"evidence_quote":"AMD CDNA 3 architecture reference; establishes that MCEs are separate functional units that can run concurrently with other CU work."},{"cited_title":"[Online]","cited_arxiv_id":null,"evidence_quote":"AMD MI300 ISA manual; provides the expected MFMA cycle counts used as the timing target and the source for the mfma_cycles lookup table."},{"cited_title":"GAP: gem5 GPU Accuracy Profiler,","cited_arxiv_id":null,"evidence_quote":"Prior gem5 GPU validation work; supplies the T_memtime = 40 and T_inst = 4 overhead constants used in Equation 1."},{"cited_title":"Closing the Gap: Improving the Accuracy of gem5’s GPU Models,","cited_arxiv_id":null,"evidence_quote":"Prior gem5 GPU accuracy work; together with [37], used to tune the simulated GPU parameters to real hardware."},{"cited_title":"Furthe r Closing the GAP: Improving the Accuracy of gem5’s GPU Models,","cited_arxiv_id":null,"evidence_quote":"Further gem5 GPU accuracy calibration; underpins the baseline parameters in Table I used for validation."},{"cited_title":"AMD lab notes,","cited_arxiv_id":null,"evidence_quote":"AMD lab notes; the source of the microbenchmark structure that the paper's timing kernels are based on."}],"review_version":1}