Pith. sign in

REVIEW 4 major objections 5 minor 51 references

Pruning Large Language Models by Identifying and Preserving Functional Networks

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that LLM pruning by preserving ICA-discovered functional networks keeps performance closer to the original than importance-based pruning.

desk verdict Interesting idea, weak evidence: the ICA-based pruning masks aren't shown to beat simple baselines, and the SOTA claim doesn't survive the paper's own numbers. read the letter →

arxiv 2508.05239 v1 pith:2W6TGLZ6 submitted 2025-08-07 cs.CL cs.AIcs.LG

Yiheng Liu , Junhao Ning , Sichen Xia , Xiaohui Gao , Ning Qiang , Bao Ge , Junwei Han , Xintao Hu This is my paper
classification cs.CLcs.AIcs.LG
keywords structuredpruninglargelanguagemodelsfunctionalnetworksindependentcomponentanalysisCanICAMLPmodelcompressionbrain-inspiredAI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a large language model can be pruned more faithfully by treating its MLP neurons as though they were brain voxels and identifying 'functional networks'—groups of neurons that activate together—with independent component analysis (CanICA). Existing structured pruners score each neuron or structural unit and cut the low scorers, which the authors argue ignores how neurons collaborate; removing the wrong neurons can tear the model's macro functional architecture even when each removal looks harmless. The proposed pipeline collects gate- and up-projection activations on calibration text, z-scores them, decomposes each layer with CanICA into 128 components, thresholds the component maps into binary masks, and OR-aggregates masks across 3200 samples to decide which MLP neurons survive. On LLaMA2-7B-chat, Vicuna-7B-v1.5 and ChatGLM3-6B-base at 20–30% pruning, this selection matches or beats LLM-Pruner, FLAP, SliceGPT and Shortened LLaMA on most zero-shot tasks and gives lower Wikitext-2 perplexity in most settings. If the claim holds, co-activation groupings rather than individual importance scores are the right unit for structured pruning, which would make LLM compression less destructive and give a data-driven map of where model functions actually live.

What carries the argument

CanICA (canonical independent component analysis), a spatial ICA algorithm developed for fMRI analysis. The observed signals are z-scored outputs of the gate and up projections; the source signal matrix $S$ has one row per functional network and one column per neuron, and thresholding $S$ yields binary masks. The machinery's job is to convert raw co-activation statistics into a discrete pruning decision, and the OR-aggregation across runs turns unstable single-batch estimates into a global mask.

What would settle it

Prune LLaMA2-7B at 20% using the paper's pipeline but sweep the binarization threshold from the 10th to the 90th percentile of the source signal values while holding the retained neuron count fixed; if Wikitext-2 perplexity and zero-shot scores stay flat across thresholds, then the functional-network selection itself is not doing the work, and any mask of the same size would perform the same.

Watch

Extended reading notes

Core claim

The central discovery argued for is that LLMs contain spatially independent functional networks in their MLP hidden states, and that preserving a superset of neurons belonging to these networks—rather than pruning by per-neuron importance—keeps the model's knowledge and reasoning largely intact. Concretely: the outputs of the gate and up projections are z-scored per neuron, stacked into a signal matrix, and decomposed by CanICA into 128 components per layer per batch of 40 calibration samples. Each component row is thresholded to a binary mask; OR-aggregating masks over 3200 samples and multiple runs yields a global mask that selects which MLP neurons survive. The paper reports that at 20% a

Load-bearing premise

The load-bearing premise is that the numerical weights CanICA assigns to neurons can be turned into a yes/no list of which neurons to keep; the paper never reports the cutoff used to make that turn and never checks whether the weights, rather than just the number of kept neurons, cause the gains.

Editorial extensions

If this is right

  • At 20% pruning, the CanICA mask beats the compared importance-based pruners on most of the seven evaluation tasks for both Vicuna-7B and LLaMA2-7B, and it lowers Wikitext-2 perplexity relative to FLAP and LLM-Pruner in nearly all settings.
  • At 30% pruning the method still holds the best or near-best scores on PIQA and WinoGrande and keeps perplexity far below SliceGPT and Shortened LLaMA, showing that the grouping criterion degrades more gracefully than block- or PCA-based pruning.
  • The number of ICA components is a real hyperparameter: 128 or 256 components produce the lowest perplexity, while 512 components degrade performance, so the method has a working range rather than a single magic setting.
  • Calibration data need scales: perplexity falls as samples increase from 40 to roughly 1500 and then plateaus or worsens, meaning the method has an optimal data budget rather than 'more is always better'.
  • Because the procedure runs layer-by-layer on MLP activations only, it leaves attention and embedding structure untouched, so it can be combined orthogonally with depth pruning methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A global, cross-layer version of the same idea is the natural next step the authors flag: since information flows through Transformer blocks, functional networks probably span layers, and preserving those might cut performance loss further.
  • The thresholding step is underdetermined in the paper: the threshold on the source signal matrix is never reported or ablated. Until a threshold sweep shows that mask content, not just neuron count, drives the gains, the method's stated mechanism is not fully pinned down.
  • The same networks could double as an interpretability atlas: labeling each ICA component with a larger model's explanations would test whether these functional networks correspond to coherent behaviors, and would give pruning a semantic rather than purely statistical justification.
  • A nonlinear decomposition (the authors mention autoencoders) on the same activation matrices would show whether the linear ICA assumption hides important neuron collaborations; if nonlinear networks find different essential sets, the linearity of CanICA is a real limitation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a structured pruning method for LLMs based on identifying 'functional networks' with canonical independent component analysis (CanICA), applied layer-by-layer to MLP neuron activations. The source signal matrix from CanICA is thresholded to obtain binary masks, OR-aggregated across 80 runs of 40 calibration samples each, and used to preserve 'key neurons' before pruning. Experiments on LLaMA2-7B-chat, Vicuna-7B-v1.5, and ChatGLM3-6B-base report perplexity and zero-shot accuracy at 20% and 30% pruning rates, with comparisons to LLM-Pruner, SliceGPT, FLAP, and Shortened LLaMA. The abstract claims state-of-the-art performance and successful identification of functional networks.

Significance. The cross-disciplinary idea—adapting spatial ICA from neuroimaging to discover neuron groups in LLMs and using those groups for pruning—is novel and potentially useful. If properly validated, it would offer a new grouping criterion for structured pruning and contribute to mechanistic interpretability. The paper provides a public code repository, evaluates on three model families, and reports calibration-sample-size sensitivity. However, the current evidence does not establish the central mechanism: the mask-generation threshold is undisclosed, no comparison against random or magnitude-based masks is provided, and the state-of-the-art claim is not supported at 30% pruning. These gaps are addressable but require substantial additional analysis.

major comments (4)
  1. [Identify and Preserve Functional Networks] The method relies on thresholding the CanICA source signal matrix to obtain binary masks, but the threshold is never reported, nor is the procedure for enforcing the target pruning ratio (e.g., per-layer percentile vs. global threshold). Without this information, the pruning decision is under-specified and the experiments are not reproducible. Since the threshold is the only link between ICA components and the set of preserved neurons, this is a load-bearing omission.
  2. [Results (Tables 4-5)] The abstract and conclusion claim state-of-the-art performance, but the 30% pruning results contradict this. In Table 5 (LLaMA2-7B, 30%), FLAP outperforms CanICA on all seven reported metrics; in Table 4 (Vicuna-7B, 30%), FLAP wins four metrics, including Wikitext-2 perplexity (21.4575 vs. 25.9746 for CanICA without bias). The SOTA claim is therefore not supported by the full set of results.
  3. [Impact of Hyperparameters] The hyperparameter n_components is selected by evaluating post-pruning perplexity on Wikitext-2 (the same dataset used for the main evaluation). The reported table shows n_components=256 gives lower perplexity (18.4713) than the chosen 128 (18.9421), yet 128 is used. This is test-set fitting, and the model selection is not independent. Additionally, the paper states n_components is the only hyperparameter, ignoring the mask threshold and any parameters governing the z-score preprocessing or the group aggregation; these are also hyperparameters that affect the mask.
  4. [Method / Evaluation] No control experiments isolate the contribution of the ICA-derived functional networks. A random mask of the same size, a mask based on raw z-scored activation magnitudes, or a mask with component labels permuted would be necessary to show that the identified functional networks, rather than any broad subset or a simple importance score, are responsible for the observed performance. Without such ablations, the results are consistent with the hypothesis that neuron selection via ICA masking is not causally meaningful.
minor comments (5)
  1. [Impact of Hyperparameters] The table of n_components results is referenced as 'Table??' in the text; the reference is unresolved.
  2. [Author affiliation] 'School of Pyhisic' should be 'School of Physics' (typo).
  3. [Abstract] Grammar: 'Most of them overlooks' should be 'Most of them overlook'.
  4. [Comparison Results (ChatGLM3)] Table 3 reports CanICA results on ChatGLM3 with no comparison baselines. The sentence 'the proposed method achieves the best performance in several tasks' is meaningless without competitors; please clarify that these are standalone results.
  5. [Discussion] The limitations paragraph acknowledges that ICA is linear, but does not mention the missing threshold or the absence of random-baseline controls, which are the main threats to the central claim. A revised discussion should address these.

Circularity Check

1 steps flagged · score 4.0 of 10

n_components is tuned on Wikitext-2 perplexity, and Wikitext-2 perplexity is then reported as an evaluation metric; the rest of the ICA pipeline is not circular.

  1. fitted input called prediction [Impact of Hyperparameters (after Comparison Results, before Impact of Number of Calibration Samples; the table is unnumbered and printed as 'Table??')]
    "To investigate the most suitable setting for the n components parameter, we conduct a simple analysis on a small sample of calibration data. We use the neural signals obtained from 40 input samples on the Vicuna-7B-v1.5 model and decompose these signals into 10, 20, 64, 128, 256, and 512 functional networks, respectively. Then, model pruning is performed based on these functional networks, and the pruned models are evaluated in terms of perplexity on the Wikitext2 dataset. ... Based on multiple experiments, we set the number of independent components in CanICA as 128 in this study."

    The only hyperparameter of the method, n_components, is selected by measuring the pruned model's perplexity on Wikitext-2, and the same Wikitext-2 perplexity is then reported in Tables 1–5 as one of the benchmark metrics supporting the state-of-the-art claim. For Vicuna-7B-v1.5, the model on which this selection was run, the reported Wikitext-2 value is therefore not an independent test: the configuration was chosen using that exact evaluation metric. The paper also uses Wikitext-2 as the calibration dataset for pruning, so the Wikitext-2 result is doubly in-sample. The selection is not maximally overfit—the table shows 256 components give lower perplexity than the chosen 128—but the metric used for selection and the metric used for evaluation are the same, making that column partly fitted

full rationale

The paper's derivation chain is mostly self-contained: CanICA is applied to z-scored MLP activations, the source matrix is thresholded to obtain masks, and OR-aggregation yields the global pruning mask. Those steps do not define the functional networks in terms of the final benchmark scores, so the core ICA mechanism is not circular. The self-citation to Liu et al. 2025a is motivational ('Inspired by ...') and the analogy is explicitly labeled a hypothesis, so it is not load-bearing in the sense of forcing the numerical results. There is no imported uniqueness theorem, no ansatz smuggled in via citation, and no renaming of a known result. The one concrete circular element is the choice of n_components: the only hyperparameter is selected by evaluating pruned-model perplexity on Wikitext-2, and Wikitext-2 perplexity is then reported in the main results tables as one of the metrics supporting the SOTA claim. For the model used in that selection (Vicuna-7B-v1.5), the Wikitext-2 column is partly a model-selection artifact rather than an independent prediction. The selection table itself shows 256 gives lower perplexity than 128, so the authors did not simply pick the minimum; this reduces the severity. Other benchmarks (PIQA, HellaSwag, ARC, OBQA, WinoGrande) and the LLaMA2/ChatGLM3 results were not used in hyperparameter selection and remain independent evidence, so the central claim still has independent content. The undisclosed threshold and the absence of random-mask or permutation controls are serious evidential gaps, but they are not circularity under the definitions used here: they concern causal attribution and reproducibility, not reduction of the output to the input. Overall score 4.

Assumptions & free parameters 3 free parameters · 3 assumptions · 1 invented entities

The method depends on several choices not derived from theory: the ICA component count, the mask threshold, and the calibration sample size. The central assumption that linear ICA on activations reveals functional groups whose preservation is optimal for pruning is asserted, not proven.

free parameters (3)
  • n_components = 128
    Empirically chosen based on Wikitext-2 perplexity; the table shows 256 achieves lower perplexity (18.4713) than 128 (18.9421), yet 128 is selected without justification.
  • mask_threshold = not reported
    Threshold applied to CanICA source signals to produce binary masks; critical to the pruning outcome, but no value or selection procedure is given.
  • calibration_sample_count = 3200
    3200 Wikitext-2 samples are used, though Figure 5 shows the best perplexity at about 1500 samples, so the choice is not tied to the reported optimum.
assumptions (3)
  • domain assumption LLM neuron activations can be treated as voxel-like BOLD signals for ICA decomposition.
    Stated in Preliminaries and Method; not empirically validated, and the paper itself acknowledges ICA is linear and may miss non-linear dependencies.
  • domain assumption Linear ICA components correspond to functional networks whose preservation maintains model performance.
    The pruning decision is based on these components, but no causal evidence links them to model function beyond the aggregate pruning accuracy.
  • ad hoc to paper OR-aggregation of per-group masks identifies the complete set of critical neurons.
    The OR operation is introduced in Figure 3 without analysis; it maximizes recall but may include redundant, unimportant neurons, and the union may not respect the desired pruning rate.
invented entities (1)
  • functional networks in LLM neurons
    purpose: Groups of neurons treated as cohesive functional units to define which neurons to preserve during pruning.
    The networks are extracted from calibration data with ICA, and their relevance is only evidenced by the pruning results used to validate them; there is no external falsifiable handle, e.g., an ablation predicting a specific behavior.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Pruning Large Language Models by Identifying and Preserving Functional Networks." pith.science (2026). https://pith.science/paper/2W6TGLZ6

@misc{pith2026250805239,
  author       = {Pith},
  title        = {Pith review of: Pruning Large Language Models by Identifying and Preserving Functional Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2W6TGLZ6}},
  note         = {Machine review of arXiv:2508.05239}
}
read the original abstract

Structured pruning is one of the representative techniques for compressing large language models (LLMs) to reduce GPU memory consumption and accelerate inference speed. It offers significant practical value in improving the efficiency of LLMs in real-world applications. Current structured pruning methods typically rely on assessment of the importance of the structure units and pruning the units with less importance. Most of them overlooks the interaction and collaboration among artificial neurons that are crucial for the functionalities of LLMs, leading to a disruption in the macro functional architecture of LLMs and consequently a pruning performance degradation. Inspired by the inherent similarities between artificial neural networks and functional neural networks in the human brain, we alleviate this challenge and propose to prune LLMs by identifying and preserving functional networks within LLMs in this study. To achieve this, we treat an LLM as a digital brain and decompose the LLM into functional networks, analogous to identifying functional brain networks in neuroimaging data. Afterwards, an LLM is pruned by preserving the key neurons within these functional networks. Experimental results demonstrate that the proposed method can successfully identify and locate functional networks and key neurons in LLMs, enabling efficient model pruning. Our code is available at https://github.com/WhatAboutMyStar/LLM_ACTIVATION.

Figures

Figures reproduced from arXiv: 2508.05239 by the authors.

Figure 1
Figure 1. The pipeline of identifying functional networks [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. The process of aggregating the mask matrices of [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. The perplexity results on LLaMA2-7B-chat-hf (left) and Vicuna-7B-v1.5 (right) with different pruning rates. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: The impact of the number of calibration samples [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

51 extracted references · 33 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    An, Y.; Zhao, X.; Yu, T.; Tang, M.; and Wang, J. 2024. Fluctuation-based adaptive structured pruning for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 10865--10873

    work page 2024

  4. [4]

    L.; Nascimento, M

    Ashkboos, S.; Croci, M. L.; Nascimento, M. G. d.; Hoefler, T.; and Hensman, J. 2024. Slicegpt: Compress large language models by deleting rows and columns. arXiv preprint arXiv:2401.15024

    arXiv 2024

  5. [5]

    Bai, G.; Li, Y.; Ling, C.; Kim, K.; and Zhao, L. 2024. Sparse LLM : Towards Global Pruning of Pre-trained Language Models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    work page 2024

  6. [6]

    Bills, S.; Cammarata, N.; Mossing, D.; Tillman, H.; Gao, L.; Goh, G.; Sutskever, I.; Leike, J.; Wu, J.; and Saunders, W. 2023. Language models can explain neurons in language models. https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html

    2023

  7. [7]

    Bisk, Y.; Zellers, R.; Gao, J.; Choi, Y.; et al. 2020. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, 7432--7439

    work page 2020

  8. [8]

    Bullmore, E.; and Sporns, O. 2009. Complex brain networks: graph theoretical analysis of structural and functional systems. Nature reviews neuroscience, 10(3): 186--198

    work page 2009

Show all 51 references
  1. [9]

    Chen, X.; Hu, Y.; Zhang, J.; Wang, Y.; Li, C.; and Chen, H. 2024. Streamlining Redundant Layers to Compress Large Language Models. In The Thirteenth International Conference on Learning Representations

    2024

  2. [10]

    Cheng, H.; Zhang, M.; and Shi, J. Q. 2024. A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations. IEEE Transactions on Pattern Analysis and Machine Intelligence

    2024

  3. [11]

    E.; et al

    Chiang, W.-L.; Li, Z.; Lin, Z.; Sheng, Y.; Wu, Z.; Zhang, H.; Zheng, L.; Zhuang, S.; Zhuang, Y.; Gonzalez, J. E.; et al. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90\ See https://vicuna. lmsys. org (accessed 14 April 2023), 2(3): 6

    2023

  4. [12]

    Clark, P.; Cowhey, I.; Etzioni, O.; Khot, T.; Sabharwal, A.; Schoenick, C.; and Tafjord, O. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457

    2018 arXiv

  5. [13]

    Frantar, E.; and Alistarh, D. 2023. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning, 10323--10337. PMLR

    2023

  6. [14]

    Gao, L.; Tow, J.; Abbasi, B.; Biderman, S.; Black, S.; DiPofi, A.; Foster, C.; Golding, L.; Hsu, J.; Le Noac'h, A.; Li, H.; McDonell, K.; Muennighoff, N.; Ociepa, C.; Phang, J.; Reynolds, L.; Schoelkopf, H.; Skowron, A.; Sutawika, L.; Tang, E.; Thite, A.; Wang, B.; Wang, K.; a...

    2024

  7. [15]

    GLM, T.; Zeng, A.; Xu, B.; Wang, B.; Zhang, C.; Yin, D.; Zhang, D.; Rojas, D.; Feng, G.; Zhao, H.; et al. 2024. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793

    2024 arXiv

  8. [16]

    Gromov, A.; Tirumala, K.; Shapourian, H.; Glorioso, P.; and Roberts, D. 2025. The Unreasonable Ineffectiveness of the Deeper Layers. In The Thirteenth International Conference on Learning Representations

    2025

  9. [17]

    Han, S.; Mao, H.; and Dally, W. J. 2016. Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding. In International Conference on Learning Representations

    2016

  10. [18]

    Han, S.; Pool, J.; Tran, J.; and Dally, W. 2015. Learning both weights and connections for efficient neural network. Advances in neural information processing systems, 28

    2015

  11. [19]

    G.; and Wolff, G

    Hassibi, B.; Stork, D. G.; and Wolff, G. J. 1993. Optimal brain surgeon and general network pruning. In IEEE international conference on neural networks, 293--299. IEEE

    1993

  12. [20]

    He, M.; Hou, X.; Ge, E.; Wang, Z.; Kang, Z.; Qiang, N.; Zhang, X.; and Ge, B. 2023. Multi-head attention-based masked sequence model for mapping functional brain networks. Frontiers in Neuroscience, 17: 1183145

    2023

  13. [21]

    Kim, B.-K.; Kim, G.; Kim, T.-H.; Castells, T.; Choi, S.; Shin, J.; and Song, H.-K. 2024 a . Shortened LLaMA: A Simple Depth Pruning for Large Language Models. ICLR Workshop on Mathematical and Empirical Understanding of Foundation Models (ME-FoMo)

    2024

  14. [22]

    Kim, B.-K.; Kim, G.; Kim, T.-H.; Castells, T.; Choi, S.; Shin, J.; and Song, H.-K. 2024 b . Shortened LLaMA: Depth Pruning for Large Language Models with Comparison of Retraining Methods. arXiv preprint arXiv:2402.02834

    2024 arXiv

  15. [23]

    LeCun, Y.; Denker, J.; and Solla, S. 1989. Optimal brain damage. Advances in neural information processing systems, 2

    1989

  16. [24]

    Liu, Y.; Gao, X.; Sun, H.; Ge, B.; Liu, T.; Han, J.; and Hu, X. 2025 a . Brain-Inspired Exploration of Functional Networks and Key Neurons in Large Language Models. arXiv preprint arXiv:2502.20408

    2025

  17. [25]

    Liu, Y.; Ge, E.; He, M.; Liu, Z.; Zhao, S.; Hu, X.; Qiang, N.; Zhu, D.; Liu, T.; and Ge, B. 2024 a . Mapping dynamic spatial patterns of brain function with spatial-wise attention. Journal of Neural Engineering, 21(2): 026005

    2024

  18. [26]

    Liu, Y.; Ge, E.; Kang, Z.; Qiang, N.; Liu, T.; and Ge, B. 2024 b . Spatial-temporal convolutional attention for discovering and characterizing functional brain networks in task fMRI. NeuroImage, 287: 120519

    2024

  19. [27]

    Liu, Y.; Ge, E.; Qiang, N.; Liu, T.; and Ge, B. 2023 a . Spatial-temporal convolutional attention for mapping functional brain networks. In 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), 1--4. IEEE

    2023

  20. [28]

    Liu, Y.; Han, T.; Ma, S.; Zhang, J.; Yang, Y.; Tian, J.; He, H.; Li, A.; He, M.; Liu, Z.; et al. 2023 b . Summary of chatgpt-related research and perspective towards the future of large language models. Meta-Radiology, 100017

    2023

  21. [29]

    Liu, Y.; He, H.; Han, T.; Zhang, X.; Liu, M.; Tian, J.; Zhang, Y.; Wang, J.; Gao, X.; Zhong, T.; Pan, Y.; Xu, S.; Wu, Z.; Liu, Z.; Zhang, X.; Zhang, S.; Hu, X.; Zhang, T.; Qiang, N.; Liu, T.; and Ge, B. 2025 b . Understanding LLMs: A comprehensive overview from training to inf...

    2025

  22. [30]

    Ma, X.; Fang, G.; and Wang, X. 2023. Llm-pruner: On the structural pruning of large language models. Advances in neural information processing systems, 36: 21702--21720

    2023

  23. [31]

    Men, X.; Xu, M.; Zhang, Q.; Wang, B.; Lin, H.; Lu, Y.; Han, X.; and Chen, W. 2024. Shortgpt: Layers in large language models are more redundant than you expect. arXiv preprint arXiv:2403.03853

    2024 arXiv

  24. [32]

    Merity, S.; Xiong, C.; Bradbury, J.; and Socher, R. 2016. Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843

    2016 arXiv

  25. [33]

    Mihaylov, T.; Clark, P.; Khot, T.; and Sabharwal, A. 2018. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789

    2018 arXiv

  26. [34]

    Molchanov, P.; Tyree, S.; Karras, T.; Aila, T.; and Kautz, J. 2017. Pruning Convolutional Neural Networks for Resource Efficient Inference. In International Conference on Learning Representations

    2017

  27. [35]

    Qiang, N.; Dong, Q.; Huang, H.; Wang, H.; Zhao, S.; Hu, X.; Li, Q.; Zhang, W.; Liu, Y.; He, M.; et al. 2024. Deep learning in functional brain mapping and associated applications. In Deep Learning for Medical Image Analysis, 395--423. Elsevier

    2024

  28. [36]

    Qiang, N.; Dong, Q.; Liang, H.; Li, J.; Zhang, S.; Zhang, C.; Ge, B.; Sun, Y.; Gao, J.; Liu, T.; et al. 2022. Learning brain representation using recurrent Wasserstein generative adversarial net. Computer Methods and Programs in Biomedicine, 223: 106979

    2022

  29. [37]

    Raichle, M. E. 2015. The brain's default mode network. Annual review of neuroscience, 38(1): 433--447

    2015

  30. [38]

    L.; Bhagavatula, C.; and Choi, Y

    Sakaguchi, K.; Bras, R. L.; Bhagavatula, C.; and Choi, Y. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9): 99--106

    2021

  31. [39]

    M.; Fox, P

    Smith, S. M.; Fox, P. T.; Miller, K. L.; Glahn, D. C.; Fox, P. M.; Mackay, C. E.; Filippini, N.; Watkins, K. E.; Toro, R.; Laird, A. R.; et al. 2009. Correspondence of the brain's functional architecture during activation and rest. Proceedings of the national academy of scienc...

    2009

  32. [40]

    Song, J.; Oh, K.; Kim, T.; Kim, H.; Kim, Y.; and Kim, J.-J. 2024. SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks. In International Conference on Machine Learning, 46136--46155. PMLR

    2024

  33. [41]

    Sun, M.; Liu, Z.; Bair, A.; and Kolter, J. Z. 2023. A simple and effective pruning approach for large language models. arXiv preprint arXiv:2306.11695

    2023 arXiv

  34. [42]

    M.; Pinsk, M

    Szczepanski, S. M.; Pinsk, M. A.; Douglas, M. M.; Kastner, S.; and Saalmann, Y. B. 2013. Functional and structural architecture of the human dorsal frontoparietal attention network. Proceedings of the National Academy of Sciences, 110(39): 15806--15811

    2013

  35. [43]

    Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozi \`e re, B.; Goyal, N.; Hambro, E.; Azhar, F.; et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  36. [44]

    Varoquaux, G.; Keller, M.; Poline, J.-B.; Ciuciu, P.; and Thirion, B. 2010 a . ICA-based sparse features recovery from fMRI datasets. In 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 1177--1180. IEEE

    2010

  37. [45]

    Varoquaux, G.; Sadaghiani, S.; Pinel, P.; Kleinschmidt, A.; Poline, J.-B.; and Thirion, B. 2010 b . A group model for stable multi-subject ICA on fMRI datasets. Neuroimage, 51(1): 288--299

    2010

  38. [46]

    Wang, J.; Shi, E.; Hu, H.; Ma, C.; Liu, Y.; Wang, X.; Yao, Y.; Liu, X.; Ge, B.; and Zhang, S. 2024. Large language models for robotics: Opportunities, challenges, and perspectives. Journal of Automation and Intelligence

    2024

  39. [47]

    Xia, M.; Zhong, Z.; and Chen, D. 2022. Structured Pruning Learns Compact and Accurate Models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1513--1528

    2022

  40. [48]

    Zafrir, O.; Larey, A.; Boudoukh, G.; Shen, H.; and Wasserblat, M. 2021. Prune once for all: Sparse pre-trained language models. arXiv preprint arXiv:2111.05754

    2021 arXiv

  41. [49]

    Zellers, R.; Holtzman, A.; Bisk, Y.; Farhadi, A.; and Choi, Y. 2019. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830

    2019 arXiv

  42. [50]

    X.; Zhou, K.; Li, J.; Tang, T.; Wang, X.; Hou, Y.; Min, Y.; Zhang, B.; Zhang, J.; Dong, Z.; et al

    Zhao, W. X.; Zhou, K.; Li, J.; Tang, T.; Wang, X.; Hou, Y.; Min, Y.; Zhang, B.; Zhang, J.; Dong, Z.; et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223

    2023 arXiv

  43. [51]

    Zhou, A.; Ma, Y.; Zhu, J.; Liu, J.; Zhang, Z.; Yuan, K.; Sun, W.; and Li, H. 2021. Learning N: M Fine-grained Structured Sparse Neural Networks From Scratch. In International Conference on Learning Representations

    2021

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.