Pith.

Computer Vision and Pattern Recognition · arXiv:2611.03125 v1 · submitted 2 Nov 2026 · 64 references

SurgPace: Complexity-Aware Progressive Distillation for Surgical Multimodal Large Language Models

Zhicheng He, Xiaohan Xing, Hongqiu Wang, Kun Yuan, Guankun Wang, Yanheng Li, Haoxuan Che, Nassir Navab

show 2 more authors

Hongliang Ren, Long Bai

The paper says

“Multimodal large language models (MLLMs) have shown strong potential in robotic surgery for question answering, scene understanding, and grounded assistance, but their high computational cost limits real clinical deployment.”

more of the abstract

Knowledge distillation provides a practical way to build compact surgical MLLMs, yet direct distillation is not well suited to surgical data because surgical samples vary greatly in difficulty, visual ambiguity, and required clinical context, making uniform teacher-student transfer unstable. In addition, the role of supervision changes during training: early stages require stronger teacher guidance for multimodal alignment, while later stages rely more on task supervision for effective optimization. To address these challenges, we propose SurgPace, a complexity-aware continuous curriculum framework for progressive distillation of surgical vision-language models. SurgPace estimates sample complexity using predictive confidence and Lorentz geometric consistency, and uses the fused score to guide a continuous easy-to-hard training process. We further introduce hierarchical representation alignment, which aligns the student with the teacher at the feature, logit, and probability levels, while gradually shifting the training focus from teacher imitation to task supervision. This design separates sample-level curriculum from objective-level scheduling, leading to more stable and effective distillation. We evaluate SurgPace on Surg-396K, a benchmark built from four public surgical datasets and covering five dialogue paradigms. SurgPace consistently outperforms strong MLLM distillation baselines, demonstrating its effectiveness for building compact, memory-efficient surgical MLLMs with faster decoding under controlled benchmarks. Code is available at https://github.com/hezhicheng2002/SurgPace.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 3 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A coherent distillation recipe and a real evaluation flaw: the headline numbers come from selecting the configuration on the same test split used to report them, so the central claim needs a validation-based rerun before it can be believed. This line is wrong

3 major objections · 5 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 64 references say.

The paper says, at §4.4 (Tables 11-16); §4.1-4.2

The review did not quote this passage.

Pith says: Objection 1 of 3 · major

Configuration selection is performed on the same test split that produces the headline number. Every ablation table in §4.4 reports metrics on the fixed 64,856-example test split of §4.1, and the row adopted as the final configuration is in each case the best row under seed 42 on that split (s_min in Table 11, easy-to-hard in Table 12, linear in Table 13, full in Table 14, continuous in Table 15, full alignment in Table 16).

more of objection 1

The 64,878-example validation split is never used for selection. The headline 0.6955 and the 'best on 15 of 16 metrics' claim are therefore the maximum of a selection loop over 17 distinct single-seed configurations evaluated on that test split, not an independent estimate, and the expected optimism of such selection is of the same order as the reported across-seed variability (0.0038, Table 4), which is also the scale of several ablation differences (e.g., 0.0044, Table 14). Note that Table 4 repeats the already-selected configuration over seeds and therefore measures the stability of that configuration, not the bias of the selection procedure. The required corrective step is to select the configuration using the unused validation split and report the test numbers of the validation-selected configuration, or at minimum to show that the validation ranking of the ablation rows reproduces the adopted choices.

The paper says, at §4.4, Table 14; §4.3, Table 4

The review did not quote this passage.

Pith says: Objection 2 of 3 · major

The evidence that the complexity-aware curriculum itself improves distillation is within run-to-run noise. Removing the entire sample-level curriculum changes the ten-metric ablation mean from 0.6585 to 0.6541 (+0.0044, Table 14), while the only reported across-seed variability for an aggregate, the 16-metric mean, is ±0.0038 (Table 4).

more of objection 2

Several individual metric movements are below one seed standard deviation (e.g., Grounding QA AP50 0.9625 vs 0.9619 with a seed std of 0.0031; Single Phrase QA Acc 0.9004 vs 0.8928 with a seed std of 0.0101). Because the curriculum is the paper's stated primary contribution, the on/off comparison should be repeated over multiple seeds and reported with the same uncertainty convention as Table 4; with the current evidence, the curriculum's contribution and the selection noise are statistically indistinguishable.

The paper says, at §3.2, Eqs. (1)-(10); §4.3, Table 2

The review did not quote this passage.

Pith says: Objection 3 of 3 · major

The difficulty signal may partly measure teacher format agreement rather than content difficulty. Eq. (1) averages teacher top-1 probability on the gold token sequence under teacher-forcing, but the teacher's free-form answers differ in format from the gold for some paradigms (Table 2: EndoChat's Single Phrase QA accuracy is 0.5485 under the common protocol, which the paper attributes to answer-format sensitivity).

more of objection 3

For such samples, low teacher confidence on the forced gold tokens may reflect answer-style mismatch rather than content difficulty. The permissive min rule of Eq. (10) is a load-bearing modeling choice whose evidence is a small, test-selected gain (Table 11: s_min vs s_mean, 0.6585 vs 0.6567). Please provide a diagnostic that the scores track learnability rather than teacher style, for example the distribution of s_i per task family, or the correlation of s_i with per-sample student loss or error.

5 minor objections

The paper says, at §4.3, Tables 2-3

Pith says: minor objection 1 of 5

The 'best on 15 of 16 metrics' statement rests on single-seed runs for five of the six baselines; only EfficientVLM has repeated-seed numbers (Table 4). Please add seeds for at least the closest competing baseline or qualify the claim near the sentence that makes it.

The paper says, at §5, Limitations

Pith says: minor objection 2 of 5

The limitations list (i)-(iv) does not acknowledge that the final configuration was selected on the test split, nor that the evaluation uses the authors' own teacher (EndoChat) and benchmark (Surg-396K); both facts should be stated so readers can calibrate the component-level claims.

show 3 more

The paper says, at §2.2

Pith says: minor objection 3 of 5

The method name 'LLA V ADI' has broken spacing, and the 'LLaV A-...' naming is inconsistent throughout the text (e.g., 'LLaV A-KD', 'LLaV A-MoD', 'LLaV A-DI'); please standardize.

The paper says, at §3.2, Eqs. (2) and (6)

Pith says: minor objection 4 of 5

Please clarify whether the same epsilon is used in the logit normalization of Eq. (2) and in the arcosh clamp of Eq. (6), and state its numerical value for reproducibility.

The paper says, at §4.4, Tables 11-16

Pith says: minor objection 5 of 5

The ablation means are over ten metrics with heterogeneous scales and are reported for seed 42 only; a per-seed variance for this ten-metric mean would let the reader compare the ablation gaps (0.0018-0.0081) with run-to-run noise directly.

What would settle it

Take the already-trained variants behind Tables 11-16 (or retrain them), compute their ten-metric means on the 64,878 validation instances, choose the best configuration by validation only, then evaluate that single frozen configuration on the 64,856 test instances once. If the validation-selected model's 16-metric mean falls materially below 0.6955 or no longer exceeds EfficientVLM's corresponding number, the reported advantage is selection-driven; in any case, report how many configurations were evaluated on test and the omitted schedule constants (r_end, w_min, w_max, phase ratios, and beta endpoints).

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 3 Nov 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic partly agrees 1 min 22 s · 29,859 tokens read, 12,689 written
  3. Editor wrote the verdict above 15 s · 30,814 tokens read, 2,000 written
  4. Referee major revision: 3 major objections, 5 minor 1 min 44 s · 30,238 tokens read, 18,450 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Quick take: SurgPace is a well-built distillation framework, and the compressed 3B student results are plausible. But the paper's central empirical claim is not yet trustworthy, because the final configuration was chosen from ablation means computed on the fixed test split, and that same split is used for the headline 0.6955 average and the "best on 15 of 16 metrics" statement.

The genuinely new parts are worth naming. Separating sample-level curriculum from objective-level scheduling is a sensible design move. The min-fusion of teacher confidence and Lorentz dispersion as a difficulty score is a real novelty, even if the underlying ingredients (ULD-style cross-tokenizer logit alignment, LLaVA-KD-style hierarchical alignment, curriculum extraction) are known. The main comparison is also carefully controlled: same student architecture, same teacher, same initialization, same split, common generation protocol, and three-seed repeats for SurgPace versus EfficientVLM. That is more than many distillation papers do.

The soft spots are concentrated in evaluation. Tables 11 through 16 all report numbers on the test split, each marks the row that becomes the final configuration, and no validation-based selection is reported even though a 64,878-example validation split exists. So the headline is effectively the maximum of a model-selection loop over roughly twenty single-seed configurations evaluated on the test set. That inflates the margin over baselines and makes the "best student method" claim about a selected configuration, not an independent estimate. The concern is not hypothetical: the full-versus-no-curriculum gain in Table 14 is +0.0044 on ten ablated metrics, which is about the same size as the 16-metric seed standard deviation of 0.0038 in Table 4, and it is itself selected on test. So the central curriculum contribution, as measured, is weak evidence. I also notice some schedule constants (r_end, r1/r2/r3, start/end scales, temperature, gamma) are not reported, which makes reproduction harder, and the only curriculum baseline, CKD-MDSR, is out of domain and collapses.

To be fair, the paper acknowledges limitations around teacher dependence, fixed scores, hardware scope, and the LLM-based audit. It does not acknowledge the test-based configuration selection, and that is the load-bearing issue. The teacher and benchmark overlap with the authors' group is a smaller concern and not disqualifying on its own, especially because the teacher choice is reasonable and the efficiency table is concrete.

This paper deserves a serious referee. I would ask for a major revision: rerun selection on the validation split, report the hidden test numbers, release full hyperparameters and checkpoints, and add at least one in-domain curriculum baseline. If those numbers hold, this is a useful contribution for surgical MLLM compression. As it stands, the recipe is credible but the headline is not yet demonstrated.

The strongest objection

Tables 11-16 in Section 4.4 (complexity fusion, curriculum order, pacing, phase transition, and alignment) all report numbers on the fixed 64,856-example test split of Section 4.1, and each table marks the row that becomes the final configuration by best observed metrics under seed 42. No validation-set selection is reported; the 64,878 validation examples are never mentioned in the ablations. The headline 0.6955 and the 'best on 15 of 16 metrics' statement are therefore not a pre-specified comparison but the maximum of a small model-selection loop over roughly 20 single-seed configurations evaluated on the same test split used for the headline. This inflates the margin over baselines and makes the 'best student method' claim about a selected configuration rather than an independent estimate. It also affects the curriculum evidence: the full-vs-no-curriculum gain in Table 14 is +0.0044 on the ten ablated metrics, comparable to the 16-metric seed standard deviation of 0.0038 in Table 4, and it is itself selected on test. Limitations (i)-(iv) acknowledge teacher dependence, fixed scores, hardware scope, and audit design, but not test-based configuration selection.

The reader's verdict is CONDITIONAL, and I agree with that verdict. My most load-bearing concern is not the Eq. 10 min rule: even if s_i is a noisy difficulty signal, the full framework could still beat baselines through hierarchical alignment and progressive scheduling, and Tables 11-12 show that ordering direction matters. What actually threatens the reported numbers is that the final configuration was chosen from test-set performance. Section 4.4 highlights a 'final' row in Tables 11-16 based on best metrics on the fixed test split, with no validation-based selection and no seed averaging for the ablations. The headline 0.6955 is thus a selected maximum across roughly 20 single-seed configurations. This also weakens the curriculum evidence: Table 14's full-vs-no-curriculum gap (+0.0044 on ten metrics) is close to the seed standard deviation reported in Table 4 (0.0038 on sixteen metrics), so without multiple seeds the small curriculum contribution may be noise. I credit the paper for a reasonably controlled common protocol, a shared student architecture and initialization, three-seed robustness for the SurgPace-vs-EfficientVLM comparison, and the public code link; those mitigate but do not remove the selection problem. A validation-based reselection experiment would settle whether the central empirical claim survives, so I keep the reader's CONDITIONAL verdict unchanged.

The referee's report

The paper's strengths are its controlled comparison design and its transparency: all student methods share the same architecture, teacher, initialization, split, and generation protocol; the strongest-baseline comparison is repeated over three seeds (Table 4); the difficulty-score sensitivity is quantified (Table 5); training budgets are reported (Table 17); the judge and audit prompts, sampling rules, and configuration hashes are given in the supplement; and the limitation list is honest about teacher dependence, fixed scores, hardware scope, and audit design. If the headline comparison holds, the result is a credible, practically relevant demonstration that a compact student can match or exceed a much larger teacher on several surgical dialogue paradigms at substantially lower cost. The principal weakness is evaluative rather than derivational: the final configuration is selected on the same test split that produces the headline number, so the exact magnitudes of the component contributions and the 0.6955 figure carry selection bias.

Pith's reading of the paper (a machine's summary, not the authors')

SurgPace tries to establish that knowledge distillation of a large surgical multimodal language model into a small one works substantially better when it is run as a curriculum rather than as uniform transfer: samples are fed easy-first, and supervision shifts over time from imitating the teacher to optimizing against ground truth. Its central empirical claim is that a 3B student trained this way from a 13B teacher beats six established distillation baselines on 15 of 16 benchmark metrics, with an unweighted average of 0.6955 versus 0.6221 for the strongest baseline, while using about 4x less GPU memory and decoding about 1.4x faster. The stakes are clinical deployment: surgical assistants that answer questions, describe scenes, and localize instruments must run on operating-room hardware with strict latency limits, so a compression method that keeps capability while cutting cost is what the paper is after. The authors report the student even edges past the teacher on short-answer questions and LLM-judged description quality, while the teacher keeps its expected lead on open-ended and spatially demanding tasks.

What carries the argument. The load-bearing object is the offline complexity score $s_i = \min(s_{\mathrm{conf}}(x_i), s_{\mathrm{lorentz}}(x_i))$, computed once from teacher decoding of the gold answer. $s_{\mathrm{conf}}$ is $1$ minus the teacher's mean top-1 token probability; $s_{\mathrm{lorentz}}$ lifts each teacher logit vector onto the unit Lorentz hyperboloid and measures the mean geodesic dispersion of the decoding trajectory, normalized across the whole dataset. The min-fusion is deliberately permissive: a sample counts as easy if either signal says so. The score becomes a sample weight $w'_i(p) = 1 + a(p)(w_i - 1)$ that anneals toward uniform weights as training progress $p$ reaches $r_{\mathrm{end}}$, so training starts easy-first and ends balanced. Sample pacing is coupled to hierarchical representation alignment — temperature-scaled KL on the shared vocabulary support, squared distance between centered logits, latent feature alignment through a learned head, and entropy regularization — whose combined weight $\beta_{\mathrm{dist}}(p)$ is interpolated down while the cross-entropy weight $\beta_{\mathrm{ce}}(p)$ rises through a three-phase stage coefficient $\alpha(p)$. Because teacher and student use different tokenizers, token positions are located by UTF-8 span and teacher logits are linearly interpolated at student positions before any comparison is made.

Core claim. The paper claims that distilling a 13B surgical multimodal teacher into a 3B student is markedly more effective when two training axes are coordinated: a sample-level curriculum that paces examples from easy to hard using an offline teacher-derived complexity score, and an objective-level schedule that shifts supervision from teacher imitation toward ground-truth task loss. On the Surg-396K benchmark, the produced student scores the highest unweighted average over 16 reported metrics (0.6955), is the best student method on 15 of 16 metrics, and repeats that dominance across three seeds ($0.6930\pm0.0038$ versus $0.6221\pm0.0029$ for the strongest baseline, EfficientVLM). A blinded LLM judge rates its detailed descriptions above the teacher's (77.48 versus 76.67), and controlled FP16 decoding shows $4.04\times$ fewer parameters, about $4\times$ less GPU memory, and $1.39\times$ faster end-to-end latency than the 13B teacher. The authors state that the teacher still leads on open-answer overlap metrics, region-based reasoning, and grounding, which they attribute to a remaining capacity gap, and that their clinical audit is a reference-text LLM audit, not a substitute for prospective adjudication by surgeons looking at the original images.

The premise it rests on. The load-bearing premise is that a sample the 13B teacher decodes confidently, or with a compact logit trajectory, is genuinely easy for the 3B student to learn — and that the permissive min-fusion never lets one overconfident teacher signal mark a genuinely hard sample as easy; if teacher certainty does not transfer to student learnability, the curriculum signal is noise.

If the paper is right (Pith's inferences, not the authors')

A 3B surgical assistant can match or beat a 13B teacher on short-answer QA and LLM-judged description quality while cutting loaded GPU memory from 26.03 GB to 6.46 GB and end-to-end latency by about 1.4x in the paper's controlled FP16 benchmark.

Offline difficulty scoring is cheap and stable: scoring all 591,031 instances took 3.11 GPU-hours, and perturbing the teacher logits used for scoring costs only 0.4% (temperature $T=1.3$) to 1.0% (5% logit noise) on the 16-metric average.

The direction of the curriculum is what matters, not just difficulty weighting: easy-to-hard ordering beats hard-to-easy and mixed ordering on every reported metric, and linear pacing beats root, geometric, baby-step, and one-pass schedules.

The two design axes contribute separately: the full framework's ten-metric mean is 0.6585, versus 0.6541 with only the progressive objective schedule, and removing the latent-alignment or entropy terms degrades most metrics.

The hierarchical alignment objective is the strongest single component: removing either the KL or the centered-logit L2 term causes severe degradation across all task families.

The recipe is not surgical-specific: any heterogeneous vision-language corpus with a strong teacher could reuse the confidence-plus-Lorentz scorer, with pathology imaging, endoscopy, and procedural video as nearby testbeds.

The paper's own ablations put the curriculum's standalone gain at about 0.0044 on the ten-metric mean (0.6541 to 0.6585), so a fair reading is that the multi-level alignment and the progressive schedule carry most of the weight; a decisive test would be the full curriculum on top of a static objective.

Both difficulty signals are teacher-side and fixed offline, so the framework could be made adaptive by replacing the prior with a student-side measure such as per-sample student loss or teacher-student agreement — the open direction the authors list as future work.

The min-fusion rule is a falsifiable bet: a sample is called easy when only one of two teacher signals is confident, so samples where the teacher is confidently wrong will be mislabeled, which may be why the remaining audit errors concentrate in action direction and unsupported details.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Multimodal large language models (MLLMs) have shown strong potential in robotic surgery for question answering, scene understanding, and grounded assistance, but their high computational cost limits real clinical deployment. Knowledge distillation provides a practical way to build compact surgical MLLMs, yet direct distillation is not well suited to surgical data because surgical samples vary greatly in difficulty, visual ambiguity, and required clinical context, making uniform teacher-student transfer unstable. In addition, the role of supervision changes during training: early stages require stronger teacher guidance for multimodal alignment, while later stages rely more on task supervision for effective optimization. To address these challenges, we propose SurgPace, a complexity-aware continuous curriculum framework for progressive distillation of surgical vision-language models. SurgPace estimates sample complexity using predictive confidence and Lorentz geometric consistency, and uses the fused score to guide a continuous easy-to-hard training process. We further introduce hierarchical representation alignment, which aligns the student with the teacher at the feature, logit, and probability levels, while gradually shifting the training focus from teacher imitation to task supervision. This design separates sample-level curriculum from objective-level scheduling, leading to more stable and effective distillation. We evaluate SurgPace on Surg-396K, a benchmark built from four public surgical datasets and covering five dialogue paradigms. SurgPace consistently outperforms strong MLLM distillation baselines, demonstrating its effectiveness for building compact, memory-efficient surgical MLLMs with faster decoding under controlled benchmarks. Code is available at https://github.com/hezhicheng2002/SurgPace.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-04

The authors' main claim, as the classifier picked it out 1.00

“SurgPace consistently outperforms strong MLLM distillation baselines, demonstrating its effectiveness for building compact, memory-efficient surgical MLLMs with faster decoding under controlled benchmarks.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.25 · human participants 0.15 · controlled comparison 0.88 · randomized assignment 0.01 · states quantitative result 0.12 · compares with prior results 0.97 · mentions code or data availability 0.98

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Multimodal large language models (MLLMs) have shown strong potential in robotic surgery for question answering, scene understanding, and grounded assistance, but their high computational cost limits real clinical deployment. background 1.00 0.07 0.03 asserted 0.98
Knowledge distillation provides a practical way to build compact surgical MLLMs, yet direct distillation is not well suited to surgical data because surgical samples vary greatly in difficulty, visual ambiguity, and required clinical context, making uniform teacher-student transfer unstable. gap 0.67 0.11 0.03 asserted 1.00
In addition, the role of supervision changes during training: early stages require stronger teacher guidance for multimodal alignment, while later stages rely more on task supervision for effective optimization. background 0.69 0.15 0.02 asserted 1.00
To address these challenges, we propose SurgPace, a complexity-aware continuous curriculum framework for progressive distillation of surgical vision-language models. contribution 0.99 0.80 0.07 asserted 0.98
SurgPace estimates sample complexity using predictive confidence and Lorentz geometric consistency, and uses the fused score to guide a continuous easy-to-hard training process. method 0.97 0.92 0.05 asserted 1.00
We further introduce hierarchical representation alignment, which aligns the student with the teacher at the feature, logit, and probability levels, while gradually shifting the training focus from teacher imitation to task supervision. method 0.91 0.90 0.05 asserted 1.00
This design separates sample-level curriculum from objective-level scheduling, leading to more stable and effective distillation. interpretation 0.58 0.66 0.11 asserted 1.00
We evaluate SurgPace on Surg-396K, a benchmark built from four public surgical datasets and covering five dialogue paradigms. method 0.76 0.93 0.03 asserted 0.97
SurgPace consistently outperforms strong MLLM distillation baselines, demonstrating its effectiveness for building compact, memory-efficient surgical MLLMs with faster decoding under controlled benchmarks. result 0.98 0.96 0.85 asserted 1.00
Code is available at https://github.com/hezhicheng2002/SurgPace. contribution 0.66 0.53 0.02 asserted 0.88

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.94

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-04: 1 request, 7,673 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (4)
Figure 1 of the paper
Figure 1. Overview of the proposed SurgPace framework. Given surgical multimodal data, a teacher model first estimates sample complexity o
Figure 2 of the paper
Figure 2. Overview of the proposed complexity-aware curriculum framework for progressive distillation. Teacher confidence and Lorentz geometric con￾sistency are fused into a permissive complexity score, which guides continuous curriculum pacing from easy-first sampling to balanced training. This pacing is coupled with progressive objective interpolation and hierarchical representation alignment, jointly establishing a more structured optimization process that enables stable and efficient compression of surgical vision-language models. receive greater emphasis during training, whereas…
Figure 3 of the paper
Figure 3. Qualitative visualization of a representative same-image case across five dialogue paradigms. For the same surgical scene, the figure compares ground
Figure 4 of the paper
Figure 4. Radar chart of task mean scores computed from Tables 2 and 3. Each

Figures are the authors', reproduced from the paper.

References (64)
  1. [1] Bai, L., Islam, M., Ren, H., 2023a. CAT-ViL: Co- attention gated vision-language embedding for visual 15 question localized-answering in robotic surgery, in: Medi- cal Image Computing and Computer Assisted Intervention – MICCAI 2023, Springer. pp. 397–406. doi:10.1007/ 978-3-0... on Pith
  2. [2] Bai, L., Islam, M., Seenivasan, L., Ren, H., 2023b. Surgical-vqla: Transformer with gated vision-language embedding for visual question localized-answering in robotic surgery. arXiv preprint arXiv:2305.11692 on PithPith's reviewarXiv
  3. [3] Boizard, N., Haddad, K.E., Hudelot, C., Colombo, P., on Pith
  4. [4] Cai, Y ., Zhang, J., He, H., He, X., Tong, A., Gan, Z., Wang, C., Xue, Z., Liu, Y ., Bai, X., 2025. Llava-kd: A framework of distilling multimodal large language mod- els, in: Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pp. 239–249 on Pith
  5. [5] Cao, J., Zhang, Y ., Huang, T., Lu, M., Zhang, Q., An, R., Ma, N., Zhang, S., 2025. Move-kd: Knowledge distil- lation for vlms with mixture of visual encoders, in: Pro- ceedings of the Computer Vision and Pattern Recognition Conference, pp. 19846–19856 on Pith
  6. [6] Chen, Z., Luo, X., Wu, J., Chan, D.T.M., Lei, Z., Ourselin, S., Liu, H., 2024a. SurgFC: Multimodal surgical func- tion calling framework on the demand of surgeons, in: 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 3076–3081. doi:10.110 9/BIBM... on Pith
  7. [7] Chen, Z., Luo, X., Wu, J., Chan, D.T.M., Lei, Z., Ourselin, S., Liu, H., 2024b. Vs-assistant: versatile surgery assistant on the demand of surgeons. arXiv preprint arXiv:2405.08272 URL:https://arxiv.org/abs/24 05.08272 on PithPith's reviewarXiv
  8. [8] Chen, Z., Luo, X., Yuan, K., Wu, J., Chan, D.T.M., Navab, N., Liu, H., Lei, Z., Luo, J., 2025. SurgLLM: A versa- tile large multimodal model with spatial focus and tem- poral awareness for surgical video understanding. arXiv preprint arXiv:2509.00357 URL:https://arxiv.org/ abs... on PithPith's reviewarXiv
  9. [9] Dong, W., Ge, J., Dong, W., Motani, M., 2026. Llama32- med: Parameter-efficient adaptation of multimodal llms for medical visual question answering, in: Medical Imag- ing with Deep Learning on Pith
  10. [10] Fang, L., Yu, X., Cai, J., Chen, Y ., Wu, S., Liu, Z., Yang, Z., Lu, H., Gong, X., Liu, Y ., et al., 2026. Knowledge distillation and dataset distillation of large language mod- els: Emerging trends, challenges, and future directions. Artificial Intelligence Review 59, 17 on Pith
  11. [11] Gu, Y ., Dong, L., Wei, F., Huang, M., 2024. MiniLLM: Knowledge distillation of large language models, in: The Twelfth International Conference on Learning Represen- tations on Pith
  12. [12] Gupta, S., Karmalkar, S., 2025. Efficient knowledge distillation via curriculum extraction. arXiv preprint arXiv:2503.17494 URL:https://arxiv.org/abs/25 03.17494 on PithPith's reviewarXiv
  13. [13] Hao, P., Li, S., Wang, H., Kou, Z., Zhang, J., Yang, G., Zhu, L., 2025a. Surgery-r1: Advancing surgical-vqla with reasoning multimodal large language model via reinforce- ment learning. arXiv preprint arXiv:2506.19469 on PithPith's reviewarXiv
  14. [14] Hao, P., Wang, H., Yang, G., Zhu, L., 2025b. Enhanc- ing visual reasoning with llm-powered knowledge graphs for visual question localized-answering in robotic surgery. IEEE Journal of Biomedical and Health Informatics on Pith
  15. [15] He, J., Li, P., Liu, G., Zhong, S., 2025. Parameter-efficient fine-tuning medical multimodal large language models for medical visual grounding, in: 2025 IEEE 22nd Interna- tional Symposium on Biomedical Imaging (ISBI), IEEE. pp. 1–5 on Pith
  16. [16] Holland, R., Taylor, T.R., Holmes, C., Riedl, S., Mai, J., Patsiamanidi, M., Mitsopoulou, D., Hager, P., Müller, P., Paetzold, J.C., et al., 2025. Specialized curricula for train- ing vision language models in retinal image analysis. npj Digital Medicine 8, 532 on Pith
  17. [17] Javaheripi, M., Bubeck, S., 2023. Phi-2: The surpris- ing power of small language models. Microsoft Research Blog. URL:https://www.microsoft.com/en-us/ research/blog/phi-2-the-surprising-power-o f-small-language-models/. published December 12, 2023 on Pith
  18. [18] Jia, J., Hu, Y ., Weng, X., Shi, Y ., Li, M., Zhang, X., Zhou, B., Liu, Z., Luo, J., Huang, L., et al., 2024. Tinyllava fac- tory: A modularized codebase for small-scale large mul- timodal models. arXiv preprint arXiv:2405.11788 on PithPith's reviewarXiv
  19. [19] Jiang, S., Wang, Y ., Song, S., Hu, T., Zhou, C., Pu, B., Zhang, Y ., Yang, Z., Feng, Y ., Zhou, J.T., et al., 2025a. Hulu-med: A transparent generalist model towards holis- tic medical vision-language understanding. arXiv preprint arXiv:2510.08668 on PitharXiv
  20. [20] Jiang, W., Lu, Y ., Lin, H., Han, X., Sun, L., 2025b. Teach small models to reason by curriculum distillation, in: Pro- ceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 7423–7433 on Pith
show 44 more
  1. [21] Ke, Y .H., Jin, L., Elangovan, K., Ong, B.W.X., Oh, C.Y ., Sim, J., Loh, K.W.T., Soh, C.R., Cheng, J.M.H., Lee, A.K.Y ., et al., 2026. Real-world deployment and evalua- tion of peri-operative ai chatbot (peach): a large language model chatbot for peri-operative medicine. Anaes... on Pith
  2. [22] Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., Gao, J., 2023a. Llava- med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems 36, 28541–28564 on Pith
  3. [23] Li, J., Skinner, G., Yang, G., Quaranto, B., Schwait- zberg, S., Kim, P., Xiong, J., 2024a. Llava-surg: Towards multimodal surgical assistant via structured surgical video learning. arXiv preprint arXiv:2408.07981 on PithPith's reviewarXiv
  4. [24] Li, X., Lin, L., Wang, S., Qian, C., 2023b. Unlock the power: Competitive distillation for multi-modal large lan- guage models. arXiv preprint arXiv:2311.08213 on PithPith's reviewarXiv
  5. [25] Li, X., Lin, L., Wang, S., Qian, C., 2024b. Self-improving teacher cultivates better student: Distillation calibration for multimodal large language models, in: Proceedings of the 47th International ACM SIGIR Conference on Re- search and Development in Information Retrieval, p... on Pith
  6. [26] Li, Y ., Liao, B., Liu, W., Wang, X., 2025. Matvlm: Hybrid mamba-transformer for efficient vision-language model- ing. arXiv preprint arXiv:2503.13440 on PithPith's reviewarXiv
  7. [27] Loshchilov, I., Hutter, F., 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 on PithPith's reviewarXiv
  8. [28] Low, C.H., Wang, Z., Zhang, T., Zhuo, Z., Zeng, Z., Ma- zomenos, E.B., Jin, Y ., 2026. Surgraw: Multi-agent work- flow with chain of thought reasoning for robotic surgical video analysis. IEEE Robotics and Automation Letters on Pith
  9. [29] Maier-Hein, L., Vedula, S.S., Speidel, S., Navab, N., Kiki- nis, R., Park, A., Eisenmann, M., Feussner, H., Forestier, G., Giannarou, S., et al., 2017. Surgical data science for next-generation interventions. Nature Biomedical Engi- neering 1, 691–696 on Pith
  10. [30] Min, Z., Lai, J., Ren, H., 2025. Innovating robot-assisted surgery through large vision models. Nature Reviews Electrical Engineering 2. doi:10.1038/s44287- 025 -00166-6 on Pithdoi:10.1038/s44287-
  11. [31] Ong, C.S., Obey, N.T., Zheng, Y ., Cohan, A., Schneider, E.B., 2024. Surgeryllm: a retrieval-augmented genera- tion large language model framework for surgical decision support and workflow enhancement. npj Digital Medicine on Pith
  12. [32] Perez, A., Rau, A., White, L., Mlambo, B., Nwoye, C., Jamal, M.A., Mohareri, O., 2026. Sureon: A benchmark and vision-language-model for surgical reasoning. arXiv preprint arXiv:2603.06570 on PithPith's reviewarXiv
  13. [33] URL:http://dx.doi.org/10.1038/s41746-024 -01391-3, doi:10.1038/s41746-024-01391-3 on Pithdoi:10.1038/s41746-024
  14. [34] Rajbhandari, S., Rasley, J., Ruwase, O., He, Y ., 2020. Zero: Memory optimizations toward training trillion pa- rameter models, in: SC20: international conference for high performance computing, networking, storage and analysis, IEEE. pp. 1–16 on Pith
  15. [35] Qin, G., Wang, X., Zhuo, Z., Low, C.H., Xiao, Y ., Fu, Y ., Liu, H., Wang, K., Li, C., Jin, Y ., 2026. Surgo-r1: Bench- marking and modeling contextual reasoning for operative zone in surgical video. arXiv preprint arXiv:2602.21706 on PitharXiv
  16. [36] Rueckert, T., Rauber, D., Maerkl, R., Klausmann, L., Yildiran, S.R., Gutbrod, M., Nunes, D.W., Moreno, A.F., Luengo, I., Stoyanov, D., et al., 2026. Comparative vali- dation of surgical phase recognition, instrument keypoint estimation, and instrument instance segmentation in ... on Pith
  17. [37] Rau, A., Endo, M., Aklilu, J., Heo, J., Saab, K., Paderno, A., Jopling, J., Holsinger, F.C., Yeung-Levy, S., 2025. Systematic evaluation of large vision-language models for surgical artificial intelligence. arXiv preprint arXiv:2504.02799 on PithPith's reviewarXiv
  18. [38] Schmidgall, S., Cho, J., Zakka, C., Hiesinger, W., 2024a. GP-VLS: A general-purpose vision language model for surgery. arXiv preprint arXiv:2407.19305 URL:https: //arxiv.org/abs/2407.19305 on PithPith's reviewarXiv
  19. [39] Saha, R., Fahim, A., Fyshe, A., Murphy, A., 2024. Ex- ploring curriculum learning for vision-language tasks: A study on small-scale multimodal training. arXiv preprint arXiv:2410.15509 on PithPith's reviewarXiv
  20. [40] Seenivasan, L.K., Islam, M., Krishna, A., Ren, H., on Pith
  21. [41] Schmidgall, S., Kim, J.W., Kuntz, A., Ghazi, A.E., Krieger, A., 2024b. General-purpose foundation models for increased autonomy in robot-assisted surgery. Nature Machine Intelligence 6, 1275–1283 on Pith
  22. [42] Soviany, P., Ionescu, R.T., Rota, P., Sebe, N., 2022. Cur- riculum learning: A survey. International Journal of Com- puter Vision 130, 1526–1565 on Pith
  23. [43] Sun, W., Xie, R., Zhang, J., Zhao, W.X., Lin, L., Wen, J.R., 2025. Curriculum-scheduled knowledge distillation from multiple pre-trained teachers for multi-domain se- quential recommendation. World Wide Web 28, 1–26 on Pith
  24. [44] Shu, F., Liao, Y ., Zhuo, L., Xu, C., Zhang, L., Zhang, G., Shi, H., Chen, L., Zhong, T., He, W., Fu, S., Li, H., Li, B., Yu, Z., Liu, S., Li, H., Jiang, H., 2024. LLaV A- MoD: Making LLaV A tiny via MoE knowledge distil- lation. arXiv preprint arXiv:2408.15881 URL:https: //ar... on PithPith's reviewarXiv
  25. [45] Wang, G., Bai, L., Nah, W.J., Wang, J., Zhang, Z., Chen, Z., Wu, J., Islam, M., Liu, H., Ren, H., 2024. Surgical- lvlm: Learning to adapt large vision-language model for grounded visual question answering in robotic surgery. arXiv preprint arXiv:2405.10948 on PithPith's reviewarXiv
  26. [46] Wang, G., Bai, L., Wang, J., Yuan, K., Li, Z., Jiang, T., He, X., Wu, J., Chen, Z., Lei, Z., Liu, H., Wang, J., Zhang, F., Padoy, N., Navab, N., Ren, H., 2026. Endochat: Grounded multimodal large language model for endoscopic surgery. Medical Image Analysis 107, 103789. doi:10... on Pithdoi:10.1016/j
  27. [47] Tripathi, S., Sukumaran, R., Cook, T.S., 2024. Efficient healthcare with large language models: optimizing clin- ical workflow and enhancing patient care. Journal of 17 the American Medical Informatics Association 31, 1436– 1440 on Pith
  28. [48] Wang, G., Xiao, H., Zhang, R., Gao, H., Bai, L., Yang, X., Li, Z., Li, H., Ren, H., 2025b. Copesd: A multi-level surgical motion dataset for training large vision-language models to co-pilot endoscopic submucosal dissection, in: Proceedings of the 33rd ACM International Confer... on Pith
  29. [49] Wang, T., Zhou, W., Zeng, Y ., Zhang, X., 2023. Effi- cientvlm: Fast and accurate vision-language models via knowledge distillation and modal-adaptive pruning, in: Findings of the Association for Computational Linguis- tics: ACL 2023, pp. 13899–13913. URL:https:// aclanthology... on Pithdoi:10.18653/v1/2023.findings-acl.873
  30. [50] Wang, G., Wang, J., Mo, W., Bai, L., Yuan, K., Hu, M., Wu, J., He, J., Huang, Y ., Padoy, N., Lei, Z., Liu, H., Navab, N., Ren, H., 2025a. SurgVidLM: Towards multi- grained surgical video understanding with large language model. arXiv preprint arXiv:2506.17873 URL:https: //arx... on PitharXiv
  31. [51] Wen, Z., Wang, S., Zhou, Y ., Zhang, J., Zhang, Q., Gao, Y ., Chen, Z., Wang, B., Li, W., He, C., Zhang, L., 2025. Efficient multi-modal large language models via progressive consistency distillation. arXiv preprint arXiv:2510.00515 URL:https://arxiv.org/abs/25 10.00515 on PitharXiv
  32. [52] Xu, S., Li, X., Yuan, H., Qi, L., Tong, Y ., Yang, M.H., 2024. Llavadi: What matters for multimodal large language models distillation. arXiv preprint arXiv:2407.19409 URL:https://arxiv.org/abs/24 07.19409 on PithPith's reviewarXiv
  33. [53] Wang, X., Chen, Y ., Zhu, W., 2022. A survey on curricu- lum learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 4555–4579. doi:10.1109/TPAM I.2021.3069908 on Pithdoi:10.1109/tpam
  34. [54] Yuan, K., Srivastav, V ., Navab, N., Padoy, N., 2024. Hecvl: Hierarchical video-language pretraining for zero- shot surgical phase recognition, in: International Con- ference on Medical Image Computing and Computer- Assisted Intervention, Springer. pp. 306–316 on Pith
  35. [55] Yuan, K., Srivastav, V ., Yu, T., Lavanchy, J.L., Marescaux, J., Mascagni, P., Navab, N., Padoy, N., 2025. Learning multi-modal representations by watching hundreds of sur- gical video lectures. Medical Image Analysis , 103644 on Pith
  36. [56] Yang, S., Zhou, F., Mayer, L., Huang, F., Chen, Y ., Wang, Y ., He, S., Nie, Y ., Wang, X., Jin, Y ., et al., 2026. Large- scale self-supervised video foundation model for intelli- gent surgery. npj Digital Medicine on Pith
  37. [57] Zeng, Z., Zhuo, Z., Jia, X., Zhang, E., Wu, J., Zhang, J., Wang, Y ., Low, C.H., Jiang, J., Zheng, Z., et al., 2025. Surgvlm: A large vision-language model and systematic evaluation benchmark for surgical intelligence. arXiv preprint arXiv:2506.02555 on PithPith's reviewarXiv
  38. [58] Zhai, X., Mustafa, B., Kolesnikov, A., Beyer, L., 2023. Sigmoid loss for language image pre-training, in: Pro- ceedings of the IEEE/CVF international conference on computer vision, pp. 11975–11986 on Pith
  39. [59] Zeng, Z., Xu, M., Jiang, J., Guo, P., Xu, Y ., Zhuo, Z., Low, C.H., He, Y ., Yang, D., Lin, C., et al., 2026. Surg σ: A spectrum of large-scale multimodal data and foun- dation models for surgical intelligence. arXiv preprint arXiv:2603.16822 on PitharXiv
  40. [60] Zhang, Y ., Mohamed, A., Abdine, H., Shang, G., Vazir- giannis, M., 2026. Beyond random sampling: Effi- cient language model pretraining via curriculum learning, in: Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (V ... on Pith
  41. [62] Zhang, S., Zhang, X., Sun, Z., Chen, Y ., Xu, J., on Pith
  42. [63] Dual-space knowledge distillation for large lan- guage models, in: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, As- sociation for Computational Linguistics. p. 18164–18181. URL:http://dx.doi.org/10.18653/v1/2024.emnl p-main.1010, doi:... on Pithdoi:10.18653/v1/2024.emnl
  43. [2022] Surgical-VQA: Visual question answering in surgi- cal scenes using transformer, in: Medical Image Comput- ing and Computer Assisted Intervention – MICCAI 2022, Springer. pp. 33–43. doi:10.1007/978-3-031-16449 -1_4 on Pithdoi:10.1007/978-3-031-16449
  44. [2024] Towards cross-tokenizer distillation: the uni- versal logit distillation loss for llms. arXiv preprint arXiv:2402.12030 on PithPith's reviewarXiv

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment