Pith.

Image and Video Processing · arXiv:2609.36549 v1 · submitted 29 Sep 2026 · 48 references

MedForge-RSI: Medical Deepfake Detection via Recursive Self-Improvement

Zhihui Chen, Mengling Feng

The paper says

“Text-guided image editors can generate high-fidelity medical deepfakes, challenging the reliability of clinical imagery. Although reasoning-based detectors perform strongly in distribution, they degrade substantially under deployment shift.”

more of the abstract

MedForge-Reasoner, an 8B vision-language model trained with supervised fine-tuning and reinforcement learning, achieves 99.2% accuracy on its target distribution, yet misclassifies 40% of authentic scans, reaches only 77% accuracy on unseen generators, and falls to 59% under transmission distortion. Adapting such models through conventional retraining is costly, requiring large-scale supervision and expert-designed guidelines. We introduce MedForge-RSI, a recursive self-improvement framework that enables a deployed detector to adapt while keeping its model weights frozen. Over 20 rounds, the detector analyzes verified errors, accumulates reusable experience, and autonomously develops image-analysis tools, while an independent acceptance test retains only validated improvements. Across 49 registered configurations, MedForge-RSI increases average accuracy over four test sets from 75.0% to 84.4%. On a held-out 4,000-image evaluation, it improves clean accuracy from 76.5% to 87.9% and transmission-distorted accuracy from 59.0% to 70.5%, with the largest gains in authentic-image recall. Controlled analysis across all 49 trajectories identifies which self-improvement mechanisms replicate across seeds, shows acceptance testing to be the largest individual contributor, and reveals a taxonomy of failed adaptations. We release the complete trajectories, including all rejected and rolled-back changes.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 1 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. Strong empirical study of deployment-time self-improvement with a real missing control: no frozen baseline at a matched operating point, so the headline gains may be threshold rebalancing. This line is wrong

3 major objections · 5 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 48 references say.

The paper says, at §5.1, Table 1; §7 (b154); Figure 5

The review did not quote this passage.

Pith says: Objection 1 of 3 · major

No operating-point-matched frozen baseline is reported, and this is load-bearing for the central claim. The base detector's error structure is one-sided (main-set real recall 60%, fake recall 99%; clean final evaluation 53.8% real, ~99.2% fake). Every reported gain lies exactly on the balanced-accuracy line: on the clean final evaluation, ensemble reflection moves real/fake recall from 53.8/99.2 to 89.3/86.4, and (89.3+86.4)/2 …

more of objection 1

… = 87.9 reproduces the reported accuracy; on the distorted final evaluation, (58.8+82.2)/2 = 70.5 reproduces the reported accuracy; the main-set control and ensemble points satisfy the same identity. That is, the +11.4/+11.5 point final-evaluation gains equal the average of the real-recall gain and the fake-recall loss, which is the signature of an operating-point rotation, not of added discriminative capability. The paper's own Figure 5 shows every learning configuration sliding along the real/fake recall tradeoff, and Section 7 (b154) concedes that the improved components reflect 'miscalibrated thresholds rather than missing capability.' No configuration evaluates the frozen detector at a matched operating point, e.g., a confidence threshold or a one-line leniency directive tuned on the same 100-image validation pool used by the acceptance test (§3.4, Appendix E). The tool ablation of §6.1 (Table 12) shows tools matter at the RSI-selected operating point, but it does not show the RSI outcome lies outside the frozen detector's achievable operating-point set. I recommend adding a matched-operating-point frozen baseline (threshold or fixed directive tuned on the validation pool) reported on all four sets and the final evaluation, with the real/fake recall pair, and an ROC-style comparison showing whether the RSI final state is outside the frozen detector's achievable tradeoff. Without this control, the claim of b163 that the loop adds capability is not distinguished from a threshold adjustment the frozen model could make with no self-improvement loop.

The paper says, at §6.2, Table 11; §4 (final evaluation)

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.70):

“The final evaluation is complete for six configurations; the self-reflection and case-based memory rows and the first two training-time rows are single-run or pending measurements and their final-evaluation cells are left blank accordingly. formance, as in the case of second-opinion arbi- tration, which reaches 87.5% at its best round and finishes at 78.5%.”

Pith says: Objection 2 of 3 · major

The headline magnitudes rest on single-run measurements even for the configuration that is claimed to be seed-replicated. Per Table 11, only ensemble reflection replicates, and the replication was measured on the main test set only (86.5 and 81.5; mean 84.0 versus 79.5 control).

more of objection 2

The final-evaluation figures that carry the abstract's and conclusion's main claims (clean 87.9, distorted 70.5) are from the base-seed run v12 and were not replicated; two-sided reflection (86.9/65.7) and case-based memory lack seed replication entirely (b160). Given the paper's own stated standard ('seed replication as a methodological requirement for self-improvement claims', §6.2, and condition 4 of b163), the +11-point final-evaluation and +9.4-point four-set numbers should be presented as single-run, seed-unconfirmed estimates, with the replicated estimate (roughly +4.5 points on the main set) given equal prominence in the abstract and conclusion. The acknowledgment in the Limitations section does not fully carry into how the headline numbers are framed.

The paper says, at §4; Appendix E; §3.4

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.72):

“The learning stream draws labeled images from the training pool, and a de- ployment would require some verified label source, such as adjudicated clinical flags.”

Pith says: Objection 3 of 3 · major

The deployment-assumption gap should be tightened where the quantitative claims depend on it. The loop requires 40 verified labels per round from the learning stream (drawn here from the detector's own training pool) plus a 100-image labeled validation pool drawn from the official held-out split.

more of objection 3

The paper acknowledges the label-source requirement (b159), so this is not presented as a new finding, but it does mean the demonstrated gains are conditional on a continuously labeled stream that a real deployment must supply externally; the abstract and conclusion state the four conditions of b163 without this fifth, arguably equally binding, condition.

5 minor objections

The paper says, at Abstract vs. §4/Table 1

Pith says: minor objection 1 of 5

The abstract's 'falls to 59% on unseen generators and 59% under transmission distortion' mixes bases: 59.0% is the final-evaluation distorted base, while the four-test-set distorted set's base is 64.5%. Please state which base each claim refers to, or the numbers read as inconsistent with Table 1.

The paper says, at Appendix Q

Pith says: minor objection 2 of 5

The silent-verifier narrative ends mid-sentence: 'the score-drop check The main scoreboard compared fields explicitly...' appears to be a truncated sentence. Please complete it.

This objection may describe the copy Pith gave the reviewer, not the paper.

show 3 more

The paper says, at Table 2 (v48/v49) and §5

Pith says: minor objection 3 of 5

The recommended four-mechanism combination (v48/v49) is marked 'run.' with no results anywhere in the paper; since the contribution list and Section 7 reference it, state explicitly that no results for it are available and that all headline figures are from ensemble reflection (v12) and two-sided reflection (v19).

The paper says, at §6.1

Pith says: minor objection 4 of 5

The text says ten of eleven configurations lose accuracy without their tools 'by 4.3 points on average'; the mean across the eleven rows of Table 12 is 4.27 including the zero row, which matches only if the zero row is included. Please state the computation so readers are not misled about the ten that lose.

The paper says, at General

Pith says: minor objection 5 of 5

Given the operating-point issue, consider reporting a table of (real recall, fake recall, accuracy) for the frozen control under several thresholds alongside the RSI states in Table 1; this would make the rebalancing component of the gains directly visible and would strengthen the paper's otherwise excellent disclosure.

What would settle it

On the released 4,000-image final evaluation, evaluate the frozen MedForge-Reasoner checkpoint across a sweep of operating points without any learning, memory, or tools: vary a confidence threshold on the parsed verdict (or token-level probability on the conclusion line; if the API exposes no score, vary a fixed one-sentence directive such as 'when uncertain, clear real images'). Plot accuracy and real/fake recall per threshold. If any threshold point reaches 87.9% clean and 70.5% distorted, or the 84.4% four-set average, the RSI gains are consistent with pure operating-point rebalancing. If the best threshold point falls materially below those numbers while holding both sets, the self-improvement claim survives. This requires no retraining and uses only the released harness and scorer.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 1 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic partly agrees 1 min 46 s · 30,411 tokens read, 8,790 written
  3. Editor wrote the verdict above 39 s · 31,376 tokens read, 2,968 written
  4. Referee major revision: 3 major objections, 5 minor 57 s · 30,614 tokens read, 9,573 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Read this one. It is the most careful empirical study of recursive self-improvement in a perception task I have seen, and it has one load-bearing gap. What is new: a frozen 8B medical deepfake detector gains roughly 9 to 11 accuracy points through two knowledge stores, experience memory and self-written tools, under independent-validation acceptance testing; there are 49 pre-registered single-switch configurations, a measured four-mode failure taxonomy, a negative seed-replication result, and a promised release of all trajectories. That is real work. The tool-calibration analysis is genuinely good: 61% of model-written tools emitted no parseable reading, 26% embedded hardcoded numbers unrelated to their computation, and the percentile-calibration layer plus tool ablations show the tools contribute about 4.3 points on average. Credit where due: the validation pool is disjoint from the test sets, the 4,000-image final evaluation is disjoint from training, circularity is low, and the authors disclose the resample contamination and the missing replications rather than hiding them.

Soft spots, in order of importance. First, the operating-point concern lands. The frozen detector's errors are extremely one-sided: on the clean final evaluation real recall is 53.8% and fake recall is about 99.2% at 76.5% accuracy. Ensemble reflection moves real recall from 53.8% to 89.3% and fake recall from 99.2% to 86.4%, which is an accuracy gain of 11.4 points on a balanced set. The arithmetic is exactly the mirror tradeoff along the same axis, and the distorted set shows the same pattern. No configuration evaluates the frozen base at a matched operating point, via a confidence threshold or a fixed one-line directive. Without that control, the headline gain may be a threshold adjustment available with no self-improvement loop at all. The authors say in Section 7 that the gains reflect missing evidence and miscalibrated thresholds rather than missing capability, but they never isolate the threshold component. That is the central ambiguity. Second, seed replication covers only ensemble reflection; the best single-seed configuration does not replicate, two-sided reflection and case-based memory are single-seed, and the recommended four-mechanism combination was still running at submission. The authors state all of this, but by their own methodological standard only one configuration fully supports the claim. Third, learning batches draw from the training pool, so a deployment label source is assumed; they acknowledge this in Section 7.

Who this is for: anyone working on deployment-time adaptation, verifier design, or evaluation standards for self-improvement claims. The failure taxonomy, operating-point reversal, silent verifier failure, and single-run variability, is a contribution on its own. Recommendation: it deserves a serious referee. The referee should ask for a matched-operating-point frozen baseline and completed seed replications for the principal configurations. With those, the claim becomes either much weaker or much stronger; right now it is genuinely ambiguous.

The strongest objection

The most load-bearing gap is that no frozen baseline is evaluated at a matched operating point. The base detector's error structure is extremely one-sided: main-set real recall is 60% versus fake recall 99%; on the clean final evaluation real recall is 53.8%, which with the balanced 2,000-image design implies fake recall of about 99.2% at 76.5% accuracy. The headline improvements are exactly the mirror-image tradeoff along that same axis. On the clean final evaluation, ensemble reflection reaches 89.3% real recall and 86.4% fake recall, that is, 87.9% accuracy; on the distorted set it moves 22.1% to 58.8% real and 95.9% to 82.2% fake, that is, 59.0% to 70.5% accuracy. In both cases the accuracy gain equals the average of the real-recall gain and the fake-recall loss, the signature of an operating-point shift. The paper's own Section 7 explanation says the gains reflect missing evidence and miscalibrated thresholds rather than missing capability, yet no configuration reports the frozen detector at a similarly balanced operating point, for example via a confidence threshold or a fixed one-line directive. Without that control, the observed +9.4-point four-set average and +11.4/+11.5-point final-evaluation gains do not establish that reflection, memory, self-written tools, or acceptance testing add capability; they may be a threshold adjustment the frozen model could have made with no self-improvement loop at all.

The reader's weakest assumption (deployment label source) is real but acknowledged in Section 7 (b159): the loop requires a verified label stream, and the paper explicitly conditions the deployment claim on it. I do not treat that as the decisive gap. The threshold-matched baseline is more load-bearing because it is unaddressed and directly bears on the interpretation of every headline number. The recall pairs in Section 5.1 are consistent with moving along the frozen detector's existing ROC curve: each final-evaluation gain equals half the real-recall increase plus half the fake-recall decrease, and the paper itself attributes the improvement partly to threshold miscalibration. A threshold-matched frozen control would settle whether the RSI machinery is responsible or whether a fixed operating-point change suffices. This does not overturn the paper's empirical honesty or the value of its failure taxonomy, but it is a genuine condition on the central 'self-improvement' claim, so the reader's CONDITIONAL verdict stands.

The referee's report

If the central claim held, this would be a notable result: a safety-critical perception system whose weights stay frozen yet which adapts to new generators and transmission distortions through verified artifact-level self-modification, with full trajectory disclosure and an unusually honest seed-replication analysis. The paper's strengths are real and worth naming: 49 registered single-switch configurations, pre-registration of batches, machine-checked disjointness of the validation pool from the 4,000-image final evaluation, complete release of trajectories, and a self-critical mechanism analysis (several intuitive mechanisms measurably hurt, and the best single run fails to replicate). However, the headline improvements are exactly the average of the real-recall gains and fake-recall losses of the frozen detector, which is the signature of a pure operating-point shift, and no frozen baseline at a matched operating point is reported. The central claim that the loop, memory, tools, and acceptance testing add capability beyond a threshold adjustment is therefore not yet established.

Pith's reading of the paper (a machine's summary, not the authors')

This paper claims that a medical deepfake detector with permanently frozen weights can meaningfully improve itself at deployment time, with no gradient updates and no additional training compute. Over 20 rounds, the detector examines its own verified errors, accumulates structured experience entries, and writes and repairs its own image-analysis tools, while an independent acceptance test admits only changes that pass replay and A/B validation checks. The claim is that this loop raises average accuracy over four test sets from 75.0% to 84.4%, lifts real-image recall from 60% to 80% (91% with case-based memory), and raises clean and transmission-distorted accuracy on a 4,000-image held-out evaluation from 76.5% to 87.9% and from 59.0% to 70.5%. The authors argue the gains concentrate on the error type that reinforcement learning left unfixed: the detector falsely flagged 40% of authentic scans, and this false-alarm rate drops to 20%, or to 10% with case-based memory.

What carries the argument. The machinery is a two-store adaptation loop wrapped in a safety architecture. The experience memory holds structured entries of the form (context, cue, caution), recording the imaging setting in which a lesson applies, the visual cue to seek, and the error it guards against, while the toolbox holds image-analysis tools written by the detector itself under a strict contract: a single self-contained file using only Pillow and NumPy, deterministic numeric output, and no verdict labels. Because self-written tools are unreliable instruments—61% emit output with no parseable reading and 26% embed hardcoded numbers unrelated to the computation—a calibration layer converts each raw reading into a percentile within historical fake- and real-image reading distributions, rendering directional evidence or “NO SIGNAL” when the distributions overlap. Every proposal to either store must pass a two-stage acceptance test: replay on the learning batch requiring non-negative net correct change, then an A/B comparison on a disjoint validation split allowing at most one correct-count drop, with a verifier health check that distrusts test outcomes when the replay baseline fails to reproduce 70% of known-correct predictions. The loop's write access excludes the model weights, the scoring code, the official system prompt, and all evaluation sets, which is what makes the improvement bounded self-refinement rather than unchecked self-modification.

Core claim. On its own terms, the central discovery is that recursive self-improvement works in a safety-critical perception setting when every persistent change is gated by an acceptance test on an independent validation split and when tool readings are calibrated against labeled history rather than trusted as reported. The frozen detector MedForge-Reasoner, an 8B vision–language model that reaches 99.2% accuracy in its training distribution but only 75.0% under deployment conditions, improves to 84.4% over four test sets, and on a 4,000-image final evaluation clean accuracy rises from 76.5% to 87.9% and distortion-affected accuracy from 59.0% to 70.5%. The decisive result is attribution: controlled single-switch comparisons across 49 registered configurations identify acceptance testing as the largest single contributor (a silent scoring bug that disabled it cost 17.5 points of final accuracy), tool-reading calibration as the most reliable positive mechanism (+6.5 points), and seed replication as the criterion that separates genuine mechanisms (ensemble reflection, +4.5 points over control) from single-stream luck (calibration+freeze, whose best run of 87.0 fails to replicate).

The premise it rests on. The load-bearing premise is that deployment supplies a continuous stream of correctly labeled images (40 per round) plus a labeled validation split whose distribution matches the evaluation target; the paper acknowledges in Section 7 that its learning stream draws from the training pool and that a real deployment would need a verified label source such as adjudicated clinical flags, and without such labels the loop cannot run and its gains are tuned toward a validation proxy.

If the paper is right (Pith's inferences, not the authors')

A deployed detector can track newly released generators and transmission distortions without retraining; each admitted change costs roughly 140 labeled validation images and the entire study uses fewer labels than a single SFT run of the base detector.

The false-alarm rate on authentic scans drops from 40% to 20% (10% with case-based memory), attacking the failure mode most damaging to clinical trust rather than the already-saturated fake-image side.

The gain generalizes beyond the learning stream: unseen-generator accuracy rises from 77.0% to 88.0–89.5%, and the improvement is monotonic across disjoint test sets, indicating the acquired capability is not memorization of the learning stream.

The division of labor persists: gradient updates set the base perceptual ability, verified self-improvement adapts it, and periodic lightweight retraining on the accumulated experience and tools is left as future work.

Six design guidelines transfer: test before admitting, calibrate tool readings against labeled history, write access only on enumerable artifacts, diversify proposals, protect recovery rather than only stability, and replicate across seeds.

If the result holds, the binding resource for perception self-improvement is verified label supply rather than model capability, which suggests the same loop could transfer to other imaging domains where ground-truth labels are cheap or automatic, such as satellite or industrial inspection.

The finding that acceptance testing matters more than reflection quality, and that proposal-diversity mechanisms replicate while verdict-formation mechanisms (arbitration, self-consistency voting) hurt, predicts a testable ranking for other frozen-model self-improving systems: admission gating and evidence calibration should beat proposal cleverness.

Both a silent verifier bug and an operating-point reversal went unnoticed until hand inspection, which implies that a real clinical deployment needs externally held health checks on the acceptance machinery itself; the paper's released trajectories, including rejected and rolled-back changes, are the natural audit substrate.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Text-guided image editors can generate high-fidelity medical deepfakes, challenging the reliability of clinical imagery. Although reasoning-based detectors perform strongly in distribution, they degrade substantially under deployment shift. MedForge-Reasoner, an 8B vision-language model trained with supervised fine-tuning and reinforcement learning, achieves 99.2% accuracy on its target distribution, yet misclassifies 40% of authentic scans, reaches only 77% accuracy on unseen generators, and falls to 59% under transmission distortion. Adapting such models through conventional retraining is costly, requiring large-scale supervision and expert-designed guidelines. We introduce MedForge-RSI, a recursive self-improvement framework that enables a deployed detector to adapt while keeping its model weights frozen. Over 20 rounds, the detector analyzes verified errors, accumulates reusable experience, and autonomously develops image-analysis tools, while an independent acceptance test retains only validated improvements. Across 49 registered configurations, MedForge-RSI increases average accuracy over four test sets from 75.0% to 84.4%. On a held-out 4,000-image evaluation, it improves clean accuracy from 76.5% to 87.9% and transmission-distorted accuracy from 59.0% to 70.5%, with the largest gains in authentic-image recall. Controlled analysis across all 49 trajectories identifies which self-improvement mechanisms replicate across seeds, shows acceptance testing to be the largest individual contributor, and reveals a taxonomy of failed adaptations. We release the complete trajectories, including all rejected and rolled-back changes.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-09-30

The authors' main claim, as the classifier picked it out 0.47

“Across 49 registered configurations, MedForge-RSI increases average accuracy over four test sets from 75.0% to 84.4%.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.49 · human participants 0.10 · controlled comparison 0.82 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.93 · mentions code or data availability 0.89

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Text-guided image editors can generate high-fidelity medical deepfakes, challenging the reliability of clinical imagery. background 1.00 0.08 0.02 asserted 0.99
Although reasoning-based detectors perform strongly in distribution, they degrade substantially under deployment shift. background 0.70 0.22 0.04 asserted 1.00
MedForge-Reasoner, an 8B vision-language model trained with supervised fine-tuning and reinforcement learning, achieves 99.2% accuracy on its target distribution, yet misclassifies 40% of authentic scans, reaches only 77% accuracy on unseen generators, and falls to 59% under transmission distortion. result 0.66 0.83 0.07 asserted 1.00
Adapting such models through conventional retraining is costly, requiring large-scale supervision and expert-designed guidelines. background 0.49 0.12 0.02 asserted 1.00
We introduce MedForge-RSI, a recursive self-improvement framework that enables a deployed detector to adapt while keeping its model weights frozen. contribution 0.95 0.82 0.14 asserted 1.00
Over 20 rounds, the detector analyzes verified errors, accumulates reusable experience, and autonomously develops image-analysis tools, while an independent acceptance test retains only validated improvements. method 1.00 0.90 0.09 asserted 1.00
Across 49 registered configurations, MedForge-RSI increases average accuracy over four test sets from 75.0% to 84.4%. result 1.00 0.96 0.28 asserted 1.00
On a held-out 4,000-image evaluation, it improves clean accuracy from 76.5% to 87.9% and transmission-distorted accuracy from 59.0% to 70.5%, with the largest gains in authentic-image recall. result 1.00 0.96 0.29 asserted 1.00
Controlled analysis across all 49 trajectories identifies which self-improvement mechanisms replicate across seeds, shows acceptance testing to be the largest individual contributor, and reveals a taxonomy of failed adaptations. result 0.63 0.96 0.15 asserted 1.00
We release the complete trajectories, including all rejected and rolled-back changes. contribution 0.98 0.94 0.04 asserted 0.98

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.95

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-30: 1 request, 7,703 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (7)
Figure 1 of the paper
Figure 1. The MedForge-RSI loop. a, At deployment the frozen MedForge-Reasoner processes a fresh learning batch of 40 labeled images in every round, and its errors are asymmetric: fake-image recall is 99% but real-image recall only 60%. b, Reflection examines these errors and proposes changes to two knowledge stores, structured experience entries and sandboxed tools written by the model itself. Every proposal must pass a two-stage acceptance test, consisting of replay on the learning batch and an A/B comparison on an independent validation split, and a proposal that reduces accuracy on either…
Figure 2 of the paper
Figure 2. Learning dynamics over 20 rounds on the main test set. The upper panel shows accuracy and the lower panel real-image recall. The frozen control remains constant at 79.5% and 60%. Configurations without calibration improve early in the run and subse￾quently decline, whereas the calibrated configurations maintain their improvement through the final round. Fake-image recall, which varies far more strongly for the uncalibrated configurations, appears in
Figure 3 of the paper
Figure 3. Four failure modes, plotted from measured trajectories. Dotted lines mark the frozen control’s final main-set accuracy (79.5%) in panels I and III. (I) Early peaks that decline; the arrows mark the interval from peak to final value. (II) A batch of seven admitted changes reverses the operating point in round 10, and no subsequent round reverts it. (III) The best single run of the calibration+freeze configuration does not replicate, whereas ensemble reflection does. (IV) With the acceptance test disabled by a defect in the scoring code, the same configuration finishes 17.5 points…
Figure 5 of the paper
Figure 5. Error structure of all 47 completed configu￾rations (final state, main test set): real-image recall ver￾sus fake-image recall, colored by accuracy. The frozen control (star) lies at the saturated-fake / weak-real cor￾ner; every learning configuration shifts toward higher real recall. The operating-point-reversal configurations (bottom right) increase one recall at the expense of the other. Propose round 5: checker for diaphragm, 60 lines Sandbox probe images 0 of 4 pass rejected Repair model rewrites its own code 4 of 4 pass Deploy in service rounds 5 to 11 Audit error vs ok overlap…
Figure 6 of the paper
Figure 6. Lifecycle of a self-written tool (a diaphragm￾consistency verifier). The proposal fails all four sandbox probes; the reflector repairs its own code and the tool enters service. Six rounds later the discrimination table shows no separation, and the model disables its own tool; accuracy recovers from 57.5% to 72.5% in the following round (real-image recall 25%→55%). The write-repair-deploy-audit-retire cycle runs without hu￾man intervention. structions; seat labels and JSON field names are renamed to match the terminology of this paper. Stage 1: free-form diagnosis (temperature 0.4,…
Figure 7 of the paper
Figure 7. Per-set accuracy of the four leading configu￾rations versus the frozen control. Case-based memory is strongest under distortion (77.0); two-sided reflec￾tion is strongest on the resample and unseen-generator sets (88.0, 89.5), and ensemble and two-sided reflection both improve monotonically across main → resample → unseen-generator. I Complete Self-Tool Ablation

Showing 6 of the paper's 7 figures. Figures are the authors', reproduced from the paper.

References (48)
  1. [1] Med-Banana-50K: A Cross-modality Large-Scale Dataset for Text-guided Medical Image Editing , author=. arXiv preprint arXiv:2511.00801 , year= on PithPith's reviewarXiv
  2. [2] Diffusion model-based image editing: A survey , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , year= on Pith
  3. [3] Sida: Social media image deepfake detection, localization and explanation with large multimodal model , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages= on Pith
  4. [4] FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models , author=. International Conference on Learning Representations , year= on Pith
  5. [5] Spot the fake: Large multimodal model-based synthetic image detection with artifact explanation , author=. arXiv preprint arXiv:2503.14905 , year= on PitharXiv
  6. [6] AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Models , author=. arXiv preprint arXiv:2507.02664 , year= on PithPith's reviewarXiv
  7. [7] Toward Medical Deepfake Detection: A Comprehensive Dataset and Novel Method , author=. International Conference on Medical Image Computing and Computer-Assisted Intervention , pages=. 2025 , organization= on Pith
  8. [8] MedNet: Medical deepfakes detection using an improved deep learning approach , author=. Multimedia Tools and Applications , volume=. 2024 , publisher= on Pith
  9. [9] Alsaheel, Alaa and Alhassoun, Reem and Alrashed, Reema and Almatrafi, Noura and Almallouhi, Noura and Albahli, Saleh , year =. Deep Fakes in Healthcare: How Deep Learning Can Help to Detect Forgeries , volume =. Computers, Materials Continua , doi = on Pith
  10. [10] \ CT-GAN \ : Malicious tampering of 3d medical imagery using deep learning , author=. 28th USENIX Security Symposium (USENIX Security 19) , pages= on Pith
  11. [11] The Optimal Model for Copy-Move Forgery Detection in Medical Images , author=. Journal of Medical Signals Sensors , volume=. 2024 , publisher= on Pith
  12. [12] Qwen3-VL Technical Report , author=. 2025 , eprint= on Pith
  13. [13] Chen, Zhihui and He, Kai and Lei, Qingyuan and Pu, Bin and Zhang, Jian and Xu, Yuling and Feng, Mengling. M ed F orge: Interpretable Medical Deepfake Detection via Forgery-aware Reasoning. Proceedings of the 64th Annual Meeting of the A ssociation for C omputational L inguisti... on Pithdoi:10.18653/v1/2026.acl-long.1177
  14. [14] Med-Banana: Learning Quality-Controlled Medical Image Editing from Success-and-Failure Trajectories , author=. 2026 , eprint= on Pith
  15. [15] Chen, Zhihui and He, Kai and Huang, Yucheng and Zhu, Yunxiao and Feng, Mengling. D iv S core: Zero-Shot Detection of LLM -Generated Text in Specialized Domains. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. doi:10.18653/v1/2025.e... on Pithdoi:10.18653/v1/2025.emnlp-main.971
  16. [16] Chen, Mingguang and Wang, Licheng and Qu, Bo , title =. arXiv preprint arXiv:2607.07663 , year = on PithPith's reviewarXiv
  17. [17] Xu, Guoyang and Chen, Hao , title =. arXiv preprint arXiv:2608.24302 , year = on PithPith's reviewarXiv
  18. [18] Zhang, Jenny and Hu, Shengran and Lu, Cong and Lange, Robert Tjarko and Clune, Jeff , title =. International Conference on Learning Representations (ICLR) , year = on Pith
  19. [19] Novikov, Alexander and V. arXiv preprint arXiv:2506.13131 , year = on PithPith's reviewarXiv
  20. [20] Yue, Zhenrui and Upasani, Kartikeya and Yang, Xianjun and Ge, Suyu and Nie, Shaoliang and Mao, Yuning and Liu, Zhe and Wang, Dong , title =. arXiv preprint arXiv:2601.07055 , year = on PithPith's reviewarXiv
show 28 more
  1. [21] Shi, Zhan and He, Bing and Sang, Yisi and Dumoulin, Benoit and Lu, Hanqing , title =. arXiv preprint arXiv:2606.20657 , year = on PithPith's reviewarXiv
  2. [22] Xia, Peng and Zeng, Kaide and Liu, Jiaqi and Qin, Can and Wu, Fang and Zhou, Yiyang and Xiong, Caiming and Yao, Huaxiu , title =. ICLR Workshop on Recursive Self-Improvement , year = on Pith
  3. [23] Zhao, Andrew and Wu, Yiran and Yue, Yang and Wu, Tong and Xu, Quentin and Wang, Shenzhi and Wu, Qingyun and Zheng, Zilong and Huang, Gao , title =. Advances in Neural Information Processing Systems (NeurIPS) , year = on Pith
  4. [24] Xiong, Yiming and Hu, Shengran and Clune, Jeff , title =. International Conference on Learning Representations (ICLR) , year = on Pith
  5. [25] Rank, Ben and Bhatnagar, Hardik and Nguyen, Karina and Prabhu, Ameya and Eisenberg, Shira and Bethge, Matthias and Andriushchenko, Maksym , title =. International Conference on Machine Learning (ICML) , year = on Pith
  6. [26] Duan, Yi and Liu, Ying and Tang, Zirui and Chen, Haodong and Zhou, Jun and others , title =. arXiv preprint arXiv:2609.11873 , year = on PithPith's reviewarXiv
  7. [27] Romera-Paredes, Bernardino and Barekatain, Mohammadamin and Novikov, Alexander and Balog, Matej and Kumar, M. Pawan and Dupont, Emilien and Ruiz, Francisco J. R. and Ellenberg, Jordan S. and Wang, Pengming and Fawzi, Omar and others , title =. Nature , volume = on Pith
  8. [28] Madaan, Aman and Tandon, Niket and Gupta, Prakhar and Hallinan, Skyler and Gao, Luyu and Wiegreffe, Sarah and Alon, Uri and Dziri, Nouha and Prabhumoye, Shrimai and Yang, Yiming and Gupta, Shashank and Majumder, Bodhisattwa Prasad and Hermann, Katherine and Welleck, Sean and Y... on Pith
  9. [29] Shinn, Noah and Cassano, Federico and Berman, Edward and Gopinath, Ashwin and Narasimhan, Karthik and Yao, Shunyu , title =. Advances in Neural Information Processing Systems (NeurIPS) , year = on Pith
  10. [30] Wang, Guanzhi and Xie, Yuqi and Jiang, Yunfan and Mandlekar, Ajay and Xiao, Chaowei and Zhu, Yuke and Fan, Linxi and Anandkumar, Anima , title =. Transactions on Machine Learning Research (TMLR) , year = on Pith
  11. [31] Hu, Shengran and Lu, Cong and Clune, Jeff , title =. International Conference on Learning Representations (ICLR) , year = on Pith
  12. [32] Zhang, Jiayi and Xiang, Jinyu and Yu, Zhaoyang and Teng, Fengwei and Chen, Xionghui and Chen, Jiaqi and Zhuge, Mingchen and Cheng, Xin and Hong, Sirui and Wang, Jinlin and Zheng, Bingnan and Liu, Bang and Luo, Yuyu and Wu, Chenglin , title =. International Conference on Learni... on Pith
  13. [33] Yin, Xunjian and Wang, Xinyi and Pan, Liangming and Lin, Li and Wan, Xiaojun and Wang, William Yang , title =. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL) , year = on Pith
  14. [34] Zelikman, Eric and Wu, Yuhuai and Mu, Jesse and Goodman, Noah , title =. Advances in Neural Information Processing Systems (NeurIPS) , year = on Pith
  15. [35] Chen, Zixiang and Deng, Yihe and Yuan, Huizhuo and Ji, Kaixuan and Gu, Quanquan , title =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL) , year = on Pith
  16. [36] Yuan, Weizhe and Pang, Richard Yuanzhe and Cho, Kyunghyun and Sukhbaatar, Sainbayar and Xu, Jing and Weston, Jason , title =. International Conference on Machine Learning (ICML) , year = on Pith
  17. [37] Shumailov, Ilia and Shumaylov, Zakhar and Zhao, Yiren and Papernot, Nicolas and Anderson, Ross and Gal, Yarin , title =. Nature , volume = on Pith
  18. [38] FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space , author=. 2025 , eprint= on Pith
  19. [39] Gpt-4o system card , author=. arXiv preprint arXiv:2410.21276 , year= on PithPith's reviewarXiv
  20. [40] Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities , author=. arXiv preprint arXiv:2507.06261 , year= on PithPith's reviewarXiv
  21. [41] Zhao, Andrew and Huang, Daniel and Xu, Quentin and Lin, Matthieu and Liu, Yong-Jin and Huang, Gao , title =. Proceedings of the AAAI Conference on Artificial Intelligence , year = on Pith
  22. [42] Ouyang, Siru and Yan, Jun and Hsu, I-Hung and Chen, Yanfei and Jiang, Ke and Wang, Zifeng and Han, Rujun and Le, Long T. and Daruki, Samira and Tang, Xiangru and Tirumalashetty, Vishy and Lee, George and Rofouei, Mahsan and Lin, Hangfei and Han, Jiawei and Lee, Chen-Yu and Pfi... on Pith
  23. [43] Zhang, Qizheng and Hu, Changran and Upasani, Shubhangi and Ma, Boyuan and Hong, Fenglu and Kamanuru, Vamsidhar and Rainton, Jay and Wu, Chen and Ji, Mengmeng and Li, Hanchen and Thakker, Urmish and Zou, James and Olukotun, Kunle , title =. arXiv preprint arXiv:2510.04618 , year = on PithPith's reviewarXiv
  24. [44] Li, Junkai and Lai, Yunghwei and Li, Weitao and Ren, Jingyi and Zhang, Meng and Kang, Xinhui and Wang, Siyu and Li, Peng and Zhang, Ya-Qin and Ma, Weizhi and Liu, Yang , title =. arXiv preprint arXiv:2405.02957 , year = on PithPith's reviewarXiv
  25. [45] Wang, Yajing and Bi, Chao and Sun, Junshu and Shen, Shufan and Qi, Zhaobo and Wang, Shuhui and Huang, Qingming , title =. arXiv preprint arXiv:2606.24292 , year = on PithPith's reviewarXiv
  26. [46] Schmidhuber, J. G. arXiv preprint arXiv:cs/0309048 , year = on PithPith's reviewarXiv
  27. [47] Gautam, Srishti and Radhakrishna, Arjun and Gulwani, Sumit , title =. arXiv preprint arXiv:2606.10546 , year = on PithPith's reviewarXiv
  28. [48] Chen, Xiwen and Zhu, Wenhui and Zheng, Songzhu and Rasul, Kashif and Deng, Yueyue and Li, Huayu , title =. arXiv preprint arXiv:2605.06822 , year = on PithPith's reviewarXiv
Cited by 1 Pith paper

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment