Pith. sign in

REVIEW 2 major objections 7 minor 1 cited by

Vulnerability of Text-Matching in ML/AI Conference Reviewer Assignments to Collusions

T0 review · 2 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Colluding author-reviewer pairs can steer text-based reviewer assignment, lifting a target reviewer from rank 101 to the top 5 about 92% of the time on NeurIPS 2023 data.

desk verdict SPECTER text-matching is gameable at scale; the 'get assigned' headline outruns the ranking-based evidence. read the letter →

arxiv 2412.06606 v2 pith:KDT7JZ36 submitted 2024-12-09 cs.LG cs.CRcs.DL

Jhih-Yi Hsieh , Aditi Raghunathan , Nihar B. Shah This is my paper · ORCID
classification cs.LGcs.CRcs.DL
keywords reviewerassignmenttext-matchingattacksSPECTERcollusionringsadversarialabstractspeerreviewintegrityNeurIPSarchivecuration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the text-matching component of automated reviewer assignment at major ML/AI venues is not secure against collusion. It shows that a collaborating author and reviewer, without any bid manipulation, can raise the reviewer's computed similarity to the author's paper from a natural rank of 101 to the top 5 among all reviewers roughly 92% of the time on NeurIPS 2023 data. The attack works by having the reviewer curate their profile down to one highly similar past paper and the author add theme sentences and keywords to the abstract, using an LLM. If the paper is right, venues that disabled bidding to stop collusion rings remain exposed, and text similarity cannot be treated as a trusted signal.

What carries the argument

The machinery is the cosine similarity in SPECTER's embedding space between a paper's title or abstract and each paper in a reviewer's archive, aggregated by either maximum or mean and then ranked across all reviewers. The attack exploits two control surfaces: the reviewer can adversarially shrink the archive to the single paper most similar to the target paper, and the author can add abstract sentences about the reviewer's themes and greedily selected keywords, both implemented with an LLM. These operations increase the embedding similarity without obviously changing what the paper is about.

What would settle it

Run the same attack on a venue's actual assignment solver, with reviewer load caps, conflict lists, and any randomization included, and measure the fraction of manipulated top-5 pairs that are actually assigned; if that fraction is far below the reported 92%, the practical consequence of the claim would fail even though the ranking shift itself is real.

Watch

Extended reading notes

Core claim

The central claim is that SPECTER, the text-embedding model used for reviewer-paper matching at major ML/AI venues, is practically manipulable by a colluding author-reviewer pair. Using only the publicly available model weights, the reviewer's past-publication profile, and the previous year's reviewer pool, the pair can substantially increase the colluding reviewer's similarity ranking for the target paper. In fully automatic experiments on 3,218 NeurIPS 2023 papers and 7,900 reviewers, the attack lifts a reviewer with natural rank 101 to top-5 in 93% of cases; with human oversight and early stopping the top-5 rate is 92%. Even without touching the abstract, selecting the single most similar past paper raises the reviewer to top-5 in 41% of cases. The paper interprets these numbers as evidence that the assumption that text matching is robust to manipulation is false.

Load-bearing premise

The load-bearing premise is that moving the colluding reviewer into the top-1, top-3, or top-5 similarity rankings is a faithful proxy for being assigned to review the paper, since the paper does not run the full assignment optimization with load limits, conflicts of interest, and randomization.

Editorial extensions

If this is right

  • Venues that removed bidding to thwart collusion rings can still be attacked through text matching alone.
  • Reviewer archive curation is a standalone vulnerability: keeping one highly similar paper can put the colluding reviewer in the top 5 about 41% of the time.
  • Max-pooling aggregation is more attack-prone than mean pooling, and imposing a minimum archive size or switching aggregation reduces success rates.
  • Attackers can tune their modifications on publicly available prior-year reviewer pools, because manipulated rankings correlate strongly (Spearman 0.62 to 0.93) between two consecutive NeurIPS editions.
  • Human reviewers rarely flag the manipulated abstracts as malicious, and LLM-paraphrased benign abstracts draw similar coherence complaints, giving colluders plausible deniability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If assignment optimizers add randomization or strict load and conflict constraints, the real-world assignment-level success rate could be lower than the reported top-5 ranking success, because the paper evaluates rankings rather than final assignments.
  • The same abstract-plus-profile manipulation could transfer to other text-based matching systems, such as journal or grant reviewer matching, that use comparable embedding models, though success rates would need to be measured.
  • A defense that averages similarity and caps how few papers a reviewer may keep may only shift attacks toward more sophisticated abstract edits, since the paper's evidence suggests a trade-off between robustness and matching quality.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. This paper studies whether the SPECTER-based text-matching component used in automated reviewer assignment at ML/AI venues can be manipulated by colluding author-reviewer pairs. The proposed attack combines adversarial archive curation by the reviewer with LLM-based abstract modifications (IncludeThemes and InsertKeywords), in both human-in-the-loop and fully automatic modes. Experiments on NeurIPS 2022/2023 measure the colluding reviewer's similarity rank among all reviewers, reporting high top-1/3/5 success rates (e.g., 92% top-5 for natural rank 101), strong cross-year correlations, and sensitivity to archive length and pooling method. A preregistered human-subject study examines whether adversarial abstracts are detectable. The paper concludes that text matching is vulnerable and proposes defenses such as minimum archive lengths, average pooling, and randomized assignment.

Significance. The main technical finding—that SPECTER embeddings are susceptible to simple, LLM-assisted textual manipulation—is credible and well supported by out-of-sample evaluation: hyperparameters are tuned on NeurIPS 2022 and evaluated on held-out NeurIPS 2023 data, with standard errors reported and code, data, and adversarial examples released. The threat model is realistic, and the cross-year correlation analysis is a useful contribution. If the similarity-ranking results translate to actual assignments, the implications are substantial because the affected algorithms are used at multiple venues. The central gap is that the headline 'get assigned' claim is supported only by a ranking proxy, not by assignment-level simulation; the paper's own Section 6.2 concedes that load, conflict, and geographic constraints were not factored in. This gap must be addressed before the full significance of the claim can be accepted.

major comments (2)
  1. [Abstract; §2.1; §5.1; §6.2] The abstract and introduction claim that colluders can 'get assigned their target paper,' but the experiments measure only the colluding reviewer's top-1/3/5 similarity rank among all reviewers, not whether the assignment optimization actually selects that reviewer. Section 5.1 defines success rates as ranking-based, Section 2.1 justifies rankings as a proxy for general applicability, and Section 6.2 explicitly states that constraints such as reviewer and paper load limits, conflicts of interest, and geographical considerations were not factored in. Thus the reported 92% figure in Tables 1 and 2 is the fraction of attacks reaching top-5 similarity, not the fraction of papers assigned to the colluding reviewer. Please either simulate the actual assignment optimization with realistic constraints, or consistently rephrase the central claim as 'achieve top-k similarity ranking' and avoid the unqualified 'get assigned' language. The current wording overstates the evidence.
  2. [§5.7.2; §6.2] The human-subject experiment used attack budgets and prompts that were 'erroneously tuned on the NeurIPS 2023 test data' rather than the NeurIPS 2022 training data. The manuscript argues that this should not affect the outcome because the manipulated abstracts in that experiment were edited more heavily than those in the automatic attacks. This is plausible but not demonstrated, and the detectability results are the only evidence for the paper's 'plausible deniability' conclusion. The limitations section mentions the error, but the conclusions in Section 5.7.6 should either be explicitly qualified as conditional on this tuning issue or supported by a robustness check using abstracts generated with training-data-tuned budgets.
minor comments (7)
  1. [§5.1] There is a typo in 'Similarily, we also sample'—it should be 'Similarly.'
  2. [§5.2] The 25 human-in-the-loop evaluations were not a random sample: the authors write that they 'keep the first 25 samples with paper topics we are familiar enough with.' This selection rule could bias the high success rates in Table 1; please report the number of samples excluded by this familiarity criterion and comment on the potential direction of bias.
  3. [§3, Related Work, fourth bullet] The sentence 'While [CFB+20] has only released three adversarial samples' appears to cite the wrong paper; the context indicates that the prior attack paper [EQM+23] released three samples, not the SPECTER paper [CFB+20].
  4. [§5.7.1] The preregistration URL 'https://aspredicted.org/HXF Z92' contains an erroneous space and should be corrected.
  5. [Table 6] There is a typo: 'reponses' should be 'responses.' Also, the rows for 'Irregularities related to IncludeThemes' and 'Irregularities related to InsertKeywords' should specify that the percentages are computed over the 67 experimental reviews, since the table currently mixes group sizes.
  6. [Figure 4] The x-axis label 'Target Archive Length' is ambiguous; it should be 'Reviewer archive length |Qadv_r|' or 'Required archive length' to indicate that this is the number of papers the reviewer is forced to retain.
  7. [§2.3] The threat model should state explicitly how the colluding pair avoids being flagged by conflict-of-interest screening; if the attack assumes colluders are not co-authors or institutional affiliates, that assumption should be stated and justified.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation in the central attack evaluation: hyperparameters are tuned on NeurIPS 2022 and success is measured on held-out NeurIPS 2023 rankings; the only disclosed leakage is in the secondary human-subject experiment, where test-set tuning does not determine the measured detectability outcome.

full rationale

The paper's main claim is supported by an out-of-sample evaluation rather than by a fitted constant or a self-citation. Section 5.1 states: "We explore the attack success rates under different N, M, K with the NeurIPS 2022 dataset and select the highest performing combination N = 5, M = 2, K = 5." The success rates in Tables 1 and 2 are then computed against the NeurIPS 2023 reviewer pool, so the attack does not tune on the data on which success is reported. The transferability result in Section 5.6 computes the same manipulated abstracts' rankings against both the 2022 and 2023 pools, which is a genuine external check rather than a circular reuse. The only leakage is explicitly disclosed in Section 5.7.2: "The budgets and prompts in this experiment are tuned the same way as those in the rest of this paper, but they were erroneously tuned on the NeurIPS 2023 test data instead of the NeurIPS 2022 training data." That leakage affects the secondary human-subject detectability experiment, not the headline attack-success claim, and the paper states that it is not measuring attack success in that experiment. The ranking-versus-assignment limitation (Sections 2.1 and 6.2) is an external-validity caveat, not a circular reduction: the paper explicitly defines the attacker objective as achieving top-k similarity ranking. Citations to the authors' prior work on bidding and randomized assignment are background or defensive suggestions, not load-bearing premises. I therefore find no step in which a prediction is equivalent to its input by construction.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the threat model (white-box access to SPECTER, title and abstract only, reviewer archive curation), the ranking-as-assignment proxy, and the assumption that LLM-edited abstracts retain plausible deniability. These are domain assumptions stated in the paper, not proven outside it.

free parameters (4)
  • N (number of generated abstract versions in IncludeThemes) = 5 (default), 10 (human-in-the-loop)
    Tuned on NeurIPS 2022 success rates (Section 5.1); increasing N improves attack success at diminishing returns (Appendix B).
  • M (number of keyword insertion batches) = 2
    Tuned on NeurIPS 2022; gradual insertion outperforms larger batches (Section 5.1, Appendix B).
  • K (max keywords per batch) = 5
    Tuned on NeurIPS 2022; with M=2 gives up to 10 inserted keywords (Section 5.1).
  • delta (similarity tolerance in IncludeThemes) = 0.01
    Subjectively chosen without systematic tuning (Appendix A.1); allows small similarity drops to preserve coherence.
assumptions (5)
  • domain assumption SPECTER model weights are publicly available so attackers can compute exact embeddings and query similarity.
    Stated in Section 2.3 and used throughout; the attack is white-box.
  • domain assumption The assignment similarity score is computed from title and abstract only and uses SPECTER, with max or average pooling over the reviewer archive.
    Stated in Section 2.2; if venues use other text fields or models, results may not transfer. The paper notes preliminary SPECTER v2 results are similar.
  • domain assumption Achieving a top-5 similarity ranking among all reviewers is a meaningful proxy for getting assigned, because papers are typically assigned 3-6 reviewers.
    Section 2.1; the paper does not simulate the full assignment optimization with load or conflict constraints.
  • domain assumption Reviewers can curate their archive down to one paper in the evaluated threat model.
    Section 4.1; the paper notes OpenReview and TPMS allow curation. If venues mandate minimum archive sizes, success drops (Section 5.4).
  • domain assumption Large language model edits can preserve enough coherence and consistency to escape reliable detection; supported only partially by the underpowered human study.
    The attack's practical viability depends on this; the human study found no malicious-intent detection but had low power for some comparisons and used a control pool of LLM paraphrases.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Vulnerability of Text-Matching in ML/AI Conference Reviewer Assignments to Collusions." pith.science (2026). https://pith.science/paper/KDT7JZ36

@misc{pith2026241206606,
  author       = {Pith},
  title        = {Pith review of: Vulnerability of Text-Matching in ML/AI Conference Reviewer Assignments to Collusions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KDT7JZ36}},
  note         = {Machine review of arXiv:2412.06606}
}
read the original abstract

In the peer review process of top-tier machine learning (ML) and artificial intelligence (AI) conferences, reviewers are assigned to papers through automated methods. These assignment algorithms consider two main factors: (1) reviewers' expressed interests indicated by their bids for papers, and (2) reviewers' domain expertise inferred from the similarity between the text of their previously published papers and the submitted manuscripts. A significant challenge these conferences face is the existence of collusion rings, where groups of researchers manipulate the assignment process to review each other's papers, providing positive evaluations regardless of their actual quality. Most efforts to combat collusion rings have focused on preventing bid manipulation, under the assumption that the text similarity component is secure. In this paper, we demonstrate that even in the absence of bidding, colluding reviewers and authors can exploit the machine learning based text-matching component of reviewer assignment used at top ML/AI venues to get assigned their target paper. We also highlight specific vulnerabilities within this system and offer suggestions to enhance its robustness.

Figures

Figures reproduced from arXiv: 2412.06606 by the authors.

Figure 1
Figure 1. An illustrated example of the attack procedure. 1a shows the reviewer’s action; 1b shows the [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. An example of IncludeThemes modifying the abstract. Goal navigation is an important theme in the adversarial archive Qadv r of the reviewer r. The modification (highlighted sentence) adds robot goal navigation as a motivating example for further improvements to multimodal models. The resulting abstract a adv p remains coherent and consistent with the paper’s focus on text and image embedding alignment, but has incre… view at source ↗
Figure 3
Figure 3. An example of InsertKeywords modifications. The paper in the adversarial archive Qadv r proposes mitigating label scarcity in transfer learning with data augmentation. Some inserted keywords (highlighted) match their original meanings from the paper in Qadv r , while others (highlighted + bolded) are adapted to common English. The InsertKeywords changes to this abstract, alongside adversarial reviewer archive curati… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Attack success rates when the colluding reviewers have to keep 1, 2, 5, 10 papers in the adversarial [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Manipulated rankings in 2022 and 2023 iterations of NeurIPS are strongly correlated, so colluders [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Attack success rates generally increase with [PITH_FULL_IMAGE:figures/full_fig_p028_6.png]
Figure 7
Figure 7. Figure 7: Grids of Top-1, Top-3, and Top-5 success rates for different combinations of batches of keywords [PITH_FULL_IMAGE:figures/full_fig_p028_7.png]
Figure 8
Figure 8. Figure 8: Mean similarity scores (before attack) associated with different rankings across all papers in the [PITH_FULL_IMAGE:figures/full_fig_p029_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Identity Theft in AI Conference Peer Review

    cs.DL 2025-08 conditional novelty 6.0 of 10

    OpenReview staff report 94 fraudulent reviewer profiles in AI conferences, used by dishonest researchers to favorably review their own papers.

Reference graph

Works this paper leans on

34 extracted references · 28 canonical work pages · cited by 1 Pith paper

  1. [1]

    The sigmod 2019 research track reviewing system

    Anastasia Ailamaki, Periklis Chrysogelos, Amol Deshpande, and Tim Kraska. The sigmod 2019 research track reviewing system. ACM SIGMOD Record , 48(2):47--54, 2019

    work page 2019

  2. [2]

    Combating collusion rings is hard but possible

    Niclas Boehmer, Robert Bredereck, and Andr \'e Nichterlein. Combating collusion rings is hard but possible. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 4843--4850, 2022

    work page 2022

  3. [3]

    Controlling the false discovery rate: a practical and powerful approach to multiple testing

    Yoav Benjamini and Yosef Hochberg. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological) , 57(1):289--300, 1995

    1995

  4. [4]

    SPECTER : Document-level representation learning using citation-informed transformers

    Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel S Weld. SPECTER : Document-level representation learning using citation-informed transformers. arXiv preprint arXiv:2004.07180 , 2020

    arXiv 2004

  5. [5]

    Charlin and R

    L. Charlin and R. S. Zemel. The T oronto P aper M atching S ystem: An automated paper-reviewer assignment system. In ICML Workshop on Peer Reviewing and Publishing Models , 2013

    work page 2013

  6. [6]

    The toronto paper matching system: an automated paper-reviewer assignment system

    Laurent Charlin and Richard Zemel. The toronto paper matching system: an automated paper-reviewer assignment system. The International Conference on Learning Representations , 2013

    work page 2013

  7. [7]

    No more reviewer\# 2: Subverting automatic paper-reviewer assignment using adversarial learning

    Thorsten Eisenhofer, Erwin Quiring, Jonas M \"o ller, Doreen Riepel, Thorsten Holz, and Konrad Rieck. No more reviewer\# 2: Subverting automatic paper-reviewer assignment using adversarial learning. In 32nd USENIX Security Symposium (USENIX Security 23) , pages 5109--5126, 2023

    work page 2023

  8. [8]

    HotFlip : White-box adversarial examples for text classification

    Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. HotFlip : White-box adversarial examples for text classification. arXiv preprint arXiv:1712.06751 , 2017

    arXiv 2017

Show all 34 references
  1. [9]

    K-loop free assignment in conference review systems

    Longhua Guo, Jie Wu, Wei Chang, Jun Wu, and Jianhua Li. K-loop free assignment in conference review systems. In 2018 International Conference on Computing, Networking and Communications (ICNC) , pages 542--547. IEEE, 2018

    2018

  2. [10]

    Knuth: Computers and typesetting, 2021

    Andrew McCallum Haw-Shiuan Chang. Knuth: Computers and typesetting, 2021. https://www.overleaf.com/project/5f359923225f06000134ea95. Last accessed 14 April 2024

    2021

  3. [11]

    Is BERT really robust? a strong baseline for natural language attack on text classification and entailment

    Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is BERT really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence , volume 34, pages 8018--8025, 2020

    2020

  4. [12]

    On the detection of reviewer-author collusion rings from paper bidding

    Steven Jecmen, Nihar B Shah, Fei Fang, and Leman Akoglu. On the detection of reviewer-author collusion rings from paper bidding. arXiv preprint arXiv:2402.07860 , 2024

    2024 arXiv

  5. [13]

    Tradeoffs in preventing manipulation in paper bidding for reviewer assignment

    Steven Jecmen, Nihar B Shah, Fei Fang, and Vincent Conitzer. Tradeoffs in preventing manipulation in paper bidding for reviewer assignment. In ICLR workshop on ML Evaluation Standards , 2022

    2022

  6. [14]

    Shah, and Fei Fang

    Steven Jecmen, Minji Yoon, Vincent Conitzer, Nihar B. Shah, and Fei Fang. A dataset on malicious paper bidding in peer review. In TheWebConf , 2023

    2023

  7. [15]

    Mitigating manipulation in peer review via randomized reviewer assignments

    Steven Jecmen, Hanrui Zhang, Ryan Liu, Nihar Shah, Vincent Conitzer, and Fei Fang. Mitigating manipulation in peer review via randomized reviewer assignments. Advances in Neural Information Processing Systems , 33:12533--12545, 2020

    2020

  8. [16]

    Paper matching with local fairness constraints

    Ari Kobren, Barna Saha, and Andrew McCallum. Paper matching with local fairness constraints. In ACM KDD , 2019

    2019

  9. [17]

    Matching papers and reviewers at large conferences

    Kevin Leyton-Brown, Yatin Nandwani, Hedayat Zarkoob, Chris Cameron, Neil Newman, and Dinesh Raghu. Matching papers and reviewers at large conferences. Artificial Intelligence , 331:104119, 2024

    2024

  10. [18]

    Collusion rings threaten the integrity of computer science research

    Michael L Littman. Collusion rings threaten the integrity of computer science research. Communications of the ACM , 64(6):43--44, 2021

    2021

  11. [19]

    Bert-attack: Adversarial attack against bert using bert

    Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. Bert-attack: Adversarial attack against bert using bert. arXiv preprint arXiv:2004.09984 , 2020

    2004 arXiv

  12. [20]

    Editable user profiles for controllable text recommendation

    Sheshera Mysore, Mahmood Jasim, Andrew McCallum, and Hamed Zamani. Editable user profiles for controllable text recommendation. arXiv preprint arXiv:2304.04250 , 2023

    2023 arXiv

  13. [21]

    Expertise modeling for matching papers with reviewers

    David Mimno and Andrew McCallum. Expertise modeling for matching papers with reviewers. In KDD , 2007

    2007

  14. [22]

    PDF mirage: Content masking attack against Information-Based online services

    Ian Markwood, Dakun Shen, Yao Liu, and Zhuo Lu. PDF mirage: Content masking attack against Information-Based online services. In 26th USENIX Security Symposium (USENIX Security 17) , pages 833--847, Vancouver, BC, August 2017. USENIX Association

    2017

  15. [23]

    Neighborhood contrastive learning for scientific document representations with citation embeddings

    M Ostendorff, N Rethmeier, I Augenstein, et al. Neighborhood contrastive learning for scientific document representations with citation embeddings. arXiv preprint arXiv:2202.06671 , 2022

    2022 arXiv

  16. [24]

    Slide from ICLR 2024 program chair presentation, 2024

    ICLR 2024 P rogram C hairs. Slide from ICLR 2024 program chair presentation, 2024. https://twitter.com/chriswolfvision/status/1787886748434878769. Last accessed 16 May 2024

    2024

  17. [25]

    I will have order! optimizing orders for fair reviewer assignment

    Justin Payan and Yair Zick. I will have order! optimizing orders for fair reviewer assignment. In Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems , pages 1711--1713, 2022

    2022

  18. [26]

    Scirepeval: A multi-format benchmark for scientific document representations

    Amanpreet Singh, Mike D'Arcy, Arman Cohan, Doug Downey, and Sergey Feldman. Scirepeval: A multi-format benchmark for scientific document representations. arXiv preprint arXiv:2211.13308 , 2022

    2022 arXiv

  19. [27]

    Challenges, experiments, and computational solutions in peer review

    Nihar B Shah. Challenges, experiments, and computational solutions in peer review. Communications of the ACM. Preprint available at https://www.cs.cmu.edu/ nihars/preprints/SurveyPeerReview.pdf, June 2022

    2022

  20. [28]

    PeerReview4All : Fair and accurate reviewer assignment in peer review

    Ivan Stelmakh, Nihar Shah, and Aarti Singh. PeerReview4All : Fair and accurate reviewer assignment in peer review. Journal of Machine Learning Research , 2021

    2021

  21. [29]

    PDFPhantom : Exploiting pdf attacks against academic conferences' paper submission process with counterattack

    Dat Tran and Chetan Jaiswal. PDFPhantom : Exploiting pdf attacks against academic conferences' paper submission process with counterattack. In 2019 IEEE 10th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON) , pages 0736--0743, 2019

    2019

  22. [30]

    T. N. Vijaykumar. Potential organized fraud in ACM / IEEE computer architecture conferences, 2020. https://medium.com/@tnvijayk/potential-organized-fraud-in-acm-ieee-computer-architecture-conferences-ccd61169370d. Last accessed 29 April 2024

    2020

  23. [31]

    T. N. Vijaykumar. Potential organized fraud in on-going ASPLOS reviews, 2020. https://medium.com/@tnvijayk/potential-organized-fraud-in-on-going-asplos-reviews-874ce14a3ebe. Last accessed 29 April 2024

    2020

  24. [32]

    Simple and effective paraphrastic similarity from parallel translations

    John Wieting, Kevin Gimpel, Graham Neubig, and Taylor Berg-Kirkpatrick. Simple and effective paraphrastic similarity from parallel translations. In ACL , pages 4602--4608, Florence, Italy, July 2019

    2019

  25. [33]

    Making paper reviewing robust to bid manipulation attacks

    Ruihan Wu, Chuan Guo, Felix Wu, Rahul Kidambi, Laurens Van Der Maaten, and Kilian Weinberger. Making paper reviewing robust to bid manipulation attacks. In International Conference on Machine Learning , pages 11240--11250. PMLR, 2021

    2021

  26. [34]

    A one-size-fits-all approach to improving randomness in paper assignment

    Yixuan Xu, Steven Jecmen, Zimeng Song, and Fei Fang. A one-size-fits-all approach to improving randomness in paper assignment. Advances in Neural Information Processing Systems , 36, 2024

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.