The reviewed record of science sign in
Pith

arxiv: 2607.05968 · v1 · pith:4QN5H2XC · submitted 2026-07-07 · cs.CL · cs.AI· cs.IR

InfluMatch: Frontier-Quality KOL Search at 4B-Model Cost

Krittanon Kaewtawee , Petmongkon Pornpichitsuwan , Natchaya Temyingyong , Nutnicha Laplamoon , Wachiravit Modecrua , Krittin Pachtrachai , Touchapon Kraisingkorn This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-08 19:39 UTCglm-5.2pith:4QN5H2XCrecord.jsonopen to challenge →

classification cs.CL cs.AIcs.IR
keywords frontiercascadecostreasonersearchbaselinecandidatesend-to-end
0
0 comments X

The pith

4B models match frontier LLM on influencer search, 35× fewer tokens

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that a three-stage cascade of small (4B-parameter) open-weight models can match a frontier LLM on a real influencer-matching task — ranking Thai KOLs against free-form marketing briefs — at roughly 1/35th the output-token cost and ~4× lower latency. The cascade works by splitting labor: dense retrieval casts a wide net (50 candidates), a SimPO-tuned 4B reranker filters to a top-10 shortlist using a single Yes-token log-probability score, and an untuned 4B reasoner grades each shortlisted KOL per criterion with a Thai rationale. The key design insight is that filtering before reasoning is both cheaper and more accurate than reasoning over the full pool, because the reasoner's coarse per-criterion scoring becomes decisive when weak candidates are removed. The paper's second claim is about supervision: fine-tuning on pairwise (relative) preferences transfers to end-to-end deployment, while fine-tuning on pointwise (absolute) scores wins offline but degrades the deployed system — an inversion traced to noisy, under-specified absolute labels where only 16% of 'match' judgments are backed by verbatim profile evidence.

Core claim

The central discovery is a division-of-labor effect in retrieval cascades: a pairwise-preference-trained reranker that is mediocre at absolute judgment (topping out at ~67% P@5 alone) becomes a strong filter that enables an untuned reasoner to reach 94.1% P@5 — beating the same reasoner applied to all 50 candidates (80.0%) by 14 points while using half the tokens. The paper traces this to the reasoner's coarse resolution (five criteria scored 0/1/2 produces ties over large pools) being eliminated when the candidate set is small. The secondary discovery is the offline–end-to-end inversion: pointwise SFT+GRPO fine-tuning of the reasoner tops the offline per-criterion leaderboard (59.0 weighted

What carries the argument

Three-stage cascade (retrieval → rerank → reason); SimPO pairwise preference training on a single Yes-token log-probability; pointwise {0,1,2} rubric scoring with even-sum aggregation; evidence audit verifying label grounding via exact-substring profile quotes

If this is right

  • If relative-preference supervision consistently transfers better than absolute scoring, annotation pipelines for LLM-based retrieval should default to best/worst comparisons over rating scales, reducing label noise and improving downstream ranking.
  • The finding that filtering before reasoning improves both cost and accuracy suggests that multi-stage cascades with cheap filters and expensive reasoners may generalize beyond KOL matching to any domain where per-item scoring has coarse resolution over large candidate sets.
  • The evidence-audit methodology — checking whether human labels are backed by verbatim evidence — offers a practical diagnostic for detecting when a model's disagreements with human labels reflect label noise rather than model error, which could reframe how fine-tuning success is evaluated.
  • The deployment of 4B open-weight models at frontier-adjacent quality on a non-English (Thai) task suggests that the cost gap between frontier and small models may be narrowest in domain-specific, well-structured retrieval tasks rather than general-purpose reasoning.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The 2.3-point P@5 gap between the 4B cascade (94.1%) and the frontier baseline (91.8%) on Set 1 (n=11) is well within the noise floor — a single query reordering shifts macro-averaged P@5 by roughly 9 points at this sample size. The Set 2 replication (n=31) reverses the ranking (frontier 91.9% > 4B 89.3%), suggesting the 'matching frontier quality' claim is directional rather than statistically es
  • The inversion finding (offline-best fine-tuned model is worst end-to-end) may be partly an artifact of the specific label-collection design rather than a general property of pointwise supervision. The paper itself attributes the gap to three compounding design choices (no rubric, grouped annotation, criterion-vs-holistic mismatch); a cleaner pointwise labeling protocol might close the inversion, w
  • The '35× fewer output tokens' comparison is between the 4B cascade (2,974 output tokens/query) and Kimi-K2.6 in think mode (102,862 output tokens/query). A fairer cost comparison might use the frontier model in no-think mode (19,457 tokens, ~6.5× more), which narrows the efficiency gap substantially while still favoring the small-model cascade.

Load-bearing premise

The headline accuracy comparison rests on 11 queries where a single reordering shifts the metric by roughly 9 percentage points, and the larger replication set (31 queries) reverses which model wins — so the claim that the cascade 'matches frontier quality' is not statistically distinguishable from a tie at the reported sample sizes.

What would settle it

Run the same cascade and frontier baseline on a larger query set (e.g., 100+ queries with full candidate labeling). If the 4B cascade's P@5 drops below the frontier baseline by a statistically significant margin, or if a cleaner pointwise labeling protocol eliminates the offline–end-to-end inversion, the paper's two central claims would be weakened.

Figures

Figures reproduced from arXiv: 2607.05968 by Krittanon Kaewtawee, Krittin Pachtrachai, Natchaya Temyingyong, Nutnicha Laplamoon, Petmongkon Pornpichitsuwan, Touchapon Kraisingkorn, Wachiravit Modecrua.

Figure 1
Figure 1. Figure 1: The three-stage InfluMatch pipeline: retrieval [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The three annotation interfaces presented to human annotators: pointwise scoring (T2), [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
read the original abstract

Matching influencers (KOLs) to free-form, multi-part Thai marketing criteria is today served either by keyword search over structured profiles, which misses semantic fit, or by prompting frontier LLMs over every candidate, which is accurate but slow and expensive. We present InfluMatch, a low-cost three-stage cascade -- retrieval $\rightarrow$ rerank $\rightarrow$ reason -- built entirely from small open-weight models: dense retrieval returns 50 candidates, a 4B pointwise reranker scores each by the log-probability of a single Yes token and keeps 10, and a 4B reasoner grades the shortlist per criterion on a rubric with a Thai rationale. The cascade is designed for cost: reasoning over a filtered top-10 halves token spend versus reasoning over all 50 while scoring 14 points higher. End-to-end against human relevance labels on an 11-query set with all 50 candidates labeled, the full cascade reaches 94.1% P@5, versus a retrieval-only baseline near random; it matches the frontier model Kimi-K2.6 (91.8%) while emitting ${\sim}35\times$ fewer output tokens and serving a 50-KOL query in ${\sim}20$ s on one A100. Notably, the only fine-tuning that pays off is pairwise: a SimPO-tuned reranker matches the frontier baseline's best-pick accuracy (78.0 EM), whereas fine-tuning the reasoner on pointwise per-criterion labels improves offline scores yet degrades end-to-end ranking -- an inversion we trace to the design of the absolute labeling task -- leaving the untuned base model as the strongest deployed reasoner. The result is a deployable, explainable KOL search system at a small fraction of frontier serving cost.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 7 minor

Summary. InfluMatch presents a three-stage cascade (retrieval → rerank → reason) for matching Thai marketing briefs to influencers (KOLs), using only 4B open-weight models. The reranker is a SimPO-tuned 4B model scoring candidates by logp(Yes); the reasoner is an untuned 4B model grading shortlisted candidates per criterion. The paper reports 94.1% P@5 on an 11-query set (Set 1), matching a frontier model (Kimi-K2.6, 91.8%) at ~35× fewer output tokens. A key secondary finding is that pairwise SimPO fine-tuning transfers end-to-end while pointwise SFT+GRPO does not, despite the latter winning offline — an inversion traced via an evidence audit to label quality issues.

Significance. The paper makes several valuable contributions: (1) a deployable cascade architecture with explicit per-stage cost budgets, (2) a SimPO formulation for single-token reranking (Eq. 1) that achieves training-serving consistency by design, (3) a well-motivated evidence audit (Table 2) that grounds the offline–end-to-end inversion in measurable label quality problems, and (4) a practical finding that pairwise supervision transfers while pointwise does not. The cost–accuracy Pareto analysis (Table 10) is informative and the system-level engineering is sound. The work is positioned within the relevant neural IR and LLM-as-judge literature.

major comments (3)
  1. §6.3, Tables 8–9: The headline claim that the cascade 'matches frontier quality' rests on a 2.3-point P@5 margin (94.1 vs. 91.8) over n=11 queries. The authors acknowledge in §Limitations that a single flipped triplet moves EM by ~2.4pp; for P@5 at k=5 over 11 queries, a single query reordering can shift the macro-average by roughly 9 points. Critically, Set 2 (n=31) reverses the ordering: Kimi-K2.6 (91.9) beats Base (89.3). The paper frames this as 'gaps compress,' but it is a direct reversal of the headline comparison on a larger sample. No confidence intervals, bootstrap distributions, or significance tests are reported. The claim should be softened to reflect that the cascade is 'competitive with' rather than 'matching' frontier quality, or the authors should provide bootstrap CIs to support the stronger claim.
  2. §6.3, Table 8: The cascade result (94.1% P@5) is notably higher than reason-on-all-50 (80.0% P@5) — a 14-point gain from filtering. This is a counterintuitive and load-bearing finding (it underpins the cascade design). The explanation in §7 (filtering removes diluting candidates, reducing ties) is plausible but not tested directly. A controlled experiment — e.g., injecting random noise into the top-10 vs. top-50 pools, or measuring tie rates at each pool size — would strengthen this claim. As stated, the 14-point gap could also reflect that the top-10 by SimPO rerank happens to contain more relevant KOLs by construction, which would be a retrieval-quality effect rather than a reasoning-quality effect.
  3. §4.3, Table 2: The evidence audit is produced by Kimi-K2.6, the same model used as the frontier baseline throughout the paper. While the authors note that quote verification is model-independent (exact substring check), the selection of which quotes to verify and the grounding judgment itself inherit the auditor's biases, as acknowledged in §Limitations. Since the audit underpins the central claim that pointwise labels are noisy (and thus that the offline–end-to-end inversion is label-imposed rather than capacity-imposed), using the same model as both auditor and baseline competitor creates a potential conflict. The authors should discuss whether an alternative auditor (or human spot-check of the audit) was considered.
minor comments (7)
  1. §3.4, Eq. (1): The values of β and γ are not stated in the text. These are free parameters that affect the margin and calibration of the SimPO objective. Please report the chosen values and briefly justify them.
  2. §5.3: The relevance criterion uses binary majority voting with a tie-breaking rule (f=0.5 dropped). The number of annotators per KOL and inter-annotator agreement statistics are not reported. This information would help calibrate the reliability of the end-to-end labels.
  3. Table 6: The SimPO row reports 78.0 EM (32/41) on the indirect logp(Yes) path, matching Kimi-K2.6's 78.0 EM (32/41) on the direct judge path. The raw counts (32/41) are the same, but the items being judged differ (indirect scoring of individual KOLs vs. direct A-vs-B comparison). The text should clarify that these are not the same 32 correct items.
  4. §4.2: The T2 annotation interface did not provide a scoring rubric defining 0/1/2 boundaries. This is discussed later in §7 as a contributing factor to the inversion, but it would be more transparent to flag this design choice when the annotation protocol is first described, so readers understand the label quality before reaching §7.
  5. §6.4, Table 10: Wall-clock timings are noted as not like-for-like across models (different concurrency levels). The text correctly directs attention to the token axis, but the abstract's claim of '~20s on one A100' could be misread as a fair latency comparison. Consider adding a parenthetical in the abstract noting that wall-clock is indicative and cost comparisons rest on tokens.
  6. The abstract states 'a 50-KOL query in ~20s on one A100.' Table 10 reports 20.2s for the deployed configuration. It would be useful to clarify whether this includes retrieval time or only the rerank+reason stages.
  7. References [20] and [21] cite model release blogs rather than technical reports. If technical reports or model cards with architecture details are available, citing those would help reproducibility.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive reading. The three major comments are well-taken; we address each below and indicate revisions where the manuscript will change.

read point-by-point responses

  1. Referee: §6.3, Tables 8–9: The headline claim that the cascade 'matches frontier quality' rests on a 2.3-point P@5 margin (94.1 vs. 91.8) over n=11 queries. Set 2 (n=31) reverses the ordering. No confidence intervals, bootstrap distributions, or significance tests are reported. The claim should be softened or bootstrap CIs provided.

    Authors: The referee is correct on all counts. The 2.3-point margin on n=11 is not statistically distinguishable from zero, and Set 2 does reverse the ordering. We acknowledge that 'matches frontier quality' overstates what the data support. We will make two changes in the revision. First, we will soften the headline claim throughout the paper (abstract, §1.1, §6.3, §8) to 'competitive with frontier quality' or 'within frontier range,' and we will explicitly note the Set 2 reversal in §6.3 rather than framing it as 'gaps compress.' Second, we will add bootstrap 95% confidence intervals (10,000 resamples, stratified by query) for all P@5 comparisons in Tables 8 and 9. We expect these intervals to be wide and overlapping, which is itself the honest message: the 4B cascade and the frontier model are in the same ballpark, but we cannot claim superiority at these sample sizes. We agree this is the correct framing. revision: yes

  2. Referee: §6.3, Table 8: The 14-point gain from filtering (94.1 vs. 80.0) is counterintuitive and load-bearing but not directly tested. The gap could reflect retrieval-quality effects rather than reasoning-quality effects. A controlled experiment would strengthen this claim.

    Authors: This is a fair point and we agree the current explanation is plausible but untested. The referee's alternative hypothesis — that the top-10 by SimPO rerank simply contains more relevant KOLs by construction, so the reasoner starts from a better pool — is partially correct but does not fully explain the pattern. Note that 'reason on all 50' already has access to the same relevant KOLs; the question is why seeing more candidates hurts. That said, we cannot rule out that the larger pool introduces tie-breaking noise that depresses the reasoner's ranking, which is a retrieval-pool-size effect rather than a 'filtering removes dilution' effect. We will add a controlled experiment in the revision: we will measure tie rates (fraction of candidate pairs with identical summed scores) at pool sizes 10, 20, and 50, holding the reasoner fixed. If tie rates increase substantially at larger pool sizes, this directly supports the dilution mechanism. We will also report the number of relevant KOLs in the top-10 vs. top-50 pools per query, to quantify how much of the gap is a retrieval-quality effect. We agree that without this experiment the 14-point claim is under-supported as currently stated, and we will temper the language in §7 accordingly. revision: partial

  3. Referee: §4.3, Table 2: The evidence audit is produced by Kimi-K2.6, the same model used as the frontier baseline. The audit underpins the central claim that pointwise labels are noisy. Using the same model as auditor and baseline competitor creates a potential conflict. Should an alternative auditor or human spot-check be considered?

    Authors: We agree this is a legitimate concern. To be precise about what does and does not inherit the auditor's bias: the quote verification step (exact substring match against the profile text) is model-independent and mechanical — a quote either appears verbatim or it does not. What does inherit potential bias is (a) which quotes the auditor selects to extract and (b) the binary grounding judgment (is this quote sufficient evidence for the score?). These are genuine sources of auditor bias. We did not run an alternative auditor or human spot-check in the current study, and we cannot retroactively claim independence. In the revision we will: (1) add an explicit discussion of this conflict in §4.3 and §Limitations, acknowledging that the audit's central finding (score-2 labels are weakly grounded) could be partially an artifact of the auditor's quote-selection tendencies; (2) report inter-rater agreement on a small human spot-check of the audit if we can complete it before the revision deadline — we will attempt this but cannot guarantee it at this time. If the spot-check is not feasible within the revision window, we will at minimum reframe the audit's role: rather than presenting it as definitive proof that pointwise labels are noisy, we will present it as consistent with the label-noise hypothesis, noting that the offline–end-to-end inversion itself (SFT+GRPO wins offline, loses end-to-end) is a model-independent observation that holds regardless of the audit. The audit is corroborating evidence, not the sole basis for the claim. revision: partial

Circularity Check

0 steps flagged

No significant circularity found; the derivation is self-contained against external human-labeled benchmarks.

full rationale

The paper's central claims are evaluated against human-labeled binary pass/fail relevance judgments (E2E task, §4.2) that are independent of the trained models. The SimPO objective (Eq. 1) directly optimizes the logp(Yes) used at inference — this is training-serving consistency by design, not circularity; the paper explicitly states 'training and serving optimize the same score' (§3.4), which is a feature of the method, not a definitional reduction of the output to the input. The offline reranker EM is measured against held-out human pairwise labels (T10 test split, §5.1), and the end-to-end P@5 is measured against held-out human binary labels (Sets 1 and 2, §5.3). No prediction is renamed from a fitted parameter. The evidence audit (§4.3, Table 2) uses Kimi-K2.6 as auditor while Kimi-K2.6 is also the baseline, which is a minor concern, but the audit's grounding check (exact substring verification of profile quotes) is model-independent by the paper's own description, and the audit is used diagnostically rather than as load-bearing support for the headline accuracy claim. The one minor self-citation pattern — citing the authors' own prior work on best-worst scaling reliability [15] to justify the pairwise label geometry — is not load-bearing for the central result; the end-to-end evaluation directly tests whether pairwise supervision transfers, regardless of that citation. No step in the derivation chain reduces to its inputs by construction.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

No new entities, particles, forces, or dimensions are introduced. The paper is an applied systems paper using existing models and techniques.

free parameters (5)
  • β (SimPO inverse temperature) = not stated
    Controls the sharpness of the logistic loss in Eq. 1; value not reported in the paper.
  • γ (SimPO target margin) = not stated
    Fixed target margin in the SimPO loss (Eq. 1); value not reported.
  • top-50 retrieval cutoff = 50
    Chosen as the retrieval pool size; no sensitivity analysis provided.
  • top-10 rerank cutoff = 10
    Chosen as the shortlist size for the reason stage; no sensitivity analysis provided.
  • even sum aggregation s(k) = Σ si(k) = equal weights
    All five criteria weighted equally in the final ranking score; no justification or alternative weighting tested.
axioms (4)
  • domain assumption Relative (pairwise) judgments are more reliable than absolute rating scales at equal annotation budget.
    Invoked in §2 and §7 to explain the offline-online inversion; cited from [15]. This is a standard result from the annotation literature.
  • domain assumption The five-criterion framework (audience fit, content-product relevance, selling style, credibility, format) fully captures marketer intent for Thai KOL matching.
    Stated in §4.1; all criteria are generated to align to these five dimensions. No evidence that five is sufficient or that important criteria are omitted.
  • domain assumption Synthetic briefs generated by DeepSeek-V3.2 mirror the distribution of real proprietary Thai marketing briefs.
    Stated in §4.1 with 'schema-mirroring and web grounding'; acknowledged as a limitation in §Limitations. No quantitative distributional comparison provided.
  • domain assumption Binary majority vote (pass fraction > 0.5) is a valid ground-truth relevance signal for end-to-end evaluation.
    Defined in §5.3; the threshold and vote-counting method are not validated against an external gold standard.

pith-pipeline@v1.1.0-glm · 14796 in / 4062 out tokens · 250154 ms · 2026-07-08T19:39:03.834933+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 26 canonical work pages · 15 internal anchors

  1. [1]

    Dense passage retrieval for open-domain question answering

    Vladimir Karpukhin, Barlas Oğuz, Sewon Min, et al. Dense passage retrieval for open-domain question answering. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781. Association for Computational Linguistics,

    work page 2020

  2. [2]

    doi: 10.18653/v1/2020.emnlp-main.550

    work page doi:10.18653/v1/2020.emnlp-main.550 2020

  3. [3]

    Passage Re-ranking with BERT

    Rodrigo Nogueira and Kyunghyun Cho. Passage re-ranking with BERT.arXiv preprint arXiv:1901.04085, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1901

  4. [4]

    Multi-Stage Document Ranking with BERT

    Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, et al. Multi-stage document ranking with BERT. arXiv preprint arXiv:1910.14424, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1910

  5. [5]

    Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting

    Zhen Qin, Rolf Jagerman, Kai Hui, et al. Large language models are effective text rankers with pairwise ranking prompting. InFindings of the Association for Computational Linguistics: NAACL 2024. Association for Computational Linguistics, 2024. URLhttps://arxiv.org/ abs/2306.17563

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  6. [6]

    Document ranking with a pretrained sequence-to-sequence model

    Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, et al. Document ranking with a pretrained sequence-to-sequence model. InFindings of the Association for Computational Linguistics: EMNLP 2020, pages 708–718. Association for Computational Linguistics, 2020. doi: 10.18653/ v1/2020.findings-emnlp.63. URLhttps://aclanthology.org/2020.findings-emnlp.63/

    work page 2020

  7. [7]

    Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents

    Weiwei Sun, Lingyong Yan, Xinyu Ma, et al. Is ChatGPT good at search? investigating large language models as re-ranking agents. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 14918–14937, Singapore, 2023. Association for Computational Linguistics. URLhttps://arxiv.org/abs/2304.09542

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  8. [8]

    RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Models

    Ronak Pradeep, Sahel Sharifymoghaddam, and Jimmy Lin. RankVicuna: Zero-shot listwise document reranking with open-source large language models.arXiv preprint arXiv:2309.15088, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  9. [9]

    RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!

    Ronak Pradeep, Sahel Sharifymoghaddam, and Jimmy Lin. RankZephyr: Effective and robust zero-shot listwise reranking is a breeze!arXiv preprint arXiv:2312.02724, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  10. [10]

    Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, et al. Judging LLM-as-a-judge with MT-bench and chatbot arena. InAdvances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, 2023. URLhttps://arxiv.org/abs/2306.05685

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  11. [11]

    Prometheus: Inducing fine-grained evaluation capability in language models

    Seungone Kim, Jamin Shin, Yejin Cho, et al. Prometheus: Inducing fine-grained evaluation capability in language models. InInternational Conference on Learning Representations (ICLR),

    work page

  12. [12]

    URLhttps://arxiv.org/abs/2310.08491

    work page internal anchor Pith review Pith/arXiv arXiv

  13. [13]

    Large Language Models are not Fair Evaluators

    Peiyi Wang, Lei Li, Liang Chen, et al. Large language models are not fair evaluators. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9440–9450, Bangkok, Thailand, 2024. Association for Computational Linguistics. URLhttps://arxiv.org/abs/2305.17926

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  14. [14]

    Direct Preference Optimization: Your Language Model is Secretly a Reward Model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, et al. Direct preference optimization: Your language model is secretly a reward model. InAdvances in Neural Information Processing Systems (NeurIPS), 2023. URLhttps://arxiv.org/abs/2305.18290. 14

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  15. [15]

    SimPO: Simple Preference Optimization with a Reference-Free Reward

    Yu Meng, Mengzhou Xia, and Danqi Chen. SimPO: Simple preference optimization with a reference-free reward. InAdvances in Neural Information Processing Systems (NeurIPS), 2024. URLhttps://arxiv.org/abs/2405.14734

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  16. [16]

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, et al. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  17. [17]

    Mohammad

    Svetlana Kiritchenko and Saif M. Mohammad. Best-worst scaling more reliable than rating scales: A case study on sentiment intensity annotation. InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 465–470, Vancouver, Canada, 2017. Association for Computational Linguistics

    work page 2017

  18. [18]

    Offline and online evaluation of news rec- ommender systems at swissinfo.ch

    Florent Garcin, Boi Faltings, Olivier Donatsch, et al. Offline and online evaluation of news rec- ommender systems at swissinfo.ch. InProceedings of the 8th ACM Conference on Recommender Systems (RecSys), pages 169–176, 2014. doi: 10.1145/2645710.2645745

    work page doi:10.1145/2645710.2645745 2014

  19. [19]

    LoRA: Low-Rank Adaptation of Large Language Models

    Edward J. Hu, Yelong Shen, Phillip Wallis, et al. LoRA: Low-rank adaptation of large language models. InInternational Conference on Learning Representations (ICLR), 2022. URL https://arxiv.org/abs/2106.09685

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  20. [20]

    Nemotron-personas-USA: Synthesized personas grounded in real-world distribu- tions

    NVIDIA. Nemotron-personas-USA: Synthesized personas grounded in real-world distribu- tions. https://huggingface.co/datasets/nvidia/Nemotron-Personas-USA, 2025. CC BY 4.0. Generated with NeMo Data Designer

    work page 2025

  21. [21]

    DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models

    DeepSeek-AI. DeepSeek-V3.2: Pushing the frontier of open large language models.arXiv preprint arXiv:2512.02556, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  22. [22]

    Qwen3.5: Towards native multimodal agents.https://qwen.ai/blog?id=qwen3

    Qwen Team. Qwen3.5: Towards native multimodal agents.https://qwen.ai/blog?id=qwen3. 5, 2026. Release blog, February 2026. Small model series (0.8B–9B) released March 2026

    work page 2026

  23. [23]

    Kimi K2.6

    Moonshot AI. Kimi K2.6. https://huggingface.co/moonshotai/Kimi-K2.6, 2026. Model card and technical blog, April 2026

    work page 2026

  24. [24]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, et al. Chain-of-thought prompting elicits reasoning in large language models. InAdvances in Neural Information Processing Systems (NeurIPS),

    work page

  25. [25]

    URLhttps://arxiv.org/abs/2201.11903

    work page internal anchor Pith review Pith/arXiv arXiv

  26. [26]

    The “problem” of human label variation: On ground truth in data, modeling and evaluation

    Barbara Plank. The “problem” of human label variation: On ground truth in data, modeling and evaluation. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, United Arab Emirates, 2022. Association for Computational Linguistics. 15

    work page 2022