Competition-Level Code Generation with AlphaCode

arxiv: 2203.07814 · v1 · submitted 2022-02-08 · 💻 cs.PL · cs.AI· cs.LG

Competition-Level Code Generation with AlphaCode

Yujia Li , David Choi , Junyoung Chung , Nate Kushman , Julian Schrittwieser , R\'emi Leblond , Tom Eccles , James Keeling

show 18 more authors

Felix Gimeno Agustin Dal Lago Thomas Hubert Peter Choy Cyprien de Masson d'Autume Igor Babuschkin Xinyun Chen Po-Sen Huang Johannes Welbl Sven Gowal Alexey Cherepanov James Molloy Daniel J. Mankowitz Esme Sutherland Robson Pushmeet Kohli Nando de Freitas Koray Kavukcuoglu Oriol Vinyals

This is my paper

Pith reviewed 2026-05-18 06:39 UTC · model grok-4.3

classification 💻 cs.PL cs.AIcs.LG

keywords code generationcompetitive programmingtransformer modelsprogram samplingbehavioral filteringCodeforceslarge language models

0 comments p. Extension

The pith

AlphaCode generates novel solutions to complex programming problems by sampling many programs and filtering them according to observed behavior on test cases.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to demonstrate that large language models can tackle competition-level programming tasks requiring algorithm understanding and complex reasoning rather than simple translation of instructions. It reports that a system called AlphaCode reaches an average ranking in the top 54.3 percent of participants across recent Codeforces contests with more than 5,000 entrants. The authors identify three necessary ingredients: a large clean dataset of competitive programming problems, efficient large transformer models, and the practice of drawing thousands of samples per problem then keeping only those that pass observed test behavior. If the approach holds, it shows a practical route from current code-generation models to systems that can produce working solutions for unseen, algorithmically demanding problems.

Core claim

By training large transformer models on a curated collection of competitive programming problems and solutions, then generating thousands of candidate programs for each new problem and retaining a small number whose execution matches the behavior shown on available tests, AlphaCode produces submissions that rank on average in the top 54.3 percent of participants in simulated Codeforces contests with over 5,000 entrants.

What carries the argument

Large-scale sampling from transformer models followed by filtering of candidates according to their execution behavior on provided test cases.

If this is right

Code-generation systems can move from completing short tasks to producing complete, algorithmically nontrivial programs when allowed to explore many possibilities and discard those that fail observed tests.
Performance on competition problems improves substantially once sampling volume is increased and a reliable filter based on program execution is applied.
Training on a large, high-quality dataset of competition problems is a prerequisite for the models to internalize the necessary algorithmic patterns.
The same sampling-plus-filtering recipe could be applied to other structured generation tasks that admit cheap behavioral checks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If behavioral filtering continues to work without overfitting, future systems may rely more on search volume than on perfect single-shot prediction.
Real deployments would need ways to generate or approximate test cases when gold tests are unavailable.
The method invites experiments that replace the current filter with learned verifiers or formal proof checkers.
Similar techniques might transfer to generating solutions in mathematics or scientific modeling where candidate outputs can be checked against known cases.

Load-bearing premise

Selecting a few programs from a large sample on the basis of how they behave on test cases reliably identifies correct solutions for problems the model has never seen.

What would settle it

Running the same sampling-and-filtering procedure on a fresh set of Codeforces problems released after the model's training data and checking whether the submitted solutions still pass the hidden test suites at the reported rate.

read the original abstract

Programming is a powerful and ubiquitous problem-solving tool. Developing systems that can assist programmers or even generate programs independently could make programming more productive and accessible, yet so far incorporating innovations in AI has proven challenging. Recent large-scale language models have demonstrated an impressive ability to generate code, and are now able to complete simple programming tasks. However, these models still perform poorly when evaluated on more complex, unseen problems that require problem-solving skills beyond simply translating instructions into code. For example, competitive programming problems which require an understanding of algorithms and complex natural language remain extremely challenging. To address this gap, we introduce AlphaCode, a system for code generation that can create novel solutions to these problems that require deeper reasoning. In simulated evaluations on recent programming competitions on the Codeforces platform, AlphaCode achieved on average a ranking of top 54.3% in competitions with more than 5,000 participants. We found that three key components were critical to achieve good and reliable performance: (1) an extensive and clean competitive programming dataset for training and evaluation, (2) large and efficient-to-sample transformer-based architectures, and (3) large-scale model sampling to explore the search space, followed by filtering based on program behavior to a small set of submissions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

AlphaCode reaches top-half Codeforces rankings by sampling a million programs and filtering on sample-test behavior, but that filtering step is the part that still needs checking.

read the letter

AlphaCode reaches top-half Codeforces rankings by sampling a million programs and filtering on sample-test behavior, but that filtering step is the part that still needs checking. The new piece is the full pipeline: a clean competitive-programming dataset, large efficient transformers, and then generating far more candidates than usual before keeping only the ones whose outputs match on the visible samples. That combination is what moves the needle on problems that actually require algorithmic reasoning rather than simple translation. The paper is straightforward about which three ingredients mattered and shows the ranking result on recent contests with thousands of participants. That is concrete and worth having in the literature. The soft spot is the one the stress test flags. Filtering works only if the sample inputs catch the same failures that the hidden tests will. If the hidden cases have different distributions or extra edges, programs can pass the filter yet fail real judging, which would make the simulated ranking look stronger than it is. The abstract does not give ablations on how much the filter adds versus just sampling more, nor does it detail checks for leakage between the training set and the evaluation contests. Those details matter because the headline number rests on the filter doing its job reliably. This is for anyone working on code generation or on turning language models into tools that can handle non-trivial programming tasks. The empirical result is sharp enough and the method is simple enough to build on that it deserves a serious referee to verify the evaluation protocol and see how far the approach generalizes.

Referee Report

2 major / 1 minor

Summary. The paper introduces AlphaCode, a transformer-based code generation system trained on a large competitive programming dataset. It claims that large-scale sampling of solutions followed by filtering based on observed behavior on public sample inputs enables the system to achieve an average ranking of top 54.3% in simulated evaluations on recent Codeforces competitions with more than 5,000 participants. The authors identify three critical components: the training dataset, large efficient transformer architectures, and the sampling-plus-filtering pipeline.

Significance. If the results hold under rigorous scrutiny, this constitutes a notable advance in automated program synthesis by showing that scaling sampling volume and behavioral filtering can produce competition-level solutions to problems requiring algorithmic reasoning, rather than simple translation of specifications. The approach of generating ~1M candidates and reducing them via execution on visible tests offers a concrete, scalable technique that could transfer to other synthesis domains.

major comments (2)

The simulated evaluation procedure that produces the headline top-54.3% ranking (abstract) is insufficiently specified with respect to the filtering step. The manuscript must clarify how clustering or deduplication by output behavior on the public sample inputs provided in problem statements avoids selecting programs that pass visible tests yet fail the private test suites that determine real Codeforces scores. Because the abstract explicitly lists 'large-scale model sampling ... followed by filtering based on program behavior' as one of the three critical components, any gap here directly undermines the central performance claim.
No quantitative ablations, confidence intervals, or variance estimates are reported for the ranking metric or for the individual contributions of the three key components. Without these, it is impossible to assess whether the observed performance is robust or sensitive to the free parameters (sampling volume and filtering thresholds) listed in the axiom ledger.

minor comments (1)

The abstract refers to 'large and efficient-to-sample transformer-based architectures' without giving model sizes, parameter counts, or sampling throughput numbers; adding these would improve reproducibility and allow readers to gauge the scale relative to prior work.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments on our manuscript. We address each major comment below, providing clarifications and indicating where revisions will be made to improve the description of our methods and results.

read point-by-point responses

Referee: The simulated evaluation procedure that produces the headline top-54.3% ranking (abstract) is insufficiently specified with respect to the filtering step. The manuscript must clarify how clustering or deduplication by output behavior on the public sample inputs provided in problem statements avoids selecting programs that pass visible tests yet fail the private test suites that determine real Codeforces scores. Because the abstract explicitly lists 'large-scale model sampling ... followed by filtering based on program behavior' as one of the three critical components, any gap here directly undermines the central performance claim.

Authors: We thank the referee for this observation and agree that greater detail on the filtering procedure will strengthen the manuscript. The process generates approximately one million candidate solutions per problem and executes them on the visible sample inputs from the problem statement. We then cluster solutions by their output behavior on these samples (using exact output matching after normalization) and select a small number (typically up to 10) of representative programs from the clusters to form the final submission set. This selection is performed solely to respect the real-world limit on submissions per problem in Codeforces contests; the final ranking is obtained by submitting these programs to the actual Codeforces judge, which evaluates them on the private test suites. The filtering therefore does not attempt to predict or guarantee performance on hidden tests—it only ensures the submitted programs pass the visible samples while covering diverse behaviors. We will revise the methods section to include a precise description of the clustering algorithm, behavior equivalence definition, and selection criteria, along with a diagram of the pipeline, to eliminate any ambiguity. revision: yes
Referee: No quantitative ablations, confidence intervals, or variance estimates are reported for the ranking metric or for the individual contributions of the three key components. Without these, it is impossible to assess whether the observed performance is robust or sensitive to the free parameters (sampling volume and filtering thresholds) listed in the axiom ledger.

Authors: We acknowledge the value of quantitative ablations and statistical measures for evaluating robustness. The three components were identified through iterative development experiments, but the submitted manuscript focused on the end-to-end system performance rather than exhaustive ablations. In the revision we will add a dedicated section presenting smaller-scale ablations that isolate the effects of training dataset composition, model size, and sampling volume on a held-out set of problems. We will also report the range and standard deviation of per-contest rankings across the evaluated Codeforces rounds as a measure of variance for the headline metric. For sensitivity to sampling volume and filtering thresholds, we will include performance curves showing how the number of solved problems changes with these parameters. Full-scale ablations matching the main experimental budget are not feasible within a reasonable revision timeline due to computational cost, but the added smaller-scale studies and variance reporting will allow readers to assess sensitivity. revision: partial

Circularity Check

0 steps flagged

No significant circularity; results benchmarked on external Codeforces platform

full rationale

The paper reports an empirical performance result (average top 54.3% ranking in Codeforces competitions) obtained by submitting filtered program samples to an independent external platform whose private test suites and ranking mechanics are outside the paper's control or definitions. The three critical components (dataset, model scale, sampling+filtering) are described as engineering choices whose effectiveness is measured against that external benchmark rather than derived from internal equations or self-referential fits. No self-definitional loops, fitted inputs renamed as predictions, or load-bearing self-citations appear in the chain leading to the headline ranking claim. The evaluation therefore remains self-contained against an external standard.

Axiom & Free-Parameter Ledger

2 free parameters · 1 axioms · 0 invented entities

The headline result depends on the representativeness of the competitive-programming training set and on the unstated details of how behavior-based filtering selects submissions; these are treated as given rather than derived.

free parameters (2)

sampling volume
Number of programs generated per problem is chosen to balance coverage and compute; exact count not derivable from first principles.
filtering thresholds
Criteria for keeping or discarding programs based on execution behavior are tuned to produce a small submission set.

axioms (1)

domain assumption A sufficiently large and clean dataset of competitive programming problems and solutions exists and is representative of unseen contest problems.
The abstract lists this as the first critical component without independent verification of coverage or absence of leakage.

pith-pipeline@v0.9.0 · 5865 in / 1228 out tokens · 35164 ms · 2026-05-18T06:39:26.191298+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

PhiForcing phi_equation unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

AlphaCode achieved on average a ranking of top 54.3% in competitions with more than 5,000 participants

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 19 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Primal Generation, Dual Judgment: Self-Training from Test-Time Scaling
cs.LG 2026-05 unverdicted novelty 7.0

DuST uses on-policy RL to train code models on ranking their own sampled solutions by sandbox execution correctness, improving judgment NDCG, pass@1, and Best-of-4 accuracy while showing that SFT on the same data does...
Primal Generation, Dual Judgment: Self-Training from Test-Time Scaling
cs.LG 2026-05 conditional novelty 7.0

DuST self-trains LLMs for code generation by ranking their own test-time samples via sandbox execution and applying GRPO, improving judgment by +6.2 NDCG and single-sample pass@1 by +3.1 on LiveCodeBench.
MR-Coupler: Automated Metamorphic Test Generation via Functional Coupling Analysis
cs.SE 2026-04 conditional novelty 7.0

MR-Coupler leverages functional coupling analysis and LLMs to generate valid metamorphic test cases for over 90% of tasks while detecting 44% of real bugs, outperforming baselines by 64.90% in validity and 36.56% in f...
Automating Database-Native Function Code Synthesis with LLMs
cs.DB 2026-04 conditional novelty 7.0

DBCooker automates synthesis of database native functions via LLM-guided characterization, coding plans, hybrid filling, and progressive validation, delivering 34.55% higher accuracy than baselines on SQLite, PostgreS...
Synthesis-in-the-Loop Evaluation of LLMs for RTL Generation: Quality, Reliability, and Failure Modes
cs.AR 2026-03 unverdicted novelty 7.0

LLM evaluation for RTL generation identifies three performance tiers with frontier models reaching high synthesis quality and reveals systematic failure differences between proprietary and open models.
Massive Activations in Large Language Models
cs.CL 2024-02 unverdicted novelty 7.0

Massive activations are constant large values in LLMs that function as indispensable bias terms and concentrate attention probabilities on specific tokens.
Voyager: An Open-Ended Embodied Agent with Large Language Models
cs.AI 2023-05 unverdicted novelty 7.0

Voyager achieves superior lifelong learning in Minecraft by combining an automatic exploration curriculum, a library of executable skills, and iterative LLM prompting with environment feedback, yielding 3.3x more uniq...
InCoder: A Generative Model for Code Infilling and Synthesis
cs.SE 2022-04 unverdicted novelty 7.0

InCoder is the first generative model to directly perform zero-shot code infilling via bidirectional context from a masked-then-appended training scheme, matching left-to-right models on synthesis while improving on t...
Schedule-and-Calibrate: Utility-Guided Multi-Task Reinforcement Learning for Code LLMs
cs.SE 2026-05 unverdicted novelty 6.0

ASTOR improves a single code LLM across four tasks by 9.0-9.5% over the best specialist and 7.5-12.8% over prior multi-task RL baselines via utility-driven data scheduling and adaptive KL regularization.
AI-Generated Smells: An Analysis of Code and Architecture in LLM and Agent-Driven Development
cs.SE 2026-05 unverdicted novelty 6.0

More capable LLMs and agents generate code with greater volume and architectural decay, following a Volume-Quality Inverse Law that neither functional correctness nor prompting mitigates.
Tracing the Thought of a Grandmaster-level Chess-Playing Transformer
cs.LG 2026-04 unverdicted novelty 6.0

Sparse replacement layers decompose the MLP and attention modules of a chess-playing transformer to reveal verifiable tactical reasoning pathways and parallel computation patterns.
SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding
cs.DC 2026-02 unverdicted novelty 6.0

SPEED-Bench is a new standardized benchmark for speculative decoding that supplies semantically diverse qualitative data and throughput-oriented splits across concurrency levels, integrated with vLLM and TensorRT-LLM.
Process Reinforcement through Implicit Rewards
cs.LG 2025-02 conditional novelty 6.0

PRIME enables online process reward model updates in LLM RL using implicit rewards from rollouts and outcome labels, yielding 15.1% average gains on reasoning benchmarks and surpassing a stronger instruct model with 1...
Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive
cs.CL 2024-02 conditional novelty 6.0

DPOP is a new loss function that prevents DPO from lowering preferred response likelihoods and outperforms standard DPO on diverse datasets, MT-Bench, and enables Smaug-72B to exceed 80% on the Open LLM Leaderboard.
CodeT5+: Open Code Large Language Models for Code Understanding and Generation
cs.CL 2023-05 conditional novelty 6.0

CodeT5+ is a flexible encoder-decoder LLM family for code pretrained with diverse objectives on multilingual corpora and initialized from existing LLMs, achieving state-of-the-art results on code generation, completio...
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
cs.CL 2022-11 unverdicted novelty 6.0

BLOOM is a 176B-parameter open-access multilingual language model trained on the ROOTS corpus that achieves competitive performance on benchmarks, with improved results after multitask prompted finetuning.
Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models
cs.LG 2025-10 unverdicted novelty 5.0

GenCluster scales test-time compute via large-scale generation, behavioral clustering, ranking, and round-robin submission to achieve IOI gold medal performance with the open-weight gpt-oss-120b model.
PaLM 2 Technical Report
cs.CL 2023-05 unverdicted novelty 5.0

PaLM 2 reports state-of-the-art results on language, reasoning, and multilingual tasks with improved efficiency over PaLM.
StarCoder: may the source be with you!
cs.CL 2023-05 accept novelty 5.0

StarCoderBase matches or beats OpenAI's code-cushman-001 on multi-language code benchmarks; the Python-fine-tuned StarCoder reaches 40% pass@1 on HumanEval while retaining other-language performance.

Reference graph

Works this paper leans on

14 extracted references · 14 canonical work pages · cited by 18 Pith papers · 2 internal anchors

[1]

Accessed: 2021-12-04. S. Edunov, M. Ott, M. Auli, and D. Grangier. Understanding back-translation at scale. InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 489–500, Brussels, Belgium, Oct.-Nov. 2018. Association for Computational Linguistics. doi: 10.18653/v1/D18-1045. URL https://aclanthology.org/D18-1045. ...

work page doi:10.18653/v1/d18-1045 2021
[2]

Scaling Laws for Neural Language Models

Accessed: 2021-12-04. ICPC Rules. ICPC rules.https://icpc.global/worldfinals/rules, 2021. Accessed: 2021- 12-09. IOI. International olympiad in informatics. https://ioinformatics.org/, 2021. Accessed: 2021-12-04. N. P. Jouppi, D. H. Yoon, M. Ashcraft, M. Gottscho, T. B. Jablin, G. Kurian, J. Laudon, S. Li, P. Ma, X. Ma, et al. Ten lessons from three gener...

work page internal anchor Pith review Pith/arXiv arXiv doi:10.1145/362566.362568 2021
[3]

Accessed: 2021-12-04. V. Murali, L. Qi, S. Chaudhuri, and C. Jermaine. Neural sketch learning for conditional program generation. arXiv preprint arXiv:1703.05698, 2017. Y. Nandwani, D. Jindal, Mausam, and P. Singla. Neural learning of one-of-many solutions for combi- natorial problems in structured output spaces. InInternational Conference on Learning Rep...

work page internal anchor Pith review Pith/arXiv arXiv 2021
[4]

38 A.2 Program judging

Appendix A Problem setup 38 A.1 Hidden tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 A.2 Program judging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 A.3 Evaluation metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 B Datasets 41 B.1 GitHub dat...

work page 2022
[5]

Submissions for duplicate problems were merged

Removed problems that are duplicates of each other, ignoring whitespace. Submissions for duplicate problems were merged

work page
[6]

Removed submissions that are duplicates of others, ignoring whitespace

work page
[7]

CleanedC++submissionstocompilewithourcompilerandsandboxes,forexamplebyadding int in front ofmain() where it was missing. We further formatted C++code usingclang-format, replaced all the includes with#include <bits/stdc++.h> , and expanded all other prepro- cessor directives andtypedefs,15 which also made the code shorter. Note that this cleaning 15#define...

work page
[8]

divide and conquer

Executed all Python and C++solutions on all the test cases, and removed in order: (a) all submissions that pass no tests, (b) all tests that less than 10% of the remaining submissions produce non-empty outputs on, (c) all submissions that pass less than 10% of the remaining tests. B.3. Data leakage and temporal split Transformer language models trained on...

work page 2020
[9]

min(2, 4) = 4 . 2 = 8. * f(1, 3) = max(a_1, a_2, a_3) . min(a_1, a_2, a_3) = max(2, 4, 3) . min(2, 4, 3) = 4 . 2 = 8. * f(2, 3) = max(a_2, a_3) . min(a_2, a_3) = max(4,

work page
[10]

min(4, 3) = 4 . 3 = 12. 1554A Cherry – Simpliﬁed You are given n integers a_1, a_2, ..., a_n. Find the maximum value of a_l times a_{l + 1} for an integer l for which 1 <= l < n. Input The first line contains a single integer t (1 <= t <= 10 000) - the number of test cases. The first line of each test case contains a single integer n (2 <= n <= 10^5). The...

work page
[11]

Each test case consists of two lines

- the number of test cases. Each test case consists of two lines. The first line of each test case contains a single integer n (1 <= n <= 100) - the length of the sequence. The second line of each test case contains n integers a_1, a_2, ..., a_n (0 <= a_i <= 10^9). Output For each test case, print one integer - the minimal value of the maximum value in th...

work page
[12]

baba" and

- the number of test cases. Each test case consists of two lines. The first line of each test case contains a single integer n (1 <= n <= 100) - the length of the sequence. The second line of each test case contains n integers a_1, a_2, ..., a_n (0 <= a_i <= 10^9). Output For each test case, print one integer - the bitwise AND of all elements of a. Exampl...

work page
[13]

abbaba" and substring

- the number of testcases. Then the descriptions of t testcases follow. The first line of the testcase contains a single integer n (1 <= n <= 50) - the length of the string. The second line of the testcase contains a string s, consisting of n letters, each letter is either ’a’ or ’b’. Output For each testcase print two integers. If there exists a non-empt...

work page
[14]

Then the descriptions of t testcases follow

- the number of testcases. Then the descriptions of t testcases follow. The first line of the testcase contains a single integer n (1 <= n <= 50) - the length of the string. The second line of the testcase contains a string s, consisting of n letters, each letter is either ’a’ or ’b’. Output For each testcase print two integers. If there is an adjacent pa...

work page

[1] [1]

Accessed: 2021-12-04. S. Edunov, M. Ott, M. Auli, and D. Grangier. Understanding back-translation at scale. InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 489–500, Brussels, Belgium, Oct.-Nov. 2018. Association for Computational Linguistics. doi: 10.18653/v1/D18-1045. URL https://aclanthology.org/D18-1045. ...

work page doi:10.18653/v1/d18-1045 2021

[2] [2]

Scaling Laws for Neural Language Models

Accessed: 2021-12-04. ICPC Rules. ICPC rules.https://icpc.global/worldfinals/rules, 2021. Accessed: 2021- 12-09. IOI. International olympiad in informatics. https://ioinformatics.org/, 2021. Accessed: 2021-12-04. N. P. Jouppi, D. H. Yoon, M. Ashcraft, M. Gottscho, T. B. Jablin, G. Kurian, J. Laudon, S. Li, P. Ma, X. Ma, et al. Ten lessons from three gener...

work page internal anchor Pith review Pith/arXiv arXiv doi:10.1145/362566.362568 2021

[3] [3]

Accessed: 2021-12-04. V. Murali, L. Qi, S. Chaudhuri, and C. Jermaine. Neural sketch learning for conditional program generation. arXiv preprint arXiv:1703.05698, 2017. Y. Nandwani, D. Jindal, Mausam, and P. Singla. Neural learning of one-of-many solutions for combi- natorial problems in structured output spaces. InInternational Conference on Learning Rep...

work page internal anchor Pith review Pith/arXiv arXiv 2021

[4] [4]

38 A.2 Program judging

Appendix A Problem setup 38 A.1 Hidden tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 A.2 Program judging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 A.3 Evaluation metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 B Datasets 41 B.1 GitHub dat...

work page 2022

[5] [5]

Submissions for duplicate problems were merged

Removed problems that are duplicates of each other, ignoring whitespace. Submissions for duplicate problems were merged

work page

[6] [6]

Removed submissions that are duplicates of others, ignoring whitespace

work page

[7] [7]

CleanedC++submissionstocompilewithourcompilerandsandboxes,forexamplebyadding int in front ofmain() where it was missing. We further formatted C++code usingclang-format, replaced all the includes with#include <bits/stdc++.h> , and expanded all other prepro- cessor directives andtypedefs,15 which also made the code shorter. Note that this cleaning 15#define...

work page

[8] [8]

divide and conquer

Executed all Python and C++solutions on all the test cases, and removed in order: (a) all submissions that pass no tests, (b) all tests that less than 10% of the remaining submissions produce non-empty outputs on, (c) all submissions that pass less than 10% of the remaining tests. B.3. Data leakage and temporal split Transformer language models trained on...

work page 2020

[9] [9]

min(2, 4) = 4 . 2 = 8. * f(1, 3) = max(a_1, a_2, a_3) . min(a_1, a_2, a_3) = max(2, 4, 3) . min(2, 4, 3) = 4 . 2 = 8. * f(2, 3) = max(a_2, a_3) . min(a_2, a_3) = max(4,

work page

[10] [10]

min(4, 3) = 4 . 3 = 12. 1554A Cherry – Simpliﬁed You are given n integers a_1, a_2, ..., a_n. Find the maximum value of a_l times a_{l + 1} for an integer l for which 1 <= l < n. Input The first line contains a single integer t (1 <= t <= 10 000) - the number of test cases. The first line of each test case contains a single integer n (2 <= n <= 10^5). The...

work page

[11] [11]

Each test case consists of two lines

- the number of test cases. Each test case consists of two lines. The first line of each test case contains a single integer n (1 <= n <= 100) - the length of the sequence. The second line of each test case contains n integers a_1, a_2, ..., a_n (0 <= a_i <= 10^9). Output For each test case, print one integer - the minimal value of the maximum value in th...

work page

[12] [12]

baba" and

- the number of test cases. Each test case consists of two lines. The first line of each test case contains a single integer n (1 <= n <= 100) - the length of the sequence. The second line of each test case contains n integers a_1, a_2, ..., a_n (0 <= a_i <= 10^9). Output For each test case, print one integer - the bitwise AND of all elements of a. Exampl...

work page

[13] [13]

abbaba" and substring

- the number of testcases. Then the descriptions of t testcases follow. The first line of the testcase contains a single integer n (1 <= n <= 50) - the length of the string. The second line of the testcase contains a string s, consisting of n letters, each letter is either ’a’ or ’b’. Output For each testcase print two integers. If there exists a non-empt...

work page

[14] [14]

Then the descriptions of t testcases follow

- the number of testcases. Then the descriptions of t testcases follow. The first line of the testcase contains a single integer n (1 <= n <= 50) - the length of the string. The second line of the testcase contains a string s, consisting of n letters, each letter is either ’a’ or ’b’. Output For each testcase print two integers. If there is an adjacent pa...

work page