Pith. sign in

REVIEW 1 cited by

CS1QA: A Dataset for Assisting Code-based Question Answering in an Introductory Programming Course

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2210.14494 v1 pith:4TCQ57DR submitted 2022-10-26 cs.CL

Changyoon Lee , Yeon Seonwoo , Alice Oh This is my paper
classification cs.CL
keywords codequestioncs1qaansweringdatasetprogrammingchatcode-based
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We introduce CS1QA, a dataset for code-based question answering in the programming education domain. CS1QA consists of 9,237 question-answer pairs gathered from chat logs in an introductory programming class using Python, and 17,698 unannotated chat data with code. Each question is accompanied with the student's code, and the portion of the code relevant to answering the question. We carefully design the annotation process to construct CS1QA, and analyze the collected dataset in detail. The tasks for CS1QA are to predict the question type, the relevant code snippet given the question and the code and retrieving an answer from the annotated corpus. Results for the experiments on several baseline models are reported and thoroughly analyzed. The tasks for CS1QA challenge models to understand both the code and natural language. This unique dataset can be used as a benchmark for source code comprehension and question answering in the educational setting.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SWE-QA: Can Language Models Answer Repository-level Code Questions?

    cs.CL 2025-09 unverdicted novelty 7.0 of 10

    SWE-QA creates a new repository-level code QA benchmark with 576 pairs and an agentic LLM framework, showing promise but open challenges for models handling complex codebases.

Pith tools