{"id":"78aff705-37ca-457e-9cec-d8cad05da536","arxiv_id":"2606.21837","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":3.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"G-Issue mines issue artifacts faster than Python APIs but uses more memory, and lifetime/evolution data can help prioritize issues in open source projects.","lead":"This paper introduces G-Issue, a tool for mining issue-related artifacts from open source repositories, and compares its performance to other tools while analyzing issue lifetime and evolution. The analysis aims to support better prioritization of issues for software quality and developer management.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"G-Issue extraction accuracy lacks ground-truth validation, undermining both performance and prioritization results","rationale":"The reader's weakest assumption matches the load-bearing point exactly. The abstract-only review already flagged it; the full text does not appear to add the missing validation step, so the empirical claims remain conditional on that check.","tokens_in":1672,"tokens_out":259,"duration_ms":20462,"concrete_test":"Pick the smallest repository analyzed in the paper; run G-Issue and the official GitHub API to extract the full set of issues plus comments/events; compute precision/recall on issue IDs and comment counts; if either metric falls below 95%, the accuracy assumption fails.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The performance comparison (G-Issue faster but higher memory than 'general Python API') and the claim that lifetime/evolution data enable prioritization both presuppose that G-Issue extracts issue artifacts completely and correctly. No section describes a validation step (e.g., manual audit or cross-check against GitHub REST responses) that would detect systematic omissions, duplicate handling errors, or missing event timelines. If extraction is incomplete, the reported timings are on a non-comparable subset and the evolution statistics are biased.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces G-Issue, a tool for mining issue-related artifacts from open source repositories. It compares the tool's performance against state-of-the-art alternatives, reporting that G-Issue is faster but uses more memory. It further examines issue lifetime and evolution patterns across well-known maintained repositories and concludes that these patterns support issue prioritization for management and developer assignment.","tokens_in":1769,"tokens_out":345,"duration_ms":26700,"significance":"If the extraction accuracy were validated and the performance/prioritization analyses rested on documented datasets and statistical procedures, the work could supply practical empirical guidance for issue tracking in open-source projects.","major_comments":[{"comment":"Abstract: performance and prioritization conclusions are stated without any description of the datasets used, statistical methods, error handling, or validation procedures, so the claims cannot be assessed from the given text.","section":"Abstract"},{"comment":"Results (performance and prioritization sections): both the claim that G-Issue is faster yet higher-memory than the general Python API and the claim that lifetime/evolution data enable prioritization presuppose that G-Issue extracts issue artifacts completely and correctly; no ground-truth validation step (manual audit, cross-check against GitHub REST responses, or duplicate-handling audit) is described, which is load-bearing for the reported timings and evolution statistics.","section":"Results"}],"minor_comments":[{"comment":"Abstract: the phrase 'during mining issue mining' is redundant and should be rephrased for clarity.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive comments and the recommendation for major revision. The points raised about the abstract and the need for validation of extraction accuracy are important for strengthening the manuscript. We address each major comment below and will incorporate revisions accordingly.","responses":[{"response":"We agree that the abstract would benefit from greater specificity to allow assessment of the claims. In the revised version, we will expand the abstract to reference the specific well-known open-source repositories examined, the direct timing and memory measurements against the Python GitHub API, and the descriptive (rather than inferential) nature of the lifetime and evolution patterns used for prioritization. This change will improve transparency without exceeding abstract length limits.","revision_made":"yes","referee_comment":"[Abstract] Abstract: performance and prioritization conclusions are stated without any description of the datasets used, statistical methods, error handling, or validation procedures, so the claims cannot be assessed from the given text."},{"response":"The referee correctly identifies that the performance and prioritization results rest on the assumption of accurate extraction. The manuscript describes G-Issue's use of the GitHub REST API but does not present an explicit validation procedure. We will add a dedicated validation subsection that reports a manual audit of a random sample of extracted issues against direct GitHub API responses, along with discussion of duplicate handling. This will directly address the load-bearing concern for both the timing results and the evolution statistics.","revision_made":"yes","referee_comment":"[Results] Results (performance and prioritization sections): both the claim that G-Issue is faster yet higher-memory than the general Python API and the claim that lifetime/evolution data enable prioritization presuppose that G-Issue extracts issue artifacts completely and correctly; no ground-truth validation step (manual audit, cross-check against GitHub REST responses, or duplicate-handling audit) is described, which is load-bearing for the reported timings and evolution statistics."}],"tokens_in":1261,"tokens_out":418,"duration_ms":18860,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper builds and releases a tool called G-Issue for pulling issue-related artifacts from open source repositories, then compares its speed and memory use against a general Python API and looks at how issues evolve over time in a few well-known repos. It suggests the lifetime data could support prioritization for developer assignment and quality work.\n\nThe tool release itself is straightforward and the URL is given, which lets others inspect the code. The focus on practical issue management in open source projects is a reasonable target for software engineering work.\n\nThe soft spots are more substantial. The abstract and description give no account of how G-Issue was checked for completeness or correctness against the actual GitHub data. No manual audits, cross-checks with REST responses, or handling of duplicates and missing events are mentioned. Without that step, the reported timing advantage and the evolution statistics are both on uncertain ground; any systematic omissions would make the performance comparison non-comparable and the prioritization suggestion unsupported. The paper also gives no list of the specific repositories, no statistical methods, and no error analysis, so the results cannot be reproduced or assessed from the text.\n\nThis follows the standard pattern of repository-mining studies already common in the field and adds little that is distinct. The stress-test concern about missing ground-truth validation holds up on the available description.\n\nThe work is aimed at practitioners or students who want a ready tool for issue tracking rather than researchers seeking new methods or rigorously validated findings. It does not show the methodological detail or evidential sharpness needed for a serious referee process.","headline":"G-Issue is a new mining tool with performance numbers and lifetime observations, but the work provides no validation of extraction accuracy so the claims rest on unverified output.","tokens_in":2227,"tokens_out":391,"would_cite":false,"duration_ms":18546,"reading_group":"no","serious_thinker":"unclear","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"G-Issue mines issue artifacts faster than standard Python APIs while using more memory and supports prioritization by issue lifetime.","keywords":["issue mining","open source repositories","software issues","issue lifetime","issue evolution","software quality","G-Issue tool"],"falsifier":"Direct head-to-head timing and memory measurements of G-Issue versus a standard Python API on identical large repositories, plus manual spot-checks of extracted artifacts against the original repository contents.","tokens_in":2586,"feed_emoji":"🔧","tokens_out":535,"duration_ms":10539,"temperature":0.7,"pith_summary":"The paper presents G-Issue, a tool designed to extract issue-related artifacts such as bugs, errors, and documentation gaps from open source repositories. It benchmarks G-Issue against general Python APIs, showing faster extraction speeds offset by higher memory consumption. The work further tracks how issues evolve and persist over time across well-known repositories, demonstrating that lifetime patterns can guide which issues receive priority attention. These capabilities are positioned to support more effective issue management, developer assignment, and overall software quality improvements.","feed_headline":"G-Issue extracts issues faster than Python APIs","feed_subtitle":"Lifetime data from maintained repositories enables prioritization for better management.","key_machinery":"G-Issue tool, which performs mining of issue artifacts and tracks their lifetime and evolution patterns to support prioritization.","core_discovery":"G-Issue extracts issue-related artifacts more quickly than general Python APIs during mining tasks but requires more memory, and the resulting data on issue lifetime and evolution across maintained repositories enables prioritization of issues for better management and quality outcomes.","pith_inferences":["Similar lifetime tracking could be applied to other project artifacts such as pull requests or commits.","The memory trade-off might be mitigated by selective data loading in future versions of the tool.","Patterns identified in well-known repositories could serve as baselines for comparing newer or less-maintained projects."],"forward_implications":["Faster mining allows larger-scale analysis of issue data without proportional time increases.","Lifetime and evolution metrics can be used to rank issues for developer attention.","Improved prioritization supports more targeted developer assignment and quality management.","The approach offers a practical way to monitor software health through issue behavior over time."],"fun_headline_variants":["G-Issue mines issues faster than Python APIs but uses more memory","Issue lifetime enables prioritization in open source repositories","G-Issue studies evolution of issues over time in repos","Prioritizing issues based on lifetime and evolution data"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The selected well-known and maintained repositories provide representative data and that G-Issue accurately extracts all relevant issue artifacts without systematic omissions or errors.","fun_headline_variants_meta":{"raw":{"variants":["G-Issue mines issues faster than Python APIs but uses more memory","Issue lifetime enables prioritization in open source repositories","G-Issue studies evolution of issues over time in repos","Prioritizing issues based on lifetime and evolution data"]},"model":"grok-4.3","cost_usd":0.00444,"raw_usage":{"total_tokens":2180,"prompt_tokens":593,"num_sources_used":0,"completion_tokens":63,"cost_in_usd_ticks":44399500,"prompt_tokens_details":{"text_tokens":593,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1524,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":593,"tokens_out":63,"duration_ms":9470,"temperature":1.0,"reasoning_tokens":1524,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-26T12:02:47.467587+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Direct head-to-head timing and memory measurements of G-Issue versus a standard Python API on identical large repositories, plus manual spot-checks of extracted artifacts against the original repository contents.","supporting_citations":[],"review_version":1}