{"id":"0cafc918-ba87-4670-b0dc-401f2060183e","arxiv_id":"2502.08621","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"SportsBuddy, a deployed AI-assisted web editor, lets sports users add tracked highlights, tactical drawings, and captions to game videos without professional editing skills.","lead":"SportsBuddy is a web tool that lets coaches, athletes, and fans turn game footage into narrated highlight videos with click-to-track visual effects. A three-month public deployment with more than 150 users and case studies with college teams and influencers suggests that AI-assisted editing can make sports storytelling accessible to non-professionals.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Impact claim hinges on unverified self-reports and a 150M-view figure with no analytics source; objective log/analytics verification is needed before accessibility and fan-engagement conclusions are accepted.","rationale":"The reader's weakest assumption correctly identifies the self-selected, self-reported nature of the user data in Sec. 5.1. I agree that this is a serious limitation. My stress-test adds a more specific and more load-bearing variant: the one quantitative impact statistic in the paper, the 'over 150 million views' for 14 Instagram videos, is extraordinary and entirely unsourced within the manuscript. If that number is inaccurate, the strongest objective evidence for 'positive impact' and 'fan engagement' collapses, leaving only qualitative testimonials. Even if the number is correct, the accessibility claim lacks behavioral verification: no log-based measure of editing time, no task success rates, and no comparison against a baseline editing workflow. These gaps do not make the paper's design contributions invalid; the system is clearly deployed and used, and the qualitative feedback is directionally consistent. The appropriate posture remains CONDITIONAL: the claims should be treated as promising but unverified until the authors provide the underlying analytics and logs. My recommendation is therefore UNCHANGED relative to the reader's verdict, with the condition made more concrete: release the Instagram analytics and server logs that would substantiate the headline impact figures.","tokens_in":15638,"tokens_out":4226,"duration_ms":113968,"concrete_test":"Obtain the Instagram analytics for the 14 Harvard Athletics posts (per-video view counts, reach, date range) and the SportsBuddy server logs for the three-month deployment (per-user uploads, exports, session duration). Check (a) whether the sum of post views is approximately 150M, and (b) whether a meaningful fraction of the 163 registered users (e.g., >20%) exported at least one highlight. If either check fails, the deployment claims should be revised to reflect the verified usage level.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that SportsBuddy is accessible and impactful rests on two evidentiary pillars: (i) qualitative feedback from 66 of 163 registered users (Sec. 5.1) and (ii) two case studies, including the statement in Sec. 5.2.1 that 'These 14 videos have collectively attracted over 150 million views.' Both pillars are currently unverifiable from the manuscript. There is no SUS score, task-completion rate, or log-based measure of editing time; the 'over 30 minutes to under 10 minutes' quote is a single self-report, not a measured workflow comparison. The 150M-view figure is the only quantitative evidence for fan-engagement impact, yet no analytics source, per-video breakdown, or time window is provided. If this figure is a typo or reflects aggregate account reach rather than video views, the impact conclusion loses its objective anchor. The accessibility claim would still be plausible as a design case study, but it would not be established for the broader 'diverse audiences' stated in the abstract.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents SportsBuddy, a web-based AI-powered tool for authoring sports video highlights. The system combines player tracking, embedded canvas-based interactions, timeline-based editing, and AI captioning to let coaches, athletes, creators, parents, and fans produce context-rich highlight videos. The authors report a three-month public deployment (163 registered users), survey and interview feedback from a subset of users, and two case studies with Harvard Athletics and three basketball influencers. Based on this evidence, the paper claims that SportsBuddy improves accessibility and ease of use for sports storytelling and has positive impact on coaching communication, game analysis, and fan engagement.","tokens_in":15775,"tokens_out":2966,"duration_ms":28458,"significance":"The paper's main strength is its real-world deployment and the detailed description of a working system that operationalizes prior academic frameworks for embedded sports visualization. The design goals and implementation are clearly presented, and the iterative improvements based on user feedback demonstrate practical value. The claim that non-professional sports stakeholders can create engaging video stories with SportsBuddy is plausible and, if supported by stronger evidence, would be a useful contribution to sports visualization and authoring tools. However, the evaluation is largely qualitative and self-selected, and the headline quantitative figure (150 million views) is unverified. As written, the paper is a solid design case study, but its broader impact and accessibility claims outrun the evidence provided.","major_comments":[{"comment":"The sentence \"These 14 videos have collectively attracted over 150 million views\" is load-bearing for the fan-engagement conclusion, yet no analytics source, per-video breakdown, or time window is provided. This figure cannot be checked from the manuscript, and if it is incorrect or refers to aggregate account reach rather than video views, the impact claim loses its objective anchor. The authors should either supply verifiable analytics evidence (e.g., platform screenshots, a per-video table with view counts and dates) or remove the figure and temper the corresponding claims in Section 5.2.1 and the abstract.","section":"5.2.1"},{"comment":"The accessibility and efficiency claims rest on self-reported feedback from 66 of 163 registered users, with no log-based measures of editing time, task completion, or usability. The representative quote \"over 30 minutes to under 10 minutes\" is a single anecdote, not a measured workflow comparison. To substantiate the abstract's claim that the tool is accessible to \"diverse audiences,\" the authors should report objective system logs (e.g., session durations, feature usage counts, completion rates) or explicitly acknowledge the limitation and generalize only to the interviewed users.","section":"5.1"},{"comment":"The Future Work section states that \"SportsBuddy currently focuses on basketball but can expand to sports like soccer and tennis,\" which contradicts Sections 3.4 and 4.2.1 that describe support for basketball, soccer, volleyball, lacrosse, and tennis. This internal inconsistency matters because the paper's central claim of supporting diverse sports roles and sports types depends on the tool's actual coverage. The authors should clarify whether the deployed system supported all five sports or whether the deployment only exercised basketball (and perhaps a subset), and adjust the claims accordingly.","section":"6.1"},{"comment":"The two case studies rely on collaborators (Harvard Athletics and named influencers) who may be positively predisposed toward the tool; no independent evaluation, adversarial testing, or neutral third-party assessment is described. This is not a fatal flaw, but the paper's unqualified statement that feedback was \"overwhelmingly positive\" should be contextualized as coming from motivated collaborators, and the limitations of this evaluation mode should be stated explicitly.","section":"5.2"}],"minor_comments":[{"comment":"The abstract and Section 1 say \"over 150 sports users\" while Section 5 reports 163 registered users; the numbers should be aligned or the earlier statement should say \"over 160\" for consistency.","section":"1"},{"comment":"The sentence \"The entire creation process took Mike less than 5 minutes\" is presented as a demonstration; consider labeling it as an illustrative scenario rather than a measured performance claim, since no timing methodology is given.","section":"3.4"},{"comment":"The claim that rendering time is \"within 10 milliseconds for each frame\" should state whether this is a measured benchmark and under what conditions (e.g., video resolution, number of render objects), or it should be described as a design target.","section":"4.2.2"},{"comment":"The statement that each pipeline branch \"demonstrates robust performance\" is not supported by quantitative results in this paper; the authors should clarify that they defer to the original method papers for accuracy evaluation and that no new benchmark is provided.","section":"4.1"},{"comment":"Reference [23] is missing publication details (year, venue, or a stable URL); please complete the citation.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of a visualization or HCI venue and the system description is valuable. However, the evaluation needs substantial strengthening before the claims of broad accessibility and fan-engagement impact can be endorsed. In particular, the 150 million views figure should be verified or removed; the self-report-only basis of Section 5.1 should be acknowledged; and the internal inconsistency about supported sports should be fixed. I would not reject the paper, but the current evidence does not support the abstract's strong claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"SportsBuddy is a straightforward systems paper: the authors took their earlier augmented-video work, wrapped it in a web app with GPT-4o captions and five sports, and let the public use it for three months. That is genuinely useful. Most sports-visualization papers stop at a lab study; this one has 163 registered users, 66 questionnaires, two case studies, and a list of iterative changes driven by feedback. The tool itself is real, and the writing is honest about its limitations, including tracking failures, fixed transformation matrices, and browser memory constraints.\n\nThe accessibility claim is plausible and I think it largely holds up as a design case study. The evidence, though, is softer than the abstract implies. The 66 respondents are self-selected, there is no baseline or control, and the \"30 minutes to 10 minutes\" quote is one user's self-report, not a measured workflow. The 150-million-views figure for the Harvard videos is the only quantitative anchor for fan engagement, and the paper gives no analytics source, per-video breakdown, or time window. That number could be accurate, but right now it is an assertion.\n\nNone of this is disqualifying for a CHI-style systems paper. The deployment itself is the contribution, and the authors are not hiding the rough edges. What I would ask for in revision is straightforward: add whatever server-side logs they have (processing times, export counts are already there), separate measured numbers from self-reported ones, and either justify the 150M figure with a link or drop it. Also, tone down the abstract's \"demonstrate impact\" to something like \"suggest potential.\"\n\nI would send this to referees. The systems community will find it a useful data point, and the evaluation weaknesses are the kind that a good review can fix.","headline":"A real deployed sports video storytelling tool with a plausible accessibility story, but the evidence is mostly self-reported and the 150M-view figure needs verification.","tokens_in":16355,"tokens_out":1979,"would_cite":true,"duration_ms":18231,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An AI-powered web tool lets coaches, athletes, parents, and fans turn raw game footage into narrated, annotated highlight reels without professional editing skills, and a three-month public deployment with 163 registered users suggests…","keywords":["sports video storytelling","embedded visualization","highlight authoring","player tracking","real-world deployment","human-AI collaboration","video editing accessibility","case study"],"falsifier":"Server logs from the three-month deployment should show per-user session lengths, editing actions, and export counts; if most of the 163 accounts never exported a highlight, or if the average editing time for a minute-long highlight among logged-in users was comparable to manual editing in a general-purpose tool, the central accessibility claim would fail.","tokens_in":15418,"feed_emoji":"🏀","tokens_out":3690,"duration_ms":34876,"temperature":0.7,"pith_summary":"SportsBuddy is a web-based authoring tool that combines AI player tracking, direct on-video interaction, and timeline visualizations so that coaches, athletes, parents, creators, and fans can turn raw game footage into narrated highlight reels without professional video-editing skills. The paper argues that domain-specific tools like this, rather than general-purpose editors or generative video models, are what sports practitioners need to communicate insights. It supports the argument with a three-month public deployment: 163 registered users, 66 of whom provided feedback, plus case studies with collegiate marketing staff and online basketball creators. The reported outcome is that editing tasks that once took over thirty minutes came down to under ten, and that users adopted the tool for player synergy, tactical breakdowns, and spatial-action analysis. The paper's contribution is the design, implementation, and real-world evaluation of the system.","feed_headline":"AI web tool turns game clips into narrated highlights in minutes","feed_subtitle":"Real-world test with 163 sports users claims editing time drops from 30 minutes to under 10.","key_machinery":"The load-bearing mechanism is the render object: a structured data entity that encodes each visual effect's type, start and end frame, and effect parameters, and is drawn onto an HTML canvas in real time, with layers placed between the background and foreground so that overlays either sit on the court or above players. Player tracking via a sports-optimized tracker and a pose-estimation model, plus foreground-background segmentation, let effects like Circle, Spotlight, and Text attach to a player and follow them automatically, while Path, Zone, and Marker accept freehand tactical drawings that are transformed with a fixed perspective matrix. A timeline of color-coded tracks gives users direct control over each effect's timing. This combination is what replaces frame-by-frame manual editing with object-level, context-embedded authoring.","core_discovery":"The paper's central claim is that an interactive video authoring tool built around automatically tracked players and object-level visualization primitives can make sports video storytelling accessible to people who are not media professionals, and that this accessibility shows up in real use: over 150 registered users across five sports created and exported highlights, coaches used the visualizations to 'show, don't tell' during game review, and creators switched from static images to annotated video. The evidence is self-reported feedback from 66 questionnaire respondents, a 90.8 percent video upload success rate and 87.9 percent export success rate across 1,021 uploads and 814 exports, and two case-study collaborations. The claim is that these observations demonstrate reduced editing burden and improved storytelling quality for a diverse audience.","pith_inferences":["A controlled comparison using server logs rather than self-reports would probably show a smaller average time saving than the headline '30 to 10 minutes' figure, but even a moderate reduction could be enough to change adoption habits.","The render-object architecture could transfer to other domains where object tracking and layered annotation matter more than generative content, such as medical procedure review or refereeing analysis.","The accessibility claim is likely capped by tracking robustness: when player tracking fails under occlusion or rapid motion, the object-level interaction advantage disappears and users fall back to manual adjustment, so improving tracking may matter more than adding new visualization features.","The paper's emphasis on real-world deployment suggests a broader evaluation model for visualization research, where external validity comes from observing diverse self-selected users rather than from controlled laboratory tasks."],"forward_implications":["If the reported accessibility holds, sports organizations without broadcast budgets can produce broadcast-style tactical breakdowns for social media; the paper's collegiate case study reports 14 shared videos drawing over 150 million views.","Editing workflows that previously required switching between separate tools for clipping, annotation, narration, and export can be combined in one browser session, which the paper links to editing times falling from over 30 minutes to under 10.","Non-video professionals such as parents and youth coaches can generate recruiting-style player highlights, a use case the paper identifies as important for scholarships and team applications.","The paper's four design goals—intuitive sports features, object-level visualizations, integrated narratives, and a streamlined end-to-end workflow—provide a template that could transfer to other domain-specific video authoring tools.","Automated features like player tracking and AI-generated captions reduce the burden of manual annotation, which the paper reports was the greatest time-saver for less experienced users such as interns and youth coaches."],"supporting_citations":[{"why":"Supplies the data-driven approach and design space for augmented sports videos that SportsBuddy builds on.","marker":"[19]"},{"why":"Provides the embedded visualization framework and design guidance for sports video augmentation.","marker":"[28]"},{"why":"The sports multi-object tracking dataset and tracker used for player detection in the video processing pipeline.","marker":"[22]"},{"why":"The pose estimation toolbox used to predict player keypoints from tracking bounding boxes.","marker":"[21]"},{"why":"The segmentation model used to separate foreground players from the background court for layered rendering.","marker":"[25]"}],"fun_headline_variants":["AI storytelling tool helps sports users create highlights","SportsBuddy: AI video authoring for coaches, fans, and creators","Real-world deployment shows AI tool eases sports video storytelling","Player tracking meets timeline visuals for easy game stories","Over 150 users validate AI tool for sports video narratives"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The impact story depends on the 66 users who answered the questionnaire being a fair sample of the 163 registrants and on their self-reported time savings and enjoyment matching what they actually did in the tool.","fun_headline_variants_meta":{"raw":{"variants":["AI storytelling tool helps sports users create highlights","SportsBuddy: AI video authoring for coaches, fans, and creators","Real-world deployment shows AI tool eases sports video storytelling","Player tracking meets timeline visuals for easy game stories","Over 150 users validate AI tool for sports video narratives"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000232,"raw_usage":{"total_tokens":1458,"prompt_tokens":880,"completion_tokens":578,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":496,"completion_tokens_details":{"reasoning_tokens":498}},"tokens_in":496,"tokens_out":578,"duration_ms":6419,"temperature":1.0,"reasoning_tokens":498,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T00:00:40.119194+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Server logs from the three-month deployment should show per-user session lengths, editing actions, and export counts; if most of the 163 accounts never exported a highlight, or if the average editing time for a minute-long highlight among logged-in users was comparable to manual editing in a general-purpose tool, the central accessibility claim would fail.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the data-driven approach and design space for augmented sports videos that SportsBuddy builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the embedded visualization framework and design guidance for sports video augmentation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The sports multi-object tracking dataset and tracker used for player detection in the video processing pipeline."},{"cited_title":"Contributors","cited_arxiv_id":null,"evidence_quote":"The pose estimation toolbox used to predict player keypoints from tracking bounding boxes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The segmentation model used to separate foreground players from the background court for layered rendering."}],"review_version":1}