Pith. sign in

REVIEW 3 major objections 7 minor 58 references

First systematic benchmark of AI video chatbots reveals no app is both fast and accurate.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Commercial AI video chat apps differ by 4× in video bitrate, 10× in framerate, and from zero to 10+ minutes of visual memory, with none replying in under 1.5 seconds.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection A genuinely first measurement paper on AI video chat; the core benchmark is worth taking seriously, but the headline GPT-4o degradation claim is not proven and the artifact has internal inconsistencies. the 3 major comments →

arxiv 2510.00481 v2 pith:56AYYAHD submitted 2025-10-01 cs.NI cs.AIcs.HCcs.MMcs.PF

Make a Video Call with LLM: A Measurement Campaign over Six Mainstream Apps

Jiayang Xu , Xiangjie Huang , Zijie Li , Antariksh Verma , Zili Meng This is my paper
classification cs.NI cs.AIcs.HCcs.MMcs.PF
keywords AI video chatreal-time communicationLLM benchmarkingmultimodal understandingRTC measurementvisual memorylatencysystem overhead
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper presents the first systematic benchmark for AI video chat, a new class of real-time applications where an LLM receives live video and responds in speech. It measures five mainstream chatbots—ChatGPT, Gemini, Grok, Doubao, and Yuanbao—across quality, latency, internal design, and overhead. The central claim is that none achieve conversational parity: response delays never go below 1.5 seconds, no app can proactively speak, and visual memory ranges from over ten minutes to zero. The paper also finds that AI capability, not network bitrate, dominates user experience, and that underlying architectures remain heterogeneous.

Core claim

Using a custom automated testbed on cloud emulators and a local rooted phone, the authors measure five commercial AI video chat apps and find a clear quality-latency trade-off: no app is both fast and accurate. Grok and Yuanbao respond in about 2.5 seconds but with lower quality, while ChatGPT, Gemini, and Doubao take longer (ChatGPT averages ~4s, Doubao ~3.6s with a 90th percentile near 6s) for better answers. All apps score zero on proactive output. Visual memory varies: ChatGPT recalls over ten minutes, Yuanbao nothing beyond the current frame. The paper also documents divergent transport choices (RTP vs QUIC), framerates from ~1 to 30 fps, and a split between audio-native and cascaded sp

What carries the argument

The carrying mechanism is the black-box testbed: a cloud setup with a Genymotion Android emulator fed by a Linux media source providing virtual camera and microphone, plus a local setup with a rooted Android phone and Bluetooth audio redirection. Combined with a four-dimension metric framework (quality, latency, internal mechanisms, overhead) and a dataset mixing StreamingBench-style visual tasks with AI-RTC-specific scenarios (memory, named-entity recognition, math), this enables automated, reproducible measurement of opaque commercial apps. The released dataset and testbed constitute the reusable benchmark.

Load-bearing premise

The claim that real-time video chat degrades ChatGPT's visual understanding rests on the assumption that the model powering ChatGPT's video chat is the same GPT-4o evaluated in StreamingBench; if the production model is a different version, fine-tuned, or served with different preprocessing, the drop could be an artifact of the comparison.

What would settle it

Run the same open-ended audio questions from this benchmark against the official GPT-4o API (or offline StreamingBench protocol) while feeding identical video frames; if the API scores match the live ChatGPT scores (~53) rather than the offline StreamingBench scores (~74), the paper's RTC-degrades-understanding conclusion is wrong.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If this baseline holds, future optimization should prioritize AI-side capabilities—memory, proactive output, audio-video integration—over network transport alone.
  • The finding that server-side request scheduling contributes at least 40% of total response delay suggests capacity planning could cut latency without changing the model.
  • The lack of proactive output across all five apps is a concrete, testable product gap.
  • The heterogeneity in framerates and input modality shows the industry has not converged, so principled design choices based on this benchmark could still shape the field.
  • Because the testbed and dataset are public, future app releases can be re-measured against this first baseline.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The live-versus-offline comparison for ChatGPT (53.39 vs 74.54) assumes the production model is the same GPT-4o; if that identity fails, the RTC degradation conclusion could be partly an artifact.
  • The quality scores rely on an LLM-as-judge with manual review; running the same protocol with human raters on a subset would validate the ranking.
  • The memory findings suggest a reusable probe: a blank-screen moment separates memory failure from overwrite failure, as Grok recalls only when no new frames arrive.
  • AI chatbots needing higher bandwidth than humans, yet seeing better within range, implies bitrate adaptation tuned for human perception is misfit for machine vision.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper presents a measurement campaign and benchmark for emerging AI video chat applications, proposing four evaluation dimensions (quality, latency, internal mechanisms, system overhead). Using custom cloud and local testbeds, the authors evaluate five commercial apps — ChatGPT, Gemini, Grok, Doubao, and Yuanbao — over a 21-day period with more than 60 hours of active sessions. They report response quality scores on a curated StreamingBench subset plus a custom AI-RTC dataset, response delay and setup time, streaming protocols and bitrates, client CPU/memory usage, and behavior under bandwidth throttling. The headline results are: (1) large cross-app heterogeneity in delay, quality, memory, and streaming design; (2) a quality-latency trade-off; (3) a substantial quality drop for ChatGPT relative to offline GPT-4o benchmarks, attributed to RTC constraints; and (4) significant bandwidth-constrained performance differences versus traditional RTC. The authors state they will release the dataset, testbeds, and an online evaluation platform.

Significance. If its claims hold, this is a timely and useful contribution. AI video chat is a genuinely new RTC application class, and the community lacks systematic measurements of commercial deployments. The paper's strengths are the custom testbeds (virtual cameras/microphones, echo cancellation, automated UI control), the multi-region deployment, the 60+ hours of active measurement, and the open release of dataset/testbed artifacts. The measurements of bitrate, framerate, protocol, CPU/memory, and response delay are plausible and reproducible in design. The main risk is interpretive: one of the paper's central quality findings depends on an unverified model-identity and task-equivalence assumption, and cross-app latency comparisons mix local and cloud testbeds without an explicit correction. These issues are fixable and do not undermine the value of the measurement infrastructure, but they currently prevent the paper's strongest qualitative conclusions from being accepted as stated.

major comments (3)
  1. [§6.2.1, Table 5] The claim that 'the model powering ChatGPT for AI video chat is identical to the one evaluated in StreamingBench (both GPT-4o)' is unsupported. Production ChatGPT video chat is not documented as using the exact model version evaluated in StreamingBench, and the real-time pipeline may involve frame sampling, fine-tuning, or different serving conditions. This identity assumption is load-bearing for the headline comparison of 53.39 vs 74.54/73.28 and for the conclusion that 'even benchmarks designed for streaming fail to capture AI performance under RTC constraints.' Please verify the deployed model via official documentation or behavioral probes, or reframe the comparison as approximate and not a direct model-equivalent measurement. The blackbox limitation in §8 does not cover this specific assumption.
  2. [§4.1 and §6.2.1, Table 2] The assertion that 'subset bias can be ruled out' because the sample was randomly selected and covered all sub-dimensions is not justified. A 54-question random subset can still differ in difficulty, and the protocol was changed from multiple-choice text with full-video input to open-ended audio questions with live streaming. There is no offline control run on the same subset to quantify the effect of these protocol changes. The observed 21-point drop could be an artifact of task format or subset selection rather than RTC constraints. Please run the same subset under the StreamingBench offline protocol, report per-category subset scores from StreamingBench, or downgrade the conclusion to a hypothesis.
  3. [§5.2, Table 4, Fig. 7] Cross-app response-delay comparisons mix testbeds: ChatGPT and Grok are local-only (Portugal), while Gemini, Doubao, and Yuanbao are cloud-only, with a reported VM-to-VM video delay of ~300 ms and a Bluetooth audio delay of ~200 ms. The paper does not state whether these measured testbed delays were subtracted from the reported response delays. Since the app-level differences in Fig. 10 are on the order of 1–2 s, an uncorrected 200–300 ms infrastructure offset is material for the quality-delay trade-off in Fig. 7. Please report the delay budget for each testbed, state the correction procedure explicitly, and provide a local/cloud consistency check (e.g., the Doubao/Yuanbao local validation mentioned in §5.2).
minor comments (7)
  1. [Title/Abstract] The title says 'Five Mainstream Apps' but the abstract says 'six'; the body evaluates five. Please make this consistent.
  2. [Abstract] The abstract states that 'network latency of AI video chat matters not as much as human video chat,' but I could not find a direct measurement or comparison supporting this claim in §6. Either add the corresponding evidence or remove the claim.
  3. [Abstract] The abstract mentions 'an online platform for user study,' but no user-study results are reported in the body. Clarify whether this is future work or a platform release only.
  4. [§5.2] The computation of response delay from parecord recordings should specify the audio-silence detection thresholds and how partial words or overlapping speech are handled. Without these details, replication of the latency numbers is difficult.
  5. [§5.2, §6.2.1] The LLM judge is GPT-4o, and one tested product is also GPT-4o-based. To mitigate same-model bias, report judge agreement or use a second independent judge for a subset of the scored answers.
  6. [Table 5, §6.2.1] The Grok visual-content-memory entry is listed as '0*' with a footnote 'in certain cases,' while the text reports recall of up to 10 seconds during blank screens. Please clarify the scoring rule that yields 0 overall despite these cases.
  7. [§6.4] The Gemini framerate is approximated from packet burst frequency over an opaque QUIC stream. This should be stated more prominently as an estimate, not a direct measurement, in both the text and Table 7.

Circularity Check

0 steps flagged

No significant circularity: the benchmark results are independent measurements; the flagged §6.2.1 comparison rests on an unverified external identity assumption, not on a definitional or fitted reduction.

full rationale

The paper's derivation chain is empirical: construct a dataset (borrowed from StreamingBench plus self-built scenarios), run black-box testbeds, capture network traces, and score responses with Whisper and an LLM judge. No equation defines a predicted quantity in terms of a fitted parameter, and no benchmark score is constructed from its own inputs. The paper's only self-citation, [23] (ACE, same group), appears as a qualitative justification for bursty transmission trade-offs ('This behavior may reflect trade-offs between network link utilization and the risk of overflow [23]'), not as evidence for any measured quantity; it is not load-bearing. The potentially problematic step is §6.2.1, where ChatGPT's live video-chat score (53.39) is compared with StreamingBench's offline GPT-4o scores (74.54/73.28) to conclude that RTC constraints degrade visual understanding. This comparison assumes the production ChatGPT video-chat model is 'identical to the one evaluated in StreamingBench (both GPT-4o)' and that converting offline multiple-choice tasks to live open-ended audio questions is task-neutral. That is a real validity and correctness risk, and the paper itself discloses black-box opacity in §8, but it is not circularity: the paper neither defines the production model in terms of StreamingBench nor fits any parameter that forces the observed 21-point drop. No circular step can be exhibited under the required quote-and-reduction standard, so the appropriate finding is no significant circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The central claim is empirical, so the ledger records calibration/design choices and background assumptions rather than fitted parameters. The most consequential entries are the unverified ChatGPT/GPT-4o identity (load-bearing for the performance-drop claim) and the background assumption that synthetic testbed media is faithful to real user input. No new physical or conceptual entities are introduced.

free parameters (5)
  • Background audio attenuation ratio = 20% (video audio vs question audio)
    Used to simulate environmental background sound in omni-source understanding tests (§4.1). This hand-chosen ratio affects the omni-source scores because the AI must separate the question from background noise; a different ratio would likely change measured scores.
  • Testbed video/audio delay constants = ≈300 ms (cloud video between VMs); ≈200 ms (local Bluetooth audio round trip)
    Reported in §5.2 as measured system delays. The full text does not explicitly state they are subtracted from the reported response-delay values; if not subtracted, the measured latencies are inflated by these constants, which is a systematic bias in the latency findings.
  • Peak/off-peak time windows = 02:00–07:00 vs 14:00–19:00 (peak in figure is 15:00, off-peak 03:00)
    Chosen to segment the latency time series into 'peak' and 'off-peak'. The 40% scheduling attribution is computed from this binary segmentation of a single day (§6.3.1).
  • Bandwidth throttling decrement = 100 kbps
    In §6.6, available bandwidth was reduced in 100 kbps steps; this step size limits the precision of the reported 'minimal acceptable bitrate' (e.g., Gemini at 300 kbps vs Doubao at 800 kbps).
  • Dataset subset sizes = 63 videos, 136 questions; 5 memory videos; 4 proactive-output questions
    Chosen for practicality (§4.1) rather than statistical power; the paper asserts subset bias can be ruled out for the StreamingBench subset, but the proactive (n=4) and memory (n=5) dimensions have very small samples and no error bars.
axioms (5)
  • domain assumption The AI video chat system follows a cascaded (STT→LLM→TTS) or end-to-end audio+video pipeline as depicted in Figure 1.
    The whole benchmark framing assumes the app processes user audio and video at a cloud LLM backend and returns audio only; if a product uses different internal architecture (e.g., local processing, video understanding), some metrics could be mis-attributed.
  • domain assumption Virtual camera/microphone inputs from the testbed are equivalent to real user camera/mic signals.
    Both testbeds simulate video/audio at the OS level (v4l2loopback, OBS, vcam, PulseAudio). If apps detect or react differently to synthetic streams (e.g., lower resolution, missing sensor metadata, system-app restrictions), the measured behavior may not represent real use.
  • domain assumption Whisper transcription and GPT-4o judge correctly grade open-ended answers, and manual review resolves partials.
    Quality scores depend entirely on the judge pipeline (§5.2); no inter-annotator agreement or judge validation other than the MT-Bench citation is reported.
  • ad hoc to paper The model powering ChatGPT's video chat is identical to the GPT-4o evaluated in StreamingBench.
    Stated in §6.2.1 as the basis for comparing real-time ChatGPT score (53.39) to StreamingBench's offline GPT-4o score (74.54); the equivalence is not independently verifiable for a black-box commercial service.
  • domain assumption LLM inference dominates server-side latency, so network latency is a minor factor.
    Assumed in §3 (Non-network bottlenecks attract more attention) and used to interpret response-delay differences; the paper never decomposes response delay into network vs inference vs scheduling components.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Make a Video Call with LLM: A Measurement Campaign over Six Mainstream Apps." pith.science (2026). https://pith.science/paper/56AYYAHD

@misc{pith2026251000481,
  author       = {Pith},
  title        = {Pith review of: Make a Video Call with LLM: A Measurement Campaign over Six Mainstream Apps},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/56AYYAHD}},
  note         = {Machine review of arXiv:2510.00481}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

In 2025, Large Language Model (LLM) services have launched a new feature -- AI video chat -- allowing users to interact with AI agents via real-time video communication (RTC), just like chatting with real people. Despite its significance, no systematic study has characterized the performance of existing AI video chat systems. To address this gap, this paper proposes a comprehensive benchmark across four dimensions: quality, latency, internal mechanisms, and system overhead. Using custom testbeds, we further evaluate six mainstream AI video chatbots with this benchmark. We also build an online platform for user study. The measurement leads to interesting findings that could be beneficial to the future optimizations. For example, the network latency of AI video chat matters not as much as human video chat. The capabilities of AI agents matters most in the user experience. Our benchmarking results also open up several research questions for future optimizations of AI video chatbots. Availability: https://callarena.net/ for the online evaluation platform and our open-sourced dataset and testbed.

Figures

Figures reproduced from arXiv: 2510.00481 by Antariksh Verma, Jiayang Xu, Xiangjie Huang, Zijie Li, Zili Meng.

Figure 1
Figure 1. Figure 1: AI video chat paradigm (Google), Grok (xAI), Doubao (ByteDance) [6] and Yuanbao (Tencent) [46]. The first three chatbots are the only video￾enabled applications from the global top 15 [43], while the last two are the only ones with this feature among the top five in China [40]. We develop the testbeds for real phones and cloud emulated phones to evaluate the performance both faith￾fully and broadly. The me… view at source ↗
Figure 3
Figure 3. Figure 3: Evaluation objectives for AI video chat mission and client-side processing. In AI video chat, how￾ever, the primary bottleneck is server-side, dominated by the heavy autoregressive inference of the LLM. AI video chat vs. AI text/audio chat. The primary feature distinguishing AI video chat from text- or audio-only chat is the addition of a continuous video input stream. This multi￾modal input introduces two… view at source ↗
Figure 4
Figure 4. Figure 4: The definition of AI visual content memory [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Overview of cloud and local testbeds Audio Loopback Sink Source Record Original Redesigned AI Chat Bot Audio AI Chat Bot Record [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Reroute audio for echo cancellation speaker and microphone and set them as the default devices. Audio files are then streamed using the same approach as in the cloud testbed. • Video stream. On the rooted Android device, video input is simulated using the vcam module, which creates a virtual camera fed by a local video file. Note that the vcam approach for video simulation does not currently function with … view at source ↗
Figure 7
Figure 7. Figure 7: Overall performance of AI video chat applications [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 10
Figure 10. Figure 10: Response delay across apps on users by different applications and (2) the speaking speed of the AI agent itself. To further verify the underlying cause of the response du￾ration discrepancy, we additionally measured the speaking rate of each AI application during the test sessions. As shown in [PITH_FULL_IMAGE:figures/full_fig_p009_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Response delay across different regions ChatGPTGeminiGrok DoubaoYuanbao 0 1 2 3 4 5 Peak Hour Off-Peak Hour Response Delay (s) [PITH_FULL_IMAGE:figures/full_fig_p010_11.png] view at source ↗
Figure 13
Figure 13. Figure 13: Response delay throughout the day (Doubao) ChatGPT Gemini Grok Doubao Yuanbao 0 1 2 3 4 5 Setup Time (s) [PITH_FULL_IMAGE:figures/full_fig_p010_13.png] view at source ↗
Figure 17
Figure 17. Figure 17: Frame intervals CDF ChatGPTGeminiGrok DoubaoYuanbao 0 100 200 300 0 2 4 6 8 10 CPU Memory CPU Usage (%) Memory Usage (%) [PITH_FULL_IMAGE:figures/full_fig_p012_17.png] view at source ↗
Figure 19
Figure 19. Figure 19: Minimal acceptable bitrate with recognizing the title of a paper (12-point font) printed on an A4 sheet, where the distance between the smartphone’s camera and the paper was fixed at 30 cm. For WhatsApp video calls, two participants were asked to visually identify the same paper title, with the metric being the minimum bi￾trate at which the title remained distinguishable to both. As shown in [PITH_FULL_I… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

58 extracted references · 9 linked inside Pith

  1. [1]

    10 years of siri: the history of apple’s voice assistant | techradar

    ALLEN, J. 10 years of siri: the history of apple’s voice assistant | techradar. https://www.techradar.com/news/siri-10-year-a nniversary, 2021

    2021

  2. [2]

    (A SUBSIDIARY OFAMAZON.COM, I

    AMAZONWEBSERVICES, INC. (A SUBSIDIARY OFAMAZON.COM, I. Aws cloud: Amazon web services – comprehensive cloud computing platform.https://aws.amazon.com/, 2006

    2006

  3. [3]

    N., BEGEN, A

    BENTALEB, A., LIM, M., AKCAY, M. N., BEGEN, A. C., HAM- MOUDI, S.,ANDZIMMERMANN, R. Toward one-second latency: Evolution of live media streaming.IEEE Communications Surveys & Tutorials(2025)

    2025

  4. [4]

    I am trying to use chatgpt when i try to login showing me this message error

    BLUESTACKS. I am trying to use chatgpt when i try to login showing me this message error. https://www.reddit.com/r/BlueStacks/ comments/1ft5hsf/comment/n4h6z6e/?context=3, 2024

    2024

  5. [5]

    Bluestacks: Play games on pc & mac, android emulator and ...https://www.bluestacks.com/, 2025

    BLUESTACKS. Bluestacks: Play games on pc & mac, android emulator and ...https://www.bluestacks.com/, 2025

    2025

  6. [6]

    Doubao.https://doubao.com/, 2023

    BYTEDANCE. Doubao.https://doubao.com/, 2023

    2023

  7. [7]

    Can you see me now? a measurement study of zoom, webex, and meet

    CHANG, H., VARVELLO, M., HAO, F.,ANDMUKHERJEE, S. Can you see me now? a measurement study of zoom, webex, and meet. In Proceedings of the 21st ACM internet measurement conference(2021), pp. 216–228

    2021

  8. [8]

    Reality check of metaverse: A first look at commercial social virtual reality platforms

    CHENG, R., WU, N., CHEN, S.,ANDHAN, B. Reality check of metaverse: A first look at commercial social virtual reality platforms. In2022 IEEE conference on virtual reality and 3D user interfaces abstracts and workshops (VRW)(2022), IEEE, pp. 141–148

    2022

  9. [9]

    Alibaba cloud

    CLOUD, A. Alibaba cloud. https://www.alibabacloud.com/, 2009

    2009

  10. [10]

    Moshi: a speech-text foun- dation model for real-time dialogue.arXiv preprint arXiv:2410.00037 (2024)

    DÉFOSSEZ, A., MAZARÉ, L., ORSINI, M., ROYER, A., PÉREZ, P., JÉ- GOU, H., GRAVE, E.,ANDZEGHIDOUR, N. Moshi: a speech-text foun- dation model for real-time dialogue.arXiv preprint arXiv:2410.00037 (2024)

    Pith/arXiv arXiv 2024

  11. [11]

    Sustainable llm serving: Environmental impli- cations, challenges, and opportunities

    DING, Y.,ANDSHI, T. Sustainable llm serving: Environmental impli- cations, challenges, and opportunities. In2024 IEEE 15th International Green and Sustainable Computing Conference (IGSC)(2024), IEEE, pp. 37–38

    2024

  12. [12]

    N.,ANDCONTRIBUTORS

    DURETTE, P. N.,ANDCONTRIBUTORS. gtts: Google text-to-speech. https://pypi.org/project/gTTS/, 2014. Python library for inter- acting with Google’s Text-to-Speech API

    2014

  13. [13]

    Professional android emulator

    GENYMOTION. Professional android emulator. https://www.geny motion.com/, 2025

    2025

  14. [14]

    Pexels: A library of free royalty-free images and videos

    GMBH, P. Pexels: A library of free royalty-free images and videos. https://www.pexels.com/, 2014

    2014

  15. [15]

    Gemini live with camera

    GOOGLE. Gemini live with camera. https://www.youtube.com/ watch?v=2Db6dBT6Vwg, 2025

    2025

  16. [16]

    Google gemini.https://gemini.google.com/, 2025

    GOOGLE. Google gemini.https://gemini.google.com/, 2025

    2025

  17. [17]

    Pop quiz gemini live with camera

    GOOGLE. Pop quiz gemini live with camera. https://www.youtub e.com/shorts/q78Z8ZEc3Cs, 2025

    2025

  18. [18]

    Project astra | exploring the future of learning with an ai tutor research prototype

    GOOGLE. Project astra | exploring the future of learning with an ai tutor research prototype. https://www.youtube.com/watch?v=MQ 4JfafE5Wo, 2025

    2025

  19. [19]

    Project astra: Our vision for the future of ai assistants

    GOOGLE. Project astra: Our vision for the future of ai assistants. https://www.youtube.com/shorts/1ritVbXeMbg, 2025

    2025

  20. [20]

    Spot a mystery bird? https://www.youtube.com/watc h?v=MjcpIuW5fG0, 2025

    GOOGLE. Spot a mystery bird? https://www.youtube.com/watc h?v=MjcpIuW5fG0, 2025

    2025

  21. [21]

    Wikipedia rabbit hole with screen share

    GOOGLE. Wikipedia rabbit hole with screen share. https://www. youtube.com/watch?v=MQ4JfafE5Wo, 2025

    2025

  22. [22]

    Kv prediction for improved time to first token

    HORTON, M., CAO, Q., SUN, C., JIN, Y., MEHTA, S., RASTEGARI, M.,ANDNABI, M. Kv prediction for improved time to first token. arXiv preprint arXiv:2410.08391(2024)

    Pith/arXiv arXiv 2024

  23. [23]

    D., SHI, S.,ANDMENG, Z

    HUANG, X., XU, J., WANG, H., YU, H., SATHYANARAYANA, S. D., SHI, S.,ANDMENG, Z. Ace: Sending burstiness control for high- quality real-time communication. InProceedings of the ACM SIG- COMM 2025 Conference(New York, NY , USA, 2025), SIGCOMM ’25, Association for Computing Machinery, p. 1182–1198

    2025

  24. [24]

    Assessing opportunities and dis- ruptions in an evolving trillion-dollar market

    INTELLIGENCE, B. Assessing opportunities and dis- ruptions in an evolving trillion-dollar market. https: //assets.bbhub.io/promo/sites/16/Bloomberg-Intelli gence-NVDA-Gen-AIs-Disruptive-Race.pdf , 2025. Accessed: 2025-08-30

    2025

  25. [25]

    Best global brands 2024

    INTERBRAND. Best global brands 2024. https://interbrand.c om/best-global-brands/, 2024

    2024

  26. [26]

    One-way transmis- sion time for the general recommendations on the transmission quality for an entire international telephone connection

    INTERNATIONALTELECOMMUNICATIONUNIONTELECOMMUNI- CATIONSTANDARDIZATIONSECTOR(ITU-T). One-way transmis- sion time for the general recommendations on the transmission quality for an entire international telephone connection. Tech. Rep. G.114, International Telecommunication Union (ITU), 05 2003. ITU-T Rec- ommendation G.114 (2003-05)

    2003

  27. [27]

    Ipinfo.io: Ip address data and geolocation api

    IPINFO.IO, I. Ipinfo.io: Ip address data and geolocation api. https: //ipinfo.io/, 2013

    2013

  28. [28]

    Llm inference serving: Survey of recent advances and opportunities

    LI, B., JIANG, Y., GADEPALLY, V.,ANDTIWARI, D. Llm inference serving: Survey of recent advances and opportunities. In2024 IEEE High Performance Extreme Computing Conference (HPEC)(2024), IEEE, pp. 1–8

    2024

  29. [29]

    Beyond static datasets: A deep interaction approach to llm evaluation.arXiv preprint arXiv:2309.04369(2023)

    LI, J., LI, R.,ANDLIU, Q. Beyond static datasets: A deep interaction approach to llm evaluation.arXiv preprint arXiv:2309.04369(2023)

    Pith/arXiv arXiv 2023

  30. [30]

    Qserve: W4a8kv4 quantization and system co-design for efficient llm serving.arXiv preprint arXiv:2405.04532(2024)

    LIN, Y., TANG, H., YANG, S., ZHANG, Z., XIAO, G., GAN, C.,AND HAN, S. Qserve: W4a8kv4 quantization and system co-design for efficient llm serving.arXiv preprint arXiv:2405.04532(2024)

    Pith/arXiv arXiv 2024

  31. [31]

    tc(8) — linux manual page: show / manipulate traffic control settings

    LINUX MAN-PAGES PROJECT. tc(8) — linux manual page: show / manipulate traffic control settings. https://man7.org/linux/ man-pages/man8/tc.8.html, 2025. Accessed from the iproute2 project repository; Last repository commit date: 2025-08-08; HTML page retrieved: 2025-08-11. Documentation for the Linux kernel traf- fic control utility, covering qdiscs, class...

    2025

  32. [32]

    Measuring the performance and network utilization of popular video conferencing applications

    MACMILLAN, K., MANGLA, T., SAXON, J.,ANDFEAMSTER, N. Measuring the performance and network utilization of popular video conferencing applications. InProceedings of the 21st ACM Internet Measurement Conference(New York, NY , USA, 2021), IMC ’21, As- sociation for Computing Machinery, p. 229–244

    2021

  33. [33]

    Maxmind.https://www.maxmind.com/, 2002

    MAXMIND. Maxmind.https://www.maxmind.com/, 2002. 13

    2002

  34. [34]

    Enabling passive measurement of zoom performance in production networks

    MICHEL, O., SENGUPTA, S., KIM, H., NETRAVALI, R.,ANDREX- FORD, J. Enabling passive measurement of zoom performance in production networks. InProceedings of the 22nd ACM internet mea- surement conference(2022), pp. 244–260

    2022

  35. [35]

    Chatgpt.https://openai.com/blog/chatgpt/, 2022

    OPENAI. Chatgpt.https://openai.com/blog/chatgpt/, 2022

    2022

  36. [36]

    A survey of useful llm evaluation.arXiv preprint arXiv:2406.00936(2024)

    PENG, J.-L., CHENG, S., DIAU, E., SHIH, Y.-Y., CHEN, P.-H., LIN, Y.-T.,ANDCHEN, Y.-N. A survey of useful llm evaluation.arXiv preprint arXiv:2406.00936(2024)

    Pith/arXiv arXiv 2024

  37. [37]

    Phone heats up during video calls/zoom meet- ing

    PIXEL4A5G. Phone heats up during video calls/zoom meet- ing. https://www.reddit.com/r/Pixel4a5G/comments/s6myt 9/phone_heats_up_during_video_callszoom_meeting/, 2022

    2022

  38. [38]

    10 most popular foods in the world

    POTHURAJU, S. 10 most popular foods in the world. https://voca l.media/education/10-most-popular-foods-in-the-world , 2022

    2022

  39. [39]

    T., BA ¸ SAR, T.,ANDIYER, R

    QIU, H., MAO, W., PATKE, A., CUI, S., JHA, S., WANG, C., FRANKE, H., KALBARCZYK, Z. T., BA ¸ SAR, T.,ANDIYER, R. K. Efficient inter- active llm serving with proxy model-based sequence length prediction. arXiv preprint arXiv:2404.08509(2024)

    Pith/arXiv arXiv 2024

  40. [40]

    The number of mobile application users reaches 645 million

    QUESTMOBILE. The number of mobile application users reaches 645 million. https://news.qq.com/rain/a/20250916A02I7100,

    arXiv

  41. [41]

    QUORA. Video editing: How loud should background music be in terms of volume percent? https://www.quora.com/Video-Editi ng-How-loud-should-background-music-be-in-terms-of-v olume-percent, 2024

    2024

  42. [42]

    W., XU, T., BROCKMAN, G., MCLEAVEY, C.,ANDSUTSKEVER, I

    RADFORD, A., KIM, J. W., XU, T., BROCKMAN, G., MCLEAVEY, C.,ANDSUTSKEVER, I. Robust speech recognition via large-scale weak supervision, 2022

    2022

  43. [43]

    The best ai chatbots in 2025

    REBELO, M. The best ai chatbots in 2025. https://zapier.com /blog/best-ai-chatbot/, 2025. Accessed: 2025-08-30

    2025

  44. [44]

    Create and manage virtual devices

    STUDIO, A. Create and manage virtual devices. https://develo per.android.com/studio/run/managing-avds, 2025

    2025

  45. [45]

    You can now do ’live video’ with google gemini

    TECHKAFEVER. You can now do ’live video’ with google gemini. https://x.com/techkafever/status/1931961962214342989? referrer=grok-com, 2025

    arXiv 2025

  46. [46]

    Tencent yuanbao

    TENCENT. Tencent yuanbao. https://yuanbao.tencent.com/, 2025

    2025

  47. [47]

    Video conferencing market analysis

    UNKNOWN. Video conferencing market analysis. https://www.fortunebusinessinsights.com/industry-r eports/video-conferencing-market-100293 , 2025. Accessed: 2025-08-30

    2025

  48. [48]

    The world’s 47 best tourist attractions

    USNEWS. The world’s 47 best tourist attractions. https://travel.usnews.com/gallery/the-worlds-best-t ourist-attractions, 2025

    2025

  49. [49]

    C.,ANDCHU, X

    WANG, Y., CHEN, Y., LI, Z., TANG, Z., GUO, R., WANG, X., WANG, Q., ZHOU, A. C.,ANDCHU, X. Towards efficient and reliable llm serving: A real-world workload study.CoRR(2024)

    2024

  50. [50]

    Livebench: A challenging, contamination-free llm benchmark.arXiv preprint arXiv:2406.19314 4(2024)

    WHITE, C., DOOLEY, S., ROBERTS, M., PAL, A., FEUER, B., JAIN, S., SHWARTZ-ZIV, R., JAIN, N., SAIFULLAH, K., NAIDU, S.,ET AL. Livebench: A challenging, contamination-free llm benchmark.arXiv preprint arXiv:2406.19314 4(2024)

    Pith/arXiv arXiv 2024

  51. [51]

    A stroll in the sun with gemini live

    WITHGOOGLE, T. A stroll in the sun with gemini live. https: //x.com/ThinkwithGoogle/status/1934971969054617605?re ferrer=grok-com, 2025

    arXiv 2025

  52. [52]

    R., LIN, C.-Y., CHEN, Y.-N

    WU, C.-K., TAM, Z. R., LIN, C.-Y., CHEN, Y.-N. V.,ANDLEE, H.- Y. Streambench: Towards benchmarking continuous improvement of language agents.Advances in Neural Information Processing Systems 37(2024), 107039–107063. [53]XAI. Grok ai.https://grokdemo.com/, 2025

    2024

  53. [54]

    Measurement of cloud-based game streaming system response to competing tcp cubic or tcp bbr flows

    XU, X.,ANDCLAYPOOL, M. Measurement of cloud-based game streaming system response to competing tcp cubic or tcp bbr flows. InProceedings of the 22nd ACM Internet Measurement Conference (2022), IMC ’22

    2022

  54. [55]

    Youtube.https://www.youtube.com/, 2005

    YOUTUBE. Youtube.https://www.youtube.com/, 2005

    2005

  55. [56]

    Glm-4-voice: Towards intelligent and human-like end-to-end spoken chatbot.arXiv preprint arXiv:2412.02612(2024)

    ZENG, A., DU, Z., LIU, M., WANG, K., JIANG, S., ZHAO, L., DONG, Y.,ANDTANG, J. Glm-4-voice: Towards intelligent and human-like end-to-end spoken chatbot.arXiv preprint arXiv:2412.02612(2024)

    Pith/arXiv arXiv 2024

  56. [57]

    Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in neural information processing systems 36(2023), 46595–46623

    ZHENG, L., CHIANG, W.-L., SHENG, Y., ZHUANG, S., WU, Z., ZHUANG, Y., LIN, Z., LI, Z., LI, D., XING, E.,ET AL. Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in neural information processing systems 36(2023), 46595–46623

    2023

  57. [58]

    X., CHEN, X., LIN, Y., WEN, J.-R.,ANDHAN, J

    ZHOU, K., ZHU, Y., CHEN, Z., CHEN, W., ZHAO, W. X., CHEN, X., LIN, Y., WEN, J.-R.,ANDHAN, J. Don’t make your llm an evaluation benchmark cheater.arXiv preprint arXiv:2311.01964(2023). 14

    Pith/arXiv arXiv 2023

  58. [2025]

    Accessed: 2025-08-30

    2025

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.