Pith. sign in

REVIEW 4 major objections 4 minor 73 references

AutoBridge: Automating Smart Device Integration with Centralized Platform

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AutoBridge claims to automatically generate usable integration code for 34 smart devices across Home Assistant and openHAB, averaging 93.87% success and 94.87% function coverage without human help.

desk verdict A useful LLM-based pipeline for IoT integration code with a real benchmark, but the headline coverage metric is measured against the system's own knowledge base and needs an independent ground-truth check. read the letter →

arxiv 2507.23178 v1 pith:3DXNCI22 submitted 2025-07-31 cs.SE cs.AI

Siyuan Liu , Zhice Yang , Huangxun Chen This is my paper
classification cs.SEcs.AI
keywords IoTmanagementlargelanguagemodelsprogramsynthesiscodegenerationintegrationHomeAssistantopenHABhardware-in-the-loopdebugging
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that an LLM-based agent can take over the job of writing IoT integration code—the bridge code that lets platforms like Home Assistant and openHAB talk to a new device—a task that currently requires expert programmers. AutoBridge splits the job into two knowledge-retrieval phases, first writing device control logic from device manuals, SDKs, and repositories, then reworking it against platform specifications, and verifies it in two debugging stages that end with a human only saying yes or no to observed device behavior. On a benchmark of 34 devices it reports an average 93.87% generation success rate and 94.87% function coverage with no human input, rising to 100% function coverage with a few yes/no answers. If those numbers hold, adding a new device becomes a specification and verification task rather than a development task, which matters because no centralized platform can manually keep up with the growing range of IoT devices.

What carries the argument

The load-bearing mechanism is a divide-and-conquer generation strategy followed by a two-stage debugging pipeline. The code generator first produces device control code using device-specific knowledge pulled from manuals, API/SDK documentation, and official GitHub repositories stored in a vector database, then re-targets that logic to the platform using a separate platform-specific knowledge base, guided by a ReAct-style reasoning-acting loop that decides when to retrieve additional information. The automated debugger then registers a virtual device inside the platform's execution environment, runs generated unit tests covering both basic integration and device functionality, and iteratively fixes code based on test failures. Finally, the hardware-in-the-loop debugger drives the real device, asks a human observer a binary yes/no question for each function, and revises the code whenever the answer is no, continuing until every known function is verified.

What would settle it

Take a physical device with a function deliberately omitted from all manuals, SDK pages, and repositories that AutoBridge is allowed to access, run the full pipeline, and check whether the generated code or the yes/no debugger ever exercises or asks about that function; if the hidden function is neither implemented nor tested nor shown to the user, the claimed full coverage is coverage of the retrieved documentation, not of the device.

Watch

Extended reading notes

Core claim

The paper claims that AutoBridge, an LLM-based programming agent, can synthesize the integration code that lets a centralized IoT platform understand and control a new device, and that this can be done automatically at high accuracy. On its benchmark of 34 devices—8 real purchased devices plus 26 official expert-written integrations across Home Assistant and openHAB—the fully automated pipeline achieves an average success rate of 93.87% and average function coverage of 94.87%; after the hardware-in-the-loop debugger collects only yes/no answers from a human, every device integration in its real-hardware set reaches 100% function coverage with no more than 9 rounds of revision per device. A 15-participant user study reports that AutoBridge's generated code is 50%–80% more accurate than code written by expert programmers who were allowed to use commercial code LLMs, and that non-programmers could operate the system successfully. The intended upshot is that the expert-programming bottleneck for IoT integration can be removed, letting personalized multi-modal IoT systems absorb new devices without waiting for manual development.

Load-bearing premise

The whole pipeline—code generator, test generator, and hardware verification checklist—takes its definition of the device's functions from the documentation it retrieves, so any device function absent from those sources is invisible to the system and is not counted in coverage.

Editorial extensions

If this is right

  • Adding a new device to Home Assistant or openHAB can be reduced from an expert programming task to a specification task plus brief yes/no verification for the benchmarked device classes.
  • The gap between automatic generation at 94.87% function coverage and fully correct integration at 100% is closed by a few binary answers, with no more than 9 revision rounds per device in the real-hardware evaluation.
  • Because the method relies only on retrieved public documentation and binary user feedback, non-programmers can complete integration tasks that more than half of expert programmers failed to finish correctly within 40 minutes in the user study.
  • The accuracy gain comes from separating device knowledge from platform knowledge and from staged debugging, not from the code LLM alone, since naive LLM and copilot baselines produced incomplete or placeholder-ridden code.
  • The approach is language-independent in practice, working for both the Python-based Home Assistant platform and the Java-based openHAB platform.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported coverage is relative to the device functions that appear in the retrieved documentation; a function absent from manuals, SDK pages, and repositories would be neither implemented, tested, nor offered for yes/no verification, so absolute coverage of the physical device could be lower than the reported percentages.
  • The same two-phase separation of interface knowledge and framework knowledge could plausibly transfer to other bridge-code domains, such as device drivers, printer adapters, router firmware integrations, or robot middleware, where a small hardware-facing layer meets a large framework contract.
  • The costly part of the hardware-in-the-loop stage is the human observer; replacing yes/no answers with an automatic visual or electrical check of device state would make the entire pipeline hands-free, a direction the paper itself notes in its discussion.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. AutoBridge is an LLM-based agent that automates the generation of IoT integration code for centralized platforms such as Home Assistant and openHAB. The system uses a divide-and-conquer code generation strategy (device control logic first, then platform-compliant integration code), progressive retrieval from device- and platform-specific vector databases, an automated debugger that tests code against virtual devices, and a hardware-in-the-loop debugger that asks users only yes/no questions. The evaluation covers 34 devices (8 on real hardware, 26 based on official human-written integrations), a 15-participant user study, and ablations over backbone models, prompt granularity, and system components. The headline results are a 93.87% generation success rate and 94.87% functional coverage without human feedback, rising to 100% functional coverage on the real-hardware set with at most 9 yes/no corrective feedback rounds.

Significance. If the results hold, the paper demonstrates a substantial advance in automating a knowledge-intensive and hardware-dependent code synthesis task: it shows an LLM-based agent generating hundreds-to-thousands-of-lines integration code with high success, verified on real devices. The evaluation is unusually comprehensive for this area: it includes real hardware, two platforms in different languages (Python and Java), an open-sourced benchmark, a user study with both experts and non-experts, and systematic ablations. The principal weakness is that the headline functional-coverage metric is not independent of the system's own knowledge base: the device-function list used for the real-hardware evaluation is derived from the same retrieved documentation that drives generation, test creation, and verification. Consequently the absolute coverage numbers are likely optimistic, and the central claim needs re-derivation against an external ground truth before it can be accepted as stated.

major comments (4)
  1. [§5.2, §3.2.2] The functional coverage for EvalSet 1 (RealHardware) is measured against a device-function list produced from the same device-specific vector database that drives code generation and automated test generation. Any function absent from the retrieved manuals, API/SDK documentation, or GitHub repositories is never generated, never unit-tested, and never prompted during hardware-in-the-loop verification, so it cannot lower the reported coverage. This effectively converts 'coverage of the device' into 'coverage of the knowledge base' and makes the 94.87% and 100% figures systematically optimistic. Please report coverage separately for EvalSet 1 and EvalSet 2, and construct an independent function list for EvalSet 1 (e.g., from official product specifications not used in the retrieval pipeline) to recompute the headline numbers.
  2. [§5.2] The definition of 'usable integration code' is not given; the Pass@1 success metric depends on a binary judgment that is not operationalized. Please specify the exact criteria used to classify a generated integration as usable (e.g., it passes the automated debugger's full test suite, or it satisfies a platform-specific checklist), and state who applies those criteria (the authors, the automated debugger, or an independent evaluator). Without this, the 93.87% success rate is not reproducible or comparable with future work.
  3. [§3.2.2] The automated debugger generates its own unit tests from the same vector DB, and no independent measure of test-suite completeness is provided. Incomplete or erroneous tests can certify code that omits functions or contains incorrect logic. The paper asserts that 'few redundant or erroneous test[s]... does not significantly impact the overall process,' but this claim is not supported by evidence. Please quantify the generated test suite's coverage against tests derived from the human-written integrations in EvalSet 2, or provide another external check of test-suite adequacy.
  4. [§5.4, Abstract] The claim that 'with minimal binary yes/no feedback the code is revised to reach 100% function coverage' is only demonstrated for EvalSet 1 (RealHardware), not for the full 34-device benchmark; the abstract presents it as a general property. Please qualify the claim, and note that even for EvalSet 1 the 100% figure inherits the knowledge-base circularity described above.
minor comments (4)
  1. [Abstract, §5.2] State explicitly that the 94.87% functional coverage is computed only over usable code (Pass@1 successes). An unconditional coverage metric that counts failed generations as zero would be lower and should be reported for transparency.
  2. [Global] Typos and wording issues: 'can achieves' (Abstract), 'virual' (§3.2.2), 'dose' (Figure 9 caption), 'Retrival Method' (Table 2 header), 'sucess rate' (§5.5), 'fucntion' (§5.3).
  3. [References] Reference [35] is cited for the SUS scoring method (converting 1–5 to 0–4), but [35] is a paper on data augmentation for time-series classification; the correct SUS scoring source appears to be [10] or the original SUS paper. Please fix this citation.
  4. [§5.1, §5.3] Clarify the unit over which the averages in Figure 10 are computed (device-platform pairs, devices, or generation runs). The text mentions 'device-platform pair' for Pass@1, but the averaging across devices and tiers is not fully specified.

Circularity Check

2 steps flagged · score 5.0 of 10

Function coverage is measured against a device-function list derived from the same retrieved documentation that drives generation and testing; omitted functions are invisible, so the 94.87% and 100% coverage claims need an independent ground-truth check.

  1. self definitional [Section 3.1.2 (Device-Specific Vector DB) and Section 3.2.2 (Virtual Device and Test Case Generation)]
    "Integration code needs to cover every function of the targeted device. To serve as a reference for the code generator, we use search APIs to comprehensively retrieve the target device's user manuals, API/SDK documentation, and official GitHub repositories ... We retrieve knowledge from the device- and platform-specific vector databases, summarize a device functionality list to serve as a reference for test code generation. ... this metric quantifies the percentage of functions correctly integrated relative to the full set of device functions."

    The full set of device functions used as the coverage denominator is never obtained independently. The only function inventory described in the paper is the one summarized from the same device- and platform-specific vector DBs that also guide the generator and the automated debugger. Any function absent from the retrieved manuals, API/SDK docs, or GitHub repos is therefore never implemented, never unit-tested, and never counted as missing. The reported 94.87% coverage is thus coverage of the knowledge base's own function list, not of the physical device. The manual screening disclaimer only removes exact integration snippets; it does not validate the completeness of the function list.

  2. self definitional [Section 3.3 (Interactive Hardware-in-the-Loop Debugger) and Section 5.4 (Performance w/ Human Feedback)]
    "The debugger will continue the debugging process based on the feedback until every device function is verified to be working correctly. ... Within the 'no' feedback quota limitation, all integrations are ultimately able to achieve 100% functional coverage, as shown in Figure 10 (b)."

    The HIL verification loop asks yes/no questions only about the device-function list that the automated debugger summarized from the vector DBs. Because that list is generated from the same manuals, API docs, and GitHub repos used to write the code, a function missing from the retrieved knowledge is never presented to the human observer and cannot trigger a 'no' response. The reported 100% functional coverage after binary feedback therefore asserts completeness with respect to the self-generated function list, not with respect to an independent enumeration of the device's actual capabilities. The paper does not describe any separate ground-truth inventory for the RealHardware benchmark.

full rationale

AutoBridge is an empirical pipeline rather than a formal derivation, so the circularity question is whether the reported metrics are self-referential. The 93.87% generation success rate retains meaningful independent content: the code must load in the platform and operate in a real execution environment. The 94.87% and 100% functional coverage claims, however, are weakened by construction. In Section 3.1.2 the device-specific vector DB is built from retrieved manuals, API/SDK documentation, and GitHub repositories; in Section 3.2.2 the same vector DBs are used to summarize the device-function list that the automated debugger tests against; and in Sections 3.3 and 5.4 the hardware-in-the-loop verification and the 100% coverage result are described relative to 'every device function' without an independent ground-truth inventory. EvalSet 2 provides some independent signal because its function-specific tests are constructed from official human-written integrations, and the manual screening disclaimer prevents exact copy-paste leakage. However, the headline averages merge EvalSet 1 and EvalSet 2, and the paper does not report coverage separately in a way that isolates the self-referential part. The coverage metric is therefore partially circular: it measures the system against the knowledge the pipeline already had, rather than against an externally fixed device-function ground truth.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the completeness of the documentation that AutoBridge retrieves, on the validity of virtual-device testing as a proxy for real platform behavior, and on the reliability of binary human feedback. These are empirical assumptions, not mathematical axioms, and none is independently verified in the paper.

free parameters (2)
  • Retrieval top-k count = 5
    Section 4 Implementation: 'retaining the top 5 items from each retrieval'. Chosen by hand with no ablation; affects the knowledge available to code generation and debugging.
  • Maximum 'no' feedback cap = 10
    Section 5.2: 'The maximum number of no feedback per device function is capped at 10.' This is a stopping rule for HIL debugging; no sensitivity analysis is provided.
assumptions (3)
  • domain assumption Completeness of retrieved device documentation
    Section 3.1.2: the generator and test generator both rely on the device-specific vector DB built from search APIs and GitHub. If a function is missing from these sources, it is neither generated nor tested, so coverage is overestimated.
  • domain assumption Virtual device testing is a valid proxy for platform compliance
    Section 3.2.2: the automated debugger registers a virtual device and runs tests in the platform core environment; the paper assumes that passing these tests implies the integration will load and bind services correctly on real hardware, which is only partially addressed by the later HIL stage.
  • domain assumption Binary human feedback accurately reflects device behavior
    Section 3.3: the HIL debugger relies on yes/no answers from a human observer to decide whether a function works; this assumes the observer can reliably judge the physical outcome.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AutoBridge: Automating Smart Device Integration with Centralized Platform." pith.science (2026). https://pith.science/paper/3DXNCI22

@misc{pith2026250723178,
  author       = {Pith},
  title        = {Pith review of: AutoBridge: Automating Smart Device Integration with Centralized Platform},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3DXNCI22}},
  note         = {Machine review of arXiv:2507.23178}
}
read the original abstract

Multimodal IoT systems coordinate diverse IoT devices to deliver human-centered services. The ability to incorporate new IoT devices under the management of a centralized platform is an essential requirement. However, it requires significant human expertise and effort to program the complex IoT integration code that enables the platform to understand and control the device functions. Therefore, we propose AutoBridge to automate IoT integration code generation. Specifically, AutoBridge adopts a divide-and-conquer strategy: it first generates device control logic by progressively retrieving device-specific knowledge, then synthesizes platformcompliant integration code using platform-specific knowledge. To ensure correctness, AutoBridge features a multi-stage debugging pipeline, including an automated debugger for virtual IoT device testing and an interactive hardware-in-the-loop debugger that requires only binary user feedback (yes and no) for real-device verification. We evaluate AutoBridge on a benchmark of 34 IoT devices across two open-source IoT platforms. The results demonstrate that AutoBridge can achieves an average success rate of 93.87% and an average function coverage of 94.87%, without any human involvement. With minimal binary yes and no feedback from users, the code is then revised to reach 100% function coverage. A user study with 15 participants further shows that AutoBridge outperforms expert programmers by 50% to 80% in code accuracy, even when the programmers are allowed to use commercial code LLMs.

Figures

Figures reproduced from arXiv: 2507.23178 by the authors.

Figure 1
Figure 1. Personalized multi-modal IoT systems need to sup [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Integration code example showcases its "bridge" na [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. IoT Integration Generation with GPT-4o-mini-high [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: IoT Integration Generation with Coding Copilot [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]
Figure 5
Figure 5. Figure 5: Error Types and Distribution code is expected to cover core functionalities of the fan, includ￾ing power on/off, timer settings, speed control, oscillation, etc. As depicted in [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: System Overview of AutoBridge. Web: [other query] Device-specific Vector DB Platform-specific Vector DB Query/Results Query/Results Query/Results Query/Results Device Control Code Generation from libdyson import ... # …. class DysonFanLogic: def __init__(self, device) …
Figure 7
Figure 7. Figure 7: Workflow of Integration Code Generator. It first [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]
Figure 8
Figure 8. Figure 8: The prompt skeleton for (a) integration code Generator, (b) Test Code Generation, (c) Automated Debugger and (d) [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 9
Figure 9. Figure 9: Workflow of AutoBridge’s Multi-stage Debugging Framework. [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: Benchmark-based Performance Evaluation: (a) [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 11
Figure 11. Figure 11: Ablation Study: (a) impact of different backbone LLM models; (b) impact of prompt granularity; (c) impact of [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 12
Figure 12. Figure 12: AutoBridge’s User Study (b) Impact of Intermediate Device Control Code. As shown in [PITH_FULL_IMAGE:figures/full_fig_p011_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

73 extracted references · 46 canonical work pages

  1. [1]

    OpenAI Assistants Code Interpreter

    Online. OpenAI Assistants Code Interpreter . https://platform.openai.com/docs/ assistants/tools/code-interpreter Online; Last accessed Apr. 2025

    work page 2025

  2. [2]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774 N/A, N/A (2023)

    arXiv 2023

  3. [3]

    Anthropic. 2025. Claude-4-Sonnet. https://www.anthropic.com/news/claude-4

    work page 2025

  4. [4]

    Anthropic. 2024. Claude-3-Sonnet. https://www.anthropic.com/claude, Accessed:2024-04-01

    work page 2024

  5. [5]

    Home Assistant. 2017. Home Assistant. https://www.home-assistant.io/, Accessed:2024-02-09

    work page 2017

  6. [6]

    Home Assistant. 2017. Home Assistant Guidance to Develop Integrations. https: //developers.home-assistant.io/docs/development_index/, Accessed:2025-04-22

    work page 2017

  7. [7]

    Home Assistant. 2017. Official Integration Examples of Home Assistant. https: //www.home-assistant.io/examples/, Accessed:2024-02-13

    work page 2017

  8. [8]

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program Synthesis with Large Language Models. arXiv preprint arXiv:2108.07732 (2021)

    arXiv 2021

Show all 73 references
  1. [9]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Han...

    2025 arXiv

  2. [10]

    Aaron Bangor, Philip T Kortum, and James T Miller. 2008. An empirical evaluation of the system usability scale. Intl. Journal of Human–Computer Interaction 24, 6 (2008), 574–594

    2008

  3. [11]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating Large Language Models Trained on Code. arXiv preprint arXiv:2107.03374 (2021)

    2021 arXiv

  4. [12]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Se- bastian Gehrmann, Parker Schuh, Kensen Shi, Sashank Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, ...

    2023

  5. [13]

    Cursor. 2024. Cursor. https://www.cursor.com/, Accessed:2024-07-01

    2024

  6. [14]

    Google DeepMind. 2023. Gemini 1.0. https://deepmind.google/technologies/ gemini/, Accessed:2024-04-01

    2023

  7. [15]

    Google DeepMind. 2024. Gemini Code Assist. https://codeassist.google/, Accessed:2025-07-01

    2024

  8. [16]

    DeepSeek-AI. 2024. DeepSeek LLM: Scaling Open-Source Language Models with Longtermism. arXiv preprint arXiv:2401.02954 (2024). https://github.com/ deepseek-ai/DeepSeek-LLM

    2024 arXiv

  9. [17]

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, et al. 2024. A Survey on In-context Learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 1107–1128

    2024

  10. [18]

    Yihong Dong, Xue Jiang, Zhi Jin, and Ge Li. 2023. Self-collaboration Code Generation via ChatGPT. arXiv preprint arXiv:2304.07590 (2023)

    2023 arXiv

  11. [19]

    Shihan Dou, Yan Liu, Haoxiang Jia, Enyu Zhou, Limao Xiong, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, Zhiheng Xi, et al . 2024. Step- Coder: Improving Code Generation with Reinforcement Learning from Compiler Feedback. In Proceedings of the 62nd Annual Meeting of th...

    2024

  12. [20]

    Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2024. The Faiss library. (2024). arXiv:2401.08281 [cs.LG]

    2024 arXiv

  13. [21]

    Zachary Englhardt, Richard Li, Dilini Nissanka, Zhihan Zhang, Girish Narayan- swamy, Joseph Breda, Xin Liu, Shwetak Patel, and Vikram Iyer. 2024. Explor- ing and Characterizing Large Language Models for Embedded System Devel- opment and Debugging. In Extended Abstracts of the ...

    2024

  14. [22]

    FAISS. 2025. FAISS: IndexFlatL2. https://faiss.ai/cpp_api/struct/structfaiss_1_ 1IndexFlatL2.html, Accessed:2025-04-26

    2025

  15. [23]

    Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. A Survey on RAG meeting LLMs: Towards Retrieval-Augmented Large Language Models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining ...

    2024

  16. [24]

    Jia Feng, Jiachen Liu, Cuiyun Gao, Chun Yong Chong, Chaozheng Wang, Shan Gao, and Xin Xia. 2024. ComplexCodeEval: A Benchmark for Evaluating Large Code Models on More Complex Code. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering ....

    2024

  17. [25]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2023. Retrieval-Augmented Gen- eration for Large Language Models: A Survey. arXiv preprint arXiv:2312.10997 (2023)

    2023 arXiv

  18. [26]

    Github. 2024. Github Copilot. https://github.com/features/copilot, Accessed:2025- 07-01

    2024

  19. [27]

    Github. 2025. GitHub REST API. https://docs.github.com/en/rest?apiVersion= 2022-11-28, Accessed:2025-04-26

    2025

  20. [28]

    Kaijie Gong, Wei Dong, Yingqi Peng, Hao Wang, and Yi Gao. 2024. Poster: Enabling IoT Application Programming in Natural Language with IoTPilot. In Proceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems . 903–904

    2024

  21. [29]

    Google. 2025. Google’s Custom Search JSON API. https://developers.google. com/custom-search/v1/overview?hl=zh-cn, Accessed:2025-04-26

    2025

  22. [30]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y Wu, YK Li, et al. 2024. DeepSeek-Coder: When the Large Language Model Meets Programming–The Rise of Code Intelligence.arXiv preprint arXiv:2401.14196 (2024)

    2024 arXiv

  23. [31]

    Junda He, Christoph Treude, and David Lo. 2025. LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead. ACM Trans. Softw. Eng. Methodol. (Jan. 2025). doi:10.1145/3712003 Just Accepted

    2025 doi

  24. [32]

    Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, et al . 2021. Measuring Coding Challenge Competence With APPS. In Thirty-fifth Conference on Neural Information Processing Systems Datasets ...

    2021

  25. [33]

    Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, et al

  26. [34]

    IFTTT. 2024. IFTTT:Automation for Business and Home. https://ifttt.com/, Accessed:2024-09-06

    2024

  27. [35]

    Brian Kenji Iwana and Seiichi Uchida. 2020. An Empirical Survey of Data Augmen- tation for Time Series Classification with Neural Networks.CoRR abs/2007.15951 (2020). arXiv:2007.15951 https://arxiv.org/abs/2007.15951

    2020 arXiv

  28. [36]

    Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024. A Survey on Large Language Models for Code Generation. arXiv preprint arXiv:2406.00515 (2024)

    2024 arXiv

  29. [37]

    Evan King, Haoxiang Yu, Sangsu Lee, and Christine Julien. 2024. Sasha: Creative Goal-Oriented Reasoning in Smart Homes with Large Language Models. 8, 1, Article 12 (mar 2024), 38 pages. doi:10.1145/3643505

    2024 doi

  30. [38]

    Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettle- moyer, Wen-tau Yih, Daniel Fried, Sida Wang, and Tao Yu. 2023. DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation. In Interna- tional Conference on Machine Learning . PMLR, 18319–18345

    2023

  31. [39]

    Fu Li, Jiaming Huang, Yi Gao, and Wei Dong. 2023. ChatIoT: Zero-code Gen- eration of Trigger-action Based IoT Programs with ChatGPT. In Proceedings of the 7th Asia-Pacific Workshop on Networking (Hong Kong, China) (APNet ’23). Association for Computing Machinery, New York, NY,...

    2023

  32. [40]

    R Li, LB Allal, Y Zi, N Muennighoff, D Kocetkov, C Mou, M Marone, C Akiki, J Li, J Chim, et al. 2023. StarCoder: May the Source be With You! Transactions on machine learning research (2023). Liu et al

    2023

  33. [41]

    Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. 2022. Competition-level Code Generation with AlphaCode. Science 378, 6624 (2022), 1092–1097

    2022

  34. [42]

    Liang, Chenyang Yang, and Brad A

    Jenny T. Liang, Chenyang Yang, and Brad A. Myers. 2024. A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (Lisbon, Portugal) (ICSE ’24). Association...

    2024

  35. [43]

    Kaiwei Liu, Bufang Yang, Lilin Xu, Yunqi Guo, Guoliang Xing, Xian Shuai, Xi- aozhe Ren, Xin Jiang, and Zhenyu Yan. 2025. TaskSense: A Translation-like Approach for Tasking Heterogeneous Sensor Systems with LLMs. In Proceedings of the 23rd ACM Conference on Embedded Networked S...

    2025

  36. [44]

    Llama. 2024. CodeLlama-70B. https://llama-2.ai/codellama-70b/, Accessed:2024- 04-01

    2024

  37. [45]

    Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2024. WizardCoder: Empowering Code Large Language Models with Evol-Instruct. In The Twelfth International Conference on Learning Representations

    2024

  38. [46]

    Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2022. CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis. InThe Eleventh International Conference on Learning Representations

    2022

  39. [47]

    OpenAI. 2025. GPT-o3. https://platform.openai.com/docs/models/o3, Accessed:2025-07-01

    2025

  40. [48]

    OpenAI. 2025. GPT-o4-mini. https://openai.com/index/introducing-o3-and-o4- mini/

    2025

  41. [49]

    OpenAI. 2025. Hello GPT-4o. https://openai.com/index/hello-gpt-4o/, Accessed:2025-04-26

    2025

  42. [50]

    OpenAI. 2025. New and Improved Embedding Model. https://openai.com/index/ new-and-improved-embedding-model/, Accessed:2025-04-26

    2025

  43. [51]

    openHAB. 2019. open Home Automation Bus (openHAB). https://www.openhab. org/, Accessed:2024-04-14

    2019

  44. [52]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al

  45. [53]

    Huy Nhat Phan, Hoang Nhat Phan, Tien N Nguyen, and Nghi DQ Bui. 2024. Repohyper: Better Context Retrieval is All You Need for Repository-level Code Completion. CoRR (2024)

    2024

  46. [54]

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiao- qing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. 2023. Code Llama: Open Foundation Models for Code. arXiv preprint arXiv:2308.12950 (2023)

    2023 arXiv

  47. [55]

    Leming Shen, Qiang Yang, Xinyu Huang, Zijing Ma, and Yuanqing Zheng. 2025. GPIoT: Tailoring Small Language Models for IoT Program Synthesis and De- velopment. In Proceedings of the 23rd ACM Conference on Embedded Networked Sensor Systems. 199–212

    2025

  48. [56]

    Leming Shen, Qiang Yang, Yuanqing Zheng, and Mo Li. 2025. AutoIOT: LLM- Driven Automated Natural Language Programming for AIoT Applications. In Proceedings of the 31st Annual International Conference on Mobile Computing and Networking. 1–15

    2025

  49. [57]

    CodeGemma Team, Heri Zhao, Jeffrey Hui, Joshua Howland, Nam Nguyen, Siqi Zuo, Andrea Hu, Christopher A Choquette-Choo, Jingyue Shen, Joe Kelley, et al. 2024. CodeGemma: Open Code Models based on Gemma. arXiv preprint arXiv:2406.11409 (2024)

    2024 arXiv

  50. [58]

    Runchu Tian, Yining Ye, Yujia Qin, Xin Cong, Yankai Lin, Yinxu Pan, Yesai Wu, Hui Haotian, Liu Weichuan, Zhiyuan Liu, et al. 2024. DebugBench: Evaluating Debugging Capability of Large Language Models. In Findings of the Association for Computational Linguistics ACL 2024 . 4173–4198

    2024

  51. [59]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. LLaMA: Open and Efficient Foundation Language Models. arXiv preprint arXiv:2302.13971 (2023)

    2023 arXiv

  52. [60]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-Thought Prompting Elicits Rea- soning in Large Language Models. Advances in Neural Information Processing Systems 35 (2022), 24824–24837

    2022

  53. [61]

    Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, Zhenda Xie, Yu Wu, Kai Hu, Jiawei Wang, Yaofeng Sun, Yukun Li, Yishi Piao, Kang Guan, Aixin Liu, Xin Xie, Yuxiang You, Kai Dong, Xingkai Yu, Haowei Zhang,...

    2024 arXiv

  54. [62]

    Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. C-Pack: Packaged Resources To Advance General Chinese Embedding. arXiv:2309.07597 [cs.CL]

    2023 arXiv

  55. [63]

    Xiaomi. 2025. Xiaomi Smart Temperature and Humidity Monitor

    2025

  56. [64]

    Liwenhan Xie, Chengbo Zheng, Haijun Xia, Huamin Qu, and Chen Zhu-Tian

  57. [65]

    Hu Xu, Gargi Ghosh, Po-Yao Huang, Prahal Arora, Masoumeh Aminzadeh, Christoph Feichtenhofer, Florian Metze, and Luke Zettlemoyer. 2021. Vlm: Task- agnostic video-language model pre-training for video understanding. arXiv preprint arXiv:2105.09996 (2021)

    2021 arXiv

  58. [66]

    https://www.mi.com/global/product/xiaomi-smart-temperature-and- humidity-monitor-3/, Accessed:2025-04-26

    2025

  59. [67]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. ReAct: Synergizing Reasoning and Acting in Language Models. In The Eleventh International Conference on Learning Representations . N/A, N/A, N/A

    2022

  60. [68]

    In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology (Pittsburgh, PA, USA) (UIST ’24)

    WaitGPT: Monitoring and Steering Conversational LLM Agent in Data Analysis with On-the-Fly Code Visualization. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology (Pittsburgh, PA, USA) (UIST ’24). Association for Computing Machinery, New Y...

  61. [69]

    Fengji Zhang, Bei Chen, Yue Zhang, Jacky Keung, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, and Weizhu Chen. 2023. RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural L...

    2023 doi

  62. [70]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree-of-Thoughts: Deliberate Problem Solving with Large Language Models. Advances in Neural Information Processing Systems 36 (2024)

    2024

  63. [72]

    Yeelight. 2025. Yeelight LED Bulb 1S. https://www.yeelight.com/en_US/product/ lemon2-ct, Accessed:2025-04-26

    2025

  64. [2022]

    Advances in Neural Information Processing Systems 35 (2022), 27730–27744

    Training Language Models to Follow Instructions with Human Feedback. Advances in Neural Information Processing Systems 35 (2022), 27730–27744

    2022

  65. [2024]

    In The Twelfth International Conference on Learning Representations

    MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. In The Twelfth International Conference on Learning Representations

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.