{"id":"c6901168-4bd2-41a0-a5c1-356b5eef24e5","arxiv_id":"2412.13693","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"UITrans is an LLM-based tool that translates Android XML UIs to HarmonyOS ArkUI, reporting over 89% line-based success on six open-source Android apps.","lead":"This paper introduces UITrans, a tool that automatically converts Android UI layouts into HarmonyOS ArkUI code using LLM agents, retrieval, and reflection. A developer might use it to speed up porting Android apps to HarmonyOS instead of rewriting the interface by hand.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed >90% success hinges on a subjective 'Lines Modified' metric with no rubric or runtime equivalence test; until the metric is validated, the headline numbers are not established.","rationale":"I read the paper as an engineering contribution whose value depends on whether the reported translation success rates are believable. The architecture—Android project parsing, task decomposition, LLM-driven multi-agent generation with RAG and reflective regeneration—is plausible, and the authors provide a public repository and demo, which is creditworthy. The central claim, however, is an empirical one about success rates, and that claim rests on a metric that is not reproducible. The reader's weakest-assumption analysis identifies the same soft spot: Lines Modified in Appendix B is a manual, unprotocoled count with no inter-rater check and no functional equivalence test. I do not see an internal logical contradiction in the approach, and the selection of six feasible apps is not itself fatal if the claims were limited to those apps and the metric were objective. But the abstract and conclusion generalize the percentages, and the table shows that only a minority of components/pages compiled unchanged; the rest of the high success rate comes from the authors' own judgment of what needed modification. This is load-bearing because if another evaluator used a stricter rubric, the numbers could drop substantially. The concrete test I propose—objective build plus visual comparison on a held-out, not pre-screened set—would settle whether the claimed success rates reflect real translation quality or just a lenient line-edit metric. Until that test is run, the appropriate verdict remains conditional, which is exactly what the reader recommended; my stress-test does not move the recommendation.","tokens_in":7213,"tokens_out":2658,"duration_ms":28205,"concrete_test":"Run the released UITrans tool on a held-out set of 20 randomly selected open-source Android apps that are not pre-screened for buildability; for each generated HarmonyOS project, run the standard hvigor build and then compare screenshots of each Android page against the corresponding HarmonyOS page on emulators using automated UI hierarchy or pixel-diff analysis. Report the fraction of projects that build and whose pages visually match within a defined tolerance. If that objective pass rate is materially below the Appendix B percentages, the claimed success rates are an artifact of the manual line-edit metric rather than genuine translation quality.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of over 90.1%, 89.3%, and 89.2% success depends entirely on Appendix B's definition of Comp. Success, Page Success, and Project Success as 1 - Lines Modified / Total Lines. These variables are never operationalized: there is no protocol for what counts as a modified line, no inter-rater reliability check, no compile or runtime equivalence test, and no independent replication. Table 1's parenthetical counts, which show components and pages that compiled without modification, are much lower than the headline percentages—for example, forecastie has 0 pages compiled unchanged yet Page Success is 89.3%. This means the high percentages are driven by the authors' manual judgment of which lines needed modification, a judgment that can be influenced by code formatting, verbosity, or tolerance of semantic drift. The denominator is also the total lines of the translated output, so if the LLM generates verbose or differently formatted ArkUI code, the ratio is not comparable across projects and can be inflated. Because the evaluation set was pre-filtered to the six apps that could be built, compiled, and previewed, the results cannot support a general claim about Android-to-HarmonyOS translation quality. The paper is a plausible engineering contribution, but the empirical evidence for the central success-rate claim is weaker than the precise percentages suggest.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents UITrans, a tool that uses an LLM-driven multi-agent reflective collaboration framework to translate Android XML layouts into HarmonyOS ArkUI layouts. The approach consists of three phases: Android project parsing, translation task planning and decomposition, and HarmonyOS code generation with RAG and a UI mapping table, followed by reflective regeneration. The authors evaluate UITrans on six open-source Android apps and report component, page, and project-level success rates of over 90.1%, 89.3%, and 89.2%, respectively, comparing against a direct prompt-based LLM baseline. The tool, prompts, benchmark dataset, online service, and demo video are publicly released.","tokens_in":7579,"tokens_out":3945,"duration_ms":34887,"significance":"If the reported success rates were validated, UITrans would be a valuable contribution to cross-platform UI migration, addressing a real industrial pain point with a publicly available, coherently designed tool. The paper's strengths include a reproducible architecture, release of the implementation and prompts, a benchmark dataset, and an objective baseline comparison (Only-prompt) in which UITrans consistently outperforms the baseline. The parenthetical compile counts in Table 1 provide a machine-checkable signal that many components and pages indeed compile without modification. However, the headline success percentages rest on a subjective manual line-edit metric that is not operationalized, and the evaluation set is pre-filtered to apps that could already build and preview. The significance of the central empirical claim is therefore conditional on strengthening the evaluation protocol.","major_comments":[{"comment":"The success metrics are defined as one minus a ratio of 'Lines Modified' to 'Total Lines,' but 'Lines Modified' is never operationalized. There is no protocol for what constitutes a modified line, no rubric, no inter-rater reliability check, and no independent verification that the modified lines restore functional equivalence with the Android UI. The note in Appendix B stating that 'Lines Modified represent the number of modified code lines ensuring that the translated HarmonyOS code is functionally consistent with the Android UI' is an assertion, not a measurement. Without a concrete counting protocol, the headline figures of over 90.1%, 89.3%, and 89.2% are not reproducible or verifiable.","section":"Appendix B, Eqs. (1)-(3)"},{"comment":"The parenthetical counts in Table 1 report the number of components and pages that 'were successfully built and compiled without any modifications.' For forecastie, the Page Success is 89.3% while the number of pages compiled without modification is 0. This example shows that the high page success percentage is driven by the authors' manual judgment of which lines needed modification, not by objective compile success. Since a page can receive a high Page Success score even when no page compiles unchanged, the paper should either report compile-based success rates as the primary evidence or provide a demonstrated correlation between the line-edit ratio and functional correctness (e.g., screenshot comparisons, runtime tests, or an independent developer review).","section":"Table 1 (forecastie row)"},{"comment":"The evaluation is restricted to 'the top six Android applications that could be successfully built, compiled, and previewed the UI from GitHub.' This pre-filtering excludes apps that fail to build, compile, or preview, which are likely the harder cases for translation. The abstract and conclusion then generalize to Android applications without this caveat, and the stated success rates apply only to this selected set. The paper should either scope the claims explicitly to buildable/previewable apps or address the selection bias by including a consecutive or random sample of Android apps and reporting the build/preview failure rate.","section":"Section 4, dataset selection"},{"comment":"The denominator in the success metrics is 'Total Lines,' defined as the total lines in the translated output. If the LLM generates verbose or differently formatted ArkUI code, the same number of functional modifications produces a higher success score simply because the denominator grows, and the metric is not comparable across projects or methods. The paper does not report raw modified-line and total-line counts, nor does it normalize by the source Android line counts. Adding these raw counts would allow readers to assess whether the percentages are inflated by output verbosity.","section":"Appendix B, Eq. (1) and Table 1"},{"comment":"The reflective regeneration step uses a secondary LLM-driven agent that compares generated code with the original functional description and feeds discrepancies back for re-generation. This is a self-assessment within the same LLM pipeline, not an independent validation of functional equivalence. The paper should include some external validation of functional consistency, such as automated UI tests, runtime equivalence checks, or manual QA by a developer who was not involved in building the tool.","section":"Section 2.3.1, Step 3"}],"minor_comments":[{"comment":"There is a typo in 'evluate' (should be 'evaluate') in the sentence describing the experiments, and 'we are the first to provide a benchmarks' should be 'a benchmark'.","section":"Section 1"},{"comment":"The statement that 'more than 72.5% of components were successfully translated without requiring any modifications' should cite the corresponding rows of Table 1 and explain whether this is a weighted average or an unweighted mean across projects.","section":"Section 4"},{"comment":"For Transportr, the 'Complex%' column shows 0.0% while the Components column lists '15/9', suggesting 9 complex components out of 15; these entries appear inconsistent and should be checked.","section":"Appendix A, Table 2"},{"comment":"The sentence 'including ules and query instructions' appears to have a typo; it should likely read 'rules and query instructions'.","section":"Section 2.2"},{"comment":"The paper cites [4] as an existing rule-based Android-to-iOS migration approach but does not compare UITrans against it. Adding such a comparison, even qualitative, would help position the contribution against prior automated migration work.","section":"Related Work / Evaluation"}],"recommendation":"major_revision","confidential_remarks":"The paper is a promising systems contribution with a public release, but the central success-rate claim is not yet established because the 'Lines Modified' metric is not operationalized. A revised version with a rigorous evaluation protocol, inter-rater reliability, compile/runtime equivalence evidence, and a more transparent reporting of raw counts could turn this into a strong paper. The authors should also be asked to temper the 'first automated UI translation tool' claim unless they conduct a more systematic comparison with existing migration tools."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"UITrans is the first Android-to-HarmonyOS translation pipeline I've seen in the literature, and the released artifact—code, prompts, demo, mapping table—is real engineering credit. The multi-agent LLM setup with RAG and reflective regeneration is a sensible integration of known pieces, not a breakthrough in any single technique. If the headline numbers held up, it would be a meaningful step for developers porting UI-heavy apps.\n\nThey don't fully hold up. The Appendix B metrics define success as 1 - lines modified / total lines, but nothing operationalizes what counts as a modified line. There is no rubric, no inter-rater check, no runtime equivalence test. Table 1's parenthetical compile-without-modification counts are far lower than the percentages—forecastie has 0 pages compile unchanged yet shows 89.3% page success. That gap tells you the percentages are driven by manual judgment, not an objective outcome. The denominator being total lines of generated code also makes the ratio sensitive to how verbose or differently formatted the LLM output is, so cross-project comparisons are shaky. And the six apps were pre-filtered to those that could build, compile, and preview, which excludes hard cases by construction. The only baseline is direct prompting; no rule-based migration tool or manual effort is compared.\n\nThe engineering itself is coherent, the writing is clear, and the claims about the system are mostly in line with what the architecture does. The weak point is the evaluation, not the construction. The paper reads as an honest early report rather than a hype job, but the abstract's precise percentages overstate what the evidence supports.\n\nWho is this for: tool builders in cross-platform migration and developers weighing HarmonyOS ports. It deserves a serious referee because it is a first result in a live area with a public artifact, but the reviewer should demand a scoring protocol, compile and runtime success as primary metrics, a larger and less filtered dataset, and at least one non-LLM baseline. I'd engage with it. Send it to review with the expectation of heavy revision.","headline":"A genuine first Android-to-HarmonyOS translation pipeline whose headline numbers outrun the evidence behind them.","tokens_in":7972,"tokens_out":2414,"would_cite":true,"duration_ms":22308,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new tool claims to auto-translate Android UIs to HarmonyOS with over 90% success.","keywords":["UI translation","Android","HarmonyOS","ArkUI","LLM","multi-agent","retrieval-augmented generation","code generation"],"falsifier":"Take the six benchmark apps, run UITrans, and have independent developers—blind to the tool's output—mark every generated line they had to change to make the HarmonyOS UI render and behave like the Android original, then compare their counts with the paper's numbers; alternatively, automate a screenshot or view-hierarchy comparison between the original Android app and the generated HarmonyOS app on a held-out set of compilable apps. If the independently measured success rates fall well below 90%, the paper's central claim fails.","tokens_in":7014,"feed_emoji":"🔄","tokens_out":5413,"duration_ms":43519,"temperature":0.7,"pith_summary":"The paper tries to establish that automated translation of Android user interfaces into HarmonyOS ArkUI layouts is feasible at high fidelity, not just for simple widgets but for whole projects with multiple pages and interaction logic. It presents UITrans, an LLM-driven tool that takes an Android project's source code and produces compilable HarmonyOS code, and reports success rates above 90% at component, page, and project levels on six open-source apps. A sympathetic reader would care because manual rewrites of mobile UIs are costly and error-prone, and a working translator would let developers reuse Android UI work when moving to HarmonyOS. The paper also argues that its component, page, and project benchmark fills a gap by giving the community a common evaluation set for this translation task.","feed_headline":"Android UIs auto-translate to HarmonyOS at ~90% success","feed_subtitle":"LLM pipeline maps Android XML to ArkUI, cutting the manual UI rewrites needed to port mobile apps.","key_machinery":"The central mechanism is a multi-agent reflective collaboration framework: Android project parsing extracts activities, Java/XML dependencies, and interaction logic into an in-memory dictionary; translation task planning decomposes layouts recursively into minimal translatable units; and code generation runs each unit through three stages—domain knowledge retrieval (via a UI mapping table and retrieval-augmented generation), HarmonyOS source-code generation, and a reflective re-generation loop in which a second agent compares generated code against the functional description and feeds discrepancies back for revision. The UI mapping table and RAG knowledge base are the load-bearing resources that ground the LLM's output in correct ArkUI component usage, where ArkUI is HarmonyOS's declarative UI framework.","core_discovery":"On its own terms, the paper discovers that a pipeline combining project parsing, task decomposition, and LLM-driven multi-agent generation with retrieval-augmented domain knowledge and a reflective re-generation step can translate Android XML layouts into ArkUI code with component-, page-, and project-level success rates of 90.1%, 89.3%, and 89.2% respectively, and that this substantially beats a single-prompt LLM baseline on the same six applications. The tool handles component mapping, page dependencies, and interaction logic, and the authors state that the translated code needs only minor fixes, mostly attribute usage or layout alignment, which newcomers can apply.","pith_inferences":["The paper's success measure counts how many generated lines a human had to edit; an independent evaluation using runtime screenshot comparison or blind developer re-counts might yield different numbers, since \"functionally consistent\" is not defined operationally in the paper.","The six-app dataset includes only projects that the authors could build, compile, and preview; apps with heavy third-party libraries or unusual custom views are excluded by construction, so real-world success rates on messy production code are likely lower than reported.","The reflective re-generation loop and growing mapping table are not specific to HarmonyOS: the same architecture could plausibly translate between other declarative UI frameworks, such as Android XML to iOS SwiftUI, if equivalent knowledge bases were built.","A concrete testable extension would be to define success by automated UI-equivalence checks (e.g., screenshot diffing or view-hierarchy comparison) and rerun the same benchmark, which would separate \"lines are similar\" from \"the UI actually renders and behaves the same.\""],"forward_implications":["Developers porting an Android app to HarmonyOS can start from generated ArkUI code rather than writing each screen from scratch; the reported 72.5% of components needing no modification suggests most of the UI survives the translation untouched.","The success metric implies that the generated project-level code, after small fixes, preserves the layout and interaction of the original app, so the tool could lower the barrier for app migration in the HarmonyOS ecosystem.","The benchmark of six Android apps with component, page, and project annotations gives future tool builders a common yardstick for Android-to-HarmonyOS UI translation.","Because unmapped components are recorded back into the UI mapping table, the tool's coverage should grow as it is used, improving translations of custom or less common widgets over time."],"supporting_citations":[{"why":"Supplies the large language model that powers parsing, decomposition, and code generation in UITrans.","marker":"[10]"},{"why":"Motivates retrieval-augmented generation for code translation, the basis for grounding ArkUI generation in domain knowledge.","marker":"[1]"},{"why":"Provides the self-reflection mechanism that drives the reflective re-generation step.","marker":"[18]"},{"why":"Describes the rule-based Android-to-iOS UI migration approach that UITrans compares against as a prior baseline.","marker":"[4]"},{"why":"Offers the multi-agent collaborative framework that informs UITrans's agent-based design.","marker":"[5]"},{"why":"Supports the reflective optimization through verbal reinforcement learning for LLM agents.","marker":"[19]"}],"fun_headline_variants":["LLM agents translate Android UI to HarmonyOS at 90% success","UITrans: LLM-driven Android to HarmonyOS UI translation hits 90%","Android to HarmonyOS UI porting now 90% automated via LLM agents","Multi-agent LLM pipeline converts Android XML to ArkUI at 90%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole evaluation rests on the authors' manual count of \"lines modified\" as a valid measure of whether the translated UI is functionally equivalent to the original Android UI; if that count is subjective or unrepeatable, the reported 90% success rates do not establish that the translation works.","fun_headline_variants_meta":{"raw":{"variants":["LLM agents translate Android UI to HarmonyOS at 90% success","UITrans: LLM-driven Android to HarmonyOS UI translation hits 90%","Android to HarmonyOS UI porting now 90% automated via LLM agents","Multi-agent LLM pipeline converts Android XML to ArkUI at 90%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000224,"raw_usage":{"total_tokens":1416,"prompt_tokens":853,"completion_tokens":563,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":469,"completion_tokens_details":{"reasoning_tokens":479}},"tokens_in":469,"tokens_out":563,"duration_ms":4570,"temperature":1.0,"reasoning_tokens":479,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:53:04.923215+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the six benchmark apps, run UITrans, and have independent developers—blind to the tool's output—mark every generated line they had to change to make the HarmonyOS UI render and behave like the Android original, then compare their counts with the paper's numbers; alternatively, automate a screenshot or view-hierarchy comparison between the original Android app and the generated HarmonyOS app on a held-out set of compilable apps. If the independently measured success rates fall well below 90%, the paper's central claim fails.","supporting_citations":[{"cited_title":"Shinn et al","cited_arxiv_id":null,"evidence_quote":"Supports the reflective optimization through verbal reinforcement learning for LLM agents."}],"review_version":1}