{"id":"1347ee32-d6b2-4736-b4ef-2c1df1be9e5d","arxiv_id":"2608.09273","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"ECAT migrates Android repositories to HarmonyOS through an LLM generator-discriminator loop that minimizes a Code Entropy proxy, scoring 74.7% on a new 50K-300K LOC benchmark.","lead":"Huawei researchers built a two-agent system that automatically migrates Android apps to HarmonyOS: one agent writes code, another scores the result and lists defects to fix. On three real apps between 50,000 and 300,000 lines of code it reports 74.7% average migration quality, and the paper releases a benchmark for this task.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 74.7% claim is not settled: the feature_check entropy dimension (weight 0.30) overlaps Agent-as-Judge, and Code Entropy is never calibrated against human or executable ground truth; removing the overlap and measuring human agreement would settle whether ECAT truly improves functional migration…","rationale":"Reviewed in good faith. The paper's strengths: detailed reproducible protocol, Node Alignment as an independent structural metric, dynamic entropy with emulator screenshots, ablations (discriminator, memory, base LLM), cost reporting, and consistent artifact sizes. These support that ECAT produces large quantities of HarmonyOS code that builds and is structurally aligned. However, the central claim is 'functionally complete migration quality at 74.7%', and the only metric that measures function is Agent-as-Judge, an LLM judge explicitly designed to be static and tolerant of missing runtime proof (C.2). ECAT's optimization objective contains a near-twin of this metric (feature_check, weight 0.3). The paper's assertion that entropy dimensions are application-agnostic is strained by the feature_check definition. The concern is not that Code Entropy is 'not real entropy' (the paper acknowledges the analogy), nor that self-citation is involved; it is that the acceptance guarantee in Eq. (6) is only as good as the estimator, and estimator validity is never demonstrated. The proposed test — ablating feature_check and checking human-agent agreement — would settle whether ECAT's margin reflects genuine functional migration or LLM-judge alignment. Since the concern is testable but currently unresolved, the reader's CONDITIONAL verdict remains appropriate; no change.","tokens_in":19898,"tokens_out":8169,"duration_ms":82411,"concrete_test":"Re-run ECAT on all three A2H-RepoBench repositories with the feature_check dimension removed from Code Entropy (renormalize remaining weights), using the same seeds, LLMs, and thresholds as Table 1; compare Agent-as-Judge and Node Alignment against the reported 74.7%. Independently, have two human annotators score a random sample of 50 features per repository with the same Full/Partial/Missing rubric and compute Cohen's kappa against the Agent-as-Judge scores for both original and feature-check-free outputs. If the ECAT margin over ReCodeAgent collapses without feature_check, or human-agent agreement is below κ≈0.6, the central functional claim is an artifact of proxy/judge overlap rather than validated entropy-driven migration.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Sec. 4.3/Table 1's headline claim — average migration quality 74.7% vs. 46.4% — is only meaningful if Eq. (6)'s acceptance rule tracks true functional completeness. The paper never calibrates Code Entropy (Eq. 2): there is no correlation against human-verified functionality, no executable test-suite score, and no study of estimator bias. Consequently, the 'monotonic reduction' guarantee in Eq. (6) is definitional with respect to an unvalidated estimate. The risk is concrete: Table 5's feature_check dimension (weight 0.3) is semantically the same construct as the Agent-as-Judge metric — both score per-feature completeness against the Android counterpart; the discriminator is denied the checklist, but its feature_check prompt can reconstruct per-feature completeness from the Android source. Sec. 3.2's assertion that no entropy dimension carries application-specific functional requirements is therefore inconsistent with feature_check's definition. Table 2's w/o-discriminator ablation shows self-evaluation overestimation matters, but it does not establish that the independent LLM judge is calibrated. Since Sec. C.2 states Agent-as-Judge is static and does not require runtime proof, and no human agreement is reported, the reported functional advantage may reflect optimizing an LLM-judge-like objective rather than measurable migration quality.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ECAT, a generator–discriminator framework for Android-to-HarmonyOS repository migration. The discriminator estimates a weighted Code Entropy objective (Eq. 2) over 14 static and dynamic dimensions and emits structured text gradients; the generator proposes repository updates, and Eq. (6) accepts an update only when estimated Code Entropy decreases. Successful trajectories are stored in a self-evolving memory tree for cross-repository transfer. The authors introduce A2H-RepoBench, containing three real-world Android repositories (50K, 120K, and 300K LOC), and report that ECAT improves average migration quality from 46.4% (best baseline) to 74.7% under Node Alignment and Agent-as-Judge metrics.","tokens_in":20227,"tokens_out":3616,"duration_ms":36628,"significance":"If the reported results hold, the paper would make a substantial contribution to long-horizon repository migration: it provides a new multiscale benchmark (A2H-RepoBench), a reproducible evaluation skill (a2h-evaluate), an open-source implementation, and a clear adversarial-optimization architecture that decouples generation from evaluation. The detailed appendices on baseline prompting, evaluation grounding, and platform-friction filtering are valuable for reproducibility. However, the central claim depends critically on whether the optimized Code Entropy objective tracks true functional migration quality, and the current evidence for that link is incomplete; this must be addressed before the headline result is acceptable.","major_comments":[{"comment":"The headline claim that ECAT improves average migration quality from 46.4% to 74.7% is not yet settled because the optimization objective overlaps the evaluation metric. Table 5 defines feature_check (weight 0.30) as 'per-feature completeness against the Android counterpart', estimated by an LLM judge, while Agent-as-Judge (Eq. 9 and Sec. C.2) scores per-feature completeness against a sealed Android-derived feature checklist. These are semantically the same construct, even if the discriminator is not given the literal evaluation checklist. The paper should report: (i) the correlation between Code Entropy and Agent-as-Judge with the feature_check dimension excluded, (ii) an ablation that removes feature_check from the optimization objective, and (iii) a human or executable ground-truth evaluation on at least a sample of features to show that entropy reduction corresponds to real functional improvement.","section":"Sec. 4.1, Sec. 4.3, Table 5"},{"comment":"The statement that Eq. (6) 'guarantees that every accepted repository update monotonically reduces estimated Code Entropy' is correct, but it is a property of the estimator, not of migration quality. The acceptance rule guarantees monotone decrease of H(R), which is computed by the discriminator's rule-based and LLM-based estimators; it does not by itself guarantee monotone improvement in functional completeness. The paper never calibrates Code Entropy against human-verified functionality, an executable test suite, or any independent ground truth. I recommend adding a calibration study, e.g., comparing Code Entropy scores with human judgments on a stratified sample of migrated repositories, and reporting estimator bias and variance across iterations.","section":"Sec. 3.3, Eq. (6)"},{"comment":"The assertion in Sec. 3.2 that 'all entropy dimensions are defined at the level of generic migration defects and contain no application-specific functional requirements' is inconsistent with the feature_check dimension in Table 5, which explicitly evaluates 'per-feature completeness against the Android counterpart'. Per-feature completeness is an application-specific functional property. If the intended meaning is that the discriminator does not receive the curated evaluation checklist, the paper should state this precisely and provide the actual feature_check prompt so readers can assess how much application-specific information the LLM judge recovers from the Android source. As written, the isolation claim is too strong.","section":"Sec. 3.2, Table 5"},{"comment":"The w/o-Discriminator ablation demonstrates that replacing the independent evaluator with self-evaluation degrades performance, which supports the adversarial-design motivation. However, it does not establish that the independent LLM discriminator is well calibrated: a self-consistent but biased judge could still drive convergence to a low estimated entropy without improving real functionality. The conclusion that 'decoupling repository generation from quality evaluation' is the cause of the gain should be supported by evidence that the discriminator's entropy estimates agree with an external ground truth, not just that they differ from the generator's self-estimates.","section":"Sec. A.1, Table 2"}],"minor_comments":[{"comment":"The Node Alignment score is defined in Eq. (8) as |M|/|Vs|, but Sec. C.3 defines the denominator as D_s, the pre-filter Android retention denominator after excluding platform-friction entities. Please reconcile the notation and state clearly whether |Vs| in Eq. (8) is the raw node count or D_s.","section":"Sec. 4.1, Eq. (8); Sec. C.3"},{"comment":"Figure 5(a) shows a threshold at 0.01, but the stopping criterion in Eq. (7) also requires a sliding window of size l=2. The caption should describe the complete stopping rule so the reader does not infer that a single crossing of the threshold terminates optimization.","section":"Fig. 5"},{"comment":"The sentence 'higher entropy indicates greater uncertainty, inconsistency, and unresolved migration errors' uses 'entropy' as an analogy rather than an information-theoretic quantity; the paper already acknowledges this, but the phrasing in the abstract and Section 1 could be tightened to avoid implying a formal thermodynamic or Shannon interpretation.","section":"Sec. 3.2"}],"recommendation":"major_revision","confidential_remarks":"The referee report focuses on a correctness-risk concern: the optimization objective (Code Entropy) and the evaluation metric (Agent-as-Judge) share a semantic construct via the feature_check dimension, and no external calibration is provided. This is fixable within the scope of the paper by adding ablations and a human/executable validation study, but it is load-bearing for the claimed 74.7% result. I also note that the paper would benefit from clarifying the relationship between the discriminator's feature_check prompt and the evaluation checklist, since the current wording overstates the degree of isolation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, A2H-RepoBench is the real contribution: three real Android repositories at 50K/120K/300K LOC with functional checklists and a sealed evaluation protocol, two orders of magnitude larger than prior repo-translation benchmarks. Second, the headline result (74.7% vs 46.4%) is not yet trustworthy. The reader's conditional verdict is about right.\n\nWhat the paper does well: the generator/discriminator split is sensible, and the w/o-discriminator ablation is compelling — collapsing to self-evaluation drops the average score to 28.4%, and the failure mode scales with repository size. The memory-tree transfer experiment (38 vs 66 iterations, roughly half the tokens) is a clean demonstration. The appendix is unusually honest: baseline prompts, evaluation details, a sealed reference protocol, artifact sizes, and base-LLM robustness all support the framework-level claims.\n\nThe soft spots, in order of seriousness.\n\nOne: the optimization objective and the evaluation metric overlap. Code Entropy's feature_check dimension (weight 0.3) is an LLM judge scoring per-feature completeness against the Android counterpart; Agent-as-Judge is an LLM judge scoring per-feature completeness against the Android counterpart. The discriminator is denied the checklist, but the feature_check prompt can reconstruct essentially the same construct from the source. Section 3.2's claim that no entropy dimension carries application-specific functional requirements is not consistent with feature_check's own definition. This does not make ECAT useless, but it means the reported functional gain may be partly an artifact of optimizing a judge-like objective.\n\nTwo: Code Entropy is never calibrated. There is no comparison against human-verified functionality, no test-suite score, no estimator-bias study. Eq. (6) guarantees monotone decrease of the estimated score, and that is definitional, not empirical. The acceptance rule is fine as an optimization device; the problem is that we do not know whether the estimated score tracks true migration quality. The dynamic/emulator dimensions help, but the static judge dimensions dominate the weights.\n\nThree: baselines are adapted by the authors, run without error bars, and no pinned code artifact is provided. The GitHub link is nice, but a commit hash and a runnable evaluator would make the 74.7% claim checkable.\n\nNone of this sinks the paper. The benchmark, the adversarial formulation, and the memory-tree results justify a serious referee. I would send it to review and ask for: (a) calibration of Code Entropy against human or executable ground truth; (b) removal or redefinition of feature_check so the optimization signal is not the same construct as the evaluation; (c) pinned code and baseline error bars. If those land, the paper is a solid systems contribution.","headline":"The benchmark and the adversarial loop are worth taking seriously; the 74.7% headline is not yet earned because the optimization objective overlaps the evaluation metric and Code Entropy is never calibrated.","tokens_in":20782,"tokens_out":2774,"would_cite":true,"duration_ms":28486,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that long-horizon repository migration can be automated by treating it as adversarial entropy minimization, and demonstrates a generator–discriminator framework that raises Android-to-HarmonyOS migration quality to 74.7%.","keywords":["repository migration","Code Entropy","adversarial optimization","generator-discriminator","Android-to-HarmonyOS","multi-agent LLM","self-evolving memory","benchmark"],"falsifier":"Instrument ECAT to save every candidate repository that Eq. (6) rejects, then score those same candidates with the held-out Agent-as-Judge checklist; if a meaningful share of rejected candidates scores at or above the accepted final repository, then the entropy estimate is not a faithful proxy for migration quality.","tokens_in":19698,"feed_emoji":"🤖","tokens_out":8647,"duration_ms":75131,"temperature":0.7,"pith_summary":"Automatically migrating a real software repository from one platform to another is a long-horizon task where errors pile up across thousands of interdependent files. This paper argues that the migration can be framed as adversarial entropy minimization: an independent 'discriminator' agent measures the disorder of the current translation using a unified Code Entropy score, and a 'generator' agent repairs the code under text instructions from the discriminator. Only updates that actually lower the entropy estimate are accepted, so the translated repository is driven toward a low-entropy, functionally complete state over many iterations. The authors build this loop on top of LLM agents, add a self-evolving memory tree that transfers experience across repositories, and introduce a benchmark of three real Android applications ranging from 50,000 to 300,000 lines of code. On that benchmark the framework scores 74.7% average quality versus 46.4% for the strongest baseline.","feed_headline":"Adversarial entropy loop migrates Android apps to HarmonyOS at 74.7%","feed_subtitle":"An independent judge scores repository disorder; only entropy-lowering edits survive, driving 50K–300K LOC translations.","key_machinery":"The central object is Code Entropy, $H(R_t) = \\frac{W^\\top E}{\\mathbf{1}^\\top W}$, a normalized scalar in $[0,1]$ that aggregates $K=14$ entropy dimensions $e_i$, each estimated either by a deterministic rule (compilation success, leftover placeholders, permission mapping) or by an LLM judge (feature completeness, navigation reachability, UI alignment with an emulator screenshot). Each dimension carries a hand-set weight $w_i\\in\\{0.1,0.3\\}$ that governs repair priority. The generator–discriminator loop carries the argument: the discriminator produces an estimated entropy $H_t$ plus a structured text gradient $g_t$ (file-specific issues and recommended skills), the generator applies the gradient to propose $\\hat{R}_{t+1}$, and Eq. (6) accepts the candidate only when $H(\\hat{R}_{t+1}) < H(R_t)$, which the authors state guarantees monotonic entropy decrease. A sliding-window stopping rule avoids premature termination on transient fluctuations, and a three-level self-evolving memory tree stores successful low-entropy trajectories as reusable, repository-agnostic patterns. Together these components turn repository migration into a discrete-space optimization loop with an explicit, computable objective.","core_discovery":"The paper's central claim is that repository-level migration is best treated not as a single translation step but as repeated adversarial optimization, with the objective being a scalar quantity the authors call Code Entropy, defined as a weighted average of fourteen defect dimensions covering compilation, structure, fidelity, permissions, navigation, and runtime behavior. The discriminator estimates this score and emits a structured text gradient: a list of concrete file-level issues with suggested repair skills. The generator acts on that gradient, and the next candidate repository is accepted only when its estimated entropy is strictly lower, a guarantee that, by construction, every accepted update monotonically reduces the estimated disorder. The loop terminates when the estimate stays below a threshold for a sliding window of iterations, and successful trajectories are distilled into a hierarchical memory tree shared by both agents. Evaluated on the new A2H-RepoBench, the approach reaches 74.7% average quality under node alignment and an agent-based functional judge, consistently beating three agentic baselines, and the ablation shows that removing the independent discriminator drops the average score to 28.4%.","pith_inferences":["Extending beyond the paper, the same adversarial entropy loop could apply to other long-horizon repository transformations, such as iOS-to-Android, Java-to-Kotlin, or framework upgrades, provided a credible Code Entropy estimator exists; the paper only demonstrates Android-to-HarmonyOS.","A subtle consequence the authors do not discuss is that the monotonic acceptance rule makes the optimization greedy: a miscalibrated early entropy estimate could lock the system into a local minimum, since rejected candidates are discarded permanently; a slower acceptance schedule that occasionally accepts entropy-increasing edits, similar to simulated annealing, is a natural stress test.","Because several entropy dimensions are judged by LLMs, the objective itself is a moving target: as the judge's notion of migration fidelity drifts with the underlying model, the loop converges to what that judge considers low-entropy, a form of reward overfitting that the paper does not measure.","A testable extension is to measure whether distilled memory patterns generalize to a repository outside A2H-RepoBench, since the current transfer experiment only evaluates transfer among the three benchmark repositories."],"forward_implications":["If the reported results hold, a real Android application of up to 300K lines of code can be migrated to a functionally runnable HarmonyOS repository without human code repair, which is the concrete scale at which prior agentic systems collapsed.","The independent discriminator is a key ingredient: the paper's ablation shows that letting one agent both generate and evaluate collapses the average score from 74.7% to 28.4%, with failure amplifying as the repository grows.","Emulator-based dynamic entropy evaluation protects runtime-heavy applications; removing it drops the average to 72.7% and produces visible UI defects that static checks miss.","The self-evolving memory tree transfers migration knowledge across repositories, cutting convergence iterations from 66 to 38 and roughly halving token consumption on the smallest repository.","Because the entropy dimensions are defined generically, with no application-specific checklist, the same loop should transfer to other platform pairs without re-engineering the objective."],"supporting_citations":[{"why":"Supplies the adversarial generator–discriminator paradigm that ECAT adapts to repository migration.","marker":"(Goodfellow et al. 2014)"},{"why":"Supplies the entropy minimization principle that motivates using Code Entropy as the optimization objective.","marker":"(Wang et al. 2021)"},{"why":"Supplies the concept of text gradients, used here to translate repository defects into discrete-space optimization signals.","marker":"(Yuksekgonul et al. 2024)"},{"why":"Provides the RepoTransBench benchmark and the ReAct-based RepoTransAgent baseline that ECAT compares against and extends to much larger repositories.","marker":"(Wang et al. 2026)"},{"why":"ReCodeAgent is the strongest baseline (46.4% average in this paper's comparison) that ECAT must outperform.","marker":"(Ibrahimzada et al. 2026)"},{"why":"OpenHands is a general-purpose agent baseline whose self-evaluation bias motivates the independent-discriminator design.","marker":"(Wang et al. 2025)"},{"why":"Supplies the Agent-as-Judge protocol used as one of the two evaluation metrics.","marker":"(Zhuge et al. 2025)"},{"why":"Provides the CodeGraph tool used to build semantic graphs for the Node Alignment metric.","marker":"(McHenry 2026)"},{"why":"Documents LLM self-evaluation bias, which is the failure mode the independent discriminator is designed to avoid.","marker":"(Zheng et al. 2023)"}],"fun_headline_variants":["Entropy-lowering adversarial loop migrates Android to HarmonyOS","Adversarial entropy loop cuts repo migration to 74.7% quality","Generator vs discriminator: only entropy-lowering edits survive","74.7% migration via adversarial entropy minimization","Entropy-judged adversarial loop migrates repos at 74.7%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the Code Entropy scores produced by Eq. (2), with its hand-set weights and LLM-based estimators, reliably track true migration quality: the acceptance rule Eq. (6) guarantees only that the estimated score decreases, so if the estimates are biased, the loop can settle on a low-estimate repository that is still functionally incomplete.","fun_headline_variants_meta":{"raw":{"variants":["Entropy-lowering adversarial loop migrates Android to HarmonyOS","Adversarial entropy loop cuts repo migration to 74.7% quality","Generator vs discriminator: only entropy-lowering edits survive","74.7% migration via adversarial entropy minimization","Entropy-judged adversarial loop migrates repos at 74.7%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000973,"raw_usage":{"total_tokens":4171,"prompt_tokens":1018,"completion_tokens":3153,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":634,"completion_tokens_details":{"reasoning_tokens":3065}},"tokens_in":634,"tokens_out":3153,"duration_ms":23849,"temperature":1.0,"reasoning_tokens":3065,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:21:14.161695+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Instrument ECAT to save every candidate repository that Eq. (6) rejects, then score those same candidates with the held-out Agent-as-Judge checklist; if a meaningful share of rejected candidates scores at or above the accepted final repository, then the entropy estimate is not a faithful proxy for migration quality.","supporting_citations":[{"cited_title":"2026 , eprint =","cited_arxiv_id":null,"evidence_quote":"Provides the RepoTransBench benchmark and the ReAct-based RepoTransAgent baseline that ECAT compares against and extends to much larger repositories."},{"cited_title":"2026 , eprint =","cited_arxiv_id":null,"evidence_quote":"ReCodeAgent is the strongest baseline (46.4% average in this paper's comparison) that ECAT must outperform."},{"cited_title":"2026 , howpublished =","cited_arxiv_id":null,"evidence_quote":"Provides the CodeGraph tool used to build semantic graphs for the Node Alignment metric."}],"review_version":1}