{"id":"0bdda4b0-3260-47f1-8f10-feeede67f472","arxiv_id":"2411.15875","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Ease of use, usefulness, and traceability sit at the base of a five-level model whose top outcome is agile response, reached through software reliance and collaboration efficiency.","lead":"This paper analyzes 34 interviews with Chinese government employees who used e-government systems and arranges ten perceived factors into a five-level hierarchy that ends in agile response. It argues that technology use and group collaboration are two complementary pathways to agility, not a single route.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The five-level hierarchy and the dual-pathway conclusion inherit the full empirical weight of a 10x10 adjacency matrix that two researchers scored with no reported coder-level data, pairwise quote counts, or sensitivity analysis; the paper offers no way to check from the published record whether…","rationale":"I read the paper as a qualitative theory-building study whose central contribution is the five-level ISM model connecting technology usage and group collaboration to agile response. The strongest claim is exactly the hierarchy reported in Table 8 and Fig. 2, and the propositions translate that hierarchy into dual-pathway statements. What would have to be true for that claim to hold is that the adjacency matrix in Table 5 faithfully represents the interview evidence. That condition is the least secure part of the paper: the matrix is produced by two unnamed researchers, the threshold rule is stated but not operationalized, graded scores are described but a binary matrix is reported, and no coder-level agreement or per-cell evidence is given. I verified that the published reachable matrix, level partition, and level assignments follow correctly from Table 5, so the problem is not arithmetic; it is that the empirical grounding of the matrix is unavailable in the manuscript. This is precisely the weakness the reader identified, and I find no additional concern that would move me past it. The honest disposition is to keep the CONDITIONAL verdict: the model is internally coherent and the qualitative method is recognizable, but the central claim should not be treated as robust until the matrix construction is auditable and its stability is demonstrated. The proposed concrete test—independent re-scoring by coders blind to the published matrix—would settle whether the reported level structure is reproducible or idiosyncratic.","tokens_in":20418,"tokens_out":9003,"duration_ms":79036,"concrete_test":"Have at least two independent coders, blind to the published Table 5, re-score all 90 ordered pairs of the 10 constructs from the 34 interview transcripts using the stated 0/0.25/0.5/0.75/1 scale and the 'mentioned by fewer than three interviewees' threshold, then recompute the reachable matrix and level partition from each coder's binary matrix and from their consensus matrix; report per-coder matrices, inter-coder agreement, and whether level 2 in every resulting partition is still {software reliance, collaboration efficiency} with agile response at level 1. If any independent re-scoring produces a different level structure, the dual-pathway claim is an artifact of idiosyncratic matrix construction.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that software reliance and collaboration efficiency are direct antecedents of agile response, and that the five-level ISM hierarchy in Fig. 2 depicts the dual pathways—is a deterministic consequence of the adjacency matrix in Table 5. The level partition in Tables 6–8 follows mechanically from that matrix via Boolean reachability; the propositions in Section 3.4.4 restate the corresponding direct edges (e.g., H→F and B→F for Proposition 5). Therefore the empirical content of the entire model is contained in the 90 off-diagonal 0/1 judgments made by two researchers (Section 3.4.1). The paper does not report the two coders' individual matrices, the pairwise interview-mention counts behind each nonzero cell, or the operationalization of the 'fewer than three interviewees' removal rule. It also gives no agreement statistic, and the scoring criteria (0/0.25/0.5/0.75/1) are described alongside a binary final matrix without explaining how graded scores were dichotomized. Because the hierarchy is exactly the transitive closure of these judgments, any error, bias, or even an arbitrary threshold choice in Table 5 changes the level assignments and the claimed direct antecedents. This is not a mathematical error in the ISM computation—we checked that Tables 5–8 are internally consistent—but it means the central finding cannot be independently verified or falsified from the paper as published.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports a qualitative study of how Chinese government employees achieve agile response in e-governance. The authors conducted 34 semi-structured interviews, used three rounds of coding to identify constructs, and then applied Interpretative Structural Modeling (ISM) to build a five-level hierarchy. The resulting model posits two pathways: a technology-usage pathway (ease of use/usefulness, system quality, technology mindfulness, software reliance, agile response) and a group-collaboration pathway (traceability, communication transparency, trust, collaboration efficiency, agile response). The paper provides propositions for each layer and discusses practical implications for government decision-makers, system developers, and employees.","tokens_in":20730,"tokens_out":4545,"duration_ms":37841,"significance":"If the central structural claims held, the paper would make a useful contribution by shifting attention from G2C to G2E systems and by integrating technology-acceptance and collaboration literatures in a single framework. The ISM arithmetic from the reported adjacency matrix to the reachability matrix and level partitions is internally consistent and reproducible from the published tables, and the qualitative material is extensive. The practical implications follow from a plausible and clearly presented model. However, the empirical content of the model rests almost entirely on a 10x10 binary adjacency matrix populated by two researchers' judgments, and the paper does not provide the information needed to audit or independently validate that matrix; the propositions then restate the edges of the same matrix. These are load-bearing limitations that the current manuscript does not fully address.","major_comments":[{"comment":"The stress-test concern is confirmed on reading: the five-level hierarchy and the claim that software reliance and collaboration efficiency are direct antecedents of agile response are algebraic consequences of the binary adjacency matrix in Table 5, but the manuscript does not report the two researchers' individual graded matrices, the rule by which the 0/0.25/0.5/0.75/1 scores were dichotomized to 0/1, or the operationalization of the 'mentioned by fewer than three interviewees' removal rule. Because any change in these judgments can change the level partition, the central finding cannot be independently verified from the published record; the paper should provide the coder-level matrices, an agreement statistic, and a sensitivity analysis of the threshold.","section":"Section 3.4.1, Table 5"},{"comment":"There is a material inconsistency in the construct count: Table 1 reports 12 constructs and a 4-level model, but Stage 1 coding lists only 10 first-order categories (ease of use, usefulness, traceability, system quality, technology mindfulness, collaboration efficiency, communication transparency, trust, agile response, reliance) and the ISM analysis yields a 5-level model. The manuscript should correct these counts and explain why the analysis uses 10 constructs rather than the 12 listed in Table 1.","section":"Section 3.3, Table 1 and Table 3"},{"comment":"The propositions are statements of the edges of the same adjacency matrix that generated the ISM hierarchy, so they do not constitute an independent test of the hypothesized causal chains; for example, Proposition 5 asserts B-to-F and H-to-F, which are exactly the edges in Table 5. The paper should explicitly frame the ISM output as an interpretive synthesis of the interview-based judgments and position the propositions as hypotheses for future quantitative testing, rather than as empirical results.","section":"Section 3.4.4, Propositions 1-5"}],"minor_comments":[{"comment":"The output column says 'Generated a 4-level theoretical model,' but the final model in Table 8 and Fig. 2 has five levels; please reconcile the two descriptions.","section":"Table 1"},{"comment":"The sentence describing the 'principle of maximum membership degree' is not operationalized; please specify exactly how the two coders' graded matrices were combined to yield the binary Table 5.","section":"Section 3.4.1"},{"comment":"The first-order categories are introduced as '12 first-order categories in total' but only 10 are named; please add the missing two categories or correct the number.","section":"Section 3.3"},{"comment":"The manuscript uses 'two interviewers' and 'two scholars' interchangeably; please standardize the terminology and describe the coders' qualifications and coding procedure.","section":"Sections 3.3 and 3.4.1"},{"comment":"The sentence 'Findings of this study provides suggestions and implications' should be 'Findings of this study provide suggestions and implications.'","section":"Abstract"},{"comment":"Given that the adjacency matrix is the empirical backbone of the model, the authors should consider publishing the coder-level matrices or at least the pairwise interview-mention counts in an appendix, in addition to the 'reasonable request' statement.","section":"Data Availability"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for Group Decision and Negotiation, and the authors have assembled a rich qualitative dataset that is potentially useful for theory building. My main concern is the lack of transparency in the ISM input matrix and the internal inconsistencies in the reported construct counts. The circularity concern is real but is inherent to the ISM method; a careful reframing of the ISM output as an interpretive framework, together with a fully auditable matrix construction process, would make the paper publishable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Best quick take: this is a competent qualitative study that integrates TAM/IS-success and collaboration constructs into a five-level ISM hierarchy for G2E agile response. The novelty is in the configuration, not the components. The authors do a decent job describing 34 interviews and three coding rounds, and the ISM arithmetic is internally consistent — we checked Tables 5–8. The direct antecedents of agile response (software reliance, collaboration efficiency) follow from their matrix.\n\nWhat the paper does well: the qualitative grounding is visible. Table 3 gives illustrative quotes, Table 4 maps each participant to constructs, and the coding process is described step by step. The authors are appropriately cautious in the limitations, calling for future quantitative testing. The citation pattern is ordinary, no self-citation inflation.\n\nSoft spots are real but not fatal if read as an interpretive model rather than an empirical test. The entire five-level hierarchy is the transitive closure of a 10x10 adjacency matrix scored by two researchers, with no inter-coder reliability, no sensitivity analysis, and no explanation of how the graded 0/0.25/0.5/0.75/1 scores were dichotomized. The 'mentioned by fewer than three interviewees' threshold is stated but not operationalized. So the central claim cannot be independently checked from the published record. There are internal inconsistencies too: Table 1 reports 12 constructs and a 4-level model, while the analysis uses 10 constructs and 5 levels. Minor, but sloppy. The propositions at the end restate edges of the same matrix, so they shouldn't be read as tested hypotheses; the authors acknowledge this indirectly in the limitations.\n\nWho is this for? Researchers in e-government/G2E who want an organizing framework of levers for agile response, and public managers who want a checklist. It deserves a serious referee because the question matters and the method is transparent enough to scrutinize. I would not cite the hierarchy as evidence of causal ordering, but it's a reasonable hypothesis-generation device.\n\nRecommendation: engage, but push for a revised version that reports the two coders' individual matrices, the threshold rule, and the dichotomization rule before treating the dual-pathway claim as robust.","headline":"A competent qualitative ISM study whose five-level hierarchy for agile response in e-governance is only as strong as the two researchers' hand-built adjacency matrix.","tokens_in":21221,"tokens_out":3470,"would_cite":false,"duration_ms":28488,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Agile response in e-governance runs on two parallel paths, technology reliance and collaboration efficiency, organized into a five-level causal model.","keywords":["agile response in e-governance","technology usage","group collaboration","government employee","public sector","interpretative structural modeling","government-to-employee systems"],"falsifier":"Re-rate the pairwise influence matrix with a fresh panel of raters, or collect survey data measuring the ten constructs and test the structural paths. If software reliance and collaboration efficiency no longer sit directly below agile response, or if Proposition 5 fails in the quantitative test, the claimed hierarchy is falsified.","tokens_in":20253,"feed_emoji":"⚡","tokens_out":5678,"duration_ms":45874,"temperature":0.7,"pith_summary":"The paper argues that agile response in e-governance is not produced by technology alone or by collaboration alone, but by a dual pathway in which both run simultaneously. Drawing on 34 semi-structured interviews with Chinese government employees and three rounds of qualitative coding, it uses Interpretative Structural Modeling to arrange ten constructs into five hierarchical levels. The top of the hierarchy is agile response; directly beneath it sit software reliance and collaboration efficiency, fed by trust and technology mindfulness, which rest on system quality and communication transparency, with ease of use, usefulness, and traceability at the base. The practical point is that governments seeking agility should treat system usability and traceable work records as foundations, and transparency and trust as the collaborative bridge to efficient teamwork.","feed_headline":"Agile e-government needs two paths: tech reliance plus teamwork","feed_subtitle":"34 Chinese civil servant interviews put software reliance and collaboration efficiency directly under agile response.","key_machinery":"Interpretative Structural Modeling (ISM), a systems-engineering matrix method that turns pairwise influence judgments among constructs into a reachability matrix and then partitions it into hierarchical levels. Here it takes the 10x10 adjacency matrix of ten coded constructs, computes the reachability matrix by Boolean matrix powers, and extracts levels by the rule that a construct sits at a level when its reachability set equals its intersection set. This machine determines every level assignment and every proposition in the model.","core_discovery":"The central claim is that employees' agile response in e-governance is achieved through the synergy of two pathways, technology usage and group collaboration, and that these pathways converge in five layers. In the technology pathway, perceived ease of use and usefulness build system quality, which fosters technology mindfulness and then software reliance. In the collaboration pathway, system traceability enables communication transparency, which builds trust and then collaboration efficiency. Software reliance and collaboration efficiency are the direct antecedents of agile response. The paper states this result as five propositions and grounds it in the level partition of the reachability matrix of the ten constructs.","pith_inferences":["A natural next step would be to convert the five-level structure into a structural equation model and test it on survey data, which would show whether the adjacency-based hierarchy survives independent measurement.","The same dual-path logic may extend beyond government to any organization where digital tools and teamwork jointly determine responsiveness, such as remote healthcare or distributed software teams.","Because the model was built from Chinese civil servants during and after the pandemic, its transfer to other administrative cultures is an open question; the traceability path in particular may carry different weight where record-keeping norms differ.","The threshold rule, which drops links mentioned by fewer than three interviewees, means the model is deliberately conservative about rare influences; rarer factors could still matter in specialized sectors even though they do not appear in the hierarchy."],"forward_implications":["E-government investments should be aimed at the two direct antecedents, software reliance and collaboration efficiency, because these are the only constructs immediately below agile response.","System designers need to treat ease of use, usefulness, and traceability as foundational; without them the upper layers of the hierarchy lack support.","Government managers should treat communication transparency as a collaboration lever, since it feeds trust, which feeds collaboration efficiency, which feeds agility.","The five propositions P1 through P5 give researchers a ready-made set of hypotheses for quantitative testing.","Technology mindfulness acts as a bridge in the technology pathway: high-quality systems encourage employees to explore features, deepening their reliance on the system."],"supporting_citations":[{"why":"Origin of Interpretative Structural Modeling, the matrix method that produces the hierarchy.","marker":"Warfield 1974"},{"why":"Provides the inductive coding procedure used in the three-round qualitative analysis.","marker":"Miles and Huberman 1994"},{"why":"Supplies the rule to drop weak relationships mentioned by fewer than three interviewees before building the adjacency matrix.","marker":"Grunert and Grunert 1995"},{"why":"Source of the ease-of-use and usefulness constructs at the base of the technology pathway.","marker":"Davis 1989"},{"why":"Basis for treating perceived system quality as a downstream effect of usefulness and ease of use.","marker":"DeLone and McLean 2003"},{"why":"Defines agile governance as the flexible, adaptable, rapid culture that the model targets.","marker":"Mergel et al. 2018"},{"why":"Establishes the trust-to-collaboration-efficiency link used in the group collaboration pathway.","marker":"Cheng et al. 2016b"}],"fun_headline_variants":["Two paths to agile e-government: tech reliance and teamwork","E-governance agility from tech use plus team collaboration","Synergy of tech and teamwork drives e-government agility","How civil servants get agile: tech and group synergy","Agile e-governance: tech reliance meets collaboration"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire five-level hierarchy is computed from a 10x10 adjacency matrix that two researchers filled in, rating each pair of constructs on a 0-to-1 influence scale and dropping links mentioned by fewer than three interviewees; if those ratings or that threshold are idiosyncratic, every level assignment and proposition changes.","fun_headline_variants_meta":{"raw":{"variants":["Two paths to agile e-government: tech reliance and teamwork","E-governance agility from tech use plus team collaboration","Synergy of tech and teamwork drives e-government agility","How civil servants get agile: tech and group synergy","Agile e-governance: tech reliance meets collaboration"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000589,"raw_usage":{"total_tokens":2706,"prompt_tokens":829,"completion_tokens":1877,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":445,"completion_tokens_details":{"reasoning_tokens":1798}},"tokens_in":445,"tokens_out":1877,"duration_ms":11789,"temperature":1.0,"reasoning_tokens":1798,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:46:18.545658+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-rate the pairwise influence matrix with a fresh panel of raters, or collect survey data measuring the ten constructs and test the structural paths. If software reliance and collaboration efficiency no longer sit directly below agile response, or if Proposition 5 fails in the quantitative test, the claimed hierarchy is falsified.","supporting_citations":[],"review_version":1}