{"id":"b7e4cd4c-ae11-4780-8522-896e3f7b1812","arxiv_id":"2501.06191","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A conceptual schema and management framework called DLOM2 are proposed to select or create optimized deep learning models for IoT devices, but the design remains unvalidated.","lead":"This paper proposes a model and a software framework to help companies choose among compressed and optimized deep learning models for Internet of Things devices. It is a design sketch, not a working system: the authors describe the plan and an imagined example, but no implementation or test results.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The DL Modeling Network requires a huge corpus of validated optimization models that is never sourced; the DSS feedback loop also stores unvalidated predictions, so the generative core of DLOM2 is unimplementable as specified.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the DL Modeling Network's training corpus is absent, unsourced, and unplanned. My stress-test confirms this is the most critical point because the DLN is the only component that creates new models; the rest of DLOM2 is a weighted-sum query engine (Equation 1) over a repository. Without a validated corpus, the framework cannot be instantiated, and the central claim that it 'would help organizations choose the optimal DL optimization model' is unsupported. I also note a reinforcing internal inconsistency: the DSS saves DLN-generated configurations back into the repository without requiring measured performance, so the feedback loop would pollute the training data. This is not an external-consensus disagreement but a missing-evidence problem, and the authors themselves concede the design is initial. The concrete test I propose would settle the concern by checking whether a sufficient corpus exists and whether the DLN can outperform random search; until then, the paper is a reasonable research proposal, not a demonstrated framework. The reader's CONDITIONAL verdict is appropriate, and my analysis does not change it.","tokens_in":9096,"tokens_out":5773,"duration_ms":53887,"concrete_test":"Assemble a seed repository from public edge-inference benchmarks (e.g., MLPerf Tiny, papers-with-code) for a fixed device such as Raspberry Pi 3, recording configuration, optimization techniques, and measured accuracy/latency; then train the proposed unsupervised DLN on this corpus and compare its generated configurations against random search on a held-out target. If the corpus contains fewer than ~10^4 schema-compliant records, or generated configurations do not beat random search, the DLN training assumption fails and the framework's central claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that DLOM2 helps organizations choose optimal DL optimization models depends on the DL Modeling Network (Section 5.3) generating new configurations when no stored model matches. This component is trained on 'a huge number of successful optimization models' previously stored in the repository; however, the paper neither provides such a corpus, cites an existing one, nor outlines how it would be assembled. Without this training data the DLN cannot learn, and the framework reduces to Equation 1's weighted sum over a repository that is empty in any current instantiation. The problem is compounded by the DSS workflow (Section 5.3): when no match is found, the DSS sends criteria to the DLN and then 'saves a copy of the new model configurations in the repository' without requiring measured performance. These unvalidated predictions would then contaminate the same repository that is supposed to supply successful training examples for future DLN iterations, so even a seed corpus would degrade. The illustrative example in Section 6 does not exercise the DLN, so it provides no evidence for this component. The paper's own Section 7 concedes the framework is an 'initial abstract design iteration,' but the missing corpus is not a matter of implementation detail: it is the sole source of training signal for the only component that creates new models, and its absence leaves the central claim unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses the problem of selecting and reusing Deep Learning (DL) optimization models for Internet of Things (IoT) applications. It proposes a six-class schema for describing DL optimization models (Model, Cloud Configuration, End Device Specifications, Main DLN, Optimization, and Performance classes) and an accompanying framework, DLOM2, composed of a cloud repository, a GUI, a decision support system (DSS), and a DL Modeling Network (DLN). The DSS ranks candidate models using a weighted additive score over six objectives (performance, reliability, security, cost, latency, complexity) as given in Equation (1). When no stored model matches the user criteria, the DLN is supposed to generate a new model configuration. The paper provides an illustrative example of the first three steps of the framework but includes no implementation, no data, and no evaluation.","tokens_in":9363,"tokens_out":6300,"duration_ms":55399,"significance":"If the proposed design were implemented and validated, the schema in Table 3 and the DSS workflow could provide a useful starting point for practitioners who need to choose among DL optimization techniques for edge and end devices. The paper usefully organizes known optimization methods around six explicit objectives and makes the selection procedure transparent and testable. Equation (1) is a standard multi-criteria aggregation, so there is no circularity in the ranking step itself. However, the paper's central claim, that the framework 'maximizes performance without sacrificing quality,' is not supported by any empirical evidence, and the generative DLN component depends on a training corpus that is never sourced. The main value of the paper at this stage is conceptual: the taxonomy and the decision workflow. The missing validation and the unspecified DLN training data are the key obstacles to accepting the stronger claims.","major_comments":[{"comment":"The DL Modeling Network is the only component that creates genuinely new optimization models, but the paper gives no source for the 'huge number of successful optimization models' it requires for training. Section 5.3 states the requirement without providing a corpus, citing an existing repository, or outlining an acquisition procedure. Because the DSS workflow queries the repository first and only invokes the DLN when no match is found, the entire generative claim in the abstract ('maximizes performance without sacrificing quality') depends on this missing training resource. The illustrative example in Section 6 never exercises the DLN, so it offers no evidence that such a network can be trained from the proposed schema. The authors should either supply a concrete data-source and feasibility argument, or explicitly restrict the paper's claims to retrieval and ranking of existing models.","section":"§5.3 (DL Modeling Network) and §6 (Illustrative Example)"},{"comment":"The DSS saves 'a copy of the new model configurations in the repository' immediately after the DLN proposes a new model, without any measured performance or validation step. This would insert unvalidated predictions into the same repository that is supposed to contain 'successful optimization models' used as training input for the DLN. The feedback loop therefore degrades the training signal as soon as the system begins generating models. The paper needs an explicit provenance and validation mechanism, such as storing only models with measured results or tagging predicted versus validated models, before this component can support the framework's claims.","section":"§5.3 (DSS workflow)"},{"comment":"The only evaluation is an illustrative example that walks through the GUI steps and applies Equation (1) to three retrieved models. It does not compare the framework against existing model management systems, measure any quality metric, test the DLN, or use a real repository. Section 7 itself concedes that the framework is 'an initial abstract design iteration.' The abstract's strong claim that the framework 'would help organizations choose the optimal DL optimization model that maximizes performance without sacrificing quality' is therefore not supported by the evidence presented. The claims need to be brought in line with the design-science stage, or a prototype and evaluation need to be added.","section":"§6 and §7"},{"comment":"Equation (1) is a standard weighted additive score: it ranks models according to the user's preference weights over six objectives. Consequently, the 'optimal' model is optimal only relative to the weights elicited from a particular user. The paper's wording that the framework selects an optimal model that 'maximizes performance without sacrificing quality' extends beyond what Equation (1) can deliver, since a weighted sum explicitly encodes trade-offs among performance, cost, latency, and so on. The claim should be scoped to 'the model ranked best under the user's stated preferences,' and the relationship between the weights and the performance/quality trade-off should be stated explicitly.","section":"Equation (1)"}],"minor_comments":[{"comment":"The DLN is described as an 'unsupervised DL approach' but is also said to be 'trained with different optimization models previously stored in the repository'; the learning paradigm and the output of the network (a new configuration versus a ranking) should be clarified.","section":"§5.3"},{"comment":"The subscripts in Equation (1) are garbled in the submitted text (e.g., w89:, w;<=, w><?), and it is not stated whether the weights are normalized or how the reverse-scaled objectives (cost, latency, complexity) are combined with the forward-scaled ones; the notation should be fixed and the scale convention stated.","section":"Equation (1)"},{"comment":"Query 1 in Section 6 is not valid SPARQL as printed: the WHERE clause is not closed and the FILTER expression is incomplete; a correct query should be provided.","section":"§6"},{"comment":"Table 2 lists 'Design Evaluation' with the future-tense phrase 'will be evaluated using illustrative example,' which conflicts with the present-tense claims elsewhere in the paper and with the design-science methodology cited in Section 3; the research stage should be stated consistently.","section":"Table 2"},{"comment":"Section 5.2 describes four repositories but does not specify which of the six schema classes from Table 3 are stored in each; a mapping would help readers understand the storage design.","section":"§5.2"},{"comment":"There are numerous typographical and formatting issues, including 'a Deep Learning Networks (DLNs)' in Section 1, 'the research add s' in Section 1, 'thar' in Section 7, and inconsistent titles for the DLOM2 acronym; a careful copyedit is needed.","section":"Throughout"},{"comment":"The DSS paragraph is placed under the heading 'The DL Modeling Network' in Section 5.3; the subsection structure should separate the DSS description from the DLN description.","section":"§5.3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is better characterized as a design-science research-in-progress paper than as a completed systems contribution. The main risk for the journal is that the strong claims in the abstract will be taken as validated results. If the journal accepts design science submissions, the paper could be publishable after major revision, but the authors should be asked to either provide a prototype with a concrete repository of validated models or substantially reduce the scope of the claims. I would also ask the authors to acknowledge and discuss the DLN training corpus problem explicitly in the limitations section, as Section 7 currently treats the framework as merely incomplete rather than identifying this specific bottleneck."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a clearly written design-science paper, and the authors deserve credit for what it is: an initial abstract design, not a demonstrated system. The one genuinely new piece is the six-class schema (Table 3) that organizes DL optimization for IoT specifically — model, performance, end-device specs, main DLN, optimization, cloud config — and ties it to a management framework with a repository, a DSS, and a DL Modeling Network. The gap claim holds up: ModelDB, ModelHub, and MLflow are general model management systems, not aimed at DL optimization for constrained IoT devices. The authors also state their limitations plainly in Section 7, which is more than many design papers do.\n\nThat said, the soft spots are significant and the central claim overreaches. There is no implementation and no evaluation beyond a mock illustrative example. The abstract says the framework \"maximizes performance without sacrificing quality,\" but no evidence supports that. The weighted-sum scoring in Equation 1 is a standard multi-criteria aggregation, so the novelty has to carry the schema and the DLN — and the DLN is where the paper breaks down. Section 5.3 says training it \"needs a huge number of successful optimization models,\" but no such corpus is provided, cited, or shown to be assemblable. Without that, the DLN cannot learn. The problem is worse: when the DSS generates a new model, it \"saves a copy of the new model configurations in the repository\" without requiring measured performance. Those unvalidated predictions then sit in the same repository that is supposed to supply successful training examples for future DLN iterations. The seed corpus would be contaminated by its own output.\n\nThe illustrative example in Section 6 only exercises the DSS query path and the weighted sum, so it gives no evidence about the DLN. The authors are honest that this is one iteration in a design-science program, and that framing helps, but the missing corpus is not an implementation detail; it is the sole training signal for the only component that creates new models. Until that is addressed, the framework is essentially a weighted-sum query engine over an empty database.\n\nWho is this for? Researchers and practitioners in information-systems design science who want a structured vocabulary for DL model selection on IoT. It deserves serious peer review because the schema and the research agenda are usable, but the referee report should demand a proof-of-concept instantiation and a concrete plan for the DLN's training data. I would not cite it as a validated framework; I might cite it as an early design proposal if I were working on the same problem.","headline":"An honest, well-scoped design-science proposal for a DL-optimization model-management framework, but it is an unvalidated artifact whose generative core needs a training corpus that is neither provided nor accounted for.","tokens_in":9882,"tokens_out":1518,"would_cite":false,"duration_ms":16285,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A six-objective weighted score can rank deep-learning optimization models for IoT devices, and a learned network can propose new ones when none match.","keywords":["deep learning","Internet of Things","DL optimization model management","edge computing","model selection","decision support system","DL tunneling"],"falsifier":"Try to assemble a corpus of successful DL optimization models with their device specs and outcomes; if no such corpus can be collected, or if a trained network produces no configuration that beats the best stored model on the same device, the framework's generative claim fails and only the scoring rule remains testable.","tokens_in":8886,"feed_emoji":"🤖","tokens_out":4008,"duration_ms":34316,"temperature":0.7,"pith_summary":"This paper argues that organizations embedding deep-learning networks in IoT devices lack criteria for choosing among competing optimization techniques such as pruning, quantization, and knowledge distillation. To fill that gap, it proposes a six-class modeling schema that captures the model itself, its optimization methods, cloud and end-device configurations, and performance measures. The schema feeds a management framework, DLOM2, that scores stored models by a weighted sum of performance, reliability, security, cost, latency, and complexity, and lets users pick the highest-scoring model. When no stored model matches, a DL Modeling Network is meant to learn from the repository and propose a new configuration. The paper positions this as an initial design-science artifact, not a deployed system.","feed_headline":"A six-objective score picks the best deep-learning model for IoT","feed_subtitle":"The DLOM2 framework stores past optimization models, ranks them by weighted objectives, and proposes new ones when none fit.","key_machinery":"The load-bearing object is the six-class DL optimization modeling schema (Table 3), paired with the weighted-score formula in equation (1) and the DL Modeling Network. The schema defines what metadata about a model is recorded—network name, layers, hyperparameters, optimization methods, cloud configuration, end-device specs, and user ratings—which is what makes storage, retrieval, and comparison possible. Equation (1) collapses the six objectives into a single number for ranking. The DL Modeling Network is the only component that genuinely creates new models, and it is assumed to do so by learning patterns from a large repository of successful optimization models.","core_discovery":"The paper's central claim is that the DL optimization problem for IoT can be structured as a queryable, reusable decision process rather than an ad-hoc trial-and-error exercise. The proposed six-class schema—Model, Cloud Configuration, End-device Specifications, Main DLN, Optimization, and Performance—defines what metadata a stored optimization model must carry, enabling comparison and retrieval. Selection is driven by equation (1), a weighted sum of six objectives with weights derived from pairwise preference elicitation. The DLOM2 framework wraps this schema with a GUI, a cloud repository, a decision support system, and a DL Modeling Network; the last component is trained on previously stored models to infer new combinations of optimization techniques when the repository contains no match. The paper evaluates the framework only through a step-by-step illustrative example of a medical company, making clear that this is an initial design iteration.","pith_inferences":["If the DL Modeling Network proves infeasible to train for lack of data, the framework still operates as a weighted query engine, so its practical value depends on assembling a corpus before its generative promise can be tested.","The six-class schema could generalize beyond IoT to other model-selection settings, such as federated learning or model cards for transparency, because the same metadata pattern applies wherever models are chosen by context.","A concrete testable extension is to seed the repository with published compression results and check whether the weighted-score ranking matches expert choices in a user study.","Another testable extension is to measure how sensitive the top-ranked model is to the six weights, since pairwise preference weights can be noisy; the paper does not address that robustness question."],"forward_implications":["If the schema is adopted, organizations can share and compare DL optimization models in a common format, lowering the cost of reusing prior work across projects.","The weighted-score rule gives IT managers a transparent, reproducible way to trade off accuracy, latency, cost, security, reliability, and complexity instead of relying on undocumented judgment.","When no stored model matches a request, the DL Modeling Network could synthesize a new configuration, making the repository grow with each query and user decision.","The paper's illustrative example shows a plausible workflow: capture requirements, query the repository, elicit preference weights, rank models, and optionally ask for a new model.","Because the design-science framing treats the artifact as an initial iteration, later work can revise the schema and scoring rule without discarding the overall management approach."],"supporting_citations":[{"why":"Supplies the DM3 ontology that the proposed six-class schema is built on.","marker":"[13]"},{"why":"Provides the preference-elicitation method used to derive the weights in equation (1).","marker":"[36]"},{"why":"Provides the complexity-control design-science perspective that justifies the artifact as an initial iteration.","marker":"[31]"},{"why":"Supplies the design-science research guidelines that structure the research activities in Table 2.","marker":"[34]"},{"why":"ModelHub, a deep-learning management system, defines the gap this framework extends toward IoT.","marker":"[8]"},{"why":"ModelDB, an early model-management system, sets the baseline the authors position their IoT-targeted framework against.","marker":"[16]"}],"fun_headline_variants":["AI model picker for IoT uses six weighted goals","Scoreboard picks deep-learning model for IoT tasks","DLOM2 framework ranks DL models for IoT by six objectives","Reusable model selection for IoT deep learning","Six-objective scoring eases DL model choice for IoT"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework's ability to create new models rests on training a DL Modeling Network on 'a huge number of successful optimization models,' but no such corpus is provided, sourced, or shown to exist; without it, DLOM2 is a query-and-rank engine over a database that is empty at present.","fun_headline_variants_meta":{"raw":{"variants":["AI model picker for IoT uses six weighted goals","Scoreboard picks deep-learning model for IoT tasks","DLOM2 framework ranks DL models for IoT by six objectives","Reusable model selection for IoT deep learning","Six-objective scoring eases DL model choice for IoT"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000165,"raw_usage":{"total_tokens":1219,"prompt_tokens":881,"completion_tokens":338,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":497,"completion_tokens_details":{"reasoning_tokens":262}},"tokens_in":497,"tokens_out":338,"duration_ms":3921,"temperature":1.0,"reasoning_tokens":262,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:32:02.640463+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Try to assemble a corpus of successful DL optimization models with their device specs and outcomes; if no such corpus can be collected, or if a trained network produces no configuration that beats the best stored model on the same device, the framework's generative claim fails and only the scoring rule remains testable.","supporting_citations":[{"cited_title":"Procedia computer science, 2017","cited_arxiv_id":null,"evidence_quote":"Supplies the DM3 ontology that the proposed six-class schema is built on."},{"cited_title":"Yoon, and M","cited_arxiv_id":null,"evidence_quote":"Provides the preference-elicitation method used to derive the weights in equation (1)."},{"cited_title":"26ms Inference Time for ResNet-50: Towards Real-Time Execution of all DNNs on Smartphone","cited_arxiv_id":"1905.00571","evidence_quote":"Provides the complexity-control design-science perspective that justifies the artifact as an initial iteration."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the design-science research guidelines that structure the research activities in Table 2."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ModelHub, a deep-learning management system, defines the gap this framework extends toward IoT."},{"cited_title":"Thomas, and K","cited_arxiv_id":null,"evidence_quote":"ModelDB, an early model-management system, sets the baseline the authors position their IoT-targeted framework against."}],"review_version":1}