{"id":"ce7e12f3-f5a0-47b0-a145-97981aa4dae6","arxiv_id":"2501.09326","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"A POS-driven rule-based algorithm extracts subject-predicate-object triples from Kiswahili text and answers questions from the resulting semantic network with up to 78.6% exact match.","lead":"The authors present a rule-based algorithm that converts natural language text in Swahili, a low-resource SVO language, into semantic network triples using only part-of-speech tags and no training data. The method is evaluated on question answering, reporting up to 78.6% exact match on a Swahili QA dataset.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 78.6% QA claim depends on an unspecified query-generation step; without an automated question-to-SPARQL module, the result may reflect hand-written queries rather than the algorithm.","rationale":"The paper's strongest claim is that raw Kiswahili text can be mapped into a semantic network that answers questions with 78.6% exact match, with no training data. For this claim to hold, two automated pieces are required: the SN generator (described in Fig. 2) and a question-to-query translator. Only the first is specified. The example SPARQL query in Section 3.4 is clearly hand-written, with manually chosen nodes and predicates. Tables 2 and 3 in Section 3.5 report aggregate accuracy without any description of how each question became a query. Without that description, the 78.6% number could be achieved by a human who reads the generated triples and writes a bespoke query for each test question. That would mean the algorithm's contribution—the SN itself—is not what is being measured. The reader's identified weakest assumption (Cartesian-product noise) is real but secondary: if an automated query generator existed, the noise could be partially mitigated by query design; conversely, even a perfect SN would fail without a query translator. Therefore the missing question-to-query pipeline is the most load-bearing concern. The proposed test—rerun the KenSwQuAD sample with an end-to-end automated pipeline, or determine that no such pipeline existed—would settle whether the headline result is attributable to the algorithm. Since this concern supports the reader's rejection but for a more fundamental reason, the verdict remains unchanged.","tokens_in":12397,"tokens_out":3168,"duration_ms":29390,"concrete_test":"Obtain the authors' original 365-question KenSwQuAD sample and the Python/RDFLIB code (or an independent implementation) that takes as input only the raw question text and the context text, and produces the answer. If the protocol required hand-written SPARQL per question, the claim is not supported. If an automated question-to-query module exists, specify it and rerun the sample; if EM drops materially below 78.6% when queries are generated automatically (e.g., by template rules based on POS tags), the headline claim is overstated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the rule-based SN generation enables QA with 78.6% exact match on KenSwQuAD. To answer a question, two components are needed: (1) generate SN from context; (2) translate the natural-language question into a query against that SN. The paper specifies (1) in Fig. 2, but never specifies (2). Section 3.4 shows a manually written SPARQL query for one example, with UNION patterns and manual choices of concepts (:chelsea, :mwaka). Section 3.5 reports aggregate EM numbers (35/54 on TyDiQA, 287/365 on KenSwQuAD) with no description of how those queries were produced. If queries were hand-crafted per question, then the reported accuracy is not a property of the algorithm; it is a property of a human query writer exploiting the generated triples. Moreover, without a stated query mechanism, there is no way to reproduce the experiment or separate the algorithm's contribution from the query writer's. This is more load-bearing than the Cartesian-product noise: even a perfect triple store needs a query translator, and the paper provides none. The self-admitted noise (named entities like 'Chelsea Football Club' split into three subjects) matters only if the query translator is forced to use the noisy triples; a human query writer can work around it, masking the algorithm's limitations.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a rule-based algorithm that uses part-of-speech (POS) tags to convert raw Kiswahili text, which has subject-verb-object (SVO) structure, into RDF semantic-network triples with subject-predicate-object (SPO) structure. The algorithm anchors on the verb, collects all nouns on its left and right, forms Cartesian-product triples, and adds supplementary rules for constructions such as 'is-a'. The authors report QA experiments on Tusome, TyDiQA, and KenSwQuAD, with the headline result being 78.6% exact match on a purposive sample of 365 KenSwQuAD questions, with no training data for the semantic network generation. The paper also provides a worked example, per-question-type accuracy tables, and an explicit list of limitations including named-entity splitting and coreference failures.","tokens_in":12627,"tokens_out":5288,"duration_ms":52045,"significance":"If substantiated, the work would offer a training-data-free pipeline for structuring low-resource-language text and supporting simple entity-lookup question answering. The paper gives an explicit algorithm listing, a concrete example of generated triples, and an honest discussion of limitations such as named-entity decomposition and comma ambiguity. However, the central QA claim is currently not attributable to the proposed algorithm: the question-to-SPARQL translation step is unspecified, the evaluation uses small purposive samples with no baselines, and the main benchmark was created by the same authors. The contribution is best regarded as a preliminary demonstration, not a fully validated method.","major_comments":[{"comment":"The exact-match results in Tables 2 and 3 cannot be attributed to the proposed algorithm because the paper never specifies how natural-language questions are converted into SPARQL queries. Section 3.4 presents a single hand-written query with UNION patterns and manually chosen concept URIs (:chelsea, :mwaka), but the 54 TyDiQA and 365 KenSwQuAD questions are not accompanied by any automated query-generation procedure. If the queries were composed by a human using the visualized graph, the reported 64.8% and 78.6% figures measure the human query writer and the graph, not the SN-generation algorithm alone. A reproducible evaluation requires either a fully specified deterministic question-to-query module or a clear separation of human-authored queries from algorithm-only queries.","section":"§3.4–3.5"},{"comment":"The core extraction step forms the Cartesian product of every noun left of the verb with every noun right of the verb (lines 22–38). The paper acknowledges that this produces spurious triples, as when 'Chelsea Football Club' is decomposed into three separate subjects, each linked to 'club'. However, it does not quantify the proportion of spurious triples, nor does it describe how the QA pipeline selects among the many candidate triples when multiple answers are possible. Without triple-level precision or recall, or a filtering mechanism, the semantic network's quality is unknown and the QA accuracy cannot be interpreted.","section":"Algorithm lines 22–38"},{"comment":"The evaluation uses purposive samples of 33, 54, and 365 questions with no sampling protocol, no confidence intervals, and no baseline comparisons. Table 3 shows that the 'Define/How/Why' category has 0 correct out of 36, and Table 2 also shows 0 for 'Define'. This indicates the system is limited to simple entity lookup, yet the paper's abstract and conclusion frame the QA performance as general evidence of successful structuring. The authors should report how the sample was selected, include simple baselines (e.g., extracting the most frequent date or number from the context, or lexical-overlap retrieval), and qualify the QA claim to entity-lookup questions only.","section":"§3.5, Tables 2 and 3"},{"comment":"The main QA evaluation uses KenSwQuAD (Wanjawa et al., 2023), a dataset created by the same authors, while the algorithm builds on the authors' own prior model (Wanjawa & Muchemi, 2021). This creates a risk of implicit circularity: the rule set and the dataset annotations may share unstated assumptions about Kiswahili structure. The authors should test the algorithm on an independent Swahili QA corpus (e.g., the full TyDiQA test set with a defined query generator, or AfriQA), or at a minimum describe how the development of the rule set was kept independent from the KenSwQuAD annotation scheme.","section":"§3.5 and §5"},{"comment":"The paper claims that the method requires 'no training data,' but the algorithm depends on a POS tagger. The POS tagger used in the actual experiments is not identified; the text only mentions an online demo tool (aflat) in the worked example. The choice of tagger and its error rate are load-bearing for the reproducibility of the QA results, because POS errors propagate directly to the generated triples. The authors should name the tagger, its version or model, and report its accuracy on Kiswahili.","section":"§2 and §3.2.1"}],"minor_comments":[{"comment":"The phrase 'upto 78.6% exact match' contains a typo and should be 'up to 78.6%'. The claim should also mention the dataset, sample size, and that this is on a purposive sample of 365 questions.","section":"Abstract and §5"},{"comment":"The table entries for T7, T9, and T10 are not self-explanatory: the rows labeled 'N PROPNAME' produce triples with predicate :ni, but it is unclear how the 'is-a' rule maps the copula to :ni and how this differs from the verb-anchored triples in T1–T3.","section":"§3.2.1.1, Table 1"},{"comment":"The SPARQL query uses the concept :mwaka, but the triple-generation example in Table 1 does not show a triple containing :mwaka. The paper should explain how question tokens such as 'mwaka' are lemmatized and linked to nodes in the semantic network, including whether this linking is automatic or manual.","section":"§3.4"},{"comment":"The rule set in the 'OTHER_RULES' procedure is left open with 'etc.'; the authors should list all implemented rules and their coverage across the test corpora, since these rules produce a substantial fraction of the triples in the worked example.","section":"Algorithm lines 43–46"},{"comment":"Several references are incomplete or informal (e.g., 'Wu & Wu, n.d.', 'omniglot, 2021', and the online POS tagger reference). The authors should provide complete bibliographic details and, where possible, archival versions or DOIs.","section":"References"},{"comment":"The visualization is used as evidence that a connected semantic network is formed, but the figure is not self-contained. It would help to annotate a few of the displayed triples or list them in the caption so that a reader can connect the graph to Table 1.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The paper relies heavily on the authors' own prior model and dataset (Wanjawa & Muchemi, 2021; Wanjawa et al., 2023). This is not by itself disqualifying, but it makes independent validation essential. The core idea is simple and potentially useful, yet the QA evaluation as presented is not reproducible because the query-generation step is absent. I recommend major revision rather than rejection because the missing components can, in principle, be supplied by the authors: a deterministic query translator, triple-level error analysis, baselines, and an evaluation on an independent dataset."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one for the algorithm description, not for the 78.6% claim. The paper gives a concrete, step-by-step procedure for converting Kiswahili text into RDF triples using POS tags, with a worked Chelsea example and honest discussion of failure modes (named entity splitting, coreference, comma ambiguity). That is a real engineering contribution for low-resource languages, and the authors don't hide the limitations.\n\nThe soft spot is the QA evaluation, and it's load-bearing. To answer questions from the generated network, you need a mechanism that turns a natural-language question into a SPARQL query. The paper never specifies this. The one example query in Section 3.4 is hand-written, with manual choices of concepts (e.g., :chelsea, :mwaka) that look tailored to the known answer. Sections 3.5 report 35/54 and 287/365 exact matches with no description of how those queries were produced. If those were also hand-crafted per question, the accuracy is measuring the query writer, not the algorithm. This is a bigger problem than the Cartesian-product noise, because a human query writer can work around noisy triples.\n\nThe other weaknesses are in proportion. The samples are small and purposive. The POS tagger used in experiments isn't identified (the paper points to an online tool in the example but doesn't say which one produced the reported numbers). No code or data is shipped, so nothing is independently checkable. KenSwQuAD is the authors' own dataset, and the approach builds on their own prior model, which is fine as long as the evaluation pipeline is transparent—but here it isn't.\n\nWhat holds up: the core SVO-to-SPO mapping idea is straightforward and the algorithm as written does generate triples; the authors' admission that explanatory questions (define, why) score 0 and that NER is unsolved is refreshingly direct. This is not a deceptive paper; it's an under-validated one.\n\nWho should read it: people working on rule-based/low-resource NLP who want a starting point for POS-driven triple extraction. It deserves a serious referee because the problem is relevant and the method is described in enough detail to be tested, but the revision must specify the query generation, release code and data, and report error rates from the triple extraction itself. Right now the headline result is not reproducible.","headline":"A clearly specified rule-based triple extractor for Swahili, but the headline QA numbers rest on an unspecified question-to-query step.","tokens_in":13194,"tokens_out":1834,"would_cite":false,"duration_ms":46058,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A purely rule-based algorithm maps Kiswahili's subject-verb-object sentences into semantic-network triples from part-of-speech tags alone, and the resulting network answers sampled Swahili questions with up to 78.6% exact match, with no…","keywords":["algorithm","low-resource languages","Kiswahili","semantic networks","question answering","part-of-speech tagging","rule-based system","SVO word order"],"falsifier":"A concrete check: run the algorithm on a set of fifty Kiswahili contexts from the TyDiQA gold set, list every emitted triple, and have native speakers label each one as a valid fact or a spurious pairing. If the majority of triples are spurious yet exact-match QA still reaches the reported 78.6%, then the paper's account of how the network answers questions is incomplete; if the majority are valid, the Cartesian-product rule is doing the work the paper claims.","tokens_in":12146,"feed_emoji":"🔗","tokens_out":8132,"duration_ms":73003,"temperature":0.7,"pith_summary":"This paper proposes a rule-based algorithm that turns raw Kiswahili text into a semantic network without any training data, using only part-of-speech tags. The claim is that because Kiswahili sentences follow a subject-verb-object order, each verb can anchor a mapping into subject-predicate-object triples, the same shape as a semantic network triple. The authors test the generated networks on three Swahili question-answering sets, reporting exact match scores of 63.7%, 64.8%, and 78.6%, the last on the KenSwQuAD dataset. The point of the work is to show that low-resource languages can be structured for machine processing through language-structure rules rather than large annotated corpora.","feed_headline":"POS tags alone deliver 78.6% exact match on Swahili QA","feed_subtitle":"A no-training-data algorithm turns subject-verb-object Swahili text into queryable semantic networks.","key_machinery":"The load-bearing mechanism is an SVO-to-SPO rule set anchored on the verb. For each sentence, the algorithm tags parts of speech, finds each verb, lists every noun to its left as a subject candidate and every noun to its right as an object candidate, then emits all left-noun plus verb plus right-noun combinations as triples; when no verb is present, it applies fallback rules such as noun-'is-a'-noun. This Cartesian-product step is what generates the semantic network, and it is also the main source of spurious triples when named entities or long phrases are split into separate nouns.","core_discovery":"The paper shows that a semantic network for Kiswahili can be generated from raw text by finding each verb, collecting all nouns on the left and right of the verb, and forming the Cartesian product of those nouns into subject-predicate-object triples, with additional 'is-a' rules for noun-noun phrases. The resulting RDF triples can be queried with SPARQL, and the paper reports that this network supports question answering: 287 of 365 sampled KenSwQuAD questions answered exactly (78.6% exact match). The authors argue that this demonstrates that part-of-speech tags alone, without word embeddings or trained models, are sufficient to create a usable knowledge structure for a low-resource SVO language.","pith_inferences":["Beyond the paper, publishing the full triple lists with false-triple counts would let others see how much noise the Cartesian-product step produces and how much the SPARQL queries tolerate.","Beyond the paper, replacing the missing named-entity and coreference resolution with a small resolver would provide a direct test of whether spurious triples are the main limitation on accuracy.","Beyond the paper, the same rule set should transfer to other SVO low-resource languages that have a POS tagger, turning the claim from a Kiswahili-specific result into a general method.","Beyond the paper, the near-zero scores on 'define' and 'why' questions suggest the semantic network is best used as a fact retriever feeding a separate reasoning component rather than as a complete question-answering system."],"forward_implications":["A POS tagger, not training data, is the only language resource needed to build a queryable semantic network for an SVO low-resource language.","The generated RDF triple store can be queried with SPARQL, so the structured output can plug into existing linked-data tooling.","Question answering from the network is strong on factual lookup questions such as who, what, where, and when, and weak on explanatory questions such as define and why, matching the paper's per-type results.","Adapting the rules to SOV-type languages should be possible, extending the method beyond Kiswahili.","Named-entity decomposition and coreference gaps leave the network noisy and partially disconnected, so better preprocessing tools would likely raise accuracy."],"supporting_citations":[{"why":"supplies the prior Swahili NLP pipeline whose SVO-identification stage this algorithm expands.","marker":"B. Wanjawa & Muchemi, 2021"},{"why":"provides the TyDiQA Swahili gold-standard set used for the 64.8% exact-match test.","marker":"Clark et al., 2020"},{"why":"provides the KenSwQuAD dataset and the deep-learning baseline (F1 59.4%, EM 48%) that the reported 78.6% is measured against.","marker":"B. W. Wanjawa et al., 2023"},{"why":"provides the Tusome early-literacy corpus used in the third QA test with 63.7% exact match.","marker":"Piper et al., 2018"},{"why":"is the Kiswahili POS tagger that produces the part-of-speech input the algorithm consumes.","marker":"aflat, 2020"},{"why":"supports the premise that Kiswahili follows the SVO word order that makes the direct mapping possible.","marker":"Sánchez-Martínez et al., 2020"}],"fun_headline_variants":["Swahili semantic nets from POS tags alone hit 78.6% QA","No training data: Swahili QA solved with POS tags at 78.6%","Cartesian product of nouns builds Swahili knowledge graph for QA","POS-only semantic network yields 78.6% exact match on Swahili QA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The algorithm assumes that any noun to the left of a verb and any noun to the right of it can be stitched together into a true fact, even though real sentences contain named entities, pronouns, and clauses that do not pair that way.","fun_headline_variants_meta":{"raw":{"variants":["Swahili semantic nets from POS tags alone hit 78.6% QA","No training data: Swahili QA solved with POS tags at 78.6%","Cartesian product of nouns builds Swahili knowledge graph for QA","POS-only semantic network yields 78.6% exact match on Swahili QA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000501,"raw_usage":{"total_tokens":2404,"prompt_tokens":853,"completion_tokens":1551,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":469,"completion_tokens_details":{"reasoning_tokens":1464}},"tokens_in":469,"tokens_out":1551,"duration_ms":11068,"temperature":1.0,"reasoning_tokens":1464,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:06:01.667974+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check: run the algorithm on a set of fifty Kiswahili contexts from the TyDiQA gold set, list every emitted triple, and have native speakers label each one as a valid fact or a spurious pairing. If the majority of triples are spurious yet exact-match QA still reaches the reported 78.6%, then the paper's account of how the network answers questions is incomplete; if the majority are valid, the Cartesian-product rule is doing the work the paper claims.","supporting_citations":[],"review_version":1}