{"id":"b9559e30-4bad-4873-b320-4e7bc27989d0","arxiv_id":"1908.07857","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Combining three nearest-neighbor classifiers with confidence-weighted voting is reported to reach 88.6 percent on Tibetan characters, but the comparison discards samples all classifiers missed.","lead":"This thesis combines three nearest-neighbor classifiers for Tibetan character recognition and reports that weighted voting raises accuracy above the best single classifier. The claimed gain is weakened because final accuracy is measured only on the subset of samples that at least one classifier predicted correctly, while baseline accuracy is measured on a different, unfiltered test set.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported ensemble accuracies are computed on a filtered subset (samples where at least one base classifier is correct), not on the full validation set; recomputing them with discarded samples counted as errors drops every reported final accuracy below the best single-classifier baseline.","rationale":"The reader's weakest assumption identifies the same load-bearing flaw: Section 4.1.8 defines the final accuracy only over validation samples where at least one base classifier is correct, and Chapter 4 repeats this protocol for every hypothesis. The individual classifier accuracies in Table 5.1 are not conditioned on that subset, so the comparison in the paper's strongest claim is not apples-to-apples. The concern is not merely stylistic: using the paper's own numbers, 223 correct out of 255 becomes 223 out of 320 when discarded samples are counted as errors, reducing the headline 87.45% to 69.69%, far below the 84.06% baseline. This is an internal validity problem, not a disagreement with prevailing consensus. The paper itself flags the issue in Section 4.1.8 and in Chapter 6, but the conclusion still asserts success. A single recomputation on the full validation set, or a conditional-accuracy comparison on the same subset, would settle the matter. No independent support (code, data, or formal verification) is provided, so the reported arithmetic is the only evidence for the central claim, and that arithmetic is unreliable.","tokens_in":31291,"tokens_out":4824,"duration_ms":44918,"concrete_test":"Recompute the results from Chapter 4 using the full 320-sample validation set: for each hypothesis, count every discarded sample as an error and report ensemble accuracy as correct/total 320. Also report each base classifier's accuracy on the same 255/233/227/228-sample subset used for that hypothesis. If the full-set ensemble accuracy is below the best single classifier's full-set accuracy (as the arithmetic here suggests), the central claim fails.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim is Table 5.1's 'overall accuracy increases' for all five hypotheses. In Sections 4.1.8, 4.2.2, 4.3.6, 4.4.6, and 4.5.1 the 'final accuracy' is defined as (correct ensemble predictions) / (validation samples for which at least one base classifier is correct). Samples on which all three classifiers are wrong are explicitly discarded. The baseline accuracies in Table 5.1 (e.g., 84.06% for Classifier #1) are unconditional accuracies on the test split. This mixes denominators. Recomputing the ensemble results over all 320 validation samples (64 classes x 5 validation samples) turns Hypothesis #1's 223/255=87.45% into 223/320=69.69%, and Hypothesis #2's 226/255=88.63% into 226/320=70.63%, both below the 84.06% baseline. Similar arithmetic applies to Hypotheses #3-5 (194/320=60.63%, 190/320=59.38%, 191/320=59.69%). Thus the claimed gain is an artifact of evaluating on an easier, selected subset. The text itself discloses the conditioning in Section 4.1.8 ('at least one classifier should give the correct prediction') and Chapter 6 admits that discarding samples 'affected in the overall accuracy,' but this limitation is not carried into the conclusion. A fair comparison requires either evaluating the ensemble on the full validation set (with discards counted as errors) or comparing against each base classifier's accuracy on the same filtered subset, with coverage reported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript tests five ensemble strategies for combining three nearest-neighbour classifiers on a 64-class Tibetan character recognition task. The classifiers use raw pixels, centroid-distance structural features, and Gabor-filter responses reduced by PCA; the five 'hypotheses' differ in how per-class confidence weights are derived and combined. The paper's central claim, in Table 5.1, is that every hypothesis improves on the best single classifier, with Hypothesis #2 reaching 88.63% versus 84.06% for the best base classifier. The thesis also documents the data distribution, confusion matrices, and lists of discarded validation samples.","tokens_in":31655,"tokens_out":9286,"duration_ms":86631,"significance":"If the reported comparison were valid, the work would give a simple demonstration that confidence-weighted voting of three nearest-neighbour classifiers improves character-recognition accuracy, and the explicit recording of discarded samples would be a useful auditing feature. However, the central comparison is not valid: ensemble accuracies are computed on a filtered validation subset while the baseline accuracies are unconditional and from a different split. The paper also selects hyperparameters on the test set and never tests its stated independence assumption. The manuscript does not, as presented, establish the claimed improvement, and the experimental design cannot be repaired without changing the conclusion.","major_comments":[{"comment":"The headline accuracies are computed on the subset of validation samples for which at least one base classifier is correct, not on the full validation set. For Hypothesis #1, Section 4.1.8 reports 223 correct out of 255 such samples (87.45%), and the 65 samples for which all three classifiers were wrong are explicitly listed and discarded; Sections 4.2.2, 4.3.6, 4.4.6, and 4.5.1 do the same. The baseline accuracies in Table 5.1 (e.g., 84.06% for Classifier #1) are unconditional accuracies on the separate test split. Counting the discarded samples as errors gives 223/320=69.69% for Hypothesis #1 and 226/320=70.63% for Hypothesis #2, both below the 84.06% baseline; the corresponding unconditional values for Hypotheses #3-#5 are 194/320=60.63%, 190/320=59.38%, and 191/320=59.69%, all below the 77.81% baseline used for those hypotheses. The conclusion in Chapter 5 that 'the overall accuracy increases' is therefore unsupported. A valid comparison requires either counting discarded samples as errors on the full validation set or reporting the base-classifier accuracies on the same filtered subset with the coverage rate.","section":"Sections 4.1.8, 4.2.2, 4.3.6, 4.4.6, 4.5.1; Table 5.1"},{"comment":"The same test split is used for multiple purposes: Tables 4.1 and 4.2 select the PCA component count (110) and Gabor kernel size (7) by maximizing accuracy, Section 4.1.6 builds the confidence matrices from the classifiers' confusion matrices on that split, and Table 5.1 reports the base-classifier accuracies from that split. The ensemble is then evaluated on the validation split. This protocol provides no independent estimate of generalization: the reported base accuracies are optimistic because the hyperparameters were tuned on the test set, and the confidence weights are not validated on data unseen during their construction. A separate validation set should be used for model selection and confidence estimation, or nested cross-validation should be employed.","section":"Sections 3.3.3, 4.1.3, 4.1.4, 4.1.6"},{"comment":"The manuscript states as a basic assumption that the classifiers are conditionally independent and do not make the same mistakes, but it provides no diversity measure or error-correlation analysis, and Chapter 6 admits that this property was neither achieved nor proven. The coverage numbers themselves (255/320 for Hypotheses #1-#2, 233/320 for Hypothesis #3, 227/320 for Hypothesis #4, 228/320 for Hypothesis #5) show that a substantial fraction of validation samples are missed by all three classifiers, which is direct evidence that errors are not complementary in the strong sense required by the motivation. The independence assumption needs to be tested (e.g., with pairwise Q-statistics or disagreement measures) before the success of the method can be attributed to diversity.","section":"Chapter 3 (basic assumption) and Chapter 6"}],"minor_comments":[{"comment":"Hypothesis #2 accuracy is given as 88.62% in the text and 88.63% in the table; please reconcile.","section":"Section 4.2.2 and Table 5.1"},{"comment":"Classifier #3 accuracy for Hypothesis #3 is 60.62% in Section 4.3.3 and 60.63% in Table 5.1; for Hypothesis #4 it is 4.37% in Section 4.4.3 and 4.38% in Table 5.1; please reconcile these values.","section":"Sections 4.3.3, 4.4.3 and Table 5.1"},{"comment":"Section 3.3.3 says PCA reduces the Gabor feature dimension to 100 as the optimum, while Section 4.1.3 concludes that 110 components is optimal; the text should be consistent about the actual setting used for Classifier #3.","section":"Sections 3.3.3 and 4.1.3"},{"comment":"The reference list is incomplete: the citation for Freund and Schapire/Quinlan in Section 2.2 is [5], but the list jumps from [4] to [6] without an entry for [5].","section":"References"},{"comment":"The 15/5/5 train/test/validation split is described only at the level of counts; please specify how the random selection was performed and whether the split was stratified, to make the experiments reproducible.","section":"Section 3.1"},{"comment":"There are numerous typographical and OCR artifacts (e.g., 'Bressenham' for Bresenham, broken equations in Chapter 2) that should be cleaned up in a revised version.","section":"General"}],"recommendation":"reject","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe reader's high-confidence reject is right, and the stress-test arithmetic holds up. The headline \"overall accuracy increases\" is an artifact of comparing a conditional accuracy on a filtered subset to an unconditional accuracy on the full set. Section 4.1.8 states the conditioning plainly: only 255 of 320 validation samples survive (samples where at least one classifier is correct), and the hypothesis gets 223 of those right, so the reported 87.45% is 223/255. Put the 65 discarded samples back as errors and you get 223/320 = 69.69%, well below the 84.06% baseline. The same arithmetic applies to the other four hypotheses. Since this is the load-bearing number of the thesis, the main conclusion falls.\n\nTo give credit: this is a real student effort. It builds three genuinely different nearest-neighbor classifiers (raw pixels, centroid-distance, Gabor+PCA) and applies them to a Tibetan character dataset, which is a legitimate, niche, new application. The literature survey is competent, and the five fusion rules are described concretely. The text is honest about the filtering, and the future-work chapter admits that discarding samples \"affected in the overall accuracy.\" So this is an evaluation error, not a hidden manipulation.\n\nThe soft spots beyond the main one: the five hypotheses are minor variants of confidence-weighted voting, so the novelty is low. Hyperparameters (number of PCA components, Gabor kernel size) appear to be tuned using the test set, with no error bars or independent validation. No code or data is released, and there is no comparison against a simple majority-vote baseline on the same subset. The per-hypothesis classifier accuracies also differ, making the comparison table in Chapter 5 internally messy.\n\nWho gets value from this? Someone studying Tibetan OCR might find the dataset construction and feature ideas worth a look, but without released data or code that value is mostly aspirational. For ensemble methodology, the paper adds nothing over the literature it cites.\n\nMy recommendation: desk reject. The central comparison is invalid, and the contribution is incremental. If the authors fixed the evaluation and released the data, a short application note might be publishable, but not this version.","headline":"The headline accuracy gains vanish once the discarded samples are counted as errors; the Tibetan OCR setting is real, but the central comparison is not valid.","tokens_in":32134,"tokens_out":4138,"would_cite":false,"duration_ms":40960,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a confidence-weighted combination of three nearest-neighbour classifiers raises Tibetan character recognition accuracy from 84.06% to 88.63%.","keywords":["multi-hypothesis classifier","classifier ensemble","confidence matrix","nearest neighbour","Gabor filter","principal component analysis","Tibetan character recognition","weighted majority voting"],"falsifier":"Recompute the ensemble accuracy for Hypothesis #2 on the full validation set without discarding samples where all three classifiers err, and compute the three individual classifiers' accuracies on that same validation set. If the unconditional ensemble accuracy no longer exceeds 84.06%, or the gain shrinks once baseline and ensemble are measured on identical samples, the reported improvement is an artifact of conditioning. The 64 discarded sample numbers listed in the paper provide the exact subset to restore.","tokens_in":31094,"feed_emoji":"🔤","tokens_out":4698,"duration_ms":45255,"temperature":0.7,"pith_summary":"This thesis tries to establish that a small committee of three deliberately different nearest-neighbour classifiers, each built on a different representation of Tibetan character images, can be fused by confidence values so that the committee beats every one of its members. The three base classifiers use raw pixels, centroid-distance shape features, and Gabor-filter features reduced by PCA, and the paper devises five fusion hypotheses based on confidence matrices. It reports that all five combinations improve on the best individual classifier, with the top result of 88.63% against an 84.06% single-classifier baseline. The broader point is that combining strong classifiers is worthwhile, not only combining many weak ones.","feed_headline":"Ensemble vote lifts Tibetan character accuracy to 88.6%","feed_subtitle":"A confidence-weighted committee of nearest-neighbour models outdoes every single model on 64 Tibetan character classes.","key_machinery":"The central object is the confidence matrix: for each base classifier, each diagonal entry of its confusion matrix is divided by the column sum, giving a per-class belief that the classifier is right when it predicts that class. Fusion works by treating this matrix as a lookup table, adding or comparing confidence values for the class labels proposed by the three classifiers, and then picking the label with the highest weighted support. Hypothesis #2 substitutes a sample-level confidence: in nearest-neighbour search, the ratio of the second-best distance to the best distance measures how decisive the top match is, and this sample-level confidence drives the vote. This machinery is what converts three separate label predictions into a single ensemble decision.","core_discovery":"On the paper's own terms, the central claim is that a multi-hypothesis classifier improves accuracy in all five tested fusion strategies. Each base classifier is treated as a hypothesis whose confusion matrix can be converted into a confidence matrix; the ensemble then consults these confidence values and combines the individual predictions by weighted majority or maximum confidence. The best variant, Hypothesis #2, builds confidence from the ratio between the distance to the best match and the distance to the second-best match in nearest-neighbour search, and reaches 88.63%, compared with 84.06% for the strongest single classifier. The paper also observes that certain visually similar Tibetan character classes are consistently confusable and remain errors even after fusion.","pith_inferences":["A fair test of the claimed gain would compute both the ensemble and the 84.06% baseline on the identical, complete validation set, including samples every classifier gets wrong; until then the reported gain is conditional, not guaranteed.","The confidence-matrix design is generic, so the same five hypotheses could be run on printed Latin, Devanagari, or Chinese character datasets with fixed train/test splits; if the conditional gain disappears there, the method is specific to this Tibetan dataset.","The failure cases named by the paper suggest a testable extension: add a rejection stage that sends low-confidence samples to a dedicated pairwise discriminator rather than forcing the ensemble to choose among visually similar classes."],"forward_implications":["A committee of strong, feature-diverse classifiers can outperform each of its members on the same data.","The decisive-match distance ratio in nearest-neighbour search is a usable confidence signal for voting, not just a ranking criterion.","Fusion cannot repair confusions that all base classifiers share, such as visually near-identical Tibetan character classes.","The reported gain is measured only on validation samples where at least one base classifier is correct, so the ensemble's operational accuracy on every sample is a separate question.","Adding more diverse classifiers or replacing random sample selection with deliberate sampling could further raise ensemble accuracy, as the paper's future-work discussion suggests."],"supporting_citations":[{"why":"Supplies the statistical, computational, and representational rationale for why classifier ensembles can beat a single classifier.","marker":"[1]"},{"why":"Provides the general framework for combining pattern classifiers that the thesis builds on.","marker":"[2]"},{"why":"Establishes that combining classifiers yields little if they make the same errors, which motivates the need for diverse base classifiers.","marker":"[3]"},{"why":"Derives the fixed combination rules (product, sum, max, min, median) under which the confidence-weighted voting used here is framed.","marker":"[8]"},{"why":"Presents Bayesian and Dempster-Shafer classifier-combination approaches that inform the confidence-based fusion strategies.","marker":"[9]"},{"why":"Defines diversity levels for classifier ensembles, including the level where at least one classifier is correct for each pattern, which the paper's evaluation protocol mirrors.","marker":"[11]"}],"fun_headline_variants":["Confidence-weighted NN committee hits 88.63% on Tibetan","Weighted vote lifts Tibetan character accuracy to 88.6%","Multi-hypothesis classifier tops single NN by 4.6 points","Ensemble of NNs scores 88.6% on 64 Tibetan classes","Hypothesis fusion outscores best single classifier"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that every counted validation sample is correctly labelled by at least one of the three classifiers; the paper discards all samples where all three are wrong and reports accuracy only on the remaining samples, so the headline 88.63% is a conditional success rate rather than an accuracy over all validation samples.","fun_headline_variants_meta":{"raw":{"variants":["Confidence-weighted NN committee hits 88.63% on Tibetan","Weighted vote lifts Tibetan character accuracy to 88.6%","Multi-hypothesis classifier tops single NN by 4.6 points","Ensemble of NNs scores 88.6% on 64 Tibetan classes","Hypothesis fusion outscores best single classifier"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000575,"raw_usage":{"total_tokens":2680,"prompt_tokens":875,"completion_tokens":1805,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":491,"completion_tokens_details":{"reasoning_tokens":1715}},"tokens_in":491,"tokens_out":1805,"duration_ms":13596,"temperature":1.0,"reasoning_tokens":1715,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:18:35.514647+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the ensemble accuracy for Hypothesis #2 on the full validation set without discarding samples where all three classifiers err, and compute the three individual classifiers' accuracies on that same validation set. If the unconditional ensemble accuracy no longer exceeds 84.06%, or the gain shrinks once baseline and ensemble are measured on identical samples, the reported improvement is an artifact of conditioning. The 64 discarded sample numbers listed in the paper provide the exact subset to restore.","supporting_citations":[{"cited_title":"In the above equation  denotes the logical AND operator or binary multiplication and in the following equations  denotes the logical OR operator or binary summation","cited_arxiv_id":null,"evidence_quote":"Supplies the statistical, computational, and representational rationale for why classifier ensembles can beat a single classifier."},{"cited_title":"In other words, the classifiers that reject the sample x will have no impact on the combined classifier unless all the classifier reject x","cited_arxiv_id":null,"evidence_quote":"Provides the general framework for combining pattern classifiers that the thesis builds on."},{"cited_title":"drugpa kunleg","cited_arxiv_id":null,"evidence_quote":"Establishes that combining classifiers yields little if they make the same errors, which motivates the need for diverse base classifiers."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Derives the fixed combination rules (product, sum, max, min, median) under which the confidence-weighted voting used here is framed."},{"cited_title":"Which samples to collect from each class of each book, given the number of samples to be collected from each book for each class?","cited_arxiv_id":null,"evidence_quote":"Presents Bayesian and Dempster-Shafer classifier-combination approaches that inform the confidence-based fusion strategies."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines diversity levels for classifier ensembles, including the level where at least one classifier is correct for each pattern, which the paper's evaluation protocol mirrors."}],"review_version":1}