{"id":"2d2b8c52-c3cf-48a2-9c01-d884f8fee661","arxiv_id":"1908.09428","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"CoinNet, a convolutional network combining compact bilinear pooling, residual blocks, and attention, achieves 98.5% top-1 accuracy on the new 100-class RRCD-Main dataset of Roman Republican coin reverses.","lead":"This paper trains a deep learning model, CoinNet, to classify ancient Roman Republican coins by recognizing objects on their reverse sides, reaching over 98% accuracy on a new dataset of more than 18,000 coin images. It introduces RRCD, a large public dataset of Roman Republican coin reverse motifs that could help historians and numismatists automate coin identification.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The §5.6 disjoint-test protocol counts any output among the 12 biga training classes as correct for unseen biga images, so the reported >30% generalization margins measure coarse object recognition, not recognition of unseen coin types.","rationale":"The reader's weakest-assumption analysis identifies exactly the issue I consider most load-bearing: the disjoint-test metric in Section 5.6 uses a coarse object-level proxy and then interprets it as fine-grained generalization to unseen coin classes. The paper's main classification accuracy claim of 98.5% on RRCD-Main is plausible, and the dataset release and ablation study are valuable, but the generalization contribution is the weakest link because it is supported only by this proxy. My concrete test would settle whether the concern lands by comparing all methods at a true object-level granularity and by inspecting whether predictions spread across semantically distinct training classes. Since the reader's conditional verdict already requires re-evaluating the generalization protocol, I do not see grounds to move the verdict; the conditional recommendation stands.","tokens_in":12710,"tokens_out":4967,"duration_ms":50320,"concrete_test":"Re-evaluate Table 5 under an explicit object-level protocol: merge the 12 biga, 4 quadriga, and 1 curule-chair RRCD-Main outputs into single 'biga', 'quadriga', and 'curule' categories, and compute per-object accuracy for CoinNet, VGG, and NASNet on the same 1,155 disjoint images. If CoinNet's margin over the baselines persists, the coarse generalization claim is supported; if it shrinks or reverses, the 30%+ margins are an artifact of the 12-way proxy. Additionally, on a random subset of disjoint images, manually label the exact Crawford type and report the distribution of predicted training classes, to verify that predictions are not dominated by one or two training classes.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.6 evaluates on RRCD-Disjoint, which consists of 128 unseen coin classes (700 biga, 111 curule-chair, and 344 quadriga images), but the paper supplies no ground-truth class labels for these images. The stated workaround is that a disjoint test image whose main object is 'Biga' is counted as correctly classified if the model outputs any one of the 12 RRCD-Main classes containing a biga. This collapses all biga variants into a single semantic bucket, and similarly for quadriga and curule chair. Consequently, the Table 5 numbers (96.56% biga, 68.15% quadriga, 79.28% curule) do not demonstrate that CoinNet recognizes unseen coin classes; they show that, under this proxy, CoinNet places most unseen images into some training class sharing the same coarse object. The claimed 30%+ margin over VGG and NASNet is therefore not evidence for the paper's stated contribution that the solution 'generalizes to completely disjoint test sets' at the level of coin classes. The object-level result may still be useful, and the 98.5% RRCD-Main result is unaffected, but the generalization claim is overstated as written because Table 5 is the only experimental support for it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces RRCD, a dataset of 18,285 images of ancient Roman Republican coin reverses spanning 228 motif classes (100 in RRCD-Main and 128 in RRCD-Disjoint), and proposes CoinNet, a network that fuses DenseNet161 and ResNet50 feature maps via compact bilinear pooling, applies residual groups and spatial attention, and classifies reverse motifs. On a 30%-70% train-test split of RRCD-Main, CoinNet reports 98.5% top-1 accuracy, outperforming fine-tuned VGG (97.4%), NASNet (97.8%), and earlier BoVWs methods. The paper also claims strong generalization on RRCD-Disjoint, where test images share only a coarse object label (biga, quadriga, curule chair) with the training set, and reports large margins over baselines under a proxy that counts a prediction as correct if it falls into any training class with the same coarse object.","tokens_in":12946,"tokens_out":4009,"duration_ms":40281,"significance":"The RRCD dataset is a potentially valuable resource for cultural-heritage and fine-grained recognition research, and the paper's main accuracy result on RRCD-Main is well supported by the reported experiments. The use of compact bilinear pooling with multi-network feature fusion is a sensible technical direction, and the ablation study in Table 3 shows robustness across feature extractors. The paper also makes model and dataset availability claims, which supports reproducibility. However, the generalization claim is overstated because the disjoint-set evaluation collapses 128 unseen coin classes into three coarse object buckets, so the reported >30% margins do not demonstrate recognition of unseen coin types at the class level. This does not undermine the main accuracy figure, but it does affect one of the paper's stated contributions.","major_comments":[{"comment":"The disjoint-test evaluation counts a test image as correctly classified if the model outputs any training class sharing the same coarse object (biga, quadriga, or curule chair). Because the 128 RRCD-Disjoint classes are not individually labeled, this proxy collapses all variants of each object into a single semantic bucket and measures object-level agreement, not recognition of unseen coin types. The >30% margins over VGG and NASNet therefore do not support the claimed generalization to 'completely disjoint test sets' at the coin-class level. I recommend either annotating the disjoint images with their Crawford class labels and reporting fine-grained top-1 accuracy, or clearly reframing the claim as object-level generalization.","section":"Section 5.6, Table 5"},{"comment":"The reported improvements over VGG (1.1%) and NASNet (0.7%) come from a single 30%-70% split with no variance estimates, confidence intervals, or multiple runs. Given the small margins, the claim that CoinNet significantly outperforms these baselines on RRCD-Main is not statistically substantiated. I ask for repeated runs (or at least a per-class breakdown and statistical test) to strengthen the central accuracy claim.","section":"Section 5.2, Table 2"}],"minor_comments":[{"comment":"The dimension flow in Figure 6 (e.g., '14 x 14 x 16k' and '14 x 14 x 2k') is not explained in the text; please clarify whether compact bilinear pooling is applied per spatial location and what the output dimensionality of the residual group is.","section":"Section 4.2, Figure 6"},{"comment":"The text says vocabulary sizes are 'empirically selected' but does not state which setting from Table 4 is used for the BoVWs and RT results in Table 2; please specify the chosen vocabulary size explicitly.","section":"Section 5.5, Table 4"},{"comment":"The loss function uses both qi(y) and pi(y) in Eq. (6) but then refers to qi(x) in the following sentence; please make the notation consistent.","section":"Section 4.2, Equation (6)"},{"comment":"The Limitations subsection discusses only low-resolution and blur effects; it would be appropriate to also acknowledge the coarse-label proxy used in the disjoint-set evaluation, since that limits the interpretation of the generalization results.","section":"Section 5.7"},{"comment":"The precision and recall figures are reported without specifying whether they are macro-averaged or micro-averaged; please clarify.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"The main accuracy result and the dataset contribution are credible, and the code release promise is a positive sign. The key weakness is the disjoint-set protocol: the generalization claim is load-bearing for the paper's third contribution, and the proxy metric inflates the apparent margins. This is fixable by re-annotating the disjoint images or tempering the claim, so I recommend major revision rather than rejection. I would also encourage the editor to verify that the GitHub repository actually contains the dataset and evaluation scripts, since the abstract and conclusion promise public availability."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is the dataset: 18,285 images across 228 reverse-motif classes, collected from museum and auction sources and cross-checked against Crawford. That is a real resource for computational numismatics, and the public release makes it usable. The 98.5% top-1 accuracy on RRCD-Main is also plausible and supported by the reported comparisons against fine-tuned VGG and NASNet. The ablations and Grad-CAM examples are standard but adequately done.\n\nThe soft spot is exactly where the stress-test note lands. Section 5.6 evaluates on RRCD-Disjoint, but there are no ground-truth class labels for those 128 unseen coin classes. The workaround counts a biga image as correctly classified if the model outputs any of the 12 biga-containing training classes, and similarly for quadriga and curule chair. That measures coarse object recognition, not recognition of unseen coin types. The >30% margins over VGG and NASNet in Table 5 therefore do not support the paper's claim of generalizing to completely disjoint coin classes. The authors should either annotate the disjoint images with their true classes or explicitly reframe the evaluation as coarse-grained object-level generalization and soften the conclusion.\n\nTwo smaller issues. First, the dataset curation discards deteriorated images where the reverse motif is hard for experts to distinguish. That is a reasonable practical choice, but it means the reported accuracy likely overstates performance on coin images in the wild; a sentence acknowledging that selection bias would help. Second, there are no variance or significance numbers anywhere, so the 0.7–1.1% margins over baselines in Table 2 may be within run-to-run noise. Minor formatting issues (precision/recall with a stray % sign) are not worth referee time.\n\nThe citation pattern is fine. The BoVWs baselines are from the authors' own prior work, but those are genuine comparisons, not inputs to the model.\n\nThis paper deserves a serious referee. The dataset alone justifies that. The fix is not conceptually hard: redo the generalization evaluation with proper labels or an explicit coarse-grained metric, add confidence intervals, and moderate the claims. I would bring it to reading group as an example of a domain-specialized dataset paper with a cautionary tale about evaluation proxies.","headline":"The RRCD dataset is the real contribution and the 98.5% main-split number is credible, but the §5.6 generalization claim rests on a permissive proxy that collapses unseen coin types into coarse object buckets.","tokens_in":13497,"tokens_out":1163,"would_cite":true,"duration_ms":14128,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Reverse-motif recognition identifies ancient Roman Republican coins with 98.5% top-1 accuracy.","keywords":["ancient coin classification","Roman Republican coins","reverse motif recognition","compact bilinear pooling","convolutional neural networks","feature attention","deep learning for cultural heritage","coin dataset"],"falsifier":"Retrain or reuse the described CoinNet on RRCD-Main, then test it on the 128 RRCD-Disjoint classes with the exact motif class per image as the label rather than the grouped main object. If accuracy on biga, quadriga, and curule-chair images drops toward chance (roughly 1/128, or at least far below the reported 68–96%), the generalization claim is refuted.","tokens_in":12488,"feed_emoji":"🪙","tokens_out":8346,"duration_ms":76516,"temperature":0.7,"pith_summary":"This paper argues that the main object on the reverse side of an ancient Roman Republican coin—its reverse motif—can serve as a reliable, coarse-grained key to identifying the coin. To test that, the authors assemble a dataset of 18,285 images spanning 228 reverse-motif classes and train CoinNet, a network that fuses features from two pre-trained convolutional networks by compact bilinear pooling, then refines them with residual blocks and spatial attention. On a 70/30 split of the 100 main classes, the paper reports 98.5% top-1 accuracy, ahead of bag-of-visual-words baselines and fine-tuned VGG and NASNet. The authors further report large margins over baselines on a disjoint set of 128 unseen style classes, arguing that the model generalizes to coin types it never trained on. If this holds, automated reverse-motif recognition could narrow down a query coin to the few classes sharing that motif, reducing the manual effort of numismatic attribution.","feed_headline":"Reverse motifs identify ancient Roman coins at 98.5%","feed_subtitle":"CoinNet fuses two CNN feature maps with attention, beating VGG, NASNet, and bag-of-words on 228 classes.","key_machinery":"The load-bearing mechanism is the fusion of two feature maps, $\\alpha_I$ from DenseNet161 and $\\beta_I$ from ResNet50, produced by the two networks before their final layers. Instead of a full bilinear outer product, which would explode memory, CoinNet uses compact bilinear pooling: each feature map is passed through a randomized count-sketch projection and the two sketches are convolved, equivalently $\\varphi(\\alpha,\\beta,u,v)=F^{-1}(F(\\varphi(\\alpha,u,v))\\odot F(\\varphi(\\beta,u,v)))$, so pairwise interactions are captured in a low-dimensional representation. A group of residual blocks then refines the fused vector, an $\\ell^2$ normalization stabilizes it, and a learned soft-attention map over the $14\\times14$ spatial grid emphasizes informative regions before the final classifier. The paper's ablation shows the accuracy stays near 98.4–98.5% when the backbone pair is replaced, which it interprets as evidence that the fusion, residual, and attention stages, rather than the specific encoder, carry the discriminative information.","core_discovery":"CoinNet's central claim is that a compact bilinear fusion of two deep feature maps, followed by residual groups and soft attention, is sufficient to classify eroded and stylistically varied Roman Republican reverse motifs at near-ceiling accuracy. On RRCD-Main (100 classes, roughly 70% training after augmentation), it reaches 98.5% top-1 accuracy, with precision 0.907 and recall 0.951, outperforming VGG (97.4%), NASNet (97.8%), and the spatial bag-of-visual-words baselines (up to 84.4%). The paper also claims strong generalization: using a protocol that treats any training class sharing the main object as a correct match, CoinNet scores 96.56% on 81 biga classes, 68.15% on 35 quadriga classes, and 79.28% on 12 curule-chair classes, all unseen, beating VGG and NASNet by more than 30% in several comparisons.","pith_inferences":["A stricter test—assigning each RRCD-Disjoint image its actual unseen motif class and evaluating 128-way classification—would probably reduce the reported margins, because the paper's protocol counts any class containing the same main object as correct; this is a natural next experiment.","Because motif and legend are complementary cues and the paper notes legends are more worn, combining reverse-motif recognition with a separate legend reader could yield finer-grained coin attribution than either cue alone.","The architecture's invariance to encoder choice suggests the fusion and attention stages are where the discriminative signal lives; an ablation replacing compact bilinear pooling with average-pooled concatenation would make that dependency explicit.","The protocol itself, where a prediction is correct if it shares a semantic object with the true coin, could be reused as a benchmark for object-level attribute recognition rather than exact coin-type classification."],"forward_implications":["If 98.5% reverse-motif accuracy holds, a practical coin-attribution tool can narrow a query to the small subset of catalog classes showing that motif, leaving legend and symbol reading for final disambiguation.","The disjoint-set results imply that models trained on well-preserved museum images can still recognize motif styles from auction photos, provided the evaluation groups classes by shared object.","The insensitivity to backbone choice means the method can be reimplemented with lighter encoders without expected loss, which matters for deployment on modest hardware.","The released 18,285-image, 228-class RRCD gives later work a fixed benchmark for coarse-grained ancient coin recognition."],"supporting_citations":[{"why":"Defines the standard reference numbering for Roman Republican coin classes that structures the RRCD categories and collection process.","marker":"[1]"},{"why":"Provides the previous largest reverse-motif dataset and the bag-of-visual-words baseline that CoinNet compares against.","marker":"[3]"},{"why":"Supplies a CNN-based ancient-coin recognition baseline and dataset that the paper contrasts with its own.","marker":"[7]"},{"why":"Introduces compact bilinear pooling, the fusion operator at the center of CoinNet.","marker":"[33]"},{"why":"Provides the multimodal compact bilinear pooling approach used for combining the two feature maps.","marker":"[34]"},{"why":"Supplies the DenseNet161 encoder that produces one of the two fused feature maps.","marker":"[37]"},{"why":"Supplies the ResNet50 encoder that produces the other fused feature map.","marker":"[38]"},{"why":"Provides the large generic image corpus used to pretrain the backbone networks before fine-tuning on coins.","marker":"[39]"},{"why":"Is the NASNet baseline whose fine-tuned accuracy (97.8%) CoinNet is compared against.","marker":"[44]"},{"why":"Is the VGG baseline whose fine-tuned accuracy (97.4%) CoinNet is compared against.","marker":"[45]"}],"fun_headline_variants":["CoinNet reads eroded Roman coins with 98.5% accuracy","Attention and feature fusion crack ancient coin puzzles","Roman coin classification hits 98.5% via deep fusion","Eroded Roman coins identified by fused CNN features","CoinNet beats baselines on 228 Roman coin motifs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The system's strongest reported evidence rests on a proxy: a disjoint test coin is deemed correct if it falls into any one of the training classes whose reverse motif contains the same main object, so 'generalization' means recognizing the object, not the unseen class.","fun_headline_variants_meta":{"raw":{"variants":["CoinNet reads eroded Roman coins with 98.5% accuracy","Attention and feature fusion crack ancient coin puzzles","Roman coin classification hits 98.5% via deep fusion","Eroded Roman coins identified by fused CNN features","CoinNet beats baselines on 228 Roman coin motifs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000445,"raw_usage":{"total_tokens":2260,"prompt_tokens":964,"completion_tokens":1296,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":580,"completion_tokens_details":{"reasoning_tokens":1216}},"tokens_in":580,"tokens_out":1296,"duration_ms":10098,"temperature":1.0,"reasoning_tokens":1216,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:11:00.646361+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain or reuse the described CoinNet on RRCD-Main, then test it on the 128 RRCD-Disjoint classes with the exact motif class per image as the label rather than the grouped main object. If accuracy on biga, quadriga, and curule-chair images drops toward chance (roughly 1/128, or at least far below the reported 68–96%), the generalization claim is refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the standard reference numbering for Roman Republican coin classes that structures the RRCD categories and collection process."},{"cited_title":"Anwar, S","cited_arxiv_id":null,"evidence_quote":"Provides the previous largest reverse-motif dataset and the bag-of-visual-words baseline that CoinNet compares against."},{"cited_title":"Schlag, O","cited_arxiv_id":null,"evidence_quote":"Supplies a CNN-based ancient-coin recognition baseline and dataset that the paper contrasts with its own."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces compact bilinear pooling, the fusion operator at the center of CoinNet."},{"cited_title":"Fukui, D","cited_arxiv_id":null,"evidence_quote":"Provides the multimodal compact bilinear pooling approach used for combining the two feature maps."},{"cited_title":"Huang, Z","cited_arxiv_id":null,"evidence_quote":"Supplies the DenseNet161 encoder that produces one of the two fused feature maps."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the ResNet50 encoder that produces the other fused feature map."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the large generic image corpus used to pretrain the backbone networks before fine-tuning on coins."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Is the NASNet baseline whose fine-tuned accuracy (97.8%) CoinNet is compared against."},{"cited_title":"Simonyan, A","cited_arxiv_id":null,"evidence_quote":"Is the VGG baseline whose fine-tuned accuracy (97.4%) CoinNet is compared against."}],"review_version":1}