{"id":"5a861385-c361-4b86-8705-3a761d1bb08e","arxiv_id":"2505.03371","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Neural networks assess two-qubit entanglement mainly from coherence-sensitive measurements, while random forests rely most on occupation measurements; the discrepancy disappears when both learn from density matrix elements directly.","lead":"Two machine learning methods, a neural network and a random forest, disagree about which tomography measurements matter most for detecting two-qubit entanglement. The neural network focuses on coherence measurements, while the random forest leans on occupation measurements, a difference that affects which method survives noisy data.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed NN/RF contradiction may reflect task and metric mismatch (classification vs regression, Eq. 17 vs Eq. 18) rather than model class; Shapley results show RF's MA dominance is class-dependent.","rationale":"The reader's weakest assumption focused on MDI bias and non-commensurable perturbation measures. My concern is closely related but more central: the two models are trained for different tasks (classification vs regression) and evaluated with different importance metrics, so the observed 'contradiction' may be a task artifact. This is directly testable and would settle whether the central claim holds. I also identified an internal inconsistency in the paper's Shapley results: the RF's MA dominance is not global but specific to strongly entangled states, which the abstract and introduction do not qualify. This strengthens the need for the conditional acceptance already recommended by the reader. Since the paper's qualitative pattern is plausible and several methods are used, I do not recommend rejection, but the central claim requires the additional task-matched control before it can be considered established. The verdict remains CONDITIONAL, i.e., unchanged from the reader's recommendation, because the requested revision should include this control experiment.","tokens_in":16817,"tokens_out":8496,"duration_ms":87151,"concrete_test":"Retrain the NN as a binary classifier using the same architecture, optimizer, and dataset as the NN regressor, but with a sigmoid output and binary cross-entropy loss (with the classification threshold chosen on a validation set). Evaluate both the NN classifier and the RF classifier with the same perturbation drop measure, Eq. (17), on the same test set. If the NN classifier's block ordering becomes MA-dominant like the RF, the reported contradiction is an artifact of the regression-vs-classification task difference. If the NN classifier remains MD-dominant, then recompute the RF perturbation drop separately on separable-only and entangled-only subsets of the test set; if MD is most important on the separable subset, the global MA dominance is distribution-dependent and the central claim must be qualified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that RF and NN attribute contradictory importance to measurement blocks, but the comparison is confounded by task and metric mismatches. The RF is a binary classifier (Sec. V A) whose perturbation importance is the drop in classification accuracy, Eq. (17). The NN is a regressor trained with MSE (Eq. 11) whose perturbation importance is the mean absolute error of the perturbed predictions, Eq. (18). These are not commensurable: Eq. (18) reports the absolute error level, not the change caused by the perturbation, so it includes the model's baseline error and cannot be directly compared with Eq. (17). The Shapley values in Sec. VI B 3 are computed for the RF's class-probability output but for the NN's continuous C output, again different quantities. This alone makes the block-importance ordering in Fig. 5 an apples-to-oranges comparison. Moreover, the paper's own Shapley results undermine the unqualified 'MA most important' claim for RF: for separable states (Fig. 6a) the RF's most important block is MD, the same non-local coherence block favored by the NN; only for strongly entangled states (Fig. 6b) is MA dominant. The global perturbation measure (Fig. 5a) averages over a test set that is about 53% entangled, so the MA dominance could be an artifact of class balance rather than a model-class property. The density-matrix control experiment does not resolve this because the RF/NN task and metric mismatch persists. Thus the headline assertion that feature importance is model-class dependent is not securely established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper trains a two-hidden-layer MLP regressor and several random-forest classifiers to predict entanglement (concurrence) of two-qubit states from the 16 measurement outcomes of the James-Kwiat-Munro-White tomography protocol. On a synthetic data set of 460,000 training and 46,000 test states, the MLP achieves higher classification accuracy (0.922 vs. 0.879 for the best RF). Using three interpretability tools (MDI, noise-perturbation importance, and Shapley values), the authors report that the RF attributes most importance to the occupation-measurement block MA, whereas the NN attributes most importance to the non-local-coherence block MD. They interpret this as a model-class-dependent effect caused by information scattering in the linear tomography equations, and they claim the discrepancy disappears when the models are trained directly on density-matrix elements.","tokens_in":17214,"tokens_out":7319,"duration_ms":66508,"significance":"If the reported discrepancy is real, the paper provides a useful cautionary example that feature-importance rankings in machine-learned entanglement witnesses can be model-class-dependent rather than reflecting the intrinsic physical importance of the measurements. The work combines three interpretability routes and includes hyperparameter details and class-conditional Shapley analyses, which is a strength. However, several methodological mismatches (between the RF and NN perturbation metrics, between the SHAP targets, and in the threshold-selection procedure) currently prevent the headline claim from being established at the level of rigor required for a journal publication. The results are nonetheless amenable to correction within the manuscript's scope.","major_comments":[{"comment":"The perturbation importance measures for the RF and NN are not commensurable. E_P^RF in Eq. (17) is the drop in classification accuracy caused by perturbing one measurement, whereas E_P^NN in Eq. (18) is the mean absolute error between the perturbed NN output and the ground-truth concurrence without subtracting the unperturbed baseline error. Consequently, the block orderings in Fig. 5(a) and Fig. 5(c) may differ simply because E_P^NN includes the model's baseline error, not because the models use the measurements differently. The authors should redefine E_P^NN as a drop in accuracy (using the same classification threshold as for the RF) or as the increase in error relative to the unperturbed case, and use identical metrics for both model classes.","section":"Sec. VI B 2, Eqs. (17) and (18)"},{"comment":"The Shapley values for the RF are computed for the class-probability output (with TreeExplainer, as stated in Appendix C), while those for the NN are computed for the continuous concurrence output, and the SHAP explainer used for the MLP is not specified. These are different target functions, so the averaged SV patterns in Figs. 6(a,b) and Figs. 6(e,f) are not directly comparable. Additionally, Fig. 6(a) shows that for separable states the RF's most important block is MD, contradicting the unqualified claim that the RF always favors MA; the MA dominance appears only for strongly entangled states (Fig. 6(b)). The claims should be made class-conditional and the SHAP settings for the NN should be reported.","section":"Sec. VI B 3, Fig. 6"},{"comment":"The NN classification threshold tau_NN is chosen by maximizing the area under the precision-recall curve, but the manuscript does not state whether this is done on the test set Stest or on a validation set. If the threshold is optimized on the test set, the accuracy, precision, and recall values for the MLP in Table I are optimistically biased relative to the RF classifiers, whose decision threshold is not tuned. The threshold-selection procedure should be described precisely and performed on a validation split.","section":"Sec. VI A, Eq. (14)"},{"comment":"Mean Decrease in Impurity is known to be biased for correlated predictors, and the tomography measurement features are strongly interdependent through the linear relations in Eqs. (7)-(10). The paper does not discuss this limitation or provide a complementary permutation-based importance for the RF, so the MDI ranking in Fig. 4 should be treated with caution as evidence for the claim that occupations are the most important features for the RF.","section":"Sec. VI B 1, Fig. 4"},{"comment":"The statement that the discrepancy disappears when the models are trained on density-matrix elements is a central piece of the proposed information-scattering explanation, but no quantitative result, figure, or table is presented for this control experiment. The authors should provide the corresponding feature-importance results for both models trained on density-matrix inputs, or explicitly mark this as a claim requiring further verification.","section":"Sec. VI B 2, final paragraph"}],"minor_comments":[{"comment":"In the abstract, 'measurements which arthe most important' contains a typo; it should read 'are the most important'.","section":"Abstract"},{"comment":"The subscript notation E_jk in Eq. (18) is not defined; specify that j indexes test samples and k indexes independent noise realizations.","section":"Eq. (18)"},{"comment":"The vertical axis labels in Figs. 5 and 10 are not fully defined; Fig. 10 in particular should state whether the ordinate is classification accuracy or the perturbation measure E_P, and the color scale in Fig. 5 should be labeled.","section":"Figs. 5 and 10"},{"comment":"The sentence 'utilizing a ReLU activation function' should be plural because each hidden layer has its own ReLU activation.","section":"Sec. V B"},{"comment":"The phrase 'it's results' should be 'its results'.","section":"Sec. VI B 2"},{"comment":"The statement that 'The explanations of these hyperparameters can be found within the main text' is inaccurate; the hyperparameters are defined in the appendix itself, so the cross-reference should be corrected.","section":"Appendix B, Table II"}],"recommendation":"major_revision","confidential_remarks":"The manuscript does not state whether code or data are available; given the sensitivity of the conclusions to the SHAP implementation and threshold choices, I would encourage the editor to request that the authors release the code. The paper is likely of interest to the quantum-ML community if the methodological issues are resolved, but the current version overstates the certainty of the RF/NN discrepancy."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear Colleague,\n\nHere's the bottom line: this paper reports that a neural network and a random forest trained on the same two-qubit tomography data attribute opposite importance to measurement blocks—the NN leans on non-local coherences (block MD), the RF on occupation measurements (block MA)—and that this difference vanishes when both models are trained on density matrix elements instead. That is a new empirical result in the ML-entanglement literature, and it is the sort of finding that should make experimentalists think twice before trusting a single model's feature importance ranking as a window into the physics.\n\nThe paper earns credit for not relying on one interpretability tool. The RF's MA dominance shows up in MDI, perturbation, and Shapley analyses (with the class-dependent caveat below), and the density-matrix control is a sensible check. The qualitative story—that occupations are needed to invert the tomography equations, so the RF builds its decision rules around them—is plausible and worth testing further.\n\nThe soft spots are manageable but real. The perturbation measure for the RF (Eq. 17) is a drop in classification accuracy, while for the NN (Eq. 18) it is the mean absolute error of the regression output, baseline included. That makes cross-model magnitudes meaningless, though within-model rankings over blocks remain informative because the baseline is fixed. The authors should harmonize the measure or explicitly compare relative increases over baseline. More substantively, the Shapley values reveal that the RF's preferred block depends on the class: for separable states, MD is the most important block for the RF; MA dominates only for strongly entangled states. The paper's global claim that \"RF signifies dominance of occupations\" is therefore overstated—it is an average over a test set in which entangled states are slightly overrepresented. The class-resolved Shapley results deserve a more central place in the narrative.\n\nTwo smaller issues: the NN classification threshold τ_NN is tuned on the test set, which inflates the reported accuracy edge, and no code, data, or error bars are provided. The threshold tuning does not affect the regression-based feature importance, but it should still be fixed.\n\nI left this paper feeling the central qualitative finding—model-class-dependent measurement importance rooted in the structure of tomography—survives a careful read. It deserves a serious referee. My recommendation is conditional acceptance: harmonize the perturbation metrics, re-frame the RF claim to reflect the class dependence, and clean up the test-set tuning.","headline":"A genuinely new empirical observation—NN and RF rank tomography blocks oppositely for entanglement—but the global RF claim is overstrong given the class-dependent Shapley results; deserves refereeing after harmonizing the perturbation metrics.","tokens_in":887,"tokens_out":844,"would_cite":true,"duration_ms":99007,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":["03.67.Mn","03.67.-a"],"model":"deepseek-v4-flash","headline":"A neural network and a random forest learn two-qubit entanglement from opposite measurements.","keywords":["entanglement quantification","two-qubit tomography","random forests","neural networks","feature importance","Shapley values","concurrence","measurement importance"],"falsifier":"Retrain both models from scratch after deleting each of the four measurement blocks in turn (leave-block-out). The paper's claim predicts that removing $M_A$ damages the random forest most while the neural network is damaged most by removing $M_D$; if retraining produces the same damage ranking for both models, the reported contradiction is an artifact of the attribution tools rather than a genuine difference in how the models learn.","tokens_in":16618,"feed_emoji":"⚛️","tokens_out":7571,"duration_ms":69763,"temperature":0.7,"pith_summary":"The paper claims that when a neural network and a random forest are trained to quantify two-qubit entanglement from standard projective tomography data, they reach comparable predictions but assign importance to opposite kinds of measurements. The neural network treats measurements in block $M_D$, which carry information about non-local coherences, as most important; the random forest treats the occupation measurements in block $M_A$ as most important. The paper argues this is not a quirk of one dataset: the same ordering appears under mean-decrease-in-impurity, perturbation-based, and Shapley-value analyses, and it disappears when the models are trained on density-matrix elements directly. The result matters because experimental tomography data are often noisy or incomplete, so knowing which measurements a model actually relies on determines how much one can trust its prediction. The authors suggest this difference can be exploited to obtain more reliable entanglement information from noisy tomography data.","feed_headline":"Neural nets and random forests trust opposite tomography measurements","feed_subtitle":"A neural net relies on coherence data; the forest relies on occupations—and the split vanishes for density-matrix inputs.","key_machinery":"The load-bearing mechanism is the linear scattering of information in two-qubit projective tomography, encoded in the four measurement blocks of Eq. (6). The measurement outcomes are linear functions of the density-matrix elements, Eqs. (7)--(10): $M_A$ gives the diagonal elements directly, $M_B$ and $M_C$ mix diagonals with local coherences, and $M_D$ mixes diagonals, local coherences, and non-local coherences. Because the diagonals must be known to convert $M_B$ and $M_C$ into actual off-diagonal elements, the $M_A$ block functions as a gatekeeper for all coherence information in the measurement representation; this is what the random forest's split-based learning exploits. The neural network, by contrast, learns an internal representation that recovers the direct role of $M_D$. The supporting analytic comparison is the Shapley-value analysis of the concurrence computed from quantum-state reconstruction, which shows the same ranking as the neural network. The disappearance of the contradiction for density-matrix inputs is the control that isolates the scattering mechanism.","core_discovery":"The central discovery is a model-class-dependent reversal in feature importance for entanglement quantification. For two-qubit states labeled by concurrence $C(\\rho)=\\max(0,\\lambda_1-\\lambda_2-\\lambda_3-\\lambda_4)$, the 16 projective measurements split naturally into four blocks defined in Eq. (6): block $M_A$ contains the occupation probabilities $m_{00},m_{01},m_{10},m_{11}$, blocks $M_B$ and $M_C$ contain information about local coherences, and block $M_D$ contains information about non-local coherences, which the paper identifies as critical for entanglement. The neural network, like the analytic quantum-state reconstruction, ranks $M_D$ first, then $M_B,M_C$, then $M_A$; the random forest ranks $M_A$ first and $M_D$ last. The paper traces the inversion to the tomography equations (7)--(10): the diagonal outcomes in $M_A$ enter the reconstruction of every off-diagonal density-matrix element, so the random forest, which builds splits on single features, latches onto $M_A$ as the gatekeeper for all coherence information, while the neural network effectively inverts the linear relations and reads the entanglement-relevant coherences directly. When both models are fed density-matrix elements instead of raw measurements, their importance rankings coincide.","pith_inferences":["A testable extension is to train the same model families on three-qubit or higher-dimensional partial tomography and compare leave-one-block-out retraining; the prediction would be that tree ensembles again shift importance onto directly measured diagonal blocks while networks track coherence-relevant blocks, with the gap widening as Hilbert-space dimension grows.","The same informational-scattering mechanism should appear in any sensor-fusion task where raw features are linear mixtures of latent variables and one class of features is needed to calibrate the others; random forests would be expected to key on the calibrating features, neural networks on the latent-relevant ones.","A practical diagnostic for noisy experiments is to compare model predictions under targeted measurement noise and use disagreement between the two models as a flag that the tomography data are unreliable in a particular block.","Because Shapley values for the random forest differ between separable and strongly entangled samples (block $M_D$ dominating for separable, block $M_A$ for entangled), model confidence could be recalibrated separately for the two regimes, potentially improving low-concurrence classification."],"forward_implications":["On full, clean tomography data, the neural network classifies two-qubit entanglement more accurately than the random forest (accuracy 0.922 versus 0.879 on the test set), so the better learner and the more interpretable learner are not the same model.","When a single measurement is corrupted by noise, the random forest degrades more gracefully at high noise levels than the neural network, because many trees vote over different feature subsets; the neural network is more accurate at low noise.","The block-importance rankings provide a practical criterion for choosing a model: if occupation measurements are the reliable part of a tomography run, the random forest's output should be preferred, while the neural network is the safer predictor when coherence measurements carry the signal.","Feature importance in machine-learning entanglement quantification is not a property of the data alone; it depends on the model family, so interpretability claims should be stated relative to the learner.","The same models trained on density-matrix elements agree on importance, confirming that the discrepancy is caused by the dispersed, interdependent structure of measurement data rather than by different notions of entanglement."],"supporting_citations":[{"why":"Supplies the projective-measurement tomography set and the linear formulas (Eqs. 7--10) that define the four measurement blocks and their information content.","marker":"[1]"},{"why":"Defines random forests and the mean-decrease-in-impurity feature importance that produces the $M_A$-first ranking.","marker":"[21]"},{"why":"Defines concurrence, the entanglement measure used to label all training states and target the models.","marker":"[31]"},{"why":"Supplies the Shapley-value model-agnostic attribution method used to compare feature importance across the neural network, random forest, and quantum-state reconstruction.","marker":"[42]"},{"why":"Supplies the random-state generation approach (quantum circuits) that creates the balanced training and test datasets.","marker":"[11]"},{"why":"Supplies the random forest classifier implementation used for the RF models in the study.","marker":"[46]"}],"fun_headline_variants":["Neural nets and random forests read entanglement from opposite data","Random forests see entanglement in occupations, neural nets in coherences","Why random forests and neural nets rank entanglement measurements oppositely","Study: neural nets and forests disagree on key entanglement measurements","Entanglement learning: neural nets and random forests choose different signals"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The conclusion rests on the assumption that the feature-importance measures, especially the random forest's mean-decrease-in-impurity and the perturbation-based accuracy drops, correctly reveal which measurements the models actually use even though the tomography features are strongly correlated with one another.","fun_headline_variants_meta":{"raw":{"variants":["Neural nets and random forests read entanglement from opposite data","Random forests see entanglement in occupations, neural nets in coherences","Why random forests and neural nets rank entanglement measurements oppositely","Study: neural nets and forests disagree on key entanglement measurements","Entanglement learning: neural nets and random forests choose different signals"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0004,"raw_usage":{"total_tokens":2119,"prompt_tokens":1003,"completion_tokens":1116,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":619,"completion_tokens_details":{"reasoning_tokens":1031}},"tokens_in":619,"tokens_out":1116,"duration_ms":8691,"temperature":1.0,"reasoning_tokens":1031,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:53:54.576570+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain both models from scratch after deleting each of the four measurement blocks in turn (leave-block-out). The paper's claim predicts that removing $M_A$ damages the random forest most while the neural network is damaged most by removing $M_D$; if retraining produces the same damage ranking for both models, the reported contradiction is an artifact of the attribution tools rather than a genuine difference in how the models learn.","supporting_citations":[{"cited_title":"In an RF, feature importance is typically determined by measuring the impact of each feature on the model’s predictive accuracy, e.g","cited_arxiv_id":null,"evidence_quote":"Supplies the projective-measurement tomography set and the linear formulas (Eqs. 7--10) that define the four measurement blocks and their information content."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines random forests and the mean-decrease-in-impurity feature importance that produces the $M_A$-first ranking."},{"cited_title":"Gibney and D","cited_arxiv_id":null,"evidence_quote":"Defines concurrence, the entanglement measure used to label all training states and target the models."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the random forest classifier implementation used for the RF models in the study."}],"review_version":1}