{"id":"ca002c9d-85d6-4644-a7d4-13ac3e760639","arxiv_id":"2501.00623","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":8,"one_line_summary":"A shared-parameter alternating Tweedie model learns word embeddings from cooccurrence counts, with a Fisher scoring update and a decaying learning rate.","lead":"The authors propose fitting Tweedie regression models to word cooccurrence counts to learn dense word vectors, alternating between updating row and column embeddings with a Fisher scoring algorithm. The paper reports better Named Entity Recognition results than GloVe vectors in a simple BiLSTM, but the comparison and underlying assumptions have significant gaps.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 4.1 fits SA-Tweedie to log(co-occurrence+1) while Section 2's likelihood and Section 3's simulation assume raw Tweedie counts; no derivation or experiment bridges this gap, so the NER and Fisher-scoring claims are not actually tested on the model's response.","rationale":"The reader's weakest assumption and my independent reading converge on the same point: the model is derived and simulated for raw Tweedie counts, but Section 4.1 explicitly switches to log counts. This matters because every derivative, information matrix, and p/phi calibration in Section 2 is tied to the Tweedie likelihood, and the zero atom has a specific meaning under the Compound Poisson Gamma model. Applying that likelihood to log(co-occurrence+1) is not a harmless transform: it conflates raw counts 0 and 1, changes the support, and is not covered by the Section 3 simulation. The authors themselves flag related trouble when they find p-hat intervals outside the valid Tweedie range and mention zero-inflated Tweedie as a possibility, yet proceed. The NER result and the convergence comparison therefore do not validate the proposed model for the actual response used. I would not call for a different verdict; the reader's REJECT remains appropriate. I would add that even the BERT comparison in Section 4.2 is not apples-to-apples because SA-Tweedie embeddings are fine-tuned while the quoted feature-based BERT result freezes the pretrained representations, and no code is provided, but the distributional mismatch is the decisive issue. The paper does offer a concrete scalability implementation sketch and a reproducible simulation setup via GloVe-initialized parameters, but those assets do not repair the missing bridge between the simulated raw-count model and the log-count application.","tokens_in":29292,"tokens_out":8616,"duration_ms":87543,"concrete_test":"Re-run the Section 3 simulation exactly, but set the observed response to Z_ij = log(Y_ij + 1) for Y_ij generated from the Tweedie law, and fit Algorithm 1 with the Section 2 likelihood on Z. Then compare the with-lr/no-lr/Adam loss curves (Figures 4-8) and measure recovery of the planted GloVe-based word vectors up to orthogonal rotation. If the convergence ranking or recovery behavior changes materially, the Wikipedia application is not supported by the simulation. As a second check, re-estimate p and phi from Z as in Section 2.2 and verify that the p values used in Algorithm 1 remain in [1,2); if they fall outside, the Tweedie likelihood used for training is outside its stated domain.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing weakness is the mismatch between the response used in the application and the response assumed by the model and simulation. Section 2.1 develops a Compound Poisson Gamma/Tweedie likelihood with P(Y=0)=exp(-lambda) and a positive density for Y>0, and Section 3 generates data directly from that Tweedie law. Section 4.1, however, after reporting raw-count skewness, states: 'we decided to model the log-counts using the alternating Tweedie regression model.' The same score equations, Fisher information, and loss are then applied to Z_ij = log(X_ij+1). No derivation shows that log-transformed weighted counts follow a Tweedie law; in fact the transformation collapses raw counts 0 and 1 to the same value 0, so the zero-atom interpretation changes. The Section 2.2 estimates of p on this log-scale data include intervals with p-hat in (0,1), where the paper itself says the Tweedie distribution is not well defined. The simulation therefore validates the optimizer on a different generative process than the one used for the reported 91% F1 result and the claimed advantage of Fisher scoring with learning-rate adjustment. Unless the model is redefined as a Tweedie GLM on log counts, with new likelihood, p/phi estimation, and simulations, the central claim is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Shared-parameter Alternating Tweedie (SA-Tweedie) model for co-occurrence matrices, in which the log of each co-occurrence count is modeled as the inner product of a row embedding and a column embedding plus row/column biases, with Tweedie noise and a zero mass. The authors derive score functions and Fisher information for the Tweedie likelihood, introduce a learning-rate-adjusted Fisher scoring update, and compare it with plain Fisher scoring and Adam on simulated data. They then apply the method to a Wikipedia corpus with WordPiece tokens, building embeddings that are fed into a simple BiLSTM for CoNLL-2003 named entity recognition, where they report about 91% test weighted F1, above their GloVe and random baselines and above the feature-based BERT embeddings result cited from Devlin et al. (2019). The paper also describes a SQLite-based storage and CPU-GPU pipelining strategy to make the algorithm scalable.","tokens_in":29660,"tokens_out":6592,"duration_ms":66509,"significance":"If the statistical model were correctly specified for the response actually used, the paper would offer a potentially interesting probabilistic alternative to GloVe that can produce useful token embeddings with a comparatively light architecture. The manuscript also demonstrates a nontrivial engineering effort: storing the sparse matrix in SQLite, using rowid ranges, chunked GPU computation with einsum, and CPU-GPU queue-based parallelism. The central claim, however, is not adequately supported because the model fitted to the Wikipedia data is not the model developed in Section 2, and the simulation does not validate the modeling assumptions used in the application. These issues bear directly on the NER result and on the claimed advantage of the proposed optimizer, so the significance of the contribution as stated cannot be affirmed.","major_comments":[{"comment":"The model developed in Section 2 assumes the response Y_ij itself is Tweedie-distributed with mean exp(eta_ij) and zero mass exp(-lambda_ij). Section 4.1, however, states that the authors 'decided to model the log-counts using the alternating Tweedie regression model,' i.e., the response is Z_ij = log(X_ij + 1). No derivation is given for the claim that log-transformed weighted counts follow a Tweedie law; indeed a log transformation changes the zero atom (counts 0 and 1 both map to 0) and alters the support, so the likelihood in equations (2.6)-(2.7) is not the correct likelihood for the data actually fitted. The simulation in Section 3 generates raw Tweedie counts, not log-counts, so the simulation does not validate the application and the NER result does not test the proposed model.","section":"Section 4.1, Eq. (2.6), Section 3"},{"comment":"The estimated power parameters in Table 1 include several intervals with p-hat between 0 and 1 (intervals 1, 2, 3, and 7). The paper itself states on page 20 that for these values 'Tweedie distribution is not well defined' and that a zero-inflated Tweedie distribution may be needed. Yet the subsequent application uses the Tweedie likelihood and score equations with these p values as if they were valid. This means that, for a substantial fraction of the data, the likelihood used for estimation is not the likelihood of any proper distribution, undermining both the estimation procedure and the interpretation of the resulting embeddings.","section":"Section 2.2, Table 1"},{"comment":"The loss used for the convergence criterion and for the loss-curve comparisons is defined as -sum_{i,j}(y_ij*theta_ij - kappa(theta_ij)). This is not the negative log-likelihood of the proposed Tweedie model: for y_ij = 0 the Tweedie log-likelihood contributes -lambda_ij = -mu_ij^(2-p)/(phi_ij(2-p)), whereas the loss contributes kappa(theta_ij) = mu_ij^(2-p)/(2-p), missing the 1/phi factor. The updates in (2.8), by contrast, use the score and Fisher information derived from the actual Tweedie likelihood. The algorithm therefore optimizes a different objective from the one whose score and information matrix it uses, and the reported loss curves do not measure likelihood maximization.","section":"Section 2.1, 'Loss' definition, Eq. (2.8)"},{"comment":"The simulation study only reports training-loss trajectories and score-vector norms; it never checks whether the estimated embedding vectors recover the true vectors used to generate the data. Because the loss is the same objective being optimized, 'outperforms' in the simulation partly reflects the algorithm's progress on its own training criterion rather than statistical estimation accuracy. Moreover, the claimed advantage of the learning-rate-adjusted Fisher scoring over plain Fisher scoring is small (the final losses differ only after the second decimal place in most of the eight replicates), and no standard errors or tests are provided to show the difference is meaningful.","section":"Section 3, Figures 4-8"},{"comment":"The comparison with BERT is not an apples-to-apples evaluation. The BERT numbers reproduced in Table 4 come from Devlin et al. (2019) using cased WordPiece models and a different BiLSTM setup, while the SA-Tweedie experiments lowercase all tokens and use a 512-dimensional BiLSTM. The feature-based BERT row labeled 'Embeddings' reports a validation F1 of 91.0 and no test F1, whereas the paper's statement that SA-Tweedie is 'much more effective than BERT embeddings' is based on this non-comparable setting. The claim is therefore overstated and should be removed or replaced with a matched comparison.","section":"Section 4.2, Tables 3 and 4"}],"minor_comments":[{"comment":"Figure 1 describes a '0 vs positive: Bernoulli' component with a logit link, which suggests a two-stage hurdle model, but the equations in Section 2 do not include such a Bernoulli component and instead use the Tweedie zero mass exp(-lambda). Section 5 also refers to a 'two-stage Bernoulli-Tweedie model.' These descriptions are inconsistent and should be reconciled.","section":"Figure 1"},{"comment":"The infinite sum in equation (2.3) contains an undefined variable x; it should likely read y^{ka_ij - 1}. The notation for the product and the indices elsewhere in the equation also needs cleaning.","section":"Equation (2.3)"},{"comment":"The abstract and introduction mention that the Tweedie model handles 'positive probability mass for zero observations,' but the connection to the actual Compound Poisson Gamma model with p in (1,2) could be stated more precisely, especially since later intervals use p outside that range.","section":"Abstract and Section 1"},{"comment":"The text says '4 Billon tokens' (typo for 'billion') and refers to the 'January 20th, 2022 version of the Wikipedia dump' without specifying the exact preprocessing (e.g., sentence splitting, window size k) used to build the co-occurrence matrix, making the experimental setup only partially reproducible.","section":"Section 4.2"},{"comment":"The convergence claim for the alternating procedure is asserted through an analogy with the generalized EM algorithm, but no formal theorem or proof is given; the two conditions stated are not verified for the SA-Tweedie model. This should either be made rigorous or explicitly labeled as a conjecture.","section":"Section 5"}],"recommendation":"reject","confidential_remarks":"The paper's core problem is that the statistical model developed in Section 2 is not the model fitted in Section 4.1. This is not a presentation-level flaw but a mismatch that invalidates the simulation-application link and the reported NER comparison. The authors would need to redefine the model for log-counts, re-derive the likelihood and score equations, re-run the simulation for that response, and re-estimate p and phi under the new model. Even then, the current evidence that the optimizer advantage is meaningful is weak. Given the scope of the required changes, I do not see a path to acceptance without essentially rewriting the modeling component. The engineering and scalability parts are useful, but they do not rescue the statistical claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the SA-Tweedie model is a genuinely new combination — shared row/column Tweedie power and dispersion parameters, alternating Fisher scoring with a decaying learning rate — and the score-equation derivations are careful. But the load-bearing application is not testing the model. Section 4.1 says raw cooccurrence counts are too skewed, so they fit log(X+1), while Section 2's likelihood and Section 3's simulation are for raw Tweedie counts. No derivation connects log-transformed counts to a Tweedie law, and the transformation collapses 0 and 1 to the same value, changing the zero-atom interpretation. Table 1 then shows estimated p in (0,1) for several intervals, which the paper itself says is not a well-defined Tweedie range; the mention of zero-inflated Tweedie is a pointer, not a fix. So the 91% NER F1 and the Fisher-scoring advantage are at best evidence about a different model than the one proposed and simulated.\n\nWhat the paper does well: the model is original, the Fisher information and score equations are derived in detail, and the simulation is reproducible in principle (GloVe vectors, 8 datasets). The engineering — SQLite row retrieval, chunking, einsum-based information matrices, CPU-GPU pipeline — is a real contribution for anyone wanting to fit such models at scale. I also give credit for being honest about non-identifiability and about p estimates falling outside the valid Tweedie range.\n\nThe soft spots, in proportion: the log-count mismatch is major and load-bearing. The simulation only reports loss curves, not whether the true embedding vectors are recovered, so 'our optimizer wins' is partly tautological. The NER evaluation is thin: three seeds, no code or data release, no comparison to FastText or other non-contextual embeddings, and the BERT comparison is apples-to-oranges — SA-Tweedie's validation score is compared to BERT's feature-based validation, while its test score is compared to fine-tuned BERT's test; fine-tuned BERT still scores higher. The convergence discussion in Section 5 is asserted rather than proved, and the acknowledged orthogonal non-identifiability is not addressed in the algorithm.\n\nBottom line: this deserves a serious referee, not a desk rejection, because the statistical model is original and the central flaw is diagnosable. I would not accept it in current form. A revision needs to either fit raw counts with proper handling of p, or re-derive the Tweedie model for log counts with new simulations, and it needs code/data plus fair baselines. I'd bring it to a reading group to discuss the distributional mismatch, but I wouldn't cite it yet.","headline":"A real statistical proposal — shared-parameter alternating Tweedie with damped Fisher scoring — but the main application fits log counts while the model and simulation assume raw counts, so the headline claims are not actually tested.","tokens_in":30147,"tokens_out":3823,"would_cite":false,"duration_ms":38297,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62-08","62J99"],"pacs":[],"model":"deepseek-v4-flash","headline":"Modeling word co-occurrence counts with a Tweedie distribution yields embeddings that beat GloVe and rival feature-based BERT on named entity recognition.","keywords":["word embedding","Tweedie distribution","high-dimensional co-occurrence matrix","matrix factorization","named entity recognition","Fisher scoring","Adam","recommender systems"],"falsifier":"Fit a comparison embedding method — GloVe's weighted least squares, or the same Tweedie loss with the power fixed at $p=1$ — to the exact same distance-weighted Wikipedia WordPiece counts used for SA-Tweedie, and run both sets of vectors through the identical BiLSTM with the same seeds; if the test F1 gap against 91% collapses, the improvement comes from the counts and preprocessing rather than from the Compound Poisson Gamma likelihood. The distributional premise can be checked in the same pass: bin held-out rows of $\\log(X_{ij}+1)$ by fitted mean and compare empirical variance to the assumed $\\phi\\mu^p$; the paper's own Table 1 already finds estimated $p$ values between 0 and 1 on several mean intervals, where the Tweedie family is undefined, so an explicit mean-variance check on the log scale would settle whether the likelihood is misspecified.","tokens_in":29048,"feed_emoji":"🔤","tokens_out":18792,"duration_ms":154440,"temperature":0.7,"pith_summary":"This paper proposes the Shared parameter Alternating Tweedie (SA-Tweedie) model, a probabilistic treatment of sparse co-occurrence data — word pairs in text, or user-item pairs on shopping platforms — that produces dense vector representations without any observed covariates. The model assumes each entry follows a Tweedie distribution, the family that includes the Compound Poisson Gamma law with positive probability mass exactly at zero, and writes the log-mean as a dot product of learned vectors plus per-token biases, with the power and dispersion parameters shared additively and multiplicatively across the two sides of the matrix. The paper's central empirical claim is that 300-dimensional embeddings fitted to log-weighted token counts from a 4-billion-token Wikipedia dump lift a simple BiLSTM to about 91% weighted F1 on CoNLL-2003 named entity recognition, above the roughly 88% of GloVe in the identical architecture and above the validation F1 that a feature-based BERT setup reports. A second claim is algorithmic: alternating Fisher scoring whose step size shrinks as $t^{-1/4}$ converges to a lower training loss than plain Fisher scoring, which can stall, and than Adam, which descends more slowly. The paper also reports that pseudo-likelihood estimation fails for this shared-parameter setup because no covariates are observed.","feed_headline":"Tweedie word vectors hit 91% F1, beating GloVe","feed_subtitle":"A probabilistic model of co-occurrence beats GloVe while training on a tenth of the text.","key_machinery":"The load-bearing object is the shared-parameter alternating Tweedie regression: a two-part likelihood in which zero entries contribute their exact probability $\\Pr[Y_{ij}=0]=\\exp(-\\lambda_{ij})$ with $\\lambda_{ij}=\\mu_{ij}^{2-p_{ij}}/(\\phi_{ij}(2-p_{ij}))$, and positive entries contribute the Tweedie exponential-family density whose mean-variance relation is $\\mathrm{Var}(Y_{ij})=\\phi_{ij}\\mu_{ij}^{p_{ij}}$. Information is shared across the whole matrix by factoring the power and dispersion parameters as $p_{ij}=p_i+\\tilde{p}_j$ and $\\phi_{ij}=\\phi_i\\tilde{\\phi}_j$, so each row and each column carries its own latent scalar while the vectors $w_i$ and $\\tilde{w}_j$ carry the geometry. Estimation alternates between two Fisher-scoring passes: with all column vectors held fixed, each row block $(w_i,b_i)$ is updated by adding $I_{\\beta_i}^{-1}U_{\\beta_i}$ scaled by $\\text{lr}/t^{1/4}$, and then the symmetric pass updates columns with rows held fixed; the information matrices are block-diagonal, so the row updates separate cleanly and each row's data can be retrieved once. The normalizing term $c(y_{ij},\\phi_{ij},p_{ij})$ in the Tweedie density is dropped from the loss because it does not depend on the regression parameters, leaving the deviance-like loss $-\\sum_{ij}(y_{ij}\\theta_{ij}-\\kappa(\\theta_{ij}))$. The power and dispersion values themselves are set before fitting by regressing the log sample variance on the log sample mean over the data matrix and reading off slope $\\hat{p}$ and intercept $\\hat{\\delta}=\\log\\hat{\\phi}$.","core_discovery":"The central claim is that a full generative model of the co-occurrence matrix — with $\\log \\mu_{ij} = w_i^\\top \\tilde{w}_j + b_i + \\tilde{b}_j$, zero probability $\\Pr[Y_{ij}=0]=\\exp(-\\lambda_{ij})$, and shared parameters $p_{ij}=p_i+\\tilde{p}_j$, $\\phi_{ij}=\\phi_i\\tilde{\\phi}_j$ — can be estimated at Wikipedia scale and yields embeddings that outperform standard alternatives in a lightweight downstream model. Fitted to $\\log(X_{ij}+1)$ of distance-weighted WordPiece co-occurrences from the January 2022 Wikipedia dump (about 4 billion tokens, 26,531 token types), 300-dimensional SA-Tweedie vectors give a simple BiLSTM about 95% weighted F1 on the CoNLL-2003 validation set and 91% on the test set, versus roughly 88% for GloVe under the same architecture and seeds; the paper reads this as more effective than BERT's feature-based embeddings, whose reported validation F1 is 91%. The paper further claims that the estimation scheme is decisive: Fisher scoring with a learning-rate factor $\\text{lr}/t^{1/4}$ reaches a lower final training loss than unscaled Fisher scoring, which can stall and occasionally jump to a much worse solution, and than Adam, which reduces loss more slowly both within epochs and across iterations; nine simulated raw-Tweedie datasets support the comparison. Taken together, the claims position SA-Tweedie as a resource-efficient route to global word representations that rival much larger contextual models when used as features.","pith_inferences":["The NER gain may owe more to the variance weighting implicit in the deviance loss, which down-weights high-mean pairs, than to the zero-mass Tweedie form; retraining GloVe's weighted least squares on the identical Wikipedia counts would isolate how much of the improvement is the model rather than the counts and preprocessing.","Because the paper validates the likelihood only on raw counts while fitting $\\log(X+1)$, the NER number is the strongest evidence for the model; the paper's own Table 1, with fitted $p$ between 0 and 1 on several intervals, hints that a zero-inflated Tweedie or another log-scale family might fit the actual response better.","The $t^{-1/4}$ schedule is a heuristic whose practical role is likely stabilization under a non-stationary alternating objective; comparing other damped step rules (line search, cosine decay, per-coordinate scaling) on the same simulated data would show how much the specific choice matters.","The additive $p_{ij}=p_i+\\tilde{p}_j$ factorization is motivated by symmetric item-item matrices; user-item matrices are asymmetric and far sparser, so applying the model to recommender data will likely require separate treatment of row and column power parameters."],"forward_implications":["Feature-based use of SA-Tweedie embeddings (91% test F1 with a one-layer BiLSTM) lands close to fine-tuned BERT (92.4–92.8 test F1) while using none of BERT's 110–345 million pretrained parameters.","The objective is corpus-efficient: a single 4-billion-token Wikipedia dump outperforms GloVe's 42-billion-token vectors in the identical downstream architecture, implying more signal extracted per token.","The estimation recipe transfers: Fisher scoring with a $t^{-1/4}$ decaying learning rate provides reliable convergence for alternating dot-product models, avoiding Adam's slow descent and unscaled Fisher scoring's late-iteration stalls.","The same likelihood machinery applies to any sparse two-sided count matrix, such as user-item interaction counts for recommender systems, as long as the additive row/column sharing of $p$ and the multiplicative sharing of $\\phi$ are appropriate.","Pseudo-likelihood is a closed path for this problem: with no covariates observed, the surrogate conditional distributions push the alternating updates in wrong directions, so full-likelihood or alternating full-likelihood estimation is required."],"supporting_citations":[{"why":"GloVe; supplies the pretrained 42-billion-token embeddings that serve as the main NER baseline (about 88% test F1) and the word vectors used to initialize the simulation.","marker":"Pennington et al. (2014)"},{"why":"BERT; provides the WordPiece tokenizer used to build the Wikipedia co-occurrence counts and the reported F1 table that the paper uses as its comparison point.","marker":"Devlin et al. (2019)"},{"why":"Supplies the Tweedie regression parameterization, including $\\lambda=\\mu^{2-p}/(\\phi(2-p))$ and the profile-likelihood approach to estimating $p$ and $\\phi$.","marker":"Bonat and Kokonendji (2017)"},{"why":"The CRFjoin setup whose reported F1 the paper uses to confirm that its GloVe baseline matches published numbers.","marker":"Wang and Manning (2013)"},{"why":"The EM algorithm theory invoked to justify convergence of the alternating procedure, treating the unobserved column vectors as missing data.","marker":"Dempster et al., 1977"},{"why":"The WordPiece tokenizer that defines the 26,531-token vocabulary of the Wikipedia co-occurrence matrix.","marker":"Wu et al. 2016"},{"why":"The dissertation containing the pseudo-likelihood derivation and experiments that the paper summarizes as a negative result.","marker":"Kim (2023)"}],"fun_headline_variants":["Tweedie co-occurrence model tops GloVe on NER with 91% F1","SA-Tweedie vectors: 91% F1 on NER, beat GloVe and BERT features","Shared-parameter Tweedie outshines GloVe with 10x less data","Fisher scoring trumps Adam for Tweedie co-occurrence fitting","Tweedie embeddings beat BERT features with a tenth the corpus"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the log-transformed co-occurrence responses actually follow the Tweedie law being fitted, a premise the paper never checks on its real data because its simulations validate the likelihood only on raw Tweedie counts, and if that premise fails the likelihood is misspecified and the NER result is not a clean test of the model.","fun_headline_variants_meta":{"raw":{"variants":["Tweedie co-occurrence model tops GloVe on NER with 91% F1","SA-Tweedie vectors: 91% F1 on NER, beat GloVe and BERT features","Shared-parameter Tweedie outshines GloVe with 10x less data","Fisher scoring trumps Adam for Tweedie co-occurrence fitting","Tweedie embeddings beat BERT features with a tenth the corpus"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000948,"raw_usage":{"total_tokens":4138,"prompt_tokens":1125,"completion_tokens":3013,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":741,"completion_tokens_details":{"reasoning_tokens":2900}},"tokens_in":741,"tokens_out":3013,"duration_ms":21918,"temperature":1.0,"reasoning_tokens":2900,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:46:39.017077+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Fit a comparison embedding method — GloVe's weighted least squares, or the same Tweedie loss with the power fixed at $p=1$ — to the exact same distance-weighted Wikipedia WordPiece counts used for SA-Tweedie, and run both sets of vectors through the identical BiLSTM with the same seeds; if the test F1 gap against 91% collapses, the improvement comes from the counts and preprocessing rather than from the Compound Poisson Gamma likelihood. The distributional premise can be checked in the same pass: bin held-out rows of $\\log(X_{ij}+1)$ by fitted mean and compare empirical variance to the assumed $\\phi\\mu^p$; the paper's own Table 1 already finds estimated $p$ values between 0 and 1 on several mean intervals, where the Tweedie family is undefined, so an explicit mean-variance check on the log scale would settle whether the likelihood is misspecified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GloVe; supplies the pretrained 42-billion-token embeddings that serve as the main NER baseline (about 88% test F1) and the word vectors used to initialize the simulation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"BERT; provides the WordPiece tokenizer used to build the Wikipedia co-occurrence counts and the reported F1 table that the paper uses as its comparison point."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Tweedie regression parameterization, including $\\lambda=\\mu^{2-p}/(\\phi(2-p))$ and the profile-likelihood approach to estimating $p$ and $\\phi$."},{"cited_title":"and Manning, C","cited_arxiv_id":null,"evidence_quote":"The CRFjoin setup whose reported F1 the paper uses to confirm that its GloVe baseline matches published numbers."},{"cited_title":"P., Laird, N","cited_arxiv_id":null,"evidence_quote":"The EM algorithm theory invoked to justify convergence of the alternating procedure, treating the unobserved column vectors as missing data."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The dissertation containing the pseudo-likelihood derivation and experiments that the paper summarizes as a negative result."}],"review_version":1}