Pith. sign in

REVIEW 5 major objections 5 minor 17 references

Global dense vector representations for words or items using shared parameter alternating Tweedie model

T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Modeling word co-occurrence counts with a Tweedie distribution yields embeddings that beat GloVe and rival feature-based BERT on named entity recognition.

desk verdict A real statistical proposal — shared-parameter alternating Tweedie with damped Fisher scoring — but the main application fits log counts while the model and simulation assume raw counts, so the headline claims are not actually tested. read the letter →

arxiv 2501.00623 v1 pith:3ZNRQFJT submitted 2024-12-31 cs.LG stat.ML

Taejoon Kim , Haiyan Wang This is my paper · ORCID
classification cs.LGstat.ML MSC 62-0862J99
keywords wordembeddingTweediedistributionhigh-dimensionalco-occurrencematrixfactorizationnamedentityrecognitionFisherscoringAdamrecommendersystems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes the Shared parameter Alternating Tweedie (SA-Tweedie) model, a probabilistic treatment of sparse co-occurrence data — word pairs in text, or user-item pairs on shopping platforms — that produces dense vector representations without any observed covariates. The model assumes each entry follows a Tweedie distribution, the family that includes the Compound Poisson Gamma law with positive probability mass exactly at zero, and writes the log-mean as a dot product of learned vectors plus per-token biases, with the power and dispersion parameters shared additively and multiplicatively across the two sides of the matrix. The paper's central empirical claim is that 300-dimensional embeddings fitted to log-weighted token counts from a 4-billion-token Wikipedia dump lift a simple BiLSTM to about 91% weighted F1 on CoNLL-2003 named entity recognition, above the roughly 88% of GloVe in the identical architecture and above the validation F1 that a feature-based BERT setup reports. A second claim is algorithmic: alternating Fisher scoring whose step size shrinks as $t^{-1/4}$ converges to a lower training loss than plain Fisher scoring, which can stall, and than Adam, which descends more slowly. The paper also reports that pseudo-likelihood estimation fails for this shared-parameter setup because no covariates are observed.

What carries the argument

The load-bearing object is the shared-parameter alternating Tweedie regression: a two-part likelihood in which zero entries contribute their exact probability $\Pr[Y_{ij}=0]=\exp(-\lambda_{ij})$ with $\lambda_{ij}=\mu_{ij}^{2-p_{ij}}/(\phi_{ij}(2-p_{ij}))$, and positive entries contribute the Tweedie exponential-family density whose mean-variance relation is $\mathrm{Var}(Y_{ij})=\phi_{ij}\mu_{ij}^{p_{ij}}$. Information is shared across the whole matrix by factoring the power and dispersion parameters as $p_{ij}=p_i+\tilde{p}_j$ and $\phi_{ij}=\phi_i\tilde{\phi}_j$, so each row and each column carries its own latent scalar while the vectors $w_i$ and $\tilde{w}_j$ carry the geometry. Estimation alternates between two Fisher-scoring passes: with all column vectors held fixed, each row block $(w_i,b_i)$ is updated by adding $I_{\beta_i}^{-1}U_{\beta_i}$ scaled by $\text{lr}/t^{1/4}$, and then the symmetric pass updates columns with rows held fixed; the information matrices are block-diagonal, so the row updates separate cleanly and each row's data can be retrieved once. The normalizing term $c(y_{ij},\phi_{ij},p_{ij})$ in the Tweedie density is dropped from the loss because it does not depend on the regression parameters, leaving the deviance-like loss $-\sum_{ij}(y_{ij}\theta_{ij}-\kappa(\theta_{ij}))$. The power and dispersion values themselves are set before fitting by regressing the log sample variance on the log sample mean over the data matrix and reading off slope $\hat{p}$ and intercept $\hat{\delta}=\log\hat{\phi}$.

What would settle it

Fit a comparison embedding method — GloVe's weighted least squares, or the same Tweedie loss with the power fixed at $p=1$ — to the exact same distance-weighted Wikipedia WordPiece counts used for SA-Tweedie, and run both sets of vectors through the identical BiLSTM with the same seeds; if the test F1 gap against 91% collapses, the improvement comes from the counts and preprocessing rather than from the Compound Poisson Gamma likelihood. The distributional premise can be checked in the same pass: bin held-out rows of $\log(X_{ij}+1)$ by fitted mean and compare empirical variance to the assumed $\phi\mu^p$; the paper's own Table 1 already finds estimated $p$ values between 0 and 1 on several mean intervals, where the Tweedie family is undefined, so an explicit mean-variance check on the log scale would settle whether the likelihood is misspecified.

Watch

Extended reading notes

Core claim

The central claim is that a full generative model of the co-occurrence matrix — with $\log \mu_{ij} = w_i^\top \tilde{w}_j + b_i + \tilde{b}_j$, zero probability $\Pr[Y_{ij}=0]=\exp(-\lambda_{ij})$, and shared parameters $p_{ij}=p_i+\tilde{p}_j$, $\phi_{ij}=\phi_i\tilde{\phi}_j$ — can be estimated at Wikipedia scale and yields embeddings that outperform standard alternatives in a lightweight downstream model. Fitted to $\log(X_{ij}+1)$ of distance-weighted WordPiece co-occurrences from the January 2022 Wikipedia dump (about 4 billion tokens, 26,531 token types), 300-dimensional SA-Tweedie vectors give a simple BiLSTM about 95% weighted F1 on the CoNLL-2003 validation set and 91% on the test set, versus roughly 88% for GloVe under the same architecture and seeds; the paper reads this as more effective than BERT's feature-based embeddings, whose reported validation F1 is 91%. The paper further claims that the estimation scheme is decisive: Fisher scoring with a learning-rate factor $\text{lr}/t^{1/4}$ reaches a lower final training loss than unscaled Fisher scoring, which can stall and occasionally jump to a much worse solution, and than Adam, which reduces loss more slowly both within epochs and across iterations; nine simulated raw-Tweedie datasets support the comparison. Taken together, the claims position SA-Tweedie as a resource-efficient route to global word representations that rival much larger contextual models when used as features.

Load-bearing premise

The load-bearing premise is that the log-transformed co-occurrence responses actually follow the Tweedie law being fitted, a premise the paper never checks on its real data because its simulations validate the likelihood only on raw Tweedie counts, and if that premise fails the likelihood is misspecified and the NER result is not a clean test of the model.

Editorial extensions

If this is right

  • Feature-based use of SA-Tweedie embeddings (91% test F1 with a one-layer BiLSTM) lands close to fine-tuned BERT (92.4–92.8 test F1) while using none of BERT's 110–345 million pretrained parameters.
  • The objective is corpus-efficient: a single 4-billion-token Wikipedia dump outperforms GloVe's 42-billion-token vectors in the identical downstream architecture, implying more signal extracted per token.
  • The estimation recipe transfers: Fisher scoring with a $t^{-1/4}$ decaying learning rate provides reliable convergence for alternating dot-product models, avoiding Adam's slow descent and unscaled Fisher scoring's late-iteration stalls.
  • The same likelihood machinery applies to any sparse two-sided count matrix, such as user-item interaction counts for recommender systems, as long as the additive row/column sharing of $p$ and the multiplicative sharing of $\phi$ are appropriate.
  • Pseudo-likelihood is a closed path for this problem: with no covariates observed, the surrogate conditional distributions push the alternating updates in wrong directions, so full-likelihood or alternating full-likelihood estimation is required.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The NER gain may owe more to the variance weighting implicit in the deviance loss, which down-weights high-mean pairs, than to the zero-mass Tweedie form; retraining GloVe's weighted least squares on the identical Wikipedia counts would isolate how much of the improvement is the model rather than the counts and preprocessing.
  • Because the paper validates the likelihood only on raw counts while fitting $\log(X+1)$, the NER number is the strongest evidence for the model; the paper's own Table 1, with fitted $p$ between 0 and 1 on several intervals, hints that a zero-inflated Tweedie or another log-scale family might fit the actual response better.
  • The $t^{-1/4}$ schedule is a heuristic whose practical role is likely stabilization under a non-stationary alternating objective; comparing other damped step rules (line search, cosine decay, per-coordinate scaling) on the same simulated data would show how much the specific choice matters.
  • The additive $p_{ij}=p_i+\tilde{p}_j$ factorization is motivated by symmetric item-item matrices; user-item matrices are asymmetric and far sparser, so applying the model to recommender data will likely require separate treatment of row and column power parameters.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a Shared-parameter Alternating Tweedie (SA-Tweedie) model for co-occurrence matrices, in which the log of each co-occurrence count is modeled as the inner product of a row embedding and a column embedding plus row/column biases, with Tweedie noise and a zero mass. The authors derive score functions and Fisher information for the Tweedie likelihood, introduce a learning-rate-adjusted Fisher scoring update, and compare it with plain Fisher scoring and Adam on simulated data. They then apply the method to a Wikipedia corpus with WordPiece tokens, building embeddings that are fed into a simple BiLSTM for CoNLL-2003 named entity recognition, where they report about 91% test weighted F1, above their GloVe and random baselines and above the feature-based BERT embeddings result cited from Devlin et al. (2019). The paper also describes a SQLite-based storage and CPU-GPU pipelining strategy to make the algorithm scalable.

Significance. If the statistical model were correctly specified for the response actually used, the paper would offer a potentially interesting probabilistic alternative to GloVe that can produce useful token embeddings with a comparatively light architecture. The manuscript also demonstrates a nontrivial engineering effort: storing the sparse matrix in SQLite, using rowid ranges, chunked GPU computation with einsum, and CPU-GPU queue-based parallelism. The central claim, however, is not adequately supported because the model fitted to the Wikipedia data is not the model developed in Section 2, and the simulation does not validate the modeling assumptions used in the application. These issues bear directly on the NER result and on the claimed advantage of the proposed optimizer, so the significance of the contribution as stated cannot be affirmed.

major comments (5)
  1. [Section 4.1, Eq. (2.6), Section 3] The model developed in Section 2 assumes the response Y_ij itself is Tweedie-distributed with mean exp(eta_ij) and zero mass exp(-lambda_ij). Section 4.1, however, states that the authors 'decided to model the log-counts using the alternating Tweedie regression model,' i.e., the response is Z_ij = log(X_ij + 1). No derivation is given for the claim that log-transformed weighted counts follow a Tweedie law; indeed a log transformation changes the zero atom (counts 0 and 1 both map to 0) and alters the support, so the likelihood in equations (2.6)-(2.7) is not the correct likelihood for the data actually fitted. The simulation in Section 3 generates raw Tweedie counts, not log-counts, so the simulation does not validate the application and the NER result does not test the proposed model.
  2. [Section 2.2, Table 1] The estimated power parameters in Table 1 include several intervals with p-hat between 0 and 1 (intervals 1, 2, 3, and 7). The paper itself states on page 20 that for these values 'Tweedie distribution is not well defined' and that a zero-inflated Tweedie distribution may be needed. Yet the subsequent application uses the Tweedie likelihood and score equations with these p values as if they were valid. This means that, for a substantial fraction of the data, the likelihood used for estimation is not the likelihood of any proper distribution, undermining both the estimation procedure and the interpretation of the resulting embeddings.
  3. [Section 2.1, 'Loss' definition, Eq. (2.8)] The loss used for the convergence criterion and for the loss-curve comparisons is defined as -sum_{i,j}(y_ij*theta_ij - kappa(theta_ij)). This is not the negative log-likelihood of the proposed Tweedie model: for y_ij = 0 the Tweedie log-likelihood contributes -lambda_ij = -mu_ij^(2-p)/(phi_ij(2-p)), whereas the loss contributes kappa(theta_ij) = mu_ij^(2-p)/(2-p), missing the 1/phi factor. The updates in (2.8), by contrast, use the score and Fisher information derived from the actual Tweedie likelihood. The algorithm therefore optimizes a different objective from the one whose score and information matrix it uses, and the reported loss curves do not measure likelihood maximization.
  4. [Section 3, Figures 4-8] The simulation study only reports training-loss trajectories and score-vector norms; it never checks whether the estimated embedding vectors recover the true vectors used to generate the data. Because the loss is the same objective being optimized, 'outperforms' in the simulation partly reflects the algorithm's progress on its own training criterion rather than statistical estimation accuracy. Moreover, the claimed advantage of the learning-rate-adjusted Fisher scoring over plain Fisher scoring is small (the final losses differ only after the second decimal place in most of the eight replicates), and no standard errors or tests are provided to show the difference is meaningful.
  5. [Section 4.2, Tables 3 and 4] The comparison with BERT is not an apples-to-apples evaluation. The BERT numbers reproduced in Table 4 come from Devlin et al. (2019) using cased WordPiece models and a different BiLSTM setup, while the SA-Tweedie experiments lowercase all tokens and use a 512-dimensional BiLSTM. The feature-based BERT row labeled 'Embeddings' reports a validation F1 of 91.0 and no test F1, whereas the paper's statement that SA-Tweedie is 'much more effective than BERT embeddings' is based on this non-comparable setting. The claim is therefore overstated and should be removed or replaced with a matched comparison.
minor comments (5)
  1. [Figure 1] Figure 1 describes a '0 vs positive: Bernoulli' component with a logit link, which suggests a two-stage hurdle model, but the equations in Section 2 do not include such a Bernoulli component and instead use the Tweedie zero mass exp(-lambda). Section 5 also refers to a 'two-stage Bernoulli-Tweedie model.' These descriptions are inconsistent and should be reconciled.
  2. [Equation (2.3)] The infinite sum in equation (2.3) contains an undefined variable x; it should likely read y^{ka_ij - 1}. The notation for the product and the indices elsewhere in the equation also needs cleaning.
  3. [Abstract and Section 1] The abstract and introduction mention that the Tweedie model handles 'positive probability mass for zero observations,' but the connection to the actual Compound Poisson Gamma model with p in (1,2) could be stated more precisely, especially since later intervals use p outside that range.
  4. [Section 4.2] The text says '4 Billon tokens' (typo for 'billion') and refers to the 'January 20th, 2022 version of the Wikipedia dump' without specifying the exact preprocessing (e.g., sentence splitting, window size k) used to build the co-occurrence matrix, making the experimental setup only partially reproducible.
  5. [Section 5] The convergence claim for the alternating procedure is asserted through an analogy with the generalized EM algorithm, but no formal theorem or proof is given; the two conditions stated are not verified for the SA-Tweedie model. This should either be made rigorous or explicitly labeled as a conjecture.

Circularity Check

0 steps flagged · score 2.0 of 10

No substantive circularity: the NER benchmark is external, and the optimizer comparison is an optimization-loss evaluation; at most a minor non-load-bearing self-citation.

full rationale

The paper's main external claim, the 91% weighted F1 on CoNLL-2003, is tested with a separate BiLSTM trained on annotated labels; the SA-Tweedie embeddings are learned unsupervised from Wikipedia co-occurrence data, so the downstream benchmark does not reduce to the training loss. The simulation comparison of Fisher scoring variants and Adam is measured on the same negative Tweedie deviance these updates optimize; that is a legitimate optimizer benchmark rather than a prediction of held-out data, so it does not make the central claims circular. The Section 4.1 decision to apply the Tweedie likelihood to log(X+1) after deriving it for raw counts (and after Section 3 simulated raw counts) is a distributional assumption gap that the paper itself flags when estimating p in intervals where the Tweedie law is not well defined (Section 2.2, Table 1); this is a correctness and misspecification risk, not a circular reduction, because the loss and score equations remain well-defined functions of the chosen response and the NER evaluation is independent. The only self-citation, Kim (2023), appears in Section 5 as a pointer for omitted pseudo-likelihood details; the pseudo-likelihood discussion is argued from non-identifiability and surrogate-distribution considerations, and it is not load-bearing for the SA-Tweedie or NER claims. No uniqueness theorem or ansatz is imported from prior work by the same authors; p and phi estimates come from the data's mean-variance relationship. Overall, the derivation chain is not circular; the low score reflects only a minor non-load-bearing self-citation.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The model relies on a Tweedie distributional assumption for log-transformed counts, fixed p and phi estimated from the same data, and an unproved convergence argument. No new physical entities are introduced. The dot product parameterization has a known non-identifiability that is acknowledged only in the pseudo-likelihood discussion.

free parameters (8)
  • Tweedie power p per interval = 0.485 to 1.230 in Table 1
    Estimated by piecewise linear regression of log sample variance on log sample mean from the Wikipedia cooccurrence data, then fixed as known in the likelihood.
  • Dispersion intercept delta = log(phi) per interval = -1.483 to 3.030 in Table 1
    Same piecewise regression fit on the same data; treated as known in subsequent model fitting.
  • Initial learning rate lr = 0.5
    Chosen by hand to control the Fisher scoring step size; no tuning procedure is described.
  • Learning rate decay exponent = 1/4
    Introduced ad hoc in equation (2.10) to shrink update size as iterations grow; no justification or sensitivity analysis.
  • Convergence threshold epsilon and maxit = 1e-4 and 100
    Chosen by hand for the relative loss convergence criterion in equations (2.9) and (4.2).
  • Number of chunks per row = 30, 10, 4 for dimensions 300, 100, 50
    Selected through trial and error to balance GPU memory and computation time.
  • Embedding dimension d = 100 and 300
    Two values are used in the Wikipedia experiment; no model selection criterion is given.
  • Cooccurrence window size k = not specified
    Equation (4.1) defines weighted counts using a window threshold k, but the value of k used in the experiments is never reported.
assumptions (5)
  • domain assumption The log-transformed cooccurrence count log(X_ij + 1) follows a Tweedie distribution with log link.
    Section 4.1 models log counts with the Tweedie likelihood from Section 2, which was derived for raw counts. No distributional justification is given for the log transform.
  • ad hoc to paper The Tweedie power parameter decomposes as p_ij = p_i + p_j and the dispersion as phi_ij = phi_i * phi_j.
    Introduced in Section 2 to reduce the parameter count for symmetric matrices. No empirical or theoretical justification is provided for these particular decompositions.
  • domain assumption The dispersion and power parameters p and phi can be treated as known after a piecewise linear mean-variance regression on the same data.
    Section 2.2 estimates p and phi from the data and then fixes them, ignoring estimation uncertainty and the fact that the loss function is not comparable across different p values.
  • ad hoc to paper The per-block Fisher information matrices used in Algorithm 1 are invertible at every iteration.
    Algorithm 1 requires I^{-1} to exist. Because the dot product parameterization is invariant to orthogonal transformations, the full likelihood is not identifiable and rank deficiency is possible, but the paper does not address this.
  • ad hoc to paper The alternating update converges because the log likelihood is bounded and sufficient statistics are linear in the data, as in the EM algorithm.
    Section 5 asserts these conditions are met for SA-Tweedie, but no theorem or proof is given. Treating the other embedding matrix as missing data with a distribution is not formalized.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Global dense vector representations for words or items using shared parameter alternating Tweedie model." pith.science (2026). https://pith.science/paper/3ZNRQFJT

@misc{pith2026250100623,
  author       = {Pith},
  title        = {Pith review of: Global dense vector representations for words or items using shared parameter alternating Tweedie model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3ZNRQFJT}},
  note         = {Machine review of arXiv:2501.00623}
}
read the original abstract

In this article, we present a model for analyzing the cooccurrence count data derived from practical fields such as user-item or item-item data from online shopping platform, cooccurring word-word pairs in sequences of texts. Such data contain important information for developing recommender systems or studying relevance of items or words from non-numerical sources. Different from traditional regression models, there are no observations for covariates. Additionally, the cooccurrence matrix is typically of so high dimension that it does not fit into a computer's memory for modeling. We extract numerical data by defining windows of cooccurrence using weighted count on the continuous scale. Positive probability mass is allowed for zero observations. We present Shared parameter Alternating Tweedie (SA-Tweedie) model and an algorithm to estimate the parameters. We introduce a learning rate adjustment used along with the Fisher scoring method in the inner loop to help the algorithm stay on track of optimizing direction. Gradient descent with Adam update was also considered as an alternative method for the estimation. Simulation studies and an application showed that our algorithm with Fisher scoring and learning rate adjustment outperforms the other two methods. Pseudo-likelihood approach with alternating parameter update was also studied. Numerical studies showed that the pseudo-likelihood approach is not suitable in our shared parameter alternating regression models with unobserved covariates.

Figures

Figures reproduced from arXiv: 2501.00623 by the authors.

Figure 1
Figure 1. Illustration of model input and desired output. Left panel: Model input - the natural log of (weighted occurrence count +1) matrix for top 300 words from Reuter Business news data. Right panel: Shared parameter Tweedie modeling process and output where λij = µ 2−pij ij ϕij (2 − pij ) (see page 5 of Bonat and Kokonendji 2017) For yij > 0 case, f (yij ) = exp  yijθij − κ (θij ) ϕij + c (yij , ϕij , pij )  [PITH_FUL… view at source ↗
Figure 2
Figure 2. Computed log(loss) and log(overall loss) from simulated dataset using the Fisher scoring with or without learning rate adjustment, and gradient descent algorithm with Adam method for parameter update. The left panel depicts how the loss changes over 10 epochs for one row of the parameter update. As epoch number grows, the loss has a general decreasing trend but the Adam’s loss has higher values and reduces slower th… view at source ↗
Figure 4
Figure 4. The loss reduction was compared within epochs among three different updates: the [PITH_FULL_IMAGE:figures/full_fig_p021_4.png] view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: The overall loss over iterations among three different update methods: with or without [PITH_FULL_IMAGE:figures/full_fig_p023_5.png]
Figure 6
Figure 6. Figure 6: The log10 scaled norm of the score vector and the overall loss as iteration proceeds from simulated data. The top panel shows norm of the score vector in log10 scale for two cases: with learning rate, and without learning rate. The bottom panel illustrates log10 overal…
Figure 7
Figure 7. Figure 7: The overall loss in log10 scale during iteration between 110 and 170 for the two cases: with learning rate, and without learning rate. The update without learning rate adjustment is stabilized in a certain value before reaching the minimum overall loss. The algorithm w…
Figure 8
Figure 8. Figure 8: Comparing performance of the alternating Tweedie regression algorithm with or without learning rate over 8 simulated datasets. Each row is for one dataset [PITH_FULL_IMAGE:figures/full_fig_p026_8.png]
Figure 9
Figure 9. Figure 9: Histogram of skewness for each row in raw co-occurrence count matrix (left panel) and the log co-occurrence count (right panel) constructed from Wikipedia dump. We use relative convergence criteria on the loss function by checking if the magnitude of change in the loss…
Figure 10
Figure 10. Figure 10: Trajectory of the training process of SA-Tweedie. Two embedding dimensions (100 and 300) are considered. The model achieved lower loss with higher embedding dimension. 4.2. Application to NER task on CoNLL-2003 data In this section, we consider Named Entity Recognitio…
Figure 11
Figure 11. Figure 11: shows the comparison of performance from all three embedding methods. For the NER task, the SA-Tweedie embedding gives the best results in that the overall F1-weighted scores are consistently higher than those from GloVe and random embedding. This pattern can be seen …
Figure 12
Figure 12. Figure 12: Training and validation loss along with training and validation weighted F1 score for 15 epochs. Top row: random embedding. Middle row: GloVe embedding. Bottom row: SA￾Tweedie embedding. All parameter initialization used identical global seed 42. The loss and weighted…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 14 canonical work pages

  1. [1]

    Bonat, W. H. and Kokonendji, C. C. (2017). Flexible tweedie regression models for continuous data. Journal of Statistical Computation and Simulation, 87(11):2138–2152

    work page 2017

  2. [2]

    Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. (2020). Language models are few-shot learners. In

    work page 2020

  3. [3]

    Curran Associates, Inc

    Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H., editors, Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc

    work page 1901

  4. [4]

    P., Laird, N

    Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977). Maximum likelihood from in- complete data via the em algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39(1):1–22

    work page 1977

  5. [5]

    Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In North American Chapter of the Association for Computational Linguistics

    work page 2019

  6. [6]

    and Karmaker, S

    Freestone, M. and Karmaker, S. S. (2024). Word embeddings revisited: Do llms offer some- thing new? ArXiv, abs/2402.11094

    arXiv 2024

  7. [7]

    Kim, T. (2023). Matrix factorization and prediction for high dimensional zero inflated co- occurrence count data via shared parameter alternating generalized linear regression with application in nlp. PhD Dissertation, Kansas State University

    work page 2023

  8. [8]

    Snyder, W., Chen, K., Kakade, S., Jain, P., and Farhadi, A. (2024). Matryoshka represen- Kim & Wang /Matrix factorization and prediction using SA-Tweedie 43 tation learning. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY, USA. Curran Associates Inc

    work page 2024

Show all 17 references
  1. [9]

    Pennington, J., Socher, R., and Manning, C. (2014). GloVe: Global vectors for word represen- tation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar. Association for Computational Lin- guistics

    2014

  2. [10]

    E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., and Zettlemoyer, L

    Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., and Zettlemoyer, L. (2018). Deep contextualized word representations. ArXiv, abs/1802.05365

    2018 arXiv

  3. [11]

    Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67

    2020

  4. [12]

    Raza, S., Rahman, M., Kamawal, S., Toroghi, A., Raval, A., Navah, F., and Kazemeini, A. (2024). A comprehensive review of recommender systems: Transitioning from theory to practice

    2024

  5. [13]

    M., Subramanian, R., Tan, X

    Reizenstein, J., Rungta, R., Saladi, K., Schelten, A., Silva, R., Smith, E. M., Subramanian, R., Tan, X. E., Tang, B., Taylor, R., Williams, A., Kuan, J. X., Xu, P., Yan, Z., Zarov, I., Zhang, Y., Fan, A., Kambadur, M., Narang, S., Rodriguez, A., Stojnic, R., Edunov, S., and S...

    2023

  6. [14]

    Polosukhin, I. (2017). Attention is all you need

    2017

  7. [15]

    and Manning, C

    Wang, M. and Manning, C. D. (2013). Effect of non-linear deep architecture in sequence labeling. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, pages 1285–1291, Nagoya, Japan. Asian Federation of Natural Language Pro- cessing

    2013

  8. [16]

    Gao, Q., Macherey, K., Klingner, J., Shah, A., Johnson, M., Liu, X., Kaiser, L., Gouws, S., Kato, Y., Kudo, T., Kazawa, H., Stevens, K., Kurian, G., Patil, N., Wang, W., Young, C., Smith, J., Riesa, J., Rudnick, A., Vinyals, O., Corrado, G., Hughes, M., and Dean, J. (2016). Go...

    2016 arXiv

  9. [17]

    J., and Yalcın, S

    Zanbouri, K., Darbandi, M., Nassr, M., Heidari, A., Navimipour, N. J., and Yalcın, S. (2024). A gso-based multi-objective technique for performance optimization of blockchain- based industrial internet of things. International Journal of Communication Systems, 37(15):e5886

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.