{"id":"635ef9b4-64f7-4a8a-9ac2-b4c51514d415","arxiv_id":"1908.07186","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Explicit stick-breaking representations are given for Pitman-Yor processes conditioned on a mixed Poisson species-sample size, recovering the normalized generalized gamma process as the m=0 case.","lead":"This paper derives explicit stick-breaking formulas for random discrete distributions, such as the Pitman-Yor process, when the number of observed species is fixed. These formulas matter because they make conditional versions of widely used Bayesian clustering priors easier to sample and analyze.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"General-m stick-breaking construction is internally inconsistent: with the paper's Ω_m, the q_{k,m} in Prop. 4.1 does not sum to 1 for m=2, so the N_{\\ell,m} variables in Thm 4.1 are not well-defined.","rationale":"The reader's weakest assumption focused on the unpublished Pitman manuscript [30] and the conditioning premise. That concern is less severe than it appears: the densities in (1.2)-(1.3) follow by elementary conditioning on the mixed Poisson likelihood, and the independence of (P_ℓ) from N_A given A is a standard species-sampling construction. The genuinely load-bearing weakness is internal: the general-m construction in Proposition 4.1 and Theorem 4.1 relies on Ω_m as defined in Section 2.2, and that definition is inconsistent with the moment identity (2.4) already at m=2. As a result, the claimed probability mass function for the augmentation variable N^{(1)}_m(λ) sums to 1/2, so the stick-breaking variables in Theorem 4.1 are not even well-defined for general m. This is a concrete, checkable mathematical error rather than a dependence on an unavailable source. Because the error appears to be a repairable normalization mistake rather than a conceptual failure, the appropriate disposition is conditional acceptance pending correction of Ω_m and reverification of the normalization of Lemma 4.1 and Proposition 4.1 for m≥2.","tokens_in":19171,"tokens_out":22687,"duration_ms":209464,"concrete_test":"Set m=2 and let α∈(0,1), λ>0. Using the paper's own definition Ω_2=Γ(2)[P(K_2=1)+P(K_2=2)λ^α]=2(1-α+αλ^α), compute q_{1,2} and q_{2,2} from Proposition 4.1(ii): q_{1,2}=αλ^α/Ω_2 and q_{2,2}=(1-α)/Ω_2. Their sum equals 1/2, not 1, for every λ>0. Alternatively, compare the two sides of (2.4) at m=2 with the direct derivative of e^{-λ^α}; the right-hand side is exactly twice the left-hand side. Either computation settles whether the Ω_m normalization is consistent with the claimed augmentation.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The load-bearing problem is in Section 2.2 and Sections 4.1-4.3, not in the conditioning premise from [30]. The formulas (1.2)-(1.3) are elementary: conditional on A=a, P(N_A(λ)=m|A=a)=e^{-λa}(λa)^m/m!, which immediately gives the kernel a^m e^{-λa}; so the unpublished source is not the weak point. The real issue is the definition of Ω_m. The paper defines Ω_m(λ^α)=Γ(m)Σ_{ℓ=1}^m P^{(m)}_{α,0}(ℓ)(λ^α)^{ℓ-1}/Γ(ℓ), where P^{(m)} is claimed to be the pmf of K_m, and asserts (2.4): E[S_α^m e^{-λS_α}]=α e^{-λ^α}λ^{α-m}Ω_m(λ^α). For m=2, the Chinese-restaurant probabilities are P(K_2=1)=1-α and P(K_2=2)=α, so Ω_2=2(1-α+αλ^α). Direct differentiation of e^{-λ^α} gives E[S_α^2 e^{-λS_α}]=α e^{-λ^α}λ^{α-2}(1-α+αλ^α). Thus (2.4) is off by a factor Γ(2). Equivalently, the marginal pmf in Proposition 4.1(ii) is not a pmf: for m=2, q_{1,2}+q_{2,2}=((1-α)+αλ^α)/(2(1-α+αλ^α))=1/2. Therefore the augmentation variable N^{(1)}_m(λ) does not exist with the stated distribution, and the N_{\\ell,m}(λ) used in Theorem 4.1(4.7) is invalid as stated. The defect is repairable by removing the spurious Γ(m) from Ω_m (or placing it consistently in (2.4) and Prop 4.1), but as written the general-m stick-breaking representation is not established.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper derives explicit stick-breaking representations for Pitman-Yor random probability measures conditioned on the value m of a mixed Poisson process with random rate equal to the α-diversity. It first translates the conditioning into a tilted Poisson-Kingman mixing distribution (Section 1.2), then proves a randomization identity (Proposition 1.1 and Corollary 2.1) that reduces the problem to the stable case P^{[m]}_α(λ). The main results are Theorem 3.1 (m=0, the normalized generalized gamma process), Theorem 3.2 (m=1), and Theorem 4.1 (general m), in which the size-biased permutation has stick-breaking weights W_ℓ = [β_{N_{\\ell,m}-α,α}((1-R_{\\ell,m})/R_{\\ell,m})+1]^{-1}, with R_{\\ell,m} and N_{\\ell,m} defined in Sections 4.2-4.3. A Brownian/local-time specialization for α=1/2 is given in Corollary 4.4.","tokens_in":1259,"tokens_out":2357,"duration_ms":526116,"significance":"The results are nontrivial and, if the displayed typos are corrected, constitute a solid contribution: Theorem 4.1 gives an explicit constructive algorithm for sampling conditional Pitman-Yor processes, and the m=0 case gives a stick-breaking representation for the normalized generalized gamma process, recovering and placing in context an earlier unpublished result. I checked the algebraic core of the construction: the factor Γ(m) in the definition of Ω_m is not an error (for m=2, Γ(2)=1 and Eq. (2.4) matches direct differentiation), and the probabilities q_{k,m} in Proposition 4.1(ii) sum to 1. The main issue is that several intermediate displayed densities contain typographical errors in normalizing constants and exponents; these are local but must be fixed.","major_comments":[{"comment":"As printed, Eq. (3.6) has the wrong normalizing coefficient: for n=1 the coefficient should be α/Γ(1−α), and in general α^n/Γ(1−α)^n, rather than 1/(α^n Γ(1−α)^n). Correspondingly, in Lemma 3.3 and in the augmentation argument in the proof of Theorem 3.1, the factor (1−w_k)^{α−1} should be (1−w_k)^{−α}. With the printed exponent the indicated density does not integrate to 1 and is incompatible with the stated representation W_k=1−β_{1−α,α}(1−R_k). The same normalizing error propagates to Eq. (4.1) in the general-m case. These are local corrections, but they occur in the proof of the central theorem.","section":"Section 3.3, Eq. (3.6), Lemma 3.3, and Theorem 3.1 proof"},{"comment":"The formulas defining S_{α,m}(λ) and Y^{(j)}_{m,ℓ}(x_j) are not well-formed as printed: the argument of τ_α in Eq. (2.6) reads 'λ^α + G_m^α − K_m(λ)' and the second display has a syntactically garbled form, and the display in Section 4.2 has the same problem. Since these identities are used to generate the variables in the general-m stick-breaking representation, they need to be restated precisely with correct superscripts and parentheses.","section":"Section 2.2, Eq. (2.6); Section 4.2, definition of Y^{(j)}_{m,ℓ}(x_j)"}],"minor_comments":[{"comment":"There are numerous typographical errors: 'thoeory' (page 3), 'statisics' (abstract), 'porposition' (page 14), and inconsistent use of multiplication in Eq. (3.7).","section":"Throughout"},{"comment":"In Eq. (4.9), θ appears on the right-hand side in the definition of f^{[m]}_{1/2}(t|λ), although the left-hand side is the θ-free density from (2.5). Please clarify whether θ is a free parameter in this display or whether the display is intended as the conditional density of S_{1/2,θ}.","section":"Section 4.5, Eq. (4.9)"},{"comment":"The paper relies on the unpublished manuscript [30] and the author's unpublished manuscript [18]; since the conditioning formulas in Section 1.2 are elementary and stated explicitly, this is not a correctness issue, but it would help readers if the status of [30] were clarified.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"None beyond the report; perhaps editors should ensure [30] is available or the essential formulas are reproduced, given the paper's reliance on it. The stress-test concern about a Γ(2) factor is based on an arithmetic error (Γ(2)=1) and does not affect the paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper derives stick-breaking representations for Pitman-Yor processes conditioned on a mixed Poisson count, which is a natural thing to do, and it has a real bug. The general-m theorem (Theorem 4.1) is not established as written because of a missing normalization in Omega_m.\n\nWhat's good: the m=0 and m=1 cases are plausible and worth reading. The randomization identity in Proposition 1.1 is a clean tool, and the connection between conditional PY processes and the normalized generalized gamma process is clearly drawn. James is upfront about the debt to Pitman's unpublished notes, and the conditioning formulas themselves are elementary and check out. The m=0 result recovers known published work by Favaro et al., and the paper credits that.\n\nThe problem is Section 2.2. Omega_m is defined with a leading Gamma(m), and (2.4) uses that definition for the m-th moment of the tilted stable density. Direct differentiation of e^{-lambda^alpha} gives E[S^2 e^{-lambda S}] = alpha e^{-lambda^alpha} lambda^{alpha-2} (1 - alpha + alpha lambda^alpha), but the paper's Omega_2 yields twice that. This factor propagates into Proposition 4.1: for m=2, the claimed probability masses sum to 1/2, so the augmentation variable N_m^{(1)}(lambda) does not have a valid distribution and the stick-breaking weights in Theorem 4.1 are not well-defined. This is load-bearing, not cosmetic. The fix looks easy - drop the Gamma(m) from Omega_m - and I'd expect the corrected construction to go through, but the paper as written cannot be trusted at general m.\n\nOne more issue: Lemma 3.2 is asserted with \"proof omitted\". It is probably true, but a referee would want the calculation filled in. The reliance on Pitman's unpublished [30] is a lesser concern, since the specific conditioning formulas are elementary.\n\nThis is a paper for people working on stick-breaking constructions for normalized random measures. The m=0 and m=1 parts are useful, but the advertised general-m result is currently broken. It deserves a serious referee, who should ask for the normalization fix and a proof of Lemma 3.2. I would not cite the general-m theorem until that is done.","headline":"The paper has a nice idea and a repairable but load-bearing bug: the Gamma(m) factor in Omega_m breaks the general-m stick-breaking construction.","tokens_in":20158,"tokens_out":3859,"would_cite":false,"duration_ms":32853,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["60C05","60G09","60G57","60E99"],"pacs":[],"model":"deepseek-v4-flash","headline":"A Pitman-Yor process conditioned on the species-sampling count $N_{S_{\\alpha,\\theta}}(\\lambda)=m$ still has an explicit stick-breaking representation, with the normalized generalized gamma process as the $m=0$ special case.","keywords":["Pitman-Yor process","two-parameter Poisson-Dirichlet distribution","stick-breaking representation","species sampling models","mixed Poisson processes","normalized generalized gamma process","size-biased permutation","conditioning on species sampling size"],"falsifier":"Take $\\alpha=1/2$ and $m=1$, simulate a 1/2-stable subordinator until total time $\\lambda$, reject all paths that do not have exactly one arrival before $\\lambda$, form the size-biased permutation of the normalized jumps, and compare the empirical density of the first stick $W_1$ with the explicit density (3.9) or with Theorem 4.1 specialized to $\\alpha=1/2$. Since 1/2-stable jumps can be generated exactly, this is a direct Monte Carlo check; systematic mismatch at any $\\lambda>0$ would refute the theorem.","tokens_in":18962,"feed_emoji":"🎲","tokens_out":15846,"duration_ms":150606,"temperature":0.7,"pith_summary":"Species sampling models are random discrete distributions used for clustering in Bayesian statistics and machine learning, and the Pitman-Yor process is a central example whose ranked masses follow the two-parameter Poisson-Dirichlet law. This paper establishes that conditioning such a process on the number of species seen by time $\\lambda$, modeled as a mixed Poisson count with random rate equal to the process's $\\alpha$-diversity, does not destroy the stick-breaking structure: the conditional size-biased weights are given explicitly in closed form. The main theorem writes each conditional stick as $W_\\ell = [\\beta((1-R_\\ell)/R_\\ell)+1]^{-1}$, with an independent $\\beta$ variable, a random ratio $R_\\ell$ that encodes the conditioning, and an auxiliary integer variable with explicit probabilities. The case $m=0$ recovers the normalized generalized gamma process, so the paper also yields explicit stick-breaking weights for that widely used nonparametric prior. If the result is correct, conditional Pitman-Yor processes can be sampled by direct construction rather than by rejection or conditioning on simulations.","feed_headline":"Explicit stick-breaking survives conditioning on species count","feed_subtitle":"The conditional weights come in closed form; m=0 is the normalized generalized gamma case.","key_machinery":"The machinery has three parts. First, the mixed-Poisson conditioning identity for the total abundance $A$: $P(A\\in da\\,|\\,N_A(\\lambda)=m)=a^m e^{-\\lambda a}P(A\\in da)/E[A^m e^{-\\lambda A}]$, which converts conditioning on the species count into a size-biased exponential tilt of the stable density. Second, the standard size-biased deletion construction for the ranked jumps of a stable subordinator, giving a product-form joint density for the first $n$ stick weights and the remaining stable mass. Third, a $\\beta$-gamma integral identity (Proposition 3.1) that rewrites the tilted density so that, conditionally on auxiliary variables $R_k(\\lambda)=((\\tilde G_{k-1}+\\lambda^\\alpha)/(\\tilde G_k+\\lambda^\\alpha))^{1/\\alpha}$, the sticks become conditionally independent with densities supported on $(r,1)$. The $R_k$ variables are the load-bearing mechanism: they carry the dependence induced by conditioning and reduce to independent $\\beta$ variables in the classical GEM$(\\alpha,\\theta)$ recovery.","core_discovery":"The central claim is Theorem 4.1. For $(P^{[m]}_\\ell(\\lambda))$---the law of the ranked masses $(P_\\ell) \\sim \\mathrm{PD}(\\alpha,0)$, the two-parameter Poisson-Dirichlet law with $\\theta=0$, conditioned on $N_{S_\\alpha}(\\lambda)=m$---the size-biased permutation $(\\tilde P_k(\\lambda))$ takes the stick-breaking form $\\tilde P_k=(1-W_k)\\prod_{l<k}W_l$, where $W_\\ell$ is given by $W_\\ell := [\\beta^{(\\ell)}_{N_{\\ell,m}(\\lambda)-\\alpha,\\alpha}((1-R_{\\ell,m}(\\lambda))/R_{\\ell,m}(\\lambda))+1]^{-1}$. Here the $\\beta^{(\\ell)}$ are independent $\\mathrm{Beta}(1-\\alpha,\\alpha)$ variables independent of the $R$'s, the $R_{\\ell,m}(\\lambda)$ are ratio variables built from gamma partial sums and a tilted stable variable, and $N_{\\ell,m}(\\lambda)$ is a discrete random variable whose probability mass function is explicit. Given the $R$'s, the $W$'s are conditionally independent; unconditionally they are dependent, which is exactly how the conditioning on the count $m$ breaks the classical GEM independence. The paper also proves the $m=0$ specialization $W_k=1-\\beta^{(k)}_{1-\\alpha,\\alpha}(1-R_k(\\lambda))$, identifying the normalized generalized gamma process, and shows that randomizing $\\lambda$ by a gamma variable recovers the classical independent-$\\beta$ GEM$(\\alpha,\\theta)$ stick weights.","pith_inferences":["An implicit consequence: the same tilted-density route should yield explicit conditional stick-breaking for any Poisson-Kingman partition whose L\\'evy density admits a tractable exponential tilt, not only the stable case; the beta-gamma integral would need a tailored analogue.","The simple binomial description of $N_{\\ell,m}(\\lambda)$ in the GEM recovery hints at a sequential-arrival-time reading, connecting the conditional sticks to Chinese-restaurant-type occupancy counts.","A testable extension: Theorem 4.1 could serve as the sampling core of a Gibbs scheme for normalized generalized gamma mixture models that condition on the observed number of clusters at a latent time, possibly removing auxiliary-variable layers from current posterior algorithms.","Because the $R$ variables encode the tilted law of the total mass, the representation also opens a route to large-time asymptotics of the first stick, such as the behaviour of $W_1$ as $\\lambda$ grows."],"forward_implications":["For $m=0$, the normalized generalized gamma process has an explicit stick-breaking representation whose sticks are simple functions of beta variables and the gamma-ratio variables $R_k(\\lambda)$.","For $m\\geq 1$, exact simulation of the conditional process is possible without rejection sampling: draw $N_{\\ell,m}(\\lambda)$ and $R_{\\ell,m}(\\lambda)$, then draw independent beta variables and form $W_\\ell$ by the formula in Theorem 4.1.","At $\\alpha=1/2$ the results specialize to explicit stick-breaking for the normalized inverse Gaussian process and for the conditional laws $P^{[m]}_{1/2}(\\lambda)$, expressed through inverse Gaussian and Brownian variables.","Randomizing $\\lambda$ via a gamma variable recovers GEM$(\\alpha,\\theta)$, and in that case the auxiliary $R$-variables become independent beta variables while the $W_\\ell$ reduce to the classical independent $\\mathrm{Beta}(\\theta+\\ell\\alpha,1-\\alpha)$ sticks."],"supporting_citations":[{"why":"It supplies the mixed-Poisson species-sampling conditioning identities (1.2)-(1.3) on which all conditional laws rest.","marker":"[30]"},{"why":"It establishes the size-biased sampling construction and the joint law of stick weights and stable mass used in Lemma 3.1.","marker":"[24]"},{"why":"It provides the Poisson-Kingman formalism and the stable-case representation $\\mathrm{PD}(\\alpha,0)=\\int \\mathrm{PD}(\\alpha|t)f_\\alpha(t)\\,dt$ that defines $P^{[m]}_\\alpha(\\lambda)$.","marker":"[28]"},{"why":"It gives the block-count distribution $K_m$ and the moment identity used to define $\\Omega_m$ and the tilted densities.","marker":"[29]"},{"why":"It supplies the two-parameter Poisson-Dirichlet size-biased deletion structure used to recover GEM$(\\alpha,\\theta)$ and the independent-beta special cases.","marker":"[32]"},{"why":"It gives the explicit Brownian/local-time size-biased ordering of $\\mathrm{PD}(1/2|s^{-2}/2)$ used for the $\\alpha=1/2$ results in Section 4.5.","marker":"[1]"}],"fun_headline_variants":["Conditional PY stick-breaking: closed-form weights","Closed-form stick-breaking for Pitman-Yor given species count","PY conditioning yields explicit stick weights, incl. gamma case","Stick-breaking weights after fixing species count: explicit","When m=0, normalized generalized gamma emerges in sticks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes the accuracy of an unpublished predecessor's claim that conditioning on the mixed-Poisson count $N_A(\\lambda)=m$ is fully captured by the tilted density $a^m e^{-\\lambda a}$ times the law of $A$, and that the conditional law of the ranked masses depends on $A$ only through its value; if that premise fails, the stick-breaking formulas describe tilted mixtures rather than the conditional Pitman-Yor process as claimed.","fun_headline_variants_meta":{"raw":{"variants":["Conditional PY stick-breaking: closed-form weights","Closed-form stick-breaking for Pitman-Yor given species count","PY conditioning yields explicit stick weights, incl. gamma case","Stick-breaking weights after fixing species count: explicit","When m=0, normalized generalized gamma emerges in sticks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000788,"raw_usage":{"total_tokens":3623,"prompt_tokens":1242,"completion_tokens":2381,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":858,"completion_tokens_details":{"reasoning_tokens":2302}},"tokens_in":858,"tokens_out":2381,"duration_ms":18338,"temperature":1.0,"reasoning_tokens":2302,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:23:19.831679+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take $\\alpha=1/2$ and $m=1$, simulate a 1/2-stable subordinator until total time $\\lambda$, reject all paths that do not have exactly one arrival before $\\lambda$, form the size-biased permutation of the normalized jumps, and compare the empirical density of the first stick $W_1$ with the explicit density (3.9) or with Theorem 4.1 specialized to $\\alpha=1/2$. Since 1/2-stable jumps can be generated exactly, this is a direct Monte Carlo check; systematic mismatch at any $\\lambda>0$ would refute the theorem.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It supplies the mixed-Poisson species-sampling conditioning identities (1.2)-(1.3) on which all conditional laws rest."},{"cited_title":"and Yor, M","cited_arxiv_id":null,"evidence_quote":"It establishes the size-biased sampling construction and the joint law of stick weights and stable mass used in Lemma 3.1."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the Poisson-Kingman formalism and the stable-case representation $\\mathrm{PD}(\\alpha,0)=\\int \\mathrm{PD}(\\alpha|t)f_\\alpha(t)\\,dt$ that defines $P^{[m]}_\\alpha(\\lambda)$."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It gives the block-count distribution $K_m$ and the moment identity used to define $\\Omega_m$ and the tilted densities."},{"cited_title":"and Yor, M","cited_arxiv_id":null,"evidence_quote":"It supplies the two-parameter Poisson-Dirichlet size-biased deletion structure used to recover GEM$(\\alpha,\\theta)$ and the independent-beta special cases."},{"cited_title":"and Pitman, J","cited_arxiv_id":null,"evidence_quote":"It gives the explicit Brownian/local-time size-biased ordering of $\\mathrm{PD}(1/2|s^{-2}/2)$ used for the $\\alpha=1/2$ results in Section 4.5."}],"review_version":1}