REVIEW 2 major objections 5 minor 3 references
Introducing the PIT-plot -- a new tool in the portfolio manager's toolkit
T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Portfolio sensitivity analysis should measure each project's impact on the whole portfolio, and the PIT-plot charts that impact.
desk verdict Useful tornado-style visualization for portfolio metrics, but the Success bar only works for independent projects and Section 4 overclaims; should be revised before serious use. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the PIT-plot, a horizontal two-bar chart in the style of a Tornado diagram. For each project $i$, one bar is the Exclusion bar $\Delta^{(i)} = M_P^{(i)} - M_P$, the change in the portfolio metric when the project is removed, and the other is the Success bar $\tilde{\Delta}^{(i)} = \tilde{M}_P^{(i)} - M_P$, the change when the project's success is taken as guaranteed. In the cash-flow setting, the portfolio metric is the Productivity Index $PI = (R-C)/C$, where $R$ and $C$ are risk-adjusted expected revenue and development cost; exclusion sums $R$ and $C$ over the remaining projects, while the Success bar replaces project $i$'s contribution by its conditional values over Monte Carlo iterations in which $i$ succeeds. The plot's ordering by $\Delta^{(i)}$ is what turns the bars into decision rules: top bars are projects whose removal hurts the portfolio, bottom bars are projects whose removal helps it.
What would settle it
Build a two-project portfolio where both projects target the same market, simulate the full joint distribution, and compare the Success bar computed by the paper's formula (conditional cash flows of project $i$ added to unconditional cash flows of the other) with the portfolio metric actually computed from the joint conditional distribution given $i$'s success; a gap between the two shows the formula's breakdown under interdependence.
Extended reading notes
Core claim
The paper's central claim is that portfolio-level sensitivity analysis should be based on changes in portfolio composition rather than on individual project parameters. For a portfolio metric $M_P$, the Exclusion bar $\Delta^{(i)} = M_P^{(i)} - M_P$ measures the effect of removing project $i$, while the Success bar $\tilde{\Delta}^{(i)} = \tilde{M}_P^{(i)} - M_P$ measures the effect of guaranteeing project $i$'s success. The PIT-plot displays both bars for every project, sorted by $\Delta^{(i)}$, and the paper proposes that projects with positive Exclusion bars are termination candidates, projects with large Success bars are candidates for risk mitigation, and projects with strongly negative Exclusion bars but modest Success bars should continue on plan. The construction is metric-agnostic, and the paper illustrates it with the Productivity Index $PI = (R-C)/C$ computed from Monte Carlo cash-flow simulations. It further asserts that the same principles remain valid when projects are interdependent, provided the underlying quantitative model captures the dependencies.
Load-bearing premise
The Success bar calculation assumes that a project's success changes nothing about the other projects' cash flows; if projects cannibalize each other's markets or share resources, the bar can misstate the true portfolio impact.
Editorial extensions
If this is right
- Portfolio managers can rank projects by their marginal impact on the portfolio metric instead of by standalone project ratios, which changes which projects look strong or weak.
- Projects with positive Exclusion bars are candidates for termination when budget is scarce, because removing them improves portfolio efficiency.
- Projects with large positive Success bars are candidates for risk mitigation and success-probability investments, because guaranteeing their success creates the largest portfolio upside.
- For relative efficiency metrics like ROI, IRR, or Productivity Index, the PIT-plot is more informative than for simple additive metrics, which can be ranked by direct calculation.
- The tool transfers to any industry where portfolio-level efficiency matters, not just pharmaceutical R&D.
Reading between the lines
- A natural extension is a mirror-image 'Addition bar' for projects not yet in the portfolio, measuring the impact of adding rather than removing a project, which would turn the PIT-plot into a prioritization tool for candidate selection.
- The Success bar as defined is an 'all-else-unchanged' counterfactual; in real portfolios with cannibalization or synergy, the true conditional portfolio metric should be recomputed with the model's dependence structure, which would require a dependency-adjusted formula.
- One could attach uncertainty intervals to the bars via bootstrapping the Monte Carlo iterations, so that the ordering of projects respects simulation noise rather than appearing exact.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a portfolio-level sensitivity chart, the PIT-plot. For each project i it defines an Exclusion bar Δ(i)=M_P^(i)-M_P, the change in a portfolio metric when project i is excluded, and a Success bar Δ̃(i)=M̃_P^(i)-M_P, the change when project i's success is guaranteed. The authors argue that sensitivity analysis in portfolio management should be based on changes in portfolio composition rather than on individual project parameters, and they instantiate the definitions for a Productivity Index computed from Monte Carlo cash-flow simulations. A ten-project pharmaceutical example is used to draw prescriptive conclusions about termination, risk mitigation, and continued investment. Section 4 asserts that the principles remain valid under project dependencies and with input and simulation uncertainty.
Significance. The perspective shift is a genuine and potentially useful contribution: the PIT-plot is simple, needs no fitted parameters, and directly addresses the portfolio-level question of which projects matter most. The Exclusion bar is a clean counterfactual that a model can compute by re-running without a project. However, the Success bar as defined in Section 2.2 mixes conditional cash flows for project i with unconditional cash flows for all other projects, so it is only valid under independence, and the paper's Section 4 claim that the principles cover dependency is not supported by the equations. The illustrative example also reports point estimates without any uncertainty bounds, while Section 4 acknowledges that uncertainty can change both bar lengths and ordering. These are fixable gaps, and a revised version could be a useful decision-support tool.
major comments (2)
- [Section 2.2 and Section 4] The equations for R̃_P^(i) and C̃_P^(i) use the unconditional portfolio means R_i* and C_i* for all projects other than i, while conditioning only project i on success. If project i affects the revenue or cost distribution of other projects, then E[R_i* | i successful] ≠ R_i*, and the Success bar is not the conditional portfolio impact. This is exactly the shared-market or resource-competition case that Section 4 claims to cover. Moreover, because the Productivity Index is a ratio, inserting conditional mean cash flows into a nonlinear function does not give the conditional expectation of the index. The Success bar should be computed by averaging the portfolio metric over Monte Carlo iterations in which i succeeds, or by re-running the model with i's success imposed, and the claims in Section 4 should be restricted to estimators that account for dependencies. Since the Success bar is the basis for risk-mitigation recommendations in Section 2.3, this is a load-bearing issue rather than a presentation concern.
- [Section 3 and Section 4] The bullets under Figure 3 (for example, 'Projects 1 and 5 might be candidates for termination' and 'Projects 6 and 9 warrant additional attention') present point-estimate conclusions without any error bars, confidence intervals, or sensitivity bounds. Section 4 itself notes that simulation variability and input uncertainty can change the lengths of the bars and the ordering of projects. For a decision-support tool, the lack of any quantification means the reader cannot tell whether the illustrative ordering is robust to Monte Carlo noise. At minimum, add a bootstrap or other uncertainty analysis, or explicitly state that the example is a schematic illustration and soften the prescriptive wording.
minor comments (5)
- [Section 2.2] After the definition of the exclusion-bar PI, the text says 'Calculating the PI for each project' but the quantity is computed for the portfolio, and later 'D̃_P^(i)' should be 'C̃_P^(i)'.
- [Section 1] The affiliation line contains 'Go teborg' instead of 'Gothenburg'.
- [Section 4] The sentence 'these uncertainties propagate to the metrics visualized in the PIT-plot, obviously impacting the length of the bars but it may also impact the ordering' has a subject-verb agreement issue; 'it' should be 'they'.
- [References] The reference list has formatting inconsistencies, such as 'Raada et al.' and 'Mohagheghi et al. (2020) ... 2019'.
- [Section 2.1] Figure 1 is referenced, but only a caption appears in the text; ensure that the final version includes the actual schematic figure.
Circularity Check
No significant circularity: the PIT-plot bars are definitional contrasts, and the cited model is only an input for the illustrative example.
full rationale
The paper's central quantities are defined directly as contrasts of portfolio metrics under different scenarios: the Exclusion bar is Δ(i) = M_P^(i) − M_P and the Success bar is Δ̃(i) = M̃_P^(i) − M_P. These are neither fitted to data nor derived from any prior result of the authors; they are explicit definitions in Sections 2.1 and 2.2. The only self-citation is Wiklund (2019), which supplies the generic drug-development model used to construct the illustrative ten-project example, and Farid et al. (2021), cited among general portfolio-optimization methods in the introduction. Neither citation is invoked to justify the validity of the PIT-plot or to rule out alternative formulations, so the self-citations are not load-bearing. There is no fitted parameter later relabeled as a prediction, no uniqueness theorem imported from the authors' prior work, and no ansatz smuggled in via citation. The strongest correctness concern — that the Success bar in Section 2.2 mixes unconditional cash flows of other projects with conditional cash flows of project i, which is only exact when projects are independent — is a validity or modeling limitation, not circularity, because the formula does not assume the conclusion it is used to establish; it simply computes a defined contrast incorrectly for dependent projects. Hence the derivation chain is self-contained and the appropriate circularity score is 0.
Assumptions & free parameters
free parameters (1)
- Illustrative portfolio input parameters =
Hand-set in Table 1, not fitted, not part of the general method
assumptions (4)
- domain assumption Portfolio revenues and costs are additive across projects (R_P = sum R_i, C_P = sum C_i)
- domain assumption Conditional independence of project outcomes for the success bar: conditioning on project i success leaves other projects' risk-adjusted values unchanged
- standard math The underlying Monte Carlo simulation yields unbiased estimates of expected revenues and costs
- domain assumption The generic drug development model (Wiklund 2019) adequately represents project cash flows
Cite this review
Pith. "Pith review of Introducing the PIT-plot -- a new tool in the portfolio manager's toolkit." pith.science (2026). https://pith.science/paper/IN6S2G4A
@misc{pith2026250612068,
author = {Pith},
title = {Pith review of: Introducing the PIT-plot -- a new tool in the portfolio manager's toolkit},
year = {2026},
howpublished = {\url{https://pith.science/paper/IN6S2G4A}},
note = {Machine review of arXiv:2506.12068}
}
read the original abstract
Project portfolio management is an essential process for organizations aiming to optimize the value of their R&D investments. In this article, we introduce a new tool designed to support the prioritization of projects within project portfolio management. We label this tool the PIT-plot, an acronym for Project Impact Tornado plot, with reference to the similarity to the Tornado plot often used for sensitivity analyses. Many traditional practices in portfolio management focus on the properties of the projects available to the portfolio. We are with the PIT-plot changing the perspective and focus not on the properties of the projects themselves, but on the impact that the projects may have on the portfolio. This enables the strategic portfolio management to identify and focus on the projects of largest impact to the portfolio, either for the purpose of risk mitigation or for the purpose of value-adding efforts.
Reference graph
Works this paper leans on
-
[1]
(1992) Spiderplots versus Tornado Diagrams for Sensitivity Analysis
Eschenbach TG. (1992) Spiderplots versus Tornado Diagrams for Sensitivity Analysis. Interfaces 22(6):40-46. https://doi.org/10.1287/inte.22.6.40 Farid M, Chaudhry A, Ytterstad M, Wiklund SJ. (2021). Pharmaceutical portfolio optimization under cost uncertainty via chance constrained-type method. Journal of Mathematics in.Industry 11,
-
[3]
https://doi.org/10.1186/s13362-021-00099-3 10 Hu Q, Szmerekovsky J. (2017). Project Portfolio Selection: A Newsvendor Approach. Decision Sciences, 48: 176-199. https://doi.org/10.1111/deci.12214 Jekunen A. (2014). Decision-making in product portfolios of pharmaceutical research and development--managing streams of innovation in highly regulated markets. D...
-
[824]
doi:10.1177/009286159803200321 Vieira GB, Oliveira HS, de Almeida JA, Belderrain MCN. (2024). Project Portfolio Selection considering interdependencies: A review of terminology and approaches, Project Leadership and Society, Volume 5, 100115, ISSN 2666-7215, https://doi.org/10.1016/j.plas.2023.100115. Wiklund SJ. (2019). A modelling framework for improved...
arXiv 2024
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.