pith. sign in

arxiv: 2605.04259 · v1 · submitted 2026-05-05 · 💻 cs.SE · cs.HC

EngThrive: Make It Fast and Easy to Do Great Work

Brian Houck , Tim Bozarth , David Liu , Dean Carignan This is my paper

Pith reviewed 2026-05-08 17:20 UTC · model grok-4.3

classification 💻 cs.SE cs.HC
keywords developer productivitymeasurement frameworksoftware engineering metricswellbeingNorth Star metricstelemetry and surveysorganizational improvement
0
0 comments X

The pith

EngThrive organizes developer productivity around speed, ease, quality and wellbeing using paired North Star and diagnostic metrics.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents EngThrive as a concrete system for turning multidimensional productivity measurement into sustained improvements inside large engineering organizations. It defines three core dimensions—Speed, Ease, and Quality—protected by a Thriving guardrail that keeps developer wellbeing from declining as performance rises. Within each dimension, high-level outcome metrics are paired with finer diagnostic submetrics drawn from both system telemetry and developer surveys. The design deliberately chooses metrics so that efforts to improve the numbers also produce genuine gains rather than gaming behavior. Case studies illustrate how this approach supports system-level changes in tools, policies, and work environments.

Core claim

EngThrive is a measurement and improvement system that organizes productivity around three dimensions—Speed, Ease, and Quality—with Thriving as a guardrail to ensure developer wellbeing improves alongside performance. Within each dimension, outcome-oriented North Star metrics are paired with diagnostic submetrics that combine system telemetry with developer surveys. This structure supplies both scale and context, and the metric-selection principles are chosen so that optimizing the numbers produces real outcome gains. The same framework functions as a general-purpose evaluation language for developer tools, AI systems, organizational policies, and physical or cultural work environments.

What carries the argument

The EngThrive framework, which defines Speed, Ease, Quality, and Thriving dimensions and pairs each with North Star outcome metrics plus diagnostic submetrics from telemetry and surveys.

If this is right

  • Teams gain specific, actionable diagnostics for bottlenecks in speed, ease, or quality without losing sight of overall outcomes.
  • Metric selection principles reduce the risk that measurement drives superficial optimization.
  • The framework extends from tool evaluation to policies and work environments as a common evaluation language.
  • Sustained system-level improvements become possible because North Star metrics stay anchored to developer experience.
  • Wellbeing is treated as a first-class dimension rather than an afterthought.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Organizations adopting similar paired-metric designs could reduce developer burnout by making wellbeing visible alongside performance targets.
  • The approach implies that single-metric systems like raw velocity or commit counts are structurally incomplete for guiding engineering work.
  • If the alignment between metrics and behavior holds, the same structure could be tested on non-software knowledge work such as research or design teams.
  • Widespread use would create comparable data across companies, enabling industry-level benchmarking of developer experience.

Load-bearing premise

That well-chosen metrics will steer teams toward genuine improvements rather than just better scores, and that the framework will work outside the Microsoft deployment setting described in the case studies.

What would settle it

A controlled deployment of EngThrive at another large engineering organization that produces no measurable rise in outcome metrics or wellbeing scores after twelve months.

read the original abstract

Frameworks such as SPACE, DevEx, and DORA established that developer productivity is inherently multidimensional, but left practitioners with a practical question: what should we measure, and how should we use it to improve? This paper introduces Engineering Thrive (EngThrive), a measurement and improvement system developed and deployed across Microsoft's engineering organization. EngThrive organizes productivity around three dimensions - Speed, Ease, and Quality - with Thriving as a guardrail to ensure developer wellbeing improves alongside performance. Within each dimension, outcome-oriented North Star metrics are paired with diagnostic submetrics, combining system telemetry with developer surveys to provide both scale and context. We describe the design principles that guide metric selection, including an approach in which well-chosen metrics align "gaming" behavior with genuine improvement. We also outline the data platform, survey program, and dashboard ecosystem required to operationalize this approach in practice, and present case studies demonstrating how outcome-oriented measurement enables sustained, system-level improvements. Finally, we show that EngThrive functions as a general-purpose evaluation language, applicable not only to developer tools and AI, but to organizational policies, work environments, and other factors that shape how developers experience their work. We offer EngThrive as a concrete model for organizations seeking to move beyond measuring activity toward improving outcomes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper introduces Engineering Thrive (EngThrive), a deployed measurement and improvement system at Microsoft that organizes developer productivity around three dimensions—Speed, Ease, and Quality—with Thriving as a guardrail for wellbeing. Within each dimension, outcome-oriented North Star metrics are paired with diagnostic submetrics drawn from system telemetry and developer surveys. The work describes metric-selection design principles (including heuristics to align gaming behavior with genuine improvement), the supporting data platform, survey program, and dashboard ecosystem, presents case studies of sustained system-level improvements, and positions EngThrive as a general-purpose evaluation language applicable to tools, AI, policies, and work environments.

Significance. If the framework and its claimed outcomes hold, the paper supplies a concrete, large-scale example of shifting from activity metrics to outcome-oriented measurement in software engineering, with explicit attention to anti-gaming design and multi-source data integration. The deployed Microsoft implementation and case studies offer practical guidance on operationalization that could inform other organizations. Credit is due for the explicit framing of Thriving as a non-negotiable guardrail and for treating the framework as an extensible evaluation language rather than a one-off tool.

major comments (3)
  1. [Case Studies] Case Studies section: observed metric shifts are presented as demonstrations of sustained improvement without reported controls, pre/post statistical tests, matched non-EngThrive cohorts, or error bars, so attribution to the framework versus concurrent changes cannot be verified.
  2. [Design Principles] Design Principles section: the claim that well-chosen metrics align gaming with genuine improvement is stated as a selection heuristic but is not accompanied by empirical tests against observed gaming incidents or quantitative validation of its effectiveness.
  3. [Generalization and Broader Applicability] Generalization claim (final section): EngThrive is asserted to function as a general-purpose evaluation language beyond the Microsoft context, yet all supporting examples and data remain internal to one organization with no external validation or cross-organization replication.
minor comments (2)
  1. [Abstract] The abstract states that the system combines telemetry with surveys 'to provide both scale and context' but does not define the precise weighting or integration method used in the dashboards; this should be clarified with a brief example.
  2. [Data Platform and Dashboards] Figure captions for the dashboard ecosystem should explicitly label which metrics are North Star versus diagnostic and indicate data sources (telemetry vs. survey) to improve readability.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their constructive and insightful review of our manuscript on the EngThrive framework. The comments identify key areas where evidence presentation and claims of applicability can be strengthened. We respond to each major comment below and indicate planned revisions.

read point-by-point responses

  1. Referee: [Case Studies] Case Studies section: observed metric shifts are presented as demonstrations of sustained improvement without reported controls, pre/post statistical tests, matched non-EngThrive cohorts, or error bars, so attribution to the framework versus concurrent changes cannot be verified.

    Authors: We agree that the case studies are observational and do not include controls, statistical tests, matched cohorts, or error bars, limiting the ability to attribute changes solely to EngThrive. These deployments occurred as part of operational improvements at Microsoft rather than as designed experiments, making such rigorous controls impractical at the time. We will revise the Case Studies section to explicitly note these limitations, frame the examples as illustrations of the framework in use rather than causal demonstrations, and suggest opportunities for future controlled evaluations. revision: yes

  2. Referee: [Design Principles] Design Principles section: the claim that well-chosen metrics align gaming with genuine improvement is stated as a selection heuristic but is not accompanied by empirical tests against observed gaming incidents or quantitative validation of its effectiveness.

    Authors: The metric selection heuristics, including alignment of gaming incentives with genuine improvement, are presented as principles derived from iterative design experience and patterns observed in our telemetry. The manuscript does not include formal empirical tests or quantitative validation of these heuristics against specific incidents. We will revise the Design Principles section to clarify their status as experiential heuristics, elaborate on the rationale with additional qualitative context from our deployments, and acknowledge the lack of controlled validation in this paper. revision: partial

  3. Referee: [Generalization and Broader Applicability] Generalization claim (final section): EngThrive is asserted to function as a general-purpose evaluation language beyond the Microsoft context, yet all supporting examples and data remain internal to one organization with no external validation or cross-organization replication.

    Authors: We acknowledge that all examples, data, and case studies are internal to Microsoft. The positioning of EngThrive as a general-purpose evaluation language rests on the abstract structure of the framework (North Star metrics paired with diagnostics across dimensions and data sources) rather than multi-organization evidence. We will revise the final section to moderate the generalization language, presenting EngThrive as a model developed and validated at scale in one large organization that other entities may adapt and test, while noting the value of future external replications. revision: yes

Circularity Check

0 steps flagged

No circularity: EngThrive is a descriptive framework without derivations or fitted predictions.

full rationale

The paper introduces a three-dimension productivity framework (Speed/Ease/Quality + Thriving guardrail) with North Star and diagnostic metrics, referencing prior work such as SPACE, DevEx, and DORA only for context. No mathematical equations, parameter fits, or predictions are presented that reduce by construction to the paper's own inputs. Design principles for metric selection and anti-gaming alignment are stated as heuristics, and case studies are offered as operational illustrations rather than self-referential tests. The central claims rest on applied description and organizational deployment, not on any self-definitional, fitted-input, or self-citation load-bearing chain.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on the domain assumption that productivity can be usefully decomposed into Speed, Ease, Quality, and Thriving, and that telemetry-plus-survey data will yield actionable diagnostics. No free parameters or invented entities are introduced.

axioms (2)
  • domain assumption Developer productivity is inherently multidimensional and requires both outcome metrics and diagnostic submetrics.
    Stated in the opening paragraph as the motivation for EngThrive.
  • domain assumption Well-chosen metrics can align gaming behavior with genuine improvement.
    Presented as one of the design principles guiding metric selection.

pith-pipeline@v0.9.0 · 5532 in / 1431 out tokens · 42823 ms · 2026-05-08T17:20:40.546115+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

12 extracted references

  1. [1]

    The SPACE of Developer Productivity

    Forsgren, N., Storey, M.-A., Maddila, C., Zimmermann, T., Houck, B., and Butler, J. “The SPACE of Developer Productivity.” ACM Queue, 19(1), 2021

    2021

  2. [2]

    DevEx in Action

    Forsgren, N., Kalliamvakou, E., Noda, A., Greiler, M., Houck, B., and Storey, M.-A. “DevEx in Action.” ACM Queue, 2024

    2024

  3. [3]

    The SPACE of AI

    Houck, B., Lowdermilk, T., Beyer, H., Clarke, K., and Hanrahan, C. “The SPACE of AI.” 2025

    2025

  4. [4]

    SPACE in the Age of AI: Measuring What Matters for Developers

    Houck, B. “SPACE in the Age of AI: Measuring What Matters for Developers.” Presented at STACK Conference, 2024

    2024

  5. [5]

    The Impact of AI on Developer Productivity: Evidence from GitHub Copilot

    Peng, S., Kalliamvakou, E., Cihon, P., and Demirer, M. “The Impact of AI on Developer Productivity: Evidence from GitHub Copilot.” arXiv preprint, 2023

    2023

  6. [6]

    Time Warp: How Developers Actually Spend Their Time

    Zimmermann, T., et al. “Time Warp: How Developers Actually Spend Their Time.” Microsoft Research, 2023

    2023

  7. [7]

    The New Future of Work: Research from Microsoft into the Pandemic's Impact on Work Practices

    Teevan, J., Baym, N., Butler, J., Hecht, B., Jaffe, S., Nowak, K., Sellen, A., and Yang, L. (Eds.). “The New Future of Work: Research from Microsoft into the Pandemic's Impact on Work Practices.” Microsoft Research Tech Report, 2021

    2021

  8. [8]

    Developer Productivity Engineering Summit

    Houck, B. and Nachi, S. “Developer Productivity Engineering Summit.” Panel Discussion, 2025

    2025

  9. [9]

    DORA 2025 State of AI-assisted Software Development Report

    DeBellis, D., Storer, K., Harvey, N., Beane, M., Edwards, R., Fraser, E., Good, B., Kalliamvakou, E., Kim, G., Maxwell, E., D'Angelo, S., Inman, S., Murillo, A., and Villalba, D. "DORA 2025 State of AI-assisted Software Development Report." DORA, Google, 2025

    2025

  10. [10]

    Why Microsoft Measures Employee Thriving, Not Engagement

    Klinghoffer, D. and McCune, E. “Why Microsoft Measures Employee Thriving, Not Engagement.” Harvard Business Review, June 2022

    2022

  11. [11]

    Accelerate: The Science of Lean Software and DevOps

    Forsgren, N., Humble, J., and Kim, G. Accelerate: The Science of Lean Software and DevOps. IT Revolution Press, 2018

    2018

  12. [12]

    The Best of Both Worlds: Unlocking the Potential of Hybrid Work for Software Engineers

    Houck, B., Yelin, H., Butler, J., Forsgren, N., and McMartin, A. “The Best of Both Worlds: Unlocking the Potential of Hybrid Work for Software Engineers.” Microsoft Research, 2023

    2023