Pith. sign in

REVIEW 3 major objections 4 minor 27 references

Clinical Audit Logs as Multi-Axial Traces of Care Delivery

T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read One EHR audit-log entry is simultaneously clinician work, patient trajectory, team activity, and workflow—motivating one pretrained model over the raw stream.

desk verdict Read this if you work with EHR audit logs: the four-axis framing is a genuine synthesis and a useful organizing device, but the paper's central pretraining prediction needs sharper operationalization before it can actually be tested. read the letter →

arxiv 2607.15397 v1 pith:QOVXJ4YW submitted 2026-07-16 cs.CY

Braden Eberhard , Nate Apathy , Kevin Johnson This is my paper
classification cs.CY
keywords EHRauditlogsmulti-axialeventstreamsfoundationmodelsrepresentationlearningclinicalworkflowcareteamcoordinationpretraininggovernance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's central claim is that a single EHR audit-log entry carries several clinically meaningful relations at once: it is an act by a clinician, a step in a patient's trajectory, a contribution to a care team's work, and a step in a process. The authors argue that this multi-axial structure is why the same raw stream has supported such different lines of research, from clinician effort to patient outcomes to workflow discovery. From that reading, they derive a concrete prediction: a representation pretrained over the full audit-log stream should help most for outcomes whose signal is spread across multiple orderings of the stream, and least for outcomes contained within one axis. The paper is a Perspective—it makes this case with a worked stroke-alert example and proposes an evaluation plan, rather than reporting new experiments. If the prediction holds, a single shared representation could replace task-specific pipelines and let sites with little annotation inherit a working signal.

What carries the argument

The central object is the multi-axial event stream: a way of viewing each audit-log row as simultaneously participating in several orderings (axes) of the stream—clinician, patient, team, and process. The paper defines an axis as a way of ordering events from a single vantage point, and notes that any single ordering is partial. The load-bearing mechanism is the claim that axes meet at shared events rather than running in parallel, so cross-axis meaning (e.g., whether an access reflects ordinary review or a coordination gap) is visible only at the intersection. This structure is what motivates foundation-model pretraining: a representation learned over the raw stream should preserve the cont

What would settle it

Run a multi-site experiment where a model pretrained on raw audit logs at one site is fine-tuned and evaluated at another, alongside a strong single-axis baseline trained from scratch at that site; the central claim is falsified if the pretrained model does not outperform the baseline on cross-axis tasks at the second site, or if its advantage does not increase as tasks shift from within-axis to cross-axis.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that audit logs are not just operational metadata or a source of task-specific measures; they are multi-axial event streams. An event like a physician opening a CT angiography result at 07:42 is at once an attention shift in a clinician session, a moment in a patient's stroke evaluation, one access within a team's overlapping chart use, and the imaging-review step of a time-sensitive protocol. Because all four meanings are present in the raw row before any task selects among them, the authors propose that a single representation pretrained over the full stream could serve many endpoints simultaneously, and they predict its advantage will be greatest

Load-bearing premise

The load-bearing premise is that unsupervised pretraining over raw audit-log streams learns reusable cross-axis structure that transfers across institutions; if event vocabularies and screen paths are too site-specific, or if the multi-axial signal is buried in noise, a shared pretrained representation could fail even though the multi-axial reading of logs remains true.

Editorial extensions

If this is right

  • A single pretrained representation over the raw audit-log stream could replace separate pipelines for clinician effort, patient deterioration, care-team coordination, and workflow analysis.
  • The advantage of pretraining should be largest for outcomes whose signal is distributed across multiple orderings (e.g., coordination breakdown, inequity in conduct of care) and modest for within-axis measures like active time.
  • A useful benchmark must span axes, include multi-site data, and compare joint representations against single-axis models, ensembles, and hand-engineered baselines on identical raw inputs.
  • The representation must preserve time as a carrier of clinical meaning—relative intervals, event order, encounter and session boundaries—and should travel across sites, so a site with little annotation can inherit a working signal.
  • Governance should treat individual-evaluation uses (performance, discipline, productivity scoring) as high-risk and separate from system-learning uses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editor inference: A concrete testable extension would be to construct a two-task benchmark on the same stream—one within-axis (e.g., clinician active time) and one cross-axis (e.g., handoff integrity)—and check whether a pretrained model's gain over single-axis baselines grows with cross-axis dependence; the paper motivates this but does not run it.
  • Editor inference: If the multi-axial reading is right, audit logs could serve as an early behavioral marker for disparities in care delivery, catching inequity in timing, attention, and escalation before it surfaces in outcomes—a use the paper flags but leaves at the level of a capability row rather than a measurement protocol.
  • Editor inference: The transferability premise, if it holds, would make audit-log representations a kind of infrastructure for cross-institution workflow benchmarking, enabling sites to compare coordination quality rather than only prediction performance; this goes beyond the paper's stated benchmark to imply a quality-improvement use.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This Perspective argues that EHR audit logs are best understood as multi-axial event streams: each logged action simultaneously belongs to a clinician's work session, a patient's trajectory, a care team's activity, and a clinical process. The authors illustrate the claim with a stroke-alert example (Section 2), then use it to motivate foundation-model pretraining over the raw stream (Section 3), outline implementation and benchmark requirements (Section 4), and discuss governance and equity (Section 5). The paper's central empirical prediction is that pretraining value should concentrate in outcomes whose signal is spread across multiple axes rather than contained within one, and it calls for a shared multi-site benchmark to test this. No new experiments are reported; the contribution is conceptual and programmatic.

Significance. If the multi-axial framing is accepted, it provides a useful organizing vocabulary for a fragmented literature: audit-log research on clinician workload, patient outcomes, teamwork, and workflow currently reconstructs different views from the same stream, and the paper shows that these views share a substrate. The worked example in Section 2 is effective, the governance discussion is appropriately cautious, and the proposed benchmark idea is a constructive next step. The paper is also explicit that the needed evidence is absent, which is a strength, and it makes a falsifiable prediction in principle. The main limitation is that the central predictive claim—that pretraining is most advantageous for cross-axis outcomes—is not yet operationally defined, and the cross-site transfer assumption rests on an analogy to structured-EHR foundation models rather than audit-log-specific evidence.

major comments (3)
  1. [Abstract; §3; §6] The central prediction is not operationally defined. The paper repeatedly claims that pretraining's advantage 'should be greatest for outcomes whose signal is distributed across multiple orderings of the stream rather than contained within one,' and §4 says a benchmark should grade tasks by their 'dependence on cross-axis structure.' But no definition is given of what it means for a task's signal to depend on cross-axis structure, nor how dependence would be measured. Without such a definition, the prediction can accommodate almost any positive pretraining result: tasks that benefit from longer context can be labeled cross-axis after the fact. I agree with the stress-test concern that the prediction is not currently falsifiable. Please specify a mechanism or measurement—e.g., comparing a joint representation against single-axis baselines under controlled event ordering, or defining cross
  2. [§4, second paragraph] The representation 'ought to travel across sites' is asserted from an analogy to [26], which is a multi-center study of structured EHR foundation models, not of audit logs. Audit-log action vocabularies, screen names, workflows, and configurations are site-specific in ways that may break transfer even if the multi-axial reading is correct. This is an empirical assumption, not an implication of the framing. Please either state it explicitly as a testable hypothesis with criteria for success in the proposed benchmark, or provide audit-log-specific preliminary evidence; otherwise the §4 discussion overstates what the analogy to [26] supports.
  3. [§2 and Table 1] The four axes are not defined formally enough to sustain the benchmark proposal. 'Clinician' and 'patient' are per-entity orderings, while 'team' and 'process' appear to be aggregate relational patterns inferred across events; it is not clear how an event is assigned to each axis or whether axis membership is task-dependent. This matters because the whole program depends on distinguishing within-axis from cross-axis structure. Please give a more precise definition of an axis (e.g., a projection of the event stream onto a sequence) and a rule for when an event participates in an axis, or acknowledge that axis labels are analytic constructs chosen for particular questions.
minor comments (4)
  1. [Abstract; §6] The term 'prediction' is used for what is, at this stage, a hypothesis about where pretraining value concentrates. Consider softening to 'hypothesis' or adding a sentence that the prediction is prospective and would need to be operationalized before empirical evaluation.
  2. [Figure 1] Panel (b) would benefit from a more explicit indication of which row is highlighted and how the markers were assigned. In the text-only rendering the reading is not obvious.
  3. [§5] The phrase 'system-learning uses' is not defined; consider defining it or using 'system-level learning uses' and clarifying the distinction from individual-evaluation uses.
  4. [General] The paper has no limitations section. Given the acknowledged absence of audit-log-specific transfer evidence and the underspecification of the key prediction, a brief limitations paragraph would make the perspective more balanced.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the paper is a conceptual perspective whose central claims are testable hypotheses, not derivations from fitted inputs; only a minor non-load-bearing self-citation appears.

full rationale

The paper is a Perspective, not an empirical derivation. It contains no equations, no fitted parameters, and no result that is later relabeled as a prediction. The central claim—that audit logs are multi-axial—is introduced as a proposed reading: 'We use axis to mean a way of ordering events in the stream from a single vantage point... We call audit logs multi-axial because a single event can be ordered in several ways at once' (Section 2). This is a conceptual definition, not a circular derivation. The predictive claim about pretraining is explicitly presented as something a future benchmark would test: 'Its advantage should be greatest on the questions that remain hard precisely because they cross axes or lack labels, and that prediction is what a shared, multi-site benchmark would test' (Section 6). It is not fitted to any subset of data or imported from a self-citation. The only author self-citation is [7] (Apathy et al.), used in Section 1 as one example of documentation-burden measurement; the multi-axial framing and the pretraining hypothesis do not depend on it. A possible concern that 'cross-axis dependence' is not operationally defined is a falsifiability or correctness issue, not circularity, because the paper does not use any benchmark result as evidence for its own claims.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No parameters are fitted anywhere in the paper. The central claim rests on conceptual assumptions: the existence of the four axes as the privileged clinically meaningful orderings, the scalability/learnability of audit-log streams, and the transferability of a shared representation. The last two are argued by analogy, not demonstrated.

assumptions (4)
  • domain assumption Audit logs are generated continuously and at scale, far exceeding the number of available labels.
    Section 3 states this as the first condition making foundation-model pretraining worth considering; no quantitative characterization or citation supports it.
  • ad hoc to paper The four orderings (clinician, patient, team, process) are the clinically meaningful ones through which care is decided, delivered, and received.
    Introduced in Section 2 as the organizing choice; the paper notes other orderings (workstation, chart object, session) exist but privileges these four without derivation.
  • domain assumption A representation pretrained over the raw stream transfers across sites, giving sites with little annotation a working signal.
    Section 4 relies on analogy with shared structured-EHR foundation models [26]; the clickstream-specific variant is untested.
  • ad hoc to paper Pretraining value concentrates on cross-axis targets rather than within-axis ones.
    Sections 3 and 6 present this as a prediction implied by multi-axiality; it is a conjecture that the proposed benchmark would test, not an established result.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Clinical Audit Logs as Multi-Axial Traces of Care Delivery." pith.science (2026). https://pith.science/paper/QOVXJ4YW

@misc{pith2026260715397,
  author       = {Pith},
  title        = {Pith review of: Clinical Audit Logs as Multi-Axial Traces of Care Delivery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QOVXJ4YW}},
  note         = {Machine review of arXiv:2607.15397}
}
read the original abstract

Electronic health record audit logs record timestamped actions through which clinical work is carried out. Generated as operational metadata, they now support research on clinician effort, patient outcomes, care-team coordination, and workflow structure. This Perspective explains that breadth by articulating audit logs as multi-axial event streams and drawing implications for representation learning, evaluation, and governance. Each logged action belongs simultaneously to multiple clinically meaningful relations: a clinician's work, a patient's trajectory, a team's activity, and a recurring workflow. This structure motivates foundation-model pretraining to learn reusable representations over the raw stream. Reading audit logs as multi-axial traces specifies what such representations must preserve, how their value should be tested, and how their use should be governed.

Figures

Figures reproduced from arXiv: 2607.15397 by the authors.

Figure 1
Figure 1. Audit logs as a multi-axial substrate. We propose that a single logged action belongs to several clinically meaningful relations at once. (a) A stroke-alert episode in which one anchor event is read along four axes: a step in the physician’s session, a moment in the patient’s evaluation after the alert and before treatment, one access within a window of overlapping chart use, and the imaging-review step of a time-se… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

27 extracted references · 1 canonical work pages

  1. [26]

    A multi-center study on the adaptability of a shared foundation model for electronic health records.npj Digit Med, 7:171, 2024

    Lin Lawrence Guo, Jason Fries, Ethan Steinberg, Scott Lanyon Fleming, Keith Morse, Cather- ine Aftandilian, Jose Posada, Nigam Shah, and Lillian Sung. A multi-center study on the adaptability of a shared foundation model for electronic health records.npj Digit Med, 7:171, 2024. 7 (a) (b) Patient Clinician Team Process Open Chart BPA Triggered Order Entry ...

    2024

  2. [1]

    Adelman, Ming Tai-Seale, Vimla L

    Julia Adler-Milstein, Jason S. Adelman, Ming Tai-Seale, Vimla L. Patel, and Chris Dymek. EHR audit logs: A new goldmine for health services research?J Biomed Inform, 101:103343,

  3. [2]

    Chiang, and Michelle R

    Adam Rule, Michael F. Chiang, and Michelle R. Hribar. Using electronic health record audit logs to study clinical activity: A systematic review of aims, measures, and methods.J Am Med Inform Assoc, 27(3):480–490, 2020

    2020

  4. [3]

    Using electronic health record audit log data for research: Insights from early efforts.J Am Med Inform Assoc, 30(1):167–171, 2023

    Thomas Kannampallil and Julia Adler-Milstein. Using electronic health record audit log data for research: Insights from early efforts.J Am Med Inform Assoc, 30(1):167–171, 2023

    2023

  5. [4]

    Malin, Mayur B

    Xinmeng Zhang, Chao Yan, Bradley A. Malin, Mayur B. Patel, and You Chen. Predicting next-day discharge via electronic health record access logs.J Am Med Inform Assoc, 28(12): 2670–2680, 2021

    2021

  6. [5]

    Rossetti, Patricia C

    Sarah C. Rossetti, Patricia C. Dykes, Chris Knaplund, Sandy Cho, Jennifer Withall, Graham Lowenthal, David Albers, Rachel Y. Lee, Haomiao Jia, Suzanne Bakken, et al. Real-time surveillance system for patient deterioration: A pragmatic cluster-randomized controlled trial. Nat Med, 31(6):1895–1902, 2025

    1902

  7. [6]

    Warner, Daphne Lew, Sunny S

    Seunghwan Kim, Benjamin C. Warner, Daphne Lew, Sunny S. Lou, and Thomas Kannampallil. Measuring cognitive effort using tabular transformer-based language models of electronic health record-based audit log action sequences.J Am Med Inform Assoc, 31(10):2228–2235, 2024

    2024

  8. [7]

    i had not time to make it shorter

    Nate C. Apathy, Allison J. Hare, Sara Fendrich, and Dori A. Cross. “i had not time to make it shorter”: An exploratory analysis of how physicians reduce note length and time in notes.J Am Med Inform Assoc, 30(2):355–360, 2023

    2023

Show all 27 references
  1. [8]

    Lorenzi, Warren S

    You Chen, Nancy M. Lorenzi, Warren S. Sandberg, Kelly Wolgast, and Bradley A. Malin. Identifying collaborative care teams through electronic medical record utilization patterns.J Am Med Inform Assoc, 24(e1):e111–e120, 2017

    2017

  2. [9]

    Lee, Hannah C

    Olga Yakusheva, Lara Khadr, Kathryn A. Lee, Hannah C. Ratliff, Deanna J. Marriott, and Deena Kelly Costa. An electronic health record metadata-mining approach to identifying 5 patient-level interprofessional clinician teams in the intensive care unit.J Am Med Inform Assoc, 32(...

    2025

  3. [10]

    Hribar, Sarah Read-Brown, Isaac H

    Michelle R. Hribar, Sarah Read-Brown, Isaac H. Goldstein, Leah G. Reznick, Lorinna Lombardi, Mansi Parikh, Winston Chamberlain, and Michael F. Chiang. Secondary use of electronic health record data for clinical workflow analysis.J Am Med Inform Assoc, 25(1):40–46, 2018

    2018

  4. [11]

    Inferring EHR utilization workflows through audit logs

    Xinmeng Zhang, Yuying Zhao, Chao Yan, Tyler Derr, and You Chen. Inferring EHR utilization workflows through audit logs. InAMIA Annu Symp Proc, volume 2022, pages 1247–1256, 2023

    2022

  5. [12]

    Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S

    Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models.arXiv, 2108.07258, 2021. https: //arxiv.org/abs/2108.0725...

    2021 arXiv

  6. [13]

    Fries, Conor K

    Ethan Steinberg, Ken Jung, Jason A. Fries, Conor K. Corbin, Stephen R. Pfohl, and Nigam H. Shah. Language models are an effective representation learning technique for electronic health record data.J Biomed Inform, 113:103637, 2021

    2021

  7. [14]

    Kannampallil, Julia Adler-Milstein, and You Chen

    Chao Yan, Xinmeng Zhang, Thomas G. Kannampallil, Julia Adler-Milstein, and You Chen. Reimagining clinical AI: From clickstreams to clinical insights with EHR use metadata.npj Health Syst, 2(1):33, 2025

    2025

  8. [15]

    Jay Holmgren, Sunny S

    Daniel Tawfik, Adam Rule, Aram Alexanian, Dori Cross, A. Jay Holmgren, Sunny S. Lou, et al. Emerging domains for measuring health care delivery with electronic health record metadata. J Med Internet Res, 27:e64721, 2025

    2025

  9. [16]

    Lou, and Thomas Kannampallil

    Brian Bartek, Sunny S. Lou, and Thomas Kannampallil. Measuring the cognitive effort associated with task switching in routine EHR-based tasks.J Biomed Inform, 141:104349, 2023

    2023

  10. [17]

    Clancy, David Schlessinger, et al

    Christian Rose, Robert Thombley, Morteza Noshad, Yun Lu, Heather A. Clancy, David Schlessinger, et al. Team is brain: Leveraging EHR audit log data for new insights into acute care processes.J Am Med Inform Assoc, 30(1):8–15, 2023

    2023

  11. [18]

    Malin, and You Chen

    Xinmeng Zhang, Chao Yan, Yuyang Yang, Zhuohang Li, Yubo Feng, Bradley A. Malin, and You Chen. Optimizing large language models for discharge prediction: Best practices in leveraging electronic health record audit logs. InAMIA Annu Symp Proc, volume 2024, pages 1323–1331, 2025

    2024

  12. [19]

    Lou, Benjamin C

    Hanyang Liu, Sunny S. Lou, Benjamin C. Warner, Derek R. Harford, Thomas Kannampallil, and Chenyang Lu. HiPAL: A deep framework for physician burnout prediction using activity logs in electronic health records. InProceedings of the 28th ACM SIGKDD Conference on Knowledge Discov...

    2022

  13. [20]

    Lou, Hanyang Liu, Benjamin C

    Sunny S. Lou, Hanyang Liu, Benjamin C. Warner, Derek Harford, Chenyang Lu, and Thomas Kannampallil. Predicting physician burnout using clinical activity logs: Model performance and lessons learned.J Biomed Inform, 127:104015, 2022

    2022

  14. [21]

    Lou, Hanyang Liu, Derek Harford, Chenyang Lu, and Thomas Kannampallil

    Sunny S. Lou, Hanyang Liu, Derek Harford, Chenyang Lu, and Thomas Kannampallil. Char- acterizing the macrostructure of electronic health record work using raw audit logs: An unsupervised action embeddings approach.J Am Med Inform Assoc, 30(3):539–544, 2023. 6

    2023

  15. [22]

    A language model framework for sequence modeling of EHR audit logs to characterize clinician-EHR interactions.medRxiv, 2026

    Seunghwan Kim, Sunny Lou, Adam Cobb, Susmit Jha, and Thomas Kannampallil. A language model framework for sequence modeling of EHR audit logs to characterize clinician-EHR interactions.medRxiv, 2026. doi: 10.64898/2026.06.24.26356449. Preprint. https://doi.org/ 10.64898/2026.06...

    2026 doi

  16. [23]

    Fries, Yizhe Xu, and Nigam H

    Ethan Steinberg, Jason A. Fries, Yizhe Xu, and Nigam H. Shah. MOTOR: A time-to-event foundation model for structured medical records. InInternational Conference on Learning Representations, 2024. https://openreview.net/forum?id=NialiwI2V6 (accessed Jun 2026)

    2024

  17. [24]

    PinFM: Foundation model for user activity sequences at a billion-scale visual discovery platform

    Xiangyi Chen, Kousik Rajesh, Matthew Lawhon, Zelun Wang, Hanyu Li, Haomiao Li, et al. PinFM: Foundation model for user activity sequences at a billion-scale visual discovery platform. arXiv, 2507.12704, 2025.https://arxiv.org/abs/2507.12704(accessed Jun 2026)

    2025 arXiv

  18. [25]

    BehaveGPT: A foundation model for large-scale user behavior modeling.arXiv, 2505.17631, 2025.https://arxiv.org/abs/2505.17631(accessed Jun 2026)

    Jiahui Gong, Jingtao Ding, Fanjin Meng, Chen Yang, Hong Chen, Zuojian Wang, et al. BehaveGPT: A foundation model for large-scale user behavior modeling.arXiv, 2505.17631, 2025.https://arxiv.org/abs/2505.17631(accessed Jun 2026)

    2025 arXiv

  19. [2020]

    doi: 10.1016/j.jbi.2019.103343

    2019

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.