Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Measuring What Matters: A Framework for Evaluating Safety Risks in Real-World LLM Applications

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that LLM safety evaluation must happen at the application level, not just on foundation models.

desk verdict A genuinely useful practitioner's framework for application-level LLM safety evaluation; the abstract's 'validated' claim outruns the evidence, but the framework itself is worth review. read the letter →

arxiv 2507.09820 v1 pith:OD7T7KHG submitted 2025-07-13 cs.SE cs.CY

Jia Yi Goh , Shaun Khoo , Nyx Iskandar , Gabriel Chua , Leanne Tan , Jessica Foo This is my paper
classification cs.SEcs.CY
keywords LLMapplicationsafetyrisktaxonomyblack-boxevaluationrefusaldetectionadversarialpromptsapplication-leveltestingAIgovernance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language model safety is usually judged on foundation models in isolation, but this paper claims the real risks live in the full application, where system prompts, retrieval pipelines, fine-tuning, and guardrails can each change what is unsafe. It proposes a two-part framework for application-level safety evaluation: principles for building a risk taxonomy customized to an organization's context, and practices for testing the whole application as a black box with adversarial prompts. In an internal pilot with two external-facing chatbots and a two-level benchmark of 1,600 basic and 33,600 intermediate prompts, the framework used refusal to respond as the working definition of a safe outcome and surfaced emerging risks before public release. The paper's aim is to give other organizations a repeatable, continuous method for assessing their own LLM deployments instead of relying on generic foundation-model benchmarks.

What carries the argument

The key machinery is the pairing of a customized safety risk taxonomy with a black-box testing loop. The loop treats the entire application as a single API, probes it with curated adversarial prompts, and turns each response into a binary safe-or-unsafe verdict using refusal as the safety proxy, then aggregates those verdicts into a safety score that can be sliced by risk category, subcategory, and severity level.

What would settle it

Run the same pipeline on two versions of one chatbot, one with a system prompt engineered to encourage harmful outputs and one hardened against them; if the safety score stays flat across the two versions, the method cannot detect the very application-level risk it is meant to measure.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that a usable measure of application-level safety can be produced from outside the system: build a taxonomy fitted to the organization's risks and legal context, curate adversarial prompts that are meaningful, diverse, localized, and incrementally complex, run them through a single application endpoint as a black box, judge each response as safe or unsafe with refusal as the conservative proxy, and aggregate the share of safe responses into a safety score. That score is explicitly an empirical estimate, not a guarantee, and the paper reports that in its pilot the process found weaknesses early enough for developers to harden the chatbots before launch. The authors treat this as evidence that safety evaluation does not have to be left to foundation-model benchmarks and can be adapted by other organizations to their own risk priorities.

Load-bearing premise

The framework's generalizability rests on an internal pilot of two chatbots whose results are described only qualitatively, with no safety scores or evaluator accuracy reported, so the assumption that the pilot represents other settings is unquantified.

Editorial extensions

If this is right

  • Organizations can replace generic safety benchmarks with context-specific taxonomies, and the paper's three risk categories for a public-sector setting (undesirable content, specialized advice, political content) show what such a taxonomy looks like.
  • Because the application is scored as a single black-box endpoint, the same evaluation pipeline can be reused across different applications, model providers, and internal component configurations.
  • The safety score gives development teams a baseline to track over time, with results aggregated by risk category so mitigations can be aimed at the specific vulnerabilities the tests expose.
  • The framework is designed to extend beyond chatbots, and the paper lists multi-turn, multilingual, multimodal, and automated red-teaming as planned directions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension of the paper's logic is to run the same black-box pipeline against a matched foundation-model benchmark; the gap between the two safety scores would quantify how much application components actually shift measured risk.
  • The refusal-as-safety-proxy is clearest for assistants where the correct safe behavior is to decline; for applications expected to redirect or provide safe alternatives, organizations would need a different evaluator, a boundary the paper leaves implicit.
  • The paper's own caveat that a perfect score does not imply zero risk suggests the safety score is best used as a relative baseline for internal improvement, not as an absolute certification for external stakeholders.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a framework for application-level safety evaluation of LLM systems, consisting of (1) principles for developing customized safety risk taxonomies and (2) practices for black-box safety testing of LLM applications. The framework is illustrated through an internal pilot at a government agency involving two external-facing chatbots and a two-level internal benchmark. The abstract claims the framework is 'validated through real-world deployment,' while the paper also positions the work as a reference point for other organizations.

Significance. If the framework is adopted, it offers a practical alternative to foundation-model-only safety benchmarks, with explicit attention to organizational context, regulatory alignment, and black-box end-to-end evaluation. The taxonomy examples and prompt-curation guidance in the appendices are concrete and actionable, and the paper's emphasis on evaluating the evaluator and acknowledging the limits of safety scores (Section 4.4) reflects good practice. However, the central validation claim is not currently supported by quantitative evidence from the pilot, which substantially weakens the contribution as a validated framework rather than a set of plausible guidelines.

major comments (3)
  1. [Abstract, Section 1, Section 4.5] The claim that the framework is 'validated through real-world deployment' is not supported by the reported case study. Section 4.5 describes the pilot setup (two chatbots, 1,600 basic and 33,600 intermediate prompts) but provides no safety scores, no refusal rates, no per-risk results, no evaluator accuracy, and no comparison with baselines. The only outcomes are qualitative ('notably different responses', 'surfaced emerging risks'). Without any quantitative results, the reader cannot assess whether the framework produces reliable measurements or whether the pilot actually improved safety. This is load-bearing because validation is a stated central contribution.
  2. [Section 4.3 vs Section 4.5] The paper states in Section 4.3 that 'organizations should evaluate the evaluator, ideally against human annotations, as its accuracy directly affects the reliability of LLM application safety assessments.' The pilot in Section 4.5 uses LLM-as-a-judge for refusal detection but reports no validation of that judge against human annotations. This is an internal inconsistency: the framework's own requirement is not followed in the demonstration, so the pilot does not illustrate the framework as specified. The authors should either report the evaluator validation or explicitly state that it was omitted and discuss the implications.
  3. [Section 4.4] The safety score is defined as the proportion of safe responses, and Section 4.4 appropriately notes that a perfect score does not imply zero risk because of evaluator error. However, the paper never reports the score, its uncertainty, or the evaluator's error rate for the pilot. Given the definition of the metric, the absence of any measured value or error analysis leaves the framework's core quantitative output unillustrated. Reporting at least aggregate scores with breakdowns by risk category, along with evaluator agreement or confidence intervals, would demonstrate the metric's practical utility.
minor comments (5)
  1. [Section 3.3] Figures 1 and 2 are referenced in the text but do not appear in the manuscript; the figures (taxonomy diagram and hateful risk subcategory definition) should be embedded and legible.
  2. [Section 4.5] The two external-facing chatbot applications are not described, making it difficult to interpret the claim that the framework generalizes; consider adding a sentence describing their purpose, intended users, and system configuration.
  3. [Appendix B.3] The reference to the 'Workplace Fairness Act 2025 [No. 8 of 2025]' is formatted inconsistently with the other references; unify the citation style.
  4. [Appendix C] Appendix C lists refusal evaluation methods but does not specify the exact LLM-as-a-judge prompt or the judge model used in the pilot; providing these details would improve reproducibility.
  5. [Section 4.1] The bullet 'Incrementally Complex' is capitalized inconsistently with the other bullets ('Meaningful', 'Diverse', 'Contextualized'); fix capitalization.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the paper is a guidance framework with no fitted parameters or derivations that reduce to inputs; the only self-referential element is the 'validated through real-world deployment' claim, which rests on an unquantified internal pilot and is an evidentiary gap rather than a circular reduction.

full rationale

This paper is a framework/guidance paper rather than a derivation. It contains no fitted parameters, no equations whose outputs are defined by their inputs, and no imported uniqueness theorems. The taxonomy principles in Section 3 draw on external references (MIT AI Risk Repository, NIST AI RMF, EU AI Act, Council of Europe Framework Convention), and the evaluation practices in Section 4 reference external tools and methods (Garak, Inspect, Moonshot, Llama Guard, WildGuard, G-Eval, Alt-Test). The organization-specific taxonomy in Section 3.3 and Appendix A is an application of the proposed principles, not a derived prediction. The safety score in Section 4.4 is explicitly defined as 'the proportion of safe responses out of the total number of prompts it was probed with' and is called 'simply' that proportion; this is a definition of a metric, not a claimed derivation. The abstract's phrase 'validated through real-world deployment across multiple use cases within our organization' relies on the authors' own internal pilot (Section 4.5), which reports no safety scores, no evaluator accuracy, no baselines, and no quantitative results. The paper itself states in Section 4.3 that organizations 'should evaluate the evaluator, ideally against human annotations, as its accuracy directly affects the reliability of LLM application safety assessments,' but Section 4.5 does not report such validation. This is a real evidentiary weakness in the validation claim, but it is not circularity: the framework's content does not reduce by construction to the pilot's outputs. The pilot is a self-reported illustration, and the framework is grounded in external resources, so the score is low rather than zero only because the central 'validated' claim is self-referential in its evidence base.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters or invented theoretical entities. The central claim rests on domain assumptions about what safety measurement means and on the unquantified internal pilot as validation evidence.

assumptions (4)
  • domain assumption LLM application safety cannot be theoretically guaranteed, making empirical probing the only practical assurance.
    Stated in Section 4 as a 'well-accepted fact' and used to justify safety testing as an empirical exercise.
  • domain assumption Black-box evaluation through a single API endpoint reflects real-world user interaction.
    Section 4.2 assumes that treating the application as a black box mirrors how actual users interact.
  • domain assumption The proportion of safe responses to the curated prompt set is a meaningful measure of application safety.
    Section 4.4 defines safety score as the proportion of safe responses, but no evidence connects this to real-world safety outcomes.
  • ad hoc to paper The internal two-chatbot pilot is representative enough to validate a generalizable framework.
    Section 4.5 uses two specific chatbots and one benchmark with no reported results; the validation claim depends on this pilot.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Measuring What Matters: A Framework for Evaluating Safety Risks in Real-World LLM Applications." pith.science (2026). https://pith.science/paper/OD7T7KHG

@misc{pith2026250709820,
  author       = {Pith},
  title        = {Pith review of: Measuring What Matters: A Framework for Evaluating Safety Risks in Real-World LLM Applications},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OD7T7KHG}},
  note         = {Machine review of arXiv:2507.09820}
}
read the original abstract

Most safety testing efforts for large language models (LLMs) today focus on evaluating foundation models. However, there is a growing need to evaluate safety at the application level, as components such as system prompts, retrieval pipelines, and guardrails introduce additional factors that significantly influence the overall safety of LLM applications. In this paper, we introduce a practical framework for evaluating application-level safety in LLM systems, validated through real-world deployment across multiple use cases within our organization. The framework consists of two parts: (1) principles for developing customized safety risk taxonomies, and (2) practices for evaluating safety risks in LLM applications. We illustrate how the proposed framework was applied in our internal pilot, providing a reference point for organizations seeking to scale their safety testing efforts. This work aims to bridge the gap between theoretical concepts in AI safety and the operational realities of safeguarding LLM applications in practice, offering actionable guidance for safe and scalable deployment.

Figures

Figures reproduced from arXiv: 2507.09820 by the authors.

Figure 1
Figure 1. Our organization’s taxonomy of safety risks We identified three primary categories of harm to prioritize: (1) Undesirable content, which may cause psychological harm to individuals or reputational damage to the organiza￾tion; (2) Specialized advice, which may lead to physical or financial harm if users act on unqualified guidance, espe￾cially given the perceived authority of government chatbots; and (3) Political co… view at source ↗
Figure 2
Figure 2. Our organization’s hateful risk subcategory definition Some risk subcategories are further broken down by severity levels, as illustrated in the case of hateful content in Fig￾ure 2. This added granularity supports more proportionate consequences; for example, discriminatory language may warrant a warning, while outright hate speech could result in an immediate ban. To support consistent interpretation of these cate… view at source ↗
Figure 3
Figure 3. Overview of the safety testing pipeline for LLM applications manageable for manual review if needed. Starting small and expanding iteratively is often the most practical approach. The prompt distribution does not need to be uniform across risks, as this variation may actually reflect real-world preva￾lence and severity, though care should be taken to avoid excessive data imbalance that could skew evaluation. 4.2. Au… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ROK-FORTRESS: Measuring the Effect of Geopolitical Transcreation for National Security and Public Safety

    cs.CL 2026-05 unverdicted novelty 7.0 of 10

    ROK-FORTRESS shows Korean-language prompts increase LLM safety suppression compared with English, while Korean geopolitical grounding often reduces that suppression, indicating translation-only evaluations miss langua...

Reference graph

Works this paper leans on

32 extracted references · 19 canonical work pages · cited by 1 Pith paper

  1. [1]

    8 of 2025

    Workplace fairness act 2024, no. 8 of 2025. https://sso.agc.gov.sg/Act/WFA2025/Uncommenced/20250214020905?DocDate=20250213, 2025. s. 8(1)

    work page arXiv 2024

  2. [2]

    Inspect AI: Framework for Large Language Model Evaluations

    AI Security Institute, U. Inspect AI: Framework for Large Language Model Evaluations . URL https://github.com/UKGovernmentBEIS/inspect_ai

    work page

  3. [3]

    Standards database

    AI Standards Hub . Standards database. URL https://aistandardshub.org/ai-standards-search/

    work page

  4. [4]

    Model ai governance framework for generative ai, 2024

    AI Verify Foundation . Model ai governance framework for generative ai, 2024. URL https://aiverifyfoundation.sg/resources/mgf-gen-ai/. Accessed: 2025-05-10

    work page 2024

  5. [5]

    Ai verify moonshot

    AI Verify Foundation, S. Ai verify moonshot. URL https://aiverify-foundation.github.io/moonshot/

    work page

  6. [6]

    RAG LLM s are not safer: A safety analysis of retrieval-augmented generation for large language models

    An, B., Zhang, S., and Dredze, M. RAG LLM s are not safer: A safety analysis of retrieval-augmented generation for large language models. In Chiruzzo, L., Ritter, A., and Wang, L. (eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long P...

    work page 2025

  7. [7]

    The alternative annotator test for llm-as-a-judge: How to statistically justify replacing human annotators with llms, 2025

    Calderon, N., Reichart, R., and Dror, R. The alternative annotator test for llm-as-a-judge: How to statistically justify replacing human annotators with llms, 2025. URL https://arxiv.org/abs/2501.10970

    arXiv 2025

  8. [8]

    Framework convention on artificial intelligence and human rights, democracy and the rule of law

    Council of Europe . Framework convention on artificial intelligence and human rights, democracy and the rule of law. https://www.coe.int/en/web/artificial-intelligence/the-framework-convention-on-artificial-intelligence, 2024. Accessed: 2025-06-07

    work page 2024

Show all 32 references
  1. [9]

    Investigating data contamination in modern benchmarks for large language models

    Deng, C., Zhao, Y., Tang, X., Gerstein, M., and Cohan, A. Investigating data contamination in modern benchmarks for large language models. In Duh, K., Gomez, H., and Bethard, S. (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Com...

    2024 doi

  2. [10]

    garak: A framework for security probing large language models, 2024

    Derczynski, L., Galinkin, E., Martin, J., Majumdar, S., and Inie, N. garak: A framework for security probing large language models, 2024. URL https://arxiv.org/abs/2406.11036

    2024 arXiv

  3. [11]

    Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act)

    European Parliament and Council . Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) . Official Journal of the European Union L 2024/1689, 12 July 2024, 2...

    2024

  4. [12]

    Ai guardian: Safeguarding ai applications for singapore's public sector

    GovTech Singapore . Ai guardian: Safeguarding ai applications for singapore's public sector. https://www.aiguardian.gov.sg/, 2025. URL https://www.aiguardian.gov.sg/. Accessed 9 July 2025

    2025

  5. [13]

    Y., Lambert, N., Choi, Y., and Dziri, N

    Han, S., Rao, K., Ettinger, A., Jiang, L., Lin, B. Y., Lambert, N., Choi, Y., and Dziri, N. Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms. In Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., and Zhang, C....

    2024

  6. [14]

    Llama guard: Llm-based input-output safeguard for human-ai conversations, 2023

    Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y., Tontchev, M., Hu, Q., Fuller, B., Testuggine, D., and Khabsa, M. Llama guard: Llm-based input-output safeguard for human-ai conversations, 2023. URL https://arxiv.org/abs/2312.06674

    2023 arXiv

  7. [15]

    SALAD -bench: A hierarchical and comprehensive safety benchmark for large language models

    Li, L., Dong, B., Wang, R., Hu, X., Zuo, W., Lin, D., Qiao, Y., and Shao, J. SALAD -bench: A hierarchical and comprehensive safety benchmark for large language models. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Findings of the Association for Computational Linguistics...

    2024 doi

  8. [16]

    G -eval: NLG evaluation using gpt-4 with better human alignment

    Liu, Y., Iter, D., Xu, Y., Wang, S., Xu, R., and Zhu, C. G -eval: NLG evaluation using gpt-4 with better human alignment. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp.\ 2511--2522, Sin...

    2023 doi

  9. [17]

    Longsafety: Evaluating long-context safety of large language models, 2025

    Lu, Y., Cheng, J., Zhang, Z., Cui, S., Wang, C., Gu, X., Dong, Y., Tang, J., Wang, H., and Huang, M. Longsafety: Evaluating long-context safety of large language models, 2025. URL https://arxiv.org/abs/2502.16971

    2025 arXiv

  10. [18]

    Harmbench: a standardized evaluation framework for automated red teaming and robust refusal

    Mazeika, M., Phan, L., Yin, X., Zou, A., Wang, Z., Mu, N., Sakhaee, E., Li, N., Basart, S., Li, B., Forsyth, D., and Hendrycks, D. Harmbench: a standardized evaluation framework for automated red teaming and robust refusal. In Proceedings of the 41st International Conference o...

    2024

  11. [19]

    Nist ai risk management framework playbook, 2023

    National Institute of Standards and Technology . Nist ai risk management framework playbook, 2023. URL https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook. Accessed: 2025-05-11

    2023

  12. [20]

    Moderation guide, 2025

    OpenAI. Moderation guide, 2025. URL https://platform.openai.com/docs/guides/moderation

    2025

  13. [21]

    Fine-tuned distilroberta-base for rejection in the output detection, 2024

    ProtectAI.com . Fine-tuned distilroberta-base for rejection in the output detection, 2024. URL https://huggingface.co/ProtectAI/distilroberta-base-rejection-v1. Accessed: 2025-05-11

    2024

  14. [22]

    Fine-tuning aligned language models compromises safety, even when users do not intend to!, 2023

    Qi, X., Zeng, Y., Xie, T., Chen, P.-Y., Jia, R., Mittal, P., and Henderson, P. Fine-tuning aligned language models compromises safety, even when users do not intend to!, 2023. URL https://arxiv.org/abs/2310.03693

    2023 arXiv

  15. [23]

    Roose, K. Can a.i. be blamed for a teen’s suicide?, 2024. URL https://www.nytimes.com/2024/10/23/technology/characterai-lawsuit-teen-suicide.html. The New York Times, Accessed: 2025-05-11

    2024

  16. [24]

    do anything now

    Shen, X., Chen, Z., Backes, M., Shen, Y., and Zhang, Y. "do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models, 2024. URL https://arxiv.org/abs/2308.03825

    2024 arXiv

  17. [25]

    K., Grundy, E

    Slattery, P., Saeri, A. K., Grundy, E. A. C., Graham, J., Noetel, M., Uuk, R., Dao, J., Pour, S., Casper, S., and Thompson, N. The ai risk repository: A comprehensive meta-review, database, and taxonomy of risks from artificial intelligence, 2025. URL https://arxiv.org/abs/2408.12622

    2025 arXiv

  18. [26]

    M., Akinwande, V., Al-Nuaimi, N., Alfaraj, N., Alhajjar, E., Aroyo, L., Bavalatti, T., Bartolo, M., Blili-Hamelin, B., Bollacker, K., Bomassani, R., Boston, M

    Vidgen, B., Agrawal, A., Ahmed, A. M., Akinwande, V., Al-Nuaimi, N., Alfaraj, N., Alhajjar, E., Aroyo, L., Bavalatti, T., Bartolo, M., Blili-Hamelin, B., Bollacker, K., Bomassani, R., Boston, M. F., Campos, S., Chakra, K., Chen, C., Coleman, C., Coudert, Z. D., Derczynski, L.,...

    2024 arXiv

  19. [27]

    Do-not-answer: Evaluating safeguards in LLM s

    Wang, Y., Li, H., Han, X., Nakov, P., and Baldwin, T. Do-not-answer: Evaluating safeguards in LLM s. In Graham, Y. and Purver, M. (eds.), Findings of the Association for Computational Linguistics: EACL 2024, pp.\ 896--911, St. Julian ' s, Malta, March 2024. Association for Com...

    2024

  20. [28]

    On regulating downstream ai developers, 2025

    Williams, S., Schuett, J., and Anderljung, M. On regulating downstream ai developers, 2025. URL https://arxiv.org/abs/2503.11922

    2025

  21. [29]

    Z., Tu, Y., Mai, Y., Klyman, K., Pan, M., Jia, R., Song, D., Liang, P., and Li, B

    Zeng, Y., Yang, Y., Zhou, A., Tan, J. Z., Tu, Y., Mai, Y., Klyman, K., Pan, M., Jia, R., Song, D., Liang, P., and Li, B. AIR - BENCH 2024: A safety benchmark based on regulation and policies specified risk categories. In The Thirteenth International Conference on Learning Repr...

    2024

  22. [30]

    S afety B ench: Evaluating the safety of large language models

    Zhang, Z., Lei, L., Wu, L., Sun, R., Huang, Y., Long, C., Liu, X., Lei, X., Tang, J., and Huang, M. S afety B ench: Evaluating the safety of large language models. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for...

    2024 doi

  23. [31]

    On prompt-driven safeguarding for large language models

    Zheng, C., Yin, F., Zhou, H., Meng, F., Zhou, J., Chang, K.-W., Huang, M., and Peng, N. On prompt-driven safeguarding for large language models. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2024

    2024

  24. [32]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.