Usage Governance Advisor: From Intent to AI Governance

Christopher Giblin; Dhaval Salwala; Elizabeth M. Daly; Frank Bagehorn; Inge Vejsbjerg; Ioana Giurgiu; Luis Garces-Erice; Michael Hind; Mira L. Wolf-Bauwens; Peter Urbanetz

REVIEW 3 major objections 5 minor 1 cited by

Usage Governance Advisor: From Intent to AI Governance

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A pipeline that turns an AI use-case intent into a prioritized risk report, model recommendation, and mitigation plan.

desk verdict A clearly described but thinly evaluated proof-of-concept that integrates risk taxonomy, knowledge graph, and LLM components into an AI governance workflow; the abstract's central claims about risk prioritization and mitigation are not directly tested. read the letter →

arxiv 2412.01957 v2 pith:6AKWBNHA submitted 2024-12-02 cs.AI

Elizabeth M. Daly , Sean Rooney , Seshu Tirupathi , Luis Garces-Erice , Inge Vejsbjerg , Frank Bagehorn , Dhaval Salwala , Christopher Giblin

show 4 more authors

Mira L. Wolf-Bauwens Ioana Giurgiu Michael Hind Peter Urbanetz

This is my paper · ORCID

classification cs.AI

keywords AIgovernanceknowledgegraphriskassessmentlargelanguagemodelsLLM-as-a-judgeChain-of-Thoughtmodelrecommendationmitigationstrategies

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Usage Governance Advisor, a proof-of-concept system that semi-automates AI governance. It takes a user's statement of intent, infers the AI task and associated risks, recommends suitable models and benchmarks, and proposes mitigation actions, all organized around a knowledge graph. The authors claim this integrated workflow is feasible, and they support it with evaluations of the two generative-AI components: knowledge-graph extraction from model documentation reaches F1 around 0.82, and an auto-assist questionnaire reaches accuracy up to 0.915 when users select from suggested answers. If correct, the system lowers the barrier for organizations to assess and document risks before deploying AI systems, helping them comply with regulations such as the EU AI Act.

What carries the argument

The central mechanism is the AI governance knowledge graph, whose ontology is defined in LinkML and includes classes for AI systems, models, datasets, licenses, evaluations, and risks. Risk vocabulary comes from the IBM AI Risk Atlas, with SKOS mappings to NIST, MIT, and OWASP taxonomies. The graph is populated by a generative AI pipeline that classifies documents, chunks them, extracts entities and relationships (simplified to entity pairs with implicit relation types), and uses an LLM-as-a-judge to reconcile matches against ground truth. Risk prioritization uses Chain-of-Thought prompting and LLM-as-a-judge over questionnaire answers; model recommendation normalizes quantitative and categorical risk scores to a common scale; and mitigation recommendation links risks to guardrails and curated action lists.

What would settle it

Run the extraction pipeline on a diverse set of model cards from different vendors and document formats; if triple-level F1 falls well below the reported ~0.82 or shows systematic misses for certain entity types, the knowledge graph and the recommendations that depend on it would inherit those errors, undermining the central claim of practical feasibility.

Watch

Extended reading notes

Core claim

The paper claims that a knowledge graph, populated by generative AI from unstructured technical documentation, can serve as the backbone of an end-to-end AI governance workflow. The system connects a user's intended use case to a risk taxonomy, prioritizes risks via an LLM-as-a-judge over questionnaire answers, recommends models based on normalized evaluation scores, selects relevant benchmarks, and suggests guardrails or manual actions. The authors demonstrate that each step can be automated with current LLMs, with the key measured components—entity/relationship extraction into the knowledge graph and Chain-of-Thought questionnaire completion—performing well enough to be practical, while noting that generative AI cannot yet guarantee full accuracy.

Load-bearing premise

The reliability of the whole governance chain rests on the generative AI pipeline that extracts entities and relationships from unstructured model documentation, which was tested on only one model card and reaches F1 around 0.82, so errors in extraction would cascade into the risk reports, model recommendations, and mitigations built on the knowledge graph.

Editorial extensions

If this is right

If the pipeline is adopted, organizations can produce a documented audit trail—intent, risks, model choices, benchmarks, and mitigations—before deployment, which supports regulatory compliance evidence.
The knowledge graph can be incrementally extended with new models, benchmarks, and risk taxonomies, making governance advice easier to update as regulations and model inventories evolve.
The evaluation suggests that LLM-based extraction, combined with a judge model, is accurate enough to bootstrap a knowledge graph, though human verification remains necessary for low-confidence facts.
The risk-to-benchmark and risk-to-guardrail links enable automated selection of relevant evaluations, reducing the cost of running all benchmarks for every use case.
The mitigation recommender, by associating actions and guardrails with specific risks, moves governance from risk identification toward actionable risk treatment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The authors' simplification of the ontology—promoting properties to entities and ignoring explicit relationship types—likely trades expressiveness for extraction reliability; a more expressive ontology might require more sophisticated extraction and lower F1.
The reported F1 is based on a single model card (granite-8b-code-base-4k); generalizing to other model families or document formats is an untested but plausible extension, and the pipeline's independence from the ontology suggests it could be retargeted with modest effort.
Improving the judge component or adding self-consistency checks could push extraction F1 closer to the overfitted upper bound of 0.92, making the whole governance chain more trustworthy.
The questionnaire accuracy gains from user selection of suggested answers hint that a human-in-the-loop design is key to acceptable performance, which the authors already incorporate.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The genuinely new thing here is the integration: an AI governance pipeline that takes a user intent, runs an auto-assist questionnaire, maps answers to risks, recommends models, suggests evaluations, and proposes mitigations. That end-to-end workflow does not appear in the cited prior work, and the system architecture is described with enough clarity to be reproduced by someone with similar components. The use of a knowledge graph with an evidence graph, and the link to external taxonomies like the AI Risk Atlas and NIST Gen AI Profile, is a sensible design choice. The paper also earns credit for being honest about its scope: it says proof-of-concept up front and acknowledges that generative AI cannot guarantee the validity of extracted facts.

The component evaluations are plausible but narrow. The best generic KG extraction reaches F1 0.82 on a single IBM model card, and the CoT questionnaire beats zero-shot on 42 synthetic intents. Those are reasonable sanity checks, not strong evidence. The stress-test note is right about the biggest gap: the risk-prioritization step (LLM-as-judge mapping question/answer pairs to risks and severity levels) and the mitigation recommender are never evaluated at all. The abstract claims the system 'identifies and prioritizes risks' and 'proposes mitigation strategies,' but no experiment measures whether those inferences are correct. The KG extraction metric also deliberately ignores relationship types, which matters a lot for a governance graph; knowing whether a model 'is trained on' a dataset versus 'has a license' is exactly the kind of distinction risk assessment depends on. So the reported F1 does not establish that risk-relevant relations are captured correctly.

The small sample sizes and absence of error bars are minor relative to that missing end-to-end validation, but they reinforce the verdict: this is a systems paper with a credible architecture and initial component feasibility, not a validated governance tool. The citation pattern looks fine, and the authors do not overclaim in the body, even if the abstract gets ahead of the evidence.

Who is this for? Researchers and practitioners building AI governance tooling who want a concrete integration blueprint. It deserves a serious referee, but more for a systems or industry track than a top-tier AI conference. The authors should be asked to evaluate the risk-prioritization and mitigation outputs, report relationship-type-aware extraction metrics, and ideally release artifacts or run a broader, statistically grounded evaluation. If those additions land, it becomes a genuinely useful reference.

Referee Report

3 major / 5 minor

Summary. The paper presents Usage Governance Advisor, a proof-of-concept system that takes a free-text user intent and produces semi-structured governance information, a prioritized risk list, recommended models, automated risk evaluations, and mitigation actions. The system is organized around an AI governance ontology materialized as a knowledge graph, populated by a generative-AI ingestion pipeline that extracts entity-relationship triples from unstructured model documentation. Additional components include an auto-assist Chain-of-Thought questionnaire, an LLM-as-a-judge risk prioritization step, a model recommender that combines normalized benchmark scores with customer policy, and a mitigation recommender that links risks to guardrails and manually curated action items. The evaluation covers two upstream components: KG triple extraction from a single IBM model card, with best generic-config F1 of 0.82 (Table 1), and auto-assist questionnaire accuracy on 42 synthetic intents, with few-shot/CoT outperforming zero-shot (Table 3). The abstract claims the system 'identifies and prioritizes risks according to the intended use case' and 'proposes mitigation strategies and actions,' but these downstream capabilities are not directly evaluated.

Significance. If the full pipeline were validated, the system would be a valuable integration of knowledge-graph construction, risk identification, and mitigation planning for AI governance, addressing a real need in regulatory compliance and responsible deployment. The paper's strengths include a clearly described LinkML ontology, the use of an LLM-as-a-judge to improve triple matching, a systematic comparison of several extraction pipelines, and evidence that few-shot/CoT improves questionnaire completion. The component evaluations are informative and follow standard metrics, though they are narrow: one model card, 42 synthetic intents, and no validation of the risk inference or mitigation steps. As a proof-of-concept, the work is useful for practitioners, but the evidence falls short of the abstract's claims. The most significant contribution is the articulation of an end-to-end intent-to-governance workflow and the identification of the individual technical challenges involved.

major comments (3)

[Abstract and Analysis] The abstract's central claim that the system 'identifies and prioritizes risks according to the intended use case' and 'proposes mitigation strategies and actions' is never directly evaluated. The Analysis section reports only two component evaluations: KG triple extraction on a single model card (Table 1) and auto-assist questionnaire accuracy on 42 synthetic intents (Table 3). Neither measures whether the inferred risks are correct, whether the severity assignments match expert judgment, whether the model recommendations are appropriate, or whether the mitigation actions are helpful. As a result, the strongest claim in the abstract is unsupported by the presented evidence.
[Entity/Relationship Extraction (KG Construction)] The KG extraction metric deliberately ignores relationship types, with the paper stating 'it does not matter what the relationship type is, only that the two entities in the triple are related.' For a governance knowledge graph, the relationship type is semantically important: knowing that a model 'is trained on' a dataset has different risk implications from knowing that it 'has license' or 'is evaluated by' a benchmark. The reported F1 scores therefore do not establish that the extracted triples are correct for risk assessment; they only establish that entity co-occurrence was detected. This simplification should be either justified with an explicit ontology mapping from entity-type pairs to fixed relationships, or the evaluation should be repeated with relationship types included.
[Auto-assist questionnaire and Risk Prioritization] The questionnaire evaluation rests on 42 synthetically generated intents with human-annotated ground truth, and accuracy values are reported without confidence intervals or significance tests. More importantly, accuracy on questionnaire answers does not validate the downstream risk inference: the paper does not evaluate whether the answers-to-risks mapping described in the Risk Prioritization section produces correct or well-prioritized risks. Since risk identification and prioritization are central to the claimed contribution, this missing evaluation is load-bearing and should be addressed before the paper's main claims can be accepted.

minor comments (5)

[Analysis (KG Construction)] The text contains a typo: 'certainity' should be 'certainty.' In the same passage, the sentence about measuring certainty 'below an acceptable threshold' would benefit from specifying how that threshold is set and how a human would verify low-confidence facts.
[Table 3] The column headers 'Few-shot/CoT (1 choice)' and 'Few-shot/CoT (user choice)' are not explained in the caption. Clarify that the latter refers to the setting where the user selects from a subset of LLM-suggested answers, since this materially affects the interpretation of the accuracy gains.
[User scenario and Mitigation Recommender] The mitigation recommender section distinguishes guardrails from manually curated actions and notes that manual curation is required. Consider qualifying the abstract's unqualified claim that the system 'proposes mitigation strategies and actions' to reflect the human-in-the-loop nature of the mitigation recommendations.
[Figure 8 and Risk Prioritization] The prompt in Figure 8 includes a 'normalized average score of all the individual scores associated with the risk.' The paper should clarify whether this score comes from the automated risk evaluations (the 1/0/-1 normalization) or from the questionnaire answers, as the two would lead to different interpretations.
[References] The OWASP reference is written as 'OW ASP' with a space in two places; use 'OWASP' consistently.

Circularity Check

0 steps flagged · score 0.0 of 10

No material circularity: the governance outputs are composed from externally sourced benchmarks, policy inputs, and manually curated risk links, not from the outcomes they claim to predict.

full rationale

The paper's claimed derivation chain is not circular: no step defines its output in terms of the quantity it is supposed to predict, and no fitted parameter is renamed as a prediction. The knowledge-graph workflow is evaluated against manually obtained ground truth from a model card, and while the best generic configuration uses a granite-3-8b-instruct judge alongside a granite-3-8b-instruct extractor, this is a possible same-family evaluation bias rather than a definitional reduction of F1 to the model's own output. The questionnaire study uses 42 synthetic intents whose answers were human-validated, so the reported accuracy is not self-agreement by construction. Risk prioritization, model recommendation, and mitigation recommendation are algorithmic compositions of a customer policy, benchmark scores normalized against a reference collection, and manually maintained links between risk-taxonomy entries and benchmarks/guardrails; these inputs are external to the final risk report. The paper's main weakness is that the abstract's strongest claim about end-to-end risk identification, prioritization, and mitigation is not directly evaluated, and the KG extraction metric deliberately ignores relationship types, which weakens risk-relevant semantics. These are validation and design limitations, not circularity. The self-citations (e.g., the IBM AI Risk Atlas and IBM-related guardrails work) are background resources rather than load-bearing uniqueness arguments, and the paper explicitly maps its base taxonomy to external taxonomies such as NIST, MIT, and OWASP. Therefore no circular step can be exhibited from the paper's own equations or constructions.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no fitted numerical parameters and no new postulated scientific entities. Its contributions are architectural: a LinkML-based ontology, a knowledge graph, and generative AI pipelines. The central unproved assumptions are domain assumptions about the adequacy of the chosen risk taxonomy, the accuracy of extraction, and the reliability of LLM-based inference, none of which are independently validated beyond the reported component tests.

assumptions (4)

domain assumption The IBM AI Risk Atlas is a suitable base risk taxonomy for linking AI use cases to risks.
The choice of base taxonomy determines which risks the system can identify. The paper justifies it only by noting the Atlas is 'widely used in industry' (Section 'Applying the Ontology to Risk Mapping').
domain assumption Generative AI extraction from unstructured documentation, plus an LLM judge, yields knowledge-graph triples accurate enough for governance decisions.
The entire KG population pipeline depends on this. The evaluation covers one IBM model card (best generic F1 0.82), and the authors state that generative AI 'does not allow 100% certainty' (Analysis, Knowledge Graph Construction).
domain assumption The SKOS mappings between risk taxonomies (IBM, NIST, MIT, OWASP) are meaningful enough for cross-taxonomy risk comparison.
The paper notes 'it is difficult to draw simple isomorphic equivalences' between taxonomies (Section 'Applying the Ontology to Risk Mapping'), yet relies on these mappings for risk assessments.
domain assumption An LLM-as-judge can reliably infer risk severity from questionnaire answers and risk descriptions.
Risk prioritization depends on this inference (Section 'Risk Prioritization'), but the paper reports no accuracy evaluation for the severity classification itself.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Usage Governance Advisor: From Intent to AI Governance." pith.science (2026). https://pith.science/paper/6AKWBNHA

@misc{pith2026241201957,
  author       = {Pith},
  title        = {Pith review of: Usage Governance Advisor: From Intent to AI Governance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6AKWBNHA}},
  note         = {Machine review of arXiv:2412.01957}
}

read the original abstract

Evaluating the safety of AI Systems is a pressing concern for organizations deploying them. In addition to the societal damage done by the lack of fairness of those systems, deployers are concerned about the legal repercussions and the reputational damage incurred by the use of models that are unsafe. Safety covers both what a model does; e.g., can it be used to reveal personal information from its training set, and how a model was built; e.g., was it only trained on licensed data sets. Determining the safety of an AI system requires gathering information from a wide set of heterogeneous sources including safety benchmarks and technical documentation for the set of models used in that system. In addition, responsible use is encouraged through mechanisms that advise and help the user to take mitigating actions where safety risks are detected. We present Usage Governance Advisor which creates semi-structured governance information, identifies and prioritizes risks according to the intended use case, recommends appropriate benchmarks and risk assessments and importantly proposes mitigation strategies and actions.

Figures

Figures reproduced from arXiv: 2412.01957 by the authors.

**Figure 1.** Usage Governance Advisor pipeline cuss governance by considering the relevant stakeholders (e.g. managers, investors etc.) and their roles in AI governance. (Angstr ˚ om et al. 2023) surveys the challenges compa- ¨ nies face with AI implementation and governance. Finally, (Ferdaus et al. 2024) identifies the opportunities, challenges and limitations of trustworthy AI with a particular focus on LLMs. The paper also … view at source ↗

**Figure 3.** Risk Cards AiModels. LargeLanguageModel, a sub-class of AiModel, is trained on Datasets. Both models and datasets have Licenses. AiEvaluations are associated with AiEvalResults and Risks which can belong to a RiskTaxonomy. Risks may have mappings to Risks in other taxonomies. We materialize this ontology in the KG by ingesting information from multiple sources. Essential information about the AI models includes: h… view at source ↗

**Figure 4.** AI Governance Ontology KG Construction As much of the information stored in the KG exists in unstructured documentation, our system uses a generative AI pipeline to ingest data from these sources. This allows the system to scale as it doesn’t require the manual population of the KG by a human, but involves some probability of incorrect information being added. We will consider this issue further in the analysis sect… view at source ↗

Figures from the paper (2 more)

**Figure 6.** Figure 6: Intent and conversion of free-form answers to bul [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]

**Figure 8.** Figure 8: Risk severity prompt with an LLM response. [PITH_FULL_IMAGE:figures/full_fig_p005_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

PASTA: A Scalable Framework for Multi-Policy AI Compliance Evaluation
cs.HC 2026-01 conditional novelty 6.0 of 10

PASTA is a model-card-based LLM pipeline that evaluates an AI system against five regulations in minutes for about $3, with expert-aligned violation and relevance scores.

Reference graph

Works this paper leans on

33 extracted references · 20 canonical work pages · cited by 1 Pith paper

[1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

A.; Anaby-Tavor, A.; Baldini, I.; Berger, S

Achintalwar, S.; Garcia, A. A.; Anaby-Tavor, A.; Baldini, I.; Berger, S. E.; Bhattacharjee, B.; Bouneffouf, D.; Chaudhury, S.; Chen, P.-Y.; Chiazor, L.; et al. 2024. Detectors for safe and reliable llms: Implementations, uses, and limitations. arXiv preprint arXiv:2403.06009

arXiv 2024
[4]

o m, R. C.; Bj \

ngstr \"o m, R. C.; Bj \"o rn, M.; Dahlander, L.; M \"a hring, M.; and Wallin, M. W. 2023. Getting AI Implementation Right: Insights from a Global Survey. California Management Review, 66(1): 5--22

work page 2023
[5]

Bandel, E.; Perlitz, Y.; Venezian, E.; Friedman-Melamed, R.; Arviv, O.; Orbach, M.; Don-Yehyia, S.; Sheinwald, D.; Gera, A.; Choshen, L.; et al. 2024. Unitxt: Flexible, shareable and reusable data preparation and evaluation for generative ai. arXiv preprint arXiv:2401.14019

arXiv 2024
[6]

Bommasani, R.; Klyman, K.; Longpre, S.; Kapoor, S.; Maslej, N.; Xiong, B.; Zhang, D.; and Liang, P. 2023. The Foundation Model Transparency Index. arXiv:2310.12941

arXiv 2023
[7]

Cihon, P.; Schuett, J.; and Baum, S. D. 2021. Corporate governance of artificial intelligence in the public interest. Information, 12(7): 275

work page 2021
[8]

European Parliament ; and Council of the European Union . 2024. Regulation ( EU ) 2024/1689 of the European Parliament and of the Council . OJ, (L 2024/1689)

work page 2024

Show all 33 references

[9]

M.; Abdelguerfi, M.; Ioup, E.; Niles, K

Ferdaus, M. M.; Abdelguerfi, M.; Ioup, E.; Niles, K. N.; Pathak, K.; and Sloan, S. 2024. Towards Trustworthy AI: A Review of Ethical and Robust Large Language Models. arXiv preprint arXiv:2407.13934

2024 arXiv
[10]

Golpayegani, D.; Pandit, H.; and Lewis, D. 2022. AIRO: An Ontology for Representing AI Risks Based on the Proposed EU AI Act and ISO Risk Management Standards. In International Conference on Semantic Systems

2022
[11]

IBM. 2023. AI Risk Atlas. https://www.ibm.com/docs/en/watsonx/saas?topic=ai-risk-atlas

2023
[12]

IBM. 2024. granite-8b-code-base-4k model card. https://huggingface.co/ibm-granite/granite-8b-code-base-4k

2024
[13]

Inan, H.; Upasani, K.; Chi, J.; Rungta, R.; Iyer, K.; Mao, Y.; Tontchev, M.; Hu, Q.; Fuller, B.; Testuggine, D.; et al. 2023. Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674

2023 arXiv
[14]

K.; and Davis, J

Jiang, W.; Yasmin, J.; Jones, J.; Synovic, N.; Kuo, J.; Bielanski, N.; Tian, Y.; Thiruvathukal, G. K.; and Davis, J. C. 2024. PeaTMOSS: A Dataset and Initial Analysis of Pre-Trained Models in Open-Source Software. In Proceedings of the 21st International Conference on Mining S...

2024
[15]

P.; Miller, M

Joachimiak, M. P.; Miller, M. A.; Caufield, J. H.; Ly, R.; Harris, N. L.; Tritt, A.; Mungall, C. J.; and Bouchard, K. E. 2024. The Artificial Intelligence Ontology: LLM-assisted construction of AI concept hierarchies. arXiv:2404.03044

2024 arXiv
[16]

LangChain. 2024. http://www.langchain.com

2024
[17]

u ttler, H.; Lewis, M.; Yih, W.-t.; Rockt\

Lewis, P.; Perez, E.; Piktus, A.; Petroni, F.; Karpukhin, V.; Goyal, N.; K\" u ttler, H.; Lewis, M.; Yih, W.-t.; Rockt\" a schel, T.; Riedel, S.; and Kiela, D. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In Larochelle, H.; Ranzato, M.; Hadsell, R.; ...

2020
[18]

Magooda, A.; Helyar, A.; Jackson, K.; Sullivan, D.; Atalla, C.; Sheng, E.; Vann, D.; Edgar, R.; Palangi, H.; Lutz, R.; et al. 2023. A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications. arXiv preprint arXiv:2310.17750

2023 arXiv
[19]

F.; and Lata, K

Mihindukulasooriya, N.; Tiwari, S.; Enguix, C. F.; and Lata, K. 2023. Text2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Text. In Payne, T. R.; Presutti, V.; Qi, G.; Poveda-Villal \'o n, M.; Stoilos, G.; Hollink, L.; Kaoudi, Z.; Cheng, G.; and Li, J....

2023
[20]

Mishra, L.; Dhibi, S.; Kim, Y.; Berrospi Ramis, C.; Gupta, S.; Dolfi, M.; and Staar, P. 2024. Statements: Universal Information Extraction from Tables with Large Language Models for ESG KPIs. In Proceedings of the 1st Workshop on Natural Language Processing Meets Climate Chang...

2024
[21]

MIT. 2024. The AI Risk Repository. https://airisk.mit.edu/

2024
[22]

A.; Solbrig, H.; Unni, D

Moxon, S. A.; Solbrig, H.; Unni, D. R.; Jiao, D.; Bruskiewich, R. M.; Balhoff, J. P.; Vaidya, G.; Duncan, W. D.; Hegde, H.; Miller, M.; et al. 2021. The Linked Data Modeling Language (LinkML): A General-Purpose Data Modeling Framework Grounded in Machine-Readable Semantics. IC...

2021
[23]

NIST. 2023. AI Risk Management Framework. https://www.nist.gov/itl/ai-risk-management- framework

2023
[24]

OWASP. 2024. OWASP Top 10 for LLMs and Generative AI Apps. https://genai.owasp.org/llm-top-10/

2024
[25]

M.; Dremel, C.; Mikalef, P.; and Krogstie, J

Papagiannidis, E.; Enholm, I. M.; Dremel, C.; Mikalef, P.; and Krogstie, J. 2023. Toward AI governance: Identifying best practices and potential barriers and outcomes. Information Systems Frontiers, 25(1): 123--141

2023
[26]

Schillaci, Z. 2024. LLM Adoption Trends and Associated Risks. In Large Language Models in Cybersecurity: Threats, Exposure and Mitigation, 121--128. Springer Nature Switzerland Cham

2024
[27]

Schwabe, D.; Laufer, C.; and Casanovas, P. 2020. Knowledge graphs: trust, privacy, and transparency from a legal governance approach. Law Context: A Socio-Legal J., 37: 24

2020
[28]

W3C. 2009. SKOS Simple Knowledge Organization System. https://www.w3.org/2004/02/skos/

2009
[29]

Weidinger, L.; Mellor, J.; Rauh, M.; Griffin, C.; Uesato, J.; Huang, P.-S.; Cheng, M.; Glaese, M.; Balle, B.; Kasirzadeh, A.; et al. 2021. Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359

2021 arXiv
[30]

Xu, L.; Zhao, K.; Zhu, L.; and Xue, H. 2023. Sc-safety: A multi-round open-ended question adversarial safety benchmark for large language models in chinese. arXiv preprint arXiv:2310.05818

2023 arXiv
[31]

Zhang, Z.; Lei, L.; Wu, L.; Sun, R.; Huang, Y.; Long, C.; Liu, X.; Lei, X.; Tang, J.; and Huang, M. 2023. Safetybench: Evaluating the safety of large language models with multiple choice questions. arXiv preprint arXiv:2309.07045

2023 arXiv
[32]

Zheng, L.; Chiang, W.-L.; Sheng, Y.; Zhuang, S.; Wu, Z.; Zhuang, Y.; Lin, Z.; Li, Z.; Li, D.; Xing, E.; et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36: 46595--46623

2023
[33]

Zhu, K.; Zhao, Q.; Chen, H.; Wang, J.; and Xie, X. 2024. Promptbench: A unified library for evaluation of large language models. Journal of Machine Learning Research, 25(254): 1--22

2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

A.; Anaby-Tavor, A.; Baldini, I.; Berger, S

Achintalwar, S.; Garcia, A. A.; Anaby-Tavor, A.; Baldini, I.; Berger, S. E.; Bhattacharjee, B.; Bouneffouf, D.; Chaudhury, S.; Chen, P.-Y.; Chiazor, L.; et al. 2024. Detectors for safe and reliable llms: Implementations, uses, and limitations. arXiv preprint arXiv:2403.06009

arXiv 2024

[4] [4]

o m, R. C.; Bj \

ngstr \"o m, R. C.; Bj \"o rn, M.; Dahlander, L.; M \"a hring, M.; and Wallin, M. W. 2023. Getting AI Implementation Right: Insights from a Global Survey. California Management Review, 66(1): 5--22

work page 2023

[5] [5]

Bandel, E.; Perlitz, Y.; Venezian, E.; Friedman-Melamed, R.; Arviv, O.; Orbach, M.; Don-Yehyia, S.; Sheinwald, D.; Gera, A.; Choshen, L.; et al. 2024. Unitxt: Flexible, shareable and reusable data preparation and evaluation for generative ai. arXiv preprint arXiv:2401.14019

arXiv 2024

[6] [6]

Bommasani, R.; Klyman, K.; Longpre, S.; Kapoor, S.; Maslej, N.; Xiong, B.; Zhang, D.; and Liang, P. 2023. The Foundation Model Transparency Index. arXiv:2310.12941

arXiv 2023

[7] [7]

Cihon, P.; Schuett, J.; and Baum, S. D. 2021. Corporate governance of artificial intelligence in the public interest. Information, 12(7): 275

work page 2021

[8] [8]

European Parliament ; and Council of the European Union . 2024. Regulation ( EU ) 2024/1689 of the European Parliament and of the Council . OJ, (L 2024/1689)

work page 2024