factgenie: A Framework for Span-based Evaluation of Generated Texts

Ond\v{r}ej Du\v{s}ek; Ond\v{r}ej Pl\'atek; Patr\'icia Schmidtov\'a; Simone Balloccu; Zden\v{e}k Kasner

REVIEW

factgenie: A Framework for Span-based Evaluation of Generated Texts

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.17863 v1 pith:DTH5HGKS submitted 2024-07-25 cs.CL

Zdeněk Kasner , Ondřej Plátek , Patrícia Schmidtová , Simone Balloccu , Ondřej Dušek This is my paper · ORCID

classification cs.CL

keywords annotationsfactgenieframeworkspan-basedtextannotatingcapturecodebase

verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments

read the original abstract

We present factgenie: a framework for annotating and visualizing word spans in textual model outputs. Annotations can capture various span-based phenomena such as semantic inaccuracies or irrelevant text. With factgenie, the annotations can be collected both from human crowdworkers and large language models. Our framework consists of a web interface for data visualization and gathering text annotations, powered by an easily extensible codebase.

Discussion (0). Continue with ORCID to comment.

Pith tools