OpenAI Cribbed Our Tax Example, But Can GPT-4 Really Do Tax?

Andrew Blair-Stanek; Benjamin Van Durme; Nils Holzenberger

arxiv: 2309.09992 · v2 · pith:TUX772XVnew · submitted 2023-09-15 · 💻 cs.AI · cs.CL

OpenAI Cribbed Our Tax Example, But Can GPT-4 Really Do Tax?

Andrew Blair-Stanek , Nils Holzenberger , Benjamin Van Durme This is my paper

classification 💻 cs.AI cs.CL

keywords gpt-4exampleopenaianswerauthorscalculatecribbeddemonstration

0 comments

read the original abstract

The authors explain where OpenAI got the tax law example in its livestream demonstration of GPT-4, why GPT-4 got the wrong answer, and how it fails to reliably calculate taxes.

This paper has not been read by Pith yet.

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

DeonticBench: A Benchmark for Reasoning over Rules
cs.CL 2026-04 unverdicted novelty 7.0

DEONTICBENCH is a new benchmark of 6,232 deontic reasoning tasks from U.S. legal domains where frontier LLMs reach only ~45% accuracy and symbolic Prolog assistance plus RL training still fail to solve tasks reliably.