{"work":{"id":"618db55c-6906-42e3-a4c5-c17088cb1df8","openalex_id":null,"doi":null,"arxiv_id":"2309.05463","raw_key":null,"title":"Textbooks Are All You Need II: phi-1.5 technical report","authors":null,"authors_text":"Yuanzhi Li, S\\'ebastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, Yin Tat Lee","year":2023,"venue":"cs.CL","abstract":"We continue the investigation into the power of smaller Transformer-based language models as initiated by \\textbf{TinyStories} -- a 10 million parameter model that can produce coherent English -- and the follow-up work on \\textbf{phi-1}, a 1.3 billion parameter model with Python coding performance close to the state-of-the-art. The latter work proposed to use existing Large Language Models (LLMs) to generate ``textbook quality\" data as a way to enhance the learning process compared to traditional web data. We follow the ``Textbooks Are All You Need\" approach, focusing this time on common sense reasoning in natural language, and create a new 1.3 billion parameter model named \\textbf{phi-1.5}, with performance on natural language tasks comparable to models 5x larger, and surpassing most non-frontier LLMs on more complex reasoning tasks such as grade-school mathematics and basic coding. More generally, \\textbf{phi-1.5} exhibits many of the traits of much larger LLMs, both good -- such as the ability to ``think step by step\" or perform some rudimentary in-context learning -- and bad, including hallucinations and the potential for toxic and biased generations -- encouragingly though, we are seeing improvement on that front thanks to the absence of web data. We open-source \\textbf{phi-1.5} to promote further research on these urgent topics.","external_url":"https://arxiv.org/abs/2309.05463","cited_by_count":null,"metadata_source":"pith","metadata_fetched_at":"2026-05-24T00:13:39.474471+00:00","pith_arxiv_id":"2309.05463","created_at":"2026-05-10T02:53:29.830700+00:00","updated_at":"2026-06-05T21:23:00.469572+00:00","title_quality_ok":true,"display_title":"Textbooks Are All You Need II: phi-1.5 technical report","render_title":"Textbooks Are All You Need II: phi-1.5 technical report"},"hub":{"state":{"work_id":"618db55c-6906-42e3-a4c5-c17088cb1df8","tier":"hub","tier_reason":"10+ Pith inbound or 1,000+ external citations","pith_inbound_count":40,"external_cited_by_count":null,"distinct_field_count":8,"first_pith_cited_at":"2023-11-09T09:25:37+00:00","last_pith_cited_at":"2026-05-21T14:52:48+00:00","author_build_status":"not_needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"not_needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-06-11T12:28:04.844303+00:00","tier_text":"hub"},"tier":"hub","role_counts":[{"context_role":"background","n":9},{"context_role":"method","n":4},{"context_role":"dataset","n":1}],"polarity_counts":[{"context_polarity":"background","n":9},{"context_polarity":"use_method","n":4},{"context_polarity":"use_dataset","n":1}],"runs":{},"summary":{},"graph":{},"authors":[]}}