Fanar-Sadiq: A Multi-Agent Architecture for Grounded Islamic QA

Firoj Alam; Gagan Bhatia; Hamdy Mubarak; Kareem Darwish; Majd Hawasly; Mohamed Y. Eltabakh; Mohammed Qusay Hashim; Mourad Ouzzani; Omar Sinan; Ummar Abbas

Fanar-Sadiq: A Multi-Agent Architecture for Grounded Islamic QA

T0 review · 3 major / 0 minor · reviewed 2026-07-15 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Intent routing to specialized tools improves faithfulness for Islamic QA over a single retrieve-then-generate pipeline.

desk verdict Solid deployed systems paper: intent routing plus deterministic Islamic calculators and citation-grounded RAG, with real usage and competitive public-benchmark numbers; generative faithfulness gains rest on an unvalidated LLM judge. read the letter →

arxiv 2603.08501 v3 pith:KDHZXB5P submitted 2026-03-09 cs.CL

Ummar Abbas , Mourad Ouzzani , Mohamed Y. Eltabakh , Omar Sinan , Gagan Bhatia , Hamdy Mubarak , Majd Hawasly , Mohammed Qusay Hashim

show 2 more authors

Kareem Darwish Firoj Alam

This is my paper · ORCID

classification cs.CL

keywords Islamicquestionansweringmulti-agentsystemsretrieval-augmentedgenerationtooluseQuranretrievalzakatinheritancehallucinationmitigation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language models can answer religious questions fluently, yet they often invent verses, misattribute sources, or break rule-bound calculations such as zakat and inheritance. This paper argues that one fixed retrieve-then-generate pipeline cannot serve that mix of needs. It presents a bilingual Arabic–English multi-agent system that classifies each query’s intent and routes it to specialized modules: exact Quran verse lookup with quotation checks, retrieval-grounded fiqh answers with normalized citations, deterministic Sunni zakat and inheritance calculators with school-sensitive branching, and utility tools for calendar, prayer times, and dua. On public Islamic QA benchmarks the end-to-end system is competitive overall and stronger on open-ended faithfulness-critical sets than strong proprietary baselines. The live service has been accessed about 1.9 million times in under a year.

What carries the argument

Hybrid intent-aware multi-agent router: an LLM classifier with embedding-prototype fallback labels each query into nine intents and dispatches it to exact Quran retrieval (including NL2SQL for statistics), retrieval-grounded fiqh or general knowledge with citation tags, or deterministic zakat and inheritance calculators with madhhab-sensitive parallel outcomes; a response assembler attaches references and verification traces.

What would settle it

On a held-out open-ended Islamic QA set, disable routing and force every query through plain retrieve-then-generate; if faithfulness on IslamicFaithQA and FatwaQA does not fall relative to the routed system, the central claim fails.

Watch

Extended reading notes

Core claim

Heterogeneous Islamic questions—verbatim scripture, citation-backed jurisprudence, and arithmetic-legal obligations—are better handled by intent-aware routing to specialized tools than by forcing every query through one retrieve-then-generate path. With exact verse handling, deterministic calculators, and citation-normalized retrieval selected by intent, open-ended faithfulness improves while average accuracy across public Islamic QA benchmarks remains competitive with strong proprietary models.

Load-bearing premise

The gains depend on the intent classifier reliably sending scripture and calculation questions to the right specialized tools instead of free-form generation.

Editorial extensions

If this is right

Faithfulness-critical Islamic QA can treat tool routing as primary design, not optional post-hoc retrieval.
Deterministic calculators with explicit school branching can surface disputed inheritance outcomes instead of collapsing them.
Exact verse and dua lookup can remove paraphrase drift on canonical text.
A binary Islamic vs non-Islamic pre-filter can isolate specialized routing inside a general chat platform.
Benchmarks that penalize hallucination and reward citation will favor agentic tool suites over monolithic generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same intent-routed pattern could transfer to other high-stakes doctrinal domains where invented citations are costly.
Tighter coupling from computed inheritance shares to multiple-choice option letters may close the remaining gap on structured legal MCQs without dropping symbolic engines.
Live routing-error rate would be a natural production metric for the faithfulness claim.
Extending quotation validation to Hadith at Quran-level strictness would address a gap the authors flag for future work.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a working multi-agent Islamic QA stack, not a theory paper. What is new is the integrated artifact: hybrid intent routing into specialized tools (exact Quran lookup with NL2SQL, dua retrieval, calendar/prayer, deterministic Sunni zakat and madhhab-sensitive inheritance with parallel disputed outcomes), plus citation normalization and verification traces, shipped as a live bilingual Fanar component with ~1.9M accesses and 77% likes on rated replies.

They do the systems work carefully. Nine intents are motivated and measured (hybrid classifier 90.1% on 700 majority-vote chat queries, κ=0.76). Algorithms for zakat and inheritance, prayer methods, Quran schema, and response assembly are concrete. Table 1 positions them against ReAct/Toolformer/Self-RAG and prior Islamic RAG. End-to-end Table 7 is the right comparison: average 76.5, beating open baselines and competitive with GPT-5/Gemini-3-Pro, with the largest lifts on IslamicFaithQA (65.4) and FatwaQA (65.1). MCQ PalmX and IslamTrust look strong; QIAS inheritance is weaker (72.2 vs ~93–94), which they attribute to option-mapping rather than pure arithmetic—plausible and honest.

Soft spots, in proportion. The stress-test is right that the headline faithfulness claim leans on GPT-4.1 SIMPLEQA-style judging with no human agreement or fiqh-specific calibration; that is the main evaluation gap, not a reason to dismiss the engineering. Routing is evaluated on in-distribution logs, so production generalization is an open risk they already list under Limitations. No public code or full corpora, so full replication is blocked. Free thresholds (similarity, temperatures, binary Islamic router) are normal for this genre. Citations are appropriate; self-cites to Fanar/Bhatia supply platform pieces, not circular accuracy.

Who it is for: Arabic NLP, agentic RAG, and anyone building high-stakes domain assistants. Serious referee time is warranted. I would engage, cite the architecture and deployment numbers when discussing tool-routed religious QA, and push for judge validation or human eval plus tighter QIAS option coupling. Not a desk reject.

Referee Report

3 major / 0 minor

Summary. The paper presents Fanar-Sadiq, a bilingual (Arabic/English) multi-agent Islamic QA system that routes queries via a hybrid intent classifier to specialized modules: retrieval-grounded fiqh with citation normalization, exact Quran verse lookup and NL2SQL statistics, deterministic Sunni zakat and madhhab-sensitive inheritance calculators, and utility tools (calendar, prayer times, dua). It argues that a single retrieve-then-generate pipeline is insufficient for heterogeneous Islamic intents and that tool- and evidence-routed execution improves faithfulness. End-to-end evaluation on five public benchmarks (PalmX, QIAS T1, IslamTrust, IslamicFaithQA, FatwaQA) reports an average accuracy of 76.5, with the largest gains on open-ended generative sets (IslamicFaithQA 65.4, FatwaQA 65.1) relative to proprietary and open baselines (Table 7). The system is deployed in the Fanar platform with ~1.9M accesses and positive user ratings on a subset of feedback.

Significance. If the routing-plus-tools design is the main driver of the reported generative gains, the work is a solid systems contribution for high-stakes religious QA: it combines intent-aware routing, deterministic calculators with explicit madhhab branching, and citation-normalized retrieval under one architecture, with public API access and substantial real-world usage. Strengths include clear module design (Algorithms 1–2, Figure 3), a hybrid classifier with reported accuracy and IAA, specialized NL2SQL results (Table 5), and evaluation against strong proprietary baselines on public benchmarks rather than only in-house sets. The paper is useful for the community building grounded Islamic assistants even if some evaluation details need tightening.

major comments (3)

§4 and App. B.4: The headline faithfulness claim rests on IslamicFaithQA (65.4) and FatwaQA (65.1) in Table 7, scored solely by GPT-4.1 under a SIMPLEQA-style CORRECT/INCORRECT/NOT_ATTEMPTED protocol. No human agreement, inter-annotator reliability, or judge-vs-human calibration is reported for Islamic/fiqh content, where madhhab disagreement, conditional rulings, and citation nuance make binary correctness ambiguous. Without that validation (or a human-judged subsample), it is hard to rule out that the judge systematically favors citation-heavy or tool-formatted answers, which would inflate the claimed gains over GPT-5/Gemini-3-Pro.
§5 and Table 7 (QIAS T1): The system scores 72.2 vs. 93.0–94.5 for the strongest proprietary models on inheritance MCQ, despite a dedicated deterministic calculator (Figure 3, §3.3.2). The paper attributes this to option-mapping friction but does not quantify mapping errors, report calculator-only accuracy on gold distributions, or ablate the symbolic-to-option step. That gap weakens the claim that specialized tools reliably improve rule-constrained Islamic reasoning under the evaluation protocol used.
§3.1.1 and Limitations: Hybrid classifier accuracy is 90.1% on 700 majority-vote queries sampled from the system’s own chat logs (Fleiss’ κ=0.76). There is no out-of-distribution or held-out production-style evaluation, no error analysis of misroutes (e.g., calculation or scripture queries sent to free-form fiqh), and no end-to-end ablation of routing errors on faithfulness. Given that correct tool invocation is load-bearing for the central claim, this generalization risk should be measured more carefully.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: empirical multi-agent system evaluated on public benchmarks, not tautological from inputs.

full rationale

Fanar-Sadiq is an engineering/systems paper whose load-bearing claims are measured end-to-end accuracies (Table 7) on public Islamic QA benchmarks (PalmX, QIAS T1, IslamTrust, IslamicFaithQA, FatwaQA), not quantities obtained by algebraic reduction from fitted parameters or self-defined objects. Deterministic zakat/inheritance modules implement established fiqh share tables and arithmetic invariants (Algorithms 1–2, Table 4, Figure 3); NL2SQL is trained on template pairs and scored by denotational execution equality; the hybrid router is scored by majority-vote accuracy on an annotated set. Overlapping-author citations (Fanar Team 2025 for the base LLM/platform; Bhatia et al. 2026 for IslamicFaithQA and related agentic RAG) supply components and one evaluation set, but the reported numbers are not forced by those citations by construction. No uniqueness theorem, fitted-input-as-prediction, self-definitional identity, or ansatz smuggled via self-citation appears in any derivation chain. Self-contained empirical evaluation against external benchmarks yields score 0.

Assumptions & free parameters 6 free parameters · 6 assumptions · 3 invented entities

The central performance claim rests on standard agentic-RAG and embedding machinery plus fixed classical Sunni legal rules treated as oracles, together with a handful of routing and retrieval thresholds chosen for the system. No new physical constants or free-form fitted scientific parameters appear; the free parameters are engineering thresholds and the axioms are domain legal and calendrical conventions.

free parameters (6)

hybrid classifier confidence fallback threshold = 0.5
Predictions below 0.5 trigger the prototype embedding fallback (Section 3.1); value chosen by authors, not derived.
dua occasion similarity threshold = 0.2
Minimum cosine similarity 0.2 for keeping top-k occasions (Section 3.2.4).
surah fuzzy-match confidence threshold = 0.6
Levenshtein/embedding match accepted only above 0.6 (Section 3.5.3).
max_sources retrieval breadth = 12
Default top-k = 12 documents for the vector retriever (Section 3.6).
Islamic vs non-Islamic binary threshold = 0.66
Continuous classifier output binarized at 0.66 for the orchestrator (Appendix A).
generation temperatures and token budgets = 0.1–0.2 / 256–4500
Module-specific defaults (greeting 0.2/256, fiqh 0.1/4500, nl2sql 0.1, etc.) set by authors (Section 3.6).

assumptions (6)

domain assumption Classical Sunni fixed-share (fard), residuary (‘asaba), ‘awl and radd rules, with madhhab-sensitive branching for disputed cases, correctly compute estate distribution when heirs and impediments are correctly specified.
Load-bearing for the inheritance calculator (Section 3.3.2, Figure 3, Table 4); treated as fixed legal oracle.
domain assumption Zakat nisab (85 g gold / 595 g silver), 2.5 % monetary rate, irrigation-dependent agricultural rates, and Hadith-based livestock schedules are the correct computational rules.
Used directly in Algorithm 1 and Section 3.3.1; prices are external inputs.
domain assumption Umm al-Qura calendar and the listed prayer-method angular parameters (Table 3) are acceptable for Hijri conversion and prayer-time computation.
Islamic Calendar and Prayer Times tools (Sections 3.2.2–3.2.3).
ad hoc to paper Nine intent classes plus four Quran subtypes exhaust the execution strategies needed for the target Islamic query distribution.
Schema defined by the authors (Section 3.1); classifier accuracy is measured only against this closed set.
domain assumption Cosine similarity over Qwen3-Embedding-4B vectors plus optional cross-encoder reranking yields sufficiently relevant documents for grounded generation.
Retriever design (Section 3.4.3); standard RAG assumption.
ad hoc to paper GPT-4.1 as LLM-as-judge, following the SIMPLE QA protocol, correctly labels open-ended answers CORRECT / INCORRECT / NOT_ATTEMPTED relative to gold Islamic references.
Sole metric for IslamicFaithQA and FatwaQA (Section 4, Appendix B.4); no human correlation reported.

invented entities (3)

Hybrid Query Classifier (LLM primary + prototype embedding fallback)
purpose: Map free-form bilingual queries onto the nine execution routes with a confidence score.
Core routing mechanism introduced and evaluated only inside this system (Section 3.1); no external independent validation set released.
Madhhab-sensitive deterministic inheritance calculator returning parallel outcomes independent evidence
purpose: Compute estate shares under Sunni rules while surfacing Hanafi vs Jumhur (etc.) disagreement instead of collapsing to one ruling.
Distinct from generic calculators; implements the three-phase fard–asaba–awl/radd workflow of Figure 3.
Citation-normalized fiqh agent with verification traces and [CITE:N] tags
purpose: Force every jurisprudential claim to map to a stable retrieved span and reduce unsupported generation.
Post-generation assembly and verification step specific to this architecture (Sections 3.4.1, 3.6).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Fanar-Sadiq: A Multi-Agent Architecture for Grounded Islamic QA." pith.science (2026). https://pith.science/paper/KDHZXB5P

@misc{pith2026260308501,
  author       = {Pith},
  title        = {Pith review of: Fanar-Sadiq: A Multi-Agent Architecture for Grounded Islamic QA},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KDHZXB5P}},
  note         = {Machine review of arXiv:2603.08501}
}

read the original abstract

Large language models (LLMs) can answer religious knowledge queries fluently, yet they often hallucinate and misattribute sources, which is especially consequential in Islamic settings where users expect grounding in canonical texts (Qur'an and Hadith) and jurisprudential (fiqh) nuance. Retrieval-augmented generation (RAG) improves grounding, however, a single retrieve-then-generate pipeline is insufficient for diverse Islamic queries, including verbatim scripture, citation-grounded guidance, and rule-constrained computations such as zakat and inheritance. To address these challenges, we present Fanar-Sadiq, a bilingual Arabic-English Islamic QA system built on a multi-agent, tool-augmented architecture. It is a core component of the Fanar AI platform. Fanar-Sadiq routes Islamic queries to specialized modules within an agentic tool architecture. It supports intent-aware routing, retrieval-grounded fiqh answers with normalized citations and verification traces, exact verse lookup with quotation validation, and deterministic Sunni zakat and inheritance calculators with madhhab-sensitive branching. We evaluate the end-to-end system on public Islamic QA benchmarks and show strong effectiveness and efficiency. It is publicly accessible through an API and Web application and has received over 1.9M accesses in less than a year (https://api.fanar.qa/docs).

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

IslamicTurathBench: A Multi-Task, Multi-Discipline Benchmark for Evaluating Large Language Models on the Islamic Scholarly Tradition (turath)
cs.CL 2026-08 conditional novelty 6.0 of 10

IslamicTurathBench is a new expert-reviewed Arabic benchmark that tests LLMs on classical Islamic scholarship across seven disciplines, three difficulty tiers, and three task formats.

Reference graph

Works this paper leans on

17 extracted references · cited by 1 Pith paper

[1]

Technical report, Birzeit University

QuranMorph: Morphologically Annotated Quranic Corpus. Technical report, Birzeit University. Sadam Al-Azani, Maad Alowaifeer, Alhanoof Alhu- nief, and Ahmed Abdelali. 2025. Ontologyrag-q: Re- source development and benchmarking for retrieval- augmented question answering in qur’anic tafsir. In Proceedings of EMNLP 2025. Yusuf Al-Qaradawi. 1999. Fiqh az-Zak...

2025
[2]

2510.25621v1

Farsiqa: Faithful and advanced rag system for islamic question answering. 2510.25621v1. Farah Atif, Nursultan Askarbekuly, Kareem Darwish, and Monojit Choudhury. 2025. Sacred or synthetic? evaluating llm reliability and abstention for religious questions. In Proceedings of the AAAI/ACM Con- ference on AI, Ethics, and Society, volume 8, pages 217–226. Adil...

arXiv 2025
[3]

In Proceedings of ArabicNLP 2023, pages 690–701, Singapore (Hybrid)

Qur’an QA 2023 shared task: Overview of passage retrieval and reading comprehension tasks over the holy qur’an. In Proceedings of ArabicNLP 2023, pages 690–701, Singapore (Hybrid). Associa- tion for Computational Linguistics. Marryam Mohammed, Sama Ali, Salma Khaled, Ayad Majeed, and Ensaf Mohamed. 2025. Aftina: enhanc- ing stability and preventing halluc...

arXiv 2023
[4]

Corrective retrieval augmented generation. arXiv. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR). Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T K...

2023
[5]

Is X halal?

**fiqh_ruling**: Questions asking for Islamic legal rulings, permissibility, obligations, or jurisprudence Examples: "Is X halal?", "What �s the ruling on Y?", �� , ��
[6]

What does verse 2:255 say?

**quran_retrieval**: Questions asking for specific Quranic verses or ayahs Examples: "What does verse 2:255 say?", "Find ayah about patience", �� , ��
[7]

Who was Umar ibn al-Khattab?

**general_islamic**: General questions about Islamic knowledge, history, concepts, or practices % Use this when the question does NOT request a ruling/calculation/timing/retrieval explicitly. Examples: "Who was Umar ibn al-Khattab?", "What is tawakkul?", ��
[8]

Hi", "Thanks!

**greeting**: Simple greetings, thanks, or pleasantries Examples: "Hi", "Thanks!", �� , ��

Show all 17 references

[9]

How much zakat do I pay on $10,000?

**zakat_calculation**: Requests to compute Zakat owed based on assets, debts, or metal prices Examples: "How much zakat do I pay on $10,000?", ��
[10]

Split inheritance among wife and children

**inheritance_calculation**: Requests to divide an estate among heirs (Mirath/Faraid) Examples: "Split inheritance among wife and children", ��
[11]

dua for entering bathroom

**dua_lookup**: Requests for duas (supplications) or adhkar (remembrances), or what to say in specific situations Examples: "dua for entering bathroom", "morning adhkar", "what to say before sleeping", ��
[12]

What is today �s Hijri date?

**islamic_calendar**: Questions about Hijri/Islamic dates, date conversions, or Islamic events/holidays Examples: "What is today �s Hijri date?", "When is Ramadan 2025?", "Convert March 1 to Hijri", "When is Eid?", �� , �� 13

2025
[13]

What time is Fajr in Dubai?

**prayer_times**: Questions about prayer times, salah timing, or Qibla direction for a location Examples: "What time is Fajr in Dubai?", "Prayer times for London", "Which direction is Qibla from Tokyo?", �� , �� Return ONLY valid JSON ...
[14]

What does verse 2:255 say?

**specific_verse**: Asking for a specific verse by number or reference Examples: - "What does verse 2:255 say?" - "Show me ayah 7 of Al-Fatiha" - �� - �� - "What are the last three verses of Surah Al-...
[15]

Write Surah Al-Fatiha

**full_surah**: Asking for an entire surah �s text Examples: - "Write Surah Al-Fatiha" - �� - "Give me the entire Surah Nas"
[16]

How many verses in Surah Al-Baqarah?

**statistics**: Counting verses, surah metadata, or structural queries Examples: - "How many verses in Surah Al-Baqarah?" - � �� - "Which surah has the most verses?" - "Is Al-Baqarah Makki or Madani?" - ��
[17]

What is the meaning of Ayat al-Kursi?

**interpretation**: Asking for meaning, tafsir, or explanation Examples: - "What is the meaning of Ayat al-Kursi?" - �� - "Explain the interpretation of Al-Kawthar" - "What does the Quran say about patience?" Return ONLY the sub-type name (sp...

1998

Pith tools

Reviewed July 15, 2026 · model on record in the stance chip above.

[1] [1]

Technical report, Birzeit University

QuranMorph: Morphologically Annotated Quranic Corpus. Technical report, Birzeit University. Sadam Al-Azani, Maad Alowaifeer, Alhanoof Alhu- nief, and Ahmed Abdelali. 2025. Ontologyrag-q: Re- source development and benchmarking for retrieval- augmented question answering in qur’anic tafsir. In Proceedings of EMNLP 2025. Yusuf Al-Qaradawi. 1999. Fiqh az-Zak...

2025

[2] [2]

2510.25621v1

Farsiqa: Faithful and advanced rag system for islamic question answering. 2510.25621v1. Farah Atif, Nursultan Askarbekuly, Kareem Darwish, and Monojit Choudhury. 2025. Sacred or synthetic? evaluating llm reliability and abstention for religious questions. In Proceedings of the AAAI/ACM Con- ference on AI, Ethics, and Society, volume 8, pages 217–226. Adil...

arXiv 2025

[3] [3]

In Proceedings of ArabicNLP 2023, pages 690–701, Singapore (Hybrid)

Qur’an QA 2023 shared task: Overview of passage retrieval and reading comprehension tasks over the holy qur’an. In Proceedings of ArabicNLP 2023, pages 690–701, Singapore (Hybrid). Associa- tion for Computational Linguistics. Marryam Mohammed, Sama Ali, Salma Khaled, Ayad Majeed, and Ensaf Mohamed. 2025. Aftina: enhanc- ing stability and preventing halluc...

arXiv 2023

[4] [4]

Corrective retrieval augmented generation. arXiv. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR). Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T K...

2023

[5] [5]

Is X halal?

**fiqh_ruling**: Questions asking for Islamic legal rulings, permissibility, obligations, or jurisprudence Examples: "Is X halal?", "What �s the ruling on Y?", �� , ��

[6] [6]

What does verse 2:255 say?

**quran_retrieval**: Questions asking for specific Quranic verses or ayahs Examples: "What does verse 2:255 say?", "Find ayah about patience", �� , ��

[7] [7]

Who was Umar ibn al-Khattab?

**general_islamic**: General questions about Islamic knowledge, history, concepts, or practices % Use this when the question does NOT request a ruling/calculation/timing/retrieval explicitly. Examples: "Who was Umar ibn al-Khattab?", "What is tawakkul?", ��

[8] [8]

Hi", "Thanks!

**greeting**: Simple greetings, thanks, or pleasantries Examples: "Hi", "Thanks!", �� , ��