Pith. sign in

REVIEW 4 major objections 5 minor 47 references

Retrieval Augmented Decision-Making: A Requirements-Driven, Multi-Criteria Framework for Structured Decision Support

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that adding an explicit, weighted hierarchy of decision criteria to retrieval-augmented generation produces decision reports that are more detailed, reasonable, and traceable than existing RAG-based approaches.

desk verdict The RAD framework is a clearly described synthesis of RAG, ISM, AHP, and LLM agents, but the claimed empirical superiority is not supported by the same-model LLM judging and the sparse statistical reporting. read the letter →

arxiv 2505.18483 v1 pith:563MDMSY submitted 2025-05-24 cs.AI

Hongjia Wu , Hongxin Zhang , Wei Chen , Jiazhi Xia This is my paper
classification cs.AI
keywords retrieval-augmentedgenerationmulti-criteriadecisionmakinganalytichierarchyprocessinterpretivestructuralmodelinghierarchicalmodellargelanguagemodelsstructuredsupportLLMevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Retrieval-augmented generation can supply context, but it does not make the trade-offs behind a recommendation explicit. This paper argues that adding a quantitative decision layer — a hierarchy of weighted criteria — fixes that gap. It presents RAD, a pipeline that extracts decision criteria from documents, structures them into an interpretive model with explicit weights, and uses that model to score and rank candidate options before writing a report. The paper's claim is that RAD's reports are more detailed, reasonable, and well structured than those of plain RAG or GraphRAG, as measured by LLM-judged win rates. If true, the method offers a practical way to make LLM-based decision support transparent and auditable.

What carries the argument

The load-bearing object is the Weighted Hierarchical Decision Making Model $H = \{F, E, W\}$: a set of extracted criteria $F$, a relation matrix $E$ assembled from pairwise influence judgments, and weights $W$ derived from a multi-agent voting process and converted by the Analytic Hierarchy Process. This model acts as the explicit reasoning scaffold: it determines which factors matter, how they relate, and how much they count, and every score in the final report can be traced back to a criterion, a weight, and a matrix entry.

What would settle it

Give domain experts the same decision requests and both RAD and GraphRAG reports, anonymized, and ask them to rate actionability, correctness, and traceability without knowing the generator. If expert ratings do not favor RAD, the central claim fails. A faster check: re-run the comparisons with a judge from a different model family and with outputs matched for length; if RAD's win-rate advantage disappears, the reported quality gain is a judgment artifact.

Watch

Extended reading notes

Core claim

RAD's central claim is that building an explicit weighted hierarchical decision model $H = \{F, E, W\}$ before generating text measurably improves decision reports. The model is constructed automatically: an LLM extracts criteria $F$ from retrieved text blocks; pairwise influence judgments fill the relation matrix $E$; Interpretive Structural Modeling layers the criteria; a manager-plus-five-expert multi-agent team votes on importance, and AHP converts the averaged ranking into weights $W$. During decision-making, each option receives a 1–9 score per criterion, and options are ranked by weighted totals. The paper reports that this configuration, evaluated with large-language-model judges, wins consistently against a no-hierarchy ablation and against GraphRAG across three quality metrics, and that the constructed hierarchies and criteria pass forced-choice validity tests.

Load-bearing premise

The evaluation assumes an LLM judge from the same model family that generated the reports can reliably tell which report is better; if the judge favors longer or more structured-looking text, the win rates do not show that RAD helps real decision-makers.

Editorial extensions

If this is right

  • If RAD's reported win rates hold, decision-support systems can replace free-form RAG answers with reports whose conclusions map back to a visible weighted hierarchy.
  • Directory-aware segmentation appears to be load-bearing: replacing it with fixed-word splitting drops user-need consistency, suggesting that document structure carries decision-relevant signal.
  • GraphRAG's option scores are not tied to explicit factors in RAD's comparisons; a factor-grounded scoring scheme may be necessary for trustworthy option comparison.
  • The component tests suggest that non-expert LLM judges can check criterion applicability and hierarchy correctness with high agreement, which could lower the cost of validating decision models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The win-rate results depend on a single LLM judge family; an independent judge or human expert panel could change the ranking, so the performance gap should be treated as provisional until such a check is run.
  • RAD's reliance on document structure means it will inherit inconsistencies in messy or contradictory sources; adding a conflict-resolution step could make it robust for regulatory and policy corpora.
  • The weighted hierarchy could be reused beyond report generation, e.g., to expose sensitivity of the final ranking to weight changes, enabling what-if analysis for stakeholders.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes RAD, a retrieval-augmented decision-making framework that integrates multi-criteria decision analysis with LLM-based semantic understanding. For a decision request with reference documents, RAD extracts decision criteria from document chunks, builds a weighted hierarchical decision model via relation extraction, ISM layering, multi-agent voting, and AHP, then scores alternatives on a 1–9 scale and generates structured decision reports. Experiments 1 and 2 compare RAD against two ablations and GraphRAG on two small self-constructed datasets (50 requests each), using DeepSeek-V3 as both generator and judge, and test alternative discrimination on seven positive/negative prompt pairs. Appendix experiments A.2–A.5 use LLM forced-choice tasks to evaluate criterion applicability, hierarchy correctness, and weight reasonableness. The central claim is that RAD significantly outperforms existing methods in the detail, rationality, and structure of decision reports.

Significance. If the proposed method worked as claimed, it would offer a useful integration of MCDA and RAG, with explicit weight assignment and traceable reasoning paths. The pipeline is clearly specified, and the ablation design in Experiment 1 is a reasonable way to isolate the contribution of the hierarchical decision model. However, the paper provides no code, data, or prompts, and the empirical evaluation rests entirely on LLM-as-judge scores from the same model family that generates the outputs, with no human validation. The statistical reporting contains an explicit inconsistency (p=0.0579 described as significant), Experiment 2 has no baseline, and the appendix evaluations are circular self-consistency checks rather than external validation. As a result, the manuscript's central empirical claim is not presently supported; the main value of the paper is the architecture proposal rather than the demonstrated results.

major comments (4)
  1. [Section 4.1.2, Section 5.1.1] The central empirical claim rests on LLM-as-judge win rates in which the judge, DeepSeek-V3, is from the same model family that generates the outputs. The paper provides no evidence that this judge's preferences correlate with human judgments of decision quality, and the evaluation criteria C1–C3 are subjective. The judge may systematically prefer longer or more structured text, which are exactly the features RAD is designed to produce. The statistical reporting is also inconsistent: Section 5.1.1 first states that 'RAD significantly outperforms NS on all metrics (p < 0.05)' and then cites p=0.0579 for the R2/C1 comparison as evidence of an advantage; p=0.0579 is not significant at the conventional 0.05 threshold. This undermines the abstract's claim of significant improvement.
  2. [Section 4.2.2, Table 1] Experiment 2 has no baseline: the table reports only RAD's own outcomes, so the claim that RAD has 'effective capability for solution discrimination' is not established. Moreover, Table 1 shows failures on 3 of 7 cases (Cases 1, 6, and 7 fail at least one of C4–C6, with Case 6 passing C4 only through a different criterion), and the discussion in Section 5.2 attributes these failures to missing key decision points. That attribution raises a concern about the completeness of the criteria-extraction stage rather than demonstrating that the method reliably discriminates between better and worse options.
  3. [Appendix A.4, A.2, A.3] The appendix evaluations are circular. In A.4, the RAD weight scheme is produced by DeepSeek-V3 through the RAD pipeline, the alternative is hand-altered, and DeepSeek-V3 is asked to choose between them; the reported selection rates of 67–74% therefore measure self-consistency of the model family, not the correctness of the weights. The same concern applies to A.2 and A.3, where the model is asked to endorse criteria and hierarchical relationships that the same model family constructed. The paper's own statement in A.1 that 'In the absence of a gold standard or expert judgment' the authors transformed the evaluation into LLM choice tasks makes this limitation explicit, but such tests cannot validate the model against human decision-making.
  4. [General (reproducibility)] No code, data, or prompts are released, and the paper does not provide access to the evaluation harness. Since the reported results depend on undocumented prompt details, model versions, and scoring protocols, independent replication is impossible. For an empirical claim of significant superiority, this is a major reproducibility gap. The authors should either provide the artifacts or substantially temper the empirical claims.
minor comments (5)
  1. [Section 4.1.2] The phrase 'Each request was generated five times, and the average was taken' is ambiguous; please specify whether the win rates are averaged over five generations and how ties are handled.
  2. [Section 3.5.3] The conversion of the average importance ranking into AHP weights is not described in detail; please provide the exact formula or a reference to a concrete AHP weighting procedure, since the weight vector W is central to the method.
  3. [Appendix A.4.3] The analysis text refers to 'Experiment 3' when describing the weight-appropriateness experiment, which is actually Experiment 5; please correct the cross-references.
  4. [Section 2.2 and Section 3.4] There are notation and capitalization inconsistencies: 'graphRAG' and 'GraphRAG' are used interchangeably, and the notation 'T opk(C, d)' in Section 3.4 contains a typographic issue that should be fixed.
  5. [Section 5.2] The statement that 'the method's answers fully satisfy criteria C4, C5, and C6 in Cases 2, 3, 4, and 5' is correct for those four cases, but the subsequent discussion of misjudgments in Cases 1, 6, and 7 would be easier to follow if the table explicitly labeled each case and stated which criteria were met or unmet.

Circularity Check

3 steps flagged · score 6.0 of 10

Central performance claims rest on same-model LLM judging; appendix validations are self-consistency checks.

  1. other [Section 4.1.2 and Section 5.1.1 (Experiment 1 configuration and main result)]
    "Since there are no standard answers for decision generation tasks, we followed research [9] and used large language models to perform comparative evaluations of the generated results. ... The deepseek-v3 model is used for generation."

    The same model family (DeepSeek-V3) both generates the decision reports and serves as the evaluator. The reported win rates therefore measure how often the generator model prefers outputs that carry the structural features RAD inserts (hierarchy, weights, reasoning chains), rather than an independent assessment of decision-report quality. No external ground truth or human expert rating is provided, so the central claim that RAD 'significantly outperforms' existing methods is validated by the same system that produced the outputs, making the comparison self-referential.

  2. fitted input called prediction [Appendix A.4, Experiment 5 (weight appropriateness evaluation)]
    "DeepSeek-V3 was used as the evaluation model. ... We randomly select a set of sub-criteria at the same level from the hierarchical model, along with their normalized weights, as the original output of the RAD method. Then, we manually construct an alternative weight scheme... Both the original and the alternative weight schemes are presented to the LLM... The model is asked: 'Which set of weights better reflects the relative importance of these criteria for this decision-making requirement?'"

    The original weight vector W is itself produced by a DeepSeek-V3 multi-agent voting process in Section 3.5.3. The evaluation then asks DeepSeek-V3 to choose between that model-generated weight set and a hand-altered alternative. The reported selection rates (67.33% and 73.66%) show the evaluator preferring the model's own prior output, i.e., self-consistency. There is no independent criterion for 'appropriate weights,' so the validation reduces to the same model agreeing with itself.

1 more flagged steps
  1. self definitional [Appendix A.2-A.3, Experiments 3 and 4 (criterion applicability and hierarchy correctness)]
    "From the hierarchical decision model built by RAD, we randomly selected one criterion as a candidate. At the same time, we constructed a distractor criterion... We then asked the LLM a closed-ended question: 'For this user requirement, which criterion is more suitable to be included in the decision model?' ... We then compare the model's responses with the original hierarchy constructed by the RAD method to assess the accuracy and consistency of its semantic structure understanding."

    The 'correct' criterion is defined as the one RAD extracted and the 'original hierarchy' is the one RAD constructed, both generated by DeepSeek-V3. Asking DeepSeek-V3 to recognize these same outputs as correct measures agreement of the model with its own prior generations, not validity against an external standard. The ~95% agreement rates are therefore self-consistency scores, not evidence that the criteria or hierarchy are semantically correct for the decision task.

full rationale

The internal derivation chain of RAD (criteria extraction -> hierarchical modeling -> weighting -> scoring -> report generation) is not circular: no step is defined in terms of its own output, and the method genuinely integrates RAG, ISM, and AHP. However, the empirical evidence for the central claim of superiority is circular in evaluation. Experiment 1 uses DeepSeek-V3 both to generate reports and to judge them, with no human expert or external ground truth. The appendix experiments are more explicitly self-referential: the criteria, hierarchy, and weights under test are generated by DeepSeek-V3, and DeepSeek-V3 is then asked to confirm whether these outputs are appropriate, yielding high agreement rates that primarily demonstrate self-consistency. Thus the reported win rates and selection proportions do not independently establish decision quality. The mislabeled p=0.0579 as 'significant' is a correctness risk rather than circularity, and no code, prompts, or data are released for independent replication. Overall, the method has independent technical content, but the paper's load-bearing validation reduces to same-model self-assessment, so the circularity score is 6 rather than lower.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The framework rests on LLM-produced criteria, relations, and weights being decision-valid, and on LLM judges being reliable, both unverified; the only free numerical inputs are the weights and a fixed k=10, with no sensitivity analysis.

free parameters (4)
  • LLM-generated criterion weight vector W = not reported; derived from DeepSeek-V3 agent rankings
    Central to final option scoring; weights come from LLM voting without external calibration (Section 3.5.3).
  • number of criteria k = 10 in Appendix experiments; otherwise unspecified top-k
    The decision model size is set to 10 criteria in the weight and hierarchy experiments without justification; varying k would change results.
  • number of expert agents = 5
    Five domain expert agents are assumed sufficient; no sensitivity analysis.
  • AHP 1-9 scoring scale for alternatives = 1-9
    Standard scale, but scores are assigned by LLM and not checked against human judgment.
assumptions (5)
  • domain assumption LLM-extracted criteria and pairwise influence relations reflect real decision-relevant structure.
    Criteria extraction (Section 3.4) and relation extraction (Section 3.5.1) are performed by DeepSeek-V3 with no ground truth or human vetting; the entire hierarchy inherits any errors.
  • domain assumption LLM-as-judge produces valid quality ratings for decision reports.
    Experiment 1 (Section 4.1.2) uses DeepSeek-V3 to choose winners; no human evaluation or calibration is reported.
  • domain assumption Document table-of-contents hierarchy provides semantically valid segmentation units.
    The text processing step (Section 3.3.1) relies on the document outline; the only evidence for its benefit is the NS ablation, a single LLM-judged comparison.
  • ad hoc to paper Multi-agent voting with five LLM experts approximates domain-expert consensus on criterion importance.
    No comparison to human expert rankings; Appendix A.4 shows LLM judges prefer RAD's weights only about 67-74% of the time, a weak endorsement.
  • standard math AHP's rank-to-weight conversion and consistency checks are valid when inputs come from stochastic LLM outputs.
    AHP is standard, but the paper reports no consistency ratios and does not discuss how LLM variability affects the weight conversion (Section 3.5.3).
invented entities (2)
  • Manager agent AM and five expert agents Ae1-Ae5
    purpose: Simulate a panel of domain experts to rank criteria importance and produce AHP weights
    These are software constructs with no external test; the claimed validity rests on the LLM judge that shares the same model family.
  • Weighted Hierarchical Decision Making Model H = {F, E, W}
    purpose: Intermediate structured representation used to score options and generate reports
    Internal data structure; its quality is only self-evaluated in Appendices A.2-A.5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Retrieval Augmented Decision-Making: A Requirements-Driven, Multi-Criteria Framework for Structured Decision Support." pith.science (2026). https://pith.science/paper/563MDMSY

@misc{pith2026250518483,
  author       = {Pith},
  title        = {Pith review of: Retrieval Augmented Decision-Making: A Requirements-Driven, Multi-Criteria Framework for Structured Decision Support},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/563MDMSY}},
  note         = {Machine review of arXiv:2505.18483}
}
read the original abstract

Various industries have produced a large number of documents such as industrial plans, technical guidelines, and regulations that are structurally complex and content-wise fragmented. This poses significant challenges for experts and decision-makers in terms of retrieval and understanding. Although existing LLM-based Retrieval-Augmented Generation methods can provide context-related suggestions, they lack quantitative weighting and traceable reasoning paths, making it difficult to offer multi-level and transparent decision support. To address this issue, this paper proposes the RAD method, which integrates Multi-Criteria Decision Making with the semantic understanding capabilities of LLMs. The method automatically extracts key criteria from industry documents, builds a weighted hierarchical decision model, and generates structured reports under model guidance. The RAD framework introduces explicit weight assignment and reasoning chains in decision generation to ensure accuracy, completeness, and traceability. Experiments show that in various decision-making tasks, the decision reports generated by RAD significantly outperform existing methods in terms of detail, rationality, and structure, demonstrating its application value and potential in complex decision support scenarios.

Figures

Figures reproduced from arXiv: 2505.18483 by the authors.

Figure 1
Figure 1. The pipeline of the method. 3.3 Text Processing To obtain decision criteria, the document set T = {ti} n i=1 must be processed. Through the document processing workflow, documents are divided into text blocks to allow large models to extract decision criteria from each block. Research [39] shows that using the document’s table of contents structure effectively supports subsequent text analysis tasks. The document se… view at source ↗
Figure 2
Figure 2. The figure displays the direct win rate percentages of the row condition over the column [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 39 canonical work pages

  1. [1]

    Grangel-González, I., & Vidal, M. E. (2021, April). Analyzing a knowledge graph of industry 4.0 standards. In Companion Proceedings of the Web Conference 2021 (pp. 16-25)

    work page 2021

  2. [2]

    Redwood, J., Thelning, S., Elmualim, A., & Pullen, S. (2017). The proliferation of ICT and digital technology systems and their influence on the dynamic capabilities of construction firms. Procedia Engineering, 180, 804-811

    work page 2017

  3. [3]

    R., & Vanthienen, J

    Gopal, R., Marsden, J. R., & Vanthienen, J. (2011). Information mining—Reflections on recent advancements and the road ahead in data, text, and media mining. Decision support systems, 51(4), 727-731

    work page 2011

  4. [4]

    Mendoza, M. L. Z., Agarwal, S., Blackshaw, J. A., Bol, V ., Fazzi, A., Fiorini, F., ... & Smith, A. M. (2024). A framework for developing a knowledge management platform. arXiv preprint arXiv:2406.12313

    work page Pith review arXiv 2024

  5. [5]

    Chalkidis, I., Fergadiotis, M., Manginas, N., Katakalou, E., & Malakasiotis, P. (2021). Regulatory compliance through Doc2Doc information retrieval: A case study in EU/UK legislation where text similarity has limitations. arXiv preprint arXiv:2101.10726

    work page Pith review arXiv 2021

  6. [6]

    G., & Medsker, L

    Small, S. G., & Medsker, L. (2014). Review of information extraction technologies and applications. Neural computing and applications, 25, 533-548

    work page 2014

  7. [7]

    (2024, October)

    Shu, D., Zhao, H., Liu, X., Demeter, D., Du, M., & Zhang, Y . (2024, October). LawLLM: Law large language model for the US legal system. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management (pp. 4882-4889)

    work page 2024

  8. [8]

    H., Jin, L., Elangovan, K., Abdullah, H

    Ke, Y . H., Jin, L., Elangovan, K., Abdullah, H. R., Liu, N., Sia, A. T. H., ... & Ting, D. S. W. (2025). Retrieval augmented generation for 10 large language models and its generalizability in assessing medical fitness. npj Digital Medicine, 8(1), 187

    work page 2025

Show all 47 references
  1. [9]

    & Larson, J

    Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., ... & Larson, J. (2024). From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130

    2024 arXiv

  2. [10]

    (2025, April)

    Xia, Y ., Zhou, J., Shi, Z., Chen, J., & Huang, H. (2025, April). Improving retrieval augmented language model with self-reasoning. In Proceedings of the AAAI conference on artificial intelligence (V ol. 39, No. 24, pp. 25534-25542)

    2025

  3. [11]

    & Wang, H

    Gao, Y ., Xiong, Y ., Gao, X., Jia, K., Pan, J., Bi, Y ., ... & Wang, H. (2023). Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997, 2, 1

    2023 arXiv

  4. [12]

    & Seifert, C

    Nauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y ., ... & Seifert, C. (2023). From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable ai. ACM Computing Surveys, 55(13s), 1-42

    2023

  5. [13]

    Rave, J. I. P., Álvarez, G. P. J., & Morales, J. C. C. (2021). Multi-criteria decision-making leveraged by text analytics and interviews with strategists. Journal of Marketing Analytics, 10(1), 30

    2021

  6. [14]

    Wu, Y ., Zhang, Z., Kou, G., Zhang, H., Chao, X., Li, C. C., ... & Herrera, F. (2021). Distributed linguistic representations in decision making: Taxonomy, key elements and applications, and challenges in data science and explainable artificial intelligence. Information Fusion...

    2021

  7. [15]

    Ghisellini, R., Pareschi, R., Pedroni, M., & Raggi, G. B. (2025). Recommending Actionable Strate- gies: A Semantic Approach to Integrating Analytical Frameworks with Decision Heuristics. arXiv preprint arXiv:2501.14634

    2025 arXiv

  8. [16]

    S., & Kumar, S

    Vaidya, O. S., & Kumar, S. (2006). Analytic hierarchy process: An overview of applications. European Journal of operational research, 169(1), 1-29

    2006

  9. [17]

    Saaty, R. W. (1987). The analytic hierarchy process—what it is and how it is used. Mathematical modelling, 9(3-5), 161-176

    1987

  10. [18]

    Attri, R., Dev, N., & Sharma, V . (2013). Interpretive structural modelling (ISM) approach: an overview. Research journal of management sciences, 2319(2), 1171. 10

    2013

  11. [19]

    Singh, R., & Bhanot, N. (2020). An integrated DEMATEL-MMDE-ISM based approach for analysing the barriers of IoT implementation in the manufacturing industry. International Journal of Production Research, 58(8), 2454-2476

    2020

  12. [20]

    (2024, May)

    Lei, H., Sun, C., Nie, M., Chen, X., Dong, Q., & Ma, F. (2024, May). Using the AHP-TOPSIS Integrated Model to Assess the Quality of Urban Environments. In International Conference on Artificial Intelligence for Society (pp. 3-15). Cham: Springer Nature Switzerland

    2024

  13. [21]

    M., Ng, A

    Blei, D. M., Ng, A. Y ., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of machine Learning research, 3(Jan), 993-1022

    2003

  14. [22]

    Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781

    2013 arXiv

  15. [23]

    Zhang, J., Gui, W., & Wen, J. (2024). China’s policy similarity evaluation using LDA model: An experimental analysis in Hebei province. Journal of Information Science, 50(2), 515-530

    2024

  16. [24]

    Granger, C. W. (1969). Investigating causal relations by econometric models and cross-spectral methods. Econometrica: journal of the Econometric Society, 424-438

    1969

  17. [25]

    N., & Scheines, R

    Spirtes, P., Glymour, C. N., & Scheines, R. (2000). Causation, prediction, and search. MIT press

    2000

  18. [26]

    G., & de Andrade Lopes, A

    Drury, B., Oliveira, H. G., & de Andrade Lopes, A. (2022). A survey of the extraction and applications of causal relations. Natural Language Engineering, 28(3), 361-400

    2022

  19. [27]

    Wang, J., Fu, J., Wang, R., Song, L., & Bian, J. (2025). PIKE-RAG: sPecIalized KnowledgE and Rationale Augmented Generation. arXiv preprint arXiv:2501.11551

    2025 arXiv

  20. [28]

    & Zhou, J

    Liang, L., Sun, M., Gui, Z., Zhu, Z., Jiang, Z., Zhong, L., ... & Zhou, J. (2024). Kag: Boosting llms in professional domains via knowledge augmented generation. arXiv preprint arXiv:2409.13731

    2024 arXiv

  21. [29]

    Sarawagi, S. (2008). Information extraction. Foundations and Trends® in Databases, 1(3), 261-377

    2008

  22. [30]

    Sheikh, M., & Conlon, S. (2012). A rule-based system to extract financial information. Journal of Computer Information Systems, 52(4), 10-19

    2012

  23. [31]

    A., Escobar, J

    Vásquez, J. A., Escobar, J. W., & Manotas, D. F. (2021). AHP–TOPSIS methodology for stock portfolio investments. Risks, 10(1), 4

    2021

  24. [32]

    Brown, M., & Shackel, P. (2023). Text mining oral histories in historical archaeology. International Journal of Historical Archaeology, 27(3), 865-881

    2023

  25. [33]

    O’Connor, B., Bamman, D., & Smith, N. A. (2011, December). Computational text analysis for social science: Model assumptions and complexity. In Second workshop on comptuational social science and the wisdom of crowds (NIPS 2011)

    2011

  26. [34]

    (2025, January)

    Zhang, Z., You, W., Wu, T., Wang, X., Li, J., & Zhang, M. (2025, January). A Survey of Generative Information Extraction. In Proceedings of the 31st International Conference on Computational Linguistics (pp. 4840-4870)

    2025

  27. [35]

    S., Ceder, G.,

    Dagdelen, J., Dunn, A., Lee, S., Walker, N., Rosen, A. S., Ceder, G., ... & Jain, A. (2024). Structured information extraction from scientific text with large language models. Nature Communications, 15(1), 1418

    2024

  28. [36]

    & Kiela, D

    Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V ., Goyal, N., ... & Kiela, D. (2020). Retrieval- augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33, 9459-9474

    2020

  29. [37]

    & Zhang, N

    Zhu, Y ., Wang, X., Chen, J., Qiao, S., Ou, Y ., Yao, Y ., ... & Zhang, N. (2024). Llms for knowledge graph construction and reasoning: Recent capabilities and future opportunities. World Wide Web, 27(5), 58

    2024

  30. [38]

    Yang, Y ., Peng, Q., Wang, J., & Zhang, W. (2024). Multi-llm-agent systems: Techniques and business perspectives. arXiv preprint arXiv:2411.14033

    2024 arXiv

  31. [39]

    Zangari, A., Marcuzzo, M., Rizzo, M., Giudice, L., Albarelli, A., & Gasparetto, A. (2024). Hierarchical text classification and its foundations: A review of current research. Electronics, 13(7), 1199

    2024

  32. [40]

    J., Wang, B., & Xiao, H

    Günther, M., Mohr, I., Williams, D. J., Wang, B., & Xiao, H. (2024). Late chunking: contextual chunk embeddings using long-context embedding models. arXiv preprint arXiv:2409.04701

    2024 arXiv

  33. [41]

    F., Bahroun, Z., & Basheer, N

    Ahmed, V ., Khatri, M. F., Bahroun, Z., & Basheer, N. (2023). Optimizing smart campus solutions: An evidential reasoning decision support tool. Smart Cities, 6(5), 2308-2346

    2023

  34. [42]

    Kotonya, N., & Toni, F. (2024). Towards a framework for evaluating explanations in automated fact verification. arXiv preprint arXiv:2403.20322. 11

    2024 arXiv

  35. [43]

    Prasad, A., Saha, S., Zhou, X., & Bansal, M. (2023). Receval: Evaluating reasoning chains via correctness and informativeness. arXiv preprint arXiv:2304.10703

    2023 arXiv

  36. [44]

    Eberhard, K. (2023). The effects of visualization on judgment and decision-making: a systematic literature review. Management Review Quarterly, 73(1), 167-214

    2023

  37. [45]

    gold standard

    Oral, E., Chawla, R., Wijkstra, M., Mahyar, N., & Dimara, E. (2023). From information to choice: A critical inquiry into visualization tools for decision making. IEEE Transactions on Visualization and Computer Graphics, 30(1), 359-369. 12 A Additional Experiments The RAD metho...

    2023

  38. [46]

    Electric Vehicle Industry Documents (EVID): Multi-Pollutant Emissions Standards for Model Years 2027 and Later Light-Duty and Medium-Duty Vehicles; National Electric Vehicle Infrastructure Standards and Requirements; Securing the Information and Commu- nications Technology and...

    2027

  39. [47]

    For this user requirement, which criterion is more suitable to be included in the decision model?

    Endangered Species Protection Documents (ESPD): Designation of Experimental Popula- tions; Enhancement of Survival and Incidental Take Permits; Regulations for Interagency Cooperation; Regulations Pertaining to Endangered and Threatened Wildlife and Plants. For each dataset, w...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.