pith. sign in

arxiv: 1906.08570 · v1 · pith:7N6DHUBRnew · submitted 2019-06-20 · 💻 cs.CL

Hindi Question Generation Using Dependency Structures

Kaveri Anuranjana , Vijjini Anvesh Rao , Radhika Mamidi This is my paper

Pith reviewed 2026-05-25 19:49 UTC · model grok-4.3

classification 💻 cs.CL
keywords Hindi question generationdependency parsingkaraka rolesrule-based systemIndoWordNetquestion transformationsemantic filters
0
0 comments X

The pith

A rule-based system using karaka dependencies generates multiple diverse questions from each Hindi sentence.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper presents a rule-based system to automatically generate questions in Hindi, addressing the shortage of training data for Hindi question answering systems. It formalizes transformations based on karaka roles identified by a dependency parser and semantic categories from IndoWordNet to produce interrogatives. The system allows one sentence to yield multiple questions through different rule applications, with semantic and syntactic filters applied to maintain quality. Manual evaluation by annotators assesses the generated questions on semantic and syntactic scales. If successful, this method can substantially expand available Hindi QA datasets beyond the number of input sentences.

Core claim

By applying question transformation rules derived from karaka-dependency theory, marking roles with a Hindi dependency parser, and using IndoWordNet for semantic categorization, the system generates diverse questions from input sentences, with filters ensuring higher quality outputs that can be manually validated.

What carries the argument

Karaka-dependency based transformation rules that map sentence structures to interrogatives, guided by parser-marked roles and ontology-derived categories, with added semantic and syntactic filters.

If this is right

  • One sentence produces multiple questions, increasing dataset size significantly.
  • Semantic and syntactic filters improve the validity of generated questions.
  • Manual annotation provides evaluation on both semantic and syntactic correctness.
  • Multiple generations from the same karaka role demonstrate the flexibility of the rules.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The approach could be adapted to other Indian languages with similar dependency structures.
  • Generated questions might serve as weak supervision for training neural question generation models.
  • Integration with existing Hindi parsers could automate the entire pipeline for large-scale data creation.

Load-bearing premise

The Hindi dependency parser must accurately identify karaka roles and IndoWordNet must correctly classify semantic categories for the transformation rules to produce valid questions.

What would settle it

Run the system on a held-out set of Hindi sentences and check whether the number of generated questions fails to significantly exceed the number of input sentences or whether annotators rate most outputs as semantically or syntactically invalid.

Figures

Figures reproduced from arXiv: 1906.08570 by Kaveri Anuranjana, Radhika Mamidi, Vijjini Anvesh Rao.

Figure 1
Figure 1. Figure 1: Karaka dependency labels for the sentence “Kal Raam ne [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
read the original abstract

Hindi question answering systems suffer from a lack of data. To address the same, this paper presents an approach towards automatic question generation. We present a rule-based system for question generation in Hindi by formalizing question transformation methods based on karaka-dependency theory. We use a Hindi dependency parser to mark the karaka roles and use IndoWordNet a Hindi ontology to detect the semantic category of the karaka role heads to generate the interrogatives. We analyze how one sentence can have multiple generations from the same karaka role's rule. The generations are manually annotated by multiple annotators on a semantic and syntactic scale for evaluation. Further, we constrain our generation with the help of various semantic and syntactic filters so as to improve the generation quality. Using these methods, we are able to generate diverse questions, significantly more than number of sentences fed to the system.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript describes a rule-based system for generating Hindi questions from declarative sentences. It formalizes transformation rules based on karaka-dependency theory, employs a Hindi dependency parser to label karaka roles, and uses IndoWordNet to identify semantic categories of role heads for selecting interrogative words. The authors note that multiple questions can be generated from the same karaka role via different rules, apply semantic and syntactic filters, and evaluate outputs via manual annotation by multiple annotators on semantic and syntactic scales. The central claim is that the method produces diverse questions in significantly greater number than the input sentences.

Significance. If the quantitative results and component accuracies hold, the work could help alleviate data scarcity for Hindi question-answering systems by providing a linguistically grounded way to expand training data. The explicit use of karaka theory and existing lexical resources (IndoWordNet) is a methodological strength that could generalize to other Indic languages, though the absence of reported metrics prevents assessment of practical utility.

major comments (2)
  1. [Abstract / Evaluation] Abstract and evaluation description: the claim that the system generates 'significantly more' questions than input sentences is presented without any quantitative results (e.g., average questions per sentence, total generations, or fraction retained after filtering). This directly undermines verification of the headline multiplicity result.
  2. [Approach / Evaluation] Approach and evaluation sections: no precision, recall, or error analysis is reported for the Hindi dependency parser's karaka-role labeling or for IndoWordNet semantic-category assignment on the evaluation corpus. These two components are load-bearing for the validity of every rule application; non-trivial error rates would invalidate the diversity claim without invalidating the rule formalism itself.
minor comments (1)
  1. [Abstract] The abstract would be strengthened by including at least one key quantitative figure (e.g., questions per sentence or acceptance rate after annotation) so readers can immediately gauge the scale of the result.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the insightful comments on our manuscript. We address each major comment point by point below.

read point-by-point responses

  1. Referee: [Abstract / Evaluation] Abstract and evaluation description: the claim that the system generates 'significantly more' questions than input sentences is presented without any quantitative results (e.g., average questions per sentence, total generations, or fraction retained after filtering). This directly undermines verification of the headline multiplicity result.

    Authors: We agree that the claim in the abstract is not supported by quantitative results in the current manuscript. We will revise the abstract and evaluation section to include specific quantitative results, such as the average number of questions generated per input sentence, the total number of generations, and the fraction retained after applying filters. revision: yes

  2. Referee: [Approach / Evaluation] Approach and evaluation sections: no precision, recall, or error analysis is reported for the Hindi dependency parser's karaka-role labeling or for IndoWordNet semantic-category assignment on the evaluation corpus. These two components are load-bearing for the validity of every rule application; non-trivial error rates would invalidate the diversity claim without invalidating the rule formalism itself.

    Authors: We acknowledge that the manuscript does not report precision, recall, or error analysis for the dependency parser's karaka-role labeling and IndoWordNet assignments. These are important for validating the approach. We will add an error analysis subsection in the revised manuscript, including accuracy metrics for these components on the evaluation corpus. revision: yes

Circularity Check

0 steps flagged

No circularity: rule-based system with external components

full rationale

The paper presents a rule-based Hindi question generation system that applies transformation rules derived from karaka-dependency theory to outputs from an external Hindi dependency parser and IndoWordNet ontology. No mathematical derivations, fitted parameters, or predictions are claimed; the multiplicity of questions per sentence is an observed empirical outcome after manual filtering and annotation, not a result forced by definition or self-citation. The abstract and method sections rely on external tools without any self-referential loops or load-bearing citations to prior author work that would reduce the central claim to its inputs. This is a standard applied NLP pipeline with no circular elements.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the reliability of external tools (Hindi dependency parser and IndoWordNet) and the completeness of the hand-crafted transformation rules derived from karaka theory; no free parameters or new entities are introduced.

axioms (1)
  • domain assumption Karaka-dependency theory provides a sufficient basis for formalizing question transformation methods from parsed sentences.
    The system is built by formalizing question transformation methods based on karaka-dependency theory as stated in the abstract.

pith-pipeline@v0.9.0 · 5676 in / 1274 out tokens · 32881 ms · 2026-05-25T19:49:29.535298+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 22 canonical work pages · 4 internal anchors

  1. [1]

    Dependency annotation scheme for indian lan- guages

    [Begum et al., 2008] Rafiya Begum, Samar Husain, Arun Dhwaj, Dipti Misra Sharma, Lakshmi Bai, and Rajeev Sangal. Dependency annotation scheme for indian lan- guages. In Proceedings of the Third International Joint Conference on Natural Language Processing: V olume-II ,

    work page 2008

  2. [2]

    Indoword- net

    [Bhattacharyya, 2017] Pushpak Bhattacharyya. Indoword- net. In The WordNet in Indian Languages , pages 1–18. Springer,

    work page 2017

  3. [3]

    Webshodh: A code mixed factoid question answering sys- tem for web

    [Chandu et al., 2017] Khyathi Raghavi Chandu, Manoj Chinnakotla, Alan W Black, and Manish Shrivastava. Webshodh: A code mixed factoid question answering sys- tem for web. In International Conference of the Cross- Language Evaluation F orum for European Languages , pages 104–111. Springer,

    work page 2017

  4. [4]

    Attention-over- attention neural networks for reading comprehension

    [Cui et al., 2016] Yiming Cui, Zhipeng Chen, Si Wei, Shi- jin Wang, Ting Liu, and Guoping Hu. Attention-over- attention neural networks for reading comprehension. arXiv preprint arXiv:1607.04423,

    work page arXiv 2016

  5. [5]

    New collection an- nouncement: Focused retrieval over the web

    [Habernal et al., 2016] Ivan Habernal, Maria Sukhareva, Fi- ana Raiber, Anna Shtok, Oren Kurland, Hadar Ronen, Judit Bar-Ilan, and Iryna Gurevych. New collection an- nouncement: Focused retrieval over the web. In Pro- ceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval , pages 701–704. ACM,

    work page 2016

  6. [6]

    Teaching ma- chines to read and comprehend

    [Hermann et al., 2015] Karl Moritz Hermann, Tomas Ko- cisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching ma- chines to read and comprehend. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, ed- itors, Advances in Neural Information Processing Systems 28, pages 1693–1701. Curran Associates, Inc.,

    work page 2015

  7. [7]

    Reinforced Mnemonic Reader for Machine Reading Comprehension

    [Hu et al., 2017] Minghao Hu, Yuxing Peng, Zhen Huang, Xipeng Qiu, Furu Wei, and Ming Zhou. Reinforced mnemonic reader for machine reading comprehension. arXiv preprint arXiv:1705.02798,

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  8. [8]

    Automatic question generation from hindi text using hy- brid approach

    [Kaur and Bathla, ] Jaspreet Kaur and Ashok Kumar Bathla. Automatic question generation from hindi text using hy- brid approach. [Keikha et al., 2014] Mostafa Keikha, Jae Hyun Park, and W Bruce Croft. Evaluating answer passages using sum- marization measures. In Proceedings of the 37th interna- tional ACM SIGIR conference on Research & development in info...

    work page 2014

  9. [9]

    A framework for automatic ques- tion generation from text using deep reinforcement learn- ing

    [Kumar et al., 2018] Vishwajeet Kumar, Ganesh Ramakrish- nan, and Yuan-Fang Li. A framework for automatic ques- tion generation from text using deep reinforcement learn- ing. arXiv preprint arXiv:1808.04961,

    work page arXiv 2018

  10. [10]

    Integrating question classifica- tion and deep learning for improved answer selection

    [Madabushi et al., 2018] Harish Tayyar Madabushi, Mark Lee, and John Barnden. Integrating question classifica- tion and deep learning for improved answer selection. In Proceedings of the 27th International Conference on Com- putational Linguistics, pages 3283–3294,

    work page 2018

  11. [11]

    Gen- erating instruction automatically for the reading strategy of self-questioning

    [Mostow and Chen, 2009] Jack Mostow and Wei Chen. Gen- erating instruction automatically for the reading strategy of self-questioning. In AIED, pages 465–472,

    work page 2009

  12. [12]

    Nanda, M

    [Nanda et al., 2016] G. Nanda, M. Dua, and K. Singla. A hindi question answering system using machine learning approach. In 2016 International Conference on Compu- tational Techniques in Information and Communication Technologies (ICCTICT), pages 311–314, March

    work page 2016

  13. [13]

    SQuAD: 100,000+ Questions for Machine Comprehension of Text

    [Rajpurkar et al., 2016] Pranav Rajpurkar, Jian Zhang, Kon- stantin Lopyrev, and Percy Liang. Squad: 100,000+ ques- tions for machine comprehension of text. arXiv preprint arXiv:1606.05250,

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  14. [14]

    Mctest: A challenge dataset for the open-domain machine comprehension of text

    [Richardson et al., 2013] Matthew Richardson, Christo- pher JC Burges, and Erin Renshaw. Mctest: A challenge dataset for the open-domain machine comprehension of text. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 193–203,

    work page 2013

  15. [15]

    Hindi-english cross-lingual question-answering system

    [Sekine and Grishman, 2003] Satoshi Sekine and Ralph Gr- ishman. Hindi-english cross-lingual question-answering system. ACM Transactions on Asian Language Informa- tion Processing (TALIP), 2(3):181–192,

    work page 2003

  16. [16]

    Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus

    [Serban et al., 2016] Iulian Vlad Serban, Alberto Garc ´ıa- Dur´an, Caglar Gulcehre, Sungjin Ahn, Sarath Chan- dar, Aaron Courville, and Yoshua Bengio. Generating factoid questions with recurrent neural networks: The 30m factoid question-answer corpus. arXiv preprint arXiv:1603.06807,

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  17. [17]

    Automated question an- swering using question templates that cover the concep- tual model of the database

    [Sneiders, 2002] Eriks Sneiders. Automated question an- swering using question templates that cover the concep- tual model of the database. In International Conference on Application of Natural Language to Information Systems , pages 235–239. Springer,

    work page 2002

  18. [18]

    Reading comprehension on squad

    [Taylor et al., ] Zachary Taylor, Brexton Pham, and Meghana Rao. Reading comprehension on squad. [V oorhees and Tice, 1999] Ellen M. V oorhees and Dawn M. Tice. The trec-8 question answering track evaluation. InIn Text Retrieval Conference TREC-8, pages 83–105,

    work page 1999

  19. [19]

    Gated self-matching net- works for reading comprehension and question answering

    [Wang et al., 2017] Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, and Ming Zhou. Gated self-matching net- works for reading comprehension and question answering. In Proceedings of the 55th Annual Meeting of the Associ- ation for Computational Linguistics (V olume 1: Long Pa- pers), volume 1, pages 189–198,

    work page 2017

  20. [20]

    Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks

    [Weston et al., 2015] Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M Rush, Bart van Merri ¨enboer, Ar- mand Joulin, and Tomas Mikolov. Towards ai-complete question answering: A set of prerequisite toy tasks. arXiv preprint arXiv:1502.05698,

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  21. [21]

    Wikiqa: A challenge dataset for open-domain ques- tion answering

    [Yang et al., 2015] Yi Yang, Wen-tau Yih, and Christopher Meek. Wikiqa: A challenge dataset for open-domain ques- tion answering. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing , pages 2013–2018,

    work page 2015

  22. [22]

    Survey on trends and methods of an intelligent answering system

    [Yogish et al., 2017] Deepa Yogish, TN Manjunath, and Ravindra S Hegadi. Survey on trends and methods of an intelligent answering system. In Electrical, Electronics, Communication, Computer , and Optimization Techniques (ICEECCOT), 2017 International Conference on , pages 346–353. IEEE, 2017

    work page 2017