Pith. sign in

REVIEW 5 cited by

Improving Question Answering by Commonsense-Based Pre-Training

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1809.03568 v3 pith:EJOPEQ7U submitted 2018-09-05 cs.CL

Wanjun Zhong , Duyu Tang , Nan Duan , Ming Zhou , Jiahai Wang , Jian Yin This is my paper · ORCID
classification cs.CL
keywords commonsenseknowledgenetworkneuralansweransweringbasecommonsense-based
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Although neural network approaches achieve remarkable success on a variety of NLP tasks, many of them struggle to answer questions that require commonsense knowledge. We believe the main reason is the lack of commonsense \mbox{connections} between concepts. To remedy this, we provide a simple and effective method that leverages external commonsense knowledge base such as ConceptNet. We pre-train direct and indirect relational functions between concepts, and show that these pre-trained functions could be easily added to existing neural network models. Results show that incorporating commonsense-based function improves the baseline on three question answering tasks that require commonsense reasoning. Further analysis shows that our system \mbox{discovers} and leverages useful evidence from an external commonsense knowledge base, which is missing in existing neural network models and help derive the correct answer.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Incorporating External Knowledge into Machine Reading for Generative Question Answering

    cs.CL 2019-09 conditional novelty 6.0 of 10

    A model that learns when to draw answer words from a knowledge base instead of the passage can generate higher-quality answers on the MS MARCO benchmark.

  2. KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning

    cs.CL 2019-09 conditional novelty 6.0 of 10

    KagNet grounds question-answer pairs in ConceptNet schema graphs and uses a GCN-LSTM-HPA module to improve CommonsenseQA accuracy over BERT baselines.

  3. Align, Mask and Select: A Simple Method for Incorporating Commonsense Knowledge into Language Representation Models

    cs.CL 2019-08 conditional novelty 6.0 of 10

    Pre-training BERT on automatically generated multiple-choice questions from ConceptNet and Wikipedia improves commonsense benchmarks and leaves GLUE performance essentially unchanged.

  4. Knowledge Enhanced Attention for Robust Natural Language Inference

    cs.CL 2019-08 conditional novelty 5.0 of 10

    Injecting WordNet lexical relations as bias terms into multi-head attention improves accuracy on the adversarial SNLI test set, with BERT reaching 94.1%, equal to estimated human performance.

  5. Incorporating Relation Knowledge into Commonsense Reading Comprehension with Multi-task Learning

    cs.CL 2019-08 conditional novelty 5.0 of 10

    Adding ConceptNet relation-existence and relation-type auxiliary tasks to a BERT multiple-choice MRC model improves accuracy by about 0.7 points on SemEval-2018 Task 11 and Story Cloze Test.

Pith tools