Pith. sign in

REVIEW 5 cited by

FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1711.07341 v2 pith:PQXGQBJE submitted 2017-11-16 cs.CL cs.AI

Hsin-Yuan Huang , Chenguang Zhu , Yelong Shen , Weizhu Chen This is my paper · ORCID
classification cs.CLcs.AI
keywords fusionnetattentionsquadbestconceptdatasetsfirstfully-aware
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

This paper introduces a new neural structure called FusionNet, which extends existing attention approaches from three perspectives. First, it puts forward a novel concept of "history of word" to characterize attention information from the lowest word-level embedding up to the highest semantic-level representation. Second, it introduces an improved attention scoring function that better utilizes the "history of word" concept. Third, it proposes a fully-aware multi-level attention mechanism to capture the complete information in one text (such as a question) and exploit it in its counterpart (such as context or passage) layer by layer. We apply FusionNet to the Stanford Question Answering Dataset (SQuAD) and it achieves the first position for both single and ensemble model on the official SQuAD leaderboard at the time of writing (Oct. 4th, 2017). Meanwhile, we verify the generalization of FusionNet with two adversarial SQuAD datasets and it sets up the new state-of-the-art on both datasets: on AddSent, FusionNet increases the best F1 metric from 46.6% to 51.4%; on AddOneSent, FusionNet boosts the best F1 metric from 56.0% to 60.7%.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Data-Centric Improvements for Enhancing Multi-Modal Understanding in Spoken Conversation Modeling

    cs.CL 2024-12 conditional novelty 6.0 of 10

    Adding transcription and answer-selection auxiliary tasks to a fixed spoken-QA dataset improves MLLM performance with less data, validated on three datasets and a new ASK-QA benchmark.

  2. QAInfomax: Learning Robust Question Answering System by Mutual Information Maximization

    cs.CL 2019-08 conditional novelty 6.0 of 10

    A mutual information maximization regularizer (QAInfomax) improves BERT's robustness on Adversarial-SQuAD, achieving state-of-the-art F1 on ADDSENT and ADDONESENT.

  3. Attentive History Selection for Conversational Question Answering

    cs.IR 2019-08 conditional novelty 6.0 of 10

    A BERT-based model with position-aware history answer embeddings and a learned history attention mechanism improves QuAC F1 by about one point over strong baselines, but multi-task learning with dialog acts does not h...

  4. Ensemble approach for natural language question answering problem

    cs.CL 2019-08 conditional novelty 4.0 of 10

    A class-weighted voting ensemble of BiDAF, QANet, and Mnemonic Reader reports F1 81.96 and EM 73.77 on the SQuAD dev set, marginally above Mnemonic Reader's 81.57 and 73.25.

  5. FlowDelta: Modeling Flow Information Gain in Reasoning for Conversational Machine Comprehension

    cs.CL 2019-08 conditional novelty 4.0 of 10

    Modeling the difference between consecutive reasoning states, called FlowDelta, improves conversational machine comprehension accuracy across FlowQA and BERT on CoQA, QuAC, and SCONE.

Pith tools