FAITH / TIQ

FAITH method

Temporal question answering (QA) involves time constraints, with phrases such as “. . . in 2019” or “. . . before COVID”. In the former, time is an explicit condition, in the latter it is implicit. State-of-the- art methods have limitations along three dimensions. First, with neural inference, time constraints are merely soft-matched, giving room to invalid or inexplicable answers. Second, questions with implicit time are poorly supported. Third, answers come from a single source: either a knowledge base (KB) or a text corpus. We propose FAITH (FAIful Temporal Question Answering over Heterogeneous Sources) a temporal QA system that addresses these shortcomings. First, it enforces temporal constraints for faithful answering with tangible evidence. Second, it properly handles implicit questions. Third, it operates over heterogeneous sources, covering KB, text and web tables in a unified manner. The method has three stages: (i) understanding the question and its temporal conditions, (ii) retrieving evidence from all sources, and (iii) faithfully answering the question. As implicit questions are sparse in prior benchmarks, we introduce a principled method for generating diverse questions. Experiments show superior performance over a suite of baselines.

TIQ Benchmark

Existing benchmarks for temporal QA focus on a single information source (either a KB or a text corpus), and include only few questions with implicit constraints. we devise a new method for automatically creating temporal questions with implicit constraints, with systematic controllability of different aspects, including the relative importance of different source types (text, infoboxes, KB), fractions of prominent vs. long-tail entities, question complexity, and more.

We construct a new dataset named TIQ with 10,000 questions and answers accompanied by supporting evidence.

Sample questions from TIQ: