FAITH method

Temporal question answering (QA) involves time constraints, with phrases such as “. . . in 2019” or “. . . before COVID”. In the former, time is an explicit condition, in the latter it is implicit. State-of-the- art methods have limitations along three dimensions. First, with neural inference, time constraints are merely soft-matched, giving room to invalid or inexplicable answers. Second, questions with implicit time are poorly supported. Third, answers come from a single source: either a knowledge base (KB) or a text corpus. We propose FAITH (FAIful Temporal Question Answering over Heterogeneous Sources) a temporal QA system that addresses these shortcomings. First, it enforces temporal constraints for faithful answering with tangible evidence. Second, it properly handles implicit questions. Third, it operates over heterogeneous sources, covering KB, text and web tables in a unified manner. The method has three stages: (i) understanding the question and its temporal conditions, (ii) retrieving evidence from all sources, and (iii) faithfully answering the question. As implicit questions are sparse in prior benchmarks, we introduce a principled method for generating diverse questions. Experiments show superior performance over a suite of baselines.

Overview of the FAITH pipeline. The figure illustrates the process of answering q3 (“Queen’s record company when recording Bohemian Rhapsody?” ) and q1 (“Record company of Queen in 1975?” ). For answering q3, two intermediate questions q31 and q32 are generated, and run recursively through the entire FAITH pipeline.

Related Papers

"Faithful Temporal Question Answering over Heterogeneous Sources", Zhen Jia, Philipp Christmann, and Gerhard Weikum. In WWW '24, Singapore, 13 - 17 May 2024.
[Preprint] [Code]

"TIQ: A Benchmark for Temporal Question Answering with Implicit Time Constraints", Zhen Jia, Philipp Christmann, and Gerhard Weikum. In TempWeb@WWW '24, Singapore, 14 May 2024.
[Code]

Download TIQ benchmark

We construct a new benchmark, TIQ (Temporal Implicit Questions), for temporal QA with 10,000 implicit questions. Questions are derived from heterogeneous sources: Wikipedia text, Wikipedia tables and the Wikidata KB. You can download it below:

Train Set (6000 Questions) Dev Set (2000 Questions) Test Set (2000 Questions)

Code

FAITH code TIQ code

Contact

For feedback and clarifications, please contact:

To know more about our group, please visit https://www.mpi-inf.mpg.de/departments/databases-and-information-systems/research/question-answering/.