Multi-task Natural Language Understanding

Abstract

The field of Natural Language Understanding (NLU) concerns the mapping of textual utterances to representations of meaning upon which a machine can act. The standard representation is a dual level semantic frame; a sentence level intent and a token level annotation of the words semantically important to the fulfilment of the intent. The joint task of intent detection and slot labelling defines the framework of experimentation in NLU. There

are two problems with the current state of the field. Firstly, the two tasks should inform each other, but the models in the literature rely on implicit sharing of information, or if it is explicit it is unidirectional from one task to the other. Further, the dominant data sets used are single turn, that is single standalone sentences. However the natural environment for NLU should be multi-turn conversations between the human and the machine, with the topic of discussion changing over time.

In this thesis, we firstly design an architecture that incorporates an explicit, bi-directional exchange of information between the tasks during the learning process and outperforms the existing state of the art. We then adapt the annotation on two multi-turn conversational data sets used for dialogue state tracking to form NLU data sets. This offers an extra level to the semantic frame, that of a domain which changes during the conversation. We then pioneer NLU on this extended framework with the design of an explicit, tri-level model and show that the inclusion of an extra task improves performance of the dual level tasks. We design a multi-turn dual-level data set for toxicity detection in in-game chat. We are among the first to show that NLU architectures can be used to successfully analyse problems from other NLP fields when cast in the NLU framework.