Deep Learning with Insufficient Data

Abstract

The recent success of deep learning relies heavily on substantial computational resources and large-scale labelled data. However, data insufficiency remains a major concern. This thesis addresses multiple aspects of data insufficiency across various learning tasks.

Without enough training data, deep models cannot fully explore the data space, which hampers generalization to unseen data. This issue is examined within the attribute-based person search task, where individuals are described by attributes like gender, age, clothing, or accessories in surveillance data. The challenge arises from non-overlapping training and testing classes (attribute combinations). We present a novel data augmentation technique generating synthetic features for unseen attribute combinations.

Insufficient data labels also lead to challenges. While unannotated data is accessible, data labelling is often expensive. Active learning, which strategically selects the most informative data for annotation, emerges as a key theme. This approach optimizes the annotation budget's utility.

Privacy-aware active learning is underexplored but crucial for applications using sensitive personal or medical data. Chapter 4 introduces a responsible active learning method, considering data privacy in cloud services by protecting local data while updating the active sampler. Federated active learning, studied in Chapter 5, allows decentralized learning without sharing local data. This method addresses non-IID data distribution with a novel knowledge-specialized reweighting.

Insufficiency of data labels also emerges when complex annotations by trained professionals are required, as in human-object interaction (HOI) detection. Chapter 6 addresses this using active learning, tackling challenges such as assessing image informativeness based on object labels, bounding boxes, and interaction labels. A comprehensive method for measuring data informativeness is introduced.