Visual Pretraining on Large-Scale Image Datasets

Abstract

This thesis focuses on large-scale visual pretraining in computer vision and addresses various limitations of previous approaches. It introduces a novel technique called Relative Contrastive Loss (RCL) to learn feature representations that encompass real-world semantic variations while respecting positive-negative relativeness. The thesis also presents UniVCL, a unified framework for unsupervised visual contrastive learning methods, leveraging a graph convolutional network (GCN) layer for improved object recognition accuracy. Additionally, the thesis explores the transferability gap between unsupervised and supervised pretraining, emphasizing the role of the multilayer perceptron (MLP) projector in enhancing transfer performance. HumanBench, a comprehensive benchmark for human-centric downstream tasks, is proposed, and a pretraining method called PATH is introduced to learn knowledge in human bodies. The findings confirm the effectiveness of the proposed methods in enhancing the practicality and performance of large-scale visual pretraining.