Analysing Generalisation Error Bounds For Convolutional Neural Networks

Abstract

Analysing Generalisation Error Bounds for Convolutional Neural Networks Abstract: Convolutional neural networks (CNNs) have achieved breakthrough performance in a wide range of applications including image classification, semantic segmentation, and object detection. Previous research on characterising the generalisability of neural networks has mostly focused on fully connected neural networks (FNNs), with CNNs regarded as a special case of FNNs without taking into account the special structure of convolutional layers; therefore, the CNN bounds may not be as tight as in FNNs. Here we propose a generalisation bound for CNNs by exploiting the sparse and shared structure of weight matrices for convolutional layers. As the new generalisation bound relies on the spectral norm of weight matrices, we further discuss the spectral norms for three convolution operations including standard convolution, depthwise convolution, and pointwise convolution. We show that our new bound for CNNs is indeed tighter than previously proposed under certain conditions.