Vision Neural Architecture Designs for Improved Robustness and Accuracy
Access status:
Open Access
Type
ThesisThesis type
Doctor of PhilosophyAuthor/s
Li, YanxiAbstract
Deep neural networks (DNNs), such as Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), have achieved impressive results in computer vision but remain vulnerable to input shifts from adversarial attacks or natural distribution changes. This undermines their ...
See moreDeep neural networks (DNNs), such as Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), have achieved impressive results in computer vision but remain vulnerable to input shifts from adversarial attacks or natural distribution changes. This undermines their reliability and limits real-world deployment. Current defences like adversarial training and distributionally robust optimization (DRO) aim to improve robustness but often sacrifice accuracy. Adversarial training reduces standard accuracy, while DRO struggles to balance in-distribution (ID) and out-of-distribution (OOD) performance. Overcoming this robustness-accuracy trade-off remains a key challenge in deep learning. Firstly, we propose Neural Architecture Dilation (NAD), a framework that improves robustness by integrating searched dilation architectures into pre-trained neural backbones. Building on this, we introduce Neural Architecture Dilation for Adversarial Robustness (NADAR), which formulates the dilation process as a constrained optimization problem. Secondly, we introduce TORA-ViTs, a transformer-based architecture that explicitly disentangles robust and predictive features through lightweight adapters, while an attention-based gated fusion mechanism dynamically balances their contributions based on input conditions. To optimize this fusion, we implement a two-phase training strategy. Thirdly, we propose EdgeNet, a plug-and-play module designed to enhance robustness by incorporating shape-based edge features into vision transformers. Utilizing a "sandwich" architecture with zero convolutions, EdgeNet introduces robust structural representations while preserving the predictive power of pretrained backbones. Lastly, we introduce AdaptNAS, a Neural Architecture Search (NAS) framework that explicitly optimizes architectures for both ID accuracy and OOD robustness. AdaptNAS leverages domain adaptation principles and adversarial learning to reduce the OOD generalization gap.
See less
See moreDeep neural networks (DNNs), such as Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), have achieved impressive results in computer vision but remain vulnerable to input shifts from adversarial attacks or natural distribution changes. This undermines their reliability and limits real-world deployment. Current defences like adversarial training and distributionally robust optimization (DRO) aim to improve robustness but often sacrifice accuracy. Adversarial training reduces standard accuracy, while DRO struggles to balance in-distribution (ID) and out-of-distribution (OOD) performance. Overcoming this robustness-accuracy trade-off remains a key challenge in deep learning. Firstly, we propose Neural Architecture Dilation (NAD), a framework that improves robustness by integrating searched dilation architectures into pre-trained neural backbones. Building on this, we introduce Neural Architecture Dilation for Adversarial Robustness (NADAR), which formulates the dilation process as a constrained optimization problem. Secondly, we introduce TORA-ViTs, a transformer-based architecture that explicitly disentangles robust and predictive features through lightweight adapters, while an attention-based gated fusion mechanism dynamically balances their contributions based on input conditions. To optimize this fusion, we implement a two-phase training strategy. Thirdly, we propose EdgeNet, a plug-and-play module designed to enhance robustness by incorporating shape-based edge features into vision transformers. Utilizing a "sandwich" architecture with zero convolutions, EdgeNet introduces robust structural representations while preserving the predictive power of pretrained backbones. Lastly, we introduce AdaptNAS, a Neural Architecture Search (NAS) framework that explicitly optimizes architectures for both ID accuracy and OOD robustness. AdaptNAS leverages domain adaptation principles and adversarial learning to reduce the OOD generalization gap.
See less
Date
2025Rights statement
The author retains copyright of this thesis. It may only be used for the purposes of research and study. It must not be used for any other purposes and may not be transmitted or shared with others without prior permission.Faculty/School
Faculty of Engineering, School of Civil EngineeringAwarding institution
The University of SydneyShare