AI Assess AI: Automating Perceptual Quality Assessment for AI-Processed Immersive Media

Abstract

As AI technologies proliferate, AI-processed content (e.g., frame interpolation, ChatGPT) is everywhere. Yet judging its quality is critical. Subjective experiments, though time-consuming, yield limited, dataset-specific opinions, highlighting the need for automated QA with minimal labels.

Immersive media—VR, AR, MR—uses Light Field Imaging (LFI) and 3D reconstructions to create realistic, interactive experiences. Although methods like PSNR, SSIM, and LPIPS exist for 2D QA, immersive media remains underexplored. This thesis advances perceptual QA for immersive media, from LFI to 3D reconstructions.

Automatic QA is either full-reference or no-reference, but most modern content lacks references, making no-reference QA more relevant. First, I develop no-reference LFIQA. Because LFIs add an angular dimension, I adapt depthwise separable convolution for spatial features and propose anglewise separable convolution for comprehensive quality assessment. I also introduce anglewise attention—multihead self-attention, grid attention, and central attention—to improve feature extraction at lower complexity.

Next, I propose the first no-reference QA for photorealistic 3D reconstructed scenes from NeRF-like or 3D Gaussian methods. My approach integrates viewwise (spatial) and pointwise (angular) assessments, evaluating each synthesized view’s quality and inter-view consistency, plus the angular properties of surface points. Experiments confirm these no-reference methods outperform existing approaches, laying the groundwork for robust perceptual QA in immersive media.