Determining the Trust of Social Media Images

Abstract

Social media images play a central role in today's communication. These images are often susceptible to various forms of changes. Some changes aim at enhancing visual appeal. Some changes are deceptive, designed to manipulate perceptions, or spread misinformation. Ascertaining the trust of social media images is important due to the increasing reliance on these images. Many fake images remain visually unchanged. Instead, manipulations often occur outside of the image, such as misleading changes to time, location, or context. We propose a novel context-based service framework to ascertain the trust of social media images using only image metadata. This method is unique in its exclusive reliance on metadata. We introduce a series of analyses to ascertain the trust of social media images. Firstly, we introduce a novel framework designed to detect modifications using only the image metadata. We leverage a word-embedding-based approach to identify semantic inconsistencies in the metadata. These inconsistencies serve as indicators of modifications in images and are used to measure the severity of changes. Secondly, we develop a transformer model trained on a large image metadata corpus to detect changes in images. Experiments are conducted on Multimodal C4 dataset using both a general corpus and a context-based corpus. Experiments on context-based corpus increase accuracy by up to 20%. Further, we determine the provenance of social media images, representing changes through a version tree that structures different image versions based on the extent of modifications. Finally, we introduce a method to determine the likelihood of an image being fake by analyzing the intent behind modifications. We conduct experiments on two fact-checked image datasets, one covering general contexts and another focusing on a specific context. Results show that our approach achieves up to 80% accuracy, demonstrating the effectiveness of our framework in determining an image's trustworthiness.