Scalable surveillance of e-cigarette products on Instagram and TikTok using computer vision.

ABSTRACT

INTRODUCTION: Instagram and TikTok, video-based social media platforms popular among adolescents, contain tobacco-related content despite the platforms' policies prohibiting substance-related posts. Prior research identified themes in e-cigarette-related social media posts using qualitative or text-based machine learning methods. We developed an image-based computer vision model to identify e-cigarette products in social media images and videos. METHODS: We created a dataset of 6,999 Instagram images labeled for 8 object classes mod or pod devices, e-juice containers, packaging boxes, nicotine warning labels, e-juice flavors, e-cigarette brand names, and smoke clouds. We trained a DyHead object detection model using a Swin-Large backbone, evaluated the model's performance on 20 Instagram and TikTok videos, and applied the model to 14,072 e-cigarette-related promotional TikTok videos (2019-2022; 10,276,485 frames). RESULTS: The model achieved the following mean average precision scores on the image test set e-juice container 0.89; pod device 0.67; mod device 0.54; packaging box 0.84; nicotine warning label 0.86; e-cigarette brand name 0.71; e-juice flavor name 0.89; and smoke cloud 0.46. The largest number of TikTok videos - 9,091 (65%) - contained smoke clouds, followed by mod and pod devices detected in 6,667 (47%) and 5,949 (42%) videos respectively. Prevalence of nicotine warning labels was the lowest, detected in 980 videos (7%). CONCLUSIONS: Deep learning-based object detection technology enables automated analysis of visual posts on social media. Our computer vision model can detect the presence of e-cigarettes products in images and videos, providing valuable surveillance data for tobacco regulatory science. IMPLICATIONS Prior research identified themes in e-cigarette-related social media posts using qualitative or text-based machine learning methods. We developed an image-based computer vision model to identify e-cigarette products in social media images and videos.We trained a DyHead object detection model using a Swin-Large backbone, evaluated the model's performance on 20 Instagram and TikTok videos featuring at least two e-cigarette objects, and applied the model to 14,072 e-cigarette-related promotional TikTok videos (2019-2022; 10,276,485 frames).The deep learning model can be used for automated, scalable surveillance of image- and video-based e-cigarette-related promotional content on social media, providing valuable data for tobacco regulatory science. Social media platforms could use computer vision to identify tobacco-related imagery and remove it promptly, which could reduce adolescents' exposure to tobacco content online.