Social media surveillance by artificial intelligence improves detection of vaccine adverse events

The use of Artificial Intelligence (AI) in surveillance of social media conversations improves detection of vaccine adverse events, according to a new Murdoch Children’s Research Institute-led study.

The study, which was published in Applied Clinical Informatics, found that applying AI techniques to social media vaccine-related conversations enhanced the discovery of genuine health complaints and demonstrated that social media could be a valuable data source for detecting mentions of vaccine side effects almost in real-time.

It also showed that analysing online conversations about vaccine-related personal health experiences could provide early warning about emerging vaccine safety issues.

Centre for Health Analytics and Surveillance of Adverse Events Following Vaccination In the Community (SAEFVIC) researcher Dr Sedigh Khademi’s method combines state-of-the-art AI techniques to greatly improve the detection of vaccine adverse event mentions from social media in a timely and cost-effective way.

Dr Khademi said, “the AI method employs techniques that understand text in a similar way to humans, allowing it to identify language around personal health mentions in Twitter conversations.”

“This method is highly effective at identifying personal health mentions and, when applied to social media, can detect existing and emerging adverse events in real-time. This can complement traditional reporting systems to improve the safety of future vaccine rollouts,” Dr Khademi said.  

Dr Khademi and her team collected three million public tweets from 1.4 million Twitter users over a 12-month period during the pandemic. The researchers then used COVID vaccine and vaccine reactions keywords to single out only vaccine-related personal health mentions (compared to non-health-related comments about vaccine hesitancy, for example).

They then used 4,000 records to train the AI tool and applied it to the entire dataset to detect adverse events. The team then compared their data with SAEFVIC to verify their results. SAEFVIC is a public health initiative of the Victorian Immunisation Program, based at Murdoch Children’s and funded by the Victorian Department of Health, which focuses on vaccine safety and surveillance.

The data obtained from this study was further validated in the Social Media Mining for Health (SMM4H) global social media mining competition. 

Murdoch Children’s Professor Jim Buttery’s team of Christopher Palmer, Dr Khademi, Dr Muhammad Javed and Dr Gerardo Luis Dimaguila used the new method to come second in the international competition in a task that involved classifying tweets self-identifying COVID-19 vaccination status.

Dr Dimaguila said, “This was a particularly interesting challenge as it aligns with a key research priority of SAEFVIC, leveraging social media monitoring to improve vaccine safety surveillance. This approach could be used in other studies to improve identification and reporting of adverse events following vaccination.”   

The team’s global runner-up efforts were described for others to learn from in the computational linguistics journal ACL Anthology.

Publications

1. Sedigh Khademi Habibabadi, Christopher Palmer, Gerardo Luis Dimaguila, Muhammad Javed, Hazel J Clothier, Jim Buttery. ‘AIDH Summit 2022 - Automated social media surveillance for detection of vaccine safety signals: a validation study,’ Applied Clinical Informatics. DOI: 10.1055/a-1975-4061
2. Christopher Palmer, Sedigh Khademi Habibabadi, Muhammad Javed, Gerardo Luis Dimaguila, Jim Buttery. ‘CHAAI@SMM4H’22: RoBERTa, GPT-2 and Sampling - An interesting concoction,’ ACL Anthology.