Scraping Digital Media Platforms: Theory, Ethics and Data Analysis

Over the last five years, scraping has become increasingly popular amongst academic researchers, becoming a vital tool for digital methods research. In this three-part workshop, we will explain the theoretical underpinnings of scraping, explore high quality digital tools for academic research and introduce options for data analysis.

In the first part, we will explain scraping’s rise in importance, covering both the theoretical underpinnings of digital methods research – notably, its reliance on repurposed data – and a media environment in which access to large data sets via APIs (application programming interfaces) is increasingly restricted by technology companies. We will then explore the methodological implications of scraping, how it shifts the research perspective from platform overview to user experience and ethical considerations. In the second part, we will introduce some digital methods tools used in academic research to scrape platforms such as Google, Instagram, TikTok and X, and some options for data analysis. Finally, in the third part, we will provide time to experiment using these tools, with advice and supervision from the workshop leader.

Outcomes

Participants in this workshop will:

• Understand the broader contexts for the increasing importance of scraping in academic research;
• Understand the methodological implications of using data scraped from digital platforms;
• Become familiar with a range of scraping tools designed for academic research;
• Gain hands-on experience of using scraping tools relevant to their research.

Contributors

Dr Warren Pearce is Senior Lecturer in Digital Methods, Science and Society at University of Sheffield. Warren’s research is engaged with ‘science in public’: how science is organised, presented and reported by government, experts and media. His research has been published in a range of leading academic journals, such as Nature Climate Change; Information, Communication & Society; Public Understanding of Science; WIREs Climate Change; Environmental Communication.