Live Session
Chamber of Commerce
Poster
17 Oct
 
8:00
CEST
Thursday Posters
Add Session to Calendar 2024-10-17 08:00 am 2024-10-17 05:10 pm Europe/Rome Thursday Posters Thursday Posters is taking place on the RecSys Hub. Https://recsyshub.org
Research

It's (not) all about that CTR: A Multi-Stakeholder Perspective on News Recommender Metrics

View on ACM Digital Library

Hanne Vandenbroucke (imec-SMIT Vrije Universiteit Brussel) and Annelien Smets (imec-SMIT Vrije Universiteit Brussel)

View Paper PDFView Poster
Abstract

Recommendation systems are increasingly adopted by news media organizations. Existing literature examines various aspects of news recommendation systems (NRS) from a computational, user-centric, or normative perspective. Yet, scholars have argued to study the complexity of real-world applications surrounding the development and evaluation of NRS. More specifically, understand why different stakeholders within the organization prioritize different values for developing NRS and how those values are operationalized into metrics. Recently, a multi-stakeholder approach to NRS has been adopted, allowing to capture this complexity more comprehensively. However, limited research has focused on the distinct key performance indicators (KPIs) and metrics considered valuable by different stakeholders in evaluating NRS.Based on 11 interviews conducted with different professionals from two commercial media organizations in Flanders and The Netherlands, this paper shows that stakeholders prioritize different KPIs and metrics related to the reach-engagement-conversion-retention funnel. The evaluation of NRS performance is often limited to short-term metrics like CTR, overlooking the multiplicity of stakeholders involved. Our findings reveal how different purposes, KPIs, and metrics are valued from the journalistic, commercial, and tech logic. In doing so, this paper contributes to the multi-stakeholder approach to NRS, advancing our understanding of the real-world complexity of NRS development and evaluation.

Join the Conversation

Head to Slido and select the paper's assigned session to join the live discussion.

Conference Agenda

View Full Agenda →
No items found.