Publications
2025
Hinterreiter, Smi; Wessel, Martin; Schliski, Fabian; Echizen, Isao; Latoschik, Marc Erich; Spinde, Timo
NewsUnfold: Creating a News-Reading Application That Indicates Linguistic Media Bias and Collects Feedback Proceedings Article Geplante Veröffentlichung
In: Proceedings of the International AAAI Conference on Web and Social Media (ICWSM'25), AAAI, Copenhagen, Denmark, Geplante Veröffentlichung, (Conditionally accepted for publication).
Abstract | Links | BibTeX | Schlagwörter: crowdsourcing, HITL, linguistic bias, media bias, news bias
@inproceedings{Hinterreiter2025NewsUnfold,
title = {NewsUnfold: Creating a News-Reading Application That Indicates Linguistic Media Bias and Collects Feedback},
author = {Smi Hinterreiter and Martin Wessel and Fabian Schliski and Isao Echizen and Marc Erich Latoschik and Timo Spinde},
url = {https://media-bias-research.org/wp-content/uploads/2024/07/Preprint_ICWSM_25_NewsUnfold.pdf},
year = {2025},
date = {2025-06-01},
urldate = {2025-06-01},
booktitle = {Proceedings of the International AAAI Conference on Web and Social Media (ICWSM'25)},
volume = {19},
publisher = {AAAI},
address = {Copenhagen, Denmark},
abstract = {Media bias is a multifaceted problem, leading to one-sided views and impacting decision-making. A way to address digital media bias is to detect and indicate it automatically through machine-learning methods. However, such detection is limited due to the difficulty of obtaining reliable training data. Human-in-the-loop-based feedback mechanisms have proven an effective way to facilitate the data-gathering process. Therefore, we introduce and test feedback mechanisms for the media bias domain, which we then implement on NewsUnfold, a news-reading web application to collect reader feedback on machine-generated bias highlights within online news articles. Our approach augments dataset quality by significantly increasing inter-annotator agreement by 26.31},
note = {Conditionally accepted for publication},
keywords = {crowdsourcing, HITL, linguistic bias, media bias, news bias},
pubstate = {forthcoming},
tppubtype = {inproceedings}
}
Media bias is a multifaceted problem, leading to one-sided views and impacting decision-making. A way to address digital media bias is to detect and indicate it automatically through machine-learning methods. However, such detection is limited due to the difficulty of obtaining reliable training data. Human-in-the-loop-based feedback mechanisms have proven an effective way to facilitate the data-gathering process. Therefore, we introduce and test feedback mechanisms for the media bias domain, which we then implement on NewsUnfold, a news-reading web application to collect reader feedback on machine-generated bias highlights within online news articles. Our approach augments dataset quality by significantly increasing inter-annotator agreement by 26.31
2024
Hinterreiter, Smi; Spinde, Timo; Oberdörfer, Sebastian; Echizen, Isao; Latoschik, Marc Erich
News Ninja: Gamified Annotation of Linguistic Bias in Online News Artikel Geplante Veröffentlichung
In: Proc. ACM Hum.-Comput. Interact., Bd. 8, Nr. CHI PLAY, Geplante Veröffentlichung, (Publisher: Association for Computing Machinery. Conditionally accepted for publication).
Abstract | Links | BibTeX | Schlagwörter: crowdsourcing, Game With A Purpose, linguistic bias, media bias, news bias
@article{Hinterreiter2024News,
title = {News Ninja: Gamified Annotation of Linguistic Bias in Online News},
author = {Smi Hinterreiter and Timo Spinde and Sebastian Oberdörfer and Isao Echizen and Marc Erich Latoschik},
url = {https://media-bias-research.org/wp-content/uploads/2024/07/Preprint_News_Ninja.pdf},
doi = {10.1145/3677092},
year = {2024},
date = {2024-10-14},
urldate = {2024-10-14},
journal = {Proc. ACM Hum.-Comput. Interact.},
volume = {8},
number = {CHI PLAY},
abstract = {Recent research shows that visualizing linguistic bias mitigates its negative effects. However, reliable automatic detection methods to generate such visualizations require costly, knowledge-intensive training data. To facilitate data collection for media bias datasets, we present News Ninja, a game employing data-collecting game mechanics to generate a crowdsourced dataset. Before annotating sentences, players are educated on media bias via a tutorial. Our findings show that datasets gathered with crowdsourced workers trained on News Ninja can reach significantly higher inter-annotator agreements than expert and crowdsourced datasets with similar data quality. As News Ninja encourages continuous play, it allows datasets to adapt to the reception and contextualization of news over time, presenting a promising strategy to reduce data collection expenses, educate players, and promote long-term bias mitigation.},
note = {Publisher: Association for Computing Machinery.
Conditionally accepted for publication},
keywords = {crowdsourcing, Game With A Purpose, linguistic bias, media bias, news bias},
pubstate = {forthcoming},
tppubtype = {article}
}
Recent research shows that visualizing linguistic bias mitigates its negative effects. However, reliable automatic detection methods to generate such visualizations require costly, knowledge-intensive training data. To facilitate data collection for media bias datasets, we present News Ninja, a game employing data-collecting game mechanics to generate a crowdsourced dataset. Before annotating sentences, players are educated on media bias via a tutorial. Our findings show that datasets gathered with crowdsourced workers trained on News Ninja can reach significantly higher inter-annotator agreements than expert and crowdsourced datasets with similar data quality. As News Ninja encourages continuous play, it allows datasets to adapt to the reception and contextualization of news over time, presenting a promising strategy to reduce data collection expenses, educate players, and promote long-term bias mitigation.
2020
Spinde, Timo; Hamborg, Felix; Donnay, Karsten; Becerra, Angelica; Gipp, Bela
Enabling News Consumers to View and Understand Biased News Coverage: A Study on the Perception and Visualization of Media Bias Proceedings Article
In: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020, S. 389–392, Association for Computing Machinery, Virtual Event, China, 2020, ISBN: 978-1-4503-7585-6.
Abstract | Links | BibTeX | Schlagwörter: bias visualization, news bias, news slant, perception of news
@inproceedings{spinde_enabling_2020,
title = {Enabling News Consumers to View and Understand Biased News Coverage: A Study on the Perception and Visualization of Media Bias},
author = {Timo Spinde and Felix Hamborg and Karsten Donnay and Angelica Becerra and Bela Gipp},
url = {https://doi.org/10.1145/3383583.3398619},
doi = {10.1145/3383583.3398619},
isbn = {978-1-4503-7585-6},
year = {2020},
date = {2020-01-01},
booktitle = {Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020},
pages = {389–392},
publisher = {Association for Computing Machinery},
address = {Virtual Event, China},
series = {JCDL '20},
abstract = {Traditional media outlets are known to report political news in a biased way, potentially affecting the political beliefs of the audience and even altering their voting behaviors. Many researchers focus on automatically detecting and identifying media bias in the news, but only very few studies exist that systematically analyze how theses biases can be best visualized and communicated. We create three manually annotated datasets and test varying visualization strategies. The results show no strong effects of becoming aware of the bias of the treatment groups compared to the control group, although a visualization of hand-annotated bias communicated bias in-stances more effectively than a framing visualization. Showing participants an overview page, which opposes different viewpoints on the same topic, does not yield differences in respondents' bias perception. Using a multilevel model, we find that perceived journalist bias is significantly related to perceived political extremeness and impartiality of the article.},
keywords = {bias visualization, news bias, news slant, perception of news},
pubstate = {published},
tppubtype = {inproceedings}
}
Traditional media outlets are known to report political news in a biased way, potentially affecting the political beliefs of the audience and even altering their voting behaviors. Many researchers focus on automatically detecting and identifying media bias in the news, but only very few studies exist that systematically analyze how theses biases can be best visualized and communicated. We create three manually annotated datasets and test varying visualization strategies. The results show no strong effects of becoming aware of the bias of the treatment groups compared to the control group, although a visualization of hand-annotated bias communicated bias in-stances more effectively than a framing visualization. Showing participants an overview page, which opposes different viewpoints on the same topic, does not yield differences in respondents' bias perception. Using a multilevel model, we find that perceived journalist bias is significantly related to perceived political extremeness and impartiality of the article.
Spinde, Timo; Hamborg, Felix; Gipp, Bela
An Integrated Approach to Detect Media Bias in German News Articles Proceedings Article
In: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020, S. 505–506, Association for Computing Machinery, Virtual Event, China, 2020, ISBN: 978-1-4503-7585-6.
Abstract | Links | BibTeX | Schlagwörter: content analysis, frame analysis, media bias, news bias, news slant
@inproceedings{spinde_integrated_2020,
title = {An Integrated Approach to Detect Media Bias in German News Articles},
author = {Timo Spinde and Felix Hamborg and Bela Gipp},
url = {https://doi.org/10.1145/3383583.3398585},
doi = {10.1145/3383583.3398585},
isbn = {978-1-4503-7585-6},
year = {2020},
date = {2020-01-01},
booktitle = {Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020},
pages = {505–506},
publisher = {Association for Computing Machinery},
address = {Virtual Event, China},
series = {JCDL '20},
abstract = {Media bias may often affect individuals' opinions on reported topics. Many existing methods that aim to identify such bias forms employ individual, specialized techniques and focus only on English texts. We propose to combine the state-of-the-art in order to further improve the performance in bias identification. Our prototype consists of three analysis components to identify media bias words in German news articles. We use an IDF-based component, a component utilizing a topic-dependent bias dictionary created using word embeddings, and an extensive dictionary of German emotional terms compiled from multiple sources. Finally, we discuss two not yet implemented analysis components that use machine learning and network analysis to identify media bias. All dictionary-based analysis components are experimentally extended with the use of general word embeddings. We also show the results of a user study.},
keywords = {content analysis, frame analysis, media bias, news bias, news slant},
pubstate = {published},
tppubtype = {inproceedings}
}
Media bias may often affect individuals' opinions on reported topics. Many existing methods that aim to identify such bias forms employ individual, specialized techniques and focus only on English texts. We propose to combine the state-of-the-art in order to further improve the performance in bias identification. Our prototype consists of three analysis components to identify media bias words in German news articles. We use an IDF-based component, a component utilizing a topic-dependent bias dictionary created using word embeddings, and an extensive dictionary of German emotional terms compiled from multiple sources. Finally, we discuss two not yet implemented analysis components that use machine learning and network analysis to identify media bias. All dictionary-based analysis components are experimentally extended with the use of general word embeddings. We also show the results of a user study.
Zustimmung verwalten
Um dir ein optimales Erlebnis zu bieten, verwenden wir Technologien wie Cookies, um Geräteinformationen zu speichern und/oder darauf zuzugreifen. Wenn du diesen Technologien zustimmst, können wir Daten wie das Surfverhalten oder eindeutige IDs auf dieser Website verarbeiten. Wenn du deine Zustimmung nicht erteilst oder zurückziehst, können bestimmte Merkmale und Funktionen beeinträchtigt werden.
Funktional Immer aktiv
Die technische Speicherung oder der Zugang ist unbedingt erforderlich für den rechtmäßigen Zweck, die Nutzung eines bestimmten Dienstes zu ermöglichen, der vom Teilnehmer oder Nutzer ausdrücklich gewünscht wird, oder für den alleinigen Zweck, die Übertragung einer Nachricht über ein elektronisches Kommunikationsnetz durchzuführen.
Vorlieben
Die technische Speicherung oder der Zugriff ist für den rechtmäßigen Zweck der Speicherung von Präferenzen erforderlich, die nicht vom Abonnenten oder Benutzer angefordert wurden.
Statistiken
Die technische Speicherung oder der Zugriff, der ausschließlich zu statistischen Zwecken erfolgt.
Die technische Speicherung oder der Zugriff, der ausschließlich zu anonymen statistischen Zwecken verwendet wird. Ohne eine Vorladung, die freiwillige Zustimmung deines Internetdienstanbieters oder zusätzliche Aufzeichnungen von Dritten können die zu diesem Zweck gespeicherten oder abgerufenen Informationen allein in der Regel nicht dazu verwendet werden, dich zu identifizieren.
Marketing
Die technische Speicherung oder der Zugriff ist erforderlich, um Nutzerprofile zu erstellen, um Werbung zu versenden oder um den Nutzer auf einer Website oder über mehrere Websites hinweg zu ähnlichen Marketingzwecken zu verfolgen.