No Landslide for the Human Journalist

This is the latest addition to the research literature from our IA project, an article for IEEE –  “No Landslide for the Human Journalist: An Empirical Study of Computer-Generated Election News in Finland” with Magnus Melin as the first author.

“In an age of struggling news media, automated generation of news via natural language generation (NLG) methods could be of great help, especially in areas where the amount of raw input data is big, and the structure of the data is known in advance. One such news automation system is the Valtteri NLG system, which generates news articles about the Finnish municipal elections of 2017. To evaluate the quality of Valtteri-produced articles and to identify aspects to improve, $n=152$ users were asked to evaluate the output of Valtteri. Each evaluator rated six preselected computer-generated articles, four control articles written by journalists, and four computer-generated articles of their own choice. All the articles were evaluated along four dimensions: credibility, liking, quality, and representativeness. As expected, the texts written by Valtteri received lower ratings than those written by journalists, but overall the ratings were satisfactory (average 2.9 versus 4.0 for journalists on a five-point scale). Valtteri’s best rating (3.6) was for credibility. The computer-written articles that the evaluators could freely select got slightly better ratings than the preselected computer-written articles. When looking at the results by demographic groups, males aged 55 or more liked the automatic articles best and females aged 34 or less liked them the least. Evaluators mistook 21% of the computer-written articles as written by humans and 10% of the human-written articles as computer-written. The share of users making these mistakes grew with the age. Overall, the male evaluators made less writer-identification mistakes than female evaluators did.”

Event: The Future of News Automation

What is the future of News Automation? This is the main question we will discuss at the final event of the Immersive Automation project (2016-2018) where we will sum up what we learned. Link to video feed here 8pm CET

Our keynote speaker is David Caswell, Executive Product Manager of BBC News Labs and founder of Structured Stories. David works at the intersection between technology (especially data-driven technology), knowledge engineering and digital media.

We also have other excellent experts talking about the future of news automation, among them two pioneers from the Nordic countries, Magnus Aabech, Editor, editorial development at the Norwegian News Agency, NTB, and Sören Karlsson, CEO of United Robots from Sweden. From the Finnish news agency STT, we have Maija Paikkala, Business Developer and Key Account Manager and from the Finnish public broadcasting company Jukka Niva, Head of Yle News Lab. They are both experimenting with new forms of structured journalism.

At this event, we will also present the industry report “News Automation – A WAN-IFRA guide to the field” that we have produced for the World Association of Newspapers and News Publishers (WAN-IFRA).

In the report, we present five examples of how news automation has been implemented in newsrooms around the world: Mittmedia and United Robots (Sweden), Radar (UK), Washington Post (US), Valtteri (Finland), and Xinhua and Caixin (China).

This is a public event.

When: November 28, 2018, 9am to 12noon

Where: Swedish School of Social Science, University of Helsinki, Snellmaninkatu 12.


9:00 Introduction: Carl-Gustav Lindén, Docent, Swedish School of Social Science at University of Helsinki

9:05 Keynote: The future of news automation. David Caswell, Executive Product Manager of BBC News Labs and founder of Structured Stories.

09:45 Presentation: News Automation: A WAN-IFRA guide to the field, Carl-Gustav Lindén and Hanna Tuulonen, PhD student, University of Helsinki

10:00: Comments:

  • Magnus Aabech, Editor, editorial development, Norwegian News Agency, NTB
  • Sören Karlsson, CEO, United Robots
  • Maija Paikkala, Business Developer and Key Account Manager, Finnish News Agency, STT.
  • Jukka Niva, Head of Yle News Lab

11:00 Presentation: Embeddia, Professor Hannu Toivonen, University of Helsinki, Department of Computer Science

11:15 Panel discussion: David Caswell, Magnus Aabech, Maija Paikkala, Sören Karlsson, Jukka Niva. Moderator: Stefanie Sirén-Heikel, PhD student.

12:00 Programme ends

Contact person: Docent (Adjunct Professor) Carl-Gustav Linden,

David Caswell bio

David Caswell is the Executive Product Manager of BBC News Labs. He has previously led product management for data and machine learning at Tribune Publishing, and was the Director of Product Management for Content Understanding at Yahoo!. Caswell is also the developer of the Structured Stories platform, which demonstrated the representation of journalistic events and narratives as structured data under editorial control, and has written extensively on structured and automated journalism. He has also researched structured journalism as a Fellow at the Reynolds Journalism Institute at the Missouri School of Journalism, and co-organizes an annual workshop on event and storyline representation for news.