Benchmarking Agentic Newswriting via Journalistic Workflows

Recent advances in autonomous digital agents from industry (e.g., Manus AI and Gemini's research mode) highlight their potential for structured tasks through autonomous decision-making and task decomposition, but it remains unclear how well such systems support real-world information-intensive workflows. We study this question in journalism, where newswriting requires iterative planning, contextual reasoning, and active discovery of missing background to produce a coherent article. We introduce NEWSAGENT, a benchmark for evaluating how agents search raw materials, select relevant information, and iteratively revise drafts through core journalistic functions. Given a writing instruction and partial firsthand materials, agents must identify narrative perspectives, issue keyword-based queries, retrieve historical context, and generate complete news articles. Unlike typical summarization or retrieval tasks, essential context is not directly available and must be actively discovered, reflecting real-world reporting constraints. NEWSAGENT consists of 6k human-verified examples derived from real news. We evaluate open- and closed-sourced LLMs with commonly-used agentic frameworks on NEWSAGENT, which shows that agents are capable of retrieving relevant facts but struggling with planning and narrative integration. We believe that NEWSAGENT serves a realistic testbed for iterating and evaluating agent capabilities in terms of web data manipulation to real-world productivity.

Paper

References (23)

08. Introducing gemini deep research2025 · https://gemini.google/overview/ deep-research/
10. Introducing computer use, a new claude 3.5 sonnet, and claude 3.5 haiku2024 · Anthropic
12Reproducibility Details This section details the implementation. We first present the complete agentprompt

Scroll for more · 11 remaining

Similar papers

© 2026 NYSGPT2525 LLC