Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality

Abstract

We introduce and study the concept of a “jagged technology frontier” to describe the uneven impact of artificial intelligence (AI) capabilities, where AI assistance improves performance for some tasks but worsens it for others, even within the same knowledge workflow and with a seemingly similar level of difficulty. In collaboration with the global management consulting firm Boston Consulting Group, we have developed realistic management consulting tasks and examined the human performance implications of using AI to perform complex and knowledge-intensive work. The preregistered experiment involved 758 knowledge workers. After establishing a performance baseline on similar tasks, subjects were randomly assigned to one of three conditions: no AI access, GPT-4 AI access, or GPT-4 AI access with a prompt engineering overview. For each one of a set of 18 realistic knowledge tasks within the frontier of AI capabilities ranging from creative to analytical tasks, subjects using AI outperformed those not using AI, completing 12.2% more tasks and completing them 25.1% more quickly on average while also delivering solutions of significantly improved quality. However, for a complex managerial task selected to be outside the frontier, subjects using AI were 19% less likely to produce correct solutions compared with those without AI, pointing to potential limitations of AI supporting knowledge workers. We discuss the positive and negative implications of AI-aided human performance in knowledge-intensive tasks. Funding: Financial support of the Harvard Business School Digital Data Design Institute and Division of Research and Faculty Development is acknowledged. Supplemental Material: The online appendix is available at https://doi.org/10.1287/orsc.2025.21838 .

中文摘要

我们引入并研究了“锯齿状技术前沿”这一概念，用以描述人工智能（AI）能力的不均衡影响——AI辅助在某些任务上能提升绩效，但在其他任务上却可能降低绩效，即使这些任务属于同一知识工作流程且难度看似相近。我们与全球管理咨询公司波士顿咨询集团合作，设计了逼真的管理咨询任务，并考察了使用AI执行复杂知识密集型工作时对人类绩效的影响。这项预先注册的实验涉及758名知识工作者。在建立类似任务的绩效基线后，受试者被随机分配到三种条件之一：无AI访问、GPT-4 AI访问、或GPT-4 AI访问并附有提示工程概述。对于AI能力前沿内的一组18项逼真知识任务（涵盖从创造性到分析性的任务），使用AI的受试者表现优于未使用AI的受试者：平均完成的任务数量多12.2%，完成速度快25.1%，且提供的解决方案质量显著更高。然而，对于一项被选为处于前沿之外的复杂管理任务，使用AI的受试者得出正确解决方案的可能性比未使用AI的受试者低19%，这指出了AI支持知识工作者的潜在局限性。我们讨论了AI辅助人类绩效在知识密集型任务中的积极和消极影响。资金：感谢哈佛商学院数字数据设计研究所及研究与教师发展部的财务支持。补充材料：在线附录见https://doi.org/10.1287/orsc.2025.21838。