The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers

Abstract

This study evaluates the effect of generative artificial intelligence (AI) on software developer productivity via randomized controlled trials at Microsoft, Accenture, and an anonymous Fortune 100 company. These field experiments, run by the companies as part of their ordinary course of business, provided a random subset of developers with access to an AI-based coding assistant suggesting intelligent code completions. Although each experiment is noisy and results vary across experiments, when data are combined across three experiments and 4,867 developers, our analysis reveals a 26.08% increase (standard error: 10.3%) in completed tasks among developers using the AI tool. Notably, less experienced developers had higher adoption rates and greater productivity gains. This paper was accepted by Jean-Pierre Dube, marketing. Funding: M. Demirer and T. Salz thank the MIT GenAI Initiative for funding. Supplemental Material: The online appendices and data files are available at https://doi.org/10.1287/mnsc.2025.00535 .

中文摘要

本研究通过微软、埃森哲及一家匿名财富100强企业开展的随机对照试验，评估了生成式人工智能对软件开发者生产力的影响。这些由企业在其常规业务中实施的现场实验，为部分开发者随机提供了基于AI的代码补全智能建议工具。尽管每项实验存在噪声且结果各异，但综合三项实验及4867名开发者的数据后，分析显示使用AI工具的开发者完成任务数量增加了26.08%（标准误：10.3%）。值得注意的是，经验较少的开发者采用率更高，生产力提升也更显著。本文由Jean-Pierre Dube（市场营销领域）接收。资助信息：M. Demirer和T. Salz感谢MIT GenAI倡议提供的资助。补充材料：在线附录及数据文件可通过https://doi.org/10.1287/mnsc.2025.00535获取。