Observability and security platform Datadog has shared findings from a recent project in which its engineering team used artificial intelligence tools to upgrade a critical production storage system. Engineer Arnold Wakim documented the process, results, and lessons learned in a technical article published by the company.
The project involved migrating an existing storage backend that had reached hard performance and capacity limits. According to Wakim, the team aimed to improve system performance and scalability while minimizing disruption to users.
AI Tools Used in Migration Workflow
The engineering team employed two AI-powered developer tools: Anthropic’s Claude and Cursor, a code editor that integrates large language models. These tools were used to assist with writing and reviewing code, generating tests, and identifying potential issues during the migration.
Wakim noted that the approach was not entirely new but represented a structured attempt to integrate AI into a high-stakes production change. He described the strategy as “test-driven production migration,” where AI assistance was paired with rigorous testing practices to ensure reliability.
What Worked
According to the report, the AI tools accelerated certain tasks such as generating boilerplate code, writing unit tests, and suggesting fixes for common issues. The team also found that the tools helped junior engineers contribute more effectively by reducing the time needed to learn the codebase.
Wakim stated that the AI tools performed well when given clear, bounded tasks with explicit instructions. The automated test generation, in particular, was cited as a time-saving benefit.
What Did Not Work
The engineer acknowledged several limitations. The AI tools occasionally produced incorrect or suboptimal code for more complex, domain-specific logic. In some cases, the tools suggested changes that passed tests but introduced subtle bugs that required manual review to catch.
Wakim also reported that the AI struggled with understanding the broader architectural context of the system. This led to recommendations that, while technically valid, were not aligned with long-term design goals.
Lessons Learned for AI-Assisted Engineering
Wakim emphasized that AI tools are not a replacement for human judgment. The team found that they achieved the best results when using the tools as co-pilots rather than autonomous agents. Human engineers remained responsible for architectural decisions and final code review.
Another key lesson was the importance of comprehensive test coverage before introducing AI-assisted changes. The team relied on an existing suite of automated tests to validate the output from the AI tools and catch regressions early.
Wakim also recommended that teams clearly define the boundaries of AI usage in critical systems. He advised against allowing the tools to operate on production infrastructure without oversight.
Performance and Reliability Outcomes
The migration ultimately succeeded, according to Datadog. The new system achieved performance improvements and eliminated the capacity bottlenecks that had constrained the previous backend. Wakim did not disclose specific performance metrics in the article.
The deployment process followed standard change management procedures, including staging, testing, and gradual rollout. The AI tools were not involved in the final deployment decisions or operational monitoring.
Industry observers note that Datadog’s experience reflects a broader trend in software engineering, where companies are experimenting with AI to increase productivity in maintenance and migration tasks. However, experts caution that the technology remains best suited for well-defined, low-risk parts of the development lifecycle.
Datadog has not announced plans to expand the use of Claude and Cursor across other teams. The company indicated that it will continue evaluating AI tools based on engineering needs and safety requirements.
