Gen AI: Time for an honest assessment

Let’s be honest: many of us thought that by the end of 2024, gen AI would be well on its way to revolutionizing software development with complex code completed in seconds, independent agents all over the organization making decisions, and tools that help everyone in the business do their jobs better. It hasn’t been so easy. Despite significant effort and investments, most organizations have little tangible impact to show for it.For gen AI to start doing some of the incredible things it’s capable of, 2025 has to be the year of honest assessment in which leaders strip away the fantasy and understand what needs to be done so their people can become better engineers. Here’s the first honest assessment: the problem isn’t the technology—it’s us. We obsess over code generation and ignore the rest of the development life cycle. We throw tools at developers without changing how people work. In 2025, CIOs and other leaders will instead need to take a hard look at the entire development life cycle, from the moment someone has an idea to when code hits production. The biggest productivity gains often come from streamlining handoffs and reducing wait times, not from faster coding. Second honest assessment: most of us are flying blind and hoping for the best. Few organizations have meaningful productivity metrics in place, which makes it hard to accurately assess if what they’re doing works or to understand why it isn’t, so they can fix it. DORA metrics⁵ alone won’t cut it. Everything, from development throughput to actual value delivery to customers, has to be tracked. Third honest assessment: companies are focusing way too much on gen AI tools and not nearly enough on the people who use them. The lessons of the past—whether from cloud, software as a service (SaaS), or broader digital transformations—are particularly pertinent here. It’s never just tech when it comes to capturing gains from tech. In 2025, CIOs and their teams will need to redesign roles, adjust incentives, and potentially restructure teams to capture productivity gains. The choice is ours: keep chasing the latest AI tools and hoping for a miracle, or roll up our sleeves and do the real work of transformation. My New Year’s wish is for the latter!

⁵ A standard set of DevOps metrics from DevOps Research and Assessment (DORA) that evaluates process performance and maturity.