Gen AI as a software quality tool? Skepticism is fading as more organizations implement it

Is artificial intelligence (AI) the answer — or at least a partial answer — to nagging software quality issues? Software quality has been a challenge since the first computers were built eight decades ago, and in a world awash in technology networks and solutions, the problem has only grown more acute. A new study suggests that generative AI (gen AI) is emerging as an important step in managing quality.

According to a survey released by Capgemini and Sogeti (part of the Capgemini Group) which surveyed 1,755 tech executives, there is a growing emphasis on including gen AI within quality engineering. 68% of organizations are employing gen AI to assist their quality efforts. At least 29% of organizations have fully integrated gen AI into their test automation processes, while 42% are actively exploring its potential.

Also: Google’s new AI course will teach you to write more effective prompts – in 5 steps

“The evolution of large language models and AI tools, particularly Copilot, have enabled their seamless integration into existing software development lifecycles, ushering in a new wave of efficiency and innovation in quality engineering automation,” the survey’s team of authors, led by Jeff Spevacek of OpenText, stated.

In last year’s software quality survey, “we saw an uptick in the investments made by organizations in AI solutions to drive the quality-transformation agenda,” Spevacek and his co-authors wrote. “However, a significant number were skeptical about the value of AI in quality engineering.”

Also: Think AI can solve all your business problems? Apple’s new study shows otherwise

Moreover, attitudes toward AI have shifted significantly over the past 12 months, they further added. “A large number of organizations are now moving [away] from experimenting to real-scale implementation of gen AI to support quality engineering activities. We truly believe we will see further advancements in this area.”

Employing AI as a software quality assurance tool is not without its challenges, though. At least 61% of the respondents worry about data breaches associated with leveraging generative AI solutions.  A lack of comprehensive test automation strategies and reliance on legacy systems were identified by 57% and 64% of respondents, respectively, as key barriers to advancing automation efforts.

Also: Could AI make data science obsolete?

Some of the recommendations offered by the OpenText/Sogeti team for moving forward with automation and AI in software quality efforts include the following:

• Take an enterprise-wide view: Clearly outline “the objectives and desired outcomes of quality engineering automation and pre-selecting the areas where to apply, increase or enhance test automation.”
• Start now and keep experimenting: “If you are not yet exploring or actively using gen AI solutions, it’s crucial to begin now to stay competitive. Don’t rush to commit to a single platform or use case. Instead, experiment with multiple approaches to identify the ones that provide the most significant benefits.”  
• Leverage gen AI’s full range of capabilities: “Gen AI goes far beyond the generation of automated test scripts and helps with the realization of self-adaptive test automation systems.”
• Tie in business key performance indicators: “Identify, and leverage key business performance indicators influenced by quality engineering automation, with a clear focus on business outcomes, such as increased customer satisfaction, reduced cost of business operations, and others which are relevant to the business.”
• Rationalize quality engineering automation tools: “Ensure that your quality engineering automation tools are streamlined and capable of integrating with emerging technologies, such as gen AI, to maintain compatibility and future readiness.”
• Enhance quality engineering talent and roles: “Incorporate more full-stack quality and software development engineers in test to strengthen your team’s capabilities.”
• Enhance, don’t replace: “Understand that Gen AI will not replace your quality engineers but will significantly enhance their productivity. However, these improvements will not be immediate; allow sufficient time for the benefits to become apparent.”

Software quality engineering is rapidly evolving, the authors highlighted. “Once defined as testing human-written software, it has now evolved with AI-generated code.” Quality engineering is seeing an increased volume of code and test scripts that need to be generated, as well as requirements for testing software chains from end to end.