After all the hype of 2023, executives who funded several GenAI initiatives are impatient to see returns on investments, but organizations are struggling to prove and realize value. As the scope, scale and price of initiatives grow, aligning the economics (cost, risk and value) of GenAI is a top priority.

Gartner predicts:

At least 30% of GenAI projects will be abandoned after proof of concept by the end of 2025, due to poor data quality, inadequate risk controls, escalating costs or unclear business value.

In this podcast, expert analysts discuss the “Peculiarly Challenging Business Case for GenAI,” for which the Gartner team focuses on measuring value and quantifying return on investment.

About the Guest:

Host Frances Karamouzis is joined by expert analyst Nate Suda. Nate covers digital strategy, execution and value creation with a focus on maximizing stakeholder value. Nate is in Gartner’s CIO practice on the FEVR team (finance, economics, value and risk team).

Interest in Copilot for Microsoft 365 is surging. However, as more organizations experiment with and evaluate this generative AI offering, many questions are emerging about functionality, integration, cost, return on investment, risks, and approaches to deployment. In this podcast, Gartner experts discuss all of these areas and share current challenges, strategic recommendations and predictions for the future.

About the Guest:

Host Frances Karamouzis is joined by our expert analyst, Matthew Cain, who focuses on the intersection of technology, job skills and workforce culture. He is part of Gartner’s Digital Workplace research team, which advises executive leaders on planning and executing technology strategies that incorporate consumer, workforce and business trends.

As we think about the continued fervor over AI and generative AI (GenAI), 2024 is shaping up to be very different from 2023 in three specific ways:

Urgency and action — First, what we are starting to see among our clients is a sense of urgency, where enterprises are shifting from exploration to action. Last year was all about ideation. This year seems to be a lot more about implementation.

Technology stack — The technology stack continues to evolve across multiple layers. At the silicon layer, we are starting to see new AI supercomputing innovations and new technologies from cloud providers. In the application layers, we see application-specific integrated circuits (ASICs). There are also networking innovations as well as shifts at the model layer (from large language models to multimodal models).

Emergence of agents — We are seeing the emergence of a new area of agents as well as “agent-to-agent” ecosystems focused on connecting and planning approaches to reasoning for purposes of taking action. All of this is being combined with context and memories integrated with our systems and software.

In this podcast, our expert analyst Chirag Dekate shares his insight and recommended actions to help I&O leaders deal with the realities of AI and generative AI in all three of those dimensions.

About the Guest:

Host Frances Karamouzis is joined by our expert analyst, Chirag Dekate. Chirag’s research focuses on providing strategic advice on generative AI systems, engineering AI pilots into production across a hybrid and multicloud context with an emphasis on AI (generative AI) infrastructures, quantum technologies (quantum computing, quantum sensing, quantum networking), high-performance computing, and advanced analytics infrastructures (quantum computing, neuromorphic, GPUs and beyond).

The challenges of a modern, data-driven enterprise demand modern tools capable of dealing with the volume and, more importantly, the diverse uses of personal data. In addition, the pace at which modern privacy regulations are proposed and adopted has continued to accelerate. This has fueled adoption of privacy technology by organizations looking to standardize a global privacy approach for handling personal data. Privacy-driven trust can serve as a key differentiator when customers are looking for a reason to pick one brand over another in a homogeneous market.

Strategic Planning Assumption

By the end of 2024, three-quarters of the world’s population will have its personal data covered by modern privacy regulations.

Executives seeking a positive balance between the organization’s overall success and corporate reputation should recognize that a mature privacy program is the entire organization’s responsibility. Privacy and data protection officers may take the lead, but CxOs have their respective responsibilities as well. In this podcast, we explore these issues and more.

Host Frances Karamouzis is joined by our expert analyst, Bart Willemsen. Willemsen focuses on privacy-related challenges in an international context, as well as on ethics, digital society, and the intersection with modern technology, including AI.

For 2024 and likely the next decade, business value creation will not happen without the successful blending of data and analytics (D&A) and software engineering at the core. Technology can be a failure point when not handled correctly, but it is often not the biggest roadblock to progress. Digital business acceleration will depend equally, if not more, on how you organize the required roles, skills and culture to drive this transformation.

Information technology’s ubiquity and centrality to business performance have inspired the current generation of senior executives to unprecedented levels of digital ambition. Upward of 80% of CxOs outside IT feel responsible for leading digital transformation, building digital business strategies and fostering technology-enabled innovation. Yet, only about one in five CxOs leads digital initiatives in ways that have a high likelihood of hitting value targets.1

It should come as no surprise then that two-thirds of CFOs report that their organizations’ returns from digital spending underperform expectations.2 Why is this? What accounts for the difference between CxOs that successfully drive business outcomes from digital initiatives and those who struggle? The answer has profound implications for boards of directors and CEOs whose digital ambitions are at all-time highs.

Due to these findings, we focused our survey of CxOs on trying to understand what makes senior business executives successful at maximizing returns from digital. To that end, Gartner gathered data on CxOs’ track records with value delivery. We also collected a lot of other data about these CxOs on demographics, behaviors, resourcing and interactions with CIOs. 

One of the topline findings of this Gartner study was that only 20% of CxOs consistently meet or exceed their outcome targets for their digital initiatives. Only one in five is quite low. Gartner has coined a term for these CxOs — the Digital Vanguard. They are very successful at digital, but they also approach and resource digital initiatives very differently from other types of CxOs.

CxOs in the Digital Vanguard are characterized by three key things:

• Co-lead digital delivery with IT.
• Engage frequently with their CIOs.
• Devote more staff and attention to tech work than the rest.

Rather than sponsor IT projects, they co-lead digital delivery with their CIOs, and dedicate their own staff (not just IT’s) to building, implementing and managing their business areas’ tech stacks.

Digital Vanguard CxOs are 1.5 to 2 times more likely to achieve their value targets from digital.

The ubiquitous impact of digital technologies should be a wake-up call for CxOs reluctant to take ownership of digital initiatives. It will be up to CxOs themselves to put digital to work in their business areas. This is a significant departure from the traditional way of managing technology projects, where business leaders sponsor initiatives upfront, but IT functions handle all of the delivery and maintenance. 

It will require:

• More CxO time dedicated to digital initiatives end to end, from strategy to execution.
• More business area staff dedicated to building, implementing or managing technology.
• New teaming structures that foster collaboration with technologists within and beyond dedicated IT departments.
• Ultimately, new CxO mindsets and behaviors.

The payoff is substantial: CxOs who take on end-to-end digital leadership responsibilities for their respective business areas are twice as likely to achieve outcomes from their investments in digital technologies compared with those who abdicate their digital leadership role.

Evidence

1 2023 Gartner Board of Directors Survey on Business Strategy in an Uncertain World. This survey was conducted to understand the new approaches adopted by nonexecutive boards of directors (BoDs) to drive growth in a rapidly changing business environment. The survey also sought to understand the BoDs’ focus on investments in digital acceleration; sustainability; and diversity, equity and inclusion. The survey was conducted online from June through July 2022 among 281 respondents from North America, Latin America, Europe and Asia/Pacific. Respondents came from all industries, except governments, nonprofits, charities and NGOs, and from organizations with $50 million or more in annual revenue. Respondents were required to be a board director or member of a corporate board of directors. If respondents served on multiple boards, they answered for the largest company, defined by its annual revenue, for which they were a board member. 

2 2023 Gartner Strengthening CxO Digital Leadership Survey. This survey was conducted to investigate how CxOs outside IT take on digital leadership and execution responsibilities, the extent to which they resource digital initiatives, and how they and their teams collaborate with their CIOs and IT departments. The research was conducted online from 22 February through 28 April 2023. In total, 618 respondents were interviewed in their native language across North America (n = 303; the U.S. and Canada), Latin America (n = 68; Brazil and Mexico), Western Europe (n = 145; the U.K., Spain, Germany, France, the Netherlands, Portugal, Belgium, Denmark, Finland and Luxembourg) and Asia/Pacific (n = 102; Australia, New Zealand, China, Hong Kong, India, Taiwan and Singapore). Qualifying organizations reported enterprisewide annual revenue for fiscal 2022 of at least $50 million or equivalent. Qualified participants had a role tied to a business unit (43% of respondents) or a corporate function (57% of respondents) and were members of senior management or above the midlevel management level (with 71% of respondents reporting to a CEO). 

Disclaimer: The results of these surveys do not represent global findings or the market as a whole, but reflect the sentiments of the respondents and companies surveyed.

The 2024 Gartner CIO and Technology Executive Survey found that 81% of respondents believe that building, developing or customizing digital technologies for the business area should be the responsibility of IT departments (led by CIOs). Only 15% of CIOs believe that this responsibility should be shared equally with business areas, according to the survey.1

The ultimate responsibility for building, developing or customizing this software, whether within the CIO organization or in lines of business, falls to software engineering leaders. These leaders are in a key position to enable their organization to become builders of software, because they are at the intersection of business and technical domains, between strategy and implementation. But, to do so, they must build a world-class software engineering organization. 

In this podcast, we explore the role of software engineers and developers as AI and generative AI are infused into their future. Gartner expert analysts discuss a few of the many layers of this complex topic in the following areas:

• Reality versus hype — The quest of business and IT leaders separating out what is reality versus hype in order to be pragmatic. More specifically, software engineering leaders must ensure they stay grounded on what is reality so they can prioritize and start piloting and experimenting.
• Value and measurement — All leaders need to be able to track and measure things so they can determine what they want to put in production, scale and make part of their long-term approach. Above and beyond that, they need to be able to quantify the value. 
• Skills and talent — This involves a deeper understanding of the value of software engineering expertise, performance and productivity.
• Trends and the future — As always, it is important for leaders to stay abreast of what is coming and how to prepare for it.

To address these important focus areas, we discuss several important concepts in the podcast. A few are highlighted below.

Reframe the ROI Conversation

The current ROI conversation is focused on cost reduction. Gartner experts are focused on guiding leaders to value generation. It is important to stop thinking of AI as cost-reduction mechanisms or a tool that could help reduce headcount. Instead, it’s important to focus on AI and GenAI as force multipliers that enhance developer experience to such an extent that they enable activities that deliver real business value. 

Amplification Fallacy

There is an idea that generative AI will “amplify” people’s skills. However, if you carefully think about the concept — “amplifying” something just makes it louder, it doesn’t make it better or higher quality. As such, it is important to identify and investigate the differentiated impact across the software development life cycle and specific developer skills. 

Some initial findings show that GenAI provides a bit more of a productivity boost for junior developers. However, there is also countervailing data that less experienced developers overtrust the outputs of GenAI and are thus more error prone and more likely to introduce security vulnerabilities. 

For more senior developers, the starting point is that they have the expertise to know what good looks like, as they already have deep knowledge of a problem space, of architectural standards, of best practices and experiential knowledge. Hence, if they are open to using new tools, experimentation and tinkering, they are the ones who can quickly iterate and figure out the best ways to prompt and interact with GenAI coding assistants.

Augmentation Versus Agency

One of the most critical and foundational concepts for the success of AI is trust — engendering trust for both the creators and consumers of the solutions. Software engineers are among the creators of the solutions. The spectrum of increasing trust begins with a low trust level where augmentation rules the day. As trust increases, more tasks are offloaded but not entire roles. Imagine an AI assistant in a craftsman’s workshop. As we arrive at a level of trust where we can offload roles, think of the full apprentice or journeyman. With increasing reliability comes increasing trust, and with increasing trust we transition from “tool-based extension” (augmentation) to “social extension” (we recognize AI as having agency).

Two of the many predictions Gartner analysts have published on this topic and we explore in the podcast are:

• By 2028, 75% of enterprise software engineers will use AI coding assistants, up from less than 10% in early 2023.
• By 2028, systematic adoption of AI coding assistants in 2023 will result in at least 36% compounded developer productivity growth.

Evidence

1 2024 Gartner CIO and Technology Executive Survey. This survey was conducted online from 2 May through 27 June 2023 to help CIOs determine how to distribute digital leadership across the enterprise and to identify technology adoption and functional performance trends. Ninety-seven percent of respondents led an information technology function. In total, 2,457 CIOs and technology executives participated, with representation from all geographies, revenue bands and industry sectors (public and private).

Disclaimer: The results of this survey do not represent global findings or the market as a whole, but reflect the sentiments of the respondents and companies surveyed.

Our host Frances Karamouzis is joined by Arun Batchu and Phillip Walsh, who are both expert analysts in Gartner’s software engineering leaders team. Batchu is a vice president, and he helps software engineering leaders build their software design, development and people strategies. Walsh is a senior principal analyst who helps software engineering leaders develop and implement strategies to build a world-class software engineering organization.

This TechWave podcast is based on Gartner’s research, The Impact of the “U.S. Executive Order on AI.”

U.S. President Joe Biden has issued an 

Executive Order on the Safe, Secure and Trustworthy Development and Use of Artificial Intelligence

 (the “EO”), which also underscores AI’s promise of innovation and competitive advantage. It specifically calls for Americans’ privacy and civil liberties’ protection, equity and civil rights advancement, and consumers’ and workers’ support, reinforcing the U.S. Blueprint for an AI Bill of Rights. 

The EO considers this “the most significant actions ever taken by any government to advance the field of AI safety.” The U.S. is sending a clear signal that AI and GenAI is far more than a disruptive technology; it has far-reaching consequences to impact every aspect of daily life, national and global economies, military matters, and the future of the planet. As such, unlike many other areas of technology or disruptive forces where government organizations have a low reaction time — this is different, and executives will be tested accordingly. 

One way that our expert analyst, Lydia Clougherty Jones, euphemistically summarizes the message of the EO during the podcast is: “Step Up or Step Aside.” The overall message is that if you are in executive leadership, you have responsibilities with regard to AI. You must proactively take measures to be compliant and prevent harm. As such, “Even if you are not ready for AI, you need to be AI ready.”

Executives should adjust leadership priorities, reconcile AI investment with redistributed risk and prepare today for a more regulated tomorrow:

• The Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (the “EO”) sends a strong message to the private and public sectors: AI is not a new technology; it’s a leadership responsibility. 
• The EO’s mandates and directives redistribute the risks of loss associated with responsible AI failures, which will affect current and future AI investments, and the projected corresponding ROI.
• The EO has major impacts on U.S. federal agencies and those who work with them. It will undoubtedly have a significant impact on the global AI vendor and solutions market, including to address the new standards for the development, functionality, use and output of AI.

The EO will have a far and wide impact on AI strategy and the ROI on AI investment. It will alter and redistribute the risk of loss from AI harms considered too costly for AI benefit or value. It will also change corporate and individual behaviors arising from new regulatory frameworks, alongside the industry self-regulation we are already beginning to see take effect. Together, this creates an opportunity for new market solutions, but the oversight of vendors by the government and the commercial sector will be different. In the podcast, several examples are discussed.

Note: Gartner does not provide legal advice or services, and its research should not be construed or used as such. While this podcast involves a discussion on a topic that has many legal issues, Gartner does not provide or apply any legal rules or terms to its clients’ or prospects’ specific business.

Our host Frances Karamouzis is joined by our expert analyst Lydia Clougherty Jones, a senior research director in the Data and Analytics group. She covers data and analytics strategy, D&A value, and derisking, among many other topics. Importantly, Jones practiced law for two decades — with a focus on emerging technologies and business transformation — before joining Gartner. Nevertheless, it is crucial to understand that Gartner does not provide legal advice or services, and its research should not be construed or used as such. While this podcast involves a discussion on a topic that has many legal issues, Gartner does not provide or apply any legal rules or terms to its clients’ or prospects’ specific business.

Generative AI has revealed applications’ potential to operate intelligently, which has created the expectation for intelligent applications. IT leaders must understand the foundational changes affecting applications and decide their strategy to ensure continued alignment to target business outcomes.

What are Intelligent Applications?

Intelligent applications include intelligence — which we define as learned adaptation to respond appropriately and autonomously — as a capability. This intelligence can be utilized in many use cases to better augment or automate work. 

As a foundational capability, intelligence comprises a number of AI-based services — especially machine learning, semantic enginesvector stores and connected data. Consequently, intelligent applications deliver experiences that dynamically adapt to user context and intent. Sometimes, user experiences are no longer necessary because applications interoperate with other applications autonomously.

Intelligent applications can synthesize their interfaces between other applications (self-integrating applications) — as well as users — in ways that are appropriate to the prevailing circumstances, and they can do so proactively (see Figure 2). For example, an intelligent application can pull functionality (i.e., ordering software from a catalog) into a conversational interface based on user intent and context, or adapt it to external APIs for data exchange.

Why Is This Trend Important?

The way applications work is changing dramatically. Intelligence — in the form of a suite of AI features and functionalities — is becoming a foundational capability. This is expanding the roles that applications can play across a broad range of employee- and customer-facing business activities, and between applications themselves: increasing their level of agency.

Intelligent applications transform the experiences of customers and employees, further impacting product owners, architects, developers and governing roles. As applications play a fundamental and pervasive role throughout our working and social lives, these transformations will have far-reaching consequences (e.g., in terms of the types of jobs available to future generations).

AI is surpassing the limits reached and imposed by traditional programming that uses explicit rules, relationships and instructions. AI learns rules implicitly. Combined with access to connected data, AI can model context and intent to operate autonomously. This can improve work through augmentation, or eliminate it through automation.

As AI continues to advance, it’s causing us to reappraise its capabilities and applications. The progress and speed of such advances — especially in the wake of generative AI applications such as ChatGPT — are providing insight into the nature of intelligence itself. AI can now mimic human behavior so successfully that it can not only help or even replace people at work, but it can also, in some circumstances, fool people into believing it’s human. As such, the scope of AI’s application to work and automation is shifting from routine and mundane tasks, such as invoice processing, to nonroutine and creative tasks, such as copywriting.

Why is this Trending?

Business disruption due to talent/skill shortages is one of the biggest external threats to business after economic threats, according to the 2023 Gartner’s Board of Directors Survey. Workforce (e.g., retention and hiring) is the second biggest priority for 2023 and 2024. The top priority is digital technology initiatives, with AI/machine learning considered the top breakthrough technology. 

Intelligent applications have entered the mainstream. Over 50% of respondents to the Gartner AI Use-Case ROI Survey reported that they have a form of intelligent application in their enterprise application portfolios. Yet, a lack of effective automation/tools is the biggest barrier to worker productivity, according to one-third of respondents to the 2023 Gartner Workforce Optimization Survey. 

Key to AI’s advance is content — facts modeled for human comprehension. Content includes text, image, video and audio formats. AI can now identify and extract facts from content and remodel these as data for processing. It can use this data as the source from which to synthesize new content — the generative in generative AI. Most enterprise data is in the form of content, such as documents, and central to all activities that involve people. 

Content also makes up the interfaces through which users interact with applications, and code is itself content. As such, intelligence extends to adapting applications’ form and function through re-composition, re-engineering their parts to optimize performance, extend reach and expand purpose.

What are the Business Implications?

Intelligence as a capability can apply to all applications. The impact and implications are therefore pervasive across all use cases touched by applications (operational-, employee- and customer-centric use cases). Examples include:

• Optimization and automation of business processes, such as inventory management. For example, generative AI working with AI-based automated stockout measures can deliver natural language insights to managers — ensuring the right level of inventory to match demand. This improves customer satisfaction and related financial metrics. 
  • Example: GA Telesis leveraged an AI-based application using Google’s Vertex Generative AI Platform, with its sales processes to synthesize purchase orders for aircraft replacement parts automatically.1 This significantly cut GA Telesis’ response times to sales inquiries, thus optimizing and preserving the customer experience.
• Assistance throughout the digital workplace to help with many tasks, including drafting documents, automating process workflows, answering questions and generating business insights. For example, digital workplace application suite vendors and their intelligent assistants, such as Microsoft Copilot and Google Duet AI. 
  • Example: Bank of England created a cognitive search application solution using Squirro to enhance its document search and internal knowledge management capabilities.2 This application used machine learning term extraction workflows, coupled with dashboards, to provide a more unified knowledge search system and streamline data management.
    
     
• Customer relationship management with chat-based interfaces facilitating agent-based and self-service support. For example, generative AI can produce an automated summary of a customer service agent’s audio interaction with a customer. The code writes one summary for the customer, indicating what advice was given. The code writes a second summary that summarizes the client’s issue and adds to the customer service knowledge base. 
  • Example: CallRail partnered with AssemblyAI to provide capabilities, such as automatic transcript highlights, and redaction of personally identifiable information.3 This not only provided customer service agents with essential insights much more quickly, but also improved CallRail’s call transcription accuracy by 23%.
    
     

The opportunities created by intelligent applications should be focused on expected outcomes, such as:

• Simplification and personalization of experiences for both employees and customers.
• Optimization of processes combined with a reduction in human error.
• Simplification of applications, and reduction of their number, to deliver business processes.