2024 Itsym Keynote
2024 Itsym Keynote
Pacing Yourself in
the AI Races: 2024
IT Symposium/Xpo
Keynote Insights
21 October 2024
Pacing Yourself in the AI Races: 2024 IT
Symposium/Xpo Keynote Insights
21 October 2024 - ID G00822215 - 27 min read
By Analyst(s): Mary Mesaglio, Hung LeHong
Initiatives: CIO Innovation Leadership
Overview
Key Findings
With AI and data coming from everywhere, CIOs must deliver safe AI outcomes.
Depending on your organization’s ambitions, you will move at either an AI-steady or AI-
accelerated pace to deliver:
■ Technology outcomes. CIOs want to protect their organization’s data and govern AI
outputs while still providing enough flexibility to capitalize on new opportunities.
■ Behavioral outcomes. As AI changes their employees’ tasks, roles and lives, CIOs
want to use AI in a way that encourages positive behaviors and intentionally
manage its impact on employees.
■ Manage behavioral outcomes by clearly defining who owns which outcomes, and by
co-creating new roles with your employees. More advanced enterprises should add
human behavior experts to their teams, especially when experimenting with agentic
AI.
Introduction
Two AI races have begun.
In one race, tech vendors are relentlessly innovating and flooding the market with highly
hyped AI-embedded technologies. A new GenAI frontier model is released every 2.5 days
on average. 1
As a CIO, you’re not in that tech vendor race. You don’t have to adopt all the latest AI to get
ahead. But you are in the second race: a race to deliver AI outcomes safely and at scale.
The stakes of your AI outcomes race are high, because CEOs believe the AI hype is
justified. In the Mid-2024 Update Gartner CEO and Senior Business Executive Survey, 74%
of CEOs said AI is the technology that will most impact their industry. This is a significant
leap from 59% in early 2024, and 21% in 2023. 2
But CIOs are realizing that delivering AI outcomes is tough. In the 2024 Gartner AI Survey,
47% of CIOs said that AI has not met their ROI expectations. 3 The highly touted
productivity and business benefits of using AI don’t always materialize, and AI can create
serious risks, unpredictable costs and negative behaviors that harm your organization.
GenAI is teetering on the edge of the Peak of Inflated Expectations in Gartner’s Hype
Cycles, and it will soon slide into the Trough of Disillusionment (see Hype Cycle for
Emerging Technologies, 2024).
As a CIO, you experience the nonstop hype from AI vendors and the sobering reality of
how tough it is to deliver AI outcomes. It can feel like you’re at the peak and in the trough
at the same time!
Whether you have explicit ownership of AI or not, you are in a race to deliver three types of
outcomes (use the links below to navigate to each section):
■ Business outcomes
■ Technology outcomes
■ Behavioral outcomes
Fortunately, you get to set the pace in this race. In this research, we will focus on the two
paces that CIOs can take:
■ AI-steady pace. If you have modest AI ambitions and your industry is not yet being
disrupted by AI, you will go at a more measured pace. This pace is suitable for risk-
averse organizations, such as those in the public sector or in highly regulated
industries; as well as small and midsize organizations with limited resources to
spend on AI.
Analysis
Deliver Business Outcomes of AI
Depending on your organization’s ambitions for delivering business outcomes from AI,
you can set the appropriate pace:
To boost productivity with GenAI, you have to get people to consistently use GenAI tools.
In the 2024 Gartner Digital Worker Survey, digital workers who used everyday AI tools or
applications for work purposes at least once a week saved an average of 3.6 hours per
week. 5
However, not all employees benefit equally from using GenAI. Productivity gains from AI
vary based on the complexity of the role and the experience level of the employee (see
Who Benefits Most From Generative AI Productivity?).
Our 2024 Gartner AI Survey found that CIOs are pursuing three types of business
outcomes with enterprise GenAI initiatives:
To deliver these improvements, manage GenAI benefits like a portfolio. Direct some GenAI
initiatives at employee augmentation; some at classic ROI improvements, such as
redesigning business processes around GenAI; and some at business model innovation
centered on GenAI.
Determine the size of your bet for each benefit category, and manage risks and rewards
across this portfolio.
These costs will continue to rise. Vendors are raising prices by up to 30%. 7 By 2027,
Gartner predicts that the cost of most enterprise applications will increase by at least 40%
due to GenAI product pricing and packaging.
Regardless of your pace in the AI outcomes race, understand your AI bill. This includes
understanding the cost components. You also need to know how to reduce these costs
and negotiate with vendors (see Negotiate AI and Generative AI Pricing to Avoid
Skyrocketing Costs).
To discover potential benefits and effectively manage costs, use proofs of concept to
understand how your costs will scale. It’s not enough to prove that the tech works and
that employees like it. Use the proof of concept as a proof of value — in other words,
weigh the benefits achieved against the AI costs incurred.
Organizations with structured, centralized data in data lakes can use GenAI to derive value
from these sources. But GenAI will also unlock new value from your emails, recordings,
presentations and other unstructured data. This is a huge opportunity, as unstructured
8
data represents an estimated 70%-90% of all enterprise data.
Of course, GenAI creates new challenges, too. One key challenge is that GenAI models will
use any data they can access, including sensitive data and personally identifiable
information. Regardless of your AI pace, you must focus on managing data access
rights for your unstructured data.
Also, while GenAI may reduce the need to move, transform and structure all your data
using traditional approaches, CIOs still need sophisticated data management to unify
structured and unstructured data into data pipelines and orchestrations (see A Journey
Guide to Delivering AI Success Through ‘AI-Ready’ Data).
Allstate Technology Solutions used to take all this data and move it, clean it, tag it and
structure it. This data formed the basis for the machine learning models that properly
classified the accident, so you could get your claim check.
But now, Allstate does the opposite. Zulfi Jeevanjee, the EVP and CIO at Allstate, now
talks about leaving the data in its place — at the source. Allstate points its GenAI to the
location of call recordings, photos and other accident data. The GenAI models interpret
this raw data and combine it with structured policy data to classify the accident.
To ensure more precision, employees also get involved. They ask questions of the
GenAI to find answers in the data, in the same way that they ask you questions about
your accident.
In the 2024 Gartner AI Survey, CIOs said that their AI capabilities will, on average, be
9
spread across three main categories:
■ Embedded AI (43%). These are the AI capabilities that software vendors are adding
to the enterprise applications in your portfolio. Embedded AI is the largest and
fastest-growing segment of AI capabilities. By 2026, Gartner predicts that more than
80% of independent software vendors will have embedded GenAI capabilities in their
enterprise applications, up from less than 5% in 2024.
■ Built AI (35%). These are the centrally owned AI capabilities that your in-house
software engineering and data and analytics (D&A) teams are building.
■ Bring your own (BYO) AI (22%). These are the packaged AI software and capabilities
that departments within your organization are procuring and using.
As a CIO, you need a multilayered technology approach that supports both centralized and
decentralized ownership.
Like always, you still need a bottom layer of centralized data and AI that come from IT
and D&A — your traditional, bottom-up tech stack. But now, you also need a top layer that
harnesses data and AI coming from everywhere, such as your marketing or HR
departments.
Figure 2 shows how you can assemble these key ingredients to start building your AI tech
sandwich. For more details, see AI Technology Sandwich: A Conceptual Framework for
Executing AI.
The right strategy for building your TRiSM layer depends on your desired pace.
Use Governance Committees to Write and Enforce TRiSM Practices (AI-Steady Pace)
■ A central AI committee to manage demand (who wants AI, and where is it coming
from?)
Using these TRiSM practices is effective if you intend to scale out 10 or fewer AI
initiatives.
Every piece of AI in your organization must go through these TRiSM technologies. This
approach will enable you to safely scale AI outcomes.
The market for TRiSM technologies is moving rapidly, and it will become more
sophisticated (see Innovation Guide for Generative AI in Trust, Risk and Security
Management). As profiled below, we’re already seeing the emergence of AI agents that
are trained to keep AI behavior in check.
If you’re moving at an AI-steady pace, you will take longer to build your AI tech
sandwich. You’ll start by using AI vendor recommendations and packaged solutions, and
you’ll gradually refine your sandwich.
Every organization’s AI tech sandwich will be unique. As a starting point, consider three
archetypes:
■ Vendor-packaged sandwich. In this sandwich, you have two types of AI: embedded
AI from software upgrades and add-ons; and BYOAI capabilities from your business
units. You’ll also add a thin layer of TRiSM technologies to keep your vendors
honest. A packaged sandwich appeals to midsize enterprises that don’t have the
luxury of big AI teams.
■ Deluxe sandwich. In this sandwich, you have everything: a large layer of AI built by
your data science and engineering teams; a significant amount of BYOAI and
embedded AI; a generous helping of TRiSM technologies; and the ability to leverage
all kinds of data from everywhere. A deluxe sandwich is for the largest enterprises
that have the resources to include everything and believe that AI drives their
competitive edge.
Some employees may feel a strong affinity for AI. Others may feel threatened or
resentful. In the 2023 Gartner HR Technology Employee Experience Survey, only 39% of
employees reported that they believe AI solutions in their organization will produce fair
outcomes. 10
Figure 3 illustrates these archetypes.
But in many organizations, no one is managing these behavioral outcomes well. In the
2024 Gartner AI Survey, only 21% of CIOs said they focus on mitigating the potential
negative impacts of AI on employee work, and only 20% said they focus on mitigating the
3
potential negative impacts of AI on employee well-being.
If you’re moving at an AI-steady pace, you need to adapt your change management
approaches to focus on employee behaviors. Be intentional about who owns which
behavioral outcomes. Otherwise, you will often get accidental ownership of these
outcomes.
As part of your new change management approach, collaborate with your employees to
redesign their roles. Co-create the AI employee experience with the people whose lives are
most directly impacted.
Consider Sam, a Vizient software engineer. Sam’s work is augmented by using GitHub
Copilot. Vizient asked Sam how this might affect him. He recognized that his job
might shift to be more about problem solving and quality control than actually writing
code.
Sam realized that his identity might shift from coder to enhancer. Is that a job he
wants? Does he have the skills for that? Grappling with these questions is essential as
Vizient works with Sam and other employees to codesign new roles.
As you redesign employee roles, focus on using AI to remove drudgery from the lives of
your employees. By removing tedious tasks from an employee’s role, you’ll see more than
just productivity increases. You’ll also see an improvement in employee engagement,
product quality and customer experience.
AI has freed the natura consultants from administrative drudgery, so they can spend
their time on what matters most — using their passion and charisma to build trusted
relationships with clients.
If you are moving at an AI-steady pace, be explicit about the behavioral responses you
want to foster — and those you want to avoid — among employees, customers and
citizens. Then, augment your change management approach by rigorously tracking and
measuring those outcomes, as you would with business and technology outcomes.
The rise of “agentic AI” will lead to totally unfamiliar scenarios for humans (see
Innovation Insight: AI Agents). AI agents won’t just remind you of your meeting; they will
attend it for you. And AI agents won’t just present you with options; if you let them, they
will make decisions for you.
In your business, you might eventually have AI agents creating RFPs and procurement
agreements, evaluating responses or taking a host of other actions. For example, AI is
starting to harden the soft sciences by triggering and measuring emotions like empathy,
anger, trust and frustration.
These AI agents call patients to help them get ready for their upcoming operations or
to follow up after their visits. Hospitals can choose the tone for the AI agents.
Sometimes that tone is direct, like when reminding you to take your medicine. Other
times it is engaging, like when it’s checking up on you while you’re recovering.
Patients then rate the AI agents based on the feelings they evoke, with questions like:
With the rise of agentic AI, you have to be as rigorous about introducing humanity as you
are about introducing technology. In your organization, you need more than machine
experts. You need human experts.
If you are moving at an AI-accelerated pace, add human experts to your team — such as
behavioral scientists, ethicists, neuroscientists or social psychologists.
On the other hand, you’ll need to move at an AI-accelerated pace if your industry is being
reinvented by AI, your AI ambition is to be AI-first and you have more than 10 initiatives.
If you plan to go at an AI-accelerated pace, do all the AI-steady actions and take it a step
further. You should also:
If you deliver these outcomes, you’ll position your organization to succeed in your AI race.
Evidence
1
Gartner analysis of Stanford Institute for Human-Centered Artificial Intelligence
Ecosystem Graphs Database data downloaded on 25 July 2024, for the period of 1
2
Mid-2024 Update Gartner CEO and Senior Business Executive Survey. This survey was
fielded in June through July 2024. In total, 110 actively employed CEOs and other senior
executive business leaders qualified and participated. All respondents were screened for
active employment in organizations greater than $50 million in annual revenue. The
sample mix by role was CEOs (n = 88); CFOs (n = 9); COOs or other C-level executives (n
= 7); and chairs, presidents or board directors (n = 6). The sample mix by location was
North America (n = 42), Europe (n = 37), Asia/Pacific (n = 24), Latin America (n = 3), the
Middle East (n = 1) and South Africa (n = 3). The sample mix by size was $50 million to
less than $250 million (n = 10), $250 million to less than $1 billion (n = 25), $1 billion to
less than $10 billion (n = 46) and $10 billion or more (n = 29).
4
2024 Gartner Impact of GenAI in the Digital Workplace Survey. This survey sought to
understand the value of GenAI assistants embedded in popular digital workplace
productivity applications in the digital workplace, assessing their ability to enhance
employee productivity and efficiency. The survey was conducted online from 16 May
through 12 June 2024. A total of 152 IT leaders participated, with 61 who were members
of Gartner’s Research Circle, a Gartner-managed panel, and 91 who were contacted
through survey links via LinkedIn posts and outreach to clients. Respondents were from
EMEA (n = 94), North America (n = 46), Asia/Pacific (n = 10) and Latin America (n = 2). Of
the 152 respondents, 132 were primarily responsible for Copilot for Microsoft 365. They
were highly involved in the decision-making process or management of Copilot and were
required to be currently piloting or finished with the pilot of Microsoft Copilot in their
organizations. The remaining 20 respondents were primarily responsible for GenAI
assistants apart from Microsoft Copilot, such as Gemini for Google Workspace,
Salesforce Slack AI and Zoom AI Companion.
5
2024 Gartner Digital Worker Survey. This survey sought to understand workers’
technological and workplace experience and sentiments. The research was conducted
online from April through July 2024 among 5,141 respondents, who were from the U.S. (n
= 1,121), Australia (n = 1,086), India (n = 996), the U.K. (n = 973) and China (n = 965).
Participants were screened for full-time employment in organizations with 100 or more
employees and were required to use digital technology for work purposes. Ages ranged
from 18 through 74 years old, with quotas and weighting applied for age, gender, region
and income, so that results were representative of countries’ working populations. We
defined “digital technology” as including any combination of technological devices (such
as laptops, smartphones and tablets), applications and web services that people use for
communication, information or productivity.
7
SAP SE (NYSE:SAP) Q2 2023 Earnings Call Transcript, Yahoo Finance.
8
Gartner IT Leaders Webinar Poll, 20 August 2024.
9
2024 Gartner AI Survey: CIO and Technology Leader View — Wave 1. This survey was
conducted to capture CIOs’ and technology leaders’ sentiments toward AI and to better
understand what organizations are doing as a result of recent changes and
announcements regarding AI. The survey was conducted online in June and July 2024
among CIOs (n = 314) and other technology leaders (n = 394). The total sample was 708
respondents, with representation from North America (n = 329), EMEA (n = 253),
Asia/Pacific (n = 89) and Oceania (n = 37) and across all industry sectors.
10
2023 Gartner HR Technology Employee Experience Survey. This survey was conducted
to understand employees’ ratings of 75 technologies and innovations across seven HR
subfunctions based on the level of adoption in their organizations, the impact on current
performance and the future importance for employee performance. The research was
conducted online from 10 October through 7 November 2023 among 3,477 respondents
from various geographies, industries and functions. The survey was designed and
developed by Gartner’s HR Practice research team.
Disclaimer: The organization (or organizations) profiled in this research is (or are)
provided for illustrative purposes only, and does (or do) not constitute an exhaustive list of
examples in this field nor an endorsement by Gartner of the organization or its offerings.
© 2024 Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of
Gartner, Inc. and its affiliates. This publication may not be reproduced or distributed in any form
without Gartner's prior written permission. It consists of the opinions of Gartner's research
organization, which should not be construed as statements of fact. While the information contained in
this publication has been obtained from sources believed to be reliable, Gartner disclaims all warranties
as to the accuracy, completeness or adequacy of such information. Although Gartner research may
address legal and financial issues, Gartner does not provide legal or investment advice and its research
should not be construed or used as such. Your access and use of this publication are governed by
Gartner's Usage Policy. Gartner prides itself on its reputation for independence and objectivity. Its
research is produced independently by its research organization without input or influence from any
third party. For further information, see "Guiding Principles on Independence and Objectivity." Gartner
research may not be used as input into or for the training or development of generative artificial
intelligence, machine learning, algorithms, software, or related technologies.
Webinar Webinar
Pacing Yourself in the AI Races Industry Perspective: Set
Set your pace in the AI race to deliver AI Your Pace in the AI Race
outcomes safely and at scale. Develop your AI action plan for 2025
with insights specific to your industry.
Tool Tool
IT Score for CIOs Gartner Digital
Benchmark key functional processes and Execution Scorecard™
activities with Gartner IT Score. Comprehensive set of digital strategy
benchmarks to accelerate decision
making and drive execution.
Already a client?
Get access to even more resources in your client portal. Log In
Connect With Us
Get actionable, objective insight that drives smarter decisions and
stronger performance on your mission-critical priorities. Contact us
to become a client:
Become a Client
© 2024 Gartner, Inc. and/or its affiliates. All rights reserved. CM_GTS_ 3318142