Why is nobody asking these Big Five Questions about ‘Agent AI’?
Jamie Smith💡
Working on the next $billion market: Empowerment Tech. AI Agents, Digital Wallets, and customer engagement. Weekly newsletter at customerfutures.com
Hi everyone, thanks for coming back to Customer Futures.
Each week I unpack the ever-increasing and disruptive shifts around digital wallets, Personal AI and digital customer relationships.
If you haven’t yet signed up, why not Subscribe Now.
Two quick notices this week:
Salesforce just released AgentForce. Anthropic just updated Claude with agent skills on your computer. And Apple’s latest iPhone release has the ability to program AI ‘intents’ (more on that soon).
So it’s a good moment to revisit a 2023 blog about AI Agents by Bill Gates. Perhaps one of the most important essays on Personal AI in the last 12 months.
His headline: “AI is about to completely change how you use computers.”
At the time I wrote a response to Gates’ post, specifically to pull out the Big Five.These are the most important, and I think now urgent, questions we need to be asking about ‘Agentic AI’.
Yet so far I don’t see many people asking these questions. Let alone hearing good answers, especially from the AI platforms.
So this week we’re looking at it again, with some market updates.
We dig into:
And we wrap up with the Five Big Questions. I’d love your feedback.
Here is the original blog post by Gates (warning, it’s long). In which he says Agent AI will happen ‘within five years’. However I believe much of it will happen sooner, and many of the recent AI announcements suggest it’s accelerating.
Personal AI is going to be one of the most important aspects of Empowerment Tech. And of the future of being a truly digital customer.
So welcome back to the Customer Futures newsletter.
Grab a comfy seat, your hot drink of choice, and Let’s Go.
Why AI is now powering a new breed of digital agents
Gates opens with this:
“I still love software as much today as I did when Paul Allen and I started Microsoft. But—even though it has improved a lot in the decades since then—in many ways, software is still pretty dumb.
Hold up.
“An incomplete understanding of your personal life”?!
It’s as if he’s only just realised that businesses can only ever see their own teeny, tiny, fractional view of the customer.
And that no company or product has the full view, or could help you day-to-day.
Sing it loudly folks: The only 360-degree view of the customer… is the customer.
Ok, with that grumble out the way, let’s get to the good stuff:
“In the next five years, this will change completely.
This is very much what Apple just announced with ‘Intents’.
And it’s an important framing by Gates: ‘A personal assistant that’s powered by AI’. Because it speaks to what I’ve previously called the ‘The Customer Stack’. Where AI is just one of the layers.
New digital tools on the customer side. Tools with many varied and different components and capabilities. Including, for example
Back to Gates:
“This type of software—something that responds to natural language and can accomplish many different tasks based on its knowledge of the user—is called an agent.”
And there we have it.
One of the most important pieces of the puzzle will be the Digital Agent.
“I’ve been thinking about agents for nearly 30 years and wrote about them in my 1995 book The Road Ahead, but they’ve only recently become practical because of advances in AI.
A personal assistant for everyone
“Some critics have pointed out that software companies have offered this kind of thing before, and users didn’t exactly embrace them.
Side note: For those old enough to remember Clippy, you’ll know this is a terriblecomparison. At least the rotary phone was useful. Clippy was an irritating, interrupting, far-from-helpful assistant that didn’t assist.
But let’s move on.
“An agent will be able to help you with all your activities if you want it to. With permission to follow your online interactions and real-world locations, it will develop a powerful understanding of the people, places, and activities you engage in.”
Ok, now we are into the juice. Note the language:
“If you want…” and “Permission to follow…”
This is because Bill’s magic assistant is ‘other’. It is run by someone, somewhere, else. One assumes it’s Not By Me.
Rather, it follows you. It tracks you. With your permission of course.
These words matter. And underline how Bill sees these digital assistants working.
My leather wallet doesn’t ‘follow me’. It’s just ‘mine’ and part of me. It’s my tool. Just like a digital corkscrew or an online handbag.
Couldn’t my digital agent be the same?
“It will get your personal and work relationships, hobbies, preferences, and schedule. You’ll choose how and when it steps in to help with something or ask you to make a decision.”
He’s now describing a digital tool that can build a fulsome worldview of me and my things.
Excellent.
So it’s now time for my first question.
#1. Who does the digital assistant belong to?
If I have to give the assistant permission, and it follows me around, is it ‘mine’?
Is it Personal, like a car - belonging to you? Or is it Personalised, like a taxi - not belonging to you, but rather something you borrow, use or rent.
There’s a huge difference. I’ve written at length about it here.
‘Personalised’ has a fundamental impact on the business model. And of course privacy (more on that later).
“To see the dramatic change that agents will bring, let’s compare them to the AI tools available today. Most of these are bots. They’re limited to one app and generally only step in when you write a particular word or ask for help.
An excellent distinction. Let’s say this over and over. Bot are single-use robots, brilliant at one thing.
Perhaps we can distinguish personal digital tools (like email and spreadsheets) from bots (like WhatsApp and Calendly).
And bookmark this point: Your assistant - your agent - will remember. They will learn.
Let’s keep going.
“Agents are smarter. They’re proactive—capable of making suggestions before you ask for them. They accomplish tasks across applications.
We got there in the end.
Personal agents will DO things. Yes, based on personalised recommendations. And you will be in control of any final action. But it can ACT on your behalf.
“Imagine that you want to plan a trip. A travel bot will identify hotels that fit your budget. An agent will know what time of year you’ll be traveling and, based on its knowledge about whether you always try a new destination or like to return to the same place repeatedly, it will be able to suggest locations.
Not quite. Bill Gates’ travel agent needs a pretty broad understanding of all options available.
This isn’t a travel agent who books restaurants. It’s more like a concierge.
“The most exciting impact of AI agents is the way they will democratize services that today are too expensive for most people.”
OK, before we dive into the sector-specific examples below, let’s pause on this point.
About giving access to the many, not the (rich) few. About price.
You could imagine that these digital assistants will be quite compelling. Perhaps even become a necessity for online life.
They will commoditise for sure.
So the business model is going to matter A Great Deal.
Which leads to the next question:
#2. How will our personal agents be funded?
This is perhaps more important than who operates them. Because it speaks to incentives. To alignment. And to purpose.
More on that in a moment.
To bring things to life, Gates now goes on to describe how digital agents will impact different sectors. Specifically:
Health care
“Today, AI’s main role in healthcare is to help with administrative tasks. Abridge, Nuance DAX, and Nabla Copilot, for example, can capture audio during an appointment and then write up notes for the doctor to review.
Agents will clearly be transformational in healthcare, especially in lowering the cost of diagnosis for billions of us.
But we must revisit our two questions:
1. Who does the digital assistant really belong to; and
2. How does it make money?
When the data involved is highly sensitive, and in some cases could be related to life-or-death situations, we must be very clear about the incentives to run these agents.
And to understand the risks of unintended consequences.
Education
“For decades, I’ve been excited about all the ways that software would make teachers’ jobs easier and help students learn.
I wanted to highlight this section because it speaks to personalisation on a whole new level.
Agents will not only help us day-to-day, but will adapt in real-time to meet our own learning styles. Do you learn best when you read, listen, or watch? Do you need pictures, or must you try it yourself first?
They’ll soon understand how (and how well) we each remember things.
Agents like this will be transformative. Once we apply this thinking from one area (e.g. education) to another (e.g. health), things get very interesting.
Imagine you have a new health diagnosis and need treatment. Your agent can not only help with the medical pathway - giving you advice and information about each step.
But it can explain things to you in the right way for you personally, and at the right time.
So you have the best chance of recovering. Perhaps it’s sticking to a new routine, changing eating habits, or remembering to take your medicine.
And I like this example from Gates:
“If a tutoring agent knows that a kid likes Minecraft and Taylor Swift, it will use Minecraft to teach them about calculating the volume and area of shapes, and Taylor’s lyrics to teach them about storytelling and rhyme schemes. The experience will be far richer—with graphics and sound, for example—and more personalized than today’s text-based tutors.”
This stuff really isn’t so futuristic.
These personal assistant tools exist today. It’s just we haven’t yet trained or distributed them widely enough for people to notice…
If you’re enjoying this and want to dig deeper into digital wallets, Personal AI and the future of digital customer engagement, why not subscribe:
Subscribed
Productivity
“There’s already a lot of competition in this field. Microsoft is making its Copilot part of Word, Excel, Outlook, and other services.
I do wonder if this misses a lot about how we should think about digital agents.
Because with all new technologies, we first apply them to old problems.
When we invented the motor, we initially put it down the mine to replace the animals. It was a long time before we imagined hoovers, boilers and cars.
And our early web browsers were considered ‘just another channel’ for broadcasting existing content. Like TV and magazines. It was years before we harnessed the web to publish content. And voila, we got social media.
With NEW tech, we first solve OLD problems in NEW ways.
So it’s predictable that we’ll soon attempt to use digital agents to file our expenses and organise our calendars. We’ll try to replace today’s human assistants. And that will be a step forward. It will certainly save time and effort.
But it’s not the real opportunity for digital agents.
Which is our next big question:
#3. What will personal agents do tomorrow, that we can’t already do today?
What will be the ‘hoovers’ and ‘social media’, but for software agents?
For now let’s just recognise that the real value - the real transformation - of digital agents won’t be when we solve today’s problems (order pizza for me, help me save for retirement).
It will be when we do NEW things in NEW ways. Not just OLD things.
But agents will also change how we feel about our interactions:
“If your friend just had surgery, your agent will offer to send flowers and be able to order them for you. If you tell it you’d like to catch up with your old college roommate, it will work with their agent to find a time to get together, and just before you arrive, it will remind you that their oldest child just started college at the local university.”
Thoughtful reminder? Yes please.
Automated and instant subscriptions, where I accidentally sign-up to another service without noticing? Erm, maybe.
Let’s be careful not to automate away the ‘Good Friction’. Where we can see and feel and touch what we are doing. Where we feel in control.
The risk is that we make everything so digital and seamless that we don’t even notice any more.
Another example: when it’s really your agent that remembers your friend’s birthday, will it mean as much?
Good Friction is about the effort. About taking the time. A handwritten letter is different to an email. A phone call during a busy day feels different to a WhatsApp message.
If it’s all automated, does it count?
Yes, we want to remove Bad Friction. But we must take care to keep the Good Friction too.
The effort. The experiences. The memories.
We can design for that. But it’s going to take care and attention. I wrote much more about it here.
Entertainment and shopping
“Already, AI can help you pick out a new TV and recommend movies, books, shows, and podcasts. Likewise, a company I’ve invested in, recently launched Pix, which lets you ask questions (“Which Robert Redford movies would I like and where can I watch them?”) and then makes recommendations based on what you’ve liked in the past.
Marvellous. Just wait till Personal AI turns loyalty upside down and inside out.
A good way to think about all this is to see that there are three ages of AI:
This is what Bill is getting at. Agent AI is the next frontier.
Not just to understand our data (1). And not just to make a recommendation or predict what’s next (2). It’s about digital agents doing the doing (3). Acting on our behalf.
Now read this - bold mine:
“If you want to buy a camera, you’ll have your agent read all the reviews for you, summarize them, make a recommendation, and place an order for it once you’ve made a decision.
This is all about ACTION. About getting things DONE.
Great. But notice what Bill Gates is not saying.
In each of the points I’ve made bold above… assuming there are multiple options to recommend, why will the digital agent choose one over another?
Will it make a suggestion based on a ‘full market’ scan of the options, prices and delivery preferences? Or just on the services to which it’s already connected, perhaps ‘my favourites’?
And we’re back to the commercial model. Might there be some kind of money kickback from a provider (part of how the AI model is trained) when making a recommendation?
Surprise!
How quickly will companies notice that the customer experience has just been hijacked by the personal assistant?
Recommendations are now in the hands of an AI model.
It’s why Zuckerburg has been building Meta’s own chatbot agents for the B2B market. For businesses to be able to auto-interact with customers over WhatsApp. Because the ‘public AI’ models might just be trained to recommend a competitor product.
Once more for those at the back: WHOSE AI MODEL IS IT REALLY?
Let’s keep going:
“You’ll also be able to get news and entertainment that’s been tailored to your interests. CurioAI, which creates a custom podcast on any subject you ask about, is a glimpse of what’s coming.”
Personalised. Customised. Tailored.
This takes us neatly to the next question:
#4. Will my personal agent do things WITH me and FOR me, or TO me?
In whose interests will the AI act?
Now go back and re-read the section with bold words above. See if you can make out how Bill Gates thinks about the commercial opportunity for agents.
A shock wave in the tech industry
Now this is a critically important point:
“In short, agents will be able to help with virtually any activity and any area of life. The ramifications for the software business and for society will be profound.
Over the last 12 months, we’ve seen these platforms develop. This is already possible.
But then he says this:
Agents will affect how we use software as well as how it’s written. They’ll replace search sites because they’ll be better at finding information and summarizing it for you.
Google’s original mission was “To organise the world’s information”. Facebook’s was “To make the world more open and connected”.
What will be the mission for those companies that will dominate the market for personal agents and digital assistants?
How about, “To empower every human”?
I believe AI agents will form the foundation of a new market category: Empowerment Tech.
Here’s how Gates describes this new market for agents:
“They’ll replace word processors, spreadsheets, and other productivity apps. Businesses that are separate today—search advertising, social networking with advertising, shopping, productivity software—will become one business.
Say that again slowly:
“Companies will have an incentive to make agents work on your behalf and not an advertiser’s”
His expectation is that in most cases, agents will be paid for by us. The customers, the citizens, the consumers. Where incentives are aligned around the person.
To optimise for the customers needs, not the business’s.
“If the number of companies that have started working on AI just this year is any indication, there will be an exceptional amount of competition, which will make agents very inexpensive.
The technical challenges
I’m glad Gates goes on to say this:
“Nobody has figured out yet what the data structure for an agent will look like. To create personal agents, we need a new type of database that can capture all the nuances of your interests and relationships and quickly recall the information while maintaining your privacy.
Folks have been working for years on how to build digital agents. And not just the ‘AI part’.
But also the information architecture. What is stored where, how, and why.
And even more precisely, asking: how can data be moved around in a trusted, smart and efficient way between agents?
This takes us back to The Customer Stack.
Not just about building the ‘smarts’ in the digital assistant. But also building new approaches to know who is who, and what is what. Then adding different rules engines, consent tools and Personal UIs and so on.
“Another open question is about how many agents people will interact with. Will your personal agent be separate from your therapist agent and your math tutor? If so, when will you want them to work with each other and when should they stay in their lanes?”
Here’s my take. I believe agents will be adopted the same way as money.
Eh?
Some people choose to be ‘all in’ with one bank or financial institution. Their payments, their mortgage, their savings.
Others want to disengage from the financial system completely, and put their cash under the mattress.
But most folks today choose a simple mix of money accounts and products, from different providers, for different things.
And so will be true for digital wallets and smart agents.
Some people will opt for a ‘cover-all-my-things’ agent. Handling everything in one place, with one provider. Others will want nothing to do with it. And they’ll completely opt out of the smart agent ecosystem altogether.
But most of us will be pragmatic.
We’ll have multiple agents (and digital wallets) for different needs. For different interactions. For different “life departments”. Health, Travel, Learning, Money, Home and so on.
Bill continues:
“There are other challenges too. There isn’t yet a standard protocol that will allow agents to talk to each other.”
Now, there are entire companies already working on precisely these problems. Developing standard protocols for sharing information between agents.
Because once we start to move data - especially personal data - between and across agents, things get sticky.
“The cost needs to come down so agents are affordable for everyone. It needs to be easier to prompt the agent in a way that will give you the right answer.
So we need to ask a whole new set of questions
Worth digging into this post by Dazza Greenwood, about the governance models needed for Agentic AI. And what legal frameworks already exist, that we can use today.
This, amongst other reasons, is why we must continue to point out the great and thorny issues around AI on privacy, security and data portability.
Privacy and data protection
Nearly there.
Finally, we reach the obligatory ‘Yeah, but what about privacy’ section of these kinds of posts.
“As all of this comes together, the issues of online privacy and security will become even more urgent than they already are.
Correct.
“But who owns the data you share with your agent”
Good question.
“…and how do you ensure that it’s being used appropriately?”
Yup.
“No one wants to start getting ads related to something they told their therapist agent.”
Uhuh.
“Can law enforcement use your agent as evidence against you? “
Well now.
“When will your agent refuse to do something that could be harmful to you or someone else?”
Let’s go back to our first question. Who operates these agents?
Is there a back door? Or some central reporting requirements? If there are, will consumers just move to competitive, more private options?
Just like they did with private messaging tools when the NSA was exposed to be snooping?
“Who picks the values that are built into agents?”
And there you have it.
This question only becomes relevant if my digital agent is trained and run by someone else.
Surely my own personal agent is trained on my values?
“There’s also the question of how much information your agent should share.
This is fundamentally about:
Which boils down to digital trust.
And now we can ask the final question:
#5: Which brands will be trusted to offer personal agents?
When these products will be less ‘B2C’ and more ‘Me2B’, digital trust is going to be key. So who will be trustworthy enough to operate digital agents?
Empowerment Tech will be an entirely new category.
A new billion-dollar market. So you can bet that the big companies will rush in. But it’s always going to come back to business models and incentives.
I have already mapped out 11 different business models for how digital agents (and other parts of The Customer Stack) could work. It’s certainly going to be an exciting and frothy market once these questions are answered, and the trusted brands stake out their market position.
The wider point is that it’s coming, and faster than we think. Where my digital agent(s) will be personal, and under my control.
As Gates concludes:
“Agents are coming. In the next few years, they will utterly change how we live our lives, online and off.”
So let’s recap:
There you go.
My Big Five unanswered questions for the biggest shift in the digital economy since the web. Possibly even the internet itself.
Because personal agents just became new critical digital infrastructure. Not just about understanding our data. Not just making recommendations or coming up with new designs.
They are about doing the doing.
And this time, the agent is on my side.
That’s a wrap. Stay tuned for more Customer Futures soon, both here and over at www.customerfutures.com
And if you’re not yet signed up, why not Subscribe Now.
Digital Trust | Emerging Technology | Innovation | Education
12moHad my first meeting where the (AI enabled) solution design was presented as "agentric" a few days ago.... one of the signs of a new things becoming a thing is the development of its own vocabulary. And then two posts down in my LinkedIn feed, I come across this from SalesForce: https://www.linkedin.com/posts/marcbenioff_for-my-first-ever-post-on-linkedin-im-excited-activity-7257157359641255936-sKpZ I love that a first ever post on LinkedIn gets >17K reactions!
Helping businesses elevate their value and competitiveness. Visual Strategist | Experience Design Guide | 25+ Years Author, My NEXT ME & ERY Method | WXO Co-Founder + Council Member
12moI like the questions. I would make on suggestion to question #4 “Will my personal agent do things WITH me and FOR me, or TO me?” By adding ‘WITHOUT me.’
Superb write up Jamie. And loved the reference to Me2B…!
Speaker, management advisor, and author of such books as The Experience Economy, Infinite Possibility, Authenticity, and Mass Customization.
12moI've long harped on the difference between personal and personalized, and the frictionless is not always best, which you nicely explain here, Jamie, along with so many other insights. I love your thinking.
R&D at Avoco Secure, writer, sometimes evolutionary anthropologist
12moMy question is "How can you switch it completely off if you want to" - question based on conversations with people of all ages.