How effective are AI agents?

computer systems that can autonomously work on menial tasks have been the promise of technologists and science fiction authors for decades while we've dabbled with the likes of robotic process automation and machine learning in the past the arrival of advanced AI software has put these lofty ambitions back on the table according to AI developers new systems known as AI agents will free up workers as never before with the ability to autonomously complete tasks based on pre-approved instructions and by contextually processing new data what does all this mean for business and how far can AI agents really go you're listening to the IT Pro podcast [Music] hi I'm Rory Bathgate jane can't be with us today but I'm delighted to say that I'm joined by Paul O'Sullivan UKI CTO at Salesforce to discuss the potential of AI agents paul welcome to the show thank you Rory thank you for having me so to start with Paul for those who don't know what is an AI agent great question um an AI agent is effectively the ability for agents to now take action so obviously we've had an enormous amount of excitement in the field when we look at the advancements in AI moving from predictive AI to obviously most recently generative AI but even generative AI there is there's been a lack of value realization for for customers for businesses for consumers and what do I mean by that well we think about foundational models they're fantastic it was a huge breakthrough because it was able to predict kind of the next word in a sentence or a string and therefore now be able to kind of craft language appropriately and interact with language so understand it but when we think about the future and agents and we think about agentic AI agentic AI has far broader ability now so we're using AI to not just generate text but actually to reason on what the human is trying to achieve and actually try and understand from that reasoning capability the actions that need to be executed and then fulfilling that promise by actually allowing the agent to take the action so it's an incredible breakthrough at Salesforce we obviously launched Agent Force back on back in October last year one of the very first to be able to bring agents to customers um and it's really exciting when I think about some of the practical use cases now how agents can not just transform businesses and enterprises but how they can actually just supercharge uh people using AI technology in everyday life so really excited but in its basic format agents are the ability now with Aentic AI to not just produce words and text or images but actually to be able to fulfill actions and reason on those actions um for people so an extension of probably our digital workforce of the future so it's really putting AI in the driving seat rather than keeping it as a tool alongside workers absolutely so you know if we think about some of the the breakthroughs we've had and and like let's just look at the broader technology for a minute you know chat GPT back in November 22 um really got everybody excited um the fastest adopted technology in our lifetime with over 100 million users within two months um but again that excitement has kind of fizzled out a little bit because of the value um now obviously as we see what's happening in the field and the progression that's being made um the ability to kind of just reason to understand and then to fulfill those actions is quite incredible and it's not just if we think about kind of the co-pilot era um co-pilots were great there were an assistant that sat alongside us um that kind of enhanced our knowledge and understanding of a topic or or what we were trying to achieve but again limited in its capacity because it couldn't actually do anything with that data or that information um that was still down to the human but now with humans and agents working side by side I can collaborate with an agent in Slack we can be in a channel together talking about a particular topic or what we need to do and the agent can go and fulfill those tasks for me they can update the record in the CRM or they can go and build the campaign for me or do a kind of a customer segmentation so very useful because they're now an extension of my ability and actually frees up a lot of my time to focus on some of those real not just the highv value tasks but to be really creative and think about kind of oh how could I go and engage with a customer slightly differently or how could I do something differently so it's it's hugely it's not going to just drive an immense amount of productivity um it's going to really augment the world that we live in so to compare this with with traditional automation for a second um because there will be some listeners who will think well you know I had maybe um RPA back in the day you know kind of stepbased automation for for say scheduling emails or something like that to to compare that with AI agents is it the case that the one of the main differentiators here is context that an AI agent can can reason through steps um with with changing contexts um and can be sort of set up and set off to go and complete tasks autonomously um without needing to be um manually adjusted if say the conditions change yeah I I think that's a great point because I talked to a lot of customers who say we've invested a lot of money in kind of our RPA capability um and what's going to be really interesting is the fact that we can now have agents that can execute some of those RPA kind of tasks and workflows but but it kind of for me it goes it goes one step even further than that because if you think about RPA capability um very similar to kind of traditional chat bots everything had to be systematically programmed there was a logical kind of approach and workflow and set of steps that needed to be executed in a particular order to allow it to be successful in completing that task if we now think about kind of the RPA capability supercharged with Agentic AI the opportunity is incredible because you may well have you know 10 steps to you know an RPA process or a workflow but imagine having a Gentic AI capability that can sit on top of that that can start to kind of use its reasoning capability to to evolve the business process and to to optimize and drive efficiency so those 10 steps that you traditionally would execute using an RPA tool you know by applying a Gentic AI 10 steps could become nine and then if it still gets to the same outcome there is no reason why the reasoning capability can't kind of try and execute the same output and the same goal with eight steps and then with seven and then with six until you get to an optimized kind of business process because lots of customers have developed business processes over you know decades that served a uh served a purpose at a time in which it needed it but we see now customers businesses evolving and consumer demand and customer demand is changing rapidly so we need to be able to kind of unlock new levels of growth for customers using incredible technology and this is where when we think about RPA capability and aentic AI with agents it it it just unleashes a completely new world now customers have also said to me Rory as well they've said to me "But hang on a minute just spent millions of pounds in building out kind of my RPA processes and driving a lot of automation across my my enterprise you don't necessarily have to waste that or throw it away we can have agents that can now take can trigger those processes um and actually that can get you to greater value quicker for an enterprise when you bring you know some of that technology together um that said agents will over time um as I described kind of in the process optimization kind of piece will over time kind of I would say would leapfrog traditional API and offer something that's completely um new for businesses to kind of execute but I still think there's a place for RPA today in in some kind of opportunity for customers so at this point it might be useful to discuss some of the main um use cases for AI agents right now I know that at first there was quite a lot of um talk around their use in knowledge work but um equally now there's a lot of talk around their use for for programming and and other complex tasks so what are some of the main use cases that you see for AI agents right now yeah it's a great question i think the possibilities are endless but what are we seeing now so very early on we saw a lot of focus in the customer service domain um how can we optimize customer service how can we augment some employees in that customer service world but actually our customers are getting really creative now if I if I take um Heathrow airport you know they use Agent Force to bring together kind of lots of real-time data from from Heathrow's knowledge base beacons and flight information to respond to kind of thousands of traveler kind of questions instantly and simultaneously and actually what's really interesting uh for Heathrow and this is a combination of different technologies playing its part 40% of travelers who go through Heathrow don't speak English as their first language so Agent Force is using voice and text translation capabilities to provide real-time language support breaking down the barriers and ensuring all passengers feel informed and at ease so if you think about that from an experience perspective I'm going through the airport i'm trying to find kind of what my my gate information is where my baggage is um and I'm also kind of um worried that I need to go grab some lunch before I jump on the plane or something like that but but now rather than kind of trailing through lots of FAQs by aggregating that data and using an agent to surface that up and allowing customers to put a question or kind of an intent through to an agent you can get direct responses to your question and what you need and what is incredible with Heathrow is they've actually now seen because of the reasoning engine and and using reinforcement learning techniques and uh the more you use it the smarter it gets we're able to achieve a 95% accuracy rate with agent force uh which is remarkable but you know another customer example is somebody like Capita capital have started to use um agent force to screen CVS in the recruitment process which again if you think about they are a huge volume business with 43,000 colleagues uh simultaneously serving 35 million end users daily that's a lot of work that needs to be done and it's a great opportunity for you know aentic AI and agents to start kind of screening CVS looking for patterns understanding candidate profiles um and bringing and looking at the skills that individuals will bring so that they can actually quickly deploy them into the field and put people to work which is a great a great use case and there's tons more when we think about proactive uh fraud detection in financial services for an example you know how can you alert people and and identify things like we've done previously with kind of pattern matching with predictive AI but now actually to be able to summarize where we think the forward might take place with generative and then agentic AI and proactiveness is going to be really important for businesses moving forward as well so there's lots of huge uh use cases and I think every day I'm seeing more and more uh from customers and you know colleagues within Salesforce as well you've named a number of different use cases there is there anything that at present AI agents um maybe can't yet do or is there a specific task that you think there's a conversation around will they ever be able to to fully do this task i think I think it's an interesting question because I think we are at Salesforce we're looking at kind of what we call the multiple waves of AI so obviously wave one was predictive AI which we we've all had for a very long time um wave two was the generative wave and now we're in wave three which is the autonomous uh and kind of agentic AI wave we predict that wave four is probably the robotic wave where you have a combination of robotic uh technologies and uh artificial intelligence now I think that is a that that is another leap forward and and and another kind of chasm that we'll kind of take on together but what's really interesting I think to answer your question is are there things that aren't possible yet i I think where where we still need kind of a human connection or kind of a mechanical kind of execution of an activity I think those things are still being worked through i think we we still see huge breakthroughs in robotics and and intelligence in that space but we're not quite there yet so I think those would be some of the things that are still to be explored i mean we're constantly evolving and we're moving at a pace of change that I I've never seen before in my career um and I do wonder if this is the slowest we will ever move because of the because of how everything is evolving at such a pace um but it's um I haven't necessarily come across anything yet that is not feasible or or possible with the technology that we see we are for an example some of my team I've got an incredible bunch of creative engineers um and we are we are very kind of empowered and driven um to use AI technology in conjunction with what we do one of my engineers has actually built uh an avatar of me in a virtual world uh that is backed up by Salesforce's agent force fully trained on all of our kind of agent force material so that you can you can ask it a question you can query technical things you can ask kind of how does the trust layer really work you can dive into you know complex technical understanding you can actually even if we if we fed at the future road map you could probably ask it questions about our future roadmap as well but again this is where I think the whole user experience is going to evolve around uh AI and Aentic AI and this is where we see Slack comes into play um really well because if we think about a workforce um communicating and interacting with each other um in Slack if we we we will deploy agents into Slack and we have them already here at Salesforce that can augment and support and fulfill tasks for people um I've also been doing a lot of research because you know I long time ago you know used to be an engineer have lots of friends that are still hands- on keyboard and um I've been doing lots of research around kind of what Gartner's been saying gartner has been saying that generative AI uh one of the biggest software engineers are one of the biggest that will benefit from generative AI uh to your point Rory around kind of writing code and we're seeing what kind of Antropics are doing with um the ability to you know with Claude to write code um it's fascinating but we've been looking at agents that could mimic scrum masters for an example because scrum masters are um their role is to support engineers allowing them to focus on building the solutions or building the products so how could a scrum master uh or an agent equivalent scrum master help with things like daily standups updating tickets on a board sprint planning all of the all of your ceremonies and what's really interesting is not just the fact that you could you could build your digital scrum master agent but you've then got an infinite scalability and a scrum master that works 24/7 but what you've also got is you've got a very consistent scrum master that has learned a methodology and will execute that methodology by the book now you know I spoke to my my brother-in-law he's you know still a hands-on keyboard engineer and we were having this conversation on would it would it be a good thing or a bad thing and would the engineering community kind of embrace a digital scrum master and the you know after much debate and thought the answer was yes because actually there are some great scrum masters out there and then there are some that are not necessarily great but you'd still probably need a a kind of a human scrum master that sits above kind of multiple digital agent scrum masters that could do the orchestration um but you could start getting these agent scrum masters to go and um do all of the reporting understand your burndown you you know your velocity your throughput so there's huge opportunity I think for all of us to be really creative in our thinking and going what could be the next agent that could revolutionize our day-to-day activities you've raised an interesting point there around where an agent fits into a um hierarchy when you have human workers and you have agents working alongside one another um a traditional chatbot if you like would follow a workflow that has it answer maybe low-level questions and then eventually hand a user over to a a human worker if the user's problem isn't solved in say a set amount of time do agents raise the bar for for how complex a problem they can solve before they hand over to a human worker um or do they circumvent the need to hand over to a human worker um entirely great question Rory i think when we all looked a couple of years back at chat bots and you know digital channels of engagement with customers to to facilitate queries it was all about deflection they everybody wanted to deflect from the contact center because you know our human workers were tremendously overwhelmed i think our customers and what I'm seeing now from our customers is this presents the opportunity not to deflect but to drive resolution and I think I'm seeing lots of customers now try and set the goals of first time resolution with agents on digital channels you know and that digital channel might be you know web voice even you know WhatsApp as a channel of engagement because I think there is a realiz ization that agents can as the maturity of the agent progresses you know if you think about in the Salesforce platform how we've built agent force is fundamentally around three kind of core components um one is the topic now the topic is about the job to be done you know this may be um you know go and provide up provide information to a customer about shipping information it might be topics like change of booking uh or you know the reservation that I've made at a restaurant where I want to kind of update my party size and increase from four people to six people now if you define your topics and you build your topics out over a period of time you're effectively giving your agents more jobs to be done and again this can all be done with natural language so we're removing complexities of of having to write code the second component is about what we call the instructions so when you define the topics you then need to set the instructions and these instructions talk about things like that they act as the guard rails you know for an example don't don't necessarily talk about a competitor or always check and verify somebody's email address um or ask them to authenticate themselves so again the instructions act as guardrails and kind of keep that safety net and trust established and then the third component is for agent force is is effectively the actions so what is the action that you need to fulfill as part of that job to be done now those actions could be you know a prompt so it could be a well structured prompt to to go and use you know uh rag techniques retrieval augmentation grounding to take data objects out of different data sources to enrich the response back to the customer or again go and fetch information within multiple systems doesn't necessarily have to be the CRM because we can retrieve information from all of the middle and back office systems as well but the first could be your prompt with with rag techniques The second kind of action could be you know an Apex class so execute a piece of code it could be an API call off to another system it could be update a record it could be anything like that and then the third component is obviously flow which is our our workflow management capability where effectively you could trigger a flow that could um have multiple steps in the process uh but again the agent can do that on your behalf so for an example you may have a customer coming in with a customer inquiry they might want to update a booking you can then obviously go and update all of the necessary systems through that executed flow and some of those systems might sit downstream from kind of the Salesforce platform so as the maturity of the agents grow well to come back to your kind of original question what we will see is a slowdown in the handoff to humans because we will be able to drive more resolution across a wider set of topics for our customers in time which is really really fascinating because if you can imagine the pressure that you can take off of these contact centers will be huge in fact a a great example is a customer of ours OpenT so Open Table every time you kind of now do a reservation or you have a change to the reservation for booking a restaurant 73% of changes to a reservations are now handled uh with Agent Force which is incredible because actually you've just you've allowed the the workforce to to not uh feel the pressure of all of the different changes that you need to handle and and the different systems that you update because you've been able to orch orchestrate that across one deeply unified platform using agent force so it is quite remarkable i think all customers will be on a journey though because I think you're still going to have agents initially in their very kind of um initial use cases they won't have access to all of the different actions or jobs to be done and topics so over a period of time customers will be able to look at their consumer behaviors what their customers need and then they'll be able to evolve their topics and actions over a period of time to just drive more resolution as opposed to deflection so it's it's really interesting but I think I think businesses need to start building out their road maps not just their use cases is they need a catalog of use cases but then they need to prioritize based on their business needs and to start building out um a road map for agents across their [Music] organization as part of the road map that you've mentioned for for implementing agents when it comes to implementing agents alongside your workforce and getting your workforce used to using AI agents um what are some of the key skills that workers need to use agents effectively and how can you go about giving them these skills skills is a crucial thing that all businesses need to look at not just businesses i would also encourage you know our education system to think about you know the skills of the future and if I think about what we have invested in from a UKI perspective across Salesforce with our AI center uh it's a commitment we made we we opened our AI center back in June last year with a focus and a purpose around helping bring together kind of our customers industry experts you know and different businesses to to look at how do we close the digital skills gap how do we drive empowerment of AI across workforces etc now if I think about the skills required there is a need for for data literacy and an understanding of of kind of how you will manage your data in a compliant way not just a compliant way but understand what data that you need to put into the AI what data you shouldn't put into the the AI and again the safety and guard rails associated with that so data is an absolute crucial skill to understand for the future i think prompt engineering and the ability to shape and define prompts in a descriptive way that gets you to the output you're looking for quicker is equally crucial i talk to lots of customers and whenever we kind of do a session we talk about how many times does it take you to kind of put your prompt in before you get the response and and there is typically kind of a four or five kind of iteration before you get what you're looking for back from the the foundational model or the LLM but like you can't do that across an enterprise because if you release AI technology or generative AI technology across 10,000 people in your workforce and it takes them on average four times or or four attempts to get to the output that's not cost effective for your business so I think the skills to be able to shape prompts build prompts and then actually deploy prompts across your enterprise in templates will be absolutely crucial i think we're going to see kind of a a need for more security skills in the future um understanding the threats uh associated with you know not just AI but people external to your organization using AI for fishing attacks and other different things so I think those are kind of some of the key skills that you probably need to be thinking about within your organization if you're not already clearly there is a need for data scientists however what I would say is you know we've taken very much an approach at Salesforce around you know leveraging our low code and procode capability so we're trying to remove the complexity of needing data scientists because there isn't enough let's be honest about it there isn't enough in the field so can we still use and apply AI techniques and AI solutions without needing kind of lots of training models and creating bespoke kind of foundational models also the other thing about this is some of these models are are going to be commodities in the future we're going to see price points are going to drop therefore you need to be flexible and you need the ability to swap in and out and use what's relevant for your business which again is is a primary focus for us at Salesforce that that level of openness and flexibility in our architecture to allow customers to to do that and be adaptable over time but I think I think really for me core skills data security prompt building I think also just some foundational engineering practices around kind of API integration cuz there's a lot of ability now to bring different systems and different technologies together to craft something that is differentiated value for a business as opposed to you know I'm I have to build my own model the reality of your own model is in lots of cases it doesn't make sense um because there is a model out there that's probably more efficient and more accurate that can get you to the outcome and you need to just understand how can I leverage that for my business needs the other point I would raise is um a a clear focus on sustainability um the these models are energy hungry um so we got to be really careful here on the balances and trade-offs um as we all have over the the multiple decades of building you know IT systems um there's always a trade-off in every design we put forward pros and cons so again I think the same thing applies when we think about the AI technology that's out there we're going to need all of those skills but then we're going to have to work in an ecosystem that allows us the the flexibility the openness the creativity and the ability to bring data from multiple different sources and connect it all together so there is lots to consider and one of the things I talk to customers about is connecting your data strategy with your AI strategy is and then underpinning that with future skills because I think we've all got data strategies but the new world of AI demands us to to link the two together i think so mark Beni off CEO at Salesforce has suggested that today's leaders will be the last to have all human workforces um driven by the kind of advancements that we've been talking about um throughout the episode how close do you think we are to this reality and what do you see on the horizon for AI agents i think we're very close i I I'm seeing it in particular customers already i think it's a matter of time and I think every enterprise is going to have a agent working in their enterprise within the next 12 to 18 months if if they don't there is a huge risk to their business because the the productivity gains the efficiencies are huge and and you you can't overlook that so I think it's now I think it's happening today already i think it's um tremendously exciting i work every day with agents i have agents deployed uh within Salesforce that support me in in everything me and my team do um so it's it's happening and I look at some of our customer success stories i think the customers that are are exploring this now with a level of curiosity around their wider business and how they operate are the customers that are going to win in the future there are some that are still debating there are some that are really far ahead um and and are actively deploying i I think we're on a we're on a a huge transformation journey and I think it cannot be ignored i think if it's ignored at this point I think there's a real threat to enterprises that may not exist in 5 10 years so my recommendation is if you haven't yet explored a Gentic AI heard about it or learned about it be curious go and ask the questions because if you don't I think we're all we're going to get left behind um but it is happening now Rory it's great to see so it's super exciting well unfortunately that's all we have time for today but Paul thanks so much for joining us thank you Rory as always you can find links to all of the topics we've spoken about today in the show notes and even more on our website at itpro.com you can also follow us on LinkedIn as well as subscribe to our daily newsletter don't forget to subscribe to the ITPro podcast wherever you find podcasts and if you're enjoying the show let us know by leaving a rating or review we'll be back next week with more from the world of IT but until then goodbye [Music] [Music]