【0520直播】半導體恩智浦副總裁Computex專題演講 邊緣AI技術演進與自主智慧未來

各位女士 各位先生 讓我們歡迎 NXP 執行副總裁 Jens Henrichsen My name is Jens Henrichsen and I am with NXP Semiconductor Today I'd like to talk to you about the autonomous future So why are we here talking about the autonomous future? This is not some science fiction movie convention This is compu-tech But it is because the autonomous world A world of machines that anticipate and automate Machines that sense, think and collaborate and perform complex tasks all by themselves It's not a line of a sci-fi novel anymore It is on the cusp of becoming reality and it will affect all of us This autonomous future will make us more productive It will make us more sustainable, safer because of AI AI is this single biggest innovation of humans to make this autonomous future a reality AI becomes part of our world Of the real world, of the physical world So actually the world where we interact with others, our surroundings, with things around us That's what we call the edge And to leverage the immense potential of AI It has to be here So today I'd like to share with you How we will make this autonomous future a reality So I will guide you through a couple of steps So I will start why AI will move into the edge And how the AI in the edge will evolve And I will also explore the engineering challenges What it takes to bring it there And what the industry can do to overcome it Let me begin with the starting fact The starting fact about data we generate And that will help us to establish the stage for why the edge matters And why we cannot do it all in the cloud And why we need to empower the edge So the world is projected to hit 180 zettabyte of data this year By the way, 1 zettabyte is 1 followed by 21 zeros It's a trillion gigabytes So we have created more data in the last three years Than in our entire human history And it is not slowing down So we generate data from everywhere From factories, vehicles, homes, wearable cities But there's one problem Most of the data goes unused So let's step back for a moment, right? And let's connect to us as humans And how our brain basically works Is dealing with data On average a human takes in roundabout 74 gigabyte of data every single day So sound, sound, motion And our brains have evolved to become extremely energy-efficient Intelligent data processing centers They understand context And prioritize the processing Actually only what matters Well, when I walked on stage You watched me, I think so, right? And your brain tuned out everything around you And you didn't pay attention to the light or to the exit signs Just until I mentioned them now So this is context-based intelligent data processing So going back to our data problem, right? So context-based intelligent data processing That is what we need to handle all the data we generate The word intelligent data processing Typically brings to mind cloud Which is powerful GPUs and CPUs With this unbelievable compute processing power But most of the data are generated in the edge In the real world So we generate about half a trillion gigabyte of data every day From the millions of edge devices all around us And just as an example A modern car generates up to 4 terabyte of data every day And then again, it's not about reducing the data volume It is about how we process the data And how to extract the true value So sending this all to the cloud for real-time decision-making Is simply not feasible because of latency and cost And we need around about thousand times more bandwidth Than we have to send all the data to the cloud Our physical world is just too vast to be uploaded to the cloud So the solution is we need to bring AI to the edge And make AI useful for our day-to-day life The edge AI will break us free from encapsulated cloud-based use cases We need intelligent data processing next to where the data is created And that is the edge where the AI meets the physical world So AI at the edge has actually quite a few benefits So let's dive a bit into this one To get a good feeling on what we are talking about So let's come back to bandwidth AI at the edge Would reduce the data transport dramatically As the processing is done at the edge And only very critical metadata Will be uploaded or transported back and forth to the cloud Also, the edge AI does not need always-on cloud connectivity Which is really especially important In areas, in remote areas with very limited coverage Another benefit is the real-time capability of the edge So in real life, many processes must happen in real time You can imagine many use cases Where the real-time requirements are essential Just think about the self-driving car in heavy traffic Or the real-time traffic light control at the same time in a smart city Or smart watches detecting a health anomaly With immediate response required Edge avoids the round trips to the cloud Edge hardware is also optimized for energy efficiency A thousand-second floating-point calculation done at the edge Takes the same energy to send a single bit of data to the cloud Very simple Less data sent to the cloud equals less energy used I like to highlight already at this point of the time That this topic is not that simple Because later I will also show you That the energy efficiency in the edge Especially of the processors in the edge Is also a real engineering challenge to overcome Coming to the next point, it's trust And trust is many things Basically, it's safety, it's security And it is data privacy, personal data privacy So we need to protect the data From being abused, falling into wrong hands And the edge AI reduces the cybersecurity risk By processing the data right where it is generated So the data are distributed into many edge devices And not consolidated in one particular place in the cloud Which would make it very attractive to hack into So in security terms, the edge concept In the edge concept, actually the attack surface Is clearly reduced So edge AI has real life benefits And it has tremendous potential to help us solve Real life, real world challenges And let's have a look into this one Let's dive into one item Which is basically the energy management So we have two examples And I like to start first with the smart energy management So if you look into this one The global energy consumption is outpacing us So from now to 25, electricity consumption Is projected to double, possibly So beyond the traditional energy consuming Buildings and industries EVs also require more energy And while we are building smarter systems I'm just talking about But they also consume more and more power So what do we do When 75% of our electric vehicles Or cars are all electrified So what happens when the households The factories, the cars, the cloud All draw from the same grid We just can't throw more power at the problem And the biggest problem actually Is the peak load demand When all require energy at the same time So AI can help contain this growing challenge And we need energy systems That think and work with the grid And I can help to smartly balance the load You can think about Balancing the load across city blocks Maybe beyond the city limits Maybe even on state level So the grids do not have to be built To the peak loads But to the balance Let's move to the second real life challenge Each year we face Millions of unnecessary fatalities 1.1 million people die on the road More than 6 million people die From actually curable diseases And we are facing approximately 2.3 million workplace deaths Every one of this number Is representing a human And it is actually also Representing a system That fails to intervene in time These aren't real unpredictable events They are typically a result Of known patterns So in traffic, in health And in industrial environment So the causes are often very well understood But the system The system that should detect and prevent them Often don't work So it's not a lack of data Again it's a lack of intelligence Of the point of need Actually I can help to reinvent The so-called vision zero And we have the ability to see these risks You see these risks coming And then with edge AI We can help prevent the incidents But these are just two big areas Where AI in the edge in the real world Can make a big difference So what have we seen in these examples Actually in a nutshell AI is becoming part of our everyday life The edge is where it all comes together This is where it's becoming real for us This is the intersection of physical and digital Where the AI meets the real world AI is moving to the edge And actually it is advancing faster than ever before And this brings me to the second part Of my speech The rapid evolution of AI And the acceleration of AI moving into the edge So let's go back a bit in time In 1997 IBM's Deep Blue beat the chess world champion Gary Kasparov In I think it was even the sixth game marathon Using symbolic AI and supercomputing power No machine has ever done this before That was the first occurrence and demonstration Of the potential of AI And I will never forget that But let's leave this example a bit behind And then let's have a look what happened since So the artificial intelligence evolved significantly And the innovation speed accelerated So AI evolved from pure perception AI With object detection Computer vision, speech recognition capabilities Over generative AI by context Such as images, codes, text Is created towards agentic AI That senses things, acts and learns Practically like humans So let's have a closer look On these evolutionary steps at the end So perception AI Creates the aware And the aware edge is able to sense And interpret signals from the real world in real time It detects objects or people And it also recognizes who is who and what is what So accurate detection and recognition was made possible By the immense progress in the neural network Such as the CNNs and the RNNs These neural networks were developed in the cloud But the algorithms were optimized To run on the resource-constrained edge devices Without losing the accuracy Where it really is critical So I took It took actually 10 years To bring the perception AI into the edge To create the aware edge Then later the edge became interactive This interaction came with the breakthrough of the transformer So transformers are the foundation Behind today's generative AI model The generative AI gave the ability to interact with humans In a very natural way Through words and visuals Or even the combination of both You may recall that gen AI Appeared not that long time ago In fact the open AI large language model First became publicly available in 2021 So the explosion of progress In gen AI models since then Has benefited the cloud But an even bigger beneficiary Has been the edge Because these gen AI models Much like the large language models The vision models or the multimodal models They all became smaller and smaller More energy efficient Just suitable for the edge So with gen AI We made the edge interactive An edge that is not only able to sense But also able to think However interaction alone is not enough Because you know for a safe autonomy This is what we want to create You need AI that is able to sense Think and refines its own thinking Before it acts That leap happened with the gen AI And with a gen AI The edge becomes autonomous A gen AI comes the closest To how humans function So a gen AI ties all the AI evolution Of the past together To create an edge that has gone from Reactive to proactive So a gen AI At the edge senses things and acts Independently responding to the real world In real time it evaluates the decision Refines them and acts safely and securely So a gen AI enables the autonomous edge The real value in edge AI Comes with autonomy When the systems become human companions And the AI evolution Typically start in the cloud I mentioned this also earlier And due to its enormous compute capability In the cloud But then the capabilities Are rapidly moving into the edge Aided by innovations in many areas Such as the silicon process technology itself And therefore resulting also In the processing capabilities So of the compute processors Including their energy efficiency The AI models became lightweight And software frameworks for deployment Were also developed Everything making it fit into the edge So let's come back to our chess example So in the mid 90s Deep Blue beat the chess champion Which is more than 25 years ago Since then We have seen significant acceleration So less than 10 years for perception AI To come prevalent in the edge And then round about 5 years For generative AI and agentic AI We see less than 1 year later So this acceleration is not by accident It is because of the realization Of the true value of edge AI In our day to day life So let's look At a practical case For gen AI at the edge So we have a short video And this video is a recording Of a multimodal gen AI model Running on a discrete NPU Which is built by our partner KINARA And also for this demo We use LAVA Which is an open source multimodal gen AI model And then we took this model We ported it Optimize it to run on this particular NPU On an edge device And then we fed a video to it Of an unfortunate accident on a factory floor And asked the AI model to describe what happened So let's have a look what happened So here you see the video To a guy Lip Not serious And then we asked the model to describe What did you see And then you see what's coming out It takes a little bit here And you see, you know, reading through this You're realizing that that system Actually it is a camera The edge device where it runs on Is interpreting the situation pretty well So in this case you can clearly see That it is possible to have the AI in the edge So this is an example of Generative AI in the edge And the most important takeaway of this video is That basically the system Understands and interprets the situation very well And so this is all done by humans these days Sitting in one room But you know this can be done by machines 24 7 later on And then if you would also add a response That would make out of this gen AI Pretty much more energetic And this all runs at the edge But then the question is Is the agentic AI too complex For the edge Is it better to be in the cloud With its enormous compute resources And the answer is it depends Because the edge is more than just An edge device So today and increasingly tomorrow It will be not just the edge or the cloud So we will see more and more implementation Of local network edges The so-called near edge So this is actually a local network edge And that could be a local edge AI hub Or maybe a local interference server A local cloud so to speak That has compute capability To run the more complex agentic AI systems While then the smaller more optimized AI agents Run on the edge So in reality you will always see The cloud and the edge Coexist Augmented by a local network And then also the realization Of AI in the edge or in the cloud Is not a binary choice Trade-offs apply right So use case by use case It is an optimization decision Of multiple and partly conflicting parameters And if you look on the slide There is on the one hand the bandwidth The latency the energy efficiency On the one side And this tremendous compute power On the other side But also the number of systems And the lifespan of the use cases Really matters in this decision making So in a nutshell Edge and cloud will always coexist So what did we learn so far We see 25 years of innovation We see that AI in the edge Is evolving from perception AI Towards agentic AI And we see the coexistence of AI in the edge And in the cloud And all is accelerating to create the autonomy In our day to day life So let's dive a bit more Into the topic of agentic AI We see this as a new paradigm shift A paradigm shift from AI as something To something that works alongside us So when we talk about agentic AI We are talking about more than just a model We are talking about a software workflow construct Built for autonomy just like a brain So as mentioned earlier Agentic AI starts with an AI agent So single or multiple And then each agent is powered by AI models Perception generative or both And then each agent is then built To very specific tasks And one may interpret the video Another one may assess sound Or another one might pave the path Or a safe path for a robot to go But the AI agent The agentic AI agent Is not just a model to do a task They learn and evolve from experience And that is the real breakthrough Of agentic AI workflow Adding structure so that AI agent Become adaptive autonomous learner Let's dive into this a little bit And let's have a look on the picture here So it really starts with an AI model Being able to sense its surrounding And this is outlined here with the word perception And then this is resulting in a projected plan Of what needs to be done But then the agentic AI uses Reinforcement learning to improve itself By exploring and collecting additional data And finally it connects the actions with outcomes This so-called feedback mapping And it is rewarding itself For let's say very successful tasks to be fulfilled Or it is learning out of failures that were made So this continuous learning piece Makes agentic AI revolutionary So this AI agent learn from real life experience They learn on the job That they are no longer limited To the data set they were originally trained on So now we have an AI agent That can do meaningful tasks And it goes further Because when they collaborate They can achieve an overarching goal People need multiple AI agents To come together and collaborate And then of course the collaboration as always Requires some kind of coordination Something to orchestrate this And this is where the AI orchestrator agent comes in It is a specialized agent Built to supervise, plan and coordinate So you can say it's like a project manager For intelligent systems Turning many agents into one unified When agentic AI gets implemented at the edge AI becomes an autonomous human companion It sees, plans, acts in the way we do To fulfill an overarching goal we gave So let's have a look at a real life example How edge agentic AI performs in action Actually in this case on a factory floor And on this factory floor We gave a goal to an agentic AI system Monitor and response to any abnormal situation To protect damage to humans and property Now imagine a water line break Moisture standards and cameras Which are AI agents themselves Detect the leak and inform the orchestrator The coordinator So the orchestrator brings into action And triggers other agents to act So maybe shutting off the water supply To the leaking pipe Closing access to the area and so on And the orchestrator agent in itself Triggers maybe the cleanup And informs everyone So it does the steps Because it plans and acts to fulfill the goal we gave it But it also learned from what went right And what went wrong Using feedback mapping and reinforcement learning So the techniques I introduced early on So here you have a system that thinks Adopts and acts just like human would In a situation like this There's another benefit That agentic AI has Which is of great value Therefore let's look at the agentic AI More from an engineering perspective Actually an agentic AI workflow Is a very scalable archetype So the underlying software archetype Can be scaled to an endless number of app applications And use cases They have minimum scalability And reuse potential So just to trigger your imagination Here are a few real world examples Which can be fundamentally done All on the same underlying software archetype So let's look into automotive So can you imagine a self-healing in-vehicle network That detects faults So maybe twisted or broken cable And repairs it on the fly Or think about agriculture So maybe precision farming Where aerial ground agents inspect the crops Optimize the watering and fertilization And detect maybe an on-the-ground robot To treat specific areas with minimal resources Or smart buildings Now smart buildings Interpret occupancy patterns in real time Adjusting climate control BV charging, lighting Dynamically to comfort energy efficiency and savings So this means we can build Intelligent autonomous systems Faster and deploy them Across nearly all the use cases We can think of Because of the scalability and the reuse We will further accelerate with this The deployment of AI at the edge An intelligent platform of scalability and reuse But building the autonomous edge is not trivial Let's continue to look at it From an engineering perspective Let's explore what it takes To make this a reality And what challenges do we need to overcome So the first challenge The edge AI system needs complex integration They cannot act in isolation So what I mean with this is We need complete hardware and software solutions To cover the sense, the think and the act And most of the silicon solutions Focus on compute only But what we need are complete systems Systems of processing, connectivity And power management, security and safety To tie it all together And this also requires A very tailored software foundation To bring the hardware to life So the edge systems Are actually systems of a system Requiring complex integration It gets me to the second challenge I mentioned this a few times before AI needs to be right sized for the edge What I mean is The models developed for the cloud Are often too large, too power hungry, too complex For the edge application They must be right sized And on top of it The new models, frameworks and tool chains Are emerging constantly So the solutions in the edge Needs to constantly keep up to this one And not stay in isolation Just because they are in the edge And finally Autonomy raises the stake for trust And I told you, trust Safety, security, data privacy So you need highest level of functional safety And once you introduce Autonomous decision making into a system You need to trust those actions Will be executed safely And or at least fail safely If something goes wrong You also need maximum security You need to protect the autonomous systems From cyber security attack Any data vulnerability Or security breach is highly critical So to make the autonomous edge a reality We need solutions to overcome All of these fundamental engineering challenges And that actually gets me to the final section So what it takes To make the autonomous edge a reality From a semiconductor vendor's perspective Let's have a look How such a system would look like And you see it here on the picture And let's go step by step So let's start with the bottom Let's start with the hardware foundation The foundation of any autonomous system is silicon And a system connected to the physical world Needs a compute platform Connectivity, data networking And a power and energy network And to build these edge systems You need to have very scalable hardware Building blocks like Lego blocks To tailor these systems To individual use case needs In each particular edge feature So at NXP we provide all these Hardware building blocks out of one hand So basically we have a very scalable Processing platform Very simple microcontrollers To really high-end powerful MP largely focusing on the auto And industrial platform We have tightly integrated power management Of these processors Explicitly for maximum energy efficiency And then secure and safe Networking and connectivity capabilities Addressing all the diverse needs You need for the different use cases Including various data rates But then, you know, the hardware Is not just enough You need comprehensive software To bring the systems to life So what is needed Is a complete software foundation Including libraries, tools, drivers Middleware tailored to the hardware And bridging to the application software And accordingly for the middleware We provide a broad collection of solutions Customers can pick from For their specific use cases And some of the examples you can see Outlined on the slide Like voice, graphics, vision Low power touch, sensing, motor control And so on So with the very scalable And reusable hardware building blocks And the software foundation We offer all the building blocks To develop the systems of a system Let's discuss now How we can find a way to right size Then the AI to the edge And therefore deploying AI at the edge Is not just a copy paste from the cloud It must be right size For the constraints in the edge These are typically the compute power The memory constraints And they must be energy efficient And I've mentioned this a few times You cannot have water cooled Or dual 10 processors In the edge application So what we provide for this one Is a tool kit which we call EIQ For model development and deployment And you can see it outlined on the picture It basically starts with a model definition And right sizing So you can bring your model You can bring your data And then we can right size and optimize it For the particular edge And it will be typically Which is shown there in orange Complemented by a local database for refinements And then of course Plenty of tools for the input and output data To the physical world So with this tool kit We have really the perfect tool kit To right size and optimize for the edge And as well also to keep the edge Always up to date That brings me to the last building block The safety and the security And you know the trust Is the foundation of autonomous systems Especially for autonomous systems at the edge And therefore highest level of functional safety And security are required And for functional safety We leverage our automotive and industrial expertise So industries The highest level of functional safety are needed And we define the functional safety On system level On hardware On software And even on the application end to end And we have multiple elements To basically bring the functional safety Of the system to life And similar to functional safety Also security needs to be defined on system level And it requires multiple measures end to end There's not that one thing that ensures security It's always the combination of multiple measures Individually for the system and the use case And so security is very deep in NXP's DNA Leveraging especially the know how We have out of our passport, credit card And mobile wallet business And we deploy these highest level of security Also for the edge device And in security you always need to stay ahead So you always need to be one step ahead And therefore NXP is building already now Post quantum cryptography into the products Actually already now So to keep the security also up to date We have also plenty of lifecycle management opportunities And secure over the air And also to live up to our strategic vision We are also building to deliver to what's next And we recently announced the intent to acquire Three companies Kinara, Aviva, Lynx and TT Tech Auto And these companies will also further to help us To enhance our tool set of bringing the AI to the edge So Kinara provides industry leading Generative AI capability Capable discrete MPU That are really ideally suited For implementing agentic AI in the edge And then Aviva, Lynx Provides capabilities for high bandwidth Asynchronous data transfer And then TT Tech Auto is a software company That provides safety critical middleware platform All these acquisitions Which are currently in the status of regulatory approval Will significantly enhance our autonomous edge capability And if you have this all combined You can see that we are very well positioned Because we have all these building blocks And tool kits under one roof To address all the engineering challenges I have outlined early on And this enables us to be a true solution provider To develop and deploy the AI solutions to the edge With our partners and with our customers As solution provider We really want to reduce the burden of complexity Our customers are facing And then based on our deep application expertise We focus on solution We focus the solution offering Explicitly in the areas you see here Like the software defined vehicle The factory automation Energy management AI vision So this drives really truly As a solution provider Innovation, fast time to market And an optimization of the total cost of ownership For the industry However, enabling the autonomous edge Is bigger than just one company can do It requires a collective effort That is why we foster A broad ecosystem of partners Developers and innovators Really a network of OEM system integrators Cloud providers Semiconductor vendors, you name it Together we make the autonomy real Scalable And trusted And not just in labs But in the real world Let's see how one of our partners Brings autonomy to reality Honeywell is one of our key partners In industrial and IoT And they are implementing Leading edge AI technologies In their solutions For their customers But also for their own factory So let's listen what Rush says The CTO of Honeywell, please Honeywell is a control systems company For the past 135 years We have gone through three evolutions One, we have leveraged deterministic AI For the last 100 plus years Because we deliver Mission-critical, safety-critical systems The last 10 years We have extended cloud To connect to our install base And we have provided value-added services The last two to three years With partnership like NXP And other semiconductor providers We are pushing AI at edge To really bring in intelligence at edge To really solve the most complex issues Within the environment Where our customers demand transformation Industrial sector impacts everyone There are three fundamentals That we try to really address One, asset-to-asset performance Within buildings, industrial And aircraft subsystems Any downtime will impact The overall production throughput That's something that we increase The lifetime of those assets Second, people Labor shortages and skills gap And that's where we come heavily To really drive the transformation Of people and productivity The last thing is Anything that impacts the throughput And efficiency of their operations And we actually help in optimizing And improving the effectiveness altogether So for us, automation to autonomy Are three fundamentals One, which is assets Second, people Third one is operational throughput 謝謝Suresh For the great insight We are very happy about the partnership And talking about partnership Especially here in Taiwan We see the ecosystem come together In a very powerful way We see shaping the ecosystem Across the automotive, smart homes Smart buildings and smart industry So it's a story of deep collaboration Strong trusted partners And I see many familiar faces Here together today In this very powerful network In the powerful network of collaboration I see suppliers, co-inventors Customers, competitors So thank you very much For your partnership Let's recap now What we have talked about In the last 45 minutes So we have learned That AI brings impactful intelligence To the end AI is accelerating at the edge Faster than ever before Autonomous edge is what's next And agentic AI shows the way It's human-inspired To think, act and learn But autonomy scales only when we can trust it So it must be built with safety and security It takes an ecosystem to build it And this is basically how We make the autonomous future a reality An environment of anticipation automation Where technology around us Is intelligent companion in our day-to-day life But innovation never stops So what's next? Or maybe after next So agentic AI is today But AI will continue to progress and evolve We don't know what will come in future And many of you here right now May be working on the next evolution So what happens Better to say When that happens We, NXP and our partners Will be there right here To bring it to reality But that's for next one today Thank you very much Thank you Thank you for joining us today 感謝各位今日的蒞臨與參與 The next session Gartner Special Event and Forum will begin at 9am and at 9.30am tomorrow morning respectively Gartner 特別活動及論壇