Edge AI is enabling the next generation of smart systems by transferring the intelligence and new functionalities directly on the device, such as an intuitive human-machine interface based on speech and gestures. This is paving the way to a world of seamless interaction, transforming the way we live, work and play. And Edge AI also offers many advantages in terms of data privacy, security, energy-efficiency and latency – just to name some. With our guest, Steven Tateosian, we discuss, how Edge AI can improve our experience as a consumer – meaning the interaction with devices - and what this means technically.

Guest: Steven Tateosian, Senior Vice President of Microcontrollers, Infineon
Moderator: Thomas Reinhardt, Director Corporate Campaigns ＆ Customer Communication, Infineon

Date of publication: 23 February 2024

***

„However, with this next generation of microcontrollers, what we've started to do is, to add specific hardware accelerators into these devices that specifically process the types of models that are used in machine learning in a super-efficient way. So, we have a radar example internally where we look at kind of benchmarking on a Cortex-M4, just kind of a 150-megahertz device versus these new PSoC Edge devices with Neuronet accelerators, and we're talking about a thousand times improvement, right?”

***

Moderator:

Hi everyone. Welcome to a new episode of the #MakeIoTwork podcast. My name is Thomas Reinhardt, I am your host, and I am excited to have the opportunity sharing this podcast with all of you.

Today's podcast episode is once again all about Artificial Intelligence - but today we want to focus on AI at the edge. Edge AI is enabling the next generation of smart systems by transferring the intelligence and new functionalities directly on the device, such as an intuitive human-machine interface based on speech and gestures. This is paving the way to a world of seamless interaction, transforming the way we live, work and play. And Edge AI also offers many advantages in terms of data privacy, security, energy-efficiency and latency – just to name some. But - I don't want to get too far ahead of myself and instead introduce my guest today.

We have invited Steven Tateosian to join us for this episode. Steven is Senior Vice President of Microcontrollers at Infineon. Together with his team, he recently introduced a new microcontroller to the market - specifically for applications at the Edge. Welcome Steven.

Guest:

Thanks for inviting me. I’m really excited to be here.

Moderator:

For our listeners, for whom artificial intelligence primarily means ChatGPT and who may not have heard of Edge AI: What is it all about? Very briefly?

Guest:

So, very briefly, it is about creating an opportunity for applications to enable more local control. So, what this means in terms of things that are popular today like ChatGPT or many of this voice engines, they are operated in the cloud. So, cloud AI uses centralized servers, where information or data are transferred back and forth and processed in the cloud and the results are sent back to the edge device. When we talk about Edge AI the processing is done locally on the device. Generally speaking in very light-weight, low-power microcontrollers. The advantages of this are that this reduces wait and see, so it gives a more real-time control capability. A simple example would be voice control to turn your lights on. The human experience with turning the lights on is: you walk into a dark room, you put the switch and you expect the light to turn on immediately. If you are using your voice and if a cloud is involved it may take two seconds, five seconds, and it's uncomfortable and you don't know should I restate the command or is it just waiting for the information to come back and turn my lights on. But when things are done locally, it removes this transfer of the voice data to the cloud, to process it and send a command back. It's all done locally and frankly nearly instantaneously, just like the human experience already with turning a switch on, the lights come on. So it's an example of being able to remove or significantly reduce the latency and the determinism of these systems. It also then removes the reliance on having a cloud connection with the device, even though many of these edge devices will remain cloud connected for multiple reasons. But for the command and control or the user interpretation, that cloud connectivity becomes a less critical part for the functioning of the device. In the end, of course, the goal here is to enable smarter, faster, safer and most importantly, you know, more efficient systems for end users.

Moderator:

I said it at the beginning: Edge AI can significantly improve our experience as a consumer – meaning the interaction with devices. Also known as Human Machine Interaction. Why is that the case? And can you give us some examples?

Guest:

So maybe I'll give a couple examples. I'll start with another kind of home example, which might be your thermostat or your temperature control of your home. You know, I know in my home there's lots of opinions on what the temperature should be in the house and maybe it's okay that some people have the let's say the authority or the ability to control that and others maybe not so much. And a simple way to do this is for the thermostat or the control unit to recognize, let's say, user authority based on recognition of that individual. And that recognition can be done in multiple ways. It can be done visually with a camera by using things like Face ID, like we're used to, to open or unlock our phones. But it can also be done with unique voice signatures. So, you know, recognizing my voice versus my child's voice to enable temperature control in the home. But beyond that, you know, there's an interaction with these devices, which is explicit, but also there's an implicit one where these devices can become more aware of our environment. And if I go back to this temperature control example in the home, for example, with microphones in the device, perhaps it is intelligent enough to know when people are home based on footsteps or when they leave the home based on a door closing. So, these devices become more, well, much smarter and enable more autonomous control based on environmental awareness of the conditions they are working to control. In an industrial environment, there may be a different use case, for example, on an assembly line, there may be safety concerns. So, in this case, we may use something like a radar sensor to detect if a human or part of a human, for example, a hand is in a position or a place where it shouldn't be from a safety perspective and then can take the appropriate action, which may mean shutting down a piece of equipment or sounding an alarm so that that person moves to a safe place.

Another really favorite example of mine is, you know, we at Infineon acquired a machine learning tool suite company called Imagimob earlier in 2023. And they worked with a customer that does welding for the automotive industry. And one really interesting thing that I learned about welding there is, that the quality of the weld, you know, there are lots of ways to check that. Different tests and measurements to check the quality of a weld. But actually, it's a really well-trained welder's ear has been historically one way to determine if the weld quality is high, meaning no voids and no issues with the weld from a strength perspective. However, it takes a really skilled and well-trained welder to pick that up. And often, you know, this is happening in an otherwise potentially noisy environment. So, our partner here, Imagimob, the company that Infineon acquired, had a customer that was using machine learning essentially to learn these capabilities, to – based on the sound that was made by these automatic welding machines – be able to predict the quality of a weld and actually had quite some success in doing so. So very much a lot of opportunity for us to do these add intelligence locally on these devices across a range of end applications that augment the human experience in the environment.

Moderator:

Great examples, but they sound like a very complex interplay of different microelectronic components. How does that work?

Guest:

Yeah, sure. So, I think I'll separate it into two aspects. One is the hardware aspect, so what's happening on the device locally. And as you mentioned at the beginning of the podcast, we had recently announced a new family of microcontrollers called PSoC™ Edge, specifically targeting these kinds of capabilities at the edge. So, on the MCU then itself, it's about the compute performance. So, there is a model that will get created, and I'm going to come to the software question in a second here, but there is some software running on the device that will, based on sensor data input, make a decision. So first of all, that data has to come into the device through a sensor, and I mentioned a few sensors already. It could be a microphone, it could be radar, it could be an inertial sensor, it could be a gas sensor, lots of different sensors in our environment. It could be a combination of multiple sensors as well, depending on the application. That data gets streamed into the microcontroller, and then that microcontroller processes that data in the context of this model for the quote-unquote known environment, and then makes the decision accordingly. So, there's different ways then on the hardware to take advantage of this. One way is just brute force, and that doesn't take advantage of any specific processing capabilities, but the higher performance the processor, the quicker the math can be done. And this is why I think, you see, that historically on these Edge devices, you've had some pretty heavy microprocessors doing this, because they had a lot of, I'll call it brute force, processing power. And to do that in a timely manner, you would leverage that higher performance, more expensive, more power-consuming microprocessor. However, with this next generation of microcontrollers, what we've started to do, is, add specific hardware accelerators into these devices that specifically process the types of models that are used in machine learning in a super-efficient way. So, we have a radar example internally where we look at kind of benchmarking on a Cortex-M4, just kind of a 150-megahertz device versus these new PSoC Edge devices with Neuronet accelerators, and we're talking about a thousand times improvement, right? So, it's not a comparison of this generation of 150-megahertz MCU. Now we move to a 250-megahertz MCU, and it gets much better. It gets twice as fast. We're talking about a thousand times improvement in terms of performance in these systems by adding these hardware accelerators.

On the software side, this is also important - so, we have this hardware system, which is a combination of one or more sensors plus a microcontroller - and then on the software side, there's a tool chain that our developers use. Again, for example, Imagimob, which starts all from data collection. So, if I know a problem, but I don't have any data to solve that problem yet, you can start by collecting data, and then you have to label that data, and the tool supports labeling that data: “This is a good scenario”, and “this is a bad scenario”, for example. So, the machine can start to understand what it is, the system is looking for. And then based on that dataset, then the tool chain will create a model, and then we put that model to run locally on an MCU. So again, I'll just summarize: It's a system of hardware, software, and that hardware is split, then, also between sensors and microcontrollers.

Moderator:

What advantages do device designers have when they go for the PSoC Edge?

Guest:

That's a great question. So, I did mention on the performance advantages already. So, let me talk about what those advantages actually enable then more from a system perspective. So again, when you add more intelligence at the end, it opens up a new set of use cases and/or enables a set of existing use cases at a lower system cost, a lower system power, and often in a way that is more secure and more private as well. So, by more secure and more private there, I mean the data never needs to actually leave the device. So, we're not talking about streaming video or audio to the cloud where it gets stored somewhere in some third-party server. This actually can be all processed and then discarded locally on the device. So, when we look at this from a system perspective, we look at advantages being higher performance, more real-time control, and then a significant advantage in system costs. And that system cost is about, depending on the system, it can come through lower cloud connectivity cost. It can come from a lower bill of materials because of MCU systems are generally very optimized from a system hardware perspective. And it can also come from things like a smaller battery or longer battery life in these systems if we're talking about portable systems.

Moderator:

We've touched on this briefly before, but perhaps we should be more specific: voice recognition, presence detection, "always-on", ... This always rings alarm bells for me in terms of data protection and privacy. What is your view here?

Guest:

Yeah, so that's a great question. So, if we look at this in the context again as an example in always-on voice. So today, right, if you look at voice systems today, they're either plugged into the wall like a smart speaker. You charge it at least once a day, if not multiple times a day like your phone or maybe you have a smart watch that gets charged every night. Or it has a huge battery like your car. But really an always-on voice system that isn't plugged in and isn't recharged on a daily basis, those are kind of, actually, I don't really believe those exist today. And there's a challenge there because when these systems are on and listening, the way they're designed today, they do consume a fair amount of power. And as a result of that, you know, the system designers need to make a trade-off in terms of the battery size and the system cost. So, with PSoC Edge, what we've enabled is this always-on voice as an example in ultra-low power. So here we have a specific low-power domain on the device, which is a Cortex-M33 paired with a proprietary neural net processor we call NN Lite. So, a very lightweight processor that is specifically listening at all times for, first of all, an audio event. So, is something happening? Do we think somebody is going to say something? And then it can start to ramp up in terms of, okay, they did utter something. Is that the keyword I'm looking for? And if it is, then it gets sent to the higher performance domain, which is a Cortex-M55 with the Helium DSP extensions paired with a neural net coprocessor called the Ethos U55. And then once that ramps up, then it enables a whole set of local language commands. So here we're able to then enable devices that are battery-powered to be always-on for voice. And it could be voice, but there are other use cases as well that a device may always want to be listening to that's portable. But extending battery life from perhaps a day in most of the devices around us today, that to maybe three, four weeks between charges as an example.

Moderator:

So, data privacy is a big plus for Edge AI and system designs created with a PSoC™ Edge. What are the other advantages - at this point for the end user.

Guest:

Yeah, I think the two key things here that I would just highlight again above the data privacy is the better user experience. It's just more intuitive. And if I go back to that first example about the cloud: Did it take my command? Should I sit here and wait another second or two for the lights to come on? Or should I reissue the command? So just a better, more seamless user experience when you're explicitly interacting with the device, but also the device becoming more intelligent and predicting the user behavior. So, I'm not necessarily explicitly interacting, but implicitly interacting with the device. So overall, better user experience. And then I come back to this lower system cost as well, because that absolutely will trickle down to the end user as well as these devices. Hit new price points in the market, they will be more accessible to the broader market. And that's for end consumers as individual users, but also for businesses or industrial use cases as well.

Moderator:

Sounds a bit contradicting, right. But I am sure you will elaborate.

Guest:

Yeah, so on the lower system cost, again, if we compare it to a microprocessor-based system that is higher power consuming. So again, if we just look at a portable application, we don't have as large a battery for the same lifetime. We remove a PMIC, which is a dedicated power management controller IC that's needed in a processor-based system. Processor-based systems may have a fan to cool the device as well. So that's removed because on a very much less power being dissipated through heat. So just the overall system build, you also typically have a lot less memory in an MCU based system because MPUs are typically running a higher end operating system that consumes large amounts of memories that aren't, frankly, of direct value to the end user experience. They're about running the system, not how the user interacts with the system. So there really are, I'll say, hardware that is just removed from the system and that cost goes with it when customers are able to reach these levels of performance with a microcontroller-based system instead of a microprocessor-based system.

Moderator:

Exciting. And that's why many research companies are predicting that edge AI will overtake cloud AI in the future.

Wow, time flies, so let's already move on to the last question - a brief look into the future. Where will the journey take us in terms of microcontrollers? What is the next milestone on the roadmap?

Guest:

So, the next milestone specifically on our roadmap is the extension of this new PSoC Edge family. So, we announced the first set of devices last month and this is just the beginning of our investment the PSoC Edge family. So, there's a lot more products to come with extended sets of capabilities. But in parallel to that and very tightly linked to that, we also continue to invest in the software and tools that enable these devices to really meet and exceed customer expectations. I think that's one thing that is really special about this market today or this application area today is, that the advances in the software and the tools is also happening rapidly along with the advances in the hardware. And that's kind of a win-win for developers and, of course, end users of these products.

Moderator:

Thank you very much, Steven, for your exciting insights.

This brings us to the end of this episode.

Dear listeners: If you want to learn more about us and our world of IoT, visit our website www.infineon.io. If you're currently listening to us on Spotify or Apple Podcast, we'd love for you to subscribe to our podcast and leave a little review. Now it only remains for me to wish you a good time. Take care and see you next time.