Infineon key visual in RGB

Protection ICs in AI data centers

We’re bringing you the latest from the world of semiconductors – straight to your ears! From quick takes on trending applications to deep dives on product innovations, our experts give you their take on the tech behind the tech.

Podcast

Feb 25, 2025

Podcast

Advancements in GPUs and the transition to higher voltage architectures in AI servers are driving the need for robust protection integrated circuits (ICs). In this episode of Podcast4Engineers, our application expert, Nitish Agarwal, explores the critical role of hot swap controllers in managing inrush currents and ensuring the reliability and efficiency of AI data centers. Additionally, he will address future trends and challenges within the semiconductor industry.

Listen on Spotify Watch on Youtube

Speakers

In this episode of Podcast4Engineers, host Peter Balint speaks with Nitish Agarwal, application expert at Infineon.

Peter Balint

Host:

Peter Balint has shaped visual and audio narratives at Infineon since 2021. He’s a video producer with 20 years of experience and has produced podcasts for the past 10 years. Over his career, Peter has interviewed speakers from all over Europe, bringing high-quality media production and engaging conversations to the forefront of his work.

Nitish Agarwal

Guest:

Nitish Agarwal earned his Master’s degree in Electrical Engineering from the University of Southern California in 2019, specializing in electrical power. Following graduation, he joined SL Power Electronics, where he spent four years designing medical-grade AC-DC power supplies and developing embedded firmware solutions to support their performance and reliability. In December 2022, Nitish brought his expertise to Infineon Technologies as a Senior Application Engineer. In this role, he provides product application support for Infineon’s protection IC portfolio, working closely with customers and design teams to deliver innovative, robust solutions for modern electronic systems.

Additional information

More episodes

Guest: AI is booming. The GPUs are also getting more and more advanced. The demand for protection ICs is also going up to keep these servers protected with the low downtime.

 

Host: Hello and welcome to Podcast4Engineers. It's the podcast you just have to listen to if you're interested in what's going on in the semiconductor market. I'm your host, Peter Balint, and today we are joined by Nitish Agarwal. He and his team are defining and developing new products for AI servers in the Power IC group here at Infineon. Welcome and thanks for joining us today.

 

Guest: Thank you, Peter, for having me.

 

Host: Powering AI is not simply about providing power to a rack of servers within an AI data center. There are also other things to be considered, such as swapping out a tray within a rack. This presents a whole new set of challenges, and this is where protection ICs come into play. And this is what you and your team are focused on. Tell us a little bit about these protection ICs in the AI domain.

 

Guest: Okay, so let's break that question down. What is powering the AI? So AI is basically powered through a GPU. GPU is a graphical processing unit, which is doing all these computations for the AI, which has highly complex mathematical algorithms that's running. And these several GPUs work in parallel to solve a particular query. Now these GPUs work on a voltage of 0.8 to 1 V. Now, it's not feasible to supply 0.8 to 1 V, so the traditional architecture was to have 12 V architecture where 12 V was converted to 0.8 and the backplane was 12 V. Now, as we know, there is a boom in AI. The machine learning models are getting more and more power hungry, and even the GPUs, they are getting more and more power dense. The system, the industry moved towards 48 V architecture. Now, as I stated, we have so many of these connected in parallel. If one goes out of service, then we need to replace that GPU without affecting anything else. So, we have these trays, we take the bad GPU out and put the new one in. While we are inserting the new tray into the rack, the first thing it sees is a capacitor. So that capacitor in there, when it connects right to the 48 V, can result in a huge inrush current because the capacitor is completely discharged. And this inrush current can blow up the fuse, it can create a surge in the adjacent GPUs and also take the whole system down. With our hot swap controllers, we try to limit that inrush current by operating a FET in a linear region.

 

Host: Can you give us an overview of how these protection ICs work?

 

Guest: So, as I explained, we have to limit this inrush current because when we are connecting this 48 V into the capacitor directly, it can result in a huge inrush current. Now, we can put a resistor in there to limit that inrush current, but it can lead to a lot of losses. We will lose all the advantage of having a 48 V system. Plus, it's not efficient. What we can use, we can use a thermistor, negative temperature coefficient thermistor, which gets, once it gets hot, reduces its impedance. But it's not reliable as it changes with the temperature. We use a hot swap controller. So hot-swap controller, basically it works with an external MOSFET. Controller controls the gate voltage of the MOSFET and operates the MOSFET in a linear region. At the startup, it provides— the MOSFET acts as an impedance in between the capacitor and the backplane and slowly charges this output capacitor. And once this capacitor is charged, the FET goes in a fully on state. providing very low impedance. After this hot swapping event, the hot swap controller is also responsible for doing the telemetry like voltage, current, power, temperature, fault status warnings. It provides all that through a 1 MHz PM bus communication protocol. Plus, it also helps to isolate the faults. If anything goes wrong inside the rack, it detects that fault and isolates it and prevents it getting propagated into the rack from the tray. At the same time, if it sees anything goes wrong in the rack or in the backplane, like there is a surge, over voltage, or under voltage, it detects that and it prevents it going into the tray. These are the 3 things that a hot-swap controller does.

 

Host: Okay, now let's make things real. Give us an example of an industry or an application where we find these protection ICs.

 

Guest: As an example, let's say you are writing a question on ChatGPT. You write your query on ChatGPT, and it goes to its server where the several GPUs work in parallel and do this complex algorithm and provide you with an answer. But while doing this, it's all electronic circuits. So one, it's possible that one can get damaged. You have several of these processors working in parallel, one can get damaged. The load is transferred to the healthy ones and the damaged one through a PM bus, it'll tell the user, not the user, but the maintenance guy at the server that something is wrong with the CPU. And it will, so the maintenance guy can go in, take the bad tray out from the rack, and put a good one in without turning off the backplane. The end user who's writing the query will not see anything that has happened in the background.

 

Host: Can you give us an idea of what some of the trends are in this world of protection ICs?

 

Guest: Yes. For future trends, if you would like to say, then, you know this hot swap solution, so it's a discrete solution. You have a controller and you have a discrete MOSFET, and then you have a current sensing shunt and temperature sensor. If you, if you go back to our 12 V architecture, that's how it started, and then it switched to something called an eFuse. an integrated solution. We are seeing the same trend in the 48 V backplane where the industry is looking for an integrated solution where they will integrate the controller, the MOSFET, the current sense shunt, and the temperature sensor into one package. It'll improve the power density, plus this will be a stackable solution. Let's say you have one eFuse of 20 or 30 amps, which can do, let's say, 1 kW of power. If you need 2 kW of power, you can just stack 2 in parallel, 3 in parallel. So that's what we are looking at the trend right now. Plus, we have noticed that, as you know, this AI server backplane is growing. So as this is growing, the energy demand is also growing. The 48 V backplane is not able to meet that demand at the moment. Industry is looking to move towards 800 V and 400 V backplanes. So that will require a whole new family of hot swaps and the compatible MOSFETs for this one.

 

Host: And can you tell us some of the challenges that you and your team are facing?

 

Guest: Ah, the challenges. So, when this 48 V architecture was made and when we started working on these hot swaps, for this architecture, it was 1 kW of power roughly. We saw, we thought like 1 kW or maybe 1.5. Now we are seeing the customers are doing 4 kW and 6 kW of power. So that means you have more and more output capacitance. But controller stays the same, it's the MOSFET now. So, what we are seeing is the MOSFET needs to be capable enough to handle this much capacitive current. We have some techniques that we can use to use pulse current mode control and all that to have the FET not stay in the linear region for a long time. It can have time in between to cool down. But this is also beneficial, as now the industry wants a second source of everything because you cannot rely on single source now. But it's very difficult to find a MOSFET which is equally capable of the first one. When you do the design, you have to do it for the worst case, for the worst MOSFET you have in your design. So that's becoming quite challenging for us right now.

 

Host: And if we now take a look towards the future, what kinds of things do we see? Can you make a prediction of where things are going, where things are headed?

 

Guest: Next 5 to 10 years. What I can say immediately now, we will see eFuses, as I explained. Then maybe in 2 years timeframe, we will start seeing these 400, 800 V backplanes with their own family of hot swap and a discrete MOSFET. I don't know if we will go with eFuses on those or not. It's too much energy. But at the same time, this is all a gamble, I would say, because it'll require a lot of training for the individual who's doing the hot swapping of these 400 and 800 V. Earlier it used to be 12 V or 48 V. Now with 400, it's extra risk. It requires a whole new set of testing, added cost on it. Then probably if the GaN technology is improved. We can probably see GaN coming into pictures, maybe inside an eFuse or with an external MOSFET for doing the hot swapping. One customer guide us in one way, others in a different way. This 400-800 V architecture, we have received some guidance and for the existing hot swap, there is some direction that they're directing us to. And the eFuse, it's still new. They have to test how it goes and then probably it'll broaden in that direction and maybe we go away from the discrete, we go into the eFUSE, but it's all we need to see what happens in the next year or so.

 

Host: Well, thank you so much for taking some of your time today to join us. We really appreciate it.

 

Guest: Yeah, thank you.

 

Host: And for our listeners, thank you for being here today. If you have any comments or suggestions about future episodes, please send us an email at wepowerai@infineon.com. Thank you and see you soon.