Infineon key visual in RGB

Thermal challenges in Powering AI

We’re bringing you the latest from the world of semiconductors – straight to your ears! From quick takes on trending applications to deep dives on product innovations, our experts give you their take on the tech behind the tech.

Podcast

Mar 11, 2025

Podcast

In this episode of Podcast4Engineers, host Peter Balint sits down with Davide Chiola, VP of Systems Solutions at Infineon, to explore the critical role of thermal management in AI data centers. They discuss innovative cooling techniques, design challenges, and emerging trends like liquid cooling and superconductivity, shaping the future of high-performance computing.

Listen on Spotify Watch on Youtube

Speakers

In this episode of Podcast4Engineers, host Peter Balint speaks with Davide Chiola, VP of Systems Solutions at Infineon.

Peter Balint

Host:

Peter Balint has shaped visual and audio narratives at Infineon since 2021. He’s a video producer with 20 years of experience and has produced podcasts for the past 10 years. Over his career, Peter has interviewed speakers from all over Europe, bringing high-quality media production and engaging conversations to the forefront of his work.

Davide Chiola

Guest:

Davide Chiola is Vice President of Systems Solutions at Infineon Technologies. He specializes in thermal management for data centers, leveraging his semiconductor technology expertise. Holding several patents, Davide drives innovation in data center efficiency and sustainability. With a strong background in semiconductors, he leads the development of cutting-edge solutions. His expertise enhances the performance and environmental footprint of modern data centers. As a leader in his field, Davide Chiola is shaping the future of data center technology.

Additional information

More episodes

Guest: Temperature is the biggest enemy of power semiconductors and handling thermal management is the biggest design challenge for high-performance systems.

 

Host: Hello and welcome to Podcast4Engineers. This is the podcast you just have to listen to if you're interested in what's going on in the semiconductor market. I'm your host, Peter Balint, and today we continue our journey into the world of powering AI. And with us today is Davide Chiola. He's a VP of System Solutions here at Infineon. And welcome. Thanks for coming in today.

 

Guest: Thank you, Peter. Thank you for having me here today.

 

Host: We've talked a lot about the power required for these AI data centers. And there's a lot of things that go hand in hand. And one of the things that go hand in hand with power is the thermal effect. So, and managing this thermal energy. To kick things off, can you give us an idea of how much really in the, if you look at the total cost of ownership in the data center, how much of that is devoted strictly to thermal and cooling?

 

Guest: Sure, sure. So, Peter, let me start before I address directly your question. Let me start with a small information about the city. A city called Luleå, I hope I pronounce the name correctly, is located in Sweden, in the north of Sweden, actually close to the North Pole. And Luleå became famous in 2010, around 2011, because it was the first data center installation of Facebook outside the US. Now, what is special about Luleå is that the average temperature during the year is 1 degree Celsius. Temperature can go down, of course, in winter to -20, -30, and in summer never less than 10, 12 degrees. So now you understand how important thermal management and cooling and cool weather for data center installations is. Now, going specifically to your question about total cost of ownership, so TCO, typically it consists of 2 elements. There are fixed costs like CAPEX, capital invest. How much fixed investment we have to run, as well as OPEX, so operational expenses. How much do we have to spend every year to maintain and operate the system? For a typical AI data center, the part related to cooling, meaning, let's say, air conditioning infrastructure, for example, for the CAPEX, or paying the bill, the electricity bill related to cooling for the OPEX, can be significant. And to give you an idea, this can be in the range of 30%. On a large installation of, let's say, 2,000 racks for AI, 1,000 to 2,000 racks, which is a relatively large installation, which costs somewhere around $100 million, can be, of CAPEX, and an OPEX in 10 years of around $50 million. Then we can think about $50 million, $45 to $50 million So 30% devoted to cooling aspects. So really relevant. In fact, if we map a little bit of the installations across Europe or US, we see a lot of them in relatively cold climates. So, Ireland, for example, Netherlands, Sweden, as I mentioned, in US and Canada also. So definitely cooling, cool climate plays a big role in data center installations.

 

Host: Okay, so you could leave the windows open, basically.

 

Guest: Pretty much. Actually, in this Facebook data center, there is hardly any air conditioning cooling, but it's really big fans that are flowing cold air from the outside into the facility. So that's the idea.

 

Host: Okay, and then let's zoom in a little bit and look at something like a power supply in an AI data center, which is something you're focused on. And let's be even more specific and say the 8 kW power supply unit from Infineon. When you look at these designers' challenges, what are the biggest things that they have to overcome in the design of these power supply units?

 

Guest: Yes, Peter. So again, before I talk about that, there is another important aspect in the TCO. I want to step back one second on TCO, which is the investment for real estate. So again, real estate can be a big deal. The cost depends on the location. Of course, in highly populated urban areas, this is more expensive. In remote locations, it is less expensive, but this can also end up to something like 15% to 20% of the total installation cost and maintenance cost. That means that the 2 aspects are somehow related. So cool climate, but also having a limited real estate, meaning density, is an aspect that has to be taken into account when thinking about data center construction and installation and operations. Now, if we think about our systems, particularly a power supply unit that is one of the components of the power conversion in the overall conversion chain, main aspects that we should take into account, I will consider again density. Density is very important for what I mentioned. Managing temperature as well, as we just mentioned. And efficiency. These 3 elements are somehow related with each other in a sort of trade-off, meaning that typically high density means higher temperature. If we compress the system, of course, the heat has less space, less surface to dissipate. Temperature tends to go up. If we, at the same time, increase efficiency, then we can compensate for this effect. So typically, a more efficient system can have higher density and also lower temperature of the components. What I have here today are 2 system solutions for server data center. The specifications are very similar, so both are 3 kW ratings, 3 kW power. They have an efficiency of around 97.5%. However, as you can see from the sides, the density of the 2 solutions is totally different. The one on top is, let's say, a traditional design approach having around 30 W per cubic inch power density. And the one on the bottom here is using all these design measures that I described for a quite advanced power density of around 100 W per cubic inch. The construction is quite different, and I would like to highlight just, for example, the size of the electrolytic capacitor that are playing a big role, as you see, in volume and space. And also, the distribution of the component that here was optimized by simulation method and by advanced integration that enabled this high-density approach.

 

Host: And if you don't see video now, you should know that there's quite a bit of a size difference going on here, right?

 

Guest: Exactly. They're significantly different. And I can tell you, Peter, that now in the size of this older 3 kW design, we can squeeze actually an 8 kW output power.

 

Host: Amazing.

 

Guest: So now on the, from the design side, if we have to design a typical system, how do we take care of these elements? There are many aspects to consider. For example, for the system that you mentioned, this latest 8 kW power supply unit that we developed for AI data center and simulation. The theoretical approach first is the starting point. For thermal management, airflow simulation has to be carried out. Consider that these systems are normally included in a rack and they are stacked on top of each other or near each other. There is no way to dissipate heat vertically or laterally, but rather only longitudinally. There is a fan actually on one side that flows air. The way the air is distributed, so the air distribution and how many obstacles are found during this airflow is very important. Simulation of the airflow, placement of the component, making sure that there are no hotspots or no barrier to the airflow, even considering power dissipation. A system like an 8 kW system, runs at 97.5% efficiency. This means that around 200 W are dissipated in power. A fan like this one can consume up to 20-30 W. This can be up to 10% of the total losses. If we blow air too fast, we cool down the system, but we consume a lot of energy. This is counterproductive for efficiency. This has to be carefully considered.

Another aspect that we should take into account in improving this trade-off that I mentioned is the reduction of the passive components. If you look at the traditional system like the one we have on the table, the electrolytic capacitors that you see here are taking up a big space. If you look at the more modern system like the one I have in front with a much higher density, you can see that the size of the electrolytic capacitor is strongly reduced. This can be done by basically special circuits, special topologies. Particularly in this new system, we have a power conversion element here which is called hold-up time extension circuit that basically by using silicon, silicon components, highly efficient, we compensate somehow the size of the electrolytic cap. By having this extra small circuit, we can reduce the size of the electrolytic capacitor, for example.

Another very important aspect is integration. As mentioned, density comes with high integration. And here I want to bring also one example, which is the magnetic integration in one of the conversion steps, which is the LLC resonant converter. The system is made by a passive component, which is basically a transformer, a magnetic inductor, and a series resonant inductor. Typically, in a conventional system, there are components placed separately on the board. In an advanced high-density system, we have an integrated design where the primary and the secondary part of the transformer are integrated in the PCB and the magnetizing inductor is placed on top. This is a way to integrate basically magnetics. And in the latest integration, we also integrate the so-called synchronous rectifier, which is one additional component that we need to complete the secondary side of the system. So high integration contributes, of course, to high density. And the special design that you see here is like having a radiator effect. These PCBs are exposed to airflow and increase the area exposed to airflow to reduce, to improve the cooling capability.

Finally, very important also, we have topology. Topologies are important to increase density. Advanced topology can help to reduce the stress on the power components and also to reduce temperature. Basically, better efficiency means better thermal management and lower temperature. Summarizing again, optimized design of the airflow, reducing the size of the electrolytic capacitors, integration, especially integration of magnetics and special topologies and semiconductor quality, of course. This standard solution uses mainly silicon and silicon carbide components. This advanced solution is basically completely based on wide bandgap semiconductors. So GaN, silicon carbide, switches.

 

Host: Okay. And in addition to all of the design challenges, I'm quite certain that there are standards that are given that if you want to sell something in the marketplace, you have to be at this point temperature-wise and efficiency-wise.

 

Guest: Correct. Correct. There are standards. So let me say that in general, temperature is an enemy of power semiconductors independently of the system. Temperature is the main acceleration factor for most of the degradation effect in power semiconductor. If you think about gate oxide, if you think about interconnects, so packaging, they're all very sensitive to temperature and especially temperature variation. We want to keep these 2 elements down, absolute temperature and also temperature swing as small as possible. Now, there are standards. One standard driving the server for data center is the OCP, so Open Compute Project. One variation of the standard is talking about operating temperature for the power components should stay below 10 degrees below the maximum temperature, maximum junction temperature. And another standard, which is quite known, refers to the temperature difference of the airflow. The airflow between the input and the output should not exceed 20 degrees. In order to— this means that the whole system has to dissipate low enough not to increase the temperature beyond 20 degrees, basically.

 

Host: Okay. And when it comes to trends, you're out there, you're in the field. What kinds of trends can you identify that are happening in this area of thermal management?

 

Guest: Yes, we can definitely observe trends regarding cooling and thermal management. One very clear trend already visible is liquid cooling. So as mentioned, these systems are now airflow cooled. You see an airflow, a fan and an airflow. However, if you go to rack power on the 100-200 kW per unit up to 1 megawatt, which is what the major OEMs are having as a goal, airflow becomes not effective any longer, not so effective. So liquid cooling is already state of the art for some parts of the system. For example, the so-called IT trays are normally running on liquid cooling. And there are ideas about having also this component or a test, let's say attempt, to also have other parts of the system cooled down by liquid cooling. So liquid cooling can greatly improve the so-called power usage effectiveness index, which is an index of how effectively the input power is used to computational power. An index of 1 would be that the entire power is used to do calculation and computation, which is what these data centers are about. By the elimination of fans and chillers, which are mechanical elements of a typical airflow infrastructure, this index can be brought below 1.1, which is very good. And that means there are some tests done, for example, for some of our ODMs, a temperature reduction of up to 40% for the power component typically. It means components that run typically at 100°C, they can run at 60°C with a good liquid cooling system. Liquid cooling, there are different technologies, of course. There is this so-called cold plate, which is the one more, again, known at the time, today, already in use today, which basically means the converter is sitting on a surface which is cooled by liquid. And the heat is extracted by this cold surface, up to even more efficient system from a thermal point of view, which is immersion cooling. This is very interesting. Basically, the entire converter is immersed in a bath, in a liquid, which is a sort of oil with high dielectrics. And this allows heat extraction, even more efficient heat extraction.

However, of course, there are also reliability risks. This is a technique still, let's say, in development at the moment. At Infineon, by the way, we are looking quite actively at that. So, these, I would say, are more the short-term innovation in thermal management and cooling that we see. If I need to look, I can look a little bit beyond that. There are also, of course, research areas where big benefit can be found also for this kind of application. One is in superconductors. Everybody knows that current, when current flows in a conductor, dissipates heat. There is a law, it's called Joule law, Joule effect. And this heat dissipated is proportional to the resistance of the conductor. And by the way, to the square of the current. Systems with high current, they tend to dissipate more heat, of course. Now, that means that if we could lower the resistance of the conductor, then the dissipation would be much lower, so the temperature increase would be much lower. And there are interesting materials that if they are cooled at very low temperature, however, close to the liquid helium, so very few degrees Kelvin, they show basically zero resistance. They would have zero dissipation. So current flowing through them will have zero dissipation. Unfortunately, of course, this is very difficult to achieve because it's very expensive also.

But there are already commercial systems like the levitation, magnetic levitation train in Japan, for example, or the MRI, so the magnetic resonant system in the medical industry. That is based on this principle. So already working on this principle today. The dream in power electronics would be that a superconducting electronic would be available. And there are now research studies showing that there are materials showing superconductivity effect even to higher temperature, even close to room temperature. But in this case, you need a lot of pressure. You need to contain this system under a lot of high pressure. So very challenging. To answer your question shortly. I think the next frontier, the next step is probably superconductivity, but we need a better cytostatic technique. So cryogenic technique to reach the superconductive state for the metals.

 

Host: Yeah. All right. Very well put. Do you have any last words today?

 

Guest: Well, I can just say, Peter, it's a very exciting time because we are on the verge of a technological revolution, technological transition and revolution that doesn't happen very often. I think we, as Infineon, are proud to be part of this race, to be part of this game. And we participate in many aspects. And basically, this is what I think motivates me as an engineer, as a leader in Infineon. And we try to bring this innovation and this value also to our customers through improving the system of solutions.

 

Host: Yeah. Thank you so much for coming in today and sharing your knowledge with us.

 

Guest: Thank you, Peter.

 

Host: Thanks a lot. And to our audience, I say thank you for dropping in today. And if you would like to submit any questions or ideas for future episodes, please feel free to send an email to wepowerai@infineon.com. Thank you and see you soon.