Review and Projections of Integrated Cooling Systems for Three-Dimensional Integrated Circuits Satish G. Kandlikar Mechanical Engineering Department and Microsystems Engineering Department, Rochester Institute of Technology, Rochester, NY 14623 e-mail: sgkeme@rit.edu In an effort to increase processor speeds, 3D IC architecture is being aggressively pursued by researchers and chip manufacturers. This architecture allows extremely high level of integration with enhanced electrical performance and expanded functionality, and facilitates realization of VLSI and ULSI technologies. However, utilizing the third dimension to provide additional device layers poses thermal challenges due to the increased heat dissipation and complex electrical interconnects among different layers. The conflicting needs of the cooling system requiring larger flow passage dimensions to limit the pressure drop, and the IC architecture necessitating short interconnect distances to reduce signal latency warrant paradigm shifts in both of their design approach. Additional considerations include the effects due to temperature nonuniformity, localized hot spots, complex fluidic connections, and mechanical design. This paper reviews the advances in 3D IC cooling in the last decade and provides a vision for codesigning 3D IC architecture and integrated cooling systems. For heat fluxes of 50–100 W/cm2 on each side of a chip in a 3D IC package, the current single-phase cooling technology is projected to provide adequate cooling, albeit with high pressure drops. For future applications with coolant surface heat fluxes from 100 to 500 W/cm2, significant changes need to be made in both electrical and cooling technologies through a new level of codesign. Effectively mitigating the high temperatures surrounding local hot spots remains a challenging issue. The codesign approach with circuit, software and thermal designers working together is seen as essential. The through silicon vias (TSVs) in the current designs place a stringent limit on the channel height in the cooling layer. It is projected that integration of wireless network on chip architecture could alleviate these height restrictions since the data bandwidth is independent of the communication lengths. Microchannels that are 200 lm or larger in depth are expected to allow dissipation of large heat fluxes with significantly lower pressure drops. [DOI: 10.1115/1.4027175] Keywords: cooling, 3D IC, review, single-phase cooling, electronics cooling, flow boiling, microchannels 1 Introduction A 3D IC architecture incorporates multiple device layers that are interconnected through vertical interconnects to extend the performance of a 2D chip. A high level representation of a 3D IC structure is shown in Fig. 1. The heterogeneous/homogeneous cores are stacked in a single or multiple chip modules. The logic and memory devices are integrated at single or multiple levels depending on the system requirements. The TSVs and thermal vias play a crucial role in providing pathways for thermal and electrical communications between different layers. The main advantages of the 3D IC architecture are shorter global interconnect Contributed by the Electronic and Photonic Packaging Division of ASME for publication in the JOURNAL OF ELECTRONIC PACKAGING. Manuscript received July 30, 2013; final manuscript received February 15, 2014; published online April 29, 2014. Assoc. Editor: Gongnan Xie. Journal of Electronic Packaging Fig. 1 A schematic representation of a 3D IC cooled with a coolant layer lengths, higher circuit functionality, smaller footprint of the chip, and heterogeneous integration, for example, of memory and logic devices. A number of cooling options have been considered in the literature. Kandlikar et al. [1] presented an overview of different cooling options, including air and liquid single-phase flow, and two-phase flow (flow boiling). The interrelations between the heat transfer rate, fluid and chip surface temperature distributions, and pressure drop were discussed and the major outstanding issues in the cooling system design were outlined. Before reviewing available literature on the thermal design issues specifically related to 3D ICs, some of the commonly used terms in these systems are first introduced. The heat generated within a 3D IC chip can be removed by different cooling system configurations. For a convectively cooled 3D IC package, important heat dissipation related terms are defined below. In a chip stack, IC chips and coolant chips are alternately (or multiple IC chips followed by a cooling chip) placed to form the 3D package. Coolant chips may also be placed on the outer faces of the stack. Coolant chip—refers to a chip that integrates the cooling channels or other cooling configurations, such as coolant jets, sprays or conduction pathways. Chip stack heat density, q_ S , W/m3—defined as the total heat generated within a stack of 3D IC chips divided by the total volume of the stack including the chips and the cooling system embedded within the stack. This includes the volume of the coolant chips at the ends, if present, but excludes the piping and external pumps and other cooling system components. Chip heat density, q_ C , W/m3—refers to the total heat generated within a single chip divided by the volume of the chip. Chip heat flux, q00C , W/m2—defined as the total heat generated within a chip divided by the footprint area of the chip, irrespective of whether the chip is cooled from one side, both sides, or from the edges. It is sometimes customary to express the heat fluxes in W/cm2, although the equations are all based on the standard SI units of W/m2. Coolant chip surface heat flux, q00CS;1 and q00CS;2 , W/m2—refers to the total heat flux removed from one surface of the coolant chip divided by its projected area of the chip in the active region. The two sides of the coolant chip may have different heat fluxes. In the case of a stack with multiple coolant chips, a detailed analysis may be required to determine the fraction of the total heat generated within an adjacent 3D IC chip that is being removed from a given side of the coolant chip. This distinction is important as a 3D IC chip may be cooled from both sides. Similarly, a coolant chip provides cooling on both of its sides when embedded between two 3D IC chips. C 2014 by ASME Copyright V JUNE 2014, Vol. 136 / 024001-1 Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/ on 06/04/2014 Terms of Use: http://asme.org/terms 2 Cooling Options for High Heat Flux Electronics Significant advances have been made in cooling of high heat flux IC chips since the first introduction of microchannels by Tuckerman and Pease [2] to remove 790 W/cm2 from an IC chip using water with a temperature rise of 71 C. The high pressure drops of 200 kPa with plain microchannels and 380 kPa with pinfin enhanced microchannels posed a major challenge, although the feasibility of using microchannels to accomplish the high heat flux removal was clearly demonstrated. Dissipating this level of heat flux at low pressure drops still presents a formidable challenge. Colgan et al. [3] employed an offset strip fin configuration as an extension of plain microchannels and dissipated over 500 W/cm2 with a thermal of around 16 C-mm2/W and a pressure drop of 42 kPa using water. The heat transfer coefficient in the same offset strip fin geometry was reported by Steinke and Kandlikar [4] to be in excess of 500,000 W/m2 C. Other cooling options using jet impingement and spray cooling have been evaluated by Kandlikar and Bapat [5]. These systems suffer from their high pressures requirements, complexities of the fluidic connections and space restrictions in the IC cooling applications. Cotler et al. [6] investigated spray cooling of water directly on a lateral diffused metal oxide semiconductor field effect transistor. A maximum heat flux dissipation of 162 W/cm2 was achieved with a reduction in the junction temperature of 90 C. The spray cooling improved the cooling performance and reliability of the chip. The system pressure was not reported. Chang et al. [7,8] studied microjets though 50–200 lm slots on simulated chips and found that the heat transfer was enhanced over macrojets, and resulted in a more uniform temperature distribution on the chip surface. Although the spray cooling and microjet cooling provide good thermal performance, the pressure drop in the nozzles is high and the header connections are complex and require larger space as compared to microchannels. Flow boiling in microchannels is also attractive due to the inherently efficient boiling mechanism. However, it suffers due to boiling instabilities, a low critical heat flux (CHF) limit, and low heat transfer coefficients [9,10]. Microchannel heat sinks were studied by a number of researchers for VLSI chip cooling applications, starting with Tuckerman and Pease [1]. Jiang et al. [11] studied the thermal performance of a thermal test chip with integrated sensors. They investigated microchannels with hydraulic diameters in the range 40–80 lm. Power densities of more than 20 W/cm2 were dissipated with a water flow rate of 5 mL/min using a microchannel heat sink with 34 microchannels. Although the performance was not exemplary, the implementation of microchannel cooling in a practical device seemed quite encouraging based on their results. Koo et al. [12] modeled two-phase microchannel heat sinks and showed that it was possible to dissipate 200 W from a 20 mm 20 mm chip with a pressure drop of less than 20 kPa using a water flow rate of 15 mL/min. They indicated that deeper channels were beneficial. The depth of the channels directly competes with the requirement of shorter TSVs from an electronic standpoint. The microchannel cooler developed by Colgan et al. [3] for chip cooling applications has shown potential for handling large heat fluxes as high as 1 kW/cm2. The cooler has offset strip fins with multiple inlet and exit header connections. The multiple headers restrict the cooling ability of the cooler to only one side, unless two coolers are sandwiched together. The resulting increase in the gap between two active layers may not be acceptable due to the longer TSV lengths. The high pressure drop due to offset strip fins is also not acceptable in the 3D ICs. 3 dissipating above 50–100 W/cm2, single phase liquid cooling or flow boiling with water (at subatmospheric pressures) or a dielectric fluid may be required. A good history of the development of the fabrication technologies for monolithic 3D ICs has been presented by Petti et al. [13]. Pavlidis and Friedman [14] presented the speed and power consumption advantages of using the third dimension on the 3D network on chip (NoC). Wong et al. [15] demonstrated the performance advantages of this technology over a 2D architecture. A monolithic 3D IC is defined as an IC with circuit elements fabricated on a substrate with at least one additional circuit layer above this substrate in a single process flow. No bonding is used, thus making the process provide a pathway to go beyond the scaling limits imposed by the conventional 2D architecture. A number of researchers utilized this technology in developing multilayer CMOS devices [16–19]. The 3D IC provides an expanded area without increasing the signal delay with proper design and optimization. A clear perspective of 3D IC architecture, which is also known as vertical integration, is provided by Topol et al. [20]. The main benefit of a 3D IC over a 2D IC is in terms of the circuit performance advancement, as each transistor can access a greater number of nearest neighbors, even in the absence of device scaling. The reduction in the number of repeaters required in long interconnects also leads to a reduction in total power. Further benefits include increased packing density, immunity to noise, and reduced capacitance. Another major benefit is the ability to add functionality by integrating RF, analog, optical and micro-electromechanical systems to create a hybrid circuit. A number of researchers have provided great insight and guidance in developing the 3D IC architecture, e.g., Refs. [21–23]. However, some of the major challenges still faced are the increased cooling system complexities and cooling the hot spots created by the concentration of heat generating devices. Some of these issues can be mitigated through proper design and optimization of the layout; therefore a more careful design approach is warranted. 4. 3D IC Thermal Management Approach 4.1 Overview. The ultimate goal of any IC chip cooling system is to keep the junction temperature at or below the maximum allowable limit. As the heat generating regions become aligned in adjacent layers, the localized hot spots are created. Some of the junctions also generate large amounts of heat, creating localized hot spots. A cooling system needs to provide adequate cooling for these hot spots to limit their operating temperatures. The development in the cooling technology can be effectively summarized by illustrating the evolution from the cooling systems for a single IC chip to the embedded microchannel coolers for the 3D ICs as depicted by Koo et al. [24]. Figure 2 shows the basic 3D architecture implemented on a single chip with three device layers. The chip is cooled in a conventional way with the interconnects on one side, and a heat sink mounted with a thermal interface material to reduce the thermal contact resistance on the other Brief Review of 3D IC Architecture The cooling requirements for 3D ICs range from less than a hundred W/cm2 to several hundred W/cm2 of the projected chip area. For low heat fluxes, it may be possible to accomplish cooling of a 3D IC stack using a heat sink. However, for 3D IC packages 024001-2 / Vol. 136, JUNE 2014 Fig. 2 A schematic of a three layer 3D IC structure with an external heat sink. Redrawn from Ref. [2]. Transactions of the ASME Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/ on 06/04/2014 Terms of Use: http://asme.org/terms side. The three layers with embedded vertical interconnects provide significant electronic performance improvement. The cooling can be provided by any conventional means, such as an air cooled heat sink or a microchannel cooler embedded in the chip as demonstrated by Colgan et al. [3]. Since the electrical vias do not intersect with the cooling channels, the thermal design considerations are essentially similar to those for a 2D chip. However, the thermal profile within each layer is affected by the presence of the three device layers, which may present additional thermal resistances and heat sources in the heat flow path. The resulting thermal intensification needs to be considered during the device layout design. A number of investigators have presented conceptual designs to incorporate microchannel cooling within the 3D IC architecture. Lu et al. [25] numerically studied the integration of heterogeneous IC technologies that are applicable to smart wireless terminals, millimeter phased array radars, and smart imagers. Pande et al. [26] presented an excellent overview of a novel interconnect infrastructure for multicore chips. These concepts are applicable to 3D integration as well. Bakir et al. [27] presented a conceptual design of a 3D IC architecture that combines TSVs for providing power and signaling, optical interconnects to enable optical signal routing to all levels of the 3D IC stack, and microfluidic interconnects between the coolant channels in different layers for circulating coolant. Figure 3 shows a fully integrated 3D architecture with multiple device layers on three IC chips and two embedded coolant chips as illustrated by Koo et al. [24]. The coolant layer consists of microchannels, with the TSVs passing through the channel walls. This imposes additional constraints on both the microchannel and TSV placements, thus necessitating careful design from both electrical and thermal viewpoints. Various fabrication techniques have evolved to produce multiple device layers on a single chip. Further discussion on the current state of fabrication technology is beyond the scope of this review; this work focuses mainly on the cooling and related electronic codesign aspects of the 3D IC architecture. 4.2 Thermal Design Requirements in 3D IC Cooling. The major requirements and constraints for an embedded cooling system within a 3D IC architecture are listed below. These will be discussed further in detail in Secs. 5–13: (1) The coolant layers on the two sides of the chip with multiple device layers should provide adequate cooling to remove heat generated in these device layers, keeping the junction temperature below the allowable maximum temperature at all times. (2) The temperature nonuniformity in a device layer should be below the acceptable upper limit. The temperature differences between adjacent device layers are also of concern. Temperature nonuniformity results mainly from the following sources: (a) Hot spots with significantly high heat generation rates are of particular concern, as the local hot spot Fig. 3 A schematic representation of a microchannel cooled three layer 3D IC structure. Microchannels can be employed in both single phase and two-phase (evaporative) modes. Redrawn from Ref. [24]. Journal of Electronic Packaging temperatures could rise significantly above the allowable limit. (b) The presence of interlayer thermal resistance and the tandem location of heat sources (ICs) along the heat flow path result in localized high heat flux regions. The coolant temperature rise along the flow direction introduces another major source of temperature nonuniformity that affects the temperatures of the adjacent devices. This requires a careful control of the thermal resistance in the flow direction to provide a compensating effect. (3) The locations of the TSVs also influence the coolant layer design. For a coolant layer with embedded microchannels, the TSVs need to pass through the microchannel walls. (4) Taller channels in the coolant layer result in a lower pressure drop, while increasing the TSV length adversely impacts the electrical performance. The above considerations will be discussed in detail in the remainder of the paper. A review of some of the practical 3D IC cooling systems and research efforts on single-phase and twophase cooling technologies reported in recent years are presented in the Secs. 5–7. This is followed by a review and a detailed discussion on the energy efficiency and pressure drop considerations in the cooling system design, hot spot thermal management, temperature nonuniformity, role of TSVs and the limitations imposed by them, and the application of wireless interconnects in a 3D architecture. Further, major challenges faced in designing 3D ICs are outlined, and a vision for evolving a suitable cooling system is presented through codesign approach. 5 Practical Implementation of 3D Cooling Prior to 2005, work on 3D IC cooling was focused on multiple device layers on a single chip cooled in a conventional manner as shown in Fig. 2. Koo et al. [24] presented a clear cooling architecture for multilayered, 3D ICs with interchip cooling layers. They considered a 14.14 mm 14.14 mm chip with a channel layer thickness of 400 lm, employing eighteen microchannels, each with a width of 700 lm and a height of 300 lm. Water at a subatmospheric pressure of 0.3 bar was used with an inlet temperature of 70 C. The higher heat generating devices were placed toward the exit of the channel to reduce the two-phase pressure drop. A flow rate of 15 mL/min was employed. With a peak flux of 65 W/cm2, their analysis showed that the maximum junction temperature could be maintained below 85 C. The two-phase pressure drop was predicted to be 26.3 kPa and the maximum junction temperature gradient was 55 C/cm with a maximum junction temperature difference of only 13 C. Their predictions for a conventional heat sink (thermal resistance of 0.25 C/W) with a 28 mm 28 mm square, 1 mm thick heat spreader placed on one side of the 3D IC module indicated a temperature gradient of 300 C/cm with a maximum junction temperature difference of 45 C. The device load distribution in the above work by Koo et al. [24] considered a higher load near the exit of the channels. This helped in reducing the pressure drop. Their work clearly showed the potential of two-phase cooling using water at subatmospheric pressures directly in the silicon channels. Although the feasibility of two-phase cooling was theoretically demonstrated, they pointed out the outstanding issues of flow maldistribution and instability as reported earlier by Kandlikar [28]. Sekar et al. [29] working in a team from Georgia Institute of Technology and IBM Corporation demonstrated a practical implementation of integrated microchannels within a 3D IC stack. They employed fluidic TSVs to deliver liquid to the microchannels. The TSVs for the electrical interconnects were fabricated in the silicon as shown in Fig. 4. The microchannels were 200 lm tall and 150 lm wide, and were fabricated in a 400 lm thick silicon JUNE 2014, Vol. 136 / 024001-3 Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/ on 06/04/2014 Terms of Use: http://asme.org/terms Fig. 4 Cross sectional view of the copper TSV passing through a silicon microchannel wall. Microchannel width 150 lm, height 200 lm, and TSV diameter 50 lm. Reprinted with permission from Sekar et al. [29]. substrate. The copper TSVs were 50 lm diameter and passed through the walls of the microchannels. The silicon die contained 2500 TSVs per square cm area. The fluidic interconnects and microchannels provided the same thermal resistance as a singlechip. Significantly smaller TSVs are being employed currently with the ability to manufacture large aspect ratio channels for TSVs. The manufacturing constraint imposed by the high aspect ratio copper TSVs through the thickness of the silicon, and the thermal performance for different silicon chip thicknesses were also analyzed by Sekar et al. [29]. The silicon chips with a thickness between 75 and 400 lm were used. For a maximum pressure drop of 207 kPa (30 psi), the flow rate through each chip in the 3D stack varied from 7.6 mL/min for the 75 lm thick chip, to 1650 mL/min for the 400 lm chip. The corresponding thermal resistances were 2.05 C/W and 0.07 C/W. They employed an aspect ratio (TSV length to diameter ratio) of 6:1, with the TSV diameter varying from 13 lm to 67 lm for the 75 lm and 400 lm thick chips, respectively. With manufacturing advances, these TSV diameters could be reduced to 4.7 lm and 25 lm, respectively, for an aspect ratio of 16:1. However, much larger aspect ratio TSVs are possible with current microfabrication technology. The above analysis clearly shows the critical importance of the coolant chip thickness on the thermal as well as electrical performance parameters. A thicker silicon chip is preferred for allow- ing larger coolant flow rates with lower pressure drops and lower thermal resistances. However, higher aspect ratio TSVs is needed for thicker silicon substrates. The longer TSVs also introduce significant delays in the signals. In a later publication, Dang et al. [30] presented a detailed report on the design, fabrication, and testing of an integrated microchannel with heat sink and copper TSVs. They used thermofluidic chips with interconnections as shown in Figs. 5 and 6. The microchannels are etched with trenches and filled with a sacrificial polymer by spin-coating and then polished with a surface planarizer. The resulting surface is over-coated with a layered structure and the sacrificial polymer is decomposed and removed by heating. Further thinning of the cover is possible to provide the desired thickness. Polymer micropipes fabricated from SU-8 are used for the microfluidic connections as shown in Fig. 6. The fluid is transferred across the chips to the next microchannel layer through these fluidic interconnections. Using a flow rate of 65 mL/min, a chip temperature rise of 18.1 C was obtained with a thermal resistance of 0.4 C/W. The temperature rise was reduced to 12.7 C by increasing the flow rate to 104 mL/min, while the thermal resistance reduced to 0.28 C/W. The associated pressure drop values were not reported. Further fabrication and assembly details may be found in Ref. [30]. Dang et al. [30] further introduced a conceptual design with the electrical and fluidic TSVs in a 3D architecture as shown in Fig. 7. The microchannel cooling platform is capable of dissipating over 100 W/cm2. The monolithically integrated microchannel heat sink, copper electrical interconnects (TSVs), polymer fluidic interconnects, and solder bumps together provide a stackable structure. 6 Single-Phase Liquid Cooling Liquid cooling of 3D ICs is attractive because of the higher specific heat and thermal conductivity of liquids as compared to air. Water has the best thermal properties in this regard, and has been used extensively to show the feasibility of liquid cooling of IC chips [2,3]. Dielectric fluids are preferred because of their Fig. 5 Microchannels on a silicon chip glue bonded to a glass cover plate. Reprinted with permission from Dang et al. [30]. Fig. 6 SEM micrographs showing polymeric micropipes adjacent to solder bumps. Reprinted with permission from Dang et al. [30]. 024001-4 / Vol. 136, JUNE 2014 Transactions of the ASME Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/ on 06/04/2014 Terms of Use: http://asme.org/terms investigators have developed algorithms to analyze the thermal performance of the 3D stack, e.g., Refs. [38–41]. The applicability of CFD in single-phase liquid flow in microchannels has been verified in literature, [10,12], and these tools are recommended in designing 3D IC cooling systems. Such simulations are particularly useful in predicting the transient behavior as well as the thermal field near a hot spot, as such measurements are not readily available for characterizing the thermal performance of a cooling system. 7 Fig. 7 Schematic of a proposed 3D IC stack cooled with microchannels with fluidic and TSV interconnects. Redrawn with permission from Dang et al. [30]. electrical properties, but suffer from lower thermal conductivity and specific heat. Using deionized water, and maintaining a low ionic concentration and a high electrical resistivity, is another possible option. Some of the investigations related to liquid cooling of 3D IC chips are reviewed in this section. Alfieri et al. [31] used water to cool a stack of four 3D IC chips. The height of the channel was 100 lm, with a TSV length of 150 lm. The microchannel space was filled with full-height pin fin structures 50 lm in diameter placed at a pitch of 100 lm. The experimental work was complemented with a computational fluid dynamics (CFD) simulation considering conjugate heat transfer. The total power dissipated in the stack was 390 W at a temperature budget of 60 C. This corresponded to a volumetric cooling of 1.3 kW/cm3. The pressure drop with the aggressive pin-fin design was 11 Pa/lm corresponding to a heat flux of 125 W/cm2 at a mass flow rate of 2.16 103 kg/s (approximately 122 mL/min). For a 10 mm flow length, the resulting pressure drop is 110 kPa. The work of Alferi et al. [31] demonstrated the ability of single-phase liquid flow in meeting the IC chip cooling needs. The issues related to introducing liquid water in close proximity to the electronic devices and the high pressure drop are two main outstanding concerns with such systems. Zhang et al. [32,33] employed pin fins in the coolant layer to enhance the cooling performance. The same pin fins also provided pathways for the TSVs. The fin heights were kept low to improve the TSV performance. The coolant layer chip thickness was 200 lm, as compared to around 60 lm for a 3D stack without coolant channels. They did not report the pressure drop in their microfluidic passages with pin fins, but it is expected to be high compared to microchannels with TSVs embedded in the fins. Zhang and Bakir [34] recently studied the microchannel layers with independently controlled flow rates to reduce the temperature differences between two adjacent IC chip layers. The reduction in the temperature difference is beneficial in improving electrical performance and reducing thermal stresses. A number of investigators have analyzed the 3D IC structure with microchannels and other features such as interstrata microchannels, thermal TSVs and transient response. Lau and Yue [35] analyzed the thermal performance of a microchannel cooled 3D IC stack with copper TSVs. Ziabari and Shakouri [36] developed a fast thermal simulation program to analyze the 3D IC geometry with thermal vias without the need for remeshing. Shi et al. [37] analyzed a hybrid microchannel/TSV module and demonstrated the power savings in the circulation pumps by using the TSVs to assist in transferring heat to the microchannels. A number of Journal of Electronic Packaging Two-Phase (Flow Boiling) Cooling Flow boiling in microchannels embedded in the coolant layer is an attractive option because of its ability to remove larger quantities of heat than a single phase liquid flow system at a given fluid flow rate. However, boiling instabilities, low heat transfer coefficients, low critical heat fluxes, and high pressure drops are some of the main concerns regarding flow boiling in microchannels [28]. Koo et al. [42] considered flow boiling in microchannels in heat sinks for VLSI applications. Their modeling work demonstrated the feasibility of using microchannels directly in silicon for flow boiling. They predicted a temperature rise of 40 C while dissipating 200 W in a 25 mm square heat sink. The pressure drop was predicted to be less than 25 kPa with water. Steinke and Kandlikar [43] employed six copper microchannels, 214 lm wide 200 lm deep, spaced 570 lm apart. They observed severe backflow under low heat flux conditions. The heat transfer coefficient was found to be high (around 200 kW/m2 C) at qualities below 0.1 and decreased significantly to around 50 kW/m2 C at qualities above 0.3. Periodic dryout was also observed at higher qualities. Pressure drops in excess of 50–100 kPa were also noted. In light of these findings, it is seen that high exit qualities are detrimental to microchannel flow boiling systems. Subsequent work showed that the instabilities can be reduced by introducing pressure drop elements (flow restrictors) at the entrance, and providing nucleation cavities in the microchannels [44–46]. Addition of the restrictors increased the overall pressure drop in the flow boiling system. At higher heat fluxes however, the severity of boiling instabilities was reduced. The fluctuations combined with periodic dryout limited the maximum heat fluxes (critical heat flux, or CHF) that can be safely dissipated in a microchannel boiling system. Flow boiling in smaller microchannel geometries was recently studied by Szczukiewicz et al. [47] with refrigerants R245fa, R236fa, and R1234ze (E) using 100 lm 100 lm square microchannels. They observed that inlet orifices are needed to provide stable operation. A heat transfer coefficient of 20–25 kW/m2 C was obtained with R236fa at a mass flux of 2299 kg/m2s and a heat flux of 48.6 W/cm2 over a quality range of 0–0.2. The resulting pressure drop was 113 kPa, which is quite substantial. Intense research is being conducted on flow boiling systems to capture the benefits of the boiling process. Specifically, the DARPA/MTO ICECOOL program [48] is currently sponsoring many leading researchers to develop integrated cooling/hot spot management solutions. Such collaborative efforts are expected to provide breakthroughs in the development of 3D IC cooling systems addressing the relevant issues of high heat flux dissipation and hot spot thermal management. Recent investigations on flow boiling in microchannels have been successful in overcoming the flow instability and low performance issues reported in Ref. [28]. These concerns have been overcome by introducing the concept of tapered microchannels proposed by Mukherjee and Kandlikar [49,50]. The recent works by Lu and Pan [51], Miner et al. [52], and Kandlikar et al. [53] have shown that these expanding microchannels are quite effective in addressing the shortcomings noted earlier [28]. A record heat transfer coefficient above 300 kW/m2 C and a critical heat flux of over 500 W/cm2 with a pressure drop below 3–5 kPa were reported in Ref. [53]. However, further research is needed before JUNE 2014, Vol. 136 / 024001-5 Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/ on 06/04/2014 Terms of Use: http://asme.org/terms implementing flow boiling systems in practical 3D IC devices. Application of nanowires on all sides of microchannels was achieved by Yao et al. [54] in pool boiling application. They observed microbubble emission boiling phenomenon which greatly improved the heat transfer performance. Yang et al. [55,56] employed silicon nanowires on the microchannel surfaces to generate a large number of microbubbles emerging from the nanowires. They reached a CHF of 400 W/cm2 under stable conditions with a heat transfer coefficient of 125.4 W/m2K and a pressure drop of around 20 kPa. Application of nanowires on the microchannel surfaces is thus seen as a promising new field in flow boiling. 8 Further Considerations in Cooling System Design Recently there has been a considerable rise in awareness regarding the energy efficiency of cooling systems. The efficiency is measured in terms of coefficient of performance (COP), which is defined as the ratio of the heat removed from the chip to the pumping power expended in circulating the coolant through the IC chip. The DARPA/MTO ICECOOL program [48] has taken the lead in defining the efficiency limits in their research program. Such considerations are extremely important from energy conservation and environmental viewpoints while designing large data centers which constitute a substantial fraction of total global electricity consumption. A COP of 20 or higher is the goal proposed by the ICECOOL program. A high pressure drop is undesirable for three reasons: (i) the high pressure drop in a two-phase system leads to changes in the saturation temperature along the flow length leading to temperature nonuniformities, (ii) higher pumping power is needed, and (iii) large pressure differences introduce undesirable stresses in the silicon passages thereby adversely affecting the reliability of the system. It is therefore recommended to keep the pressure drop low, preferably below about 20–30 kPa. Even lower pressure drops would be highly desirable. Some of the outstanding challenges that are not yet addressed are: fluid flow through sharp corners and bends encountered in 3D IC cooling systems, noise due to two-phase flow, and overall reliability of such systems in conjunction with the external components of the cooling system. 9 Hot Spot Management A cooling system such as a microchannel cooler removes heat generated at different junctions by conduction through the IC layers if they are stacked together. As some of the junctions become aligned in their heat flow paths, or generate significantly high heat fluxes, it becomes necessary to implement some additional cooling techniques to reduce the localized overheating in these areas. Some of the techniques employed rely on heat spreading with diamond films and spot cooling provided by sprays or jets of liquids. Fabricating a diamond film is expensive and introduces fabrication steps with significant processing time delays. The specialized microfluidic connections needed to incorporate the nozzles and sprays are extremely complicated and introduce a high pressure drop. Additional valves required to control the coolant flow to the heated junction affect the reliability of the package. Clearly, a major breakthrough is needed to address these issues. In a more in-depth analysis of the interaction between the heat source and the liquid in a microchannel in a 3D IC stack, Mizunuma et al. [57] numerically investigated the placement of the devices along the liquid flow direction. They studied the thermal wake effect that increases the liquid temperature beyond the heat source and reduces the cooling performance for the devices located in the wake region. They recommended a thermal-aware placement of the devices in the descending order of power in the flow direction. Such a design approach reduced the maximum temperature by 25 C in their simulation. Kearney et al. [58] proposed an electrokinetically pumped microchannel loop to transfer heat from one stratum to the other. 024001-6 / Vol. 136, JUNE 2014 This solution is particularly attractive if the hot spots in the two layers are co-aligned. Figure 8 shows a schematic representation of the through-plane microchannels investigated by Kearney et al. [58]. The liquid is pumped through these microchannels using electrokinetically driven micropumps. This technique provides an efficient way to cool the hot spots. The overall effect of placing a micropump and the fabrication complexities are some of the factors that need to be considered. Sridhar et al. [38,39] numerically analyzed the compact geometries with inter strata microchannels and presented detailed results on the thermal characteristics near the hot spots. Hot spot management requires a methodical approach starting from the design at the architectural level to the software controlled redistribution of the computing resources. In spite of such efforts, specific situations exist where localized hot spots cannot be avoided. It would be most beneficial if the junctions or the clusters of junctions creating the hot spot are placed on a layer closest to the coolant channel. If the heat spreading can be accomplished with a silicon layer, it will eliminate the high cost of fabricating the diamond film. From a thermal standpoint, it is desirable to provide localized cooling with a passive device that is integrated with the overall cooling system. In this regard, localized flow disruptors, such as pin fins, nanostructures, and combined nano/microstructures are attractive candidates. A significant body of research is available on some of these systems at macroscale. Their extension to microscale single-phase and two-phase cooling systems is a recommended area for further research. 10 Temperature Nonuniformity Uneven temperature distribution can result in large thermal gradients in the IC substrate. Such large gradients result in mechanical stresses, departures in device performance, and synchronization issues which lead to reliability concerns [59,60]. Temperature nonuniformity is induced in a 3D IC chip stack due to the three main factors: (i) presence of nonuniform heat generating devices or aligning of high heat generating devices in the adjacent layers causing hot spots, (ii) temperature variation of the fluid, such as the increasing temperature of the single-phase coolant, or decreasing temperature of an evaporating coolant along the flow direction, and (iii) variation in the heat transfer coefficient along the channel length. Proper cooling system design can mitigate the temperature nonuniformity issues. For example, local hot spots can be managed by modifying the microchannel shape. Tilley [61] proposed different channel shapes that introduce advection-induced flows that locally enhance the heat transfer coefficient around a hot spot. Alternatively, localized turbulators can be introduced in front of a hot spot to reduce the temperature rise due to the hot spot. In the case of a flow boiling system, reducing the pressure drop will provide a reasonably uniform saturation temperature of the fluid. Fig. 8 Schematic representation of through-plane microchannels between two coolant layers to cool hot spots. Redrawn with permission from Kearney et al. [58]. Transactions of the ASME Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/ on 06/04/2014 Terms of Use: http://asme.org/terms One approach taken by Rubio-Jimenez et al. [62,63] was to recognize that the available temperature difference between the fluid and the channel walls decreases along the flow direction. This can be offset by an increase in the heat transfer coefficient caused by increasing the fin density from the inlet to the outlet sections. They employed offset strip fins of various shapes—circular, flat, and tear-drop—and studied the channel wall temperature uniformity for these geometries. The resulting pressure drop penalty was also studied. In a follow-on study, Lorenzini-Gutierrez and Kandlikar [64] extended this approach to the coolant passages of a 3D IC cooled with water. They demonstrated an optimum geometry that provided uniform temperature after the entrance region with a pressure drop of less than 30 kPa while dissipating 100 W/cm2 in 100 lm tall coolant passages. The variable density fins along the flow channel is seen as an effective way to improve the heat transfer performance and provide temperature uniformity, while carefully managing the available pressure drop budget. 11 TSVs TSVs are critical elements in a 3D IC architecture. They enable power and signal transfers across different device layers leading to performance improvement, which is one of the major benefits of the 3D ICs. TSVs are also used to transfer heat in the throughplane direction using the higher thermal conductivity of copper. There are four main issues related to TSVs that are relevant to the thermal management of 3D ICs: (1) (2) (3) (4) placement on a device layer length of the TSVs fabrication cost alternative technologies TSV placement is a very critical aspect in the design of the 3D IC architecture. The electrical performance enhancement depends to a large extent on the judicious placement of TSVs and devices to take full advantage of the vertical integration. A number of layout programs are available that consider the device functionality while designing the 3D IC layout plan. It is necessary to integrate the thermal profiles of the devices while designing the device layout and TSV placements. The TSVs cross the cooling layer through the microchannel walls or the pin fins that are embedded in the coolant passages. The width of the microchannels, in the absence of any pin fins, determines the distance between adjacent rows of the TSVs. From a thermal viewpoint, the microchannels are desired to be wide, between 200 lm and 1000 lm (as dictated by the mechanical strength of the silicon layer), while the TSVs are desired to be considerably closer than that. Codesign for satisfactory thermal and electrical performance will help in alleviating these problems. Using the approach of “vias first,” as described by Ramm et al. [65], the TSVs may be placed on the microchannel walls which are laid out beforehand to meet the thermal needs. If the TSV placement constraint does not permit a satisfactory electrical design, the channel and fin width of the microchannels could be reworked until a satisfactory solution emerges. The length of the TSVs impacts both the electrical and the thermal design. Shorter lengths are preferred to reduce the electrical transmission delays, while longer ones are desired from a thermal standpoint to allow for taller microchannels. For a given width, the height of the microchannel inversely dictates the pressure drop, or alternatively, it limits the liquid flow rate to meet the pressure drop constraint. Fabrication technology plays a key role in the design as well as final determination of any given architecture. Discussion on this topic is beyond the scope of this article. Readers are referred to Refs. [26,65–69] for further information on fabrication issues related to TSVs. There are a few alternative technologies to TSVs being proposed in the literature. Wireless interconnects is the leading contender, and is discussed in Sec. 12. Journal of Electronic Packaging 12 Wireless Interconnects The electrical interconnects perform three main functions—(i) to deliver power, (ii) to provide clock networks, and (iii) to provide data buses. The delays introduced by the interconnect lengths affect these three functions differently. In general, acceptable delays in the clock network are very critical. However, depending on the application, the delays, and hence the acceptable TSV lengths, could be different. This information is utilized in designing the electrical network. Several researchers have studied wireless NoC to accomplish these functions. Floyd et al. [70] developed a novel concept of incorporating antennas, receivers, and transmitters on a chip to provide wireless interconnects for clock distribution. This approach is very attractive for 3D ICs since it eliminates the need for TSVs and provides clock signal without the time-delay issues associated with TSVs. Wireless networks were also used for data transmission by Ganguly et al. [71]. They showed that incorporating wireless data transmission was vital to the successful implementation of the network-on-chip architecture as the conventional metal based interconnect technology may not be able to meet the scaling demands. They demonstrated that the hybrid wireless network outperforms the conventional metal interconnects in reducing the latency and increasing the network throughput. The on-chip wireless data transmission is seen as a revolutionary technology. In a 3D IC architecture, this technology offers significant benefits by replacing some of the on-chip interconnects as well as the TSV interconnects with the wireless network-on-chip. Some of the underlying issues related to efficient wireless communication design were addressed by Zhao and Wang [72]. They proposed a synchronous and distributed medium access control while selecting a specific channel. An in-depth understanding of the underlying wireless communication is critical when multiple transmitters and receivers are being employed in a more extensive 3D IC architecture in terms of the speed, power consumption, scalability, and cost. Reconfigurable hybrid wireless NoC on 2D ICs was studied by More and Taskin [73–75]. Their design included on-chip antennas for the wireless network layer and metal interconnects for the wired network layer. They considered two different frequency domains. The wireless network was essentially implemented for improving the performance of the 2D IC. They further extended their work to wireless interconnects between different layers in a 3D IC structure. They proposed using on-chip antennas in conjunction with TSVs for global communication in 3D ICs. They studied transmission gains for different chip bonding techniques and for different radiation frequencies. The wireless technology is sought after as an enabler for achieving performance gains even in a 2D IC. For 3D ICs, it is expected to play a crucial role in reducing the reliance on metal interconnects (TSVs). This has significant implication in terms of future cooling systems as the current TSVs pose a major challenge in designing a cooling system. 13 Major Challenges At the current juncture, 3D IC architecture is evolving through intense research worldwide. The core electrical issues being addressed by researchers are the development of new architecture that optimizes the functionality of individual devices through three-dimensional connectivity and advancements in fabrication technology. While the efforts are centered on enhancing electrical performance, the closer proximity of devices generating large amount of heat is leading to thermal challenges. The main thermal issues are summarized as follows: (a) Smaller available temperature differences than a 2D chip to account for the added thermal resistances in the multiple device layers. JUNE 2014, Vol. 136 / 024001-7 Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/ on 06/04/2014 Terms of Use: http://asme.org/terms (b) Low pressure drop due to mechanical and leakage considerations. (c) Shorter channel heights dictated by the interconnect lengths. (d) Hot spots due to high heat generating devices, or caused by stacking of multiple heat generating devices in close proximity in multiple layers. (e) Temperature nonuniformity throughout the 3D IC stack. (f) 3D IC cooling systems incorporate sharp corners and bends in the coolant flow path. Currently, we rely on the macroscale data for calculating fluid flow in these flow paths. It is recommended that the effect of these flow conditions on the heat transfer and pressure drop be investigated in the future. 14 the associated fabrication complexities, particularly the fluidic connectivity and leakage issues, need to be carefully evaluated. 14.4 TSV Lengths and Channel Heights. The TSV lengths and channel heights are interlinked and have diverse implications. While shorter TSV lengths are desirable from an electrical standpoint, larger channel heights are desirable to limit the pressure drop from a cooling system design perspective. Since a TSV crossing a cooling layer needs the extra traverse length, design compromises are needed. From the current design standpoint with two to three chip layers and a heat dissipation of 100 W/cm2 per coolant layer, the cooling passages would require a channel height of at least 100 lm. The resulting pressure drop is on the order of 30 kPa [64]. Any further increases in the heat flux would raise the pressure drop to unacceptably high levels. A Vision for 3D IC Cooling Through Codesign A number of constraints listed above are in conflict among themselves or with the electrical requirements. It is projected that these issues will be addressed through breakthrough ideas, careful design using the existing knowledge, and effective teaming. Some of the possible approaches are discussed here: 14.1 Balancing Heat Transfer and Pressure Drop Requirements—Single-Phase Enhanced Geometries. The desired high heat transfer performance requires enhanced fin designs to provide efficient heat transfer. However, the low pressure drop requirement favors designs that do not disrupt the flow aggressively. Offset strip fins are known for their superior performance, but the pressure drop is very high. Structured roughness is shown to enhance the heat transfer more favorably over the associated pressure drop penalty. Lin and Kandlikar [76] showed that a smooth sinusoidal roughness structure with a relative roughness of 0.11 provided the highest enhancement in heat transfer among different designs. They also noted that this structure provided the highest enhancement factor (defined as the enhancement in Nusselt number divided by the one-third power of the friction factor increase) in both laminar and turbulent ranges (beyond Re ¼ 900) as compared to other enhancement techniques, including alternating clockwise and anticlockwise rotating twisted tapes, porous medium inserts, and coiled inserts. Further research in this area is warranted on studying the effect of microscale roughness structure profile, pitch of the structured roughness features, and relative roughness on heat transfer and pressure drop for geometries that are relevant to 3D IC cooling systems for a specific channel height, aspect ratio and flow length. 14.2 Temperature Nonuniformity. The reduced heat transfer performance due to temperature nonuniformity induced by the temperature rise of single-phase coolant along the flow length can be balanced by increasing the heat transfer coefficient through higher fin density [62–64]. This approach is very promising as the available pressure drop budget is utilized in an efficient manner. Combining this technique with other enhancements, such as microscale roughness features, seems very promising. 14.3 Hot Spot Cooling—Codesign Approach. Cooling of hot spots is a critical consideration in designing 3D IC cooling systems. Codesigning the electrical and thermal systems will greatly reduce the occurrence and magnitude of the hot spots. Separating high heat generating devices in adjoining layers to avoid overlaps can be accomplished through such a collaborative approach. The software control process to identify the hot spots and to limit their magnitude at run time is being pursued by computer/electrical engineers [77–79]. Although a diamond substrate is an excellent heat spreader, its cost and fabrication complexity would justify its use only in some critical applications. Providing through-plane microchannels in the vicinity of the hot spots is another attractive approach. However, 024001-8 / Vol. 136, JUNE 2014 14.5 Two-Phase Cooling. Flow boiling in microchannels is an effective way to limit the coolant flow rate through the channels. However, the low heat transfer coefficients, low CHF, high pressure drop, and boiling instabilities constitute major roadblocks. These areas are being intensely investigated by the thermal research community. Some of the important outstanding topics are: (a) reduced instantaneous channel flow rate due to flow maldistribution, (b) lower maximum heat dissipation values, (c) temperature nonuniformities introduced by the varying saturation temperature and heat transfer coefficient along the flow length, and (d) proper selection of operating parameters. New configurations such as open microchannels with uniform and tapered manifolds presented by Kandlikar et al. [53] and Kalani and Kandlikar [80] that can dissipate very high heat fluxes at extremely low pressure drops of 5 kPa or lower should be considered. 14.6 Proposed 3D IC Cooling System With Wireless Interconnect Technology. Integrating direct liquid cooling is becoming more acceptable as the performance requirements push the boundaries of the existing cooling solutions. Garrou [81] reports the use of microchannels in cooling 3D ICs proposed by IBM using 50 lm channels. The pressure drop in these channels is expected to be quite high, thereby severely limiting their practical implementation. The main challenge in the realization of a 3D IC architecture in the long term is the ability to remove heat fluxes in excess of 100 W/cm2 without sacrificing the device performance. In order to realize the inherent advantages of the 3D IC architecture, heat fluxes of 100–500 W/cm2 from each side of the coolant channel are foreseen. One of the main roadblocks from the cooling system standpoint is the conflicting requirements of TSV lengths from electrical and thermal considerations. The TSV lengths, their thicknesses and placement will influence the level of integration and the cooling system design in a 3D IC module. To alleviate these cooling concerns, a conceptual 3D IC cooling design incorporating wireless transmission for clock and data as shown in Fig. 9 is presented. The wireless transmitters and receivers are added at appropriate locations in the device layers, while the conventional TSVs are still used as interconnects where the longer TSV lengths are acceptable. It is recognized that the pitch requirements are more stringent for certain interconnects, such as the clock signal. In the new conceptual design, these interconnects are handled by wireless networks that eliminate the need for TSVs. The concept originally proposed by Floyd et al. [53] for the clock, and by Ganguly et al. [71] for the data could be further expanded to provide such interconnects for devices both in the same plane as well as on chips in other layers. The issues of multiple wireless networks and their interference need to be resolved, posing research challenges to computer/electrical engineers. Secondary signal generation based on the primary wireless signal, utilizing wireless power transmission technology while optimizing the entire architecture, and efficient power management in the antennas are additional areas that need Transactions of the ASME Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/ on 06/04/2014 Terms of Use: http://asme.org/terms Acknowledgment This work is performed in the Thermal Analysis, Microfluidics and Fuel Cell Laboratory in the Mechanical Engineering Department at Rochester Institute of Technology. The author is thankful to his two colleagues in the Computer Engineering Department at RIT, Dr. Dhireesha Kudithipudi for providing the details of the overall 3D IC architecture, and Dr. Amlan Ganguly, for generously sharing his expertise and discussing his pioneering work in the area of wireless technology. The support extended by the students Ankit Kalani, Valentina Mejia, Preethi Gopalan, Andrew Greeley, and Mayuresh Kinare in the preparation of the manuscript is gratefully acknowledged. References Fig. 9 A conceptual 3D module with multiple microchannel cooling and active layers integrating wireless and TSV interconnects to be explored. There are certainly a number of challenges that will arise from an electrical standpoint that need to be resolved for incorporating the wireless network with the current chip layout design. Integrated 3D ICs with wireless network will improve both electrical and cooling performance, and enable a new level of system integration. However, integrating the wireless technology in the 3D-IC architecture poses a number of challenges and considerable research is warranted before realizing this conceptual architecture. The above conceptual design relaxes the severe height constraint imposed by the TSV based interconnects. The current limit of approximately 100 lm channel height restricts the heat flux dissipation to about 100 W/cm2, and prohibits implementation of advanced cooling techniques to dissipate higher heat fluxes, up to 500 W/cm2 at significantly reduced pressure drops of 10 kPa or lower. The larger channel gaps will also provide new opportunities for developing more efficient hot spot management strategies without significantly increasing the fabrication costs. Such approaches require a channel height of at least 200–400 lm. A shift in the paradigm of 3D architecture that takes advantage of the developments in the fields of wireless technology, software design, cooling system design, and microfabrication technology is imminent. 15 Conclusions This paper presents a review of the thermal issues and the cooling options in 3D IC chip stacks. A brief survey of the historical development is first presented, followed by a description of the four key thermal considerations: hot spot management, temperature nonuniformity, length, and placement of the TSVs, and the coolant channel height. The survey covers the current status of the single-phase and two-phase cooling technologies. The recent advances made through numerical and experimental work include: variable density fins in single-phase microchannel cooling systems, and an open microchannel design with tapered manifold employing a flow boiling two-phase system. The emerging wireless NoC technology is briefly reviewed and its potential in the future 3D IC architecture under significantly higher heat fluxes is outlined. The importance of a codesign approach among the hardware, software and thermal designers is highlighted, and the paper concludes with a conceptual 3D IC design integrating wireless network to provide faster signal transfer rates without adversely affecting the cooling performance. Journal of Electronic Packaging [1] Kandlikar, S. G., Kudithipudi, D., and Rubio-Jimenez, C. A., 2011, “Cooling Mechanisms in 3D ICs: Thermo-Mechanical Perspective,” IEEE International Green Computing Conference and Workshops (IGCC), Orlando, FL, July 25–28. [2] Tuckerman, D. B., and Pease, R. F., 1981, “High Performance Heat Sinking for VLSI,” IEEE Electron Dev. Lett., 2(5), pp. 126–129. [3] Colgan, E. G., Furman, B., Gaynes, M., LaBianca, N., Magerlein, J. H., Polastre, R., Bezama, R., Marston, K., and Schmidt, R., 2007, “High Performance and Subambient Silicon Microchannel Cooling,” ASME J. Heat Transfer, 129(8), pp. 1046–1051. [4] Steinke, M. E., and Kandlikar, S. G., 2004, “Single-Phase Liquid Heat Transfer in Plain and Enhanced Microchannels,” ASME 4th International Conference on Nanochannels, Microchannels and Minichannels, Limerick, Ireland, June 19–21, ASME Paper No. ICNMM2006-96227. [5] Kandlikar, S. G., and Bapat, A. V., 2007, “Evaluation of Jet Impingement, Spray, and Microchannel Chip Cooling Options for High Heat Flux Removal,” Heat Transfer Eng., 28(11), pp. 911–923. [6] Cotler, A. C., Brown, E. R., Dhir, V., and Shaw, M. C., 2004, “Chip-Level Spray Cooling of an LD-MOSFET RF Power Amplifier,” IEEE Trans. Comp. Packag. Technol., 27(2), pp. 411–416. [7] Chang, C. J., Chen, H. T., and Gau, C., 2013, “Flow and Heat Transfer of a Microjet Impinging on a Heated Chip: Part I—Micro Free and Impinging Jet,” Nanoscale Microscale Thermophys. Eng., 17(1), pp. 50–68. [8] Chang, C. J., Chen, H. T., and Gau, C., 2013, “Flow and Heat Transfer of a Microjet Impinging on a Heated Chip: Part II—Heat Transfer,” Nanoscale Microscale Thermophys. Eng., 17(2), pp. 92–111. [9] Kandlikar, S. G., 2012, “History, Advances, and Challenges in Liquid Flow and Flow Boiling Heat Transfer in Microchannels: A Critical Review,” ASME J. Heat Transfer, 134(3), p. 034001. [10] Kandlikar, S. G., Colin, S., Peles, Y., Garimella, S., Pease, R. F., Brandner, J. J., and Tuckerman, D. B., “Heat Transfer in Microchannels—2012 Status and Research Needs,” ASME J. Heat Transfer, 135(9), p. 091001. [11] Jiang, L., Wong, M., and Zohar, Y., 1999, “Phase Change in Micro-Channel Heat Sinks With Integrated Temperature Sensors,” J. Microelectromech. Syst., 8(4), pp. 358–365. [12] Koo, J.-M., Jiang, L., Zhang, L., Zhou, P., Banerjee, S. S., Kenny, T. M., Santiago, J. G., and Goodson, K. E., 2000, “Modeling of Two-Phase Microchannel Heat Sinks for VLSI Chips,” 14th IEEE International Conference on Micro Electro Mechanical Systems (MEMS 2001), Interlaken, Switzerland, January 21–25, pp. 422–426. [13] Petti, C., Herner, S. B., and Walker, A., “Monolithic 3D Integrated Circuits,” Wafer-Level 3D ICs Process Technology, Springer, New York, Chap. 2. [14] Pavlidis, V. F., and Friedman, E. G., 2007, “3-D Topologies for Network-onChips,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 15(10), pp. 1081–1090. [15] Wong, S., El-Gamal, A., Griffin, P., Nishio, Y., Pease, F., and Plummer, J., 2007, “Monolithic 3D Integrated Circuits,” International Symposium on VLSI Technology, Systems and Applications (VLSI-TSA 2007), Hsinchu, Taiwan, April 23–25. [16] Lee, K. F., Gibbons, J. F., and Saraswat, K. C., 1979, “Thin Film MOSFETs Fabricated in Laser-Annealed Polycrystalline Silicon,” Appl. Phys. Lett., 35(2), pp. 173–175. [17] Geis, M. W., Flanders, D. C., Antoniadis, D. A., and Smith, H. I., 1979, “Crystalline Silicon on Insulators by Graphoepitaxy,” IEDM Technical Digest, IEEE, pp. 210–212. [18] Akiyana, S., Ogawa, S., Yoneda, M., Yosii, N., and Terui, Y., 1983, “Multilayer CMOS Device Fabricated on Laser Recrystallized Silicon Islands,” 1983 International Electron Devices Meeting, Washington, DC, December, 5–7, Vol. 29, pp. 352–355. [19] Kunio, T., Oyama, K., Hayashi, Y., and Morimoto, M., 1989, “Three Dimensional ICs, Having Four Stacked Active Device Layers,” International Electron Devices Meeting (IEDM ’89), Washington, DC, December 3–6, pp. 837–890. [20] Topol, A. W., LaTulipe, D. C., Shi, L., Frank, D. J., Bernstein, K., Steen, S. E., Kumar, A., Singco, G. U., Young, A. M., Guanini, K. W., and Ieong, M., 2006, “Three-Dimensional Integrated Circuits,” IBM J. Res. Develop., 56(4/5), pp. 491–506. [21] Lee, S. B., Tam, S-W., Pefkianakis, I., Lu, S., Chang, F., Guo, C., and Reinman, G., Peng, C., Naik, M., Zhang, L., Cong, J., 2009, “A Scalable Micro Wireless JUNE 2014, Vol. 136 / 024001-9 Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/ on 06/04/2014 Terms of Use: http://asme.org/terms [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] Interconnect Structure for CMPs”, ACM 15th Annual International Conference on Mobile Computing and Networking (MobiCom ’09), Beijing, September 20–25, pp. 217–228. Shacham, A., Bergmen, K., and Carloni, L. P., 2008, “Photonic Network-onChip for Future Generations of Chip Multi-Processors,” IEEE Trans. Comput., 57(9), pp. 1246–1260. Davis, W. R., Wilson, J., Mick, S., Xu, J., Hua, H., Mineo, C., Sule, A. M., Steer, M., and Franzon, P. D., 2005, “Demystifying 3D ICs: The Pros and Cons of Going Vertical,” IEEE Des. Test Comput., 22(6), pp. 498–510. Koo, J.-M., Im, S., Jiang, L., and Goodson, K. E., 2005, “Integrated Microchannel Cooling for Three-Dimensional Electronic Circuit Architecture,” ASME J. Heat Transfer, 127(1), pp. 49–58. Lu, J.-Q., Devarajan, S., Zeng, A. Y., Rose, K., and Gutmann, R. J., 2005, “Die-on-Wafer and Wafer-Level Three-Dimensional (3D) Integration of Heterogeneous IC Technologies for RF-Microwave-Millimeter Applications,” MRS Proceedings, 833, p. G6.8. Pande, P. P., Ganguly, A., Belzar, B., Nojeh, A., and Ivanov, A., 2008, “Novel Interconnect Infrastructures for Massive Multicore Chips—An Overview,” IEEE International Symposium on Circuits and Systems (ISCAS 2008), Seattle, WA, May 18–21, pp. 2777–2780. Bakir, M. S., Sekar, D., Thacker, H., and Dang, B., 2008, “3D Heterogeneous Integrated Systems: Liquid Cooling, Power Delivery, and Implementation,” IEEE Custom Integrated Circuits Conference (CICC 2008), San Jose, CA, September 21–24, pp. 663–670. Kandlikar, S. G., 2002, “Fundamental Issues Related to Flow Boiling in Minichannels and Microchannels,” Exp. Therm. Fluid Sci., 26(2–4), pp. 389–407. Sekar, D. C., King, C., Dang, B., Thacker, H., Joseph, P., Bakir, M., and Meindl, J., 2008, “A 3D-IC Technology With Integrated Microchannel Cooling,” IEEE International Interconnect Technology Conference (IITC 2008), Burlingame, CA, June 1–4, pp. 13–15. Dang, B., Bakir, M. S., Sekar, D. C., King, C. R., Jr., and Meindl, J. D., 2010, “Integrated Microfluidic Cooling and Interconnects for 2D and 3D Chips,” IEEE Trans. Adv. Packag., 33(1), pp. 79–87. Alfieri, F., Tiwari, M. K., Zinovik, I., Poulikakos, D., Brunschwiler, T., and Michel, B., 2010, “3D Integrated Water Cooling of a Composite Multilayer Stack of Chips,” ASME J. Heat Transfer, 132(12), p. 121402. Zhang, Y., King, C. R., Zaveri, J., Yoon, J., Sahu, V., Joshi, Y., and Bakir, M. S., 2011, “Coupled Electrical and Thermal 3D IC Centric Microfluidic Heat Sink Design and Technology,” 61st IEEE Electronic Components and Technology Conference (ECTC), Lake Buena Vista, FL, May 31–June 3, pp. 2037–2044. Zhang, Y., Dembla, A., Joshi, Y., and Bakir, M. S., 2012, “3D Stacked Microfluidic Cooling for High-Performance 3D ICs,” 62nd IEEE Electronic Components and Technology Conference (ECTC), San Diego, CA, May 29–June 1, pp. 1644–1650. Zhang, Y., and Bakir, M. S., 2013, “Independent Interlayer Microfluidic Cooling for Heterogeneous 3D IC Applications,” Electron. Lett., 49(6), pp. 404–406. Lau, J. H., and Yue, T. G., 2009, “Thermal Management of 3D IC Integration With TSV (Through Silicon Via),” 59th IEEE Electronic Components and Technology Conference (ECTC 2009), San Diego, CA, May 26–29, pp. 635–640. Ziabari, A., and Shakouri, A., 2012, “Fast Thermal Simulations of Vertically Integrated Circuits (3D ICs) Including Thermal Vias,” 13th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), San Diego, CA, May 30–June 1, pp. 588–596. Shi, B., Srivastava, A., and Bar-Cohen, A., 2012, “Hybrid 3D-IC Cooling System Using Micro-Fluidic Cooling and Thermal TSVs,” IEEE Computer Society Annual Symposium on VLSI (ISVLSI), Amherst, MA, August 19–21, pp. 33–38. Sridhar, A., Vincenzi, A., Ruggiero, M., Brunschwiler, T., and Atienza, D., 2010, “Compact Transient Thermal Modeling for 3D ICs With Liquid Cooling Via Enhanced Heat Transfer Cavity Geometries,” 16th International Workshop on Thermal Investigations of ICs and Systems (THERMINIC), Barcelona, Spain, October, 6–8. Sridhar, A., Vincenzi, A., Ruggiero, M., Brunschwiler, T., and Atienza, D., 2010, “3D-ICE: Fast Compact Transient Thermal Modeling for 3D ICs With Inter-Tier Liquid Cooling,” Proceedings of the International Conference on Computer-Aided Design, pp. 463–470. Feng, Z., and Li, P., 2010, “Fast Thermal Analysis on GPU for 3D-ICs With Integrated Microchannel Cooling,” IEEE/ACM International Conference on Computer-Aided Design (ICCAD), San Jose, CA, November 7–11, pp. 551–555. Wilkerson, P., Raman, A., and Turowski, M., 2004, “Fast, Automated Thermal Simulations of Three-Dimensional Integrated Circuits,” 9th Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITHERM ’04), Las Vegas, NV, June 1–4, pp. 706–713. Koo, J.-M., Jiang, L., Zeng, L., Zhou, P., Banerjee, S. S., Kenny, T. W., Santiago, J. G., and Goodson, K. E., 2001, “Modeling of Two-Phase Microchannel Heat Sinks for VLSI Chips,” 14th IEEE International Conference on Micro Electro Mechanical Systems (MEMS 2001), Interlaken, Switzerland, January 21–25, pp. 422–426. Steinke, M. E., and Kandlikar, S. G., 2004, “An Experimental Investigation of Flow Boiling Characteristics of Water in Parallel Microchannels,” ASME J. Heat Transfer, 126(4), pp. 518–526. Kandlikar, S. G., Kuan, W. K., Willistein, D. A., and Borrelli, J., 2006, “Stabilization of Flow Boiling in Microchannels Using Pressure Drop Elements and Fabricated Nucleation Sites,” ASME J. Heat Transfer, 128(4), pp. 389–396. 024001-10 / Vol. 136, JUNE 2014 [45] Zhang, T., Peles, Y., Wen, J. T., Tong, T., Chang, J.-Y., Prasher, R., and Jensen, M. K., 2010, “Analysis and Active Control of Pressure-Drop Flow Instabilities in Boiling Microchannel Systems,” Int. J. Heat Mass Transfer, 53(11–12), pp. 2347–2360. [46] Mukherjee, A., and Kandlikar, S. G., 2009, “The Effect of Inlet Constriction on Bubble Growth During Flow Boiling in Microchannels,” Int. J. Heat Mass Transfer, 52(21–22), pp. 5204–5212. [47] Szczukiewicz, S., Borhani, N., and Thome, J. R., 2013, “Two-Phase Heat Transfer and High-Speed Visualization of Refrigerant Flows in 100 100 lm2 Silicon Microchannels,” Int. J. Refrig., 36(2), pp. 402–413. [48] DARPA, 2013, “ICECool Applications,” Microsystems Technology Office, Defense Advanced Research Projects Agency, Arlington, VA, Solicitation Number DARPA-BAA-13-21, February 6. [49] Mukherjee, A., and Kandlikar, S. G., 2005, “Numerical Study of the Effect of Inlet Constriction on Bubble Growth During Flow Boiling in Microchannels,” ASME Paper No. ICMM2005-75143. [50] Mukherjee, A., and Kandlikar, S. G., 2009, “The Effect of Inlet Constriction on Bubble Growth During Flow Boiling in Microchannels,” Int. J. Heat Mass Transfer, 52(21–22), pp. 5204–5212. [51] Lu, C. T., and Pan, C., 2009, “A Highly Stable Microchannel Heat Sink for Convective Boiling,” J. Micromech. Microeng., 19(5), p. 055013. [52] Miner, M. J., Phelan, P. E., Odom, B. A., Ortiz, C. A., Prasher, R. S., and Sherbeck, J. A., 2013, “Optimized Expanding Microchannel Geometry for Flow Boiling,” ASME J. Heat Transfer, 135(4), p. 042901. [53] Kandlikar, S. G., Widger, T., Kalani, A., and Mejia, V., 2013, “Enhanced Flow Boiling Over Open Microchannels With Uniform and Tapered Gap Manifolds,” ASME J. Heat Transfer, 135(6), p. 061401. [54] Yao, Z., Lu, Y.-W., and Kandlikar, S. G., 2012, “Fabrication of Nanowires on Orthogonal Surfaces of Microchannels and Their Effect on Pool Boiling,” J. Micromech. Microeng., 22(11), p. 115005. [55] Yang, F., Dai, X., Peles, Y., Cheng, P., Khan, J., and Li, C., 2014, “Flow Boiling Phenomena in a Single Annular Flow Regime in Microchannels (I): Characterization of Flow Boiling Heat Transfer,” Int. J. Heat Mass Transfer, 68, pp. 703–715. [56] Yang, F., Dai, X., Peles, Y., Cheng, P., Khan, J., and Li, C., 2014, “Flow Boiling Phenomena in a Single Annular Flow Regime in Microchannels (II): Reduced Pressure Drop and Enhanced Critical Heat Flux,” Int. J. Heat Mass Transfer, 68, pp. 716–724. [57] Mizunuma, H., Yang, C.-L., and Lu, Y.-C., 2009, “Thermal Modeling for 3D-ICs With Integrated Microchannel Cooling,” IEEE/ACM International Conference on Computer-Aided Design (ICCAD 2009), San Jose, CA, November 2–5, pp. 256–263. [58] Kearney, D., Hilt, T., and Pham, P., 2012, “A Liquid Cooling Solution for Temperature Redistribution in 3D IC Architectures,” Microelectron. J., 43(9), pp. 602–610. [59] Zhang, J., Bloom field, M. O., Lu, J.-Q., Gutmann, R. J., and Cale, T. S., 2005, “Thermal Stresses in 3D IC Inter-Wafer Interconnects,” Microelectron. Eng., 82(3–4), pp. 534–547. [60] Tu, K. N., 2011, “Reliability Challenges in 3D IC Packaging Technology,” Microelectronics Reliability, 51(3), pp. 517–523. [61] Tilley, B. S., 2013, “On Microchannel Shapes in Liquid-Cooled Electronics Applications,” Int. J. Heat Mass Transfer, 62, pp. 163–173. [62] Rubio-Jimenez, C. A., Kandlikar, S. G., and Hernandez-Guerrero, A., 2012, “Numerical Analysis of Novel Micro Pin Fin Heat Sink With Variable Fin Density,” IEEE Trans. Compon., Packag. Manuf., 2(5), pp. 825–833. [63] Rubio-Jimenez, C. A., Kandlikar, S. G., and Hernandez-Guerrero, A., 2012, “Performance of Online and Offset Micro Pin-Fin Heat Sinks With Variable Fin Density,” IEEE Trans. Compon., Packag. Manuf., 3(1), pp. 86–93. [64] Lorenzini-Gutierrez, D., and Kandlikar, S. G., “Numerical Simulation and Design of a Variable Density Flow Passage for Effective Cooling of a 3D IC Chip,” ASME J. Electron. Packag., 136(2), p. XXX. [65] Ramm, P., Klumpp, A., and Weber, J., 2008, “3D Integration Technologies for MEMS/IC Systems,” IEEE Bipolar/BiCMOS Circuits and Technology Meeting (BCTM 2009), Capri, Italy, October 12–14, pp. 138–141. [66] Ramm, P., and Klumpp, A., 2008, “Through-Silicon Via Technologies for Extreme Miniaturized 3D Integrated Wireless Sensor Systems (e-CUBES),” IEEE International Interconnect Technology Conference (IITC 2008), Burlingame, CA, June 1–4, pp. 7–9. [67] Taklo, M. M. V., Lietaer, N., Tofteberg, H. R., Seppanen, T., Prainsack, J., Weber, J., and Ramm, P., 2009, “3D MEMS and IC Integration,” Materials Research Society Proceedings, 1112, pp. 211–220. [68] Gagnard, X., and Mourier, T., 2010, “Through Silicon Via: From the CMOS Imager Sensor Wafer Level Package to the 3D Integration,” Microelectron. Eng., 87(3), pp. 470–476. [69] Xu, G., Huang, Q., Ning, W., Ruan, Z., and Luo, L., 2009, “A Novel MEMS Package With Three-Dimensional Stacked Modules,” International Conference on Electronic Packaging Technology & High Density Packaging (ICEPTHDP’09), Beijing, August 10–13, pp. 77–80. [70] Floyd, B. A., Hung, C.-M., and O, K. K., 2002, “Intra-Chip Wireless Interconnect for Clock Distribution Implemented With Integrated Antennas, Receivers, and Transmitters,” IEEE J. Solid-State Circuits, 37(5), pp. 543–552. [71] Ganguly, A., Chang, K., Deb, S., Pande, P. P., Belzer, B., and Teuscher, C., 2011, “Scalable Hybrid Wireless Network-on-Chip Architectures for Multicore Systems,” IEEE Trans. Comput., 60(10), pp. 1485–1502. [72] Zhao, D., and Wang, Y., 2008, “SD-MAC: Design and Synthesis of a Hardware-Efficient Collision-Free QoS-Aware MAC Protocol for Wireless Network-on-Chip,” IEEE Trans. Comput., 57(9), pp. 1230–1245. Transactions of the ASME Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/ on 06/04/2014 Terms of Use: http://asme.org/terms [73] More, A., and Taskin, B., 2011, “EM and Circuit Co-Simulation of a Reconfigurable Hybrid Wireless NoC on 2D ICs,” Proceedings of the IEEE 29th International Conference on Computer Design (ICCD), Amherst, MA, October 9–12, pp. 19–24. [74] More, A., and Taskin, B., 2010, “Wireless Interconnects for Inter-Tier Communication on 3D ICs,” European Microwave Circuits Conference (EuMC), Paris, September 28–30, pp. 105–108. [75] More, A., and Taskin, B., 2010, “Simulation Based Study of On-Chip Antennas for a Reconfigurable Hybrid 3D Wireless NoC,” IEEE International SOC Conference (SOCC), Las Vegas, NV, September 27–29, pp. 447–452. [76] Lin, T-.Y., and Kandlikar, S. G., 2012, “An Experimental Investigation of Structured Roughness Effect on Heat Transfer During Single-Phase Liquid Flow at Microscale,” ASME J. Heat Transfer, 134(10), p. 101701. [77] Li, Z., Hong, X., Zhou, Q., Bian, J., Yang, H., and Pitchumani, 2006, “Efficient Thermal-Oriented 3D Floor-Planning and Thermal Via Planning for Two- Journal of Electronic Packaging [78] [79] [80] [81] Stacked-Die Integration,” ACM Trans. Des. Autom. Electron. Syst.,” 11(2), pp. 325–345. Kudithipudi, D., Coskun, A., Reda, S., and Qiu, Q., 2012, “Temperature-Aware Computing: Achievements and Remaining Challenges,” IEEE 3rd International Green Computing Conference (IGCC), San Jose, CA, June 4–8, pp. 1–3. Mohanram, S., Brenner, D., and Kudithipudi, D., 2013, “Hierarchical Optimization of TSV Placement With Inter-Tier Liquid Cooling in 3D-IC MPSoCs,” 29th Annual IEEE Semiconductor Thermal Measurement and Management Symposium (SEMI-THERM), San Jose, CA, March 7–12, pp. 17–21. Kalani, A., and Kandlikar, S. G., 2013, “Experimental Investigation of Flow Boiling Performance of Open Microchannels With Uniform and Tapered Manifolds (OMM),” ASME Paper No. HT2013-17441. Garrou, P., 2011, “IBM to Use Water Cooling for Future 3D IC Processors,” Solid State Technol., 54(5), pp. 9. JUNE 2014, Vol. 136 / 024001-11 Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/ on 06/04/2014 Terms of Use: http://asme.org/terms