Mitigating the carbon cost of AI

Artificial intelligence (AI) operates most efficiently when it is commoditizing intelligence and decision making. We are starting to see proof of the scientific and business benefits that come from this streamlining and processing of data, and the use of AI modelling is playing out across a wide and growing spectrum of market sectors. However, as with any form of progress, there is a cost. Training just AI models can lead to significant carbon emissions. Studies have shown that state of the art models can result in hundreds of tons of emissions, and researchers and companies alike are individually training the models for their own purposes. As a result, we now have the power of a new technology that invites an exponential growth in emissions without awareness and action.

Higher compute power means higher energy consumption

When you look across the financial service and healthcare industries, we see examples of how machine learning applications are revolutionizing products, services and research. Within the financial service industry, machine learning is changing quantitative investing from a set of algorithms based on historical data to a set of models that capture and actively react to the fluctuating changes in the market. With these new tools, focus shifts from past to future and thereby the potential to dramatically reduce the friction required to achieve the next advantage on the market. For healthcare, AI modelling is enabling better disease diagnosis and prevention, all the more urgent in the current atmosphere of a global pandemic. The amount of compute power required for today’s applications when applied at scale is orders of magnitude greater than previous generations. The greater the network depth and the greater the data quantities for input, the greater the compute complexity, all of which requires high-performance computational power and longer training times.

Prioritise efficiency as a criterion for AI models

The enormous computing power needed for machine learning and deep learning applications equates to markedly high energy consumption. The conversation around the drive for accuracy – the current standard-bearer of AI research success – is beginning to be tempered by a growing concern for computation or energy efficiency. A study from the University of Massachusetts, Amherst, raised a flag last summer when it concluded that large AI models can emit more than 626,000 pounds of carbon dioxide equivalent—nearly five times the lifetime emissions of the average American car, including the manufacture of the car itself. The carbon cost of AI becomes even more substantial when you add the energy required to keep equipment cool and prevent overheating – at least 40% of all energy consumed in a conventional data centre goes towards cooling.

There are, however, steps companies can take to minimize their carbon footprint while still accessing cutting-edge supercomputing to drive their innovations. MIT President, L. Rafael Reif, reminds us that “technologies embody the values of those who make them, and the policies we build around them can profoundly shape their impact. Whether the outcome is inclusive or exclusive, fair or laissez-faire, is therefore up to all of us.” The process begins at the start of the AI project by thinking clearly about the data that is important to your business. Understanding the data at your disposal ensures higher quality results and can help identify the technology that is truly needed to support the solution. If AI is the right way forward, you can proceed more efficiently and not waste precious time and resources building and applying unnecessary models. Businesses need to approach new and innovative AI projects with the same sound practices that have been applied across other technology implementations.

Choose less carbon-intensive data centres 

A practical way of balancing the potential of AI with the need for greater sustainability is within the data centre. Many conventional corporate data centres are not well equipped to deal with the large compute required to train neural networks that are the foundation of machine learning applications. In the case of medical or financial trading applications, the training never stops and a traditional data centre that is not designed for extreme conditions will not be able to provide dependable performance. How can this be managed? Through modified data centre design.

One best practice is to place server racks closer together to maximize the bandwidth capacity between servers while minimizing the overall cost of the deployment. Traditional data centre designs have relied on widely spaced racks to help reduce the cooling stress, but this practice sacrifices the closely coupled designs required for at-scale machine learning applications. Data centre developers also want to minimize the size of their data halls to reduce the overall cost of construction and to improve the return on investment for the infrastructure. But for today’s specialised GPU boxes, this can present a challenge. For example, air-cooled systems that are positioned too close together can result in cooling deficiencies as the extreme airflow requirements of high capacity servers, such as the NVIDIA DGX-2 and the newly announced NVIDIA DGX A100, can actually blow against each other and create backpressure on the cooling fans within the equipment. As a result, data centres that are to be designed with high-intensity AI applications in mind must balance the financial pressures of reducing the footprint of the data hall with the requirements to provide sufficient space for proper cooling conditions. In addition to cooling considerations, data centres must be structurally capable to handle very heavy equipment. The best data centres today allow fully populated, very heavy AI computing cabinets to be rolled from the loading dock to their final computing location all on the same level and all on reinforced concrete slabs. Data centres that are ready for extreme AI computing must be properly designed and operated.

Location of the data centre is also paramount. A data centre campus with free cooling and powered by inexpensive and renewable sources can significantly reduce operational expenses. The majority of the equipment involved in training machine learning models does not need to be located near the end-user for performance or accessibility. As such, racks can be comfortably housed in data centres that are serious about their power sources. For example, at Verne Global’s Icelandic data centre campus, our managed colocation and cloud solutions make it simple to connect the most powerful computing resources directly to a 100% renewably powered grid that is the cleanest on the planet, all while benefiting from ambient free cooling thanks to the favourable conditions outside of our secure data centre campuses.

One exciting change that we have started to see in the conversations that we have with our customers is that a growing number of businesses are beginning to give their leadership teams financial incentives to meet sustainability targets. These companies are putting their money to work and using executive compensation as a lever for the greater good. No business will be issuing executive incentive plans that neglect the bottom-line health of the company. Therefore, savvy and sustainable businesses must search out solutions that not only protect our ecosystems but also save on the expense side of the balance sheet.

Finally, no matter how good our intentions are, if we don’t hold ourselves accountable for making these changes, it will be for nought. In the age of climate action, the public is no longer asking for claims, it is demanding action – and accountability. True accountability establishes a solid foundation for growth and enables future generations to carry our visions beyond. As we look ahead to opportunities AI can offer, we must tackle the sustainability head-on.