Understanding Power Consumption in AI Queries

Can you believe it has already been a year since #ChatGPT launched? Since its emergence, #artificialintelligence has captured global dialogue, from its potential #workforce impact to implications for education and art. But we’re missing a critical angle: AI’s #carbonfootprint. Examining ChatGPT’s usage can help us gain insight into its environmental impact. As of February 2024, the platform’s 100 million+ weekly active users are each posing an average of 10 queries… That’s ONE BILLION queries per week, each generating 4.32g of CO2. By plugging these estimations into an emissions calculator, I found that EVERY WEEK the platform is producing emissions roughly equivalent to 10,800 roundtrip flights between San Franciso and New York City (enough to melt 523,000 square feet of Arctic sea ice). Scientists have already warned the Arctic could be free of sea ice in summer as soon as the 2030s. And something tells me they weren’t factoring ChatGPT and other energy-demanding AI models into those projections. Further, this is based on estimated *current* ChatGPT use, which will only grow as society gets accustomed to the tool and as AI becomes more a part of everyday life. Some analyses indicate that by 2027, ChatGPT’s electricity consumption could rival that of entire nations like Sweden, Argentina, or the Netherlands. The platform is taking precautions, however, such as using Microsoft’s carbon-neutral #Azure cloud system and working to develop more #energyefficient chips—so it could certainly be worse. But, it could also be better. So let’s hold OpenAI accountable to mitigate their damage before it gets out of control. Join me in letting them know the public is watching their environmental impact and that they must responsibly manage the platform’s rapidly growing carbon footprint. (Pictured: Microsoft GPU server network to power OpenAI's supercomputer language model. Image courtesy of Microsoft/Bloomberg).

Energy Impact of AI I have spoken about the energy impact of software and AI in particular, and as we wind down the weekend, I came across a thought-provoking stat, which will change the way I interact on the web: A single ChatGPT query can emit up to 4.32** grams of CO₂ In comparison, a Google search emits only about 0.2 grams of CO₂ per query. To put this into perspective - Here’s what the equivalent CO2 usage is by query: 15 queries = watching one hour of videos 16 queries = boiling one kettle 20-50 queries is the equivalent of consuming 500ml of water. 139 queries = one load of laundry washed at 86 degrees Fahrenheit, then dried on a clothesline 92,593 queries = a round-trip flight from San Francisco to Seattle The computational intensity, data center operations, and energy demands of large language models like ChatGPT contribute to its significantly higher carbon footprint. As we embrace the power of AI, it's crucial to consider the environmental impact of these technologies. This disparity highlights the need for more energy-efficient AI models and the importance of using renewable energy sources to mitigate the carbon footprint of our digital interactions. So next time let's search Google before we start chatting with these LLM models! ** Source: https://lnkd.in/gCTAHBdN

Research continues to show the high environmental cost of GenAI tool development and deployment. We’ve created this classroom guide to help educators get a better understanding and engage their students in thoughtful discussions on the potential impacts of GenAI on the planet. Researchers estimate that creating ChatGPT used 1,287 megawatt hours of electricity and produced the carbon emissions equivalent of 123 gas-powered vehicles driven for one year. It's development created substantial heat that required a significant amount of water to cool down those data centers – and for every 5-50 prompts it requires about 16oz of water. Generating an image can be especially energy-intensive, similar to fully charging your smartphone. Creating 1,000 images with Stable Diffusion is responsible for as much CO2 as driving 4.1 miles in a gas-powered car. Some researchers estimate the carbon footprint of an AI prompt to be 4-5 times that of a normal search query. And the impact of escalating use predicted by 2027 could mean AI servers will use as much electricity as a small country. Check out the carousel for more including discussion questions and further reading. Or download a PDF version for your classroom here: https://lnkd.in/eaCtnN3n AI for Education #aiforeducation #aieducation #AI #GenAI #ChatGPT #environment #sustainability

Why Untether AI made my Top 13 list [https://lnkd.in/g-UBT3R5] The explosive growth in AI over the past few years has triggered an avalanche of data and one of the principal challenges to the efficiency of AI workloads is data transfer from memory to compute, which accounts for more than 90% of power consumption. And this inefficiency continues to grow as the amount of computing resources required for training generative AI models, for example, doubles every 3 months. Increasing the performance of AI systems typically means adding more hardware which has a waterfall effect of consuming an astonishing amount of energy. Though it’s difficult to know the exact environmental impact, researchers compare it to the CO2 emissions released by transportation. In a 2019 study, researchers at Cornell estimated creating the popular GPT-3 model consumed 1,287-megawatt hours of electricity generating 552 tons of carbon dioxide, the equivalent of operating 123 gas-powered vehicles for a year. Untether co-founders Raymond Chik, martin snelgrove, and Darrick Wiebe originally conceived an AI processor that would be significantly more efficient at moving data for inference applications, thereby reducing power consumption and improving latency. Their invention of at-memory compute is a groundbreaking architecture that moves the compute element adjacent to memory, resulting in a 6x reduction in power consumption AND unrivaled compute density AND lower latency. Untether’s technology can be leveraged for applications ranging from the datacenter, to industrial robots and autonomous vehicles. CEO Arun Iyengar said “Untether AI was founded to tackle the AI energy consumption problem while increasing the AI inference throughput crucial for important applications such as AVs. Our focus is on making energy-efficient, high-performance chips and creating a more sustainable future for AI” The Untether team is addressing AI energy use and their technology could not have come at a more critical time. At the AI Breakthrough Awards in July, Managing Director James Johnson highlighted the severity of the issue, saying that, “By the end of the decade, the AI portion of the chip industry will be about 50 percent and that’s an energy consumption problem we must solve.” I am a proponent of AI and believe the proliferation and democritization of AI will yield many societal benefits, like safer more efficient autonomous vehicles that are affordable to more people. However, I believe we also need to mitigate tradeoffs because we can’t have future generations bearing the burden of unmanageable energy demand. Untether.ai has made my Top 13 list because 1) They’re furthering compute performance while having a holistic view of efficiency 2) They’re solving a specific problem with broad market benefits 3) They’re helping to expand the AI industry’s consciousness about energy enabling democratization to not result in planetary suffocation. #semiconductors #semiconductorindustry #ai