Explore retrieval augmented generation (RAG) methods to investigate AI's environmental footprint using over 100 papers and public reports.
AI systems consume vast amounts of energy and water, but ask exactly how much and the answers vary wildly, scattered across academic papers, government reports, and companies' own disclosures. In this challenge, you'll build a retrieval-augmented generation (RAG) system that answers over 500 questions about AI's environmental impact from a corpus of over 100 papers and public reports. Every answer needs a citation and supporting evidence, and when the corpus can't support an answer, your system must say so instead of guessing. The goal: turn scattered evidence into transparent, actionable answers for researchers, engineers, and policymakers.
US data centers already use more than 4% of the nation's electricity, and their carbon intensity is 48% higher than the national average — they are built where power is cheap, not where it is clean. In 2023 they consumed 17.4 billion gallons of water, a figure expected to double by 2028. The pollution that comes with all this does real harm, and the harm falls hardest on low-income communities.
Good information is harder to come by than it should be. Companies rarely disclose the energy or water costs of the models people actually use, so many widely repeated numbers are estimates, extrapolations, or marketing. And efficiency improvements have not stopped the growth: as compute gets cheaper, we use more of it.
A good answer rarely comes from a single paper. Estimates get revised, methods improve, and sources disagree — sometimes because one is wrong, more often because they measured different things at different times. Answering well means finding the right sources, reasoning across them, and being honest about what nobody has measured yet. That is what this challenge scores.
Build a RAG system over the corpus — arXiv preprints, peer-reviewed work, LBNL, IEA and GAO reporting, corporate sustainability filings, and a regional cluster on Wisconsin and the Great Lakes. For each question your system returns:
answer— a concise natural-language responseanswer_value— the normalized number, term, or True/Falseref_id— the document(s) it came from, keyed to metadata.csvsupporting_materials— the verbatim quote, table, or figureexplanation— the reasoning connecting the evidence to the answerWhen the corpus cannot answer, it says so — answer_value set to is_blank — rather than guessing.
Submissions are scored with the WattBot Score, a weighted accuracy in [0, 1]. The scoring components include:
answer_value (Weight: 0.75): Numeric answers within ±0.1% relative tolerance. Derived answers may accept any value inside a tolerance band. Categorical values must match exactly after normalization. Unanswerable questions must contain is_blank.ref_id (Weight: 0.20): Citation F1 against the ground-truth set — missing a supporting document costs recall, citing one that is not in the set costs precision.is_NA (Weight: 0.05): For unanswerable questions, answer_value, ref_id, ref_url, and supporting_materials must all be is_blank.The public leaderboard reports the mean over a subset of the test set. Final standings come from a private split that is never published.
While maxing out the WattBot leaderboard is a respectable goal, teams are invited to build a chatbot application around their RAG methods as a second bonus track (not scored). This application should be documented in your writeup with screenshots, a short video, and a repo link. Criteria for a good chatbot include: