Why SoftBank paid me 17 yen to skip the air conditioning
Patrick McKenzie (patio11) reads his 2022 Bits about Money essay on the infrastructure behind American charitable giving, prompted by the wave of AI-generated wealth expected to flow into philanthropy. He explains donor-advised funds, a chimera that is legally a charity but works for donors like a financial institution that can time travel: someone with a windfall takes the tax deduction now and recommends grants for years afterward. Along the way he covers why the IRS-required donation receipt is simple enough for a seven-year-old to produce while the industry still spends billions on record-keeping, why DAFs still mail so many paper checks, and how Anthropic’s reported uncapped three-to-one donation match could push tens of billions of dollars through this machinery.
Timestamps:
(00:00) Intro
(02:58) Electricity is ubiquitous, and other polite fictions
(06:35) Electricity supplied and demanded are necessarily equal
(07:57) Demand for electricity fluctuates over the day and over the year
(09:53) The power generation mix
(13:34) Demand response and virtual power plants
(19:50) A quick aside about meter reading
(21:05) Why this sudden interest in power?
(26:22) Wrap
Transcript
This transcript will be annotated in Patrick’s usual style by the end of this week — be sure to check back.
Hideho, everyone. My name is Patrick McKenzie, better known as patio11 on the internets. With all the talk of data centers becoming the dominant marginal use case for electricity, I thought it would be useful to read a classic essay of mine on power economics. It is a good text in parallel with previous Complex Systems episodes on grid interconnection, solar economics, and similar, which I'll link in the show notes.
As always, I'll intersperse some commentary. We've had a number of very clear illustrations of the concepts from the essay in the intervening years, and regrettably have yet more examples of geopolitical events hitting the energy markets. "Markets in power," originally published in Bits About Money on September 30th, 2022.
I recently received a push notification from SoftBank, a PowerPoint design consultancy which runs diverse sidelines, including a power business here in Japan. SoftBank anticipates grid stress due to severe weather and… complicated factors.
Although this was obliquely phrased, the big two were poor decisionmaking on behalf of Russia (invading Ukraine) and Europe generally (being fundamentally unserious about power policy), leading to ripples that would affect the price of inputs to many participants in the Japanese grid. This would have the knock-on effect of potentially bringing down the grid when salarymen get home from work and turn on their air conditioners.
This was extremely not a worry without reason, by the way. The price of liquefied natural gas, which we'll talk about in a minute, increased by about 15x in 2022.
"Of course, if Russia starts a war during a hot year then there will be a risk of power outages in Tokyo; that's just science" is perhaps not the most straightforward conclusion to draw, so it's useful to go into a bit of background into how electricity markets work. This will, I promise, help explain why SoftBank wanted me to install a demand response app targeting specifically my 5 to 8 PM air conditioner use.
A few disclaimers to get out of the way: the physics of electricity generation are the same everywhere but the market structures which govern pricing it are not, sometimes due to path-dependent infrastructural reasons and sometimes due to regulatory regimes. The energy markets are a very deep topic; you'll probably want to get your week-to-week updates from someone like Doomberg (an anonymous group of energy consultants, whose work often feels like smuggling details about hidden power structures — ba dum bum — into the public view) rather than from me.
I'm going to repeatedly use two common acronyms that it is possible some readers might not understand. For their benefit, "capital expenditures" (CapEx) means long-term investments into things which will generally keep producing over a multi-year expected lifetime, and "operating expenses" (OpEx) means the day-to-day costs of running a business, like payroll or gas for a taxi. Some businesses are CapEx-intensive but OpEx-light, some are vice versa, and (relevantly) the energy business includes a lot which require both huge CapEx and high ongoing OpEx.
Electricity is ubiquitous, and other polite fictions
The first thing to understand about the energy markets is that everything written about them is a lie. Or, more to the point, everything is a simplified model, with some discontinuity between the model and behavior in the real world. This is because energy markets are almost as complex as all commerce combined and, unlike counterparties in commerce, physics tolerates infinite detail and doesn't care about your feelings.
One useful mental model: there is a Grid™ and it is everywhere, like the Internet is. Suppliers of electricity and consumers of electricity are hooked up to the grid. You push or pull your electrons as you would like to and someone, perhaps the power company, sends you a bill every month.
There is no Grid™, there are multiple grids, there are ways of transmitting power over distance that are not grids, and physics actually doesn't let you ignore distance at will.
The topology of your local energy market likely includes some power sources which do not hook up to a grid; they're frequently built next to industrial users outside cities because transmission lines are costly (in CapEx and constantly lost electricity — and, I'll note, in the lengthy process to get permitting and other approvals to hook up a new transmission line) and because industrial users have better ability to plan than society does. Power planning turns out to be an extremely deep topic, and if you have unified insight into future needs at a minute-by-minute level and do not have to worry about your neighbors, you can get tremendous savings on power.
This colocation of generation and demand leads to an interesting phenomenon called "stranded" generation, when the industrial user colocated with power generation ceases operation. The power asset is often still functional (the CapEx, heavy in most generation businesses, has been paid and if you put in OpEx you will get electricity out) but useless without a local-ish consumer of a large block of electricity.
This is much remarked upon by, of all people, Bitcoin enthusiasts, because it turns out that Bitcoin miners are almost an ideal consumer for electricity at fire-sale prices, in that you can site them just about anywhere, they have by-design absolutely insatiable demand for electricity at all margins including exactly the amount you have stranded, and capital is available to construct them in hundred million dollar increments.
(Bitcoin miners are in the business of bidding electronics depreciation and electricity consumption against each other to win a tournament in generating random numbers with mystical properties. From this tournament arises a mediocre transaction processing network and a speculative asset. You get more of the speculative asset if you're the global highest bidder in electricity used in generating random numbers and, hence, the huge overlap in professional Bitcoin miners and people with deep expertise in quirks of the power generation market. For more on this topic from someone who thinks more highly of it, see generally the work of Nic Carter.)
I'll say as an aside: this actually turned out to be extremely important in the build-out of data centers to support AI and other use cases over the last couple of years. For example, CoreWeave originally started renting out, I believe, GPUs to Ethereum miners, with the idea that Ethereum was also going to take off and become a large business in the vein of Bitcoin, as some sort of global, worldwide computer. It turns out that is actually not that valuable. But if you have a data center which is built to host GPUs for the purpose of Ethereum mining, what you have is a powered box, and you often have a multi-year commitment from the power grid to continue providing power for the box. And so you can stuff that box full of GPUs doing anything else, like, for example, H100s doing inference or training runs for AI companies. And that turns out to be a much more valuable global computer than Ethereum was.
Electricity supplied and demanded are necessarily equal
Electricity is extremely expensive to store relative to its value. This is a limitation of present battery technology but is closely related to the physics of the universe we live in. Most ways we have of generating power in the first place are only economical because they are creative cheats around physical reality, for example by burning millions of years of work by underpaid bacteria to generate a few seconds of heat and light.
As a consequence of this, the supermajority of all electricity is consumed approximately contemporaneously with being generated. Electricity moves through the less complicated bits of power infrastructure at about 0.6c, or about 110,000 miles per second. The physical processes required to generate electricity frequently require hours of time to spin up or spin down. These are fundamentally incompatible timescales and yet we somehow make it work anyway.
I will say: see the episode with Casey Handmer for the increasing advances in battery production over the last couple of years. As battery technology becomes lower cost and higher capacity, and as we have the capability to ramp production, it is increasingly the case that some of the representations made about solar not being base load power are incorrect, for at least some localities that are naturally blessed by having lots of sunshine.
Demand for electricity fluctuates over the day and over the year
Speaking generally, most readers live somewhere where there is a grid operator who coordinates energy supply by various producers to meet instantaneous power needs. Those literally change on a millisecond to millisecond basis as light switches get flicked, computers wake from sleep, and aluminum smelters ramp up during work cycles. The grid operators have demand prediction down to a science (and extremely consequential statistics exercise). This is enormously positive for you since tiny mispredictions generate brownouts.
TEPCO — the Tokyo Electric Power Company — maintains the following graph for Tokyo, current as of… the rest of today. (I love the Internet.)
Unfortunately, I can't easily narrate a graph to you, but what I see when I pull this up today is that they expect to be at roughly 90 to 91% of peak load in October, which is not exactly surprising, and 90 to 91% is thoroughly, thoroughly doable. It is more complicated when we get to 93% or 97% of peak load during summer days, because then small fluctuations in heat, and thus air conditioner usage, can cause the grid to not be able to supply everyone's demand.
On it, you can see extremely common behavior: energy demand peaks in the late afternoon (early evening, for workdays). The marginal loads include everything from restaurant kitchens to washing machines but, on weekdays, are mostly residential air conditioning or heating.
If you draw a line around 20,000 MW, you'll see that Tokyo has a certain level of work per second required to sustain civilization. That is called "base load." The 18,000 MW gap between that and peak capacity is dynamic load. (I'll note that you will generally see power generation figures quoted in MWh or kWh in English sources and, fascinatingly to me, the choice to quote in increments of 10 MW here is because it was easiest for the localization engineering team going from units of 10,000 kW, which is convenient to express in the Japanese language.)
The power markets have substantial complexity and human effort in making sure a diverse selection of sources with different physical and economic characteristics can instantly clear markets at all levels between these.
The power generation mix
You might naturally expect that, if you've spent billions of dollars on CapEx for a power plant, you would want it running continuously or as close to it as possible. This is desirable only for certain types of power plants. It turns out that the spin up and spin down times of various plants are different, and the marginal cost of their operation (fuel, mostly) and (increasingly) marginal carbon impact determine who gets lit up when to dynamically manage supply.
Speaking of lighting up: some forms of power generation are less amenable to being dynamically managed by humans. Despite millennia of research, while we have become extremely precise at predicting when the sun comes out, causing it to rise earlier or set later remains elusive. In addition to the daily cycle, which changes with the seasons, day-to-day and minute-to-minute changes in local weather greatly affect the amount of energy thrown off by solar installations.
But returning to base load power: nuclear generators operate virtually continuously (feckless political decisions notwithstanding). This is both due to the extremely low marginal fuel cost of nuclear power and the physical properties of, perhaps surprisingly, water. You can turn off a nuclear reaction in on the order of seconds but the steam produced by your reactor, which turns the turbines to generate power, will continue being hot (due to its own energy and remaining energy in the system) for a few hours.
Other base load sources include coal-fired plants. Here again, inclusion in the base load layer is a result of a combination of physics and economics. Physically, you have some choice as to whether you burn more coal in the near future but relatively little choice as to whether coal continues to burn from the near past. Coal is generally speaking cheap per kWh and available in sustained quantity in the supply chain; if your local grid counts on it, it very literally counts on it.
Geothermal and hydroelectric are also base load where they are available; while hydroelectric plants have some degree of sensitivity to the weather, both of these sources are broadly predictable and relatively constant over the clock and calendar.
The layer after base load is intermediate sources. These include solar and wind, which are unavailable to cover base load, and also generally some fossil fuels which are amenable to being started and stopped relatively quickly but have somewhat poorer margin characteristics. Natural gas commonly ends up in this layer. (The price of liquefied natural gas (LNG) in Japan is sensitive to geopolitics, critical for the Japanese economy, and is currently the subject of much commentary that you can read elsewhere.)
By the way, as an aside, it is interesting to distinguish which power and fuel markets are local and which are global. So the price of natural gas in the United States is largely locally determined because the United States is, thanks to the magic of fracking, a net exporter of natural gas. The price of natural gas in Japan is heavily dependent on global politics, because Japan is an importer of natural gas. And then the price of gasoline in the United States is sensitive to geopolitics, because we are not self-sufficient on refined petroleum products. Back to the essay.
Finally, there are "peaker" plants, which spend most of their time off and economically justify themselves by keeping society running for a few hours every day and, in some cases, only a few weeks out of every year. Peak generation is almost exclusively fossil fuels, generally the ones with the worst margin characteristics but the best minute-to-minute ability to change generation. Gasoline is one example; although gas electricity generation looks a bit different than an internal combustion engine, they roughly share the property of stopping on a proverbial dime when you shut off the flow of liquid into the controlled explosion.
Demand response and virtual power plants
So why am I getting push notifications from SoftBank to strongly consider decreasing my energy usage between 5 PM and 8 PM today? It's part of a strategy called demand response.
Because adding new MW to the grid costs a lot of CapEx, peaks which exceed rated capacity are extremely bad news. Those peaks include many individual decisions which could, in principle, be shifted. Demand response sets up economic, communications, and infrastructure layers to encourage some consumers of electricity to either shift their loads off peak or to curtail (cease using electricity) during peaks.
This has led to an innovation called "virtual power plants." Someone, probably in financial engineering, realized that there is an economic equivalency between a peaker plant, with lots of CapEx and emissions associated with it, and a well-balanced combination of software and contracts.
While there are many ways of implementing them, the typical way is to sign up a roster of customers, purchase a commitment to ongoing power from other operators to service those customers, then sell the grid an option to buy back its own power at peak times only.
Get that? Maybe it's clearer in numbers: "My customers need 40 MW which you can count on, and I will commit to purchasing that level of service, but I can make do with only 30 MW given five minutes of notice. This is economically equivalent to having an invisible 10 MW power plant powered by clean-burning unicorn flatulence. You should include it in your supply management strategy. Now, pay me for my entrepreneurial energies in finding you 10 MW of power available at the worst possible times."
How does a virtual power plant actually find those 10 MW? By contracting with customers to suggest they move power usage and, if necessary at the margin, curtailing their use (economically, contractually, or in some cases literally by having software installed on-site to remotely shut down their machines).
In SoftBank's case, with regards to this residential user, demand response is a bit more optional; they give about 24–48 hours' notice of when they expect peak demand (based on the forecast from the grid operator, based on the forecast of the weather among other factors). Software calculates what typical usage for a family like mine would be over that ~2 hour window and offers me an incentive payment for each kWh that I use less than my predicted use. There is no penalty for going over, and I'm only paid if I (trivially, one-tap) indicate my intention to respond to the call for more power. SoftBank pulls the stats off of my smart meter, calculates an incentive payment the following day, and immediately pushes it to me. (Most electricity companies would do a database update and credit my account against future usage but, because SoftBank invested in a payments company which offers zero-marginal-cost sends, they'll happily instantly transfer me 17 yen.)
As an aside: this model of having a virtual power plant has been key to the operations of some of the Bitcoin miners, and data center usage, over the last couple of years. Riot, which is a Bitcoin and similar miner, got into a bit of public hot water during the grid stress in Texas in 2023.
So, due to a confluence of ERCOT being a very quixotic institution — Texas has its own grid, because everything is bigger in Texas — and due to the wave of very hot weather they were experiencing, ERCOT was looking for power in all the places. What Riot had done was it had purchased a power forward from ERCOT. What purchasing a power forward means is: we agree in advance to the price of power, and I want to be buying that power all the time. Riot went back to ERCOT and said, "Okay, this is something we both planned for. In the event that the grid is stressed — and the grid is stressed right now — you can buy power back from me at the spot price."
And so ERCOT paid Riot several tens of millions of dollars to buy power. My impression is that people were pretty unhappy with that, because they had the impression that the Bitcoin miners were causing the grid stress by being heavy users of electricity. And I think, as someone who is not very sympathetic to Bitcoin miners, this is literally the opposite of the case. They're buying power that would be more or less otherwise unused during most [of the year, and the grid] will say, "Okay, we have a dump load — a large user of power that is capable of turning off basically on a dime if we needed to."
This is something that technologists might be familiar with from certain forms of data center use. For example, AWS and other cloud providers also have spot markets for instances. And in periods of time where everyone wants to really be doing a lot of computing as their applications scale up and down, the spot price goes up. You can buy much cheaper instances if you're willing to commit to constantly using them at some level over the course of the entire year. And in some cases, you can get some form of payment or credit or economic incentive for turning off instances or usage of other cloud resources at times where the spot market really, really wants them.
We also see this directly from AI companies, via their APIs. So, just with regards to their public pricing — which is, of course, not the best offer that they will cut you — there is a certain price for the, you know, new hotness in terms of state-of-the-art models, and then there's a different price for their smaller models. But they will cut you a 50% discount off the top if you use their batch API. Why is that? It's because the batch API allows them to decide, sometime in the next 24 hours, when they are not maxing out on their inference usage of GPUs, to run your queries and generate your tokens back. And so that is effectively offering yourself up as tribute — as a demand response user — versus everyone else that is running their inference queries in real time.
Similarly, I strongly suspect that the labs internally have ways to transition up their amount of training and down their amount of training, to use more of their committed GPU capacity for inference usage in periods where inference usage tends to spike, such as immediately after the release of a new model.
A quick aside about meter reading
Smart meters are one of the most economically important hardware technologies that you've never thought of. Thirty years ago, your monthly power consumption was likely read by a (human) "meter reader" coming to your building, who would read your meter's current number, subtract last month's number, and eventually set the power company up to bill you for the difference. This occasions what industry calls "truck rolls," which is the activity where an expensive human gets in an expensive piece of movable capital for a few hours. There is a known, high marginal cost for truck rolls and ongoing CapEx for the fixed infrastructure which makes a committed number of them possible.
A smart meter is simply a dumb meter plus a microchip plus a communications channel back to the mothership. They cut down on truck rolls by more than 99%. Upgrading the fleet to smart readers cost an incredible amount of money and paid for itself very, very quickly. (Some readers of this column still have legacy meters, and it would be an interesting dive into switching costs, regulation, and labor economics into who does and why.)
We still had to balance supply and demand prior to smart meters, and indeed prior to computers, and that this worked at all is a triumph of civilization. But smart meters are a much, much better way to do things than the status quo ante.
Why this sudden interest in power?
Power is one of the pervasive bits of infrastructure that civilization sits atop of and largely doesn't think about. This year seems like a particularly poor year to not have good intuitions about how it actually works.
We'll return to this subject later, including a fascinating angle on it that only a payments geek could enjoy: how subscription billing for utilities works and the financial consequences of it.
I'll say, in the intervening couple of years, power has become quite the political and economic issue, largely downstream of the data center build-out to support AI, which is likely going to be akin to the Industrial Revolution, if not more, in having economic impacts on the way we live.
Data centers are about 5 to 10% of the total power usage in the United States. However, given the multi-trillion-dollar infrastructure build-out to support what we assume are future AI inference loads, as AI works itself into every supply chain in every industry, data centers are likely to be a huge portion of the marginal increase in electricity usage over the next couple of years.
This has caused, you know, many political and economic issues associated with them. And some of this is exercising things that you've just heard from the essay. For example, there is a political economy question on how long the queues are for new data centers in much of the country.
So one thing you can offer for the political economy of that to work out — which also works for the economy economy of it working out — is to have some sort of demand response or curtailment agreement with your data center. This has been discussed in academic papers; I'll link an example in the show notes.
But essentially, while a particular lab or a particular industrial user of AI might have several different use cases internally — they might have training, they might have inference, and they might have some other internal usages — they support this over a geographically distributed fleet of data centers. For training runs, the latency to the end user doesn't really matter all that much. For inference, the latency to the end user matters much more. You would prefer inference being conducted on behalf of the people in Chicago to happen somewhere in the Midwest versus it happening in, I don't know, Singapore, because the, you know, silky-smooth, only-a-couple-hundred-milliseconds difference between typing something in and getting a response is not possible with that hop over the Pacific.
Singapore would also be a terrible place to site data centers for any number of reasons, but be that as it may… Sorry. Singapore is a terrible place to site data centers for AI inf— ooh, well, big can of worms, isn't it? There exist many data centers in Singapore for AI inference. That is not primarily because Singapore is an excellent place, but because it allows you to smuggle inference into China. But that's another topic entirely. Anyway, you would not choose to source compute for Americans in Singapore, given your druthers.
So, returning to the topic: if you have geographically dispersed data centers, you might say, "Okay, for the ones which are closest to the inference load at the moment, I want them running inference loads, but I will turn off some of their H100s. I will turn off some of their servers. I will turn off some of their racks. Potentially, I will turn off a data center entirely, in the case where there is a peak power demand." And that turns me into — like Riot, or like the other examples — this sort of virtual power plant. So the thing that you manage to power with, you know, unicorn flatulence is AI inference.
It's an open question whether this truly solves the political economy, partly because people have very bad intuitions for the economics of power usage, and partly because, as you expect these data centers to be by default running 24/7, there is some way in which they probably do push up marginal prices, because they are base load and not solely dump load. They're not solely designed to turn off.
Why do you expect AI data centers to run 24/7, where you don't expect that necessarily from crypto miners? It's because the expense associated with doing AI inference is mostly depreciation on the chips and not electricity. The depreciation is somewhere on the order of five to ten times more than the electricity costs. For Bitcoin mining, the depreciation on the chips is only about 1.5x, 2x, or 3x the electricity cost. And so this makes the Bitcoin miners much more sensitive to the marginal electricity cost, and so willing to play excellent games to do demand response and to find stranded energy assets, where AI is basically, "Okay, we want to hook up anywhere we can get sufficient energy to hook up. And after having achieved that hookup, ideally we would be up 24/7."
But hopefully this explanation of data centers and power, in connection with some other Complex Systems episodes that I can link you to, helps to inform your thinking about these issues, because they are certainly not going away in the next couple of years. Anyway, thank you very much for listening to Complex Systems this week, and we will see you next week.
