“Now going forward, we think that training and operating future models will be even more compute-intensive.”

In context

“rs. And there are a few key aspects of this that I want to take some time to go through today. The first is world-class compute infrastructure. I recently shared that, by the end of this year, we'll have about 350,000 H100s, and including other GPUs, that will be around 600,000 H100 equivalents of compute. We're well positioned now because of the lessons that we learned from Reels. We initially underbuilt our GPU clusters for Reels. And when we were going through that, I decided that we should build enough capacity to support both Reels and another Reels-sized AI service that we expected to emerge so we wouldn't be in that situation again. And at the time, the decision was somewhat controversial, and we faced a lot of questions about CapEx spending, but I'm really glad that we did this. Now going forward, we think that training and operating future models will be even more compute-intensive. We don't have a clear expectation for exactly how much this will be yet, but the trend has been that state-of-the-art large language models have been trained on roughly 10x the amount of compute each year. And our training clusters are only part of our overall infrastructure, and the rest, obviously, isn't growing as quickly. But overall, we're playing to win here, and I expect us to continue investing aggressively in this area. In order to build the most advanced clusters, we're also designing novel data centers and designing our own custom silicons specialized for our workloads. The second part of our playbook is open-source software infrastructure. Our long-standing strategy has been to build an open-source general infrastructure while keeping our specific product implementations pro”