“But overall, we're playing to win here, and I expect us to continue investing aggressively in this area.”

In context

“. And when we were going through that, I decided that we should build enough capacity to support both Reels and another Reels-sized AI service that we expected to emerge so we wouldn't be in that situation again. And at the time, the decision was somewhat controversial, and we faced a lot of questions about CapEx spending, but I'm really glad that we did this. Now going forward, we think that training and operating future models will be even more compute-intensive. We don't have a clear expectation for exactly how much this will be yet, but the trend has been that state-of-the-art large language models have been trained on roughly 10x the amount of compute each year. And our training clusters are only part of our overall infrastructure, and the rest, obviously, isn't growing as quickly. But overall, we're playing to win here, and I expect us to continue investing aggressively in this area. In order to build the most advanced clusters, we're also designing novel data centers and designing our own custom silicons specialized for our workloads. The second part of our playbook is open-source software infrastructure. Our long-standing strategy has been to build an open-source general infrastructure while keeping our specific product implementations proprietary. In the case of AI, the general infrastructure includes our Llama models, including Llama 3, which is training now, and it's looking great so far, as well as industry standard tools like PyTorch that we've developed. And this approach to open source has unlocked a lot of innovation across the industry, and it's something that we believe in deeply. And I know that some people have questions about how we benefit from open”