Nvidia’s big AI event: What Wall Street wants to hear
In training, cost has been an afterthought, but in inference, where the money will be made, cost is key, and computing needs are mixed. When you type a question into your favorite AI chatbot, it turns it into tokens representing words, parts of words, and punctuation. It processes all of these tokens at once, a…

