• 0 Posts
  • 37 Comments
Joined 1 year ago
cake
Cake day: July 9th, 2023

help-circle








  • This graph actually shows a little more about what’s happening with the randomness or “temperature” of the LLM.
    It’s actually predicting the probability of every word (token) it knows of coming next, all at once.
    The temperature then says how random it should be when picking from that list of probable next words. A temperature of 0 means it always picks the most likely next word, which in this case ends up being 42.
    As the temperature increases, it gets more random (but you can see it still isn’t a perfect random distribution with a higher temperature value)









  • Something to add, since I colocate my own hardware in a datacenter: Just the cost of operating a server is non-trivial too. I pay $120/month for just 2U, 1gbps networking, and power usage (600W peak usage), providing my own hardware. Things get a lot cheaper if you can rent a whole rack space though. It’s absolutely been worth it though, because to rent a similar specced server, it could easily be $1k/month.