Understand the anti hype around LLMs nowadays, but would at least slow down twisting the nail in the coffin, so to speak, before the next generation comes out from OpenAI (scheduled for summer?). This is because of the infamous Chinchilla scaling paper, which basically trained a bunch of models using increasing lvls of compute/data, and found a very reliable performance increase at each step up. It's the equivalent of Moore's law in LLM right now, and as of latest gpt4 + Claude, there has been no sign of saturation in the improvement curve
Understand the anti hype around LLMs nowadays, but would at least slow down twisting the nail in the coffin, so to speak, before the next generation comes out from OpenAI (scheduled for summer?). This is because of the infamous Chinchilla scaling paper, which basically trained a bunch of models using increasing lvls of compute/data, and found a very reliable performance increase at each step up. It's the equivalent of Moore's law in LLM right now, and as of latest gpt4 + Claude, there has been no sign of saturation in the improvement curve
Razib, thanks for an interesting discussion. The hype of most innovative research or endeavors is more prevalent than reality.