Genericness is overwhelmingly a product of RLHF rather than an innate property of LLMs. A lot of manual fine-tuning has gone into ChatGPT and Gemini to make them capable of churning out homework and marketing blogs without ever saying anything offensive.
If you make requests to the Sonnet 3.5 or DeepSeek-R1 APIs and turn up the temperature a little bit, you will get radically more interesting outputs.
Isn’t that still pulling from the same distribution with a larger standard deviation? I think the problem here is that it only covers a small part of the search space. I think the problem here is that generators are not using novel distributions. They’re still sampling from the same population (existing written works).
If you make requests to the Sonnet 3.5 or DeepSeek-R1 APIs and turn up the temperature a little bit, you will get radically more interesting outputs.