My Corner of the Web 
In coding my EZ-blog application for creating this website, I recently added a feature to use Generative AI to suggest categories based on the content of each article. To do that I send a prompt and the content to the OpenAI API and receive a response. I recently ran across a shocking error for a very commonly used parameter in large language models: Temperature. This is the error that I received from the OpenAI API:
“Unsupported parameter: ‘temperature’ is not supported with this model.”
For those not familiar with this parameter, temperature is a parameter that controls the “creativity” or randomness of the text generated by the GPT models. A higher temperature (e.g., 0.7) results in more diverse and creative output, while a lower temperature (e.g., 0.2) makes the output more deterministic and focused. In fact, in the early days when ChatGPT was integrated into the bing search engine they used to allow you to set how "creative" you wanted the chatbot to be using three buttons as depicted in the figure below.
While I understand that this change is related to reasoning models, it is a bad move for OpenAI to do this for two reasons:
- Lack of respect for compatibility. You don't break the API because you have some models that use a parameter and some that don't. You make the model that does not need that parameter just ignore it. Think about how a browser handled the rapidly changing tags over the years: If the browser encountered a tag it did not know, it just safely ignored it. That is what OpenAI should have done. To implement a change that breaks legacy code is a big deal but obviously OpenAI doesn't respect its developers enough to consider that.
- Lack of respect for transparency. Large Language Models already have a trust problem due to them hallucinating. Compounding hallucinations with a lack of transparency only increases the notion of LLMs as mysterious black boxes that us mere mortals don't have the right to know how they work. We need more control over how these neural networks function not less! In fact, we also need to know exactly when an LLM is confident in its answer. LLM's should share with us the probabilities of the tokens it is selecting as part of the answer. In fact, they should color code responses that it is highly confident in and those that it is highly uncertain. This lack of transparency is just another clear indicator that Generative AI is hitting a wall.
The solution going forward is for OpenAI to increase transparency, respect compatibility and give us greater control over these models. Anything less and generative AI will be just another fad that burns out as fast as it caught fire!