Microsoft's OpenAI API doesn't just give you basic answers to questions. It can also throw some artistic flair in its responses.
And you can tell it to do that by setting the temperature.
Yeah, the temperature.
That's the name of the property that you specify in your payload.
And in this guide, I'll show you how to do the whole thing in Java.
Keep in mind, this article is the second in a series. I strongly recommend you go through the first article because the code here will build on the code from the previous guide.
You can also check out the completed code on GitHub.
Temperature?
That's Microsoft's name. It ain't mine.
The "temperature" property in your JSON payload will look like this:
{
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "system",
"content": "You are a standup comedian like Richard Pryor."
},
{
"role": "user",
"content": "Explain what life is like in prison."
}
],
"temperature": 0.6
}You can see it there at the bottom. It's at the same level as "model" and "messages."
But what the heck is it?
It's a number between 0 and 1.
Yeah, I also don't know why they called a number between 0 and 1 a "temperature." I just work here.
In any case, that number defaults to 0. So if you don't set it, that's the value.
How Does It Work?
So how exactly does temperature work with OpenAI?
When OpenAI creates a response, it uses a model that predicts the text that follows the text that's already part of the response.
The temperature property determines the confidence level when outputting that new text.
A lower temperature means the model will take fewer risks. A temperature of 0 means that in many cases you'll get the exact same output every time.
But a higher temperature means the model will take more risks. That's when you'll see different responses to the same prompt.
So if you don't want to see the same response over and over again to a specific prompt, increase the temperature.
If you're looking for deterministic answers, set a lower temperature.
What Does This Have to Do With Creativity?
So what does any of this have to do with creativity?
Simply put: if you want more creativity and originality in your responses, raise the temperature.
That's it. That's the whole thing in a nutshell.
Let's take a look at an example.
Suppose I send the OpenAI API the following prompt: "My favorite vehicle to drive is a ". And I set the temperature to 0.
Here's what the request looks like:
{
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "My favorite vehicle to drive is a "
}
],
"temperature": 0
}But now let me try that same request with a temperature of 1.
And here's the response:
{
"id": "chatcmpl-7ijPD2bosKOJZk296xhANM9m8PfLw",
"object": "chat.completion",
"created": 1690895327,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "sports car."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 25,
"completion_tokens": 3,
"total_tokens": 28
}
}Take a look at the "content" property to see the response. In this case, it's "sports car."
That makes sense because most people probably would prefer to drive a sports car as their vehicle of choice. Remember: OpenAI completes responses based on predictive models.
Now, let me try that same request with the temperature set to 1. Here's what that request looks like:
{
"model": "gpt-3.5-turbo-0613",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "My favorite vehicle to drive is a "
}
],
"temperature": 1
}And here's the response:
{
"id": "chatcmpl-7ijG5VViqeBQss4Pyxm8ZxB8J6CzH",
"object": "chat.completion",
"created": 1690894761,
"model": "gpt-3.5-turbo",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "car. Cars offer the convenience and freedom to travel long distances, explore new places, and easily navigate through cities. With a car, you have the flexibility to go wherever you want, whenever you want. Whether it's for commuting to work, running errands, or going on road trips, cars provide comfort and reliability. Additionally, cars come in various sizes, styles, and models, allowing you to choose one that suits your personal preferences and needs."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 25,
"completion_tokens": 91,
"total_tokens": 116
}
}As you can see, the answer is no long "sports car." It's just "car."
Beyond that, OpenAI started monologuing. It didn't do that when the temperature was 0.
So you can see a bit more creativity and ad-libbing with a higher temperature.
Under the Covers
So how does this predictive model work?
Well, OpenAI breaks text down not into words but into tokens.
While some words are also single tokens, that's not always the case.
For example, let's take Mary Poppins' favorite word: "supercalifragilisticexpialidocious." That breaks down into the following tokens:
- super
- cal
- if
- rag
- il
- ist
- ice
- xp
- ial
- id
- ocious
11 tokens.
Yeah. That's an extreme example. But it highlights the larger point: there isn't necessarily a 1:1 mapping between words and tokens.
And that's important because the OpenAI algo predicts the next token (and not the next word) to appear in its output.
So you can expect to see different variations of the same word (or synonymous words) in the output when the temperature is high.
Once OpenAI has inserted a new token, then it will use its predictive model to fetch the next token. And so on. And so on.
After all of that is said and done, you've got what appears to be creative output.
What About Java?
Oh, yeah. I almost forgot.
We're supposed to be doing this in Java, aren't we?
So let's do that now.
You might recall from the first guide that you used a ChatCompletionRequest object to create the payload programmatically.
You may also recall that you used a helper class to create that object. Now it's time to update that class to support temperature.
Here's what the new static method looks like:
public static ChatCompletionRequest getRequest(final List<ChatMessage> messages, final Double temperature) {
final ChatCompletionRequest chatCompletionRequest = ChatCompletionRequest
.builder()
.model(MODEL)
.messages(messages)
.temperature(temperature)
.build();
return chatCompletionRequest;
}The method signature is the same as before except it now supports a Double value as the temperature.
So now let's create a new runnable Java class that uses that method:
public class TemperatureCheck {
private static final double TEMPERATURE = 0.0d;
public static void main(String[] args) {
final OpenAiService service = OpenAiServiceHelper.getOpenAiService();
final List<ChatMessage> messages = getMessages();
final ChatCompletionRequest chatCompletionRequest
= ChatCompletionRequestHelper.getRequest(messages, TEMPERATURE);
service.createChatCompletion(chatCompletionRequest)
.getChoices()
.forEach(System.out::println);
}
private static List<ChatMessage> getMessages() {
final List<ChatMessage> messages = new ArrayList<>();
final ChatMessage systemMessage = new ChatMessage(ChatMessageRole.SYSTEM.value(),
"You are a helpful assistant.");
final ChatMessage userMessage = new ChatMessage(ChatMessageRole.USER.value(),
"Animals are ");
messages.add(systemMessage);
messages.add(userMessage);
return messages;
}
}You can see from the static double at the top that the temperature is 0 for this request.
The only other line you need to focus on is the user prompt in the getMessages() method. That's: "Animals are ".
Everything else is pretty much the same as from the earlier guide.
The point here is to let OpenAI describe animals at it sees fit.
Now run that code. When I run it, the content I get in response looks like this:
creatures that come in a wide variety of shapes, sizes, and species. They play an important role in our ecosystem and have unique adaptations that allow them to survive in different environments.
From the majestic elephants of Africa to the tiny hummingbirds of South America, animals have captivated humans for centuries. They have the ability to communicate, form social bonds, and exhibit complex behaviors. Some animals are highly intelligent, such as dolphins and primates, while others have incredible physical abilities, like the speed of a cheetah or the strength of a gorilla.
Animals also provide us with many benefits. They can be our companions as pets, provide us with food and clothing, and even assist us in various tasks, such as guide dogs for the visually impaired or therapy animals for those in need of emotional support.
However, it is important to remember that animals deserve our respect and protection. Many species are facing threats such as habitat loss, pollution, and poaching. Conservation efforts are crucial to ensure the survival of these incredible creatures and maintain the balance of our planet.
Overall, animals are a source of wonder and inspiration. They remind us of the beauty and diversity of the natural world and teach us valuable lessons about compassion, resilience, and coexistence.
I ran it a few times and got pretty much that same response. Very little variation.
You'll probably see somthing similar.
Now change that temperature at the top to 1.0d. Run it again.
Here's what I got:
a diverse group of living organisms that can be found all over the world. They come in various shapes, sizes, and forms, ranging from microscopic organisms to large, majestic creatures.
Animals have different body structures adapted to their specific environment and lifestyle. They can be classified into various categories, such as mammals, birds, reptiles, amphibians, and fish.
One common characteristic of animals is their ability to move, which allows them to find food, escape danger, and reproduce. They have specialized organs and systems that enable them to perform essential functions, such as digestion, respiration, circulation, and reproduction.
Animals play a crucial role in ecosystems by maintaining the balance of nature. They contribute to pollination, seed dispersal, nutrient cycling, and maintaining biodiversity. Additionally, numerous animals provide companionship and support to humans as pets and service animals.
However, animals also face various challenges, including habitat loss, pollution, climate change, and hunting. It is important for us to protect and conserve animal species to ensure their survival and the health of our planet.
That's quite a bit different.
Try running again (keeping an eye on your tokens). I got this:
living beings that are part of the natural world. They can be found in a wide variety of environments, including forests, oceans, deserts, and even in human-made habitats such as farms and zoos.
Animals are classified into different groups based on characteristics such as their body structure, habitat, and how they obtain their food. Some common groups of animals are mammals, birds, reptiles, amphibians, and fish.
Animals play important roles in ecosystems. They help to maintain the balance of nature by participating in various ecological processes such as pollination, seed dispersal, and decomposition. They also serve as a source of food for other animals, including humans.
Humans have domesticated various animal species for their companionship, labor, and as a source of food and materials. Some commonly domesticated animals include dogs, cats, horses, cows, and chickens.
It is important to treat animals with kindness and respect, as they have their own needs, emotions, and rights. Many animals are vulnerable to threats such as habitat loss, pollution, climate change, and poaching. Conservation efforts are necessary to protect and preserve animal species and their habitats for future generations.
That one still includes the editorial content about climate change, but it also throws in a bit about the classification system for good measure.
And it's got stuff about domesticated animals that I didn't see the first time.
Go ahead and run a few more tests. Again, though, keep an eye on those tokens.
Wrapping It Up
There you go. You now know how to use Java with the OpenAI API to put some creativity in your responses.
Time for you to fiddle around. Use different prompts with different temperatures.
And be sure to check out the code when you get a moment.
Have fun!
Photo by cottonbro studio: https://www.pexels.com/photo/a-woman-looking-afar-5473955/