Getting more value out of ChatGPT and other LLMs with Prompt Templates

LLMs cannot read your mind!

Have you ever worked with ChatGPT and had difficulty getting it to do what you wanted it to do? Large Language Models offer a lot of value to end users, but you need to know how to interact with them in order to get the most value.

Many users when first interacting with Large Language Models initially see them as a novelty, something that can understand questions in plain language and reply to you in plain language. This change to the human/machine interface is huge by itself, but its important to understand the limitations of LLMs and how with some minor changes to your approach, you can derive a great deal more value from them.

An initial interaction with an LLM may look more like a search engine query, or looking up information in a dictionary or encyclopedia: What is the capital of Germany? Who wrote the Declaration of Independence? When did Hawaii become a state?

While this is useful, this hardly scratches the surface of the value LLMs afford us. We can leverage LLMs to derive meaning from content (What is the sentiment of this article?), summarize content, give us idea on a topic, even interact with us on a subject to get to a final answer.

These LLMs are incredibly powerful, however it is important to understand that LLMs are not magic, and despite being a subset of Artificial Intelligence, LLMs are not independently intelligent, and in reality are not “intelligent” at all. An LLM works by analyzing some input words, comparing them to a whole bunch of training data and effectively “predicting” what the next words are that apply to that input. This is important to understand because LLMs have been accused of “Hallucinating”, basically making up answers to fit some narrative. What’s actually happening is the LLM made some bad predictions based on insufficient input (Garbage in, Garbage out). That could be due to its training data being out of date (You asked about current events, even though the training cutoff was months/years ago), lack of space in its context window (The amount of “memory” an LLM has), or due to an overly broad input prompt. The latter is where Prompt Templates come into play.

What are Prompt Templates?

Prompt Templates are mechanisms to format your prompts in a memorable fashion that set the LLM up to predict the most appropriate response to your question(s).

There are many variations of prompt templates that you can use, we will discuss two different ones below: RTF and CREATE.

How do we use prompt templates

The RTF and CREATE templates are similar with RTF being fairly simple and CREATE and expansion on the same concepts.

RTF : Role, Task, Format

With RTF the goal is to define three distinct parts of your prompt, starting with the Role. By Role, we mean the position or part you want the LLM to play when generating your prompt. For example, if you are writing a post for a cooking blog, you might inform the LLM that it is a master chef. If you are generating some code, you might tell the LLM that it is a senior software engineer.

Why do we do this? When you define a role, you put are informing the LLM of a context for it to use to limit the scope of the words it will use to generate a response. In the absence of a role context, the LLM has to “guess” when words are appropriate but if you narrow that context, it can better predict the words that make sense for your particular prompt.

To Illustrate, let’s take two prompts:

“Create a recipe for me that includes tomatoes and rice” 

vs

“You are a master chef, create a recipe for me that contains tomatoes and rice”.  

Though both will produce a recipe that includes tomatoes and rice, the latter will produce a recipe that is refined and more appropriate in a restaurant setting where a master chef would work vs a generic setting for the first.

The next part is the Task. The Task is a job or process that the LLM does as part of its role. This further refines the context in which the LLM will use to generate the response. For example our master chef may be someone who use’s Mediterranean ingredients to come up with unique recipes, or a software developer who uses python to write scripts for scraping content from websites.

The last bit is the Format. The Format is how you want the output of the LLMs response to be presented. You may want to have the output presented in markdown if you are producing some documentation, or you may want the output to be in JSON format if it’s intended to be used by some function.

Putting this all together here are some examples of RTF.

You are a world class chef who specializes in Mediterranean quinine.  
Come up with a recipe that includes tomatoes and rice.  Your response should be in markdown format


You are a senior software engineer who specializes in using python.  
Write a script that takes a url as input and scrapes the content and returns the c
ontent as part of a json response.  Your response should be in JSON”

CREATE : Character, Request, Examples, Adjustments, Type, Extras/Evaluate

The CREATE prompt template is an extension of the RTF template that adds a few more dimensions to give the LLM additional opportunities to generate content that matches your expected output. While the mnemonic is useful for you to remember the parts involved, they are not necessarily used in a specific order when creating your prompt.

Character: This is the same as role, what character does the LLM play in generating the response.

Request: This is similar to the task. This is what you are wanting the LLM to produce for you.

Examples: In contrast to RTF, which contains no examples (called zero-shot), with CREATE we include one or more examples (one-shot or few-shot) in order to teach the LLM some outputs that are considered correct given a specific input

Adjustments: these are additional details that may be given in our initial prompt or subsequent prompts that direct the LLM to further refine the output based on a specific need. These may include instructions for the LLM to NOT include specific verbiage, or to alter the tone of the output for an audience

Type: Same as format above, what is the expected output type. This closely ties with the examples if a specific output is expected, like JSON or CSV.

Extra/Evaluate: This can include extra details (the more the better) or could also be secondary prompts to provide reinforcement or correction to a previous output that did not meet your expectations.

Example:

You are a sentiment analysis expert who takes a sentence as input and outputs json 
that describes the input as positive, negative or neutral.  
You only respond with JSON output, do not include any other verbiage in your response besides the JSON output.  
For example:  Input: “I was absolutely thrilled with the exceptional service I received 
at the new Italian restaurant downtown”.  Output: { “sentiment”: “positive” }.  
Input : “I was extremely disappointed with the customer service I received at the electronics store today”

In this example the Character is a sentiment analysis expert. The Request is to evaluate some input and determine its sentiment. The Type is JSON. We have several Adjustments where we indicate the LLM should only include the json output and no other verbiage. We gave it one Example in this case.

Let’s assume the response was something like “Sure here is the sentiment analysis: { “sentiment”: “negative” }”

An example Extra/Evaluate would be “That response is not in JSON format and includes additional verbiage, please fix”

———

Hopefully you found this article helpful in empowering you to get more out of LLMs, they really are powerful tools, but using with them is different from other tools and we will need to evolve how we interact not only with them, but with computers in general in the very near future.