Prompt Engineering: Learning to ride Gen AI beast ;)
Horse Riding. Driving. Prompting.
Did you guess and now, chuckling? Brevity has its own rewards.
Mastering the first two things in each generation meant that you moved away from the herd and moved up. If you couple that with first mover advantage, you gain a lot of competitive edge and perhaps financial rewards, directly or indirectly. Can we say the same about prompting or skills to use generative AI?
Prompting is the art and science of your interactions with generative AI to get you your desired outcomes. Prompts are short pieces of text that help language models understand what is expected of them.
Prompt engineering is now the field of discipline that combines logic, coding and is a bit of art but it does the following to help the LLM model.
As a starter, the following are some of the key pointers for writing efficient prompts when we engage LLMs (What is LLM?):
- Crafting clear and specific instructions
- Creating and modifying prompts to improve the performance of LLMs
- Structuring text that can be clearly understood and acted upon by LLMs
- Select appropriate phrases, examples, and sentence structures to derive the best result from LLMs.
The above sounds very common sense; why should “engineering” follow what is described above about “prompts”? This was my first reaction.
Isn’t “engineering” meant to denote something like a real expert in something — Civil/Mechanical engineering? At first glance, prompting expertise, called prompt engineering, sounded like it was imminent that there would be something called “follow-up” engineers/managers. They do nothing else apart from following up with others, and they get paid for that. They have learned the art of following up, nudging, persuading, irritating, or doing whatever it takes to follow up with much perseverance. If it became a highly-paid job, I would sit and think about the brave new world. But follow-up engineers, aka managers, do exist and get paid, so my imagination is not great.
But, as it happens with things, you see something new when you scratch the surface. Then, you can wonder about either one of these two lines from Shakespeare— “All that glitters is not gold” or “A rose by any other name would smell as sweet”. Both state the same things: what is within really matters, not a name, not the appearance, but the true value lies beneath the surface.
The key to understanding “prompting engineering”, ignoring my light-hearted dig, is the driving analogy. There are drivers like me, pretty average, and then there are drivers who compete in F1 racing. Both kinds might have started by taking a learner license, but ‘prospective’ F1 drivers eventually became something else entirely later on, wouldn’t you agree? It might have started with curiosity, attention to detail and hard work.
In the same breadth, I would say most people will chat with ChatGPT, and few others will do extraordinary things with ChatGPT, Google Bard, or something like that. I may not be an F1 driver, but I tried to learn. Doing the same with Gen AI.
ChatGPT training model used 285,000 processing cores & 10,000 graphics cards. The power consumption for training is estimated to be up to 10 gigawatt-hour. That is the annual equivalent of 1000 US household electricity.
When you are driving a ChatGPT, and you are truly in control — think about it. A F1 driver’s reaction time (0.2–0.3 sec) vs normal driver’s reaction (0.75 sec — 1.5 sec) is around 3–5 times better. An expert Gen AI user, a prompt engineer can become 10–20 times better than an average user. It is a leapfrog, it is the next level.
Remember, it is the little things that matter. Have you examined how TV reporters carefully listen to the President’s address in a foreign land? Did the President say — friend or partner, or if is it Chinese premier — Iron friendship or something like that? Stock markets, companies and other folks change strategy if the word chosen is friend vs partner vs collaborators.
The world of diplomacy pays attention to words, Gen AI does too, and so must we :)
Enough of my humour. Let us get down to business. But, I must warn you, I am no expert; just like Gen AI, I read a lot and reproduce it as my content ;)
The objective of learning prompt engineering is that you want a certain quality of output in a certain format from the LLM, and you are trying to learn the best way to prompt or ask the LLM to give you that. You might also add some efficiency as the goal of your learning. As in, how quickly can you make the LLM what you want and it generally comes back quickly.
Deep-Dive:
Once you have a clear understanding of what you want from the LLM, you need to be conscious of the following:
Prompt Format: The prompt can be a single sentence, a paragraph, or multiple sentences followed one after the others.
The prompt can be explicit or open-ended, giving the LLM the freedom to be creative about the creative content.
What are the different types of prompting, you may ask? (Details will follow in the lower section)
- Zero-shot prompting
- One shot prompting
- Few shots prompting
- Chain of thought prompting
- Self-consistency with COT
- Tree of Thoughts prompting
Instruction Clarity: We discussed it earlier, but the instructions must be very clear to the LLM. Example — If you want to summarize a huge page, please state the same. If you want the summary to be in bullet points or within 100 words or something like that. Please state the same. Clear instructions will get you your desired output quickly.
If you intend to engage LLM for three hours a day, by being very clear, you can get your work done in two hours.
Examples and Context: LLMs have been trained in extensive data sets; you want information in a given context. It is not a bad idea to ask the LLMs something and provide examples and context. The quality of your response will be more apt for your situation.
Temperature: This word might sound out of place in this context, but the temperature is a hyperparameter that controls the level of randomness in the model’s output. It is typically set during the generation process.
A higher temperature (e.g., 0.8) makes the output more diverse and creative but also can introduce more randomness and less coherence.
A lower temperature (e.g., 0.2) makes the output more focussed and deterministic, often producing more conservative and controlled responses.
Max Token Limits: This is the maximum number of words that can be given as input to an LLM. Knowing the boundary condition is important for many tasks, such as trying to understand the summary of a lengthy report.
If the whole report cannot be sent to LLM, or worse, if the whole report is accepted by LLM but only some part of it is read, and then a summary is produced, the summary would be erroneous.
Understanding Types of Prompting and Their Use Cases:
The types of prompting below might be intuitively known, but they are worth discussing and understanding a bit more.
Zero shot prompting
This is where you expect the LLM to produce an answer. You know that the LLM has been trained with that data, and you are asking for that.
Example: Instructing ChatGPT to “Please write a leave application.”
Example: Instructing ChatGPT to ask, “How many months do we have in a year?”
One shot prompting
You give the LLM one prompt or example and then ask it to do the task.
Example: Instructing ChatGPT to “Please write a leave application for me as a school student.”
Here, as a school student, is a prompt; it is one bit of instruction which brings you closer to a better result.
Few shot prompting
This is an example in which you provide multiple examples or cues to the LLM to produce an output.
Example: Write a paragraph about a beautiful beach and use words such as gentle waves, soft sands, and a light breeze.
Example: Instructing ChatGPT to “Please write a leave application for me as a school student explaining how I fell ill on 24th Oct and could not make it to the class. I also have a doctor’s prescription and medical reports to submit along with my application.”
Better Example: In this example, I am asking ChatGPT to write dialogues between two characters by giving a few examples.
Chain of thought prompting:
Introduced in Wei et al. (2022), chain-of-thought (CoT) prompting enables complex reasoning capabilities through intermediate reasoning steps. It is a type of few-shot prompting that uses a series of reasoning steps to guide a model’s reasoning process. CoT prompting can improve a model’s ability to perform reasoning tasks, such as arithmetic, commonsense, and symbolic reasoning.
Chain-of-thought (CoT) prompting
How CoT prompting works
- CoT prompting breaks down multi-step problems into intermediate steps.
- The model is provided with a few-shot exemplar that outlines the reasoning process.
- The model is expected to follow a similar chain of thought when answering the prompt.
Benefits of CoT prompting
- CoT prompting can improve a model’s ability to perform reasoning tasks.
- CoT prompting can be useful for complex computational problems where traditional methods fail to compute.
- CoT prompting can be combined with few-shot prompting for better results on more complex tasks.
In my regular work, I have not found a situation to train ChatGPT on logical problems where explaining the chain of thought will make ChatGPT perform better. Since I did not have the use case of complex logical things being solved multiple times with slight variation, automation via ChatGPT might have helped.
Another example can be using ChatGPT to create multiple choice questions about grammar by showing how three choices in a question have been drafted to mislead a student by exposing his/her lack of fundamental understanding of grammar and why the fourth answer, although confusing or awkward, is still correct. Something like GMAT 750+ questions.
Self Consistency with CoT (CoT-SC) & Tree of Thoughts (ToT)
This image from the Princeton NLP GitHub page and the research around it tells the story.
Both CoT-SC & ToT are guided prompts that allow you to use few-shot prompting and sort of engagement with LLM to come to an output. These techniques are very recent research by Yao et al. (2023) & Long(2023)
The authors, Yao and his coauthors, of the first paper used ToT to extract the better result from GPT-4 in the case of creative writing, Games of 24, and mini-crosswords.
The author, Jieyi Long, used ToT to extract better results from LLM when solving a sudoku puzzle.
Awareness of ToT and its intuitive use can help you nudge LLM to write better for you. I have experimented myself and seen that. Although when I was doing it, I thought it should be called “recursive prompts”, I later realized that the academic name of it is “Tree of Thoughts” prompts.
