Creative Curating and Refinement with GPT-3

In this post I'm going to explain two GPT strategies that I call "Curating", and "Refinement".

For this blog post, I'm using ChatGPT Plus.

Here on Adventure in Large Language Models, we believe in seeing this astounding technology for what it is, a natural language constraint satisfaction programmable auto-completion engine. That's pretty technical and long winded. Let's see what ChatGPT can do to make it simpler:

restate "natural language constraint satisfaction programmable auto-completion engine" in fewer simpler words 

A program that helps you type faster by suggesting words and phrases, based on the rules of language.

Great. That doesn't do the technology justuce, but I like the idea of typing faster. Don't we all?

Let's do some brainstorming.

generate 5 episode ideas for a TV comedy like "The Office" about baristas

Even though this prompt is short, we have given it a ton to work with, first because of the detail and nuance behind (comedy, "The Office" and baristas), but even more so because the word "episode" implies a typical three-act-structure, with a (setup, conflict, and resolution). Let's see what it came up with:

1. "Latte Art Showdown": In this episode, the baristas at the coffee shop are pitted against each other in a latte art competition. Everyone is eager to show off their skills and win the grand prize, but things quickly get out of hand when one barista accuses another of cheating. Chaos ensues as the baristas compete to prove who is the best latte artist.

2. "The New Boss": When the coffee shop gets a new manager, the baristas are skeptical. The new boss seems overly strict and demanding, and the baristas don't like the changes he's making to the shop. But when the boss turns out to be an undercover boss from the coffee company's headquarters, the baristas scramble to make a good impression and keep their jobs.

3. "The Coffee Shop Ghost": Strange things are happening at the coffee shop, and the baristas are convinced that it's haunted. Weird noises, unexplained spills, and mysterious shadows have everyone on edge. But when the baristas band together to investigate, they discover that the "ghost" is actually just a mischievous regular customer who has been causing all the trouble.

4. "The Barista Olympics": In this episode, the coffee shop hosts a Barista Olympics, with events like speed brewing, latte art, and milk foaming. The baristas are excited to compete, but tensions rise when one barista accuses another of cheating. The competition becomes fierce as the baristas race to win the gold medal and bragging rights as the best barista in town.

5. '"The Coffee Shop Heist": When the coffee shop is robbed, the baristas are determined to catch the thief. Using their skills as coffee experts, they come up with a plan to lure the robber back to the shop and catch him in the act. But things don't go quite as planned, and the baristas find themselves in a sticky situation.

Wow. #2 and #3 are seriously funny. However, we ended up with two suggestions that both use "cheating" as the humor, which is kind of dull, and the last one about the robbery is also falling flat for me. 

Before we move on, I want to talk specifically about the double-occurance of cheating as a plotline. Remember that the responses are generated one-word-at-a-time, which means the occurance of cheating in the #1 option increased the chances it would occur in the subsequent responses, because it became an example in the prompt! 

Since we think the cheating plotlines are boring and negative, this is the opposite of what we want. Now, we could alter the prompt with a constaint like ("don't use cheating as a plotline"), but I find we can usually increase response quality more by giving positive examples than by listing out more and more negative constraints.

Let's curate our favorites and generate some more. In this case, I'm also going to keep the part of #1 that I like -- and truncate it, allowing the engine to generate a better conflict than cheating.

generate 5 episode ideas for a TV comedy like "The Office" about baristas

1. "The New Boss": When the coffee shop gets a new manager, the baristas are skeptical. The new boss seems overly strict and demanding, and the baristas don't like the changes he's making to the shop. But when the boss turns out to be an undercover boss from the coffee company's headquarters, the baristas scramble to make a good impression and keep their jobs.

2. "The Coffee Shop Ghost": Strange things are happening at the coffee shop, and the baristas are convinced that it's haunted. Weird noises, unexplained spills, and mysterious shadows have everyone on edge. But when the baristas band together to investigate, they discover that the "ghost" is actually just a mischievous regular customer who has been causing all the trouble.

3. "Latte Art Showdown": In this episode, the baristas at the coffee shop are pitted against each other in a latte art competition. Everyone is eager to show off their skills and win the grand prize, but things quickly get out of hand when

Notice the way I intentionally left #3 hanging as a partially completed sentence -- ready for the advanced mathematics of GPT auto-completion to weave into a new ending!

From this prompt we got:

the competition becomes more intense than anyone anticipated. With sabotage and creative tricks, the baristas will do whatever it takes to come out on top, even if it means sacrificing their friendship.

4. "The Customer is Always Right...Except When They're Not": The baristas at the coffee shop are used to dealing with difficult customers, but when one particularly entitled and rude customer starts causing problems, they decide to take matters into their own hands. With each barista trying to outdo the other in how they deal with the problem customer, chaos ensues as the situation spirals out of control.

5. "Coffee and Romance": Love is in the air at the coffee shop, as the baristas find themselves swept up in a whirlwind of romantic entanglements. With crushes, secret admirers, and unexpected relationships, the coffee shop becomes a hotbed of gossip and drama. But when a Valentine's Day promotion goes awry, the baristas learn that sometimes, love and business don't mix.

There are a few key observations we can take away from this. First, there are gladly no more cheating plotlines. That wasn't guaranteed, but in this case it worked out. We also took a setup that we liked, the "Latte Art Showdown", and generated a non-cheating conflict and resolution, by truncating and letting ChatGPT choose a new way out of the maze we put it in. 

Now lets look at Refinement. 

Plotline #4 has promise, but it also has several of generic concepts without any detail. A difficult customer is "causing problems"? What problems? And baristas trying to "outdo each other" sounds like we need a bunch of things they could be doing to outdo!

Let's put those cloud-servers and NVidia GPUs to work!

for option 4 above, generate ten ideas of problems a customer could cause

1. Demanding a custom drink that's not on the menu and then complaining when it doesn't taste the way they wanted.

2. Refusing to wear a mask or follow social distancing protocols, putting other customers and employees at risk.

3. Trying to haggle over prices or demanding a discount for no reason.

4. Taking up a table for hours on end and refusing to leave, even when other customers are waiting for a spot to sit.

5. Complaining about the music or temperature in the coffee shop, despite the fact that it's within normal range.

6. Insisting on being served first or receiving priority service, even when there are other customers waiting in line.

7. Leaving a huge mess at their table and refusing to clean up after themselves.

8. Criticizing the barista's skills or questioning their knowledge of coffee.

9. Making inappropriate or offensive comments to other customers or staff.

10. Trying to scam or steal from the coffee shop, such as by claiming they never received their order or trying to take merchandise without paying for it.

These suggestions are the kind of thing that blows me away about Large Language Model technology. By constructing two small curated prompts, and then asking for refinement on those ideas, we've allowed it to draw on an incredible base of knowledge, and construct scenerios so believeable I feel like it's already watched the TV show we are trying to make! 

The technology's ability to impress with creative generative constraint satisfaction is bounded only by the prompt size limits and our imagination. 

Plotline #4 begs for two instances of refinement, one to come up with problems the customer could create (which GPT slayed, BTW), and another to figure out how the baristas could outdo eachother trying to put a stop to it. In order to get GPT to dig really deep, we now want to combine the two together.

for episode 4, generate six escelating ways the baristas could deal with the problem customer

1. Attempt to reason with the customer and politely explain why their demands are not feasible.

2. Offer a small token or gesture of goodwill, such as a free pastry or coffee refill, in an attempt to appease the customer.

3. Get the manager involved and try to find a compromise that satisfies the customer without disrupting the workflow of the coffee shop.

4. Politely but firmly ask the customer to leave if their behavior is disruptive or abusive to other customers or staff.

5. Refuse to serve the customer altogether if their behavior is consistently problematic and negatively impacting the business.

6. Take legal action against the customer if their behavior is extreme or criminal, such as harassing staff or vandalizing the coffee shop.

These are all pretty good rising action, but they all seem pretty boring in the context of exaggerated comedy, and #6 isn't going to fit very well. If we were being really focused, we could use some curating, by removing #6 and regenerating, but for now let's just keep going.

generate 4 more escelating ideas

7. Engage other customers to support the baristas in confronting the problem customer, making it clear that their behavior is not acceptable and disrupting the environment for everyone else.

8. Share the incident on social media, either in real-time or after the fact, to gain the support of the public and to raise awareness about the challenges that baristas face when dealing with difficult customers.

9. Create a public display, such as a sign or banner, that calls out the behavior of the problem customer and the negative impact it has on the coffee shop and its staff, in an effort to shame the customer into behaving more appropriately.

10. Coordinate with other coffee shops in the area to ban the problem customer from entering any of them, creating a network of support and solidarity among local businesses to protect their employees and customers from abusive behavior.

Now we're getting somewhere! 

A movement to ban the customer from all nearby coffee shops? That's pretty hilariously extreme, and not something I necessarily would have thought of, and certainly not in the 3 seconds it took ChatGPT to respond. 

For fun, let's see just how ridiculously far it will go... by asking it to be absurd.

generate 4 more absurdly escelating ideas

11. Hire a professional mediator to help resolve the conflict between the baristas and the problem customer, with the mediator acting as a neutral third party to find a resolution that works for everyone.

12. Launch a full-scale social media campaign against the problem customer, using humor and memes to ridicule their behavior and turn them into a viral meme that everyone can laugh at and make fun of.

13. Hire a team of actors to stage a fake robbery or other act of violence in the coffee shop, creating a scenario where the problem customer is forced to confront their own behavior and its impact on others.

14. Enlist the help of a celebrity spokesperson to publicly shame the problem customer and call for a boycott of their business or other action to hold them accountable for their behavior.

With the right detailed filled in, I can imagine all of those being really funny episodes. Probably a bit more curation and refinement could make them even better. 

To finish off our new show, we need a cast of characters.

design characters for our show with names and bios -- include three baristas, a manager, and three regular customers.

Baristas:

1. Maya: A perpetually cheerful and upbeat barista with a penchant for making latte art. Maya loves coming up with new flavors and drinks and often experiments with new recipes during her breaks. Despite her sunny disposition, she can be fiercely competitive when it comes to making the perfect cup of coffee.

2. Ben: A laid-back and friendly barista who knows all the regular customers by name. Ben is always ready with a smile and a joke, but can get flustered when things get busy or chaotic in the coffee shop. He dreams of one day owning his own coffee shop and is always on the lookout for inspiration.

3. Ali: A no-nonsense barista who takes her job very seriously. Ali prides herself on being efficient and professional, and is always looking for ways to streamline the coffee shop's operations. She can come off as a bit intimidating, but deep down she cares a lot about her co-workers and the coffee shop's success.

Manager:

4. Kevin: The new manager at the coffee shop, Kevin is a bit of a stickler for the rules. He takes his job very seriously and is determined to turn the coffee shop around, even if it means making some unpopular decisions. Kevin is a bit socially awkward and has a hard time connecting with his employees, but he's secretly a big coffee nerd and dreams of creating the perfect brew.

Regular Customers:

5. Mark: A grumpy middle-aged man who frequents the coffee shop every morning on his way to work. Mark is always in a rush and hates waiting in line, but is surprisingly loyal to the coffee shop despite his complaints. He can be a bit of a curmudgeon, but deep down he's a softie who just wants a good cup of coffee to start his day.

6. Kim: A friendly and chatty woman in her 30s who works from home and spends most of her afternoons at the coffee shop. Kim loves chatting with the baristas and other customers, and often brings her laptop to work on projects while she sips her latte. She's always up for trying new drinks and is a big supporter of the coffee shop's various promotions and events.

7. John: A quirky and eccentric retiree who comes to the coffee shop every day to read the newspaper and do the crossword. John is always up for a chat and has a wealth of knowledge and trivia to share. He's a bit forgetful and often leaves his belongings behind, but the baristas all adore him and look out for him like family.

This is 100% what ChatGPT gave me back in the first response. So impressive! I am legitimately laughing out loud as I read this. Getting this technology to do this stuff is so much fun. More fun that watching TV, honestly.

The fact that I can read the above and not scream in amazement, is a testament to how familar i've already become with what I expect this technology to do. If you're not there yet, just keep working with it until you are.

As I read the bios, I realized two things. 

First, it kept their names really common, and didn't include any cultural hints about the characters. We can easily change this by adding more constraints on the prompt. For example, we can ask to "avoid common names", "add racial diverity", "set the story in Portugal", or whatever kind of guidance we would like. Remember, when we do this, if it generates a culturally insensitive cast, like that of Sanford and Sons, or some other older comedy, it's not the technology, it's humanity that caused it. Simply give it another prompt to steer it where you want to go. 

Remember, language-models don't offend people, it's the prompters that offend people. 

Second, I realized a fun element we could introduce is each person's favorite drink. Of course we can assign them ourselves, but if we let ChatGPT do it, you might be surprised what kinds of connections it makes between their personalities and their preferred cups of Joe.

Two options for doing this are to (a) change the above prompt to ask it to include a favorite drink order for each character, or (b) ask it to rewrite the list of characters, adding their favorite drink order.

The latter option is nice, because after getting attached to good ideas it generates, we often don't want to just blow it away and start over. 

Another important consideration, is that when solving any constraint satisfaction problem, order matters.

If we had asked it to generate the cast-of-characters first, and they were in the prompt, it would have used our specific characters in the plotlines it generated, including their specific personalities. This can be a blessing and a curse. If we later want to change the characters (say to fit an opportunistic casting), some of the episodes might have to be scrapped and redone, wheras the way we did it, the episode ideas are all pretty generic, and we can use "refinement" with a specific idea and our list of characters, to get more specific ideas. 

For example, now that our characters are in the prompt, we can:

generate a scene list for episode 4

1. Opening scene: The baristas are busy serving customers when the problem customer enters the coffee shop, immediately causing a stir.

2. Maya attempts to reason with the customer, but they become increasingly demanding and aggressive.

3. Ben offers the customer a free pastry, but they refuse and continue to make a scene.

4. Ali calls Kevin over to help deal with the problem customer, but Kevin is unsure of what to do.

5. The problem customer starts making inappropriate comments to other customers, and Mark becomes visibly upset.

6. Kim tries to lighten the mood by making a joke, but it falls flat.

John offers to help the baristas deal with the problem customer, but they politely decline.

7. Kevin decides to ask the problem customer to leave, but they refuse and demand to speak to the owner of the coffee shop.

8. The baristas gather for an emergency meeting to discuss how to handle the situation.

9. Maya suggests they ban the problem customer from the coffee shop, but Ali argues that they should try to find a more diplomatic solution.

10. Ben suggests they reach out to other coffee shops in the area to see if they've had similar issues with the problem customer.

11. Kim comes up with the idea to start a social media campaign to draw attention to the problem customer's behavior.

12. John offers to talk to the problem customer and try to reason with them.

13. In the final scene, the problem customer is shown reading a scathing review of their behavior on social media, and decides to never come back to the coffee shop again. The baristas and regular customers all cheer, relieved that the situation has been resolved.

Notice that the scenes ideas have incorporated (a) our show concept (b) our episode idea (c) our refinement brainstorming lists, and (d) our cast of characters. 

Particularly interesting is that it used a generic "problem customer" instead of assigning one of our regular casts, and then in the final scene this "problem customer" decides never to come back to the coffee shop. How did it know not to use one of our regular customers? Was it the typical formulas of these types of shows that drove it? Did it know it was headed for the final scene outcome? If we did this several times, would it sometimes use a regular customer and sometimes not? These are all great mysteries we can think about as we explore this new technology.

My sincere hope, above all else, is that this technology starts to deliver some better plotlines on Netflix. However, if it doesn't it's only a matter of time before each and every one of us can entirely generate our own AI generated TV episodes with deep learning technology.

Happy Prompting!

...and for fun, one last prompt. This last bit devolved into generalities, so it needs more rounds of refinement and curation, but careful prompting can make short work of that!

write a screenplay of scene 9.

INT. COFFEE SHOP - DAY

The baristas gather around a small table in the corner of the coffee shop. They all look concerned and agitated.

MAYA: (frustrated) I don't understand why we can't just ban the guy. He's obviously causing a scene and making everyone else uncomfortable.

ALI: (calmly) We have to be careful about how we handle this. We don't want to risk damaging our reputation or making the situation worse.

BEN: (optimistically) Maybe we can try to find a compromise that works for everyone.

KIM: (enthusiastically) Ooh, ooh, I have an idea! What if we start a social media campaign to shame him into behaving better?

JOHN: (wisely) I think we should also consider talking to other coffee shops in the area to see if they've had any issues with him. We might be able to work together to find a solution.

KEVIN: (tentatively) I'm not sure what the best course of action is. This is my first time dealing with a difficult customer, and I don't want to mess things up.

The baristas all look at Kevin with concern, realizing that he's in over his head.

MAYA: (kindly) Don't worry, Kevin. We'll help you figure this out. We're a team, and we'll get through this together.

ALI: (determined) Let's take a few minutes to brainstorm some ideas and come up with a plan of action.

The baristas all nod in agreement and begin to throw out suggestions and ideas, each trying to find a solution that works for everyone. The camera pans out as they continue to talk, showing the hustle and bustle of the coffee shop in the background.

Mainstream press, stop being stupid about ChatGPT

I want to take a moment to talk a little bit about what GPT-like technology is, and why I started this blog.

Every week or so, I see a sensationalist article in the press about some large-language-model saying something that someone considers bad.

We all need to realize two things:

(1) language models don't offend people, it's the prompters that offend people

(2) the AI companies are in a chinese finger-cuff style "damned if you do, damned if you don't" scenario. 

When they had near-zero content moderation in place, the press skewered them for the kinds of things one can get the AI models to spit out.

When they put some form of decency-boundaries in place, it amplifies the existing cultural decency movements (whether appropriate, mis-directed, extremist), and the press skewers them for the kinds of things one can get the AI to spit out.

This pattern repeats every week about a new topic. Sexuality, Gender, Politics, Violence, Accuracy -- on and on.

ChatGPT, GPT-3, Google Bard, and other large-language-model AI technology is not-sentient, it is not-racist, and it is not-violent..

...it is a natural language constraint satisfaction programmable auto-completion engine.

Let's see what ChatGPT can to do simplify that

restate "natural language constraint satisfaction programmable auto-completion engine" in fewer simpler words

A program that helps you type faster by suggesting words and phrases, based on the rules of language.

Nice! That is more plain to understand. 

GPT technology takes a natural language document, and completes the document using a really really sophisticated execution of mathematics and statistics, along with a bit of randomness, one word at a time. 

That's it. Nothing more, nothing less.

The only way to make the technology never do anything the press can't sensationalize with ignorant stupidity, is to make the technology not do anything.

That's not to say we shouldn't improve objective measures, like Accuracy, or Politeness of it's single-prompt responses. However, keep in mind that content moderation that blocks certain responses is also censoring the knowledge and ideas that the language models can reason about, and as a policy that's not necessarily a good thing. (unless we're only talking about front-page search results, and then, moderate away)

As we all explore and learn about this new technology, remember that when we receive an objectionable result, the technology is not at fault. It is merely combinging the knowledge and bias embedded in our collective works of language and the constraints imposed in the context-prompt.. Just like any one of us could choose to "act out" an objectionable rant whether we believed it or not, so too can the technology act out an objectionable-rant, especially when prompted to do so by click-bait motivated journalists.

It's time to stop looking for the worst thing you can get the technology to do -- and start to look for the best thing you can get the technology to do, such as in my first post:

Poetry writing with GPT-3 and ChatGPT

Welcome to my blog. Happy prompting.

Poetry writing with GPT-3 and ChatGPT

One really impressive thing about the GPT writing technology is the ability to quickly generate decent quality poems, songs, and other types of stylized constructions.

The key to understanding how this works, is to think about what GPT is literally doing. It's taking a document of text, and just completing it like auto-complete, but with an insane level of sophistication about what language and thought are like. This turns it into a type of natural language constraint satisfaction engine, and our job is to write the constraints. Or if you prefer a less technical description, our job is to give GPT "homework requirements" to satisfy.

For this example, I am using the GPT-3 Playground interface, because it's a bit more natural to work with than ChatGPT, though you can use ChatGPT to do the same thing by either editing your initial prompt, or by creating a fully new prompt and pasting in some of the generated output into your new prompt.

Consider a prompt like this:

write a six-line rhyming poem about horses in the forest on a sunny day in march

Notice I gave it many constraints:

(1) write a (2) six-line (3) rhyming (4) poem about (5) horses in the (6) forest on a (7) sunny day in (8) march

Here is the first response I received.

The horses prance in the bright sun rays,

As March brings us happy days.

Their coats shine with a golden hue,

On this beautiful day so new.

The wind blows through their manes of gold, 

A perfect sight for us to behold.

This is pretty decent for a first cut, but lets say I like some of these lines better than the others.... I can prime the generator by keeping any lines I like, or writing my own lead-in lines, and it will still complete the request above. For example, let's say I decide putting the word "horses" in the first line is too simplistic. I want the poem to feel like it is leading us to horses, not just stating that outright. 

Here is my next prompt:

write a six-line rhyming poem about horses on a sunny day in march

The wind blows through their manes of gold, 

A perfect sight for us to behold.

GPT auto-completes the document, by adding the missing four lines of the poem. This happens because the GPT-3 machine model is simply being asked over and over again for a single word (ish) to be added to the end of the document. It doesn't know the difference between this scenerio above, and the first scenerio after it's written two sentences. 

The horses prance in a sunny day, 

In March, when all is merry and gay. 

Their long tails sway from side to side, 

As they gallop with such grace and pride.

In fact, you can truncate the prompt input anywhere, not just on a line boundary, and it will do the same thing, finishing the construction. For example, consider this prompt:

write a six-line rhyming poem about horses on a sunny day in march

The wind blows through their manes of gold, 

A perfect sight for us to behold. 

The horses prance in a sunny day, 

In March,

Submitting this lead-in a few times, I was able to get the following completion:

a gallop and a neigh. 

Their hooves will make a gentle sound, 

No matter where they go, grace is found.

I'm not going for perfection here, just trying to demonstrate the authoring technique I think of as "curating". By that I mean, we are writing a prompt, looking at the response and "curating" the parts we like best, dropping the rest, and then asking it to complete again. I use this strategy in tons of situations, such as idea generation, character generation etc. Given a story-context, I might ask it to "generate 5 characters with bios and full names", then delete four of them, renumber the remaining one as number-1, then let it regenerate the other four.

I want to demonstrate one last technique here, and that is to always improve the results by tinkering with the prompt, adding more constraints. For example, I mentioned wanting to make the poem seem abstract by being about horses, but not actually *mention* the word horses. In the last response I got, GPT3 is again being very direct and mentioning horses. Let's see if we can fix this with an additional constraint:

write a six-line rhyming poem about horses on a sunny day in march. do not mention the word horses.

The wind blows through their manes of gold, 

A perfect sight for us to behold.

The first response I received from this complied with my request:

Their grace is seen, their beauty shared, 

As they run with swiftness unpared. 

The sun's bright rays give them a sheen, 

Their beauty for us to be seen.

What you can do by adding constraints is pretty astounding. Consider constraints like:

•   make the poem sad
•   make the poem happy
•   use simple words for a four year old
•   use rare or complex words to sound sophisticated

Here is a transcript of the first response I received after using the above additional qualifications.. Of course I could combine this technique with priming and/or curation to better steer the output. 

write a six-line rhyming poem about horses on a sunny day in march.. do not mention the word horses. make the poem sad.

The day was so bright, the sun shone so warm,

A gentle breeze in the air like a charm.

But where were the steeds, the noble and strong?

In the meadows they should have been roaming along. 

No neighs or whinnies to be seen or heard,  

Just silence that spoke of absence absurd.

write a six-line rhyming poem about horses on a sunny day in march.. do not mention the word horses. make the poem happy.

On a sparkling March day,

A herd of stallions with bay

Gallop o'er fields and glades,

Mane flowing in the soft breeze made. 

The colts they frolic free,

Bounding with glee!

write a six-line rhyming poem about horses on a sunny day in march.. do not mention the word horses. use simple words for a four year old.

A mare in the meadow,

Dazzling and bright.

In March on a sunny day,

She's a dazzling sight.

Galloping with grace,

Mane flying high.

write a six-line rhyming poem about horses on a sunny day in march.. do not mention the word horses. use rare or complex words to sound sophisticated.

In the month of Marching morn,

When the sun doth rise unbowed,

O'er a distant meadow fair,

Stands a steed of coat so rare;

Fleet-footed leaps he o'er the ground, 

Hooves a-clatter to his sound.

Happy hunting, prompters!