Instructor: Okay, let's learn about
progressive summarization.
So, the idea behind this is that we want to be able
to take large documents
and make them smaller, summarize them,
but in a way that can handle an arbitrary size of document.
So we summarize a bunch of different documents
and then summarize the summaries as well.
So first, let's install OpenAI.
We're just installing the package here.
And then we want to get our API key,
so this is just to get the OpenAI client
and then we'll do a secret key,
and then we'll just, once we get that secret key,
we'll pass it in to the OpenAI client.
And then we're gonna put in the model as well.
Lemme just grab a secret key,
and I'm just gonna run this, and you see it pops up here,
"Enter your OpenAI secret key."
Just gonna paste it in there,
and now it's saved locally.
And in order to get to that, by the way,
you can just go here.
All right, now, create a function to call OpenAI.
So the way this works is typically
you wanna have something like submit_prompt
or get completion, whatever it is.
And then you're gonna get a response,
and we're gonna use the responses API, it's the newest API.
And so, instead of the completions API we used to do,
so we need to pass in the model.
And then it's like input instead of messages,
although it's still backwards compatible,
you could use input.
And we're just pasting in here the role of user
and then the content,
which is the actual prompt that we wanna put in here.
And then I'm gonna say...
Oops, sorry,
text equals format text.
So that's like the format we want back.
Temperature, max_output_tokens,
and then store false, of course.
So that's gonna be what we actually,
that's going to determine whether OpenAI stores it
on their site or whether we need to pull that back out,
like to get the next response.
But we're not gonna do that.
That's the type of function we need.
And then we're just gonna print out submit_prompt,
say, "Hey GPT are you ready?"
Let's see if it comes back or something.
We're not getting anything here, so that's because
we didn't get the response.
So we're just gonna get the response output
and the content and then the text.
So that's how you get it back from the responses API.
Here we go. It's working.
"Absolutely! I'm ready.
How can I assist you today?"
All right, we wanna create a prompt for summarization.
I'm just gonna use this one here from this
Contently piece, right?
So the prompt is "Summarize this for an eighth grade student
as a tweet of 280 characters."
And then I've added this section here.
No hashtags, no emojis, no links.
Because sometimes when you say as a tweet,
it puts like hashtags and stuff in there,
which is really annoying.
And then you'd put your text here. So that's the prompt.
And then the text gets fed in.
So what we're using here is the format response.
And this is basically looking for anything
in here in curly brackets and then filling it in.
So we've said that the variable is text,
and so it's filling in that text variable here.
So whatever prompt, whatever text we put into that prompt,
is gonna go into the template.
All right, we've got a bunch of different articles here.
These are the different articles that I wanted to summarize.
And there's a bunch of different text.
So I'm just gonna paste this in.
So I just went through and manually got this text,
but you could also write something
that searches the blog posts
and gets the text for you.
Got these different publishers as well.
And what I wanna understand is the topic
of marketing mix modeling or medium mix modeling,
and that these are a bunch of different posts about that
that are ranking on Google.
So I want to summarize all these different posts
and then summarize the summaries as well
so I can get like an overall understanding of the topic.
All right, so let's get going with it.
So let's just say text equals the first text
from that list.
So our formatted_prompt is gonna be summary_prompt,
and then we're gonna format
and we're gonna say the text equals text.
So that's gonna give us the formatted prompt
with the text from that first article.
And then we're gonna just call our function, submit_prompt,
and we're gonna stick in that formatted prompt.
Cool.
All right. And let's just print the response.
Here we go, "Media Mix Modeling
MMR, helps brands figure out
which ads really boost sales.
Old MMM worked well for TV and radio
but struggles with online ads
because platforms adjust spending," okay, cool.
So that's interesting. It's got a good summary there.
But let's hear from other voices, right?
That's just one of the articles I recast.
I wanna hear from some of the other people in the industry.
So what we're gonna do is we're gonna iterate through
everything, I'm just gonna
say all_responses,
and then I'm gonna say the formatted_prompt is text.
Oh, sorry,
for text in texts, right?
So then we're gonna have,
the formatted prompt is gonna take the text for that texts.
So it's gonna get the first result and then second result
and then third result,
it's gonna get the response each time.
Then it's gonna append all the responses together.
We can also print the response
and then maybe print like a line between them so we can see.
All right, let's just run this.
And you could also run this in parallel as well.
That would be a lot faster, but we just wanna see these
one by one and we're not doing a lot of these.
Okay, so look, we're getting the summaries each time,
Which is really helpful.
This one calls it econometrics, which is interesting.
And then, yeah, it looks like
a lot of these summaries are converging.
It's a really good kind of understanding
of what we've got across all these different things,
but maybe we don't wanna read every individual summary.
Maybe that's too much.
So we can actually go and summarize
the summaries as well here.
If we say for response in all_responses,
should say all_responses is, there we go.
So have all the responses together,
just separate it by two new lines.
Oh, that didn't work.
Okay, so we joined all our responses together here.
And then we want to get a summary of that.
So I'm just gonna submit_prompt.
Here we go, that's a concise summary of everything.
All right. See another example of this.
This is quite often used when you have a larger page
and you wanna summarize a whole article.
So if I paste in here,
this is all the Wikipedia text on marketing mix modeling,
and it's separated by commas.
So we have a bunch of different chunks here.
That's what they're called in the industry.
So it's quite common that if you can't fit the whole webpage
or you can't fit a whole report in the context window
of your LLM, then you can put all of these different chunks
together and summarize them with the same type of approach.
So lemme just try this.
We're gonna say
we're gonna get all the Wikipedia
responses
and we're gonna say for text in wikipedia_texts,
the formatted_prompt,
and then we're gonna summarize everything together.
So,
and then get that together.
And then we're gonna get the formatted_prompt,
and then we're gonna get the response,
and then we're gonna print that response.
So what we're doing here, again, this is what we just did,
except with Wikipedia,
we're getting the summaries for everything.
So it's getting the response for the summary
for each individual piece of the page.
And then it's gonna summarize everything together.
The final end of the page.
So you can see it's printing out each one.
You see it talks about Coke and Pepsi,
had to spend money on ads.
And this is, yeah, this is the final result.
And the reason this is useful is just that
typically you can't always summarize the whole article
or like all the articles in one,
because you can't put all of them
in the context window at once.
But if you use progressive summarization
where you summarize different chunks
or summarize different articles,
and then you summarize the summaries, then you have a way
to arbitrarily get to the final result.