All right.
I want us to now start talking a bit more about the actual encoding itself.
Firstly, what is and what is not a legal
URL is formally defined in the RFC 3986.
But as we're going to see a bit later, there are others.
And why is this spec so important?
Well, it's important because the W3C URI Specification has accepted this as being the formal spec that
should govern your URLs.
What are the main conclusions we can draw from the spec?
It's not that difficult.
There are just two broad categories of characters.
Reserved characters and unreserved characters.
Reserved characters have a special meaning.
They are reserved, right?
Like a forward slash or a question mark that represents the start of a query string.
We've already seen some examples of these. So, those are the reserved characters and those kind of have
to be treated in a slightly different way, as you would expect.
And then on the flip side, we've got these unreserved characters and these have no special meaning.
And because they have no special meaning, they are allowed to be in the URI itself.
Let me just get rid of all this noise.
Like I mentioned, we've got reserved characters and unreserved characters, and reserved characters
are very special.
The specification has chosen these characters to mean something very specific when it comes to URLs.
And because of this, if you want to use a reserved character in your URL, then they have to be encoded.
For example, we know that question mark is the start of a query string.
So if you want to use a question mark in your URL in some area, how does the browser know whether that's
now a query string or whether it's just your character?
And that's why it has to be encoded?
It makes sense.
Unreserved characters traditionally, like mentioned, are a limited subset of the ASCII character set.
But I've read the spec and in my opinion the spec does not explicitly state that this list is entirely
exhaustive.
I'm saying that with a bit of tongue in cheek because behind the scenes they may still be URL encoding,
but visually I'm talking about what the user sees,
that is not the case.
But don't get lost in all the detail.
Encoding is particularly important for encoding characters that are not permitted to be in a URL.
That's all that URL encoding is trying to do.
It's trying to just take non-performing characters and transform them in a way that's safe to transmit
over the web.
So all of these special characters spaces - percentage signs, tabs, colons, the equal sign and a
whole bunch of others - these need to be treated very specifically in the URL, and if we use them in
a URL, they have to be encoded to distinguish them from the reserved set itself.
Is that kind of making sense?
I hope so.
But it's not that you don't have other questions.
I mean, you might be asking, you know, why does the URL not permit certain characters in the first place?
Why can't we just have whatever we want?
Well, your browser's just trying to make sure that all the characters you want to send with a GET request
can arrive at the other end at the destination, and the browser has to encode some characters like
unprintable ones spaces, for example.
And as I just mentioned, it has to encode characters with special meaning because if it doesn't, how's
it going to know whether that query string is a query string or whether it's just a question mark from
your side?
So it just logically makes sense that the URL has to not permit all characters.
In other words, it makes logical sense that URL encoding takes place in some situations.
But let me clarify here, my dear students, encryption is not the same as encoding your URL.
Encoding is all about sending your data over a network.
It's about transport.
It doesn't make your data safe in any way.
I just wanted to clarify that.
Cool.
Have you got it?
Good.
But now how does your encoding actually take place?
What does it do?
URL encoding replaces nonconforming characters with a percentage symbol (%), followed by two hexadecimal
digits.
But what is this nonconforming character?
What do I mean by nonconforming?
Well, it's those reserved characters we were just talking about, isn't it?
If you want to use a question mark in your URL, it has to be URL encoded because it will be nonconforming.
And what is that percentage symbol?
What's that all about?
Well, in the previous lecture you'll know that it's just the hex prefix.
And after that percentage symbol, we have two hexadecimal digits.
Hexadecimal 🤷‍♀️?
Well, the good news is we spoke about that in the previous lecture, so it should all start kind of
gelling together right now.
You should start understanding the structure, what these things mean.
And before we move on, I just want to discuss one of the most frequent URL encoded characters you'll
come across, and that is the character space.
The space character is quite special because it's unprintable and therefore it makes sense that it has
to be URL encoded.
Well, the value of a space in decimal form is 32.
But we don't care about decimal forms, do we?
Because URLs only understand hex and in hex the value 20 is assigned to the character space.
Another one I want to look at actually is the plus sign as well.
That's a common one you're going to see and that is represented by 2B in ASCII.
So often in URLs you're going to be seeing %20 and %2B.
Woo!
We are cruising through this. And it may seem quite daunting, but don't feel overwhelmed.
In fact, the sets of reserved and unreserved characters are constantly changing with each revision
of the specs that govern your URLs.
So it can be very confusing. But don't worry, once you understand, once you grasp the fundamentals
of URL encoding, then it really doesn't matter what's reserved or unreserved, who cares?
Because we know exactly what's happening and we can always adapt as developers.
All right.
So I hope it's starting to gel.
I hope it's starting to make sense.
But in the next lecture, I really want to jump into these international characters.
Remember that example where we use the Japanese characters and we could see it in the URL?
Well, how is that possible when the RFC 3986 defines URLs as only containing a limited subset of ASCII
characters?
It's so weird.
Well, the good news is we're going to jump into it right now.