URL Encoding Explained: What %20 and %3D Mean

A web address can only contain a limited set of characters. Anything else, such as a space, an ampersand inside a value, or a letter like é, has to be written in a special form: a percent sign followed by two hex digits. This guide explains why that is, which characters are safe, and the mistakes that cause broken links and doubled encoding.

Why URLs need encoding

Some characters in a URL are punctuation with a job to do. A question mark starts the query, an ampersand separates parameters, an equals sign joins a name to a value, and a hash starts the fragment. If the data itself contains one of those, the browser cannot tell data from structure. Percent-encoding solves this by writing the byte as a percent sign and two hex digits, so a space is %20 and an ampersand is %26.

What is safe and what is reserved

RFC 3986 defines the rules. The unreserved characters never need encoding: the letters A to Z and a to z, the digits 0 to 9, and the four marks hyphen, period, underscore and tilde. The reserved characters are the ones with a special role, and must be encoded when they are part of the data. The general delimiters are colon, slash, question mark, hash, square brackets and at sign. The sub-delimiters are exclamation mark, dollar, ampersand, apostrophe, parentheses, asterisk, plus, comma, semicolon and equals.

The standard recommends upper-case hex digits, so %3D rather than %3d, although they mean the same.

Non-English text

A character outside basic English is first turned into bytes using UTF-8, and then each byte is written as %XX. The é in café is two bytes, so it becomes %C3%A9. The Russian word шеллы becomes %D1%88%D0%B5%D0%BB%D0%BB%D1%8B, and an emoji can need four bytes, giving four escapes. This is why a short word in Hindi or Tamil can become a long run of percent codes in a link.

encodeURI or encodeURIComponent?

JavaScript has two functions, and choosing the wrong one is a common bug. encodeURI is for a whole address. It leaves the structural characters alone, so slashes, question marks, ampersands and equals signs stay as they are. encodeURIComponent is for one piece, such as a query value. It also encodes those structural characters. For the text a=1&b=Jack & Jill, encodeURIComponent gives a%3D1%26b%3DJack%20%26%20Jill, which is right when this is a value, while encodeURI keeps the separators.

• Building a query value, a path segment or a redirect parameter: encode the piece.
• Cleaning up a full address that has spaces or non-English letters: encode the whole URL.
• Never encode a whole URL as if it were a component, because the colon and slashes would be escaped and the link would break.

The plus sign and form data

In the older form encoding used by HTML forms and many query strings, a space is written as a plus sign instead of %20. In a path, a plus sign is just a plus. This is why a search for C++ can come back as C%2B%2B: the plus signs have to be escaped so they are not read as spaces. The tool offers a form mode for this case.

Double encoding

Encoding text that is already encoded turns the percent sign itself into %25, so %20 becomes %2520. The sign of this problem is a link that shows %20 on the page, or a server that receives the literal text %20 in a value. RFC 3986 advises against encoding or decoding the same string more than once. Encode once, as late as possible, when you build the address, and decode once, when you read it.

When decoding fails

Decoding fails when a percent sign is not followed by two hex digits, as in 100%, or when the bytes do not form valid UTF-8, as with %FF on its own. The fix is to find where the text was cut or encoded with another character set. The tool reports these cases instead of returning half a result.

A short checklist

• Encode the pieces, then join them with the real separators.
• Use a component encoder for values and a URL encoder only for a whole address.
• Do not encode twice.
• Expect non-English text to grow, three or four times as long.
• If a plus sign turns into a space, the value was read as form data.

Sources

• RFC 3986: Uniform Resource Identifier (URI): Generic Syntax
• encodeURIComponent() (MDN Web Docs)