URL encoding, officially known as percent-encoding, is a mechanism for encoding information in a Uniform Resource Identifier (URI). URLs can only be sent over the Internet using the US-ASCII character set. Because URLs often contain characters outside the ASCII set or characters that have special meaning within URLs, these characters must be converted into a valid, universally accepted format.
The percent-encoding process works by replacing unsafe ASCII characters with a '%' followed by two hexadecimal digits that represent the character's ASCII value. For example, a space character corresponds to decimal 32, which is 20 in hexadecimal. Therefore, a space becomes '%20' when encoded in a URL, ensuring the web server receives a continuous string without breaking the URL structure.
Reserved vs. Unreserved Characters
The URI specification (RFC 3986) divides characters into two groups: reserved and unreserved. Unreserved characters include uppercase and lowercase letters, decimal digits, hyphen, period, underscore, and tilde (A-Z, a-z, 0-9, -, ., _, ~). These characters do not require encoding and can be safely transmitted as-is.
Reserved characters, such as '?', '&', '=', '#', and '/', have special syntactic meaning in a URL. For instance, '?' denotes the beginning of a query string, and '&' separates query parameters. If you need to include a literal '?' or '&' as data within a query parameter—rather than using them as structural delimiters—you must encode them as '%3F' and '%26', respectively.
Handling Unicode and application/x-www-form-urlencoded
When encoding characters outside the ASCII character set, such as emojis or non-Latin scripts, the character must first be converted to a byte sequence using the UTF-8 character encoding. Each byte is then percent-encoded. For example, the character 'é' (U+00E9) is represented in UTF-8 as the bytes C3 A9, which becomes '%C3%A9' when URL encoded.
It is important to distinguish standard URL encoding from the 'application/x-www-form-urlencoded' format used in HTML form submissions. In standard percent-encoding, spaces are encoded as '%20'. However, in form submissions, spaces are historically encoded as a plus sign ('+'). This discrepancy means decoding utilities must know the context of the data to correctly handle plus signs versus percent-encoded spaces.