The urllib module (specifically urllib.request) provides functions for making HTTP requests and working with URLs, without needing a third-party library.

Syntax

from urllib.request import urlopen
from urllib.parse import urlencode, urlparse

response = urlopen(url)

Deep Dive Course

urllib.request.urlopen(url) opens a connection to a URL and returns a file-like response object you can read() to get the raw response body as bytes. urllib.parse provides utilities for taking URLs apart, splitting one into scheme, host, path, and so on, and putting query strings together, turning a dict of parameters into a properly escaped query string. In modern code, the third-party requests library is generally preferred for anything beyond very simple cases, since it offers a considerably more convenient API for headers, JSON, sessions, and error handling — but urllib is always available with no extra installation.

1Understanding urllib Module

💡

For anything beyond a single, simple GET request, most developers reach for the third-party requests library instead of urllib.request — it needs a separate install, but its API for headers, JSON bodies, and error handling is much more ergonomic.

editor.html

from urllib.parse import urlencode

params = {"q": "python tutorials", "page": 2}
query_string = urlencode(params)
print(query_string)

localhost:3000

2Practical Example

Here is a real-world application of urllib Module showing how it is used in production Python code.

editor.html

from urllib.parse import urlparse

url = "https://example.com/search?q=python&page=2"
parsed = urlparse(url)
print(parsed.scheme, parsed.netloc, parsed.path)

localhost:3000

3Best Practices

Follow these guidelines when working with urllib Module:

1. Use urllib for simple scripts where avoiding a third-party dependency matters, or in restricted environments where installing packages isn't possible

2. Reach for the requests library instead of urllib for anything involving headers, authentication, JSON payloads, or more complex request handling

3. Always close, or use a with block for, the response object returned by urlopen(), the same as you would for a file

⚠️

Tip: For anything beyond a single, simple GET request, most developers reach for the third-party requests library instead of urllib.request — it needs a separate install, but its API for headers, JSON bodies, and error handling is much more ergonomic.

editor.html

from urllib.parse import urlencode

params = {"q": "python tutorials", "page": 2}
query_string = urlencode(params)
print(query_string)

localhost:3000

Examples

Example 01Basic Usage

from urllib.parse import urlencode

params = {"q": "python tutorials", "page": 2}
query_string = urlencode(params)
print(query_string)

Example 02Advanced Example

from urllib.parse import urlparse

url = "https://example.com/search?q=python&page=2"
parsed = urlparse(url)
print(parsed.scheme, parsed.netloc, parsed.path)

Best Practices

Use urllib for simple scripts where avoiding a third-party dependency matters, or in restricted environments where installing packages isn't possible
Reach for the requests library instead of urllib for anything involving headers, authentication, JSON payloads, or more complex request handling
Always close, or use a with block for, the response object returned by urlopen(), the same as you would for a file

Interview Question

Why might a project choose the third-party requests library over the built-in urllib.request module?▼

Hint: Think about ergonomics rather than raw capability — urllib can technically do the same things.

urllib.request can technically perform the same HTTP operations as requests, but its API is considerably more verbose and low-level: handling headers, sending JSON bodies, managing cookies/sessions, and dealing with errors all require more manual boilerplate. requests wraps the same underlying functionality in a much more convenient, higher-level interface, which is why it's become the de facto standard for HTTP in Python despite requiring an extra installation, while urllib remains valuable specifically when you can't or don't want to add a dependency.

Exercises

MediumPractice using urllib Module in a real scenario.

View Solution

from urllib.parse import urlencode

params = {"q": "python tutorials", "page": 2}
query_string = urlencode(params)
print(query_string)

Frequently Asked Questions

Why might a project choose the third-party requests library over the built-in urllib.request module?▼

Related Functions

json-modulewith-statementas-keyword