NetFire

Using Python’s Requests Library

In this tutorial, you’ll learn some basics for using Python’s requests library to make GET or POST requests, handle JSON data, and more.

Making GET requests

Install the library, if you haven’t already:

pip install requests

Let’s make a GET request against a server. You can use http://httpbin.org/.

Go to the python shell and import requests to import the library.

Use the GET method to make a request for the /get endpoint:

r = requests.get(‘http://httpbin.org/get’)

Now you’ve assigned the response object to the r variable. You can check the attributes of the r variable like this:

>>> r.status_code 200

Processing JSON data from an API

Get your response object:

r = requests.get(‘https://api.github.com/events’)

View the data in its raw form:

r.text

This gives you an enormous string. You can use JSON to process this string into a more user-friendly format:

events = r.json

Post data to a server using POST requests

In your terminal, enter the python shell and import requests. Then create a simple dictionary to work with that contains your payload of data. Example:

>>> payload = {‘words’: “I wear yellow socks.”, ‘user_id’: 37}

>>> r = requests.post(‘http://httpbin.org/post’, params=payload)

Now you can view the JSON in the attributes of r:

r.json

What if your endpoint has moved?

The requests library handles this issue with redirects:

>>> r = requests.get(‘http://httpbin.org/redirect/3’)

…which will redirect you 3 times. You can check to make sure this worked by viewing the status_code attribute:

>>> r.status_code

200

To disallow redirects:

>>> r = requests.get(‘http://httpbin.org/redirect/3’, allow_redirects=False)

Authentication

These authentication types are most often used when you’re requesting a web page. Here we’ll cover HTTP Basic and HTTP Digest Authentication. The primary difference between the two is that Basic sends passwords in plain text, while Digest hashes them first.

Basic:

>>> from requests.auth import HTTPBasicAuth, HTTPDigestAuth

>>> r = requests.get(‘http://httpbin.org/basic-auth/user/password’, auth=HTTPBasicAuth(‘user’,’password’))

>>> r.status_code

200

You can check it works by trying to use an invalid password, like so:

>>> r = requests.get(‘http://httpbin.org/basic-auth/user/password’, auth=HTTPBasicAuth(‘user’,’invalid_password’))

>>> r.status_code

401

When you try substituting the invalid password, you get a 401, which is forbidden.

Digest:

>>> r = requests.get(‘http://httpbin.org/digest-auth/auth/user/password’, auth=HTTPDigestAuth(‘user’,’password’))

>>> r.status_code

200

Just like Basic, you can check it works by trying to use an invalid password, like so:

>>> r = requests.get(‘http://httpbin.org/basic-auth/user/password’, auth=HTTPBasicAuth(‘user’,’invalid_password’))

>>> r.status_code

401