Accessing web APIs

Rich interfaces can be created through the web, allowing powerful interactions through HTTP. The most common interface is through RESTful APIs using JSON. These text-based interfaces are easy to understand and to program, and use common technologies that are language agnostic, meaning they can be accessed in any programming language that has an HTTP client module, including, of course, Python.

The strict definition of RESTful requires some specific characteristics, but an informal definition of RESTful is a system that describes resources through HTTP URLs. This means each URL represents a particular resource, such as an article in a newspaper or a property on a real estate site. Resources can then be manipulated through HTTP methods (GET to view, POST to create, PUT/PATCH to edit, and DELETE to delete).

Using requests is very easy with RESTful interfaces, as they include native JSON support.

Getting ready

To demonstrate how to operate RESTful APIs, we'll use the example site https://jsonplaceholder.typicode.com/. It simulates a common case with posts, comments, and other common resources. We will use posts and comments. The URLs to use will be as follows:

# The collection of all posts
/posts
# A single post. X is the ID of the post
/posts/X
# The comments of post X
/posts/X/comments

The site returns the correct result for each of them. Pretty handy!

How to do it...

1. Import requests:

   >>> import requests
   

2. Get a list of all posts and display the latest post:

   >>> result = requests.get('https://jsonplaceholder.typicode.com/posts')
   >>> result
   <Response [200]>
   >>> result.json()
   # List of 100 posts NOT DISPLAYED HERE
   >>> result.json()[-1]
   {'userId': 10, 'id': 100, 'title': 'at nam consequatur ea labore ea harum', 'body': 'cupiditate quo est a modi nesciunt soluta\nipsa voluptas error itaque dicta in\nautem qui minus magnam et distinctio eum\naccusamus ratione error aut'}
   

3. Create a new post. See the URL of the newly created resource. The call also returns the resource:

   >>> new_post = {'userId': 10, 'title': 'a title', 'body': 'something something'}
   >>> result = requests.post('https://jsonplaceholder.typicode.com/posts',
                 json=new_post)
   >>> result
   <Response [201]>
   >>> result.json()
   {'userId': 10, 'title': 'a title', 'body': 'something something', 'id': 101}
   >>> result.headers['Location']
   'http://jsonplaceholder.typicode.com/posts/101'
   

   Notice that the POST request to create the resource returns 201, which is the proper status for created.

4. Fetch an existing post with GET:

   >>> result = requests.get('https://jsonplaceholder.typicode.com/posts/2')
   >>> result
   <Response [200]>
   >>> result.json()
   {'userId': 1, 'id': 2, 'title': 'qui est esse', 'body': 'est rerum tempore vitae\nsequi sint nihil reprehenderit dolor beatae ea dolores neque\nfugiat blanditiis voluptate porro vel nihil molestiae ut reiciendis\nqui aperiam non debitis possimus qui neque nisi nulla'}
   

5. Use PATCH to update its values. Check the returned resource:

   >>> update = {'body': 'new body'}
   >>> result = requests.patch('https://jsonplaceholder.typicode.com/posts/2', json=update)
   >>> result
   <Response [200]>
   >>> result.json()
   {'userId': 1, 'id': 2, 'title': 'qui est esse', 'body': 'new body'}
   

How it works...

Two kinds of resources are typically accessed – single resources (https://jsonplaceholder.typicode.com/posts/X) and collections (https://jsonplaceholder.typicode.com/posts)

• Collections accept GET to retrieve all the members of the collection and POST to create a new resource
• Single elements accept GET to get the element, PUT and PATCH to edit, and DELETE to remove elements

All the available HTTP methods can be called in requests. In the previous recipes, we used .get(), but .post(), .patch(), .put(), and .delete() are available.

The returned response object has a .json() method that decodes the result from JSON.

Equally, to send information, a json argument is available. This encodes a dictionary into JSON and sends it to the server. The data needs to follow the format of the resource or an error may be raised.

The referred-to resource will be returned, and its URL is available in the header. This is useful when creating a new resource, as its URL is not known beforehand.

There's more...

RESTful APIs are very powerful but also have huge variability. Please check the documentation of the specific API to learn about its details.

See also

• The Downloading web pages recipe, earlier in this chapter, to learn the basics of requesting web pages
• The Installing third-party packages recipe in Chapter 1, Let's Begin Our Automation Journey, to learn the basics of installing external modules