[sunjester tuts] HTTP GET/Web Requests with Python

[18]

Introduction
I normally don't use python very much. However, I can always jump right into using it since I know how to properly read documentation. I am going to show you how you can read some documentation and start making requests to websites through python and the requests library. As with all my tutorials we will be using Linux (ubuntu xenial), and no I will not cater to the Windows kids.
 
Requirements
This tutorial will require a few things, including python. I am using 2.7 (most of you probably use 3, I don't.).

• Python >= 2.7
  
• pip >= 19.03
  
• python requests library >= 2.21
  

Checking Versions with Pip
You can easily check the requests version, or if it's even installed like this

Code:

pip show requests

 
Documentation
The website for the requests library is found here. We will be following the documentation and using the software as intended by the developers, which should give you an insight into reading documentation and finding/using other libraries.
 
Including the Library
It's pretty easy to add the library to your script. Like almost all other libraries, using the import keyword.

Code:

import requests

Getting HTML
We can use a simple line of code to requests an HTTP GET request for HTML of a web site.

Code:

req = requests.get("https://www.imdb.com/title/tt0093177/") 

This is where reading the documentation will come in handy. The code above will make the request to the IMDB page, but printing the req variable probably won't give you what you are looking for, so let's see what the will do.
 

 
It printed the response code. Back on the website (documentation) we see that it's a response object, which is probably why it has the angle brackets around it also.
 

 
Response Object Documentation
Since it returned an object, let's go see what the properties of this object are in the documentation. Here is the documentation for the Response object of the Requests library. We can see there are a few things here, let's try to use content and see if we get the source.

Code:

#!/usr/bin/env python
import requests

req = requests.get("https://www.imdb.com/title/tt0093177/") 
print req.content

 
Yep, there is the source.

Underwurld Club - RAT Museum - http://sunjester.dev

[4]

thanks ive always wanted to get into python

[10]

im glad this is here

[8]

thanks