- Python Web Scraping Cookbook
- Michael Heydt
- 57字
- 2021-06-30 18:44:04
How it works
In the case of using urlopen, the conversion was explicitly performed by using the str statement and specifying that the content should be converted to UTF-8. For requests, the library was able to determine from the content within the HTML that it was in UTF-8 format by seeing the following tag in the document:
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">