Python-Requests

Niko2022-01-172022-01-17

本教程使用Python语言，需提前安装Pip3 or Pip，例如Linux类的，请在命令行内输入:

sudo apt install python3-pip

安装

一条命令(临时换源):
Linux:

sudo pip install requests -i https://mirrors.aliyun.com/pypi/simple/

Windows:

pip install requests -i https://mirrors.aliyun.com/pypi/simple/

使用

基础Get请求

import requests
data = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36'
}  # 向服务器传递的数据
response = requests.get('https://bing.com',data=data).text # text函数:获取对应网址的源代码
print(response)

User-Agent就像是浏览器的身份证,如果是Requests默认的ua的话，对应服务器会拒绝你的爬虫请求，简而言之，拿不到数据，所以我们要获得浏览器的ua。

就在这拿User-Agent

进阶 Post请求

官网教程

带数据的 post

# -*- coding:utf-8 -*-
import requests
import json
 
host = "http://httpbin.org/"
endpoint = "post"
url = ''.join([host,endpoint])
data = {'key1':'value1','key2':'value2'}
 
r = requests.post(url,data=data)
#response = r.json()
print (r.text)

多文件上传

# -*- coding:utf-8 -*-
import requests
import json
 
host = "http://httpbin.org/"
endpoint = "post"
 
url = ''.join([host,endpoint])
#多文件上传
files = [
    ('file1',('test.txt',open('test.txt', 'rb'))),
    ('file2', ('test.png', open('test.png', 'rb')))
    ]
 
r = requests.post(url,files=files)
print (r.text)

冷门 Put 请求

import requests
import json
url_put = "http://127.0.0.1:8080/"
headers_put = {
	'Content-Type': "application/json"
}
param = {
	'myObjectField': 'hello'
}
payload = json.dumps(param)
response_put = requests.put(url, data=payload, headers=headers_put)

项目实战: 梨视频爬虫

这个项目我以前做过，就直接复制过来啦，摆烂啦（

# !/usr/bin/python3
# -*- coding:utf-8 -*-
"""
@author: 
@file: 梨视频爬虫.py
@time: 2021/7/11 21:48
@desc: 
"""
import requests

url = "https://www.pearvideo.com/video_1731260"
headers = {
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36",
    "referer": url,
}
contId = url.split("_")[1]
videoStatusUrl = f"https://video.pearvideo.com/mp4/third/20210603/cont-{contId}-15316010-202041-hd.mp4"
# 'https://video.pearvideo.com/mp4/third/20210603/cont-1731260-15316010-202041-hd.mp4'

resp = requests.get(videoStatusUrl, headers=headers).content
with open("video.mp4", mode="wb") as file:
    file.write(resp)