在Python爬虫中设置表头通常是为了模拟浏览器请求,避免被网站识别为爬虫。以下是使用`requests`库设置表头的方法:
```python
import requests
url = "http://www.example.com"
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.104 Safari/537.36",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
"Cache-Control": "max-age=0",
"Connection": "keep-alive",
"Referer": "http://www.baidu.com/"
}
response = requests.get(url, headers=headers)
print(response.text)
如果你需要使用`csv.writer`来设置表头,你可以这样做:
```python
import csv
headers = {
"Name": "John",
"Age": "30",
"City": "New York"
}
with open('output.csv', 'w', newline='') as csvfile:
writer = csv.writer(csvfile)
writer.writerow(headers.values()) 写入表头
请注意,使用表头时要确保它们与目标网站的预期格式一致,否则可能会导致请求失败或数据解析错误。