我得到一个像标题一样的错误。AttributeError:'NoneType'对象没有属性'div'
Traceback (most recent call last):
File "C:/2.py", line 40, in <module>
reviews.append(reviews_info(div))
File "C:/2.py", line 21, in reviews_info
review_text = div.find("div", "review-content").div.text
AttributeError: 'NoneType' object has no attribute 'div'
我想我可能输入了错误的元素。
我没有从零开始编写代码。我已经申请了以前爬行其他网站的代码。
如何更改元素并将其打开? 我想我写得很好。但是有一个错误。
ps我恐怕会有使用api的评论。但这是使用它的方式,因为它是有限的。
import sys
from bs4 import BeautifulSoup
import urllib.request
import requests
from urllib.parse import quote
import os
import xlwt
import re
import time
import random
import re, requests, csv
from bs4 import BeautifulSoup
from time import sleep
# CMD chcp 65001
def reviews_info(div):
review_text = div.find("div", "review-content").div.text
review_stars = div.find("div", "i-stars i-stars--regular-1 rating-large").a.text
return {
"review_text" : review_text,
"review_stars" : review_stars,
}
base_url = "https://www.yelp.com/biz/founding-farmers-d-c-washington-2?start="
reviews = []
NUM_PAGES = 36
for page_num in range(1, NUM_PAGES + 20):
print("souping page", page_num, ",", len(reviews), "data")
url = base_url + str(page_num)
soup = BeautifulSoup(requests.get(url).text, 'lxml')
for div in soup('div', 'review-content'):
reviews.append(reviews_info(div))
sleep(5)#############################################
# Save dict data
keys = reviews[0].keys()
with open('testtest.csv', 'w', encoding="utf-8") as f:
dict_writer = csv.DictWriter(f, delimiter=',', lineterminator='\n', fieldnames=keys)
dict_writer.writeheader()
dict_writer.writerows(reviews)
你描述你的刮刀和待办事项列表的方式是完全模糊的。如果你指定你想从你的刮板中提供的链接来解析的字段会更好。 – SIM