将scrapy导入json时遇到问题

Question

我试图从Craigslist中取出一些信息并将其存储在JSON文件中，但信息存储有点不对。而不是有一个[标题，链接，位置，时间]数组，我得到一个所有标题的数组，所有的链接，等等。我的标题是错的还是for循环本身是错误的？

from scrapy.spiders import BaseSpider 
from scrapy.selector import HtmlXPathSelector 
from scrapy.selector import Selector 
from craigslist_sample.items import CraigslistSampleItem 

class MySpider(BaseSpider): 
name = "craig" 
allowed_domains = ["craigslist.org"] 
start_urls = ["https://pittsburgh.craigslist.org/search/ccc"] 

def parse(self, response): 
    titles = response.selector.xpath("//p[@class='row']") 
    items = [] 
    for titles in titles: 
     item = CraigslistSampleItem() 
     item["title"] = titles.xpath("//span[@id='titletextonly']").extract() 
     item["link"] = titles.xpath("a/@href").extract() 
     item["location"] = titles.xpath("//small").extract() 
     item["time"] = titles.xpath('//time').extract() 
     items.append(item) 
    return items 

Answer 1

这是因为你的内部xpaths匹配从树根开始的元素。相反，您需要强制他们在每个项目的上下文中通过预先加点来工作：

for title in titles: 
    item = CraigslistSampleItem() 
    item["title"] = title.xpath(".//span[@id='titletextonly']").extract() 
    item["link"] = title.xpath("a/@href").extract() 
    item["location"] = title.xpath(".//small").extract() 
    item["time"] = title.xpath('.//time').extract() 
    yield item