Scrapy：类型错误：__init __（）失踪1个人需要的位置参数： '设置'

Question

我有一个Scrapy中间件：

class ProxyMiddleware(object): 
    def __init__(self, settings): 
     self.proxy_file = settings.get('PROXY_FILE') 
     fin = open(self.proxy_file) 
     self.proxy_list = list() 
     for line in fin.readlines(): 
      parts = line.strip().split() 
      proxy = parts[2] + '://' + parts[0] + ':' + parts[1] 
      self.proxy_list.append(proxy) 

    def process_request(self, request, spider): 
     request.meta['proxy'] = random.choice(self.proxy_list) 

，但出现错误在运行时，有什么设置？

Answer 1

如果您需要一个settings对象来初始化您的中间件，您需要定义一个from_crawler()类方法otherwise scrapy initializes the middleware without arguments。

看看灵感内置的中间件，例如HttpErrorMiddleware

class HttpErrorMiddleware(object): 

    @classmethod 
    def from_crawler(cls, crawler): 
     return cls(crawler.settings) 

    def __init__(self, settings): 
     self.handle_httpstatus_all = settings.getbool('HTTPERROR_ALLOW_ALL') 
     self.handle_httpstatus_list = settings.getlist('HTTPERROR_ALLOWED_CODES') 

在你的情况下，它会是这样的：

class ProxyMiddleware(object): 

    @classmethod 
    def from_crawler(cls, crawler): 
     return cls(crawler.settings) 

    def __init__(self, settings): 
     self.proxy_file = settings.get('PROXY_FILE') 
     fin = open(self.proxy_file) 
     self.proxy_list = list() 
     for line in fin.readlines(): 
      parts = line.strip().split() 
      proxy = parts[2] + '://' + parts[0] + ':' + parts[1] 
      self.proxy_list.append(proxy) 

    def process_request(self, request, spider): 
     request.meta['proxy'] = random.choice(self.proxy_list)