2016-02-28 176 views
2

当我试图通过此代码达到图像标记时,我得到以下输出。如何从链接获取图片?

url = 'https://paytm.com/shop/p/pepe-jeans-blue-slim-fit-t-shirts-APPPEPE-JEANS-BSETU2010438B648267' 

def soup_maker(url): 
    r = requests.get(url) 
    markup = r.content 
    soup = bs(markup, "html.parser") 
    return soup 

def get_images(url): 
    soup = soup_maker(url) 
    divs = soup.find_all('div', {'class': 'fixed-height'}) 
    print(divs) 
    images = soup.find_all('img') 
    print(images) 

输出

[] 
[<img alt="{{::product.text}}" ng-src="{{::product.image_url}}"/>, 
<img alt="{{item.title}}" ng-src='{{cart.imgResized(item.image_url,"50x50") }}'/>, 
<img ng-src="{{pixelSource}}"/>] 

但是,当我看到通过检查元素的存在。我不知道如何保存这些图像。

UPDATE

from selenium import webdriver 
from selenium.webdriver.common.by import By 
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC 

def soup_maker(url): 
    driver = webdriver.Chrome() 
    driver.get(url) 
    try: 
     element = WebDriverWait(driver, 20).until(
     EC.presence_of_element_located((By.CLASS_NAME, "fixed-height")) 
     ) 
     markup = driver.page_source 
     soup = bs(markup, "html.parser") 
     return soup 
    finally: 
     driver.quit() 
    driver.close() 

以上为我工作。

回答

2

这看起来像定义绑定的AngularJS模板,这意味着该网站需要一个真正的浏览器使用JavaScript引擎中被渲染让我们离开解析一部分是,但不是requests,通过selenium获取源:

from selenium import webdriver 

def soup_maker(url): 
    driver = webdriver.Firefox() # could also be Chrome(), PhantomJS() or other 
    driver.get(url) 

    # you might also need an Explicit Wait here to wait for the page to load 
    # see http://selenium-python.readthedocs.org/waits.html#explicit-waits 

    markup = driver.page_source 
    driver.close() 
    soup = bs(markup, "html.parser") 
    return soup