2013-04-22 64 views
0

我从以下网站解析html:http://www.asusparts.eu/partfinder/Asus/All在One/E系列中我只是想知道是否有任何方法可以探索python中的解析属性? 例如..下面的代码输出如下:在Python中探索一个属性

datas = s.find(id='accordion') 

    a = datas.findAll('a') 

    for data in a: 

      if(data.has_attr('onclick')): 
       model_info.append(data['onclick']) 
       print data 

[OUTPUT]

<a href="#Bracket" onclick="getProductsBasedOnCategoryID('Asus','Bracket','ET10B','7138', this, 'E Series')">Bracket</a> 

这些是我想检索值:

nCategoryID = Bracket 

nModelID = ET10B 

family = E Series 

作为页面是从AJAX呈现的,他们正在使用脚本源,从脚本文件中生成以下url:

url = 'http://json.zandparts.com/api/category/GetCategories/' + country + '/' + currency + '/' + nModelID + '/' + family + '/' + nCategoryID + '/' + brandName + '/' + null 

我怎样才能检索上面列出的3个值?


[编辑]


import string, urllib2, urlparse, csv, sys 
from urllib import quote 
from urlparse import urljoin 
from bs4 import BeautifulSoup 
from ast import literal_eval 

changable_url = 'http://www.asusparts.eu/partfinder/Asus/All%20In%20One/E%20Series' 
page = urllib2.urlopen(changable_url) 
base_url = 'http://www.asusparts.eu' 
soup = BeautifulSoup(page) 

#Array to hold all options 
redirects = [] 
#Array to hold all data 
model_info = [] 

print "FETCHING OPTIONS" 
select = soup.find(id='myselectListModel') 
#print select.get_text() 


options = select.findAll('option') 

for option in options: 
    if(option.has_attr('redirectvalue')): 
     redirects.append(option['redirectvalue']) 

for r in redirects: 
    rpage = urllib2.urlopen(urljoin(base_url, quote(r))) 
    s = BeautifulSoup(rpage) 
    #print s 



    print "FETCHING MAIN TITLE" 
    #Finding all the headings for each specific Model 
    maintitle = s.find(id='puffBreadCrumbs') 
    print maintitle.get_text() 

    #Find entire HTML container holding all data, rendered by AJAX 
    datas = s.find(id='accordion') 

    #Find all 'a' tags inside data container 
    a = datas.findAll('a') 

    #Find all 'span' tags inside data container 
    content = datas.findAll('span') 

    print "FETCHING CATEGORY" 

    #Find all 'a' tags which have an attribute of 'onclick' Error:(doesn't display anything, can't seem to find 
    #'onclick' attr 
    if(hasattr(a, 'onclick')): 
     arguments = literal_eval('(' + a['onclick'].replace(', this', '').split('(', 1)[1]) 
     model_info.append(arguments) 
     print arguments #arguments[1] + " " + arguments[3] + " " + arguments[4] 


    print "FETCHING DATA" 
    for complete in content: 
     #Find all 'class' attributes inside 'span' tags 
     if(complete.has_attr('class')): 
      model_info.append(complete['class']) 

      print complete.get_text() 

    #Find all 'table data cells' inside table held in data container  
    print "FETCHING IMAGES" 
    img = s.find('td') 

    #Find all 'img' tags held inside these 'td' cells and print out 
    images = img.findAll('img') 
    print images 

我增加了一个错误行哪里出了问题奠定...

回答

1

你可以parse that as a Python literal,如果删除了this,部分来自它,并只取括号之间的所有内容:

from ast import literal_eval 

if data.has_attr('onclick'): 
    arguments = literal_eval('(' + data['onclick'].replace(', this', '').split('(', 1)[1]) 
    model_info.append(arguments) 
    print arguments 

我们删除了this参数,因为它不是一个有效的python字符串,你不想拥有它。

演示:

>>> literal_eval('(' + "getProductsBasedOnCategoryID('Asus','Bracket','ET10B','7138', this, 'E Series')".replace(', this', '').split('(', 1)[1]) 
('Asus', 'Bracket', 'ET10B', '7138', 'E Series') 

现在你有一个Python元组,并可以挑选出你喜欢的任何值。

你想在指数1,2和4的值,例如:

nCategoryID, nModelID, family = arguments[1], arguments[3], arguments[4] 
+0

请看看我的编辑代码.. – ash 2013-04-22 13:11:08

+0

你仍然只是打印'data',你从'onclick'属性提取不算什么。 – 2013-04-22 13:36:45

+0

改为打印'arguments'; 'literal_eval'的返回值。 – 2013-04-22 13:38:41

1

到的Martijn的答案相似,但使原始的利用pyparsing(即,它可以被细化到识别功能,只采取与括号引用的字符串):

from bs4 import BeautifulSoup 
from pyparsing import QuotedString 
from itertools import chain 

s = '''<a href="#Bracket" onclick="getProductsBasedOnCategoryID('Asus','Bracket','ET10B','7138', this, 'E Series')">Bracket</a>''' 
soup = BeautifulSoup(s) 
for a in soup('a', onclick=True): 
    print list(chain.from_iterable(QuotedString("'", unquoteResults=True).searchString(a['onclick']))) 
# ['Asus', 'Bracket', 'ET10B', '7138', 'E Series']