在BeautifulSoup中只提取表（td）中的特定行和列，python

我想从网站上抓取数据并将其写入CSV。示例代码包含“列名”和“值”。在BeautifulSoup中只提取表（td）中的特定行和列，python

<div class="table-responsive"> 
    <table class="table table-bordered table-hover"> 
    <tbody> 
    <!-- required First 3 column names--> 
     <tr> 
     <td></td> 
     <td align="center" style="vertical-align:middle;" title=" Number">Number</td> 
     <td align="center" style="vertical-align:middle;" colspan="2" title=" Code ">CODE</td> 
     <td align="center" style="vertical-align:middle;">Name</td> 
     <td align="center" style="vertical-align:middle;" title="Category ">Category</td> 
     <td align="center" style="vertical-align:middle;" title="Variation ">Variation</td> 
     </tr> 
     <!-- required --> 
     <tr class="success"> 
     <td align="center" style="vertical-align:middle;"></td> 
     <td align="center" style="vertical-align:middle;"><a href="/Number/3259">3259</a> 
     </td> 
     <td style="width:125px;height:60px;vertical-align:middle;" <a href="/CODE/ABC123">ABC123</a> 
     </td> 
     <td align="center" style="vertical-align:middle;"><a title="CODE ABC123" href="/CODE/ABC123">LAND </a> 
     </td> 
     <td align="center" style="vertical-align:middle;" class="success">3 - Design Reference</td> 
     <td align="center" style="vertical-align:middle;" class="success">2 - Production Item</td> 
     </tr> 
     <!-- not-required --> 
     <tr> 
     <td colspan="7"> 
      <div id="BMCRL_36" class="collapse"> 
      <div class="list-group"> 
       <a class="list-group-item"> 
       <h4 class="list-group-item-heading"> Number 3259 - Reference Number ABC123</h4> 
       <p class="list-group-item-text"></p> 
       </a> 
       <a class="list-group-item"> 
       <h4 class="list-group-item-heading">CODE ABC123 </h4> 
       <p class="list-group-item-text"></p> 
       </a> 
      </div> 
      </div> 
     </td> 
     </tr> 
     <!-- required --> 
     <tr class=""> 
     <td align="center" style="vertical-align:middle;"></td> 
     <td align="center" style="vertical-align:middle;"><a href="/Number/3260">3260</a> 
     </td> 
     <td style="width:125px;height:60px;vertical-align:middle;" <a href="/CODE/XYZ453">XYZ453</a> 
     </td> 
     <td align="center" style="vertical-align:middle;"><a title="CODE XYZ453" href="/CODE/XYZ453">WATER </a> 
     </td> 
     <td align="center" style="vertical-align:middle;" class="success">3 Control Reference</td> 
     <td align="center" style="vertical-align:middle;" class="success">2 Item</td> 
     </tr> 

    </tbody> 
    </table> 
</div>

输出应该从第一'TD'标签像

列名时，我用这个

from bs4 import BeautifulSoup 
soup = BeautifulSoup(open('code.html'),'lxml') 
col = soup.find('tr').find_all('td') 
for c in col: 
    print(c.get_text())

所有的列打印。相反，我只需要，

['Number', 'Code', 'Name']

我不需要'TR'其中包含 “合并单元格” （第三TR）

还要去掉最后两倍中的值，

tr = soup.findChildren('tr') 
for t in tr: 
    td = t.findChildren('td') 
    for child in td: 
     print(child.text)

，我再次得到整个数据，包括列和价值观，我的期望是

['3259', 'ABC123', 'LAND'] 
      and 
['3260', 'XYZ453','WATER']

通过去除
['3 - Design Reference','2 - Production Item']

来源

2016-03-02 Suresh2692

片find_all()结果只得到相关的列和单元格。为了仅获取相关行，请检查class属性的存在。

工作样本：

import requests 
from bs4 import BeautifulSoup 

data = """ 
your HTML here 
""" 

soup = BeautifulSoup(data, "lxml") 

headers = [c.get_text() for c in soup.find('tr').find_all('td')[1:4]] 

data = [[cell.get_text(strip=True) for cell in row.find_all('td')[1:4]] 
     for row in soup.find_all("tr", class_=True)] 

print(headers) 
print(data)

打印：

['Number', 'CODE', 'Name'] 
[['3259', 'ABC123', 'LAND'], ['3260', 'XYZ453', 'WATER']]

来源

2016-03-02 18:33:20 alecxe

在BeautifulSoup中只提取表（td）中的特定行和列，python

回答

相关问题