2017-10-12 143 views
0

我正在解析具有许多超链接的网页。能够通过查找<a>标签中的所有'href'来使用美丽优点提取超链接。接下来,我试图获取所有这些链接的“标题”,这些链接显示为标签的标签内容。 [HTML新手,我认为这个尾部描述被称为标记内容吗?]下面是一个例子。字符串'哪种课堂设置适合您?'需要提取。感谢帮助。使用BeautifulSoup解析HTML标签

<a href="https://www.nytimes.com/video/education/100000005108366/classroom-waldorf-montessori-public.html"> 
Which Classroom Setting Suits You? 
</a> 

代码

import requests 
from bs4 import BeautifulSoup 
import sys 
import codecs 
sys.stdout = codecs.getwriter('utf8')(sys.stdout) 

url = 'https://www.nytimes.com' 

r = requests.get(url) 

html_as_string = r.text 

soup = BeautifulSoup(html_as_string, 'html.parser') 

for link in soup.find_all('a'): 
    print soup.a.string  ##Prints "LEARN MORE" 
    print soup.a.text  ##Prints "LEARN MORE" 
    print (link.get('text'))##prints "None" 
+0

[解析BeautifulSoup HTML标签(可能的重复https://stackoverflow.com/ questions/27581584/parsing-beautifulsoup-html-tag) – thatrockbottomprogrammer

回答

2

代码:

from bs4 import BeautifulSoup 

data = ''' 
<a href="https://www.nytimes.com/video/education/100000005108366/classroom-waldorf-montessori-public.html"> 
Which Classroom Setting Suits You? 
</a> 
''' 

bs = BeautifulSoup(data, 'html.parser') 
href = bs.find('a').text 
print(href) 

输出:

Which Classroom Setting Suits You? 

这里是你的第二个编辑的问题后,我的代码: 目前剧本全部提取从网站链接并打印文本。您需要优化脚本以专注于您想要解析的特定href链接。

这里是你的第二个编辑的问题后,我的代码:

import requests 
from bs4 import BeautifulSoup 

url = 'https://www.nytimes.com' 
r = requests.get(url) 
html_as_string = r.text 
soup = BeautifulSoup(html_as_string, 'html.parser') 
for link in soup.find_all('a'): 
    print link.text 

输出:

LEARN MORE » 
Skip to content 
Skip to navigation 
中文 (Chinese) 
Español 





Today’s Paper 
Video 
World 
U.S. 
Politics 
N.Y. 
Business 
Business 
Opinion 
Opinion 
Tech 
Science 
Health 
Sports 
Sports 
Arts 
Arts 
Style 
Style 
Food 
Food 
Travel 
Magazine 
T Magazine 
Real Estate 
Trump Allows for Cheaper, Less Regulated Health Plans 
 Comments 




1 
Open the door to relaxing A.C.A. rules on coverage. 





2 
Stop sending department officials to enrollment events. 





3 
Shut down the A.C.A. website for periods of time. 





4 
Cut funding to groups that help people enroll. 





5 
Slash spending on advertising and promotion. 





Here Are 11 Ways Trump Is Undermining Obamacare 
Trump Warns Puerto Rico: Federal Help Can’t Stay ‘Forever’ 
 Comments 


Obliterated: One of the Few Tropical Rain Forests in the U.S. 
U.S. Will Withdraw From Unesco, Citing ‘Anti-Israel Bias’ 
What’s It Like to Cover Trump? Our Reporters Answer Questions at 6:30 P.M. Eastern 
California Fires Burn ‘Faster Than Firefighters Can Run’ 
 Comments 
‘Thank God We Left’: Californians Describe Escapes 


Satellite Images Show 1,800 Buildings Destroyed by Fire in Santa Rosa 
The Daily 360 













1:32 



            Sifting Through the Charred Ruins        






















More in 
The Daily 360 » 


Weinstein Company Has Been Aware of Payouts Since 2015 

 Comments 
Facing Down the Network That Protected Weinstein 
 Comments 
How Did NBC Miss Out on a Weinstein Exposé? 
Palestinian Factions Sign Deal Paving Way for Joint Rule 
 Comments 
The Bronx’s Quiet, Brutal War With Opioids 
Comments 

California Today: A Rising Death Toll From Wildfires 
 Comments 

Your Friday Briefing 



Audio 

     Listen to ‘The Daily’ 

Apple Podcasts 
Radio Public 
Stitcher 
Smarter Living 
What to Know About Having a Microwedding 

How Would That Couch Look at Home? Check Your Phone 

 Comments 

Journalism that matters. More essential than ever. 
Subscribe to the Times 

From the Magazine 

Philip Pullman Returns to His Fantasy World 
 Comments 
U.S. Woman and Family Freed From Afghan Militants 

 Comments 
South Carolina Sidesteps the Law, and the Poor Go to Jail 

 Comments 
Protesters in N.F.L. May Have an Ally: Labor Law 

 Comments 
Different Ads, Different Ethnicities, Same Car 

Genes for Skin Color Rebut Notions of Race, Study Finds 

 Comments 
Citing Supreme Court, Judge Questions Menendez Case 

 Comments 
Leader of Philippines Threatens to Expel E.U. Diplomats 
Qatari Businessman Accused of Bribing Ex-FIFA Chief 
Yankees Complete Comeback From 0-2 to Reach A.L.C.S. 
Seychelles Reports a Case of Plague 
Fed Still Puzzled by Inflation, but Rate Increase Is on Track 

Morning Briefing: Australia Edition 

Morning Briefing: Asia Edition 

Morning Briefing: Europe Edition 

California Today 

Morning Briefing 
Got a confidential news tip? 
Learn more 





Aussie Rules: Readers Respond 





Vegemite, the Beloved Australian Spread, Goes Upmarket 





Australia’s New Offer to Refugees: A Different Detention Center 





She Is New Yorkier Than You 





Olympic Tales From a 105-Year-Old Baron 





Art Review: Collecting Strokes of Genius 
Opinion 
Mr. Trump, I Live in South Korea. You’re Scaring Me. 

The Trumps, The Poodle, The Sex Scandal 

 Comments 
Blow: Trump’s War Games 
Kristof: North Korea Fears 
Leonhardt: Falling Behind 
Rosenthal: Trump’s Latest Outrage Against Puerto Rico 
Stephens: Weinstein Enablers 
Texas Should Not Execute Robert Pruett Tonight 

Editorial: Trump Alone Can Order a Nuclear Strike 
Edsall: Democrats Play Checkers, Trump Plays Chess 
Greenhouse: Church Over State on Contraception 
Has America Forgotten the Virgin Islands? 
Congress’s Un-American, Bad-for-Israel Agenda 
From Amazon, a Change That Hurts Authors 
Ben Shapiro’s Hollow Bravery 
What Makes Eminem’s Anti-Trump Rap Different 
Getting Help Where It’s Needed in Puerto Rico 
New York Times Recipe Commenters (Politely) Spill Their Guts 
Smelly and Dirty and, in Bombing Trial, Suddenly a Star 





Times Insider » 
New York Times Recipe Commenters (Politely) Spill Their Guts 
The Crossword » 
Play Today’s Puzzle 



The Crossword » 
Play Today’s Puzzle 



Wordplay » 
timesvideo 
explore all videos » 
Television 




‘Dynasty’ or ‘Dallas’? Squaring Off 

Opinion 




You Are Special! Now Stop Being Different 

Science 




Mountain Lions Sometimes Share Food 

Food 




Our Best Pizza Recipes 

Opinion 




Getting Help Where It’s Needed in Puerto Rico 

Mind 




High Blood Pressure Tied to Later Dementia 

Travel 




What Martha Stewart Can’t Travel Without 

Opinion 




Publishing’s Unfair Gray Market 

Live 




How to Stop Bleeding and Save a Life 

Television 




Review: Netflix’s ‘Mindhunter’ 

Opinion 




Has America Forgotten the Virgin Islands? 

N.Y./Region 




Lawsuits Seek Access to College Websites 

World » 





     For Dignity and Development, East Africa Curbs Used Clothes Imports   


      Tech and Real Estate Lift the Fortunes of China’s Richest   

      Palestinian Factions, Hamas and Fatah, Sign Unity Deal   
Business Day » 





     Sheryl Sandberg Says Facebook Will Hand Over More Data on Russia-Linked Ads   


      Different Ads, Different Ethnicities, Same Car   

      Kobe Steel Problems May Be More Widespread, Raising Fears on High-Speed Rail   
Opinion » 





     Linda Greenhouse: On Contraception, It’s Church Over State   


      Editorial: Mr. Trump Alone Can Order a Nuclear Strike. Congress Can Change That.   

      Charles M. Blow: Trump’s War Games   
U.S. » 





     We’re Tracking Ways Trump Is Undermining Obamacare. Here Are 11.   


      Another Victim of Hurricane Maria: Puerto Rico’s Treasured Rainforest   

      California Wildfires Burn ‘Faster Than Firefighters Can Run’   
Technology » 


     Sheryl Sandberg Says Facebook Will Hand Over More Data on Russia-Linked Ads   


      State of the Art: The Frightful Five Want to Rule Entertainment. They Are Hitting Limits.   

      The Shift: We Asked Facebook 12 Questions About the Election, and Got 5 Answers   
Arts » 





     Critic’s Notebook: Harvey Weinstein Is Gone. But Hollywood Still Has a Problem.   


      Rose McGowan’s Twitter Account Locked After Posts About Weinstein   

      ‘Dynasty’ or ‘Dallas’? Squaring Off in the Lily Pond   
Politics » 





     Foiled in Congress, Trump Signs Order to Undermine Obamacare   


      U.S. Will Withdraw From Unesco, Citing Its ‘Anti-Israel Bias’   

      Trump’s Warning on Limits to U.S. Aid in Puerto Rico Adds ‘Insult to Injury,’ Mayor Says   
Fashion & Style » 





     Unbuttoned: Fashion’s Gossip Addiction   


      With...Tom Hanks: Hollywood’s Most Decent Fella on Weinstein, Trump and History   

      Social Q’s: What Happens When a Will is Unsigned? (It’s Not Fun)   
Movies » 





     All the Films of Studio Ghibli, Ranked   


      Critic’s Notebook: Harvey Weinstein Is Gone. But Hollywood Still Has a Problem.   

      Rose McGowan’s Twitter Account Locked After Posts About Weinstein   
New York » 





     The Bronx’s Quiet, Brutal War With Opioids   


      Cy Vance Defends Decision Not to Pursue Case Against Harvey Weinstein   

      College Websites Must Accommodate Disabled Students, Lawsuits Say   
Sports » 





     Yankees 5, Indians 2 | Yankees win series, 3-2: Didi Gregorius Leads Yankees Past Indians and Into A.L.C.S.   


      P.S.G.’s Nasser al-Khelaifi Accused of Bribing Ex-FIFA Chief Valcke   

      N.F.L. Week 6 Picks: Chiefs Over Steelers; Packers Over Vikings   
Theater » 





     ‘Brilliant,’ 41 and Lost to AIDS: The Theater World Asks Why   


      She Was About to Quit Acting. Then She Got the Role of a Lifetime.   

      Review: A Richard III Who Steps Straight Out of the Land of Id   
Science » 





     Matter: Genes for Skin Color Rebut Dated Notions of Race, Researchers Say   


      ScienceTake: How Kangaroo Rats Escape Rattlesnakes   

      Trilobites: Solitary Pumas Turn Out to Be Mountain Lions Who Lunch   
Obituaries » 





     Vincent La Selva, Dauntless Founder of a People’s Opera Troupe, Dies at 88   


      Michel Jouvet, Who Unlocked REM Sleep’s Secrets, Dies at 91   

      Y.A. Tittle, Quarterback Who Led Giants to 3 Title Games, Dies at 90   
Television » 





     ‘Dynasty’ or ‘Dallas’? Squaring Off in the Lily Pond   


      Review: Netflix’s ‘Mindhunter’ Is More Chatter Than Splatter   

      Ask a Showrunner: ‘Mr. Robot’ Season 3 Premiere: Sam Esmail on Parallel Universes and President Trump   
Health » 





     In New Test for Obamacare, Iowa Seeks to Abandon Marketplace   


      Seychelles Reports a Case of Plague   

      Phys Ed: What Half-Marathons Teach Us About Running a Marathon   
Travel » 





     36 Hours: 36 Hours in Aarhus, Denmark   


      Personal Journeys: Seeking Solitude in Japan’s Mountain Monasteries   

      Carry-On: What Martha Stewart Can’t Travel Without   
Books » 





     American Beauties: Riding Shotgun on Martha Gellhorn’s Brave and Comic Adventures   


      Books of The Times: In Ron Chernow’s ‘Grant,’ an American Giant’s Makeover Continues   

      Feature: Philip Pullman Returns to His Fantasy World   
Education » 





     On Campus: Campus Hate Lives on the Internet. Administrators Need to Catch Up.   


      Which Classroom Setting Suits You?   

      Your Money: A Game to Help Students Pay the Right Price for College   
Food » 





     The Pour: Five Common Wine Myths, Debunked   


      Hungry City: Traditional Ethiopian Food in a Cosmopolitan Setting   

      Mike Isabella, Washington’s Restless Restaurateur   
Magazine » 





     Feature: Philip Pullman Returns to His Fantasy World   


      Feature: In Northern Minnesota, Two Economies Square Off: Mining vs. Wilderness   

      Letter of Recommendation: Letter of Recommendation: Dunking   
Real Estate » 





     The Hunt: Seeking the Peace of the Suburbs   


      On the Market: Homes for Sale in New York and Connecticut   

      On the Market: Homes for Sale in New York City   
The Upshot » 





     A Bump Stock Ban Is Popular With the Public. But Experts Have Their Doubts.   


      The N.F.L. Is Now One of the Most Divisive Brands in the U.S.   

      Quality of Mercy: The Word May Be Toxic, but Amnesty Is Everywhere   
Times Insider » 





     Why Tech is Starting to Make Me Uneasy   


      Times Insider: New York Times Recipe Commenters (Politely) Spill Their Guts   

      Smelly and Dirty and, in Bombing Trial, Suddenly a Star   
Automobiles » 





     Wheels: For Electric Car Owners, ‘Range Anxiety’ Gives Way to ‘Charging Time Trauma’   


      Kobe Steel’s Falsified Data Is Another Blow to Japan’s Reputation   

      Different Ads, Different Ethnicities, Same Car   
T Magazine » 





     Home and Work: A Paris Boudoir Where Everything’s Meant to Be Touched   


      Slinky Dresses, Meet Slouchy Trousers   

      The Illustrated Interview: Gloria Vanderbilt   
Real Estate » 
International Rents Per Square Foot 

 Comments 
Search for Homes for Sale or Rent 
Mortgage Calculator 
Most Emailed 
Most Viewed 
Trending 
Recommended for you 
Go to Home Page » 

The New York Times 

World 
U.S. 
Politics 
N.Y. 
Business 
Tech 
Science 
Health 
Sports 
Education 
Obituaries 
Today's Paper 
Corrections 
Today's Opinion 
Op-Ed Columnists 
Editorials 
Op-Ed Contributors 
Letters 
Sunday Review 
Video: Opinion 
Today's Arts 
Art & Design 
Books 
Dance 
Movies 
Music 
N.Y.C. Events Guide 
Television 
Theater 
Video: Arts 
Automobiles 
Crossword 
Food 
Education 
Fashion & Style 
Health 
Jobs 
Magazine 
N.Y.C. Events Guide 
Real Estate 
T Magazine 
Travel 
Weddings & Celebrations 
Reader Center 
Classifieds 
Tools & Services 
N.Y.C. Events Guide 
Multimedia 
Photography 
Video 
NYT Store 
Times Journeys 
Subscribe 
Manage My Account 
NYTCo 
Home Delivery 
Digital Subscriptions 
Crossword 
Email Newsletters 
Alerts 
Gift Subscriptions 
Corporate Subscriptions 
Education Rate 
Mobile Applications 
Replica Edition 

        © 2017 The New York Times Company 

Contact Us 
Work With Us 
Advertise 
Your Ad Choices 
Privacy 
Terms of Service 
Terms of Sale 
Site Map 
Help 
Site Feedback 
Subscriptions 
View Mobile Version 
+0

谢谢。这对您共享的示例代码起作用。不知道为什么它不会在循环中工作....在问题文章中粘贴我的原始脚本。 –

+0

如果您分享了您的代码并显示了代码中发生问题的位置,我将能够提供帮助。 – Ali

+0

谢谢阿里! 你能否建议为什么/它在哪里拾取换行符/空格?这是否意味着\ n \ t \ r是文本的一部分。 现在,我将输出管道输出到一个文件,然后通过另一个python脚本运行flie去除空白行。理想的是找出为什么/这是在爬行.... 谢谢。 –