Python：如何搜索推文并在数据库中存储？

Question

我已经有了一个很好的Python脚本，它可以打印出来自给定用户名的过去200条推文。

但是，我想对其进行修改，以便它会收集过去200条包含特定哈希标签（来自任何用户名）的推文，然后我想将这些结果存储在数据库中。

任何人都可以提供关于如何修改下面的代码的建议吗？

import sys 
import operator 
import requests 
import json 
import twitter 

twitter_consumer_key = 'XXXX' 
twitter_consumer_secret = 'XXXX' 
twitter_access_token = 'XXXX' 
twitter_access_secret = 'XXXX' 

twitter_api = twitter.Api(consumer_key=twitter_consumer_key, consumer_secret=twitter_consumer_secret, access_token_key=twitter_access_token, access_token_secret=twitter_access_secret) 

statuses = twitter_api.GetUserTimeline(screen_name=handle, count=200, include_rts=False) 

for status in statuses: 
    if (status.lang == 'en'): 
    print status 

Answer 1

不熟悉twitter包，但这可能是一个建议，你可以继续工作。取决于你想如何保存推文，你可以用你想要的方式替换“打印状态”。 但是，这只允许您过滤200条推文，而不是获取包含特定哈希标签的200条推文。

import sys 
import operator 
import requests 
import json 
import twitter 

twitter_consumer_key = 'XXXX' 
twitter_consumer_secret = 'XXXX' 
twitter_access_token = 'XXXX' 
twitter_access_secret = 'XXXX' 

twitter_api = twitter.Api(consumer_key=twitter_consumer_key, consumer_secret=twitter_consumer_secret, access_token_key=twitter_access_token, access_token_secret=twitter_access_secret) 

statuses = twitter_api.GetUserTimeline(screen_name=handle, count=200, include_rts=False) 

tag_list = ["Xmas", "Summer"] 
for status in statuses: 
    if (status.lang == 'en'): 
    #assume there exists a hashtag in the tweet 
    for hashtag in status.entities.hashtags: 
     if hashtag.text in tag_list: 
     print status 

Answer 2

我附上一个Java代码，将打印出过去100个鸣叫包括“#engineeringproblems”＃标签（来自任何用户）。您需要在库中添加twitter API'twitter4J'。

API下载链路http://twitter4j.org/en/index.html#download

Java源代码：

public static void main(String[] args) { 

    ConfigurationBuilder cb = new ConfigurationBuilder(); 
    cb.setDebugEnabled(true) 
    .setOAuthConsumerKey("xxxx") 
    .setOAuthConsumerSecret("xxxx") 
    .setOAuthAccessToken("xxxx") 
    .setOAuthAccessTokenSecret("xxxx"); 

    Twitter twitter = new TwitterFactory(cb.build()).getInstance(); 
    Query query = new Query("#engineeringproblems "); 
    int numberOfTweets = 100; 
    long lastID = Long.MAX_VALUE; 
    ArrayList<Status> tweets = new ArrayList<Status>(); 

    while (tweets.size() < numberOfTweets) { 
     if (numberOfTweets - tweets.size() > 100) { 
      query.setCount(100); 
     } else { 
      query.setCount(numberOfTweets - tweets.size()); 
     } 
     try { 
      QueryResult result = twitter.search(query); 
      tweets.addAll(result.getTweets()); 
      System.out.println("Gathered " + tweets.size() + " tweets" + "\n"); 
      for (Status t : tweets) { 
       if (t.getId() < lastID) { 
        lastID = t.getId(); 
       } 
      } 

     } catch (TwitterException te) { 
      System.out.println("Couldn't connect: " + te); 
     }; 
     query.setMaxId(lastID - 1); 
    } 
    for (int i = 0; i < tweets.size(); i++) { 
     Status t = (Status) tweets.get(i); 


     String user = t.getUser().getScreenName(); 
     String msg = t.getText(); 

     System.out.println(i + " USER: " + user + " wrote: " + msg + "\n"); 
    } 
} 

Answer 3

很抱歉，但我真的一直在寻找一个Python的解决方案，我相信我终于找到它，并成功地进行了测试。代码如下。仍然在寻找一种方法来修改脚本以将每行输入到SQL数据库中，但我希望我可以在其他地方找到它。

PIP安装TwitterSearch

from TwitterSearch import * 
try: 
    tso = TwitterSearchOrder() # create a TwitterSearchOrder object 
    tso.set_keywords(['Guttenberg', 'Doktorarbeit']) # let's define all words we would like to have a look for 
    tso.set_language('de') # we want to see German tweets only 
    tso.set_include_entities(False) # and don't give us all those entity information 

    # it's about time to create a TwitterSearch object with our secret tokens 
    ts = TwitterSearch(
     consumer_key = 'aaabbb', 
     consumer_secret = 'cccddd', 
     access_token = '111222', 
     access_token_secret = '333444' 
    ) 

    # this is where the fun actually starts :) 
    for tweet in ts.search_tweets_iterable(tso): 
     print('@%s tweeted: %s' % (tweet['user']['screen_name'], tweet['text'])) 

except TwitterSearchException as e: # take care of all those ugly errors if there are some 
    print(e)