循环将多个列表追加到数据框python

我必须requests.get()雅虎利用YQL返回json对象的两个网址。我回来了json对象，我存储到list()。然后我循环解析数据并创建一个dic，然后创建一个熊猫数据框架。发生了只有一个列表正在追加到数据框。看起来像在上一次迭代中，第二个列表覆盖第一个列表。在这一点上，我无法弄清楚如何遍历列表中的两个元素append()。这里是我的代码...循环将多个列表追加到数据框python

import requests 
import pandas as pd 


urls = ['https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20yahoo.finance.historicaldata%20where%20symbol%20in%20(%22DIA%22%2C%22SPY%22%2C%22IWN%22)%20and%20startDate%20%3D%20%222015-01-01%22%20and%20endDate%20%3D%20%222015-10-31%22&format=json&diagnostics=true&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys&callback=', 
     'https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20yahoo.finance.historicaldata%20where%20symbol%20in%20(%22DIA%22%2C%22SPY%22%2C%22IWN%22)%20and%20startDate%20%3D%20%222015-11-01%22%20and%20endDate%20%3D%20%222016-08-31%22&format=json&diagnostics=true&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys&callback='] 


for url in urls: 
    data = requests.get(url) 
    data_json = data.json() 

    quote_list = [] 
    for quote in data_json['query']['results']['quote']: 
     quote_dic = {'symbol': quote['Symbol'], 
       'date': quote['Date'], 
       'volume': quote['Volume'], 
       'low': quote['Low'], 
       'high': quote['High'], 
       'open': quote['Open'], 
       'close': quote['Close'], 
       'adj_close': quote['Adj_Close']} 

     quote_list.append(quote_dic) 

    quote_df = pd.DataFrame(quote_list) 

quote_df.to_csv('stocks.csv')

我需要能够将整个list()追加到数据帧。这个代码的修正是什么？

来源

2016-11-08 Gilbert

为什么存储在列表中的JSON？ json本质上是一本字典，可以在不解析或翻译的情况下加载到字典中。 – zenlc2000

我可以将'json'直接附加到数据框吗？ – Gilbert

但是，我不太熟悉熊猫数据框，根据http://stackoverflow.com/questions/31695108/how-to-append-a-dictionary-to-a-pandas-dataframe它看起来很简单。 – zenlc2000

只需创建dataframes的列表，并在循环结束Concat的他们：

df_list = [] 
for url in urls: 
    data = requests.get(url) 
    data_json = data.json() 

    df = pd.DataFrame(data_json['query']['results']['quote']) 
    df_list.append(df) 

quote_df = pd.concat(df_list) 
quote_df.to_csv('stocks.csv')

来源

2016-11-08 20:52:42

这个怎么解决呢？

import urllib 
import re 
import json 

symbolslist = open("C:/Users/your_path_here/Desktop/stock_symbols.txt").read() 
symbolslist = symbolslist.split("\n") 

for symbol in symbolslist: 
    myfile = open("C:/Users/your_path_here/Desktop/" +symbol +".txt", "w+") 
    myfile.close() 

    htmltext = urllib.urlopen("http://www.bloomberg.com/markets/chart/data/1D/"+ symbol+ ":US") 
    data = json.load(htmltext) 
    datapoints = data["data_values"] 

    myfile = open("C:/Users/rshuell001/Desktop/symbols/" +symbol +".txt", "a") 
    for point in datapoints: 
     myfile.write(str(symbol+","+str(point[0])+","+str(point[1])+"\n")) 
    myfile.close()

在这个文件中的 “C：/Users/your_path_here/Desktop/symbols/amex.txt” 您有以下代号

ibm 
sbux 
msft

来源

2016-11-10 04:21:54

上面的解决方案运行良好，因为我连接到雅虎API并插入到Postgresql而不保存数据到硬盘。我有一个问题，虽然在这[后]（http://stackoverflow.com/questions/40519046/insert-dataframe-into-postgresql-sqlalchemy-with-idx-autoincrement） – Gilbert

循环将多个列表追加到数据框python

回答

相关问题