同时遍历多个数据帧

Question

我有三个用户的三个数据帧，具有相同的列名称，如时间，罗盘数据，加速计数据，陀螺仪数据和相机平移信息。我想同时遍历所有数据帧以检查特定时间哪个用户执行了摄像机平移并返回用户（例如，在特定时间内已经检测到数据帧平移）。我曾尝试使用破折号来获得并行性，但徒劳无益。下面是我的代码

import pandas as pd 
import glob 
import numpy as np 
import math 
from scipy.signal import butter, lfilter 
order=3 
fs=30 
cutoff=4.0 

data=[] 
gx=[] 
gy=[] 
g_x2=[] 
g_y2=[] 


dataList = glob.glob(r'C:\Users\chaitanya\Desktop\Thesis\*.csv') 
for csv in dataList: 
     data.append(pd.read_csv(csv)) 
for i in range(0, len(data)): 
    data[i] = data[i].groupby("Time").agg(lambda x: x.value_counts().index[0]) 
    data[i].reset_index(level=0, inplace=True) 

def butter_lowpass(cutoff,fs,order=5): 
    nyq=0.5 * fs 
    nor=cutoff/nyq 
    b,a=butter(order,nor,btype='low', analog=False) 
return b,a 
def lowpass_filter(data,cutoff,fs,order=5): 
    b,a=butter_lowpass(cutoff,fs,order=order) 
    y=lfilter(b,a,data) 
return y 

for i in range(0,len(data)): 
    gx.append(lowpass_filter(data[i]["Gyro_X"],cutoff,fs,order)) 
    gy.append(lowpass_filter(data[i]["Gyro_Y"],cutoff,fs,order)) 

    g_x2.append(gx[i]*gx[i]) 
    g_y2.append(gy[i]*gy[i]) 


g_rad=[[] for _ in range(len(data))] 
g_ang=[[] for _ in range(len(data))] 

for i in range(0,len(data)): 
    for j in range(0,len(data[i])): 
      g_ang[i].append(math.degrees(math.atan(gy[i][j]/gx[i][j]))) 


    data[i]["Ang"]=g_ang[i] 


panning=[[] for _ in range(len(data))] 
for i in range(0,len(data)): 
    for j in data[i]["Ang"]: 
     if 0-30<=j<=0+30: 
      panning[i].append("Panning") 
     elif 180-30<=j<=180+30: 
      panning[i].append("left") 
     else: 
      panning[i].append("None") 
    data[i]["Panning"]=panning[i] 
result=[[] for _ in range(len(data))] 
for i in range (0,len(data)): 
    result[i].append(data[i].loc[data[i]['Panning']=='Panning','Ang']) 

Answer 1

我打算假设你想要在时间上同时遍历。无论如何，你都希望你的三个数据框在你想要遍历的维度中有一个索引。

我会生成3个数据帧，并在9秒内产生表示随机秒数的行。

然后，我会将它们与pd.concat和ffill对齐，以便能够引用任何间隙的最后已知数据。

seconds = pd.date_range('2016-08-31', periods=10, freq='S') 

n = 6 
ssec = seconds.to_series() 
sidx = ssec.sample(n).index 

df1 = pd.DataFrame(np.random.randint(1, 10, (n, 3)), 
        ssec.sample(n).index.sort_values(), 
        ['compass', 'accel', 'gyro']) 

df2 = pd.DataFrame(np.random.randint(1, 10, (n, 3)), 
        ssec.sample(n).index.sort_values(), 
        ['compass', 'accel', 'gyro']) 

df3 = pd.DataFrame(np.random.randint(1, 10, (n, 3)), 
        ssec.sample(n).index.sort_values(), 
        ['compass', 'accel', 'gyro']) 

df4 = pd.concat([df1, df2, df3], axis=1, keys=['df1', 'df2', 'df3']).ffill() 
df4 

，那么你可以继续通过iterrows()

for tstamp, row in df4.iterrows(): 
    print tstamp