将多个文件连接成熊猫数据框

Question

我想将两个日期之间的多个文件导入熊猫数据框。但是由此产生的数据帧有多个数据副本，而不是一个副本。

我的代码如下所示：

Mu = pd.DataFrame() 
lis = [] 
for date in daterange: 
    path = 'Z:/directory/to/files' + date + '.txt' 
    frame = pd.read_csv(path,delimiter=' ', skipinitialspace=True,usecols=[0,1,2,3], 
       names = ['date','time','type1','type2'], 
       parse_dates = {'timestamp': ['date','time']}) 

    lis.append(frame) 
Mu = pd.concat(lis, axis =0, ignore_index = True) 

如果我有文件是这样的：

File A: 
20170501 00:00:11 11 1 
20170501 00:00:20 21 2 

File B: 
20170502 00:06:11 31 3 
20170502 00:30:11 41 4 

File C: 
20170503 00:40:11 51 5 
20170503 00:50:11 61 6 

生成的数据帧是这样的：

20170501 00:00:11 11 1 
20170501 00:00:20 21 2 
20170502 00:06:11 31 3 
20170502 00:30:11 41 4 
20170503 00:40:11 51 5 
20170503 00:50:11 61 6  
20170501 00:00:11 11 1 
20170501 00:00:20 21 2 
20170502 00:06:11 31 3 
20170502 00:30:11 41 4 
20170503 00:40:11 51 5 
20170503 00:50:11 61 6 
20170501 00:00:11 11 1 
20170501 00:00:20 21 2 
20170502 00:06:11 31 3 
20170502 00:30:11 41 4 
20170503 00:40:11 51 5 
20170503 00:50:11 61 6 

我想是这样的：

20170501 00:00:11 11 1 
20170501 00:00:20 21 2 
20170502 00:06:11 31 3 
20170502 00:30:11 41 4 
20170503 00:40:11 51 5 
20170503 00:50:11 61 6 

如何创建想要的数据框？

Answer 1

您可以使用drop_duplicates：

Mu = Mu.drop_duplicates() 

输出：

0 20170501 00:00:11 11 1 
1 20170501 00:00:20 21 2 
2 20170502 00:06:11 31 3 
3 20170502 00:30:11 41 4 
4 20170503 00:40:11 51 5 
5 20170503 00:50:11 61 6