删除日期不落入任何日期范围的行

Question

因此，我有一个2列数据框datetime和value，我想删除所有不属于至少一个日期范围的行。

E.g.比方说，我的有效日期范围被表示为元组

valid_date_ranges = [ 
         (2017-01-01 00:00:00.00, 2017-01-03 15:00:00.00), 
         (2017-01-04 03:25:00.00, 2017-01-06 22:56:00.00), 
         ... 
        ] 

，我有一个数据帧作为这样

datetime    value 
2017-01-01 00:00:00.00 1234 
2017-01-01 00:01:00.00 13241526 
2017-01-01 10:02:00.00 356356 
2017-01-01 10:03:00.00 17435 
2017-01-01 10:04:00.00 5234515 
2017-01-01 10:05:00.00 52452435 
... 
2017-01-03 14:59:00.00 156256 
2017-01-03 15:00:00.00 665654 
2017-01-03 15:01:00.00 890656  * 
2017-01-03 15:02:00.00 698765  * 
2017-01-03 15:03:00.00 6574  * 
... 
2017-01-04 03:23:00.00 6541632  * 
2017-01-04 03:24:00.00 1234  * 
2017-01-04 03:25:00.00 4657347 
2017-01-04 03:26:00.00 765 
2017-01-04 03:27:00.00 870089 
... 

我想在结束与明星删除行，因为它们不落入任何日期范围。

Answer 1

这里有一个办法：

# sample df and ranges to exclude, per OP 
datetime    value 
"2017-01-01 00:00:00.00" 1234 
"2017-01-01 00:01:00.00" 13241526 
"2017-01-01 10:02:00.00" 356356 
"2017-01-01 10:03:00.00" 17435 
"2017-01-01 10:04:00.00" 5234515 
"2017-01-01 10:05:00.00" 52452435 
"2017-01-03 14:59:00.00" 156256 
"2017-01-03 15:00:00.00" 665654 
"2017-01-03 15:01:00.00" 890656  
"2017-01-03 15:02:00.00" 698765  
"2017-01-03 15:03:00.00" 6574  
"2017-01-04 03:23:00.00" 6541632  
"2017-01-04 03:24:00.00" 1234 
"2017-01-04 03:25:00.00" 4657347 
"2017-01-04 03:26:00.00" 765 
"2017-01-04 03:27:00.00" 870089 

df = pd.read_clipboard(parse_dates=True, index_col='datetime') 

valid_date_ranges = [("2017-01-01 00:00:00.00", "2017-01-03 15:00:00.00"), 
        ("2017-01-04 03:25:00.00", "2017-01-06 22:56:00.00")] 
dranges = [pd.date_range(start, end, freq='min') for start, end in valid_date_ranges] 

现在下降未在该日期范围发现行，并建立剩余行新filtered数据帧：

filtered = pd.DataFrame() 
for drange in dranges: 
    filtered = pd.concat([filtered,df.drop(df.index[~df.index.isin(drange)])]) 

print(filtered) 
         value 
datetime      
2017-01-01 00:00:00  1234 
2017-01-01 00:01:00 13241526 
2017-01-01 10:02:00 356356 
2017-01-01 10:03:00  17435 
2017-01-01 10:04:00 5234515 
2017-01-01 10:05:00 52452435 
2017-01-03 14:59:00 156256 
2017-01-03 15:00:00 665654 
2017-01-04 03:25:00 4657347 
2017-01-04 03:26:00  765 
2017-01-04 03:27:00 870089 

Answer 2

这里的另一种方法。

使数据帧

valid_date_ranges = [('2017-01-01 00:00:00.00', '2017-01-03 15:00:00.00'), 
        ('2017-01-04 03:25:00.00', '2017-01-06 22:56:00.00'), 
        ] 
df=pd.DataFrame({'datetime': 
        [ 
        '2017-01-01 00:00:00.00', 
        '2017-01-03 00:01:00.00', 
        '2017-01-03 15:00:00.00', 
        '2017-01-04 01:03:00.00', 
        '2017-01-04 02:03:00.00', 
        '2017-01-04 03:25:00.00', 
        '2017-01-06 22:56:00.00', 
        '2017-01-06 23:56:00.00', 
          ], 
        'value':[1234,1324,1526, 
          356356, 
          17435,5234,515,52452435]})  
df=df.sort_values('datetime').set_index('datetime') 

标签的行为*

for i,idx in enumerate(df.index): 
    for r in valid_date_ranges: 
     if r[0]==idx: 
      star=True 
      break 
     elif r[1]==df.index[i-1]: 
      star=False 
      break 
     else: 
      star=True 
    if star: 
     df.loc[idx,'star']='*' 

输出：

      value star 
datetime        
2017-01-01 00:00:00.00  1234 * 
2017-01-03 00:01:00.00  1324 * 
2017-01-03 15:00:00.00  1526 * 
2017-01-04 01:03:00.00 356356 NaN 
2017-01-04 02:03:00.00  17435 * 
2017-01-04 03:25:00.00  5234 * 
2017-01-06 22:56:00.00  515 * 
2017-01-06 23:56:00.00 52452435 NaN 

删除与星行

df=df.loc[df.loc[:,'star']!="*",:] 

输出：

      value star 
datetime        
2017-01-04 01:03:00.00 356356 NaN 
2017-01-06 23:56:00.00 52452435 NaN