解压数据框中

Question

的列表元素我有这样的DF：

l1 = ['a', 'b', 'c'] 
l2 = ['x', ['y1', 'y2', 'y3'], 'z'] 
df = pd.DataFrame(list(zip(l1, l2)), columns = ['l1', 'l2']) 

结果：

l1   l2 
0 a    x 
1 b [y1, y2, y3] 
2 c    z 

我需要的是解压在L2内部列表，并在这样的L1普及相应的值：

l1 l2 
0 a x 
1 b y1 
2 b y2 
3 b y3 
4 c z 

这样做的正确方法是什么？ 谢谢。

Answer 1

我想你可以通过嵌套listsstr.len和平板值由chain使用numpy.repeat由legths重复值：

from itertools import chain 

df1 = pd.DataFrame({ 
     "l1": np.repeat(df.l1.values, df.l2.str.len()), 
     "l2": list(chain.from_iterable(df.l2))}) 
print (df1) 
    l1 l2 
0 a x 
1 b y1 
2 b y2 
3 b y3 
4 c z 

时序：

#[100000 rows x 2 columns] 
np.random.seed(10) 
N = 100000 
l1 = ['a', 'b', 'c'] 
l1 = np.random.choice(l1, N) 
l2 = [list(tuple(string.ascii_letters[:np.random.randint(1, 10)])) for _ in np.arange(N)] 
df = pd.DataFrame({"l1":l1, "l2":l2}) 
df.l2 = df.l2.apply(lambda x: x if len(x) !=1 else x[0]) 
#print (df) 


In [91]: %timeit (pd.DataFrame([(left, right) for outer in zip(l1, l2) for left, right in zip_longest(*outer, fillvalue=outer[0])])) 
1 loop, best of 3: 242 ms per loop 

In [92]: %timeit (pd.DataFrame({ "l1": np.repeat(df.l1.values, df.l2.str.len()), "l2": list(chain.from_iterable(df.l2))})) 
10 loops, best of 3: 84.6 ms per loop 

结论：

numpy.repeat是3 times更快，因为zip_longest解决方案在更大的df中。

编辑：

对于循环版本进行比较是necessery小DF，因为很慢：

#[1000 rows x 2 columns] 
np.random.seed(10) 
N = 1000 
l1 = ['a', 'b', 'c'] 
l1 = np.random.choice(l1, N) 
l2 = [list(tuple(string.ascii_letters[:np.random.randint(1, 10)])) for _ in np.arange(N)] 
df = pd.DataFrame({"l1":l1, "l2":l2}) 
df.l2 = df.l2.apply(lambda x: x if len(x) !=1 else x[0]) 
#print (df) 

def alexey(df): 
    df2 = pd.DataFrame(columns=df.columns,index=df.index)[0:0] 

    for idx in df.index: 
     new_row = df.loc[idx, :].copy() 
     for res in df.ix[idx, 'l2']: 
      new_row.set_value('l2', res) 
      df2.loc[len(df2)] = new_row 
    return df2 

print (alexey(df)) 

In [20]: %timeit (alexey(df)) 
1 loop, best of 3: 11.4 s per loop 

In [21]: %timeit pd.DataFrame([(left, right) for outer in zip(l1, l2) for left, right in zip_longest(*outer, fillvalue=outer[0])]) 
100 loops, best of 3: 2.57 ms per loop 

In [22]: %timeit pd.DataFrame({ "l1": np.repeat(df.l1.values, df.l2.str.len()), "l2": list(chain.from_iterable(df.l2))}) 
The slowest run took 4.42 times longer than the fastest. This could mean that an intermediate result is being cached. 
1000 loops, best of 3: 1.41 ms per loop 

Answer 2

您可以使用嵌套列表理解itertools.zip_longest。

import pandas as pd 

from itertools import zip_longest 

l1 = ['a', 'b', 'c'] 
l2 = ['x', ['y1', 'y2', 'y3'], 'z'] 

expanded = [(left, right) for outer in zip(l1, l2) 
          for left, right in zip_longest(*outer, fillvalue=outer[0])] 

pd.DataFrame(expanded) 

结果是...

0 1 
0 a x 
1 b y1 
2 b y2 
3 b y3 
4 c z 

对我来说这是对过长的列表比较的边界。还假设l1没有列表，并将进行填充。

Answer 3

蛮力，遍历数据框：

for idx in df.index: 
    # This transforms the item in "l2" into an iterable list 
    item = df.loc[idx, "l2"] if isinstance(df.loc[idx, "l2"], (list, tuple)) else [df.loc[idx, "l2"]] 
    for element in item: 
     print(df.loc[idx, "l1"], element) 

回报

a x 
b y1 
b y2 
b y3 
c z 

Answer 4

对于列不constatnt数DataFrames我现在做这样的事情：

l1 = ['a', 'b', 'c'] 
l2 = ['x', ['y1', 'y2', 'y3'], 'z'] 
df = pd.DataFrame(list(zip(l1, l2)), columns = ['l1', 'l2']) 

df2 = pd.DataFrame(columns=df.columns,index=df.index)[0:0] 

for idx in df.index: 
    new_row = df.loc[idx, :].copy() 
    for res in df.ix[idx, 'l2']: 
     new_row.set_value('l2', res) 
     df2.loc[len(df2)] = new_row 

它作品，但这看起来很像bruteforce。