2
import dask.dataframe as dd
import numpy as np
from dask import delayed
df1 = pd.DataFrame({'a': np.arange(10), 'b': np.random.rand()})
df1 = df1.astype({'a':np.float64})
df2 = pd.DataFrame({'a': np.random.rand(5), 'c': 1})
df1.to_csv('df1.csv')
df2.to_csv('df2.csv')
dd.read_csv('*.csv').compute()
给人内部联接结果:DASK:外部联接从多个CSV文件中读取
Unnamed: 0 a b
0 0 0.000000 0.218319
1 1 1.000000 0.218319
2 2 2.000000 0.218319
...
和:
df1_delayed = delayed(lambda: df1)()
df2_delayed = delayed(lambda: df2)()
dd.from_delayed([df1_delayed, df2_delayed]).compute()
给人外连接的结果:
a b c
0 0.000000 0.218319 NaN
1 1.000000 0.218319 NaN
2 2.000000 0.218319 NaN
...
如何使read_csv在相同的模式下工作?
编辑:
即使经过D型架构到大熊猫不起作用:
dd.read_csv('*.csv', dtype={'a':np.float64, 'b': np.float64, 'c': np.float64}).compute()