是否有可以用scipy.io.readsav读取的最大文件大小限制？

Question

摘要：我有一个带有905 x 905 x 762个元素的3D FLOAT数组。它存储在IDL保存文件中，该文件无法用scipy.io.readsav读取。当我将这个数组的一部分（905 x 905 x 600个元素或更少）存储在另一个IDL保存文件中时，此时scipy.io.readsav可以正常工作。

这是我做的读取任何这两个文件的内容：

from scipy.io import readsav 
filename = 'datacube.sav' 
sav = readsav(filename, verbose=True) 

这是当我读到了更大的IDL保存文件，我得到的错误：

/Users/moncho/Codes/miniconda3/lib/python3.5/site-packages/scipy/io/idl.py:473: UserWarning: Using experimental 64-bit array read 
warnings.warn("Using experimental 64-bit array read") 
Traceback (most recent call last): 

File "<ipython-input-1-c6753cefbd81>", line 1, in <module> runfile('/Users/moncho/Data/untitled7.py', wdir='/Users/moncho/Data') 
File "/Users/moncho/Codes/miniconda3/lib/python3.5/site-packages/spyder/utils/site/sitecustomize.py", line 710, in runfile 
execfile(filename, namespace) 
File "/Users/moncho/Codes/miniconda3/lib/python3.5/site-packages/spyder/utils/site/sitecustomize.py", line 101, in execfile 
exec(compile(f.read(), filename, 'exec'), namespace) 
File "/Users/moncho/Data/untitled7.py", line 11, in <module> 
sav = readsav(filename, verbose=True) 
File "/Users/moncho/Codes/miniconda3/lib/python3.5/site-packages/scipy/io/idl.py", line 799, in readsav 
r = _read_record(f) 
File "/Users/moncho/Codes/miniconda3/lib/python3.5/site-packages/scipy/io/idl.py", line 360, in _read_record 
rectypedesc['array_desc']) 
File "/Users/moncho/Codes/miniconda3/lib/python3.5/site-packages/scipy/io/idl.py", line 284, in _read_array 
array = np.fromstring(f.read(array_desc['nbytes']), 

OSError: [Errno 22] Invalid argument 

我运行在Mac OS X 10.11.6上使用python 3.5。任何想法都表示赞赏。谢谢！

Answer 1

加载文件取决于系统中可用的可用内存量。验证您是否有足够的可用内存。之前曾经遇到类似的情况 - 我能够在2.7 OS X中使用scipy加载大文件，但不能在3.5中加载。在2.7中尝试readsav。这是一个错误。