在Python中保存KDTree对象？

Question

我正在使用Scipy的KDTree实现来读取300 MB的大文件。现在，有没有办法将数据结构保存到磁盘并重新加载它，或者我坚持每次从文件中读取原始数据并在每次启动程序时构造数据结构？我正在构建KDTree，如下所示：

def buildKDTree(self): 
     self.kdpoints = numpy.fromfile("All", sep=' ') 
     self.kdpoints.shape = self.kdpoints.size/self.NDIM, NDIM 
     self.kdtree = KDTree(self.kdpoints, leafsize = self.kdpoints.shape[0]+1) 
     print "Preparing KDTree... Ready!" 

有什么建议吗？

Answer 1

KDtree使用嵌套类来定义其节点类型（innernode，leafnode）。泡菜只能在模块级的类定义，所以嵌套类车次起来：

import cPickle 

class Foo(object): 
    class Bar(object): 
     pass 

obj = Foo.Bar() 
print obj.__class__ 
cPickle.dumps(obj) 

<class '__main__.Bar'> 
cPickle.PicklingError: Can't pickle <class '__main__.Bar'>: attribute lookup __main__.Bar failed 

但是，通过猴子打补丁的类定义为scipy.spatial.kdtree在模块范围，所以，皮克勒一（哈克）解决方法可以找到他们。如果您的所有代码的读取和写入腌制KDtree对象安装这些补丁，这个技巧应该很好地工作：

import cPickle 
import numpy 
from scipy.spatial import kdtree 

# patch module-level attribute to enable pickle to work 
kdtree.node = kdtree.KDTree.node 
kdtree.leafnode = kdtree.KDTree.leafnode 
kdtree.innernode = kdtree.KDTree.innernode 

x, y = numpy.mgrid[0:5, 2:8] 
t1 = kdtree.KDTree(zip(x.ravel(), y.ravel())) 
r1 = t1.query([3.4, 4.1]) 
raw = cPickle.dumps(t1) 

# read in the pickled tree 
t2 = cPickle.loads(raw) 
r2 = t2.query([3.4, 4.1]) 
print t1.tree.__class__ 
print repr(raw)[:70] 
print t1.data[r1[1]], t2.data[r2[1]] 

输出：

<class 'scipy.spatial.kdtree.innernode'> 
"ccopy_reg\n_reconstructor\np1\n(cscipy.spatial.kdtree\nKDTree\np2\nc_ 
[3 4] [3 4]