Python：NLTK使用WordNet给计算synsets的MemoryError

Question

我想运行这个代码，在那里我计算两个单词的synsets并计算这两个单词之间的相似性。的Python代码是给的MemoryError，如下所示：

代码：

def wordSim(word1,word2): 
    maxscore = 0.0 
    word1_synsets = word1[1] 
    word2_synsets = word2[1] 
    for k,j in list(product(*[word1_synsets,word2_synsets])): 
     score = k.wup_similarity(j) # Wu-Palmer Similarity 
     maxscore = score if maxscore < score else maxscore 
    if maxscore >= 0.85: 
     return True 

def genSynsets(wordList): 
    synsetList = map(lambda x: [x,wn.synsets(x.decode('utf-8'))],wordList) 
    return synsetList 

错误消息：

Traceback (most recent call last): 
    File "<stdin>", line 1, in <module> 
    File "/global/python/2.7.5/lib/python2.7/site-packages/nltk/corpus/util.py", line 99, in __getattr__ 
    self.__load() 
    File "/global/python/2.7.5/lib/python2.7/site-packages/nltk/corpus/util.py", line 67, in __load 
    corpus = self.__reader_cls(root, *self.__args, **self.__kwargs) 
    File "/global/python/2.7.5/lib/python2.7/site-packages/nltk/corpus/reader/wordnet.py", line 1045, in __init__ 
    self._load_lemma_pos_offset_map() 
    File "/global/python/2.7.5/lib/python2.7/site-packages/nltk/corpus/reader/wordnet.py", line 1137, in _load_lemma_pos_offset_map 
    self._lemma_pos_offset_map[lemma][pos] = synset_offsets 
MemoryError 

Answer 1

从Python docs：

exception MemoryError: Raised when an operation runs out of memory...

如果您认为你仍然有一堆免费的RAM，那么很可能你正在运行32位Python，并且达到了2GB或3GB的限制。如果可能，请使用64位Python。见Why Python Memory Error with list append() lots of RAM left。