2016-06-28 64 views
1

运行pyspark我有Windows 10有继起的说明安装火花:不能在Jupyter

https://hernandezpaul.wordpress.com/2016/01/24/apache-spark-installation-on-windows-10/

现在,我打开我的jupyter记事本,然后键入以下内容:

import os 
import sys 

# Path for spark source folder 
os.environ['SPARK_HOME']="c:\\Spark" 

# Append pyspark to Python Path 
sys.path.append("C:\\Spark") 
sys.path.append("C:\\Spark\\python") 
sys.path.append("C:\\Spark\\python\\lib") 
sys.path.append("C:\\Spark\\python\\lib\\py4j-0.9-src.zip") 

from pyspark import SparkContext 
from pyspark import SparkConf 

和似乎cannoot加载累加器库,因为我得到以下错误:

--------------------------------------------------------------------------- ImportError        Traceback (most recent call last) <ipython-input-54-68cce399fff2> in <module>() 
    12 sys.path.append("C:\\Spark\\python\\pyspark") 
    13 
---> 14 from pyspark import SparkContext 
    15 from pyspark import SparkConf 
    16 

C:\Spark\python\pyspark\__init__.py in <module>() 
    39 
    40 from pyspark.conf import SparkConf 
---> 41 from pyspark.context import SparkContext 
    42 from pyspark.rdd import RDD 
    43 from pyspark.files import SparkFiles 

C:\Spark\python\pyspark\context.py in <module>() 
    26 from tempfile import NamedTemporaryFile 
    27 
---> 28 from pyspark import accumulators 
    29 from pyspark.accumulators import Accumulator 
    30 from pyspark.broadcast import Broadcast 

ImportError: cannot import name accumulators 

这些是我的sys.path的样子,我以为是正确的文件夹:

['', 
'C:\\Anaconda2\\python27.zip', 
'C:\\Anaconda2\\DLLs', 
'C:\\Anaconda2\\lib', 
'C:\\Anaconda2\\lib\\plat-win', 
'C:\\Anaconda2\\lib\\lib-tk', 
'C:\\Anaconda2', 
'c:\\anaconda2\\lib\\site-packages\\sphinx-1.3.5-py2.7.egg', 
'c:\\anaconda2\\lib\\site-packages\\setuptools-20.3-py2.7.egg', 
'C:\\Anaconda2\\lib\\site-packages', 
'C:\\Anaconda2\\lib\\site-packages\\win32', 
'C:\\Anaconda2\\lib\\site-packages\\win32\\lib', 
'C:\\Anaconda2\\lib\\site-packages\\Pythonwin', 
'C:\\Anaconda2\\lib\\site-packages\\IPython\\extensions', 
'C:\\Users\\Manuel\\.ipython', 
'C:\\Spark', 
'C:\\Spark\\python', 
'C:\\Spark\\python\\lib', 
'C:\\Spark\\python\\lib\\py4j-0.9-src.zip', 
'C:\\Spark\\python\\pyspark'] 

任何帮助将非常感激。

谢谢!

回答