尝试在SPickle中使用堆栈python(2.7.2)将测试方法通过celery发送到不同机器上执行。我希望测试方法(代码)包含在pickle中,而不是强制存在于正在执行的机器python路径中。是否可以在不使用RPC的情况下使用SPickle来序列化tasklet代码(而不仅仅是exec状态)?
去过引用下面介绍: https://ep2012.europython.eu/conference/talks/advanced-pickling-with-stackless-python-and-spickle
试图用在检查点设置滑动11所示的技术的RPC示例似乎右假设我们正在使用芹菜并不:
客户机代码:
from stackless import run, schedule, tasklet
from sPickle import SPickleTools
def test_method():
print "hello from test method"
tasks = []
test_tasklet = tasklet(test_method)()
tasks.append(test_tasklet)
pt = SPickleTools(serializeableModules=['__test_method__'])
pickled_task = pt.dumps(tasks)
Server代码:
pt = sPickle.SPickleTools()
unpickledTasks = pt.loads(pickled_task)
个
结果:
[2012-03-09 14:24:59,104: ERROR/MainProcess] Task
celery_tasks.test_exec_method[8f462bd6-7952-4aa1-9adc-d84ee4a51ea6] raised exception:
AttributeError("'module'
object has no attribute 'test_method'",)
Traceback (most recent call last):
File "c:\Python27\lib\site-packages\celery\execute\trace.py", line 153, in trace_task
R = retval = task(*args, **kwargs)
File "c:\Python27\celery_tasks.py", line 16, in test_exec_method
unpickledTasks = pt.loads(pickled_task)
File "c:\Python27\lib\site-packages\sPickle\_sPickle.py", line 946, in loads
return unpickler.load()
AttributeError: 'module' object has no attribute 'test_method'
对我在做什么不正确或者如果这甚至有可能有什么建议?
在celeryd中做动态模块加载的替代建议也是好的(作为使用sPickle的替代方案)。我已经尝试这样做:
py_mod = imp.load_source(module_name,'some script path')
sys.modules.setdefault(module_name,py_mod)
但动态加载模块似乎并不通过不同的调用celeryd,即不同的远程调用持续。
您是否同意sPickle目前没有内置的模块依赖关系和在远程机器上动态加载?我相信当我评估这一点时,我希望sPickle能够做动态模块依赖捆绑和沙盒, http://stackoverflow.com/questions/10099326/how-to-do-an-embedded-python-module-for-remote-sandbox-execution。如果你同意,会给你检查。 – 2012-06-25 19:37:12