Mapreduce作业随机失败，'BadValueError：名称必须小于500'

Question

我正在运行一个映射缩减作业，它映射大约100k条记录。经过几千年的记录（不同的地方，每次），在应用程序引擎日志BadValueError以下错误

地图步骤失败：名称必须小于500

Pipeline job id: baada59c857d11e28527e1efd5a519d7

Output Blob file name: content_contributors_report_1362478582.csv

日志转储

Final result for job '1581739058957C0CA88F5' is 'success' E2013-03-05 
12:25:21.637 name must be under 500 bytes. Traceback (most recent call 
last): File 
"/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", 
line 1535, in __call__ 
    rv = self.handle_exception(request, response, e) File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", 
line 1529, in __call__ 
    rv = self.router.dispatch(request, response) File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", 
line 1278, in default_dispatcher 
    return route.handler_adapter(request, response) File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", 
line 1102, in __call__ 
    return handler.dispatch() File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", 
line 572, in dispatch 
    return self.handle_exception(e, self.app.debug) File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", 
line 570, in dispatch 
    return method(*args, **kwargs) File "/base/data/home/apps/s~myap/a-276.365486148015822746/mapreduce/base_handler.py", 
line 68, in post 
    self.handle() File "/base/data/home/apps/s~myapp/a-276.365486148015822746/mapreduce/handlers.py", 
line 437, in handle 
    spec, state, self.base_path()) File "/base/data/home/apps/s~myapp/a-276.365486148015822746/mapreduce/handlers.py", 
line 517, in _finalize_job 
    mapreduce_spec.mapper.output_writer_class().finalize_job(mapreduce_state) 
File 
"/base/data/home/apps/s~myapp/a-276.365486148015822746/mapreduce/output_writers.py", 
line 454, in finalize_job 
    files.blobstore.get_blob_key(filename))) File "/base/data/home/apps/s~myapp/a-276.365486148015822746/mapreduce/lib/files/blobstore.py", 
line 112, in get_blob_key 
    _BLOB_FILE_INDEX_KIND, ticket, namespace='')])[0] File "/python27_runtime/python27_lib/versions/1/google/appengine/api/datastore_types.py", 
line 504, in from_path 
    ValidateString(id_or_name, 'name') File "/python27_runtime/python27_lib/versions/1/google/appengine/api/datastore_types.py", 
line 176, in ValidateString 
    raise exception('%s must be under %d bytes.' % (name, max_len)) BadValueError: name must be under 500 bytes. 

Answer 1

不同版本的mapreduce可能会导致此问题。 基于Run Map Reduce on non-default versions? 它建议使用https://code.google.com/p/appengine-mapreduce/

The stale bundle link in the docs is a bug. That'll get cleaned up soon.

A few of our SDKs have bits of MapReduce (for historic reasons), but the open source version is the way to go for now.