3

我正在使用应用引擎和大型查询作为我网站的后端。无论何时用户点击一下,我都会将它们记录到bigquery中,以便在当天晚些时候进行分析。我每天接近75,000次点击。直到上周,它一切正常。这是我使用的代码。app_identity_service.GetAccessToken()需要比可用配额更多的配额

body = {"rows":[bodyFields]} 

    credentials = appengine.AppAssertionCredentials(scope=BIGQUERY_SCOPE) 
    http = credentials.authorize(httplib2.Http()) 
    bigquery = discovery.build('bigquery', 'v2', http=http) 

    response = bigquery.tabledata().insertAll(
     projectId=PROJECT_ID, 
     datasetId=BIGQUERY_DATASETID, 
     tableId=BIGQUERY_TABLEID, 
     body=body).execute() 

现在突然间我超过了配额例外。我的应用程序是付费应用程序引擎实例。下面是我的例外

dashboard screenshot

Attempting refresh to obtain initial access_token 

The API call app_identity_service.GetAccessToken() required more quota than is available. 
Traceback (most recent call last): 
    File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__ 
    rv = self.handle_exception(request, response, e) 
    File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__ 
    rv = self.router.dispatch(request, response) 
    File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher 
    return route.handler_adapter(request, response) 
    File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__ 
    return handler.dispatch() 
    File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch 
    return self.handle_exception(e, self.app.debug) 
    File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch 
    return method(*args, **kwargs) 
    File "/base/data/home/apps/s~projectname/bigqueryapi.387952303347375306/filename.py", line 1611, in post 
    bigquery = discovery.build('bigquery', 'v2', http=http) 
    File "/base/data/home/apps/s~projectname/bigqueryapi.387952303347375306/oauth2client/util.py", line 129, in positional_wrapper 
    return wrapped(*args, **kwargs) 
    File "/base/data/home/apps/s~projectname/bigqueryapi.387952303347375306/apiclient/discovery.py", line 198, in build 
    resp, content = http.request(requested_url) 
    File "/base/data/home/apps/s~projectname/bigqueryapi.387952303347375306/oauth2client/util.py", line 129, in positional_wrapper 
    return wrapped(*args, **kwargs) 
    File "/base/data/home/apps/s~projectname/bigqueryapi.387952303347375306/oauth2client/client.py", line 516, in new_request 
    self._refresh(request_orig) 
    File "/base/data/home/apps/s~projectname/bigqueryapi.387952303347375306/oauth2client/appengine.py", line 194, in _refresh 
    scopes, service_account_id=self.service_account_id) 
    File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/app_identity/app_identity.py", line 589, in get_access_token 
    scopes, service_account_id=service_account_id) 
    File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/app_identity/app_identity.py", line 547, in get_access_token_uncached 
    return rpc.get_result() 
    File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result 
    return self.__get_result_hook(self) 
    File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/app_identity/app_identity.py", line 519, in get_access_token_result 
    rpc.check_success() 
    File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 579, in check_success 
    self.__rpc.CheckSuccess() 
    File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/apiproxy_rpc.py", line 134, in CheckSuccess 
    raise self.exception 
OverQuotaError: The API call app_identity_service.GetAccessToken() required more quota than is available. 

我的流量已经没有时间处理程序被击中也多数量上升的堆栈跟踪几乎是一样的近2个月的数据。那么为什么我得到这个错误。

回答

2

为了确定您遇到配额错误的原因,您需要共享有关您的使用情况的更多详细信息。配额应每24小时重置一次。你知道这个错误出现需要多长时间,并且你已经成功服务了多少流量?

你提到你“在今天晚些时候做分析”,这表明你可能使用的是TaskQueue APIDeferred Tasks。这些任务可能因其他原因而失败,并且可能会很快耗尽您的配额。如果您使用的是TaskQueues,则可以尝试调整queue configuration and retry options

另一种可能可以节省配额的方式是将正在构建的bigquery发现服务保存为类似于Memcache API的内容,以便它可以重复用于对BigQuery服务的多个请求。

+0

我使用吨TaskQueue调用函数,我不想阻止用户的时间。任务队列是否吃掉了access_token配额? – Sriram

+0

我也觉得问题在于发现服务。所以我重新编写了我的代码来使用bigQuery的REST API端点。问题仍然存在。另外根据我的理解,应用引擎API默认处理access_token的缓存 – Sriram

+0

TaskQueue单独使用不会影响您的'access_token'配额,使用TaskQueue没有任何问题。 'access_token'可能会被缓存,但它可能会被每个请求缓存,并且每个Task都会在一个单独的请求中运行。 您是否有统计数据可以分享您每天运行多少任务以及多少次运行成功? – erichiggins