我有3个芹菜节拍实例在我的VPS上运行(使用不同的settings.py)。其中三个实例由三个具有相同代码的网站使用。 该任务基本上发送电子邮件给几百注册用户(使用sendgrid)。芹菜任务重复问题
我的问题是,我的任务运行时使用ETA方法运行3次如下。
sdate = datetime.datetime.strptime(request.POST['schedule_date'],'%d-%m-%Y %H:%M')
tz=get_current_timezone()
celery_scheduled_campaign.apply_async(eta=tz.localize(sdate),
kwargs={'schedule_id': schedule.id })
但运行正常(仅一次)使用.delay方法时。
celery_sendmail_task.delay(pro_campaign,unsubscribe_url,ecm_host)
settings_one.py
...
BROKER_URL = 'redis://localhost:6379/0'
...
settings_two.py
...
BROKER_URL = 'redis://localhost:6379/1'
...
settings_three.py
...
BROKER_URL = 'redis://localhost:6379/2'
...
task.py
from celery import task
from bulkmailer import send_email
from models import CampaignSchedule, SendgridEmailQuota
import logging
logger = logging.getLogger("ecm_console")
#import pdb
#import time
#from django.core.mail import EmailMultiAlternatives
@task.task(ignore_result=True)
def celery_sendmail_task(obj,unsubscribe_url,host):
#time.sleep(10)
send_email(obj,unsubscribe_url,host)
obj.status=True
if obj.campaign_opt=='S':
obj.campaign_opt='R'
obj.save()
@task.task(ignore_result=True)
def sendgrid_quota_reset():
try:
quota = SendgridEmailQuota.objects.get(pk=1)
quota.used=0
quota.save()
logger.info("Success : sendgrid_quota_reset job ")
except Exception, e:
logger.error("Critical Error : sendgrid_quota_reset: {0} ".format(e))
@task.task(ignore_result=True)
def celery_scheduled_campaign(schedule_id):
try:
obj = CampaignSchedule.objects.get(pk=schedule_id)
send_email(obj.campaign, obj.unsub_url, obj.ecm_host)
obj.campaign.status = True
obj.campaign.save()
except Exception, e:
logger.error("Critical Error : celery_scheduled_campaign: {0} ".format(e))
用于运行芹菜命令
蟒manage.py芹菜工人-B -c 2 --loglevel =信息--settings = ecm.settings_one
蟒蛇manage.py芹菜工人-B -c 2 --loglevel =信息--settings = ecm.settings_two
蟒蛇管理。 PY芹菜工人-B -c 2 --loglevel =信息--settings = ecm.settings_three
版本
芹菜== 3.0.21 Django的芹菜== 3.0.21 Python 2.7版。 3
EDIT 1 芹菜日志显示任务
[2014-11-24 22:09:32,521: INFO/MainProcess] Celerybeat: Shutting down...
[2014-11-24 22:09:32,557: WARNING/MainProcess] Restoring 1 unacknowledged message(s).
[2014-11-24 22:09:40,495: INFO/Beat] Celerybeat: Starting...
[2014-11-24 22:09:40,540: WARNING/MainProcess] [email protected] ready.
[2014-11-24 22:09:40,547: INFO/MainProcess] consumer: Connected to redis://localhost:6379/3.
[2014-11-24 22:09:40,614: INFO/MainProcess] Got task from broker: ecm_core.tasks.celery_scheduled_campaign[f5c82a1d-3996-4266-9023-3f7e07538e84] eta:[2014-11-25 09:00:00+04:00]
^^氏后自动添加几个小时s是我从前端添加任务的地方。下面的任务是获取自动添加
[2014-11-24 23:09:53,039: INFO/MainProcess] Got task from broker: ecm_core.tasks.celery_scheduled_campaign[f5c82a1d-3996-4266-9023-3f7e07538e84] eta:[2014-11-25 09:00:00+04:00]
周期性任务没有ETA正常VV
[2014-11-25 00:01:00,044: INFO/Beat] Scheduler: Sending due task ecm_sendgrid_sync (ecm_sendgridapi.tasks.ecm_sendgridapi_dbsync)
[2014-11-25 00:01:00,052: INFO/MainProcess] Got task from broker: ecm_sendgridapi.tasks.ecm_sendgridapi_dbsync[37c94a3a-f6c2-433c-81a3-ae351c7018f8]
[2014-11-25 00:01:02,262: INFO/MainProcess] Success : update job
[2014-11-25 00:01:02,265: INFO/MainProcess] Task ecm_sendgridapi.tasks.ecm_sendgridapi_dbsync[37c94a3a-f6c2-433c-81a3-ae351c7018f8] succeeded in 2.18759179115s: None
再次任务,得到ETA自动添加运行。注意哈希是相同的。
[2014-11-25 00:10:12,190: INFO/MainProcess] Got task from broker: ecm_core.tasks.celery_scheduled_campaign[f5c82a1d-3996-4266-9023-3f7e07538e84] eta:[2014-11-25 09:00:00+04:00]
[2014-11-25 01:10:26,029: INFO/MainProcess] Got task from broker: ecm_core.tasks.celery_scheduled_campaign[f5c82a1d-3996-4266-9023-3f7e07538e84] eta:[2014-11-25 09:00:00+04:00]
[2014-11-25 02:10:39,025: INFO/MainProcess] Got task from broker: ecm_core.tasks.celery_scheduled_campaign[f5c82a1d-3996-4266-9023-3f7e07538e84] eta:[2014-11-25 09:00:00+04:00]
[2014-11-25 03:10:50,063: INFO/MainProcess] Got task from broker: ecm_core.tasks.celery_scheduled_campaign[f5c82a1d-3996-4266-9023-3f7e07538e84] eta:[2014-11-25 09:00:00+04:00]
[2014-11-25 04:00:00,007: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup)
[2014-11-25 04:00:00,064: INFO/MainProcess] Got task from broker: celery.backend_cleanup[35a4db80-008e-49c9-9735-2dc1df5e0ecc] expires:[2014-11-25 16:00:00.008296+04:00]
[2014-11-25 04:00:01,533: INFO/MainProcess] Task celery.backend_cleanup[35a4db80-008e-49c9-9735-2dc1df5e0ecc] succeeded in 1.01458001137s: None
[2014-11-25 04:11:03,062: INFO/MainProcess] Got task from broker: ecm_core.tasks.celery_scheduled_campaign[f5c82a1d-3996-4266-9023-3f7e07538e84] eta:[2014-11-25 09:00:00+04:00]
[2014-11-25 05:11:15,073: INFO/MainProcess] Got task from broker: ecm_core.tasks.celery_scheduled_campaign[f5c82a1d-3996-4266-9023-3f7e07538e84] eta:[2014-11-25 09:00:00+04:00]
[2014-11-25 06:11:26,101: INFO/MainProcess] Got task from broker: ecm_core.tasks.celery_scheduled_campaign[f5c82a1d-3996-4266-9023-3f7e07538e84] eta:[2014-11-25 09:00:00+04:00]
[2014-11-25 07:11:38,324: INFO/MainProcess] Got task from broker: ecm_core.tasks.celery_scheduled_campaign[f5c82a1d-3996-4266-9023-3f7e07538e84] eta:[2014-11-25 09:00:00+04:00]
[2014-11-25 08:11:53,097: INFO/MainProcess] Got task from broker: ecm_core.tasks.celery_scheduled_campaign[f5c82a1d-3996-4266-9023-3f7e07538e84] eta:[2014-11-25 09:00:00+04:00]
这可能是旧版本中的错误。我也怀疑我的VPS,内存不足(使用400 +/489)
可能是因为所有(3)任务的消息都去同一个端口号?这是导致tripple任务运行? – doniyor 2014-11-22 14:06:04
@ doniyor如何检查? – 2014-11-23 15:24:26
只是改变端口号,看看它的工作原理,让我知道如果它确实;) – doniyor 2014-11-23 16:02:18