2016-05-17 68 views
2

我想要做的是编写代码,它允许我从csv文件批量加载Django对象实例。很明显,我应该在保存任何内容之前先检查所有数据。Django模型实例full_clean方法,对吗?

tl; dr:full_clean()方法无法捕获即将尝试在没有null=True的字段中保存无。似乎不正当。这是由设计,如果是这样,为什么? Django的bug比我曾经使用过的其他任何东西都少,所以“Bug!”似乎最不可能。

完整版。我认为会工作的是每行,创建一个对象实例,用电子表格中的数据填充字段,然后调用full_clean方法。即(轮廓)

from django.core.exceptions import ValidationError 
... 

# upload a CSV file and open with a csvreader 
errors=[] 
for rownumber, row in enumerate(csvreader): 

    o = SomeDjangoModel() 
    o.somefield = row[0] # repeated for all input data row[1] ... 

    try: 
     reason = "" 
     o.full_clean() 
    except ValidationError as e: 
     reason = "Row:{} Reason:{}".format(rownumber, str(e)) 
     errors.append(reason) 
     # reason, together with the row-number of the csv file, fully explains 
     # what is wrong. 

# end of loop 
if errors: 
    # display errors to the user for him to fix 
else: 
    # repeat the loop, doing .save() instead of .full_clean() 
    # and get database integrity errors trying to save Null in non-null model field. 

麻烦的是,.full_clean()不赶领域没有值,而不null=True

我应该怎么做?想法包括

  1. 包住整个事情在一个事务中,做了一批o.save()异常处理程序内的,并滚动整个事务回,除非没有任何错误。但是,为什么在90%的尝试会以不重要的方式出错时会打扰数据库呢?

  2. 通过表单提交数据,即使与用户没有表单级的每行交互。

  3. 手动测试无应该在哪里。但是还有什么.full_clean没有检查?

我可以理解,最终赶上数据库完整性错误的唯一方法就是尝试存储数据,但为什么不Django的在空独自捕获无=假场?

BTW这是Django的1.9.6

添加的细节。这是模型定义

class OrderHistory(models.Model): 
    invoice_no = models.CharField(max_length=10, unique=True)   # no default 
    invoice_val= models.DecimalField(max_digits=8, decimal_places=2) # no default 
    date  = models.DateField()         # no default 

的相关领域,这是发生了什么,从python manage.py shell完成,以证明.full_clean方法无法发现的

>>> from orderhistory.models import OrderHistory 
>>> from datetime import date 
>>> o = OrderHistory(date=date(2010,3,17), invoice_no="21003163") 
>>> o.invoice_val=None 
>>> o.full_clean() # passes clean 
>>> o.save() # attempt to save this one which has passed full_clean() validation 
Traceback (most recent call last): 
    File "<console>", line 1, in <module> 
    File "/home/nigel/.virtualenvs/edge22/lib/python3.4/site-packages/django/db/models/base.py", line 708, in save 
force_update=force_update, update_fields=update_fields) 
    File "/home/nigel/.virtualenvs/edge22/lib/python3.4/site-packages/django/db/models/base.py", line 736, in save_base 
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields) 
    File "/home/nigel/.virtualenvs/edge22/lib/python3.4/site-packages/django/db/models/base.py", line 820, in _save_table 
result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) 
    File "/home/nigel/.virtualenvs/edge22/lib/python3.4/site-packages/django/db/models/base.py", line 859, in _do_insert 
using=using, raw=raw) 
    File "/home/nigel/.virtualenvs/edge22/lib/python3.4/site-packages/django/db/models/manager.py", line 122, in manager_method 
return getattr(self.get_queryset(), name)(*args, **kwargs) 
    File "/home/nigel/.virtualenvs/edge22/lib/python3.4/site-packages/django/db/models/query.py", line 1039, in _insert 
return query.get_compiler(using=using).execute_sql(return_id) 
    File "/home/nigel/.virtualenvs/edge22/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 1060, in execute_sql 
cursor.execute(sql, params) 
    File "/home/nigel/.virtualenvs/edge22/lib/python3.4/site-packages/django/db/backends/utils.py", line 79, in execute 
return super(CursorDebugWrapper, self).execute(sql, params) 
    File "/home/nigel/.virtualenvs/edge22/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute 
return self.cursor.execute(sql, params) 
    File "/home/nigel/.virtualenvs/edge22/lib/python3.4/site-packages/django/db/utils.py", line 95, in __exit__ 
six.reraise(dj_exc_type, dj_exc_value, traceback) 
    File "/home/nigel/.virtualenvs/edge22/lib/python3.4/site-packages/django/utils/six.py", line 685, in reraise 
raise value.with_traceback(tb) 
    File "/home/nigel/.virtualenvs/edge22/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute 
    return self.cursor.execute(sql, params) 
django.db.utils.IntegrityError: null value in column "invoice_val" violates not-null constraint 
DETAIL: Failing row contains (2, 21003163, , , 2010-03-17, , null, null, null, null, null, null). 
>>> 
>>> p = OrderHistory(invoice_no="21003164") # no date 
>>> p.date=None 
>>> p.full_clean()       # this DOES error as it should 
    Traceback (most recent call last): 
    File "<console>", line 1, in <module> 
    File "/home/nigel/.virtualenvs/edge22/lib/python3.4/site-packages/django/db/models/base.py", line 1144, in full_clean 
    raise ValidationError(errors) 
django.core.exceptions.ValidationError: {'date': ['This field cannot be null.']} 
>>> 
+0

在您的csv文件中,None值如何表示?代码缩进时出现错误,请参阅检查错误并保存的最后4行。 – ozren1983

+0

这不是代码错误。我在循环过程中积累了错误。循环后如果出现错误,我将显示所有这些错误,否则我重复执行save()而不是full_clean()的循环。 (这可能会引发数据库完整性错误,但这与此问题无关)。 – nigel222

+0

CSV文件中的空格是两个逗号之间没有任何内容。我还会将数字列中的全空白字符串视为null(这是用户间隔数据而不是删除数据的常见电子表格反模式)。 – nigel222

回答

0

我刚才重复你的步骤在外壳和full_clean()触发ValidationError为无值:

>>> from orders.models import OrderHistory 
>>> o = OrderHistory() 
>>> o.full_clean() 
Traceback (most recent call last): 
    File "<console>", line 1, in <module> 
    File "/Users/oz/.virtualenvs/full_clean_test/lib/python2.7/site-packages/django/db/models/base.py", line 1144, in full_clean 
    raise ValidationError(errors) 
ValidationError: {'date': [u'This field cannot be null.'], 'invoice_val': [u'This field cannot be null.'], 'invoice_no': [u'This field cannot be blank.']} 

我在OSX和Pyth使用Django 1.9.6和Python 2.7.10测试了新项目在Ubuntu的3.4.3上。

尝试从您的项目中删除所有* .pyc文件。如果这不起作用,请删除虚拟环境,创建新环境并重新安装依赖项。

+0

对不起,但这不是问题的答案。我想知道的是,为什么full_clean方法未能检测到这些空值。当然,我可以添加大量的特殊代码来自己检查数据,但是不应该让Django根据它对模型的了解来做到这一点? – nigel222

+0

就像你自己说的那样,这很可能不是Django或full_clean()方法的bug。我的猜测是你的csv阅读器没有像你期望的那样工作,你的空值(两个逗号之间没有任何内容)不会被读为python无类型,而可能是空字符串。您可以添加一些调试日志/打印,并检查打印“value:{},type:{}”。格式(o.somefield,type(o.somefield))时获得空值的值和类型。 – ozren1983

+0

细节已添加到问题。未显示的字段具有默认值,或者为null = True。这是'o.invoice_val = None',通过full_clean() – nigel222

相关问题