在Django上传文件时出现UnicodeEncodeError admin

Question

我的问题类似于报告的here，但建议的解决方案对我无效。我试图通过Django管理应用上传一个名为'Testaråäö.txt'的文件。

我在Debian 6服务器上运行Django 1.3.1，使用Gunicorn 0.13.4和Nginx 0.7.6.7。数据库是PostgreSQL 8.4.9。其他Unicode数据保存到数据库没有问题，所以我猜想问题必须以某种方式与文件系统。

我已经在我的nginx.conf设置

http { 
    charset utf-8; 
} 

。 LC_ALL和LANG被设置为'sv_SE.UTF-8'。运行'语言环境'验证了这一点。我甚至尝试在我的nginx初始化脚本中设置LC_ALL和LANG，以确保locale设置正确。

这里的回溯：

Traceback (most recent call last): 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/core/handlers/base.py", line 111, in get_response 
response = callback(request, *callback_args, **callback_kwargs) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/contrib/admin/options.py", line 307, in wrapper 
return self.admin_site.admin_view(view)(*args, **kwargs) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/utils/decorators.py", line 93, in _wrapped_view 
response = view_func(request, *args, **kwargs) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/views/decorators/cache.py", line 79, in _wrapped_view_func 
response = view_func(request, *args, **kwargs) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/contrib/admin/sites.py", line 197, in inner 
return view(request, *args, **kwargs) 

File "/srv/django/letebo/app/cms/admin.py", line 81, in change_view 
return super(PageAdmin, self).change_view(request, obj_id) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/utils/decorators.py", line 28, in _wrapper 
return bound_func(*args, **kwargs) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/utils/decorators.py", line 93, in _wrapped_view 
response = view_func(request, *args, **kwargs) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/utils/decorators.py", line 24, in bound_func 
return func(self, *args2, **kwargs2) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/transaction.py", line 217, in inner 
res = func(*args, **kwargs) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/contrib/admin/options.py", line 985, in change_view 
self.save_formset(request, form, formset, change=True) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/contrib/admin/options.py", line 677, in save_formset 
formset.save() 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/forms/models.py", line 482, in save 
return self.save_existing_objects(commit) + self.save_new_objects(commit) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/forms/models.py", line 613, in save_new_objects 
self.new_objects.append(self.save_new(form, commit=commit)) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/forms/models.py", line 717, in save_new 
obj.save() 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/models/base.py", line 460, in save 
self.save_base(using=using, force_insert=force_insert, force_update=force_update) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/models/base.py", line 504, in save_base 
self.save_base(cls=parent, origin=org, using=using) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/models/base.py", line 543, in save_base 
for f in meta.local_fields if not isinstance(f, AutoField)] 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/models/fields/files.py", line 255, in pre_save 
file.save(file.name, file, save=False) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/models/fields/files.py", line 92, in save 
self.name = self.storage.save(name, content) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/core/files/storage.py", line 48, in save 
name = self.get_available_name(name) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/core/files/storage.py", line 74, in get_available_name 
while self.exists(name): 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/core/files/storage.py", line 218, in exists 
return os.path.exists(self.path(name)) 

File "/srv/.virtualenvs/letebo/lib/python2.6/genericpath.py", line 18, in exists 
st = os.stat(path) 

UnicodeEncodeError: 'ascii' codec can't encode characters in position 52-54: ordinal not in range(128) 

更新：我试着调试运行Gunicorn开启，以及文件上传没有任何问题的。我想这意味着这个问题与Nginx有关。尽管如此，仍然打败我去哪里寻找。以下是Gunicorn和Nginx的，原始的响应头，如果它使任何意义：

Gunicorn：

HTTP/1.1 302 FOUND 
Server: gunicorn/0.13.4 
Date: Thu, 09 Feb 2012 14:50:27 GMT 
Connection: close 
Transfer-Encoding: chunked 
Expires: Thu, 09 Feb 2012 14:50:27 GMT 
Vary: Cookie 
Last-Modified: Thu, 09 Feb 2012 14:50:27 GMT 
Location: http://my-server.se:8000/admin/cms/page/15/ 
Cache-Control: max-age=0 
Content-Type: text/html; charset=utf-8 
Set-Cookie: messages="yada yada yada"; Path=/ 

Nginx的：

HTTP/1.1 500 INTERNAL SERVER ERROR 
Server: nginx/0.7.67 
Date: Thu, 09 Feb 2012 14:50:57 GMT 
Content-Type: text/html; charset=utf-8 
Transfer-Encoding: chunked 
Connection: close 
Vary: Cookie 

500 

Answer 1

看来，这是服务的一个常见问题运行django服务，无论是apache，gunicorn + supervisor，daemontools等......并不总是使用正确的环境变量。

在这种情况下，gunicorn由主管开始，所以你必须明确地告诉上司使用环境选项可以使用UTF-8：

environment=LANG=en_US.UTF-8, LC_ALL=en_US.UTF-8, LC_LANG=en_US.UTF-8 

Answer 2

我有genericpath.py给人一种UnicodeEncodeError同样的问题尝试上传带有非ASCII字符的文件名时。

我使用nginx，uwsgi和django与python 2.7。

一切都在本地，但没有工作正常的服务器上

下面是我所采取的步骤：

1. 添加到/ etc/nginx的/ nginx的。配置（并没有解决这个问题）

   http { 
       charset utf-8; 
   } 
   

2. 我加入这行等/默认/区域（未解决的问题）

   LANGUAGE="en_US.UTF-8" 
   

3. 我跟着标题下这里列出的说明'成功' https://code.djangoproject.com/wiki/ExpectedTestFailures（没有解决这个问题）

   aptitude install language-pack-en-base 
   

4. 过这个票找到https://code.djangoproject.com/ticket/17816 这表明服务器有什么用区域设置信息

Django的看法发生

import locale 

def locales(request): 
    """Display the locales""" 
    locales = "Current locale: %s %s -- Default locale: %s %s" % (
     locale.getlocale() + locale.getdefaultlocale()) 
    default_encoding = sys.getdefaultencoding() 
    file_system_encoding = sys.getfilesystemencoding() 

    context = { 
     'locales': locales, 
     'default_encoding': default_encoding, 
     'file_system_encoding': file_system_encoding, # affects file uploads 
    } 
    return render(request, 'testing/locales.html', context) 

Django的模板

<h2>Locales</h2> 
<p>{{ locales }}</p> 

<h2>Default Encoding</h2> 
<p>{{ default_encoding }}</p> 

<h2>File System Encoding</h2> 
<p>{{ file_system_encoding }}</p> 

对于我的测试来看，问题是，我的Ubuntu服务器上没有语言环境和默认语言环境（尽管我的本地OSX开发机上有它们），然后使用非ASCII文件名称/路径将不会正确地上传与python引发UnicodeEncodeError，但只能在生产服务器上。

解决方案

我将此添加到我的网站，我的网站管理员uwsgi配置文件 例如两在/ etc/uwsgi皇帝/诸候/我的站点，配置-INI文件

env = LANG=en_US.utf8 

更新

移动到泊坞窗后，我又开始得到同样的错误。 进一步研究后，我意识到在上面的uwsgi django实例import sys; sys.getfilesystemencoding()或{{ file_system_encoding }}返回ANSI_X3.4-1968，但是如果我启动我自己的python实例并运行import sys; sys.getfilesystemencoding()我会得到UTF-8。 ANSI_X3.4-1968格式是抛出UnicodeEncodeError的。

所以除了上面这个答案列出的uwsgi的解决方案，我也不得不加入到我的Django dockerfile

ENV LANG en_US.UTF-8 
RUN apt-get update && install -y locales && \ 
    sed -i -e "s/# $LANG.*/$LANG UTF-8/" /etc/locale.gen && \ 
    locale-gen --purge &&\ 
    update-locale LANG=$LANG 

或者，如果你不使用泊坞窗你只是在shell中运行这个

$ export LANG=en_US.UTF-8 

然后运行上面的RUN命令（但没有运行）。

参考文献：

http://stackoverflow.com/a/37246853/3003438 
https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/uwsgi/