MySQL 通过 Python 脚本将 CSV 文件导出为 UTF-8

Question

我可以通过 Python csv 模块将 MySQL table 导出到 CSV 文件中，但是没有 utf-8 字符。（示例：???? 个字符代替 ąöę）。

table 数据是 utf-8 格式（phpMyAdmin 让我看到正确的数据）。

我发现了一些信息，在 Python 中，所有数据都应该以 utf-8 解码，然后通过例如 unicodewritter 以 utf-8 编码为 CSV（因为本机 csv 模块不支持 Unicode 正确）。

我尝试了很多但没有成功。

问题：是否有任何示例脚本可以将 utf-8 格式的 MySQL 数据库导出到 Python 中的 utf-8 格式的 CSV 文件？

我使用 ubuntu 14.04，mysql.connector 有问题所以我使用 MySQLdb 和 Gord Thompson 代码：

# -*- coding: utf-8 -*-
import csv
import MySQLdb
from UnicodeSupportForCsv import UnicodeWriter
import sys
reload(sys)  
sys.setdefaultencoding('utf8')
#sys.setdefaultencoding('Cp1252')

conn = MySQLdb.Connection(db='sampledb', host='localhost',           
user='sampleuser', passwd='samplepass')

crsr = conn.cursor()
crsr.execute("SELECT * FROM rfid")
with open(r'test.csv', 'wb') as csvfile:
    uw = UnicodeWriter(
    csvfile, delimiter=',',
    quotechar='"', quoting=csv.QUOTE_MINIMAL)
for row in crsr.fetchall():
    uw.writerow([unicode(col) for col in row])

错误仍然存在：UnicodeDecodeError：'utf8'编解码器无法解码位置 2 中的字节 0xf3：连续字节无效

Answer 1

这适用于 Python 2.7.5 和 MySQL Connector/Python 2.0.4:

# -*- coding: utf-8 -*-
import csv
import mysql.connector
from UnicodeSupportForCsv import UnicodeWriter

conn = mysql.connector.connect(
    host='localhost', port=3307,
    user='root', password='whatever',
    database='mydb')
crsr = conn.cursor()
crsr.execute("SELECT * FROM vocabulary")
with open(r'C:\Users\Gord\Desktop\test.csv', 'wb') as csvfile:
    uw = UnicodeWriter(
        csvfile, delimiter=',',
        quotechar='"', quoting=csv.QUOTE_MINIMAL)
    for row in crsr.fetchall():
        uw.writerow([unicode(col) for col in row])

UnicodeWriter class 直接取自 documentation page for the csv module 上的最后一个示例，我将其存储在名为 "UnicodeSupportForCsv.py":

的文件中

import csv, codecs, cStringIO

class UTF8Recoder:
    """
    Iterator that reads an encoded stream and reencodes the input to UTF-8
    """
    def __init__(self, f, encoding):
        self.reader = codecs.getreader(encoding)(f)

    def __iter__(self):
        return self

    def next(self):
        return self.reader.next().encode("utf-8")

class UnicodeReader:
    """
    A CSV reader which will iterate over lines in the CSV file "f",
    which is encoded in the given encoding.
    """

    def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds):
        f = UTF8Recoder(f, encoding)
        self.reader = csv.reader(f, dialect=dialect, **kwds)

    def next(self):
        row = self.reader.next()
        return [unicode(s, "utf-8") for s in row]

    def __iter__(self):
        return self

class UnicodeWriter:
    """
    A CSV writer which will write rows to CSV file "f",
    which is encoded in the given encoding.
    """

    def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds):
        # Redirect output to a queue
        self.queue = cStringIO.StringIO()
        self.writer = csv.writer(self.queue, dialect=dialect, **kwds)
        self.stream = f
        self.encoder = codecs.getincrementalencoder(encoding)()

    def writerow(self, row):
        self.writer.writerow([s.encode("utf-8") for s in row])
        # Fetch UTF-8 output from the queue ...
        data = self.queue.getvalue()
        data = data.decode("utf-8")
        # ... and reencode it into the target encoding
        data = self.encoder.encode(data)
        # write to the target stream
        self.stream.write(data)
        # empty queue
        self.queue.truncate(0)

    def writerows(self, rows):
        for row in rows:
            self.writerow(row)

Answer 2

试试这个..让你轻松

https://github.com/jdunck/python-unicodecsv

unicodecsv 是 drop-in 替代 Python 2.7 的 csv 模块，它支持 unicode 字符串，没有任何麻烦。支持的版本是 python 2.6、2.7、3.3、3.4、3.5 和 pypy 2.4.0。

>>> import unicodecsv as csv
>>> from io import BytesIO
>>> f = BytesIO()
>>> w = csv.writer(f, encoding='utf-8')
>>> _ = w.writerow((u'é', u'ñ'))
>>> _ = f.seek(0)
>>> r = csv.reader(f, encoding='utf-8')
>>> next(r) == [u'é', u'ñ']
True

Answer 3

MySQL 转换字符集非常棒。但是你需要告诉它使用正确的排序规则建立连接。

默认情况下 returns 它是如何放入数据库的。将所需的字符集添加到连接：

conn = MySQLdb.Connection(db='sampledb', host='localhost',           
user='sampleuser', passwd='samplepass', charset='utf-8', )

这有帮助吗？

Answer 4

终于成功了！感谢：Gord Thompson 和 Prikkeldraad。谢谢大家！

# -*- coding: utf-8 -*-
import csv
import MySQLdb
from UnicodeSupportForCsv import UnicodeWriter
import sys
reload(sys)  
sys.setdefaultencoding('utf8')
#sys.setdefaultencoding('Cp1252')

conn = MySQLdb.Connection(db='testdb', host='localhost', user='testuser', passwd='testpasswd', use_unicode=0,charset='utf8')

crsr = conn.cursor()
crsr.execute("SELECT * FROM rfid")

with open(r'test.csv', 'wb') as csvfile:
    uw = UnicodeWriter(
        csvfile, delimiter=',',quotechar='"', quoting=csv.QUOTE_MINIMAL)

    for row in crsr.fetchall():
        uw.writerow([unicode(col) for col in row])

MySQL 通过 Python 脚本将 CSV 文件导出为 UTF-8

MySQL export to CSV file as UTF-8 via Python script

python

mysql

export-to-csv