将 csv 文件插入 SQL 服务器数据库：listdir 出错？

Question

我知道在这里扔代码并寻求帮助来解决问题是不好的。这个问题似乎有点让我头疼。

代码应该遍历所有文件和子文件夹。我不认为这里有任何日志错误。问题是我运行再次处理同一个文件并导致数据库插入在主键约束上失败的问题。

这是我的代码：

import csv
import pypyodbc
import os
import sys

extension = ".tsv"
connStr = """DSN=database_test;"""

sys.stdout = open('c:\temp\python.log', 'w')
print 'starting ...'

def LoadFile(path):   
    i = 0 
    for item in os.listdir(path): # loop through items in dir        
        full_path = os.path.join(path, item)
        if os.path.isfile(full_path) and full_path.endswith(extension): # check for ".tsv" extension

            if full_path.find('IM') > 0:
                table_name = 'table_a'
            else:
                table_name = 'table_b'

            if os.stat(full_path).st_size > 0:
                print "Processing file:", i, "|", full_path
                i = i + 1
                with open (full_path, 'r') as f:
                    reader = csv.reader(f, delimiter='\t')
                    columns = next(reader) 
                    query = 'insert into ' + table_name + '({0}) values ({1})'
                    crsr = cnxn.cursor()
                    for data in reader:
                        query = query.format(', '.join(columns), ', '.join('?' * len(columns)))          
                        #print(query, "with ", data)
                        if(data[1] != ''):
                            crsr.execute(query, data)
                    crsr.commit()  
                    crsr.close()                                
        elif os.path.isdir(full_path):                        
            print "Process Folder: ", full_path    
            LoadFile(full_path)                        
        else:
            print("invalid file name:", item)
    print "Process Folder total files: ", i, ":", full_path
    return         

cnxn = pypyodbc.connect(connStr)

dir_name = 'X:\TopLevelFolder'
LoadFile(dir_name)

cnxn.close()
print("Completed")

Answer 1

我原来的问题有点误导...由于某些无法解释的硬件问题，代码无法运行。在我切换到 python 2.7 并重新安装所有库之后。代码工作正常。

将 csv 文件插入 SQL 服务器数据库：listdir 出错？

Insert csv file into SQL Server database: error with listdir?

sql-server

csv

pypyodbc