多处理如何关闭每个线程的数据库连接

Question

我正在为每个线程创建一个数据库连接，想知道完成后关闭这些连接的最佳方法是什么。

这是我初始化数据库连接和调用池的代码

class Processor(object):
    def __init__(self, DSN):
        self.dsn0= DSN[0]
        self.dsn1= DSN[1]
        self.conn0 = None
        self.conn1 = None


    def __call__(self, data):
        if(self.conn0 is None and self.conn1 is None):
           self.conn0 = psycopg2.connect(self.dsn0)
           self.conn0Curs = self.conn0.cursor()
           self.conn1 = psycopg2.connect(self.dsn1)
           self.conn1Curs = self.conn1.cursor()
       sql = generateQuery(*data)
       print(sql)



if __name__ == '__main__':

    pool = ThreadPool()
    pool.map(Processor([DSN1, DSN2]), batches)
    pool.close()
    pool.join()

Answer 1

我建议您使用 Context Manager，而不是在线程中打开连接。它的主要优点是即使在出现错误的情况下也能确保关闭连接。

class Processor(object):
    def __init__(self, DSN):     
        self.dsn0= DSN[0]
        self.dsn1= DSN[1]
        self.conn0 = None
        self.conn1 = None 
        self.curs0 = None
        self.curs1 = None

    def __enter__(self):
        self.conn0 = psycopg2.connect(self.dsn0)
        self.conn1 = psycopg2.connect(self.dsn1)
        self.curs0 = self.conn0.cursor()
        self.curs1 = self.conn1.cursor()
        return self

    def __exit__(self, *_):
        """Close the connections."""
        self.conn0.close()
        self.conn1.close()

    def process(self, data):
        sql = generateQuery(*data)
        print(sql)


if __name__ == '__main__':
    with Processor([DSN1, DSN2]) as processor:
        pool = ThreadPool()
        pool.map(processor.process, batches)
        pool.close()
        pool.join()

多处理如何关闭每个线程的数据库连接

multiprocessing how to close db connections for each thread

python

python-multiprocessing