psycopg2 fetchmany 与命名游标

Question

根据 Psycopg2 的 server-side-cursor documentation,

If the dataset is too large to be practically handled on the client side, it is possible to create a server side cursor. Using this kind of cursor it is possible to transfer to the client only a controlled amount of data, so that a large dataset can be examined without keeping it entirely in memory.

我可以使用 fetchmany 实现类似的行为：

fetchmany(100) # fetch first 100
fetchmany(50) # fetch next 50
fetchall() # fetch remaining

那么，在获取受控数据量的情况下，fetchmany 与服务器端游标有何不同？在什么情况下我应该更喜欢其中一种？

Answer 1

我认为关键在于该部分的第一段（我的粗体字）：

When a database query is executed, the Psycopg cursor usually fetches all the records returned by the backend, transferring them to the client process. If the query returned an huge amount of data, a proportionally large amount of memory will be allocated by the client.

使用“普通”游标，整个结果集会立即传输到客户端。使用命名游标，您可以根据需要在 client/server 界面上“拉取”数据。

例如，请注意该程序执行时间的差异，这取决于它使用的游标类型（您可能需要调整连接参数）：

import psycopg2, time
conn = psycopg2.connect(dbname='template1')
if False:
    c = conn.cursor('named')
else:
    c = conn.cursor()
c.execute('SELECT GENERATE_SERIES(1,100000000)')
print('Execute ok.')
rows = c.fetchmany(1)
print('Fetch ok.')

psycopg2 fetchmany 与命名游标

psycopg2 fetchmany vs named cursor

python

postgresql

psycopg2

database-cursor