使 aiohttp 会话保持活动状态

Question

我正在尝试每隔 X 秒访问一个网站，并使用并行和单独的会话，然后分析响应中的内容以查看每个会话是否应该继续。但是，一旦代码到达第二个循环，它就会失败。

import asyncio
from aiohttp import ClientSession
import logging
import time

interval = 30
instances = 2
visit_url = 'http://www.example.org'

tasks = []

logging.basicConfig(
    format='%(asctime)s.%(msecs)03d %(message)s',  # Log in format time.milliseconds {message}
    level=logging.INFO,  # Use with logging.info()
    datefmt='%H:%M:%S')  # Display time as Hours:Minutes:Seconds


class StopException(Exception):
    pass


async def quit_app(session, task_, reason):
    logging.info("[{}] {}.".format(task_, reason))
    session.cookies.clear()  # Reset cookies
    session.headers.clear()  # Reset headers
    session.close()  # End HTTP connection
    raise StopException


async def get_status(response):
    if "abow" in response:
        return "success"
    elif "odoap" or "daoscp" in response:
        return "waiting"
    elif "nullt" in response:
        return "fail"
    elif "issue" in response:
        return "banned"
    elif "pending" in response:
        return "pending"
    else:
        return "pending"


async def initialise(headers):
    session = ClientSession()
    task_ = len(asyncio.Task.all_tasks()) - instances - 1
    passed = False
    while passed is False:
        async with session as session:
            async with session.get(visit_url, headers=headers, allow_redirects=True) as initial:
                status = await get_status(await initial.text())  # Check HTML for status
                if status == "success":
                    logging.info("[{}] {}.".format(task_, "Success"))
                    passed = True
                elif status == "pending":
                    logging.info("[{}] {}.".format(task_, "Pending.."))
                    await asyncio.sleep(interval)
                elif status == "waiting":
                    logging.info("[{}] {}.".format(task_, "Waiting..."))
                    await asyncio.sleep(interval)
                elif status == "banned":
                    await quit_app(initial, task_, "Banned")
                elif status == "fail":
                    await quit_app(initial, task_, "Failed")


if __name__ == "__main__":
    headers = {
        'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
        'accept-encoding': 'gzip, deflate, br',
        'accept-language': 'en-US,en;q=0.9',
        'upgrade-insecure-asks': '1',
        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36'
    }  # Add appropriate headers
    start = time.clock()
    loop = asyncio.get_event_loop()
    for i in range(instances):
        task = asyncio.ensure_future(initialise(headers))
        tasks.append(task)
    loop.run_until_complete(asyncio.wait(tasks))
    end = time.clock()
    print("Process took {0:.4f} seconds.".format(end - start))

此代码returns出现以下错误：

13:56:58.604 Task exception was never retrieved future: Task finished coro= initialise() done, defined at C:/Users/x/PycharmProjects/tests/src/aiohttp_main.py:49 exception=RuntimeError('Session is closed',) RuntimeError: Session is closed

我无法弄清楚如何让会话保持活动状态，直到我 .close() 它们...

Answer 1

I just can't work out how to make the sessions stay alive until I .close() them...

async with 是关闭会话的请求。这意味着在你写完之后：

async with session as session:

...一旦 async with 的主体执行完毕，您就不能再使用 session。这不是 aiohttp 特有的，它是 with 在 Python 中的工作方式。例如，在处理文件时，with 是关闭文件的请求：

with open('data.csv') as fileobj:
    # ... read stuff from fileobj

# outside the "with" block, fileobj is closed and you
# can no longer read from it

修复很简单，只需将 with 移到 while 循环之外。例如：

async def initialise(headers):
    async with ClientSession() as session:
        # the rest of the code, including the `while` loop, here

顺便说一句，您可能想用您自己的全局计数器替换 len(asyncio.Task.all_tasks())。如果您稍后将其他不相关的任务合并到事件循环中（或第三方库为您完成），以这种方式使用 Task.all_tasks() 可能会开始产生不正确的结果。

使 aiohttp 会话保持活动状态

Keep aiohttp session alive

python

python-asyncio

aiohttp