0
0

python 爬虫 入门 ：一点小实战（爬取小说）。

    小说行当不好做，笔趣阁犹如雨后春笋。你要说笔趣阁的资源哪来的？指不定也是从“笔趣阁”中来。今天咱就来爬一部小说试试（侵删）

    随便找个小说网站，随便点本小说，我就不写网址了，看下面图片,准备爬这一部小说，咱先看看网站的robots.txt，没有Disallow，都可以爬，开整。


    到小说具体页面后，先打开源代码查看，发现直接源码里有文字，而且还有一个唯一id“nr1”，可以直接使用。现在就要确定这么多章小说，每个章节的url了。


    小说首页中， 有部分章节的链接，打开源代码查看，发现小说具体章节的链接没什么规律，必须要从目录获取，目录名有规律，index_n.html,n取值1~60即可。


    到这步骤就出来了，先访问index获取具体页面，然后从页面中摘取文本保存到列表中（为了保持顺序，你也可以先每章一个文件，最后综合。），最后 将列表结果保存到txt文档里面即可。大约有1200章，263.8万字，往多了算也就二十来兆，没一个图片大，可行。

   开始编写代码， 当我们需要从源代码中找数据时，可以将源代码复制下来，粘贴到程序中测试一下。使用debug模式尝试解析代码。以避免测试过多访问导致封ip等事情。

from bs4 import BeautifulSoup

str = '''
    目录源代码 https://m.a4aa.com/5755/index_2.html
'''

bs = BeautifulSoup(str, "html.parser")  # 从目录找到章节ip
data = bs.find_all("section", class_="zxjz fk")[1]
data = [i["href"] for i in data.find_all("a")]

print(data)

from bs4 import BeautifulSoup

str = '''
小说内容页面源代码 https://m.a4aa.com/5755/3290542.html
'''

bs = BeautifulSoup(str, "html.parser")  # 找到需要保存的所有内容
title = bs.find_all("div", id="nr_title")[0]  # 章节名
text = '\n' + title.string + '\n\n'  # 这样可以识别出目录。
nr1 = bs.find_all("div", id="nr1")[0]  # 章节内容
text += nr1.text
print(text)

代码写完后，开始尝试爬取，爬取一般后禁止访问，爬的太快不让干了，所以更改逻辑，一次爬20章，爬完后歇会再爬。

代码：

import asyncio
import os
import pickle
import time

import aiohttp
from bs4 import BeautifulSoup

async def get_url(url, headers, urls, num):  # 获取目录num下全部url
    url_f = url + "/5755/index_" + str(num) + ".html"  # 产生目录url
    async with aiohttp.ClientSession() as session:
        async with session.get(url=url_f, headers=headers)as resp:  # 一个目录至少20个
            bs = BeautifulSoup(await resp.text(), "html.parser")  # 从目录找到章节ip
            data = bs.find_all("section", class_="zxjz fk")[1]
            data = [i["href"] for i in data.find_all("a")]
    for l in range(len(data)):
        urls[l + (num - 1) * 20] = url + data[l]
    print("目录", num, "完成")

async def get_text(url, headers, str_data, str_data_ok, num):  # 获取章节需要保存的内容
    if str_data_ok[num]:  # 保存过，不管了
        return
    async with aiohttp.ClientSession() as session:
        async with session.get(url=url, headers=headers)as resp:
            bs = BeautifulSoup(await resp.text(), "html.parser")  # 找到需要保存的所有内容
            title = bs.find_all("div", id="nr_title")[0]  # 章节名
            text = '\n\n' + title.string + '\n\n'  # 这样可以识别出目录。
            nr1 = bs.find_all("div", id="nr1")[0]  # 章节内容
            text += nr1.text
            str_data[num] = text  # 保存章节
            # 完工，标注该章结束
            str_data_ok[num] = True

async def get_all_url(url_1, headers, urls):  # 获取所有url存到urls
    tasks = [asyncio.create_task(get_url(url_1, headers, urls, num)) for num in range(1, 61)]
    # tasks = [asyncio.create_task(get_url(url_1, headers, urls, num)) for num in range(1, 2)]  # 先来20章试试水
    await asyncio.wait(tasks)

async def get_all_text(urls, headers, str_data, str_data_ok, start, end):  # 获取所有章节内容存到str_data
    # tasks = [asyncio.create_task(get_text(urls[num], headers, str_data, num)) for num in range(len(urls))]
    # tasks = [asyncio.create_task(get_text(urls[num], headers, str_data, num)) for num in range(20)]
    # 小网站，给人爬崩了，一次请求一部分。
    tasks = [asyncio.create_task(get_text(urls[num], headers, str_data, str_data_ok, num)) for num in range(start, end)]
    await asyncio.wait(tasks)

if __name__ == "__main__":

    # 初始化变量======================================================================

    now_time = [time.time(), 0, 0, 0]
    seat = '小说.pkl'  # python变量保存位置，用于恢复运行
    url_1 = "https://m.a4aa.com"
    max_len = 1188
    urls = [""] * max_len  # 所有章节url
    urls_ok = False  # 章节url是否保存完整
    all_text_data = [""] * max_len  # 所有数据url
    all_text_data_ok = [False] * max_len  # 相应章节是否已经保存
    save_path = "小说.txt"  # 保存位置
    headers = {
        # 用户代理，某些网站验证用户代理，微微改一下，如果提示要验证码之类的，使用它
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 "
                      "Safari/537.36 Edg/126.0.0.0",
    }

    #   加载旧变量 ======================================================================

    if os.path.exists(seat):  # 加载过去文件
        print("检测到已保存文件,是否加载(Y\\N):")
        shuru = input()
        if shuru == 'Y' or shuru == 'y':
            with open(seat, 'rb') as f:
                old_data = pickle.load(f)
                urls = old_data["urls"]  # 所有url
                urls_ok = old_data["urls_ok"]
                all_text_data = old_data["all_text_data"]  # 所有章节文本
                all_text_data_ok = old_data["all_text_data_ok"]
            print("已完成：", sum(all_text_data_ok) * 100 / len(all_text_data_ok))
            print("已加载")
        else:
            print("未加载")

    # 开始执行程序=======================================================================
    try:  # 出错就保存一下

        # 获取目录===========================================================================

        if not urls_ok:
            loop = asyncio.get_event_loop()
            loop.run_until_complete(get_all_url(url_1, headers, urls))
            urls_ok = True
            print("目录获取完成，开始获取章节", '======================================================')
        else:
            print("目录已保存过，开始获取章节", '======================================================')
        now_time[1] = time.time()
        print("用时：", now_time[1] - now_time[0], 's')

        # 获取章节===============================================================================

        now_num = 0  # 开始章节数-1
        while True:  # 一个小循环，每次拿一章内容保存，保存完休息会
            in_time = time.time()
            now_num2 = now_num + 20  # 结束章节
            if now_num2 > max_len:  # 超限结束
                now_num2 = max_len
                loop = asyncio.get_event_loop()
                loop.run_until_complete(get_all_text(urls, headers, all_text_data, all_text_data_ok, now_num, now_num2))
                break
            else:
                loop = asyncio.get_event_loop()
                loop.run_until_complete(get_all_text(urls, headers, all_text_data, all_text_data_ok, now_num, now_num2))
            print('已保存', now_num + 1, now_num2, '\n用时：', time.time() - in_time, '小睡一下')
            time.sleep(5)
            print('睡眠完毕')
            now_num += 20

        print("文章已全部获取", '======================================================')
        now_time[2] = time.time()
        print("用时：", now_time[2] - now_time[1], 's')
        print("开始保存", '======================================================')
        now_time[3] = time.time()

        # 保存文本=====================================================================

        with open(save_path, mode='w', encoding='utf-8') as f:
            for text in all_text_data:
                f.write(text)
        print("用时：", now_time[3] - now_time[2], 's')
        print("总用时：", now_time[3] - now_time[0], 's')
        print("已完成：", sum(all_text_data_ok) * 100 / len(all_text_data_ok))
        all_data = {"urls": urls,  # 所有url
                    "urls_ok": urls_ok,
                    "all_text_data": all_text_data,  # 所有章节文本
                    "all_text_data_ok": all_text_data_ok}
        with open(seat, 'wb') as f:  # 保存变量
            pickle.dump(all_data, f)

    except Exception as e:
        print("已完成：", sum(all_text_data_ok) * 100 / len(all_text_data_ok))
        all_data = {"urls": urls,  # 所有url
                    "urls_ok": urls_ok,
                    "all_text_data": all_text_data,  # 所有章节文本
                    "all_text_data_ok": all_text_data_ok}
        with open(seat, 'wb') as f:  # 保存变量
            pickle.dump(all_data, f)

结果：第一次运行：（有一些超时的，问题不大）

C:\Users\86135\AppData\Local\Programs\Python\Python39\python.exe C:/Users/86135/PycharmProjects/pythonProject/测试/测试.py
检测到已保存文件,是否加载(Y\N):
y
已完成： 0.0
已加载
目录 6 完成
目录 25 完成
目录 51 完成
目录 22 完成
目录 31 完成
目录 42 完成
目录 30 完成
目录 53 完成
目录 20 完成
目录 13 完成
目录 32 完成
目录 5 完成
目录 45 完成
目录 41 完成
目录 21 完成
目录 12 完成
目录 4 完成
目录 44 完成
目录 2 完成
目录 7 完成
目录 37 完成
目录 38 完成
目录 40 完成
目录 15 完成
目录 55 完成
目录 48 完成
目录 29 完成
目录 50 完成
目录 57 完成
目录 47 完成
目录 59 完成
目录 28 完成
目录 3 完成
目录 26 完成
目录 33 完成
目录 10 完成
目录 56 完成
目录 43 完成
目录 39 完成
目录 8 完成
目录 1 完成
目录 35 完成
目录 14 完成
目录 27 完成
目录 60 完成
目录 34 完成
目录 23 完成
目录 54 完成
目录 19 完成
目录 24 完成
目录 17 完成
目录 9 完成
目录 18 完成
目录 52 完成
目录 58 完成
目录 36 完成
目录 11 完成
目录 49 完成
目录 16 完成
目录 46 完成
目录获取完成，开始获取章节 ======================================================
用时： 26.672559022903442 s
已保存 1 20 
用时： 0.9862778186798096 小睡一下
睡眠完毕
已保存 21 40 
用时： 0.9691689014434814 小睡一下
睡眠完毕
已保存 41 60 
用时： 0.9766478538513184 小睡一下
睡眠完毕
已保存 61 80 
用时： 0.9963347911834717 小睡一下
睡眠完毕
已保存 81 100 
用时： 0.9797823429107666 小睡一下
睡眠完毕
已保存 101 120 
用时： 1.000009536743164 小睡一下
睡眠完毕
已保存 121 140 
用时： 0.9792578220367432 小睡一下
睡眠完毕
已保存 141 160 
用时： 0.9902822971343994 小睡一下
睡眠完毕
已保存 161 180 
用时： 0.9691574573516846 小睡一下
睡眠完毕
已保存 181 200 
用时： 30.67471194267273 小睡一下
Task exception was never retrieved
future: <Task finished name='Task-734' coro=<get_text() done, defined at C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py:22> exception=ConnectionTimeoutError('Connection timeout to host https://m.a4aa.com/5755/3293235.html')>
Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1081, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1111, in _create_connection_transport
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 663, in _request
    conn = await self._connector.connect(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 563, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1032, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1366, in _create_direct_connection
    raise last_exc
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1335, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 141, in __aexit__
    self._do_exit(exc_type)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 228, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py", line 26, in get_text
    async with session.get(url=url, headers=headers)as resp:
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 1359, in __aenter__
    self._resp: _RetType = await self._coro
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 667, in _request
    raise ConnectionTimeoutError(
aiohttp.client_exceptions.ConnectionTimeoutError: Connection timeout to host https://m.a4aa.com/5755/3293235.html
睡眠完毕
Task exception was never retrieved
future: <Task finished name='Task-797' coro=<get_text() done, defined at C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py:22> exception=ConnectionTimeoutError('Connection timeout to host https://m.a4aa.com/5755/3293528.html')>
Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1081, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1111, in _create_connection_transport
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 663, in _request
    conn = await self._connector.connect(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 563, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1032, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1366, in _create_direct_connection
    raise last_exc
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1335, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 141, in __aexit__
    self._do_exit(exc_type)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 228, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py", line 26, in get_text
    async with session.get(url=url, headers=headers)as resp:
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 1359, in __aenter__
    self._resp: _RetType = await self._coro
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 667, in _request
    raise ConnectionTimeoutError(
aiohttp.client_exceptions.ConnectionTimeoutError: Connection timeout to host https://m.a4aa.com/5755/3293528.html
Task exception was never retrieved
future: <Task finished name='Task-796' coro=<get_text() done, defined at C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py:22> exception=ConnectionTimeoutError('Connection timeout to host https://m.a4aa.com/5755/3293521.html')>
Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1081, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1111, in _create_connection_transport
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 663, in _request
    conn = await self._connector.connect(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 563, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1032, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1366, in _create_direct_connection
    raise last_exc
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1335, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 141, in __aexit__
    self._do_exit(exc_type)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 228, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py", line 26, in get_text
    async with session.get(url=url, headers=headers)as resp:
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 1359, in __aenter__
    self._resp: _RetType = await self._coro
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 667, in _request
    raise ConnectionTimeoutError(
aiohttp.client_exceptions.ConnectionTimeoutError: Connection timeout to host https://m.a4aa.com/5755/3293521.html
已保存 201 220 
用时： 30.99604558944702 小睡一下
睡眠完毕
已保存 221 240 
用时： 30.971959352493286 小睡一下
Task exception was never retrieved
future: <Task finished name='Task-853' coro=<get_text() done, defined at C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py:22> exception=ConnectionTimeoutError('Connection timeout to host https://m.a4aa.com/5755/3293732.html')>
Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1081, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1111, in _create_connection_transport
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 663, in _request
    conn = await self._connector.connect(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 563, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1032, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1366, in _create_direct_connection
    raise last_exc
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1335, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 141, in __aexit__
    self._do_exit(exc_type)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 228, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py", line 26, in get_text
    async with session.get(url=url, headers=headers)as resp:
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 1359, in __aenter__
    self._resp: _RetType = await self._coro
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 667, in _request
    raise ConnectionTimeoutError(
aiohttp.client_exceptions.ConnectionTimeoutError: Connection timeout to host https://m.a4aa.com/5755/3293732.html
睡眠完毕
已保存 241 260 
用时： 6.197637319564819 小睡一下
睡眠完毕
已保存 261 280 
用时： 1.0338616371154785 小睡一下
睡眠完毕
已保存 281 300 
用时： 2.0356712341308594 小睡一下
睡眠完毕
已保存 301 320 
用时： 0.9921369552612305 小睡一下
睡眠完毕
已保存 321 340 
用时： 1.0640060901641846 小睡一下
睡眠完毕
已保存 341 360 
用时： 1.0201935768127441 小睡一下
睡眠完毕
已保存 361 380 
用时： 1.0203821659088135 小睡一下
睡眠完毕
已保存 381 400 
用时： 0.9939565658569336 小睡一下
睡眠完毕
已保存 401 420 
用时： 1.0569934844970703 小睡一下
睡眠完毕
已保存 421 440 
用时： 1.0331060886383057 小睡一下
睡眠完毕
已保存 441 460 
用时： 1.0108842849731445 小睡一下
睡眠完毕
已保存 461 480 
用时： 1.013477087020874 小睡一下
睡眠完毕
已保存 481 500 
用时： 24.80794596672058 小睡一下
睡眠完毕
已保存 501 520 
用时： 1.0152294635772705 小睡一下
睡眠完毕
已保存 521 540 
用时： 22.0968017578125 小睡一下
睡眠完毕
已保存 541 560 
用时： 1.0083670616149902 小睡一下
睡眠完毕
已保存 561 580 
用时： 21.392114400863647 小睡一下
睡眠完毕
已保存 581 600 
用时： 30.079055547714233 小睡一下
Task exception was never retrieved
future: <Task finished name='Task-1949' coro=<get_text() done, defined at C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py:22> exception=ConnectionTimeoutError('Connection timeout to host https://m.a4aa.com/5755/3299056.html')>
Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1081, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1111, in _create_connection_transport
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 663, in _request
    conn = await self._connector.connect(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 563, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1032, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1366, in _create_direct_connection
    raise last_exc
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1335, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 141, in __aexit__
    self._do_exit(exc_type)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 228, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py", line 26, in get_text
    async with session.get(url=url, headers=headers)as resp:
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 1359, in __aenter__
    self._resp: _RetType = await self._coro
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 667, in _request
    raise ConnectionTimeoutError(
aiohttp.client_exceptions.ConnectionTimeoutError: Connection timeout to host https://m.a4aa.com/5755/3299056.html
睡眠完毕
已保存 601 620 
用时： 1.0188708305358887 小睡一下
睡眠完毕
已保存 621 640 
用时： 1.039496660232544 小睡一下
睡眠完毕
已保存 641 660 
用时： 1.0427892208099365 小睡一下
睡眠完毕
已保存 661 680 
用时： 1.0124883651733398 小睡一下
睡眠完毕
已保存 681 700 
用时： 1.0155725479125977 小睡一下
睡眠完毕
已保存 701 720 
用时： 1.0273873805999756 小睡一下
睡眠完毕
已保存 721 740 
用时： 1.036287546157837 小睡一下
睡眠完毕
已保存 741 760 
用时： 1.0013742446899414 小睡一下
睡眠完毕
已保存 761 780 
用时： 1.0504977703094482 小睡一下
睡眠完毕
已保存 781 800 
用时： 1.0260164737701416 小睡一下
睡眠完毕
已保存 801 820 
用时： 30.63787031173706 小睡一下
Task exception was never retrieved
future: <Task finished name='Task-2636' coro=<get_text() done, defined at C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py:22> exception=ConnectionTimeoutError('Connection timeout to host https://m.a4aa.com/5755/13370518.html')>
Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1081, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1111, in _create_connection_transport
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 663, in _request
    conn = await self._connector.connect(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 563, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1032, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1366, in _create_direct_connection
    raise last_exc
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1335, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 141, in __aexit__
    self._do_exit(exc_type)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 228, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py", line 26, in get_text
    async with session.get(url=url, headers=headers)as resp:
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 1359, in __aenter__
    self._resp: _RetType = await self._coro
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 667, in _request
    raise ConnectionTimeoutError(
aiohttp.client_exceptions.ConnectionTimeoutError: Connection timeout to host https://m.a4aa.com/5755/13370518.html
睡眠完毕
已保存 821 840 
用时： 30.976993560791016 小睡一下
Task exception was never retrieved
future: <Task finished name='Task-2679' coro=<get_text() done, defined at C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py:22> exception=ConnectionTimeoutError('Connection timeout to host https://m.a4aa.com/5755/13404059.html')>
Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1081, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1111, in _create_connection_transport
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 663, in _request
    conn = await self._connector.connect(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 563, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1032, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1366, in _create_direct_connection
    raise last_exc
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1335, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 141, in __aexit__
    self._do_exit(exc_type)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 228, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py", line 26, in get_text
    async with session.get(url=url, headers=headers)as resp:
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 1359, in __aenter__
    self._resp: _RetType = await self._coro
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 667, in _request
    raise ConnectionTimeoutError(
aiohttp.client_exceptions.ConnectionTimeoutError: Connection timeout to host https://m.a4aa.com/5755/13404059.html
睡眠完毕
已保存 841 860 
用时： 1.0284440517425537 小睡一下
睡眠完毕
已保存 861 880 
用时： 1.0212693214416504 小睡一下
睡眠完毕
已保存 881 900 
用时： 1.0207250118255615 小睡一下
睡眠完毕
已保存 901 920 
用时： 1.0172266960144043 小睡一下
睡眠完毕
已保存 921 940 
用时： 1.0471982955932617 小睡一下
睡眠完毕
已保存 941 960 
用时： 1.0365018844604492 小睡一下
睡眠完毕
已保存 961 980 
用时： 0.9948766231536865 小睡一下
睡眠完毕
已保存 981 1000 
用时： 3.99578857421875 小睡一下
睡眠完毕
已保存 1001 1020 
用时： 11.078922271728516 小睡一下
睡眠完毕
Task exception was never retrieved
future: <Task finished name='Task-3294' coro=<get_text() done, defined at C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py:22> exception=ConnectionTimeoutError('Connection timeout to host https://m.a4aa.com/5755/19071953.html')>
Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1081, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1111, in _create_connection_transport
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 663, in _request
    conn = await self._connector.connect(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 563, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1032, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1366, in _create_direct_connection
    raise last_exc
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1335, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 141, in __aexit__
    self._do_exit(exc_type)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 228, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py", line 26, in get_text
    async with session.get(url=url, headers=headers)as resp:
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 1359, in __aenter__
    self._resp: _RetType = await self._coro
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 667, in _request
    raise ConnectionTimeoutError(
aiohttp.client_exceptions.ConnectionTimeoutError: Connection timeout to host https://m.a4aa.com/5755/19071953.html
Task exception was never retrieved
future: <Task finished name='Task-3293' coro=<get_text() done, defined at C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py:22> exception=ConnectionTimeoutError('Connection timeout to host https://m.a4aa.com/5755/19071952.html')>
Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1081, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1111, in _create_connection_transport
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 663, in _request
    conn = await self._connector.connect(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 563, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1032, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1366, in _create_direct_connection
    raise last_exc
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1335, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 141, in __aexit__
    self._do_exit(exc_type)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 228, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py", line 26, in get_text
    async with session.get(url=url, headers=headers)as resp:
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 1359, in __aenter__
    self._resp: _RetType = await self._coro
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 667, in _request
    raise ConnectionTimeoutError(
aiohttp.client_exceptions.ConnectionTimeoutError: Connection timeout to host https://m.a4aa.com/5755/19071952.html
已保存 1021 1040 
用时： 30.70367741584778 小睡一下
睡眠完毕
已保存 1041 1060 
用时： 1.005244255065918 小睡一下
睡眠完毕
Task exception was never retrieved
future: <Task finished name='Task-3424' coro=<get_text() done, defined at C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py:22> exception=ConnectionTimeoutError('Connection timeout to host https://m.a4aa.com/5755/19791607.html')>
Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1081, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1111, in _create_connection_transport
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 663, in _request
    conn = await self._connector.connect(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 563, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1032, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1366, in _create_direct_connection
    raise last_exc
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1335, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 141, in __aexit__
    self._do_exit(exc_type)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 228, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py", line 26, in get_text
    async with session.get(url=url, headers=headers)as resp:
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 1359, in __aenter__
    self._resp: _RetType = await self._coro
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 667, in _request
    raise ConnectionTimeoutError(
aiohttp.client_exceptions.ConnectionTimeoutError: Connection timeout to host https://m.a4aa.com/5755/19791607.html
Task exception was never retrieved
future: <Task finished name='Task-3413' coro=<get_text() done, defined at C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py:22> exception=ConnectionTimeoutError('Connection timeout to host https://m.a4aa.com/5755/19620830.html')>
Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1081, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1111, in _create_connection_transport
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 663, in _request
    conn = await self._connector.connect(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 563, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1032, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1366, in _create_direct_connection
    raise last_exc
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1335, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 141, in __aexit__
    self._do_exit(exc_type)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 228, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py", line 26, in get_text
    async with session.get(url=url, headers=headers)as resp:
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 1359, in __aenter__
    self._resp: _RetType = await self._coro
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 667, in _request
    raise ConnectionTimeoutError(
aiohttp.client_exceptions.ConnectionTimeoutError: Connection timeout to host https://m.a4aa.com/5755/19620830.html
已保存 1061 1080 
用时： 30.98487401008606 小睡一下
Task exception was never retrieved
future: <Task finished name='Task-3412' coro=<get_text() done, defined at C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py:22> exception=ConnectionTimeoutError('Connection timeout to host https://m.a4aa.com/5755/19620828.html')>
Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1081, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1111, in _create_connection_transport
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 663, in _request
    conn = await self._connector.connect(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 563, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1032, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1366, in _create_direct_connection
    raise last_exc
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1335, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 141, in __aexit__
    self._do_exit(exc_type)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 228, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py", line 26, in get_text
    async with session.get(url=url, headers=headers)as resp:
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 1359, in __aenter__
    self._resp: _RetType = await self._coro
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 667, in _request
    raise ConnectionTimeoutError(
aiohttp.client_exceptions.ConnectionTimeoutError: Connection timeout to host https://m.a4aa.com/5755/19620828.html
睡眠完毕
已保存 1081 1100 
用时： 1.0197718143463135 小睡一下
睡眠完毕
已保存 1101 1120 
用时： 1.0471570491790771 小睡一下
睡眠完毕
已保存 1121 1140 
用时： 1.0287730693817139 小睡一下
睡眠完毕
已保存 1141 1160 
用时： 0.9939365386962891 小睡一下
睡眠完毕
已保存 1161 1180 
用时： 30.855279684066772 小睡一下
Task exception was never retrieved
future: <Task finished name='Task-3716' coro=<get_text() done, defined at C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py:22> exception=ConnectionTimeoutError('Connection timeout to host https://m.a4aa.com/5755/23189130.html')>
Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1081, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1111, in _create_connection_transport
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 663, in _request
    conn = await self._connector.connect(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 563, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1032, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1366, in _create_direct_connection
    raise last_exc
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1335, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 141, in __aexit__
    self._do_exit(exc_type)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 228, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py", line 26, in get_text
    async with session.get(url=url, headers=headers)as resp:
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 1359, in __aenter__
    self._resp: _RetType = await self._coro
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 667, in _request
    raise ConnectionTimeoutError(
aiohttp.client_exceptions.ConnectionTimeoutError: Connection timeout to host https://m.a4aa.com/5755/23189130.html
睡眠完毕
文章已全部获取 ======================================================
用时： 738.5638995170593 s
开始保存 ======================================================
Task exception was never retrieved
future: <Task finished name='Task-3772' coro=<get_text() done, defined at C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py:22> exception=ConnectionTimeoutError('Connection timeout to host https://m.a4aa.com/5755/26752119.html')>
Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1081, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1111, in _create_connection_transport
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 663, in _request
    conn = await self._connector.connect(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 563, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1032, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1366, in _create_direct_connection
    raise last_exc
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1335, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1098, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs, sock=sock)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 141, in __aexit__
    self._do_exit(exc_type)
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\async_timeout\__init__.py", line 228, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\86135\PycharmProjects\pythonProject\测试\测试.py", line 26, in get_text
    async with session.get(url=url, headers=headers)as resp:
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 1359, in __aenter__
    self._resp: _RetType = await self._coro
  File "C:\Users\86135\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 667, in _request
    raise ConnectionTimeoutError(
aiohttp.client_exceptions.ConnectionTimeoutError: Connection timeout to host https://m.a4aa.com/5755/26752119.html
用时： 0.0 s
总用时： 765.2364585399628 s
已完成： 98.82154882154882

进程已结束，退出代码为 0

第二次运行：(将之前超时失败的全部整完。)

C:\Users\86135\AppData\Local\Programs\Python\Python39\python.exe C:/Users/86135/PycharmProjects/pythonProject/测试/测试.py
检测到已保存文件,是否加载(Y\N):
y
已完成： 98.82154882154882
已加载
目录已保存过，开始获取章节 ======================================================
用时： 1.8765432834625244 s
已保存 1 20 
用时： 0.0020132064819335938 小睡一下
睡眠完毕
已保存 21 40 
用时： 0.0 小睡一下
睡眠完毕
已保存 41 60 
用时： 0.0 小睡一下
睡眠完毕
已保存 61 80 
用时： 0.0 小睡一下
睡眠完毕
已保存 81 100 
用时： 0.0 小睡一下
睡眠完毕
已保存 101 120 
用时： 0.0 小睡一下
睡眠完毕
已保存 121 140 
用时： 0.0 小睡一下
睡眠完毕
已保存 141 160 
用时： 0.0 小睡一下
睡眠完毕
已保存 161 180 
用时： 0.0 小睡一下
睡眠完毕
已保存 181 200 
用时： 1.1031506061553955 小睡一下
睡眠完毕
已保存 201 220 
用时： 1.1709847450256348 小睡一下
睡眠完毕
已保存 221 240 
用时： 1.0908143520355225 小睡一下
睡眠完毕
已保存 241 260 
用时： 0.0 小睡一下
睡眠完毕
已保存 261 280 
用时： 0.0 小睡一下
睡眠完毕
已保存 281 300 
用时： 0.0 小睡一下
睡眠完毕
已保存 301 320 
用时： 0.0 小睡一下
睡眠完毕
已保存 321 340 
用时： 0.0 小睡一下
睡眠完毕
已保存 341 360 
用时： 0.0 小睡一下
睡眠完毕
已保存 361 380 
用时： 0.0 小睡一下
睡眠完毕
已保存 381 400 
用时： 0.0 小睡一下
睡眠完毕
已保存 401 420 
用时： 0.0 小睡一下
睡眠完毕
已保存 421 440 
用时： 0.0 小睡一下
睡眠完毕
已保存 441 460 
用时： 0.0 小睡一下
睡眠完毕
已保存 461 480 
用时： 0.0 小睡一下
睡眠完毕
已保存 481 500 
用时： 0.0 小睡一下
睡眠完毕
已保存 501 520 
用时： 0.0 小睡一下
睡眠完毕
已保存 521 540 
用时： 0.0 小睡一下
睡眠完毕
已保存 541 560 
用时： 0.0 小睡一下
睡眠完毕
已保存 561 580 
用时： 0.0 小睡一下
睡眠完毕
已保存 581 600 
用时： 1.1043505668640137 小睡一下
睡眠完毕
已保存 601 620 
用时： 0.0 小睡一下
睡眠完毕
已保存 621 640 
用时： 0.0 小睡一下
睡眠完毕
已保存 641 660 
用时： 0.0 小睡一下
睡眠完毕
已保存 661 680 
用时： 0.0 小睡一下
睡眠完毕
已保存 681 700 
用时： 0.0 小睡一下
睡眠完毕
已保存 701 720 
用时： 0.0 小睡一下
睡眠完毕
已保存 721 740 
用时： 0.0 小睡一下
睡眠完毕
已保存 741 760 
用时： 0.0 小睡一下
睡眠完毕
已保存 761 780 
用时： 0.0 小睡一下
睡眠完毕
已保存 781 800 
用时： 0.0 小睡一下
睡眠完毕
已保存 801 820 
用时： 1.184851884841919 小睡一下
睡眠完毕
已保存 821 840 
用时： 1.1437313556671143 小睡一下
睡眠完毕
已保存 841 860 
用时： 0.0 小睡一下
睡眠完毕
已保存 861 880 
用时： 0.0 小睡一下
睡眠完毕
已保存 881 900 
用时： 0.0 小睡一下
睡眠完毕
已保存 901 920 
用时： 0.0 小睡一下
睡眠完毕
已保存 921 940 
用时： 0.0 小睡一下
睡眠完毕
已保存 941 960 
用时： 0.0 小睡一下
睡眠完毕
已保存 961 980 
用时： 0.0 小睡一下
睡眠完毕
已保存 981 1000 
用时： 0.0 小睡一下
睡眠完毕
已保存 1001 1020 
用时： 0.0 小睡一下
睡眠完毕
已保存 1021 1040 
用时： 1.0622766017913818 小睡一下
睡眠完毕
已保存 1041 1060 
用时： 0.0 小睡一下
睡眠完毕
已保存 1061 1080 
用时： 1.0756304264068604 小睡一下
睡眠完毕
已保存 1081 1100 
用时： 0.0 小睡一下
睡眠完毕
已保存 1101 1120 
用时： 0.0 小睡一下
睡眠完毕
已保存 1121 1140 
用时： 0.0 小睡一下
睡眠完毕
已保存 1141 1160 
用时： 0.0 小睡一下
睡眠完毕
已保存 1161 1180 
用时： 1.0669140815734863 小睡一下
睡眠完毕
文章已全部获取 ======================================================
用时： 306.5122067928314 s
开始保存 ======================================================
用时： 0.0 s
总用时： 308.38875007629395 s
已完成： 100.0

进程已结束，退出代码为 0

这是最后的文件结果：

发送到手机上看看：没有问题。

标签： 爬虫
本文转载自: https://blog.csdn.net/weixin_58196051/article/details/143159343
版权归原作者 i道i 所有， 如有侵权，请联系我们删除。
登录后发布评论

“python 爬虫 入门 ：一点小实战（爬取小说）。”的评论:

还没有评论
关于作者
...
overfit同步小助手
文章同步
相关阅读

安全见闻（9）二进制安全

C++20 STL CookBook2 更强大的编译时 + 安全比较 + spaceship比较符

Linux：login shell和non-login shell以及其配置文件

使用git命令实现对gitee仓库的下载、更新、上传、上传更新操作。

network-magic：网络管理员必备的开源网络管理工具

git-- github的使用

结合Gitee、PicGo和Typora构建一个免费的个人笔记工具

文章导航