Python SQL 没有为整个 URL 列表插入数据

Question

我正在使用 grequest 模块来使用多个 URL。但是，我这里的问题是我只是从 1 个 URL 而不是列表中的每个 URL 中将数据插入到我的数据库中。

问题：

How can I solve this?

import urllib.request as urllib
import socket
import pyodbc
from datetime import datetime
import ssl
import OpenSSL
import requests
import grequests
#
# Timestamp for undersøgelse
timestamp = datetime.now().strftime('%d-%m-%Y %H:%M:%S')

List = open("C:\Users\Farzad\Desktop\hosts.txt").read().splitlines()

rs = (grequests.get(url) for url in List)

requests = grequests.map(rs)

for response in requests:
    print(response.status_code)

#SQL Connection til local database
con = pyodbc.connect('Driver={SQL Server Native Client 11.0};'
                      'Server=DESKTOP-THV2IDL;'
                      'Database=host;'
                      'Trusted_Connection=yes;')

cursor = con.cursor()

#Insert responseheader data/sikkerhedskontrollers der bliver brugt
cursor.execute('INSERT INTO host.dbo.domain (ip, Host, HSTS, HPKP, XContentTypeOptions, XFrameOptions, ContentSecurityPolicy, XXssProtection, Server, Timestamp) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)', 
        (response.url, 1, response.headers.get('Strict-Transport-Security'), response.headers.get('Public-Key-Pins'), response.headers.get('X-Content-Type-Options'), response.headers.get('X-Frame-Options'), response.headers.get('Content-Security-Policy'), response.headers.get('X-XSS-Protection'), response.headers.get('Server'), timestamp))

con.commit()

Answer 1

在以下几行中，您将迭代响应：

for response in requests:
   print(response.status_code)

然后在这里执行你的SQL代码：

cursor.execute('INSERT INTO host.dbo.domain (ip, Host, HSTS, HPKP, XContentTypeOptions, XFrameOptions, ContentSecurityPolicy, XXssProtection, Server, Timestamp) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)', 
    (response.url, 1, response.headers.get('Strict-Transport-Security'), response.headers.get('Public-Key-Pins'), response.headers.get('X-Content-Type-Options'), response.headers.get('X-Frame-Options'), response.headers.get('Content-Security-Policy'), response.headers.get('X-XSS-Protection'), response.headers.get('Server'), timestamp))

您使用变量 response 并且它是 attributes/methods 传递给您的 SQL 代码。 但是这个响应变量只是上次请求的结果（之前迭代时写的最后一个）

如果您想为每个响应对数据库做一些事情，那么您应该在 for 循环中执行 SQL 代码。

Python SQL 没有为整个 URL 列表插入数据

Python SQL not inserting data for the whole list of URLs

python

sql-server

web-scraping

grequests