如何在 python 中多线程 post 请求

Question

我正在尝试使用 python 对 post 请求进行多线程处理，以将哈希值发送到 malwarebazaar。 1st 我正在从文件中读取哈希值，然后我尝试对其进行多线程处理，但我被卡住了我的代码没有使用所有哈希值，哪里出错了？

import re
import json
import sys
import requests
from time import time
from concurrent.futures import ThreadPoolExecutor, as_completed
f=open('hashes.txt')
lines= f.readlines()
url='https://mb-api.abuse.ch/api/v1'
array=[]
for line in lines:
    line=line.replace('\n','')
    data={'query':'get_info','hash':line}
    array.append(data)
#print(array)

#start=time()
print(url)
def function(url):
        html = requests.post(url, data=array,stream=True)
        print(html.json())
        return html

start = time()

processes = []
with ThreadPoolExecutor(max_workers=10) as executor:
    for url in url:
     processes.append(executor.submit(function, url))

#for task in as_completed(processes):
#   print(task.result())

print(f'Time taken: {time() - start}')
#c = Counter(array)
#print(c)

Answer 1

您可以使用 greenlets 进行并行请求。

from gevent.pool import Group

print (starttime)
group = Group() #start parallel fn calls
group.spawn(DoRequest,param1,param2)
group.spawn(DoRequest,param1,param2)
group.spawn(DoRequest,param1,param2)
group.join() #wait for all to finish
print (endtime)

def DoRequest(param1,param2):
    #make request and get response

Answer 2

我是这样工作的：

测试数据(hashes.txt)

bd4db5d00ba4633516169666635be7dee18a1916585a32def38498c0062b48a7
92b2ba8088561d9b67f64ee433b0f9a82599ad194672ab15f689ded2ed4d3c51
80a606be75ab17bfd2096e6f1efbb0df1968e72b4337cff0a0fc3016b088e794
b164e9f4c24d002fe0d8975972e569b3873d0d8ae3e6075f548498e261874b42
f676742212a35929267bfd3750a0bbd5609de0cc2ad43955331d2b3f27af6e8f

代码：

import re, sys, requests, time, concurrent.futures
url='https://mb-api.abuse.ch/api/v1'
array=[]
with open("hashes.txt") as f:
    for line in f:
        array.append({'query':'get_info','hash':line.rstrip("\n")})

def function(payload):
    with requests.post(url, data=payload,stream=True) as response:
        html = response.json()
        return(html)

start = time.time()

with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
    processes = {executor.submit(function, query) for query in array}
    for result in concurrent.futures.as_completed(processes):
        print(result.result()['query_status'])

print(f'Time taken: {time.time() - start}')

结果：

hash_not_found
hash_not_found
hash_not_found
hash_not_found
ok
Time taken: 1.113807201385498

如何在 python 中多线程 post 请求

How to multithread post request in python

python

multithreading

python-multithreading