使用文本文件创建列表，然后使用列表从网页中抓取以创建字典

Question

我有一个包含主机名列表的文本文件：

router-1
firewall-1
firewall-2
distro-1

我想做的是从该文件创建一个列表，然后从该列表网络抓取特定文本字符串以确定设备的 device_platform。然后用它创建一个看起来像这样的字典：

[
    {'hostname': 'router-1', 'device_platform': 'cisco_ios'},
    {'hostname': 'firewall-1', 'device_platform': 'juniper'},
    {'hostname': 'firewall-2', 'device_platform': 'cisco_asa'},
    {'hostname': 'distro-1', 'device_platform': 'broadcom_icos'}
]

到目前为止，这就是我所拥有的，网络抓取部分有效。我只需要知道如何添加一个部分来创建列表，然后从该列表中抓取信息并return它来创建一个像上面那样的字典。

import requests
import urllib3


with open('devices.txt', 'r') as f:
    host = f.read().splitlines()
    print(host)
    for device in host:
        url = f'http://nunya.com/device/{device}.nunya.com/config'
        urllib3.disable_warnings()
        request = requests.get(url).text
        if 'event manager applet' in request and 'pager' not in request:
            device_platform = 'cisco_ios'
        elif 'junos' in request:
            device_platform = 'juniper'
        elif 'pager' in request:
            device_platform = 'cisco_asa'
        else:
            device_platform = 'broadcom_icos'
        print(f'hostname: {device}, device_platform: {device_platform}')

Answer 1

我看到你最终想要的是这种格式，

[
    {'hostname': 'router-1', 'device_platform': 'cisco_ios'},
    {'hostname': 'firewall-1', 'device_platform': 'juniper'},
    {'hostname': 'firewall-2', 'device_platform': 'cisco_asa'},
    {'hostname': 'distro-1', 'device_platform': 'broadcom_icos'}
]

这是 dictionaries 的 list，您的代码无需太多更改即可获得。

只需在 for 循环开始之前启动一个空的 list，并随着循环的进行不断向其追加所需的 dictionaries。这是代码：

import requests
import urllib3


with open('devices.txt', 'r') as f:
    host = f.read().splitlines()
    print(host)
    data = []
    for device in host:
        url = f'http://nunya.com/device/{device}.nunya.com/config'
        urllib3.disable_warnings()
        request = requests.get(url).text
        if 'event manager applet' in request and 'pager' not in request:
            device_platform = 'cisco_ios'
        elif 'junos' in request:
            device_platform = 'juniper'
        elif 'pager' in request:
            device_platform = 'cisco_asa'
        else:
            device_platform = 'broadcom_icos'
        data.append({'hostname': device, 'device_platform': device_platform})
        print(f'hostname: {device}, device_platform: {device_platform}')
    print(data)

如您所见，我通过说 data=[] 启动了一个空的 list，在循环中您可以看到我将 dictionaries 和所需的数据附加到列表中在行 data.append({'hostname': device, 'device_platform': device_platform}).

希望这对你有用。

使用文本文件创建列表，然后使用列表从网页中抓取以创建字典

With Text File Create List then use List to Scrape from Webpage to create dictionary

python

dictionary

list

web-scraping