尝试从 Web 中提取数据时出现 Selenium 错误 table

Selenium error while trying to extract data from a web table

这是带有 Python 的 Selenium。这些第一行工作正常:

from selenium import webdriver
    browser = webdriver.Firefox()
    browser.get('http://www.palottery.state.pa.us/Games/Past-Winning-Numbers.aspx?id=8')
    elm = browser.find_element_by_xpath(".//*[@id='p_lt_zoneMain_pageplaceholder1_p_lt_zoneContent_pageplaceholder_p_lt_zoneContent_PaLotteryPastWinningNumbers_Button1']")
    elm.click()
    elm2 = browser.find_element_by_xpath(".//*[@id='page-content']/div[2]/div/a/img")
    elm2.click()
    browser.implicitly_wait(10)

这里我得到错误

    Dtable = browser.find_element_by_xpath('.//*[@id="p_lt_zoneLeft_PaLotteryPastWinningNumbers_Results"]/tbody')

    for i in Dtable.find_elements_by_xpath('.//tr'):
        print(i.get_attribute('innerHTML'))

elenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: {"method":"xpath","selector":".//*[@id=\"p_lt_zoneLeft_PaLotteryPastWinningNumbers_Results\"]/tbody"}

更新: 我仍然无法获取 table 的所有 250 行。由于某种原因我只得到 10 行...

def getWinNums():

    l = []

    from selenium import webdriver
    browser = webdriver.Firefox()
    browser.get('http://www.palottery.state.pa.us/Games/Past-Winning-Numbers.aspx?id=8')

    elm = browser.find_element_by_xpath(".//*[@id='p_lt_zoneMain_pageplaceholder1_p_lt_zoneContent_pageplaceholder_p_lt_zoneContent_PaLotteryPastWinningNumbers_Button1']")
    elm.click()
    elm2 = browser.find_element_by_xpath(".//*[@id='page-content']/div[2]/div/a/img")
    elm2.click()
    browser.implicitly_wait(10)

    Dtable = browser.find_element_by_xpath(".//*[@id='page-content']//table/tbody")


        # create list were elements are dates followed by 5 numbers for that date
    l = [i.text.strip() for i in Dtable.find_elements_by_xpath('.//td') if i.text != "Payout"]

    browser.close()

    # convert list into list of tuples (date, 5 numbers)
    l =  zip(*[iter(l)]*2)

    return l


def main():

    l = getWinNums()

    for el in l:
        print(el)


if __name__ == "__main__":
        main()

输出:

('09/08/2015', '2   32   35   36   39')

('09/07/2015', '14   17   19   24   43')

('09/06/2015', '10   13   15   36   38')

('09/05/2015', '4   5   24   29   34')

('09/04/2015', '1   12   18   34   36')

('09/03/2015', '4   9   15   28   40')

('09/02/2015', '14   16   17   18   34')

('09/01/2015', '7   26   33   36   41')

('08/31/2015', '17   20   22   32   41')

('08/30/2015', '11   14   23   24   38')

更新 #2

CSS 选择器的工作方式如下所示,但同样 Dtable.find_elements_by_xpath('.//td') 只产生 251 行中的 10 行。

Dtable = browser.find_element_by_css_selector("table>tbody")

更新 #3

现在我可以得到 table 的 50 行:

for i in range(1,6):

    link3 = browser.find_element_by_xpath(".//*[@id='p_lt_zoneMain_pageplaceholder1_p_lt_zoneContent_pageplaceholder_p_lt_zoneContent_PaLotteryPastWinningNumbers_Results_paginate']/span/a[{i}]".format(i=i))

    link3.click()

    Dtable = browser.find_element_by_css_selector("table>tbody>tr")

    l = [i.text.strip() for i in Dtable.find_elements_by_xpath('//td') if i.text != "Payout"]

    l_result += l

剩下的问题是如何通过单击分页按钮到达接下来的 50 行。我可以获得按钮的 xpath,它是:

.//*[@id='p_lt_zoneMain_pageplaceholder1_p_lt_zoneContent_pageplaceholder_p_lt_zoneContent_PaLotteryPastWinningNumbers_Results_next'] 

但是点击它并重复上面的 for 循环不会从 table.

中产生任何新行

我猜您想更改选择器以获取 table,如下所示:

 Dtable = browser.find_element_by_xpath('.//*[@id="p_lt_zoneLeft_PaLotteryPastWinningNumbers_Results"]/tbody')

至:

 Dtable = browser.find_element_by_css_selector("table[id^='p_lt_zoneLeft']")