使用 Search.submit() 时出现网络抓取错误

Question

我正在尝试对来自以下位置的天气数据进行网络抓取： https://www.wunderground.com/history/daily/us/dc/washington/KDCA

这里是HTML:

<div _ngcontent-app-root-c136="" id="wuForm" class="ui-front wu-form">
   <div _ngcontent-app-root-c136="" id="wuSearch-contain" class="wu-search-contain ng-star-inserted"><label _ngcontent-app-root-c136="" for="wuSearch" class="visuallyHidden">Search</label><input _ngcontent-app-root-c136="" type="search" name="query" value="" id="wuSearch" placeholder="Search Locations" aria-label="Search" autocomplete="off" class="wu-search ng-untouched ng-pristine ng-valid"><span _ngcontent-app-root-c136="" class="close-search"><i _ngcontent-app-root-c136="" class="material-icons">close</i></span><span _ngcontent-app-root-c136="" class="geolocate-wrap"><i _ngcontent-app-root-c136="" class="material-icons">gps_fixed</i></span></div>
   <!----><!---->
   <search-autocomplete _ngcontent-app-root-c136="" _nghost-app-root-c135="">
      <ul _ngcontent-app-root-c135="" tabindex="0" class="ui-autocomplete ui-front ui-menu ui-widget ui-widget-content ui-corner-all hide">
         <li _ngcontent-app-root-c135="" class="ui-autocomplete-geolocate ng-star-inserted">
            <div _ngcontent-app-root-c135="" class="mimic-a menu-geolocate"><i _ngcontent-app-root-c135="" class="material-icons">gps_fixed</i>Find Nearest Station </div>
         </li>
         <!----><!----><!----><!----><!----><!---->
         <li _ngcontent-app-root-c135="" class="ui-autocomplete-last ui-menu-item manage-favorites"><a _ngcontent-app-root-c135="" tabindex="-1" href="/member/favorites" class="ui-corner-all">Manage Favorite Cities</a></li>
      </ul>
   </search-autocomplete>
</div>

我从

开始

driver = webdriver.Chrome(r'C:\Users\bno00\Downloads\chromedriver_win32\chromedriver.exe')
driver.get('https://www.wunderground.com/history/daily/us/dc/washington/KDCA')

而且工作正常然后当我尝试使用 .submit():

将邮政编码发送到搜索字段时

zipcode='22202'
xpath='//*[@id="wuSearch"]'
search=driver.find_element_by_xpath(xpath)
search.clear() 
search.send_keys(zipcode)
search.submit() #error occurs here

错误是：

NoSuchElementException: Message: no such element: Unable to locate element:{"method":"xpath","selector":"./ancestor-or-self::form"} (Session info: chrome=89.0.4389.114)

Answer 1

<input> 标签不是表单的一部分。你不能提交它。您要么需要发送换行符，要么单击触发搜索的 gps_fixed 图标。

Answer 2

尝试使用以下方法：

input_field = driver.find_element_by_css_selector("input[id=wuSearch]")
input_field.click()
input_field.send_keys("some zip code")

或者只是：

input_field = driver.find_element_by_id("wuSearch")
input_field.send_keys("some zip code")

尽量避免click()。我不确定是否真的需要。输入邮政编码后，您应该等待搜索结果出现。

Answer 3

wait = WebDriverWait(driver, 10)
driver.get('https://www.wunderground.com/history/daily/us/dc/washington/KDCA')
zipcode='22202'
xpath='//*[@id="wuSearch"]'    
search=wait.until(EC.element_to_be_clickable((By.XPATH,xpath)))
search.clear() 
search.send_keys(zipcode)
time.sleep(1)
search.send_keys(Keys.ENTER)

稍微延迟一下。使用 webdriver 等待确保将邮政编码发送到输入，然后发送 Keys.Enter.

导入

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from time import sleep

使用 Search.submit() 时出现网络抓取错误

Web Scraping error in while using Search.submit()

python

selenium

data-analysis

web-scraping

data-science