Skip to content Skip to sidebar Skip to footer
Home / Beautifulsoup / Javascript / Python / Request / Selenium

Webscraping Javascript Page In Python

Post a Comment
Hello World, New in Python, I am trying to webscrape a javascript page : https://search.gleif.org/#/search/ Please find below the result from my code (using request)

Solution 1:

Following code is written using PySelenium.

import time
from selenium import webdriver

country = []
legal_name = []
lei = []

driver = webdriver.Chrome()
driver.implicitly_wait(5)

for i inrange(1,30395):
    driver.get('https://search.gleif.org/#/search/fulltextFilterId=LEIREC_FULLTEXT&currentPage='+str(i)+'&perPage=50&expertMode=false#results-section')

    time.sleep(5)

    country += [i.get_attribute('innerHTML') for i in driver.find_elements_by_xpath('//*[@class="table-cell country"]/a')]
    legal_name += [i.get_attribute('innerHTML') for i in driver.find_elements_by_xpath('//*[@class="table-cell legal-name"]/a')]
    lei += [i.get_attribute('innerHTML') for i in driver.find_elements_by_xpath('//*[@class="table-cell lei"]/a')]

Logging in (Change this with the respective elements.)

driver.find_element_by_id("UserName").send_keys("xxxx")
driver.find_element_by_name("Password").send_keys("yyyy")
driver.find_element_by_class("loginButton").click()

Get page content

print(driver.page_source)

You may like these posts

Post a Comment for "Webscraping Javascript Page In Python"