Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share
menu search
person
Welcome To Ask or Share your Answers For Others

Categories

I am trying to pull data from following page:

https://www.lifeinscouncil.org/industry%20information/ListOfFundNAVs

I have to select the name of the company and select a date from the calendar and click get data button. I am trying to achieve this using Selenium Web Driver using Chrome in Python, I am stuck how do i pass the date parameter to the page. it seems the page is postback after selection of date from the calendar. Date needs to be selected from the calendar else the data is not returned by the webpage. I have tried using requests Post method as well but am not able to get the NAV data.

I need to iterate this for a period of 5 years on daily (Trading Days) basis.

PS: I am bad at understanding DOM elements and have basic knowledge of Python and coding. by profession I am a data analyst.

Thanks in Advance. Kiran Jain

edit: adding current code below:

from selenium import webdriver

url='https://www.lifeinscouncil.org/industry%20information/ListOfFundNAVs'

opt = webdriver.ChromeOptions()
prefs = {"profile.managed_default_content_settings.images": 2}
opt.add_argument("--start-maximized")
# opt.add_argument("--headless")
opt.add_argument("--disable-notifications")
opt.add_experimental_option("prefs", prefs)
driver = webdriver.Chrome(options=opt)
driver.get('https://www.lifeinscouncil.org/industry%20information/ListOfFundNAVs');
insurer = driver.find_element_by_id("MainContent_drpselectinscompany")
nav_date=driver.find_element_by_id('MainContent_txtdateselect')
get_data_btn=driver.find_element_by_id('MainContent_btngetdetails')

options=insurer.find_elements_by_tag_name("option")
data=[]
row={'FirmName','SFIN','fundName','NAVDate','NAV'}
for option in options:
    print('here')
    print(option.get_attribute("value") + ' ' + option.text)
   
    if(option.text!='--Select Insurer--'):
        option.click()
        driver.find_element_by_id("MainContent_imgbtncalender").click()#Calender Icon
        driver.find_element_by_link_text("June").click()#Date
        driver.find_element_by_link_text("25").click()#Date
        get_data_btn=driver.find_element_by_id('MainContent_btngetdetails') #this is put here again because on clicking the date, the page is reloaded
        get_data_btn.click()
        print('clicked')
driver.quit()
See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
149 views
Welcome To Ask or Share your Answers For Others

1 Answer

The date is in "a" tag. You can try to do select the date using "link-text".

    driver.find_element_by_id("MainContent_imgbtncalender").click()#Calender Icon
    driver.find_element_by_link_text("27").click()#Date

As per your comment I tried to traverse through dates but it only worked for that particular month. I tried to use "send_keys()" to that text box and its not working. Below is the code to traverse it for a month.

driver.get("https://www.lifeinscouncil.org/industry%20information/ListOfFundNAVs")
driver.find_element_by_id("MainContent_drpselectinscompany").click()
driver.find_element_by_xpath("//option[starts-with(text(),'Aditya')]").click()
driver.find_element_by_id("MainContent_imgbtncalender").click()
driver.find_element_by_link_text("1").click()
driver.find_element_by_id("MainContent_btngetdetails").click()
dateval = 2

while True:
    if dateval == 32:
        break
    try:
        driver.find_element_by_id("MainContent_imgbtncalender").click()
        driver.find_element_by_link_text(str(dateval)).click()
        driver.find_element_by_id("MainContent_btngetdetails").click()
        dateval+=1
        time.sleep(2)
    except:
        driver.switch_to.default_content()
        dateval+=1
        time.sleep(2)

time.sleep(5)
driver.quit()

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share
...