python - ResultSet object has no attribute 'get'

Question

Hi I'm currently trying to scrape this https://www.sec.gov/ix?doc=/Archives/edgar/data/1090727/000109072720000003/form8-kq42019earningsr.htm SEC link with beautifulsoup to get the link containing "UPS"

pressting = soup3.find_all("a", string="UPS")
linkkm = pressting.get('href')
print(linkkm)

But when I do this I get this error:

Traceback (most recent call last):
  File "C:UsersAdminAppDataLocalProgramsPythonPython36SEC.py", line 55, in <module>
    print('Price: ' + str(edgar()))
  File "C:UsersAdminAppDataLocalProgramsPythonPython36SEC.py", line 46, in edgar
    linkkm = pressting.get('href')
  File "C:UsersAdminAppDataLocalProgramsPythonPython36libsite-packagess4element.py", line 2081, in __getattr__
    "ResultSet object has no attribute '%s'. You're probably treating a list of elements like a single element. Did you call find_all() when you meant to call find()?" % key
AttributeError: ResultSet object has no attribute 'get'. You're probably treating a list of elements like a single element. Did you call find_all() when you meant to call find()?

My expected result is to exract the href and then print that href. Any help would be appreciated.

Answer 1

Basically the page is dynamically rendered via JavaScript once it's loads. so you will not be able to parse the objects until you render it firstly. Therefore requests module will not render the JavaScript.

You can use selenium approach to achieve that. otherwise you can use HTMLSession from html_request module to render it on the fly.

from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from bs4 import BeautifulSoup
import re
from time import sleep

options = Options()
options.add_argument('--headless')
driver = webdriver.Firefox(options=options)

driver.get("https://www.sec.gov/ix?doc=/Archives/edgar/data/1090727/000109072720000003/form8-kq42019earningsr.htm")

sleep(1)
soup = BeautifulSoup(driver.page_source, 'html.parser')

for item in soup.findAll("a", style=re.compile("^text")):
    print(item.get("href"))

driver.quit()

Output:

https://www.sec.gov/Archives/edgar/data/1090727/000109072720000003/exhibit991-q42019earni.htm
https://www.sec.gov/Archives/edgar/data/1090727/000109072720000003/exhibit992-q42019finan.htm

However if you want just the first url;

url = soup.find("a", style=re.compile("^text")).get("href")
print(url)

Output:

https://www.sec.gov/Archives/edgar/data/1090727/000109072720000003/exhibit991-q42019earni.htm