python - selenium scraping data using children of elements

Question

Hi i'm trying to scrape some data from a live stocks website. I want to display the companies name and stock price, %change ect. The details of 25 companies are shown per page, and these details follow the same format, one row of data comes under the tag, have the same css selector and then the rest of the elements e.g. Name of the company share price, %change ect are children of that tag. Since all data follows the same format, my question is how do i filter through each tag displaying data for that row?

I thought about:

using a for loop to find the elements by class name (since they all have the same class) and then cycle through these but I don't how to select a child of that class:

Here' the html code:

Here's the link to the site https://uk.finance.yahoo.com/most-active

Answer 1

Here ya go man! I had similar issues when trying to scrape data from a similar stock site

from bs4 import BeautifulSoup
from urllib.request import urlopen as uReq

url = "https://uk.finance.yahoo.com/most-active"
uClient = uReq(url)
html_page = uClient.read()
uClient.close()

soup = BeautifulSoup(html_page, 'html.parser')
full_table = soup.find_all("tr", class_="simpTblRow")

for tr in full_table:
    td = tr.find_all('td')
    row = [i.text for i in td]
    print(row)

You can make your own data structure to hold each of the printed values, it runs with this output (without labels, the output looks confusing but you can look at the site for reference)

['ICON.L', 'Iconic Labs Plc', '0.0132', '0.0000', '0.00%', '1.291B', '412.755M', 
'1.299M', 'N/A', '']
['SYME.L', 'Supply@ME Capital plc', '0.8020', '+0.2320', '+40.70%', '1.145B', 
'173.333M', '262.694M', 'N/A', '']
['UJO.L', 'Union Jack Oil plc', '0.2925', '-0.0075', '-2.50%', '210.666M', '154.613M', 
'45.165M', 'N/A', '']
['TOM.L', 'TomCo Energy Plc', '0.8750', '+0.0750', '+9.37%', '197.389M', '49.702M', 
'5.894M', 'N/A', '']
['PREM.L', 'Premier African Minerals Limited', '0.0680', '-0.0045', '-6.21%', 
'193.547M', '136.558M', '8.542M', 'N/A', '']
['COPL.L', 'Canadian Overseas Petroleum Limited', '0.4000', '-0.0200', '-4.76%', 
'139.094M', '211.933M', '9.549M', 'N/A', '']
['MSMN.L', 'Mosman Oil And Gas Limited', '0.1800', '+0.0200', '+12.50%', '136.04M', 
'51.481M', '2.854M', 'N/A', '']
['VAST.L', 'Vast Resources plc', '0.2100', '-0.0090', '-4.11%', '133.599M', 
'243.076M', '25.673M', 'N/A', '']
['UKOG.L', 'UK Oil & Gas Investments PLC', '0.2250', '+0.0150', '+7.14%', '114.909M', 
'275.501M', '24.974M', 'N/A', '']
['7DIG.L', '7digital Group plc', '2.8000', '-0.3000', '-9.68%', '111.529M', '23.568M', 
'68.752M', 'N/A', '']
['ALBA.L', 'Alba Mineral Resources PLC', '0.0925', '-0.0125', '-11.90%', '99.299M', 
'39.579M', '4.472M', 'N/A', '']
['LLOY.L', 'Lloyds Banking Group plc', '28.18', '-0.44', '-1.54%', '98.424M', 
'263.235M', '19.948B', '70.46', '']
['KOD.L', 'Kodal Minerals Plc', '0.0475', '0.0000', '0.00%', '111.583M', '75.563M', 
'5.278M', 'N/A', '']
['UFO.L', 'Alien Metals Limited', '0.3600', '+0.0500', '+16.13%', '84.677M', 
'52.319M', '8.516M', 'N/A', '']
['EDL.L', 'Edenville Energy Plc', '0.0500', '-0.0025', '-4.76%', '80.171M', '47.879M', 
'4.073M', 'N/A', '']
['EQT.L', 'EQTEC plc', '0.5300', '+0.0650', '+13.98%', '73.296M', '144.02M', 
'36.929M', 'N/A', '']
['BOIL.L', 'Baron Oil Plc', '0.1000', '-0.0025', '-2.44%', '73.45M', '71.853M', 
'4.426M', 'N/A', '']
['BREE.L', 'Breedon Group plc', '83.20', '+2.20', '+2.72%', '63.05M', '4.616M', 
'1.403B', '43.79', '']
['AMGO.L', 'Amigo Holdings PLC', '9.30', '+0.55', '+6.29%', '62.892M', '20.841M', 
'44.206M', 'N/A', '']
['INSP.L', 'Inspirit Energy Holdings Plc', '0.0800', '+0.0025', '+3.23%', '61.622M', 
'73.243M', '2.323M', 'N/A', '']
['TRAF.L', 'Trafalgar Property Group plc', '0.2000', '0.0000', '0.00%', '55.959M', 
'40.707M', '2.85M', 'N/A', '']
['RBD.L', 'Reabold Resources plc', '0.7300', '-0.0350', '-4.58%', '51.697M', 
'23.499M', '51.808M', 'N/A', '']
['CPI.L', 'Capita plc', '28.70', '-7.18', '-20.01%', '48.494M', '12.711M', '478.994M', 
'N/A', '']
['GGP.L', 'Greatland Gold plc', '12.20', '-0.80', '-6.15%', '44.829M', '33.176M', 
'461.41M', 'N/A', '']
['SRES.L', 'Sunrise Resources plc', '0.3000', '-0.0100', '-3.23%', '39.45M', 
'15.895M', '9.963M', 'N/A', '']