Issue
I am currently doing a personal project, and am quite new to web scraping and the Beautiful Soups library, so any help would be much appreciated! I am currently trying to extract the R1, R2 etc text from the following HTML snippet
The code I've written for this is below:
import requests
from bs4 import BeautifulSoup
URL1 = "https://www.sportsbet.com.au/racing-schedule/horse/today"
racing = requests.get(URL1)
soup2 = BeautifulSoup(racing.content, "lxml")
race_index = soup2.findAll('div', {"class":"tableHeaderCell_fh883o"})
for race in race_index:
print(race)
However, there is clearly some text within the div tags, but the output I am getting is:
<div class="tableHeaderCell_fh883o"></div>
<div class="tableHeaderCell_fh883o"></div>
<div class="tableHeaderCell_fh883o"></div>
I am wondering why the text within the div tags are missing, and how I can extract the text.
Solution
yes you can't get it because this data is dynamically loaded not static so opening it with BeautifulSoup won't load this data.
Instead, if you open the page in your browser and open DevTools, switch to the network tab then refresh the page you will find this request being made.
So long story short, just head to that link and you will find your desired data loaded there as JSON data.
Please don't forget to mark this solution as an answer if it resolves your problem.
Answered By - D4Vinci
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.