[FIXED] I cant get the resulation of BeautifulSoup commant

Issue

I have started to using BeautifulSoup yet.I made a script like this:

import bs4
import requests
from bs4 import BeautifulSoup


def parsePrice():    
    r=requests.get('https://steamcommunity.com/market/listings/440/Mann%20Co.%20Supply%20Crate%20Key')
    soup=bs4.BeautifulSoup(r.text,"xml")
    price=soup.find_all('span',{'class':'market_commodity_orders_header_promote'})[0].find('span').text
    return price

while True:
    print('the current price: '+str(parsePrice())) 
   

When i hit the run button and it shows error as,

 runfile('C:/Users/tuck/.spyder-py3/temp.py', wdir='C:/Users/tuck/.spyder-py3')
    Traceback (most recent call last):

   File "C:\Users\tuck\.spyder-py3\temp.py", line 15, in <module>
    print('the current price: '+str(parsePrice()))

  File "C:\Users\tuck\.spyder-py3\temp.py", line 11, in parsePrice
    price=soup.find_all('span',{'class':'market_commodity_orders_header_promote'})[0].find('span').text

IndexError: list index out of range

Image for desired output:

Solution

1. Yes contents are loaded dynamically so what can you do first go to chrome developer and find Network tab and go to xhr tab

2. Then refresh your site now you can see in name tab links are present now click on that links and find your data

3. So it contains json data now you can copy the link address of link which is contain right information also i have added image for better understanding

import requests
from bs4 import BeautifulSoup
r=requests.get('https://steamcommunity.com/market/itemordershistogram?country=IN&language=english&currency=1&item_nameid=1&two_factor=0') 
main_data=r.json()
html=main_data['sell_order_summary']

soup=BeautifulSoup(html,"html.parser")
soup.text

Output:

'10079 for sale starting at $2.47'

Image:

Answered By - Bhavya Parikh