[FIXED] Interaction with simulator using python and selenium

Issue

I am trying to scrape some information from an online simulator, using Python and Selenium, but the interaction is not working. On this website (https://www.wizink.pt/public/creditos#/), I need to click on the plus button (+) to get different values on the simulator but it seems that nothing is working.

import time
from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By


options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')

driver = webdriver.Chrome(options=options)

driver.get("https://www.wizink.pt/public/creditos#/")

wait = WebDriverWait(driver, 5)
wait.until(EC.element_to_be_clickable((By.ID, "consent_prompt_submit"))).click()
time.sleep(5)

# click on "+" sign
wait.until(EC.element_to_be_clickable((By.XPATH, '//*[@id="app"]/div/div/div[2]/div/div/div/div[2]/div/div[1]/div/div/div[2]/div/div[2]/div[1]/div[2]'))).click()

I keep getting the error TimeoutException. I also can't get to read the text from TAN (10.00%) or any other text inside the simulator.

I have also tried with beautiful soup, but looks like it can't identify the objects that I am looking for.

Solution

There are shadow root elements present. Regular selenium doesn't have special selectors for reaching those directly, but https://github.com/seleniumbase/SeleniumBase has them. (pip install seleniumbase, and run with python)

from seleniumbase import SB

with SB(browser="chrome") as sb:
    sb.open("https://www.wizink.pt/public/creditos#/")
    sb.click("#consent_prompt_submit")
    sb.wait_for_element("loans-calculator")
    for i in range(6):
        sb.click("loans-calculator::shadow div.slider-more")
    info_css = "loans-calculator::shadow div.simulator_result_info_wrapper"
    print(sb.get_text(info_css))
    sb.sleep(3)

Output:

Prestação mensal
276,87 €
TAN
10,00%
TAEG
13,0%

Although SeleniumBase has its own APIs, you can access the raw driver with sb.driver.

Answered By - Michael Mintz