Issue
import requests
from bs4 import BeautifulSoup
from csv import writer
headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36'}
response = requests.get("https://www.shurgard.nl/en/self-storage-in-the-netherlands/alkmaar-hoorn/alkmaar-oudorp", headers=headers)
soup = BeautifulSoup(response.text,'html.parser')
products = soup.findAll(class_='storage-unit')
with open(r'C:\Users\Hosiwan\Desktop\python\aaaa.csv','w') as csv_file:
csv_writer = writer(csv_file)
headers = ['Title']
csv_writer.writerow(headers)
for product in products:
title = product.find(class_='price-current').get_text()
The output file only has "Title" written only without any other data..
Solution
You sure you want to do all that work with BeautifulSoup and writer, when there's an api and you can use pandas?
import pandas as pd
import requests
headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36'}
api_url = 'https://www.shurgard.nl/en/api/stores/62/units'
jsonData = requests.get(api_url, headers=headers).json()
df = pd.json_normalize(jsonData['stores'][0]['units'])
df.to_csv(r'C:\Users\Hosiwan\Desktop\python\aaaa.csv', index=False)
Output:
print(df)
id summary \
0 771 In a unit this size you can store a few boxes ...
1 770 In a unit this size you can store a few boxes ...
2 554 In a unit this size you can store a few boxes ...
3 891 In a unit this size you can store the content ...
4 537 In a unit this size you can store the content ...
5 885 In a unit this size you can store the content ...
6 513 In a unit this size you can store the content ...
7 889 In a unit this size you can store the contents...
8 538 In a unit this size you can store the contents...
9 893 In a unit this size you can store the contents...
10 892 In a unit this size you can store the contents...
11 514 In a unit this size you can store the contents...
12 570 In a unit this size you can store the contents...
13 894 In a unit this size you can store the contents...
14 522 In a unit this size you can store the contents...
15 900 In a unit this size you can store the contents...
16 890 In a unit this size you can store the contents...
17 1164 In a unit this size you can store the contents...
18 761 In a unit this size you can store the contents...
19 954 In a unit this size you can store the contents...
description sizeDefault \
0 <p>For example, you could store the following ... 1
1 <p>For example, you could store the following ... 1
2 <p>For example, you could store the following ... 1,5
3 <p>For example, you could store the following ... 2,5
4 <p>For example, you could store the following ... 3
5 <p>For example, you could store the following ... 4
6 <p>For example, you could store the following ... 4,5
7 <p>For example, you could store the following ... 5
8 <p>For example, you could store the following ... 6
9 <p>For example, you could store the following ... 7
10 <p>For example, you could store the following ... 8
11 <p>For example, you could store the following ... 9
12 <p>For example, you could store the following ... 10
13 <p>For example, you could store the following ... 11
14 <p>For example, you could store the following ... 12
15 <p>For example, you could store the following ... 17
16 <p>For example, you could store the following ... 14
17 <p>For example, you could store the following ... 13
18 <p>For example, you could store the following ... 40
19 <p>For example, you could store the following ... 50
sizeAlternative sizeDimensionDefault sizeDimensionAlternative sizeGroup \
0 1 1 m x 1 m 1 m x 1 m 1
1 3 1 m x 1 m 1 m x 1 m 1
2 4,5 1 m x 1,5 m 1 m x 1,5 m 1
3 7,5 1 m x 2,5 m 1 m x 2,5 m 1
4 9 1,5 m x 2 m 1,5 m x 2 m 1
5 12 2 m x 2 m 2 m x 2 m 2
6 13,5 1,5 m x 3 m 1,5 m x 3 m 2
7 15 2 m x 2,5 m 2 m x 2,5 m 2
8 18 2 m x 3 m 2 m x 3 m 2
9 21 2 m x 3,5 m 2 m x 3,5 m 3
10 24 2 m x 4 m 2 m x 4 m 3
11 27 3 m x 3 m 3 m x 3 m 3
12 30 2,5 m x 4 m 2,5 m x 4 m 4
13 33 2,5 m x 4,5 m 2,5 m x 4,5 m 4
14 36 3 m x 4 m 3 m x 4 m 4
15 51 3 m x 5,75 m 3 m x 5,75 m 5
16 42 3,5 m x 4 m 3,5 m x 4 m 5
17 39 3,5 m x 3,75 m 3,5 m x 3,75 m 5
18 120 4 m x 10 m 4 m x 10 m 5
19 150 5 m x 10 m 5 m x 10 m 5
sizeGtm promoGtm discountGtm isLocker \
0 1.00m2 first month 1 online special True
1 1.00m2 first month 1 online special False
2 1.50m2 first month 1 online special False
3 2.50m2 first month 1 online special False
4 3.00m2 first month 1 online special False
5 4.00m2 first month 1 online special False
6 4.50m2 first month 1 online special False
7 5.00m2 first month 1 online special False
8 6.00m2 first month 1 online special False
9 7.00m2 first month 1 online special False
10 8.00m2 first month 1 online special False
11 9.00m2 first month 1 online special False
12 10.00m2 first month 1 online special False
13 11.00m2 first month 1 online special False
14 12.00m2 first month 1 online special False
15 17.00m2 first month 1 online special False
16 14.00m2 first month 1 online special False
17 13.00m2 first month 1 online special False
18 40.00m2 first month 1 online special False
19 50.00m2 first month 1 online special False
url \
0 /en/reservationform?storeid=62&productid=771
1 /en/reservationform?storeid=62&productid=770
2 /en/reservationform?storeid=62&productid=554
3 /en/reservationform?storeid=62&productid=891
4 /en/reservationform?storeid=62&productid=537
5 /en/reservationform?storeid=62&productid=885
6 /en/reservationform?storeid=62&productid=513
7 /en/reservationform?storeid=62&productid=889
8 /en/reservationform?storeid=62&productid=538
9 /en/reservationform?storeid=62&productid=893
10 /en/reservationform?storeid=62&productid=892
11 /en/reservationform?storeid=62&productid=514
12 /en/reservationform?storeid=62&productid=570
13 /en/reservationform?storeid=62&productid=894
14 /en/reservationform?storeid=62&productid=522
15 /en/reservationform?storeid=62&productid=900
16 /en/reservationform?storeid=62&productid=890
17 /en/reservationform?storeid=62&productid=1164
18 /en/reservationform?storeid=62&productid=761
19 /en/reservationform?storeid=62&productid=954
imageUrlSmall \
0 https://shg-prd.azureedge.net/-/media/shurgard...
1 https://shg-prd.azureedge.net/-/media/shurgard...
2 https://shg-prd.azureedge.net/-/media/shurgard...
3 https://shg-prd.azureedge.net/-/media/shurgard...
4 https://shg-prd.azureedge.net/-/media/shurgard...
5 https://shg-prd.azureedge.net/-/media/shurgard...
6 https://shg-prd.azureedge.net/-/media/shurgard...
7 https://shg-prd.azureedge.net/-/media/shurgard...
8 https://shg-prd.azureedge.net/-/media/shurgard...
9 https://shg-prd.azureedge.net/-/media/shurgard...
10 https://shg-prd.azureedge.net/-/media/shurgard...
11 https://shg-prd.azureedge.net/-/media/shurgard...
12 https://shg-prd.azureedge.net/-/media/shurgard...
13 https://shg-prd.azureedge.net/-/media/shurgard...
14 https://shg-prd.azureedge.net/-/media/shurgard...
15 https://shg-prd.azureedge.net/-/media/shurgard...
16 https://shg-prd.azureedge.net/-/media/shurgard...
17 https://shg-prd.azureedge.net/-/media/shurgard...
18 https://shg-prd.azureedge.net/-/media/shurgard...
19 https://shg-prd.azureedge.net/-/media/shurgard...
imageUrlMedium \
0 https://shg-prd.azureedge.net/-/media/shurgard...
1 https://shg-prd.azureedge.net/-/media/shurgard...
2 https://shg-prd.azureedge.net/-/media/shurgard...
3 https://shg-prd.azureedge.net/-/media/shurgard...
4 https://shg-prd.azureedge.net/-/media/shurgard...
5 https://shg-prd.azureedge.net/-/media/shurgard...
6 https://shg-prd.azureedge.net/-/media/shurgard...
7 https://shg-prd.azureedge.net/-/media/shurgard...
8 https://shg-prd.azureedge.net/-/media/shurgard...
9 https://shg-prd.azureedge.net/-/media/shurgard...
10 https://shg-prd.azureedge.net/-/media/shurgard...
11 https://shg-prd.azureedge.net/-/media/shurgard...
12 https://shg-prd.azureedge.net/-/media/shurgard...
13 https://shg-prd.azureedge.net/-/media/shurgard...
14 https://shg-prd.azureedge.net/-/media/shurgard...
15 https://shg-prd.azureedge.net/-/media/shurgard...
16 https://shg-prd.azureedge.net/-/media/shurgard...
17 https://shg-prd.azureedge.net/-/media/shurgard...
18 https://shg-prd.azureedge.net/-/media/shurgard...
19 https://shg-prd.azureedge.net/-/media/shurgard...
imageUrl imageAlt \
0 https://shg-prd.azureedge.net/-/media/shurgard... 1 m² locker unit
1 https://shg-prd.azureedge.net/-/media/shurgard... 1 m² storage unit
2 https://shg-prd.azureedge.net/-/media/shurgard... 1,5 m² storage unit
3 https://shg-prd.azureedge.net/-/media/shurgard... 2,5 m² storage unit
4 https://shg-prd.azureedge.net/-/media/shurgard... 3 m² storage unit
5 https://shg-prd.azureedge.net/-/media/shurgard... 4 m² storage unit
6 https://shg-prd.azureedge.net/-/media/shurgard... 4,5 m² storage unit
7 https://shg-prd.azureedge.net/-/media/shurgard... 5 m² storage unit
8 https://shg-prd.azureedge.net/-/media/shurgard... 6 m² storage unit
9 https://shg-prd.azureedge.net/-/media/shurgard... 7 m² storage unit
10 https://shg-prd.azureedge.net/-/media/shurgard... 8 m² storage unit
11 https://shg-prd.azureedge.net/-/media/shurgard... 9 m² storage unit
12 https://shg-prd.azureedge.net/-/media/shurgard... 10 m² storage unit
13 https://shg-prd.azureedge.net/-/media/shurgard... 11 m² storage unit
14 https://shg-prd.azureedge.net/-/media/shurgard... 12 m² storage unit
15 https://shg-prd.azureedge.net/-/media/shurgard... 17 m² storage unit
16 https://shg-prd.azureedge.net/-/media/shurgard... 14 m² storage unit
17 https://shg-prd.azureedge.net/-/media/shurgard... 12 m² storage unit
18 https://shg-prd.azureedge.net/-/media/shurgard... 40 m² storage unit
19 https://shg-prd.azureedge.net/-/media/shurgard... 50 m² storage unit
thumbnailAlt promotion \
0 1 m² locker unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
1 1 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
2 1,5 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
3 2,5 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
4 3 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
5 4 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
6 4,5 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
7 5 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
8 6 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
9 7 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
10 8 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
11 9 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
12 10 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
13 11 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
14 12 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
15 17 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
16 14 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
17 12 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
18 40 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
19 50 m² storage unit {'hasFiftyPercentPromo': False, 'hasFirstMonth...
pricing
0 {'price': '€ 8,<div class='sup'>00</div>'...
1 {'price': '€ 20,<div class='sup'>00</div>...
2 {'price': '€ 25,<div class='sup'>00</div>...
3 {'price': '€ 40,<div class='sup'>00</div>...
4 {'price': '€ 44,<div class='sup'>00</div>...
5 {'price': '€ 56,<div class='sup'>00</div>...
6 {'price': '€ 60,<div class='sup'>00</div>...
7 {'price': '€ 64,<div class='sup'>00</div>...
8 {'price': '€ 83,<div class='sup'>00</div>...
9 {'price': '€ 91,<div class='sup'>00</div>...
10 {'price': '€ 102,<div class='sup'>00</div...
11 {'price': '€ 110,<div class='sup'>00</div...
12 {'price': '€ 138,<div class='sup'>00</div...
13 {'price': '€ 147,<div class='sup'>00</div...
14 {'price': '€ 156,<div class='sup'>00</div...
15 {'price': '€ 206,<div class='sup'>00</div...
16 {'price': '€ 208,<div class='sup'>00</div...
17 {'price': '€ 243,<div class='sup'>00</div...
18 {'price': '€ 492,<div class='sup'>00</div...
19 {'price': '€ 603,<div class='sup'>00</div...
Answered By - chitown88
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.