Python Web Scraping Tutorial: scraping dynamic JavaScript/AJAX websites with BeautifulSoup

This Python Web Scraping Tutorial is about scraping dynamic websites, where the content is rendered by JavaScript.
For this Python Web Scraping Tutorial I used the Steam Store as an example. Because Steam website is an example of heavy JavaScript/AJAX driven website with dynamic content.
To scrape Steamstore website with Python I used only Python Requests and BeautifulSoup (bs4) libraries. With further exporting scraped data to a csv file.

This web scraping Python tutorial is the detailed explanation of how to scrape JavaScript driven pages and websites with Python and BeautifulSoup library for absolute beginners.

To install BeautifulSoup, Requests and Lxml:
pip install bs4 requests lxml

Follow me @:
Telegram: https://t.me/red_eyed_coder_club
Twitter: https://twitter.com/CoderEyed
Facebook: https://fb.me/redeyedcoderclub

======================================
📎️ The SOURCE CODE is available via Patreon:
https://www.patreon.com/posts/steam-store-with-35670113
======================================

Timecodes:

00:00 – Beginning.
01:09 – Preliminary research (what to scrape)
03:15 – Creating a function that performs GET requests to Steam Store
06:01 – Server response research: what url should be passed in to the get_html() function
09:24 – The scraping plan
09:43 – Getting all Steam Store games with Python Requests, and BeautifulSoup. Scraping pagination.
12:40 – The algorithm of scraping all pages using the pagination GET requests
16:35 – Scraping data of a certain page with games
25:30 – Scraping hovering data of all games on each page, including the data from the hovering window
38:40 – Writing Scraped data to a CSV file

✴️✴️✴️ Also can be useful ✴️✴️✴️
Python tutorial: Namespaces and Scopes – https://youtu.be/L8UtCpEgo8I
Python Regular Expressions tutorial – https://youtu.be/TNZppdVITRE
Python tutorial: handling exceptions – https://youtu.be/vZo3EI1X6Q8
How to read and write CSV – https://youtu.be/sYZcZYoa2vM

✴️✴️✴️ Web Scraping course ✴️✴️✴️
is available via Patreon here:
https://www.patreon.com/red_eyed_coder_club

or its landing:
https://red-eyed-coder-club.github.io/courses/practical-python-web-scraping

✴️✴️✴️ PLAYLISTS ✴️✴️✴️

🔹Django 3 Tutorial: Blog Engine
https://www.youtube.com/watch?v=WeaL8o19D_E&list=PLe4mIUXfbIqbbqgDSY6z9orgzIwvYZjtw

🔹Kivy Tutorial: Coppa Project
https://www.youtube.com/watch?v=KftWys2cUfs&list=PLe4mIUXfbIqbeT-2SxDy_IbQYU4yn2gA6

🔹Telegram Bot with Python (CoinMarketCap)
https://www.youtube.com/watch?v=cX8m3sp_w84&list=PLe4mIUXfbIqbpQkE2Q_iQic6a8v-2k_m4

🔹Python Web Scraping
https://www.youtube.com/watch?v=m4hEAhHHykI&list=PLe4mIUXfbIqZkxlbr5cUVhn3LQOWQ67Bd

➥➥➥ SUBSCRIBE FOR MORE VIDEOS ➥➥➥
Red Eyed Coder Club is the best place to learn Python programming and Django:
Subscribe ⇢ https://www.youtube.com/channel/UCh_LSaTv2GeZ3woJNTGihew?sub_confirmation=1

Python Web Scraping Tutorial: scraping dynamic JavaScript/AJAX websites with BeautifulSoup
https://youtu.be/3fcKKZMFbyA

#python #pythonwebscraping #beautifulsoup #bs4 #redeyedcoderclub #webscrapingpython #beautifulsouptutorial