Project
Web Scraper
Hard
1440 completions
~ 15 hours
4.3Master web scraping and data extraction by building a robust Python application that interacts with APIs and parses dynamic web content!
Provided by
JetBrains Academy
About
You will create a function that takes a website address and a number of webpages as input arguments and then goes all over the website saving every news article on the page to a separate .txt file on your computer.
Training project
This project allows you to practice and strengthen your coding skills, helping you get ready for more advanced tasks ahead.
What you'll learn
Once you choose a project, we'll provide you with a study plan that includes all the necessary topics from your course to get it built. Here’s what awaits you:
Send an HTTP request, process the results, and learn about response status codes.
Get familiar with the BeautifulSoup library. Learn how to parse simple data from a webpage.
Learn to work with files.
Create a function that scrapes every article on a page with a for loop.
Improve the function by adding ‘webpage number’ and 'article type' parameters.
Reviews
1 month ago
you need to improve the language you are using to describe tasks. It is not English! Your wording is very polysymantic. Hire an English language editor or pass you texts by ChatGPT asking it to rewrite a return a more structural descriptions. PLEASE !!!!
1 month ago
What I learned:This project taught me valuable skills in web scraping with BeautifulSoup, HTTP requests, file handling, and dealing with real-world HTML structures. I also learned an important lesson about the challenges of maintaining tests that depend on external websites. Critical issue: test i ...
3 months ago
For the first time, HTML parsing can make you mad, but it was interesting. I recommend it.
4.3
Learners who completed this project within the Python Developer course rated it as follows: