ProjectBeta

Web Crawler

Challenging
99 completions
~ 41 hours
3.9

You seem like a quite advanced user! Here you will gain even more skills: get to know Swing library and learn how to distribute tasks among the threads and control them.

Provided by

JetBrains Academy JetBrains Academy

About

The Web is called so for a reason: all these pages are connected to many others through links, resembling a spider’s web. It’s easy to get lost and miss important links, so in this project you will write a program that might help. A Web Crawler collects and saves links from a given page, storing them in the memory for you to access later. You will practice concepts frequently tested in technical interviews at top tech companies.

Graduate project icon

Graduate project

This project covers the core topics of the Java Desktop Application Developer course, making it sufficiently challenging to be a proud addition to your portfolio.

At least one graduate project is required to complete the course.

What you'll learn

Once you choose a project, we'll provide you with a study plan that includes all the necessary topics from your course to get it built. Here’s what awaits you:
Review what you know about Swing and inheritance in order to create a simple interface: a window with text.
Feed the first link to your program so that it displays the page’s content.
Practice working with regular expressions: teach the program to parse and display the page’s title.
With the help of collections and multidimensional arrays, enable the program to scan pages, collect links and their titles and display them altogether.
You’ve got to save what you found: program your Crawler to store links in a file and save them.
Practice the art of multithreading: small tasks have to be handled simultaneously! Make the parameters of depth and time regulatable in the program.

Reviews

Dan avatar
Dan
3 years ago
Last stage is pretty challenging but overall a good project for learning multithreading and swing.
Pianoplayer1
4 years ago
One of the best projects I have had so far - it is easy enough at the beginning to get motivated, but even when the stage implementations get harder, it is explained well enough to stay maintainable.
karbon avatar
karbon
4 years ago
For me it was one of the hardest project to complete. Nevertheless during the project you can learn and get familiar to multithreading and concurrency. Anyway, if you are a totally beginner, it is not a project for you (yet ;) )

3.9

Learners who completed this project within the Java Desktop Application Developer course rated it as follows:
Usefulness
4.6
Fun
4.2
Clarity
3.0