Agency Analytics onboarding challenge (Create a PHP crawler using PHP8 & Laravel)
What is this?
Hey guys! I just received an onboarding challenge from Agency Analytics. This is my solution! In order to get this challenge going, I created a docker box. Been using it for a while and really like it's simplicity. For this challenge, I decided to create a bootstrap frontend website where the user can submit a URL and a number of pages to crawl. The request in then send by AJAX using jQuery to a server side PHP script. It receives the information and send back a JSON payload. I am using a Laravel 8 framework for this project on a PHP 8 nginx server. Please refer yourself to this readme for a full explanation on how it works.
About This challenge
Using PHP, build a web crawler to display information about a given website. Crawl 4-6 pages of our website agencyanalytics.com given a single entry point. Once the crawl is complete, display the following results:
Number of pages crawled
Number of a unique images
Number of unique internal links
Number of unique external links
Average page load in seconds
Average word count
Average title length
Also display a table that shows each page you crawled and the HTTP status code Deploy to a server that can be accessed over the internet. You can use any host of your choosing such as AWS, Google Cloud, Heroku, Azure etc. Be sure to include the url in your submission.
Repo URL: https://github.com/broomcms/crawlit_c...
Working demo: https://crawlit.webiummedia.com/