Make your Own Web Crawler - Part 4 - More on URLs
In this video we'll be making the crawler understand all of the different kinds of links that it might come across while crawling the web. We're also making the crawler ignore links that run Javascript functions as they would be of no use to us.
Full Source Code: http://howco.de/crawler_source
Don't forget to subscribe for more!