Every agency or corporate recruiter is using LinkedIn as a primary tool. It’s simple, fast, and you can reach candidates pretty easily. Other recruitment sources are job boards, Google, and so on. But not every great candidate is on LinkedIn, for various reasons, and not everyone can be found by Google X-ray because Google is not able to penetrate the whole internet and all forums. Since technical forums are quite often password-protected and therefore invisible to Google, Bing or Baidu, you can only locate those by using Booleans and advanced operators. But you cannot always access these websites, because they are hidden in the Deep or Dark Web.
Difference between Deep Web, Dark Web and Surface Web
Everything that search engines can’t access, but is still accessible by a normal browser if you have a URL, is part of the Deep Web.
Everything that is hidden for a purpose, and inaccessible through standard web browsers, is part of the Dark Web. That is a small portion of the Deep Web.
You are visiting numerous websites every day. This part of the Internet is called the Surface Web, which you can visit through search engines. But this is only a small part of the Internet, only around 4%. But what about the remaining 96%?
The vast majority of the Internet lies in the Deep Web. Some statistics mention that the Deep Web is estimated to be 500 times the size of the Surface Web, and I believe the size of the web is definitely even bigger than that.
But no one knows how big the Deep Web really is. Some data is hidden for a purpose (password protected websites/forums), and some of these sites cannot be crawled by current search engine technologies. Data that you can find in the Deep Web is hard for search engines to see, but unseen doesn’t equal unimportant. You can find such interesting data in the Deep Web like company databases, address books, etc. The potential for this data is unlimited.
Deep Web pages are operated just like any other site online, but they are invisible to web crawlers. Search engines (Google, Yahoo, Baidu, Yandex etc.) rely on programs known as crawlers or spiders that gather information by following the trails of hyperlinks that tie the Web together.