International Journal of Advancements in Technology
Open Access

Abstract

PARCAHYD: An Architecture of a Parallel Crawler based on Augmented Hypertext Documents

A. K. Sharma, J.P. Gupta, D. P. Agarwal

Search engines use web crawlers to collect documents for storage, indexing and analysis of information. Due to the phenomenal growth of web, it becomes vital to create high performance crawling systems. Augmentations to hypertext documents were proposed [6] so that the documents become suitable for parallel crawlers. PARCAHYD is an on going project aimed at designing of a Parallel Crawler based on Augmented Hypertext Documents. In this paper, the architecture of this parallel crawler is presented.