SYSTEM AND METHOD FOR CRAWL POLICY MANAGEMENT UTILIZING IP ADDRESS AND IP ADDRESS RANGE
Patent №
US 7,701,944
Granted
2010-04-20
Filed 2007
Owner
INTERNATIONAL BUSINESS MACHINES CORPORATION
Lab
AI components
3
nlp · kr · planning
Assignment
Recorded
Dataset
AIPD
2023_r1 edition
Application
11625110
The present invention relates to a method for configuring a policy management protocol for a web crawler, the method further comprising the steps of determining a web space that is to be crawled by a web crawler, wherein the web space is comprised of an IP address and/or a range of IP addresses, and determining additional hostnames that are associated with the IP address and/ range of IP addresses. The method further comprises the steps of configuring the web crawler to crawl the IP address and/ range of IP addresses, and determine additional hostnames that are associated with the IP address or range of IP addresses, and performing a web crawling function upon the determined additional hostnames by the web crawler.
AI classification
Ownership
INTERNATIONAL BUSINESS MACHINES CORPORATION
assignment · 188620915
Assignors
BHAGWAN, VARUN, DESAI, RAJESH M., JALAN, PIYOOSH
On an employer assignment, the assignors are typically the inventors.