METHOD AND SYSTEM FOR MITIGATING AUTOMATED AGENTS OPERATING ACROSS A DISTRIBUTED NETWORK

A content delivery network (CDN) service provider extends a content delivery network to gather information on atomically identifiable web clients (called “user agents”) as such computer-implemented entities interact with the CDN across different domains being managed by the CDN service provider. In one embodiment, a set of machines, processes, programs, and data comprise a data system. The data system tracks user agents, preferably via cookies, although one or more passive techniques may be used. A user agent may be a cookie-able device having a cookie store. As the user agent navigates across sites, a CDN-specific unique identifier used by the system to correlate user agents is generated. Preferably, the unique identifier is stored as an encrypted cookie. The unique identifier represents one user agent (and, thus, one cookie-able device's store). The system tracks user agent behavior on and across customer sites that are served by the CDN, and these behaviors are classified into identifiable “segments” that may be used to create a profile. CDN customers use the data system to obtain information that characterizes the user agent.