LEARNING DATA PROTOTYPES FOR INFORMATION EXTRACTION

A method for determining statistically significant token sequences lends itself for use in the recognition of broken wrappers as well as the construction of new wrapper rules. When new wrapper rules are needed as the underlying wrapped data has changed, training examples are used to recognized data rule candidates that are culled with a bias for rule candidates that would be probably more successful. The resulting rule candidate set is clustered according to feature characteristics, then compared to the training examples. Those rule candidates most similar to the training examples are used to create new wrapper rules.