HORIZON HISTOGRAM OPTIMIZATIONS

Values that occur above a threshold frequency for certain characteristic(s) of a data set are identified. A limited number of count buckets are allocated based on the threshold. Buckets store proxy counts for identifying candidate sets of values rather than actual counts. The data set is divided and each portion is analyzed separately, by iterating through each item in that portion. During each iteration, depending on an item's value(s), a bucket is incremented, all buckets are decremented, or a bucket is assigned or reassigned to count different value(s). A candidate set of values and associated counts is selected for a portion based on the buckets. The candidate sets for each portion are merged and, in some embodiments, filtered based on the associated counts. Actual frequencies are then determined for the values that remain in the merged candidate set.