Patent US 9,720,940

The focus of the present invention is the modular analysis of Big Data encompassing parallelization, chunking, and distributed analysis applications. Typical application scenarios include: (i) data may not reside in one database but alternatively exist in more non-identical databases, and analysis has to take place in situ rather than combining all databases in one big database; (ii) data exceeding the working memory of the largest available computer and has to be broken into smaller pieces that need be analyzed separately and the results combined; (c) data encompassing several distinct data types that have to be analyzed separately by methods specific to each data type, and the results combined; (iv) data encompassing several distinct data types that have to be analyzed separately by analyst with knowledge/skills specific to each data type, and the results combined; and (v) data analysis that has to take place over time as new data is coming in and results are incrementally improved until analysis objectives are met, or no more data is available. The present Big Data Parallelization/Modularization data analysis system and method—“BDP/M”)) is implemented in general purpose digital computers and is capable of dealing with the above scenarios of Big Data analysis as well as any scenario where parallel, distributed, federated, chunked and serialized Big Data analysis is desired without compromising efficiency and correctness.