AN IMPROVED DATA ETL METHOD APPLIED TO REAL-TIME DATA ETL SYSTEM

The data ETL (Extraction-Transformation-Loading) dispatching method of the present invention is applied to a real-time data ETL system, wherein a main ETL dispatching engine executes tasks according to preset sequence. In the execution, some key information are memorized by the main ETL dispatching engine, such as lscycle (latest successful data cycle), curcycle (current processing data cycle), and endcycle (processing end data cycle), etc., and are transferred to called programs. After the execution of the called programs, the main ETL dispatching engine records and keeps the updated dispatching information. In the data re-extraction, the data ETL dispatching method is adopted for the automatic re-extraction of some tasks and some cycles therein. The data ETL dispatching method, solves the problem caused by cyclic closed-loop extraction in traditional ETL dispatching programs according to timestamps in the real-time data ETL system, simplifies the tasks of the called program by allowing the called programs to focus on its own business logic, thereby giving sufficient development time for the real-time data ETL field, resulting in greatly enhancing the efficiency of project implementation.