Schema Matching using Machine Learning

Databases are the core part of any product, with new companies getting merged and creating cross-functional teams, database merging has become one of the crucial tasks to maintain the efficiency of the product. In the process of database merging, generally, Schemas get redesigned, and Schema Matching is the first step for it. Schema Matching is a method of finding attributes in database fields that are either similar to each other linguistically or represent the same information. Indeed various approaches exist to do automated schema matching, but they fail in different scenarios, such as different language-based fields. In this paper, We explored existing methods (centroid, self-organizing maps, etc.) and proposed a hybrid approach framework that uses both provided data and the schema name to solve a one-to-one schema matching problem. We also introduced the creation of a global dictionary method for one-to-many matching. In global dictionary method, we maps similar fields to same groups using feature extraction and clustering. We have performed experiments using both techniques and compared them to existing approaches. We were able to attain an F1 Score of 0.71 using our hybrid approach, an increase from 0.69 using the existing centroid approach.