Value is in the Eye of the Beholder: A Framework for an Equitable Graph Data Evaluation

Abstract

Proprietary data is a valuable asset used to develop predictive algorithms that benefit a wide range of users, including customers, business owners, and decision-makers. Consequently, there is a growing interest in developing safe and robust techniques for sharing, learning models, and distributing predictions across a wide spectrum of potential stakeholders. However, a structured process to assess the value of data assets, and thus enabling collaborations among stakeholders, remains largely unexplored. This is particularly challenging when the data to be shared has a networked structure, where increasing the shared data samples potentially connects information observed by different data owners, providing new knowledge that is unavailable to any data owner individually. Here, we propose E-GraDE, a framework that assists organizations in assessing the value of their networked data to better address graph machine learning tasks. This framework includes a step-by-step analysis of the requirements of different stakeholders, such as the accuracy or fairness requisites of the models, ensuring a fair evaluation process and stronger alignment in the development of a data federation consortium. Additionally, we propose an approach to estimate the value of networked data to be shared while disclosing only a small fraction of the original information. We support our approach with extensive computational experiments, analysing each part of it through simulated use cases.