Finding Compact, Isolated Clusters in Data Lakes

Published: 2 August 2021| Version 1 | DOI: 10.17632/k96yvyjb68.1
Contributor:
Patricia Jimenez

Description

These are the research materials that accompany article "On Exploring Data Lakes by Finding Compact, Isolated Clusters", by Patricia Jiménez, Juan C. Roldán, and Rafael Corchuelo. This package includes the following: - "data-lakes": it contains a compressed archive whose organisation is driven by data lakes. Each subfolder corresponds to data lake, and each CSV file inside a data-lake corresponds to a dataset. The last column of the datasets is called "clazz", but it is set to "0" in all cases. The column is present because many software packages expect that column to be present, although it is completely ignored. - "results": it provides the results of testing RóMULO and other competitors on the previous data lakes. Each CSV file in this folder provides the experimental data gathered for all of the competitors on all of the datasets using a particular performance measure. - "system": it provides the python code required to run and test RóMULO. There is a "launch.cmd" script that launches the experimentation. The implementation of the competitors can be found elsewhere. The implementation of GSPPCA is available from the authors at https://github.com/pamattei/GSPPCA. The implementation of AffinityPropagation, Meanshift, and OPTICS-XI is available from SckitLearn at https://scikit-learn.org/stable/install.html. The implementation of PQC is available from the authors at https://github.com/racaes/PQC. The implementation of DCC is also available from the authors at https://github.com/shahsohil/DCC.

Files

Steps to reproduce

1) Open a command shell, make it for the system folder. 2) Create a new conda environment using command "conda create --name romulo python=2.7"; 3) Activate the new enviroment with "conda activate romulo; 4) Install the requirements by using command "pip install -r requirements.txt"; 5) Execute the "launch.cmd" script.