CAZy-HABench30K: A Homology-Aware Benchmark Dataset for CAZy Enzyme Family Classification

Description

This dataset contains the CAZy-HABench30K benchmark used for homology-aware CAZy enzyme family classification. The dataset includes 30,000 protein sequences collected from UniProt and annotated with CAZy family and class labels. It covers 60 CAZy families across the six major CAZy classes: glycoside hydrolases, glycosyltransferases, polysaccharide lyases, carbohydrate esterases, auxiliary activities, and carbohydrate-binding modules. The package provides FASTA files, family/class labels, MMseqs2 cluster assignments, and fixed train/validation/test splits generated using a cluster-disjoint homology-aware splitting protocol. The final split contains 21,000 training sequences, 4,500 validation sequences, and 4,500 test sequences. The dataset also includes prediction-level outputs, homology leakage analysis results, threshold sensitivity tables, ablation results, truncation analysis outputs, and per-family performance summaries used in the associated manuscript. This benchmark is intended to support reproducible evaluation of CAZy enzyme family classification models under homology-aware conditions. It is particularly suitable for studying performance inflation caused by random sequence-level splitting, remote-homology generalization within known CAZy families, protein language model evaluation, and comparison with homology-based inference methods such as MMseqs2 nearest-neighbor transfer. No external funding was received for the creation, preparation, or publication of this dataset. Keywords: CAZy; enzyme family classification; homology-aware benchmark; MMseqs2; protein language models; ESM-2; homology leakage; UniProt; bioinformatics; benchmark dataset

Steps to reproduce

1. Download and extract the CAZy-HABench30K dataset package. 2. Use the provided FASTA and label files in the `sequences/` and `labels/` folders. The complete dataset is provided as `all.fasta` and `labels.csv`, while the final homology-aware train/validation/test partitions are provided as `train.fasta`, `val.fasta`, `test.fasta`, `train_labels.csv`, `val_labels.csv`, and `test_labels.csv`. 3. Use the provided MMseqs2 cluster assignments in `splits/mmseqs2_clusters_id20_cov60.tsv` and the split summary in `splits/split_report.json` to verify that the benchmark follows a cluster-disjoint homology-aware split protocol. 4. Clone the associated GitHub repository containing the source code and install the required Python dependencies using the provided `requirements.txt` file. 5. Run the model training and evaluation scripts using the fixed train/validation/test files provided in this dataset. The final split files should be used as the reference split for reproducing the reported results. 6. Use the prediction outputs and result tables provided in the `predictions/` and `results/` folders to reproduce the reported performance comparisons, homology leakage analysis, threshold sensitivity analysis, multi-task learning ablation, unfreezing ablation, truncation analysis, and per-family performance summaries. 7. To reconstruct the dataset generation process from public resources, use the dataset construction script in the associated GitHub repository. This script retrieves UniProt protein sequences, assigns CAZy family/class labels, and generates the initial 30,000-sequence benchmark. The final homology-aware split should then be generated using the MMseqs2-based splitting script, or alternatively verified using the split files already provided in this dataset package.

Institutions

• Afyon Kocatepe University

  Afyonkarahisar Province, Afyonkarahisar
• Ostim Technical University

  Ankara, Ankara

Categories

Related Links

Licence