FashionVerse-MME (Multimodal Metadata Enrichment)

Description

FashionVerse-MME: A Multimodally Enriched Fashion Dataset Derived from Polyvore FashionVerse-MME is a multimodally enriched fashion dataset derived from the Polyvore dataset, designed for research in fashion retrieval, recommendation systems, multimodal learning, and vision-language modeling. The dataset contains 21,889 outfit compositions comprising 126,928 fashion items across 20 apparel and accessory categories. To enhance the semantic richness of the original dataset, each fashion item was processed through a multimodal enrichment pipeline consisting of: (1) BLIP-based image caption generation, (2) LLaMA-based caption refinement to incorporate fashion-specific semantic context, and (3) K-Means-based extraction of dominant colors and representative color palettes. The resulting metadata provides structured visual, textual, and attribute-level representations for every fashion item. The dataset includes the original fashion item images, individual metadata JSON files for each item, and a consolidated metadata JSON file containing annotations for the entire dataset. Each metadata record contains category information, image references, BLIP-generated captions, LLaMA-refined fashion descriptions, a concatenated **text_input** field combining the category name, BLIP caption, and refined caption, dominant color annotations, representative color palettes, and outfit-level associations. By combining visual content with enriched textual and color-based attributes, FashionVerse-MME supports multimodal retrieval, image-to-image retrieval, text-to-image retrieval, image-plus-text retrieval, fashion recommendation, fashion compatibility prediction, cross-modal search, and multimodal embedding learning. This dataset is derived from the publicly available Polyvore Dataset released under the Apache License 2.0. The contribution of FashionVerse-MME lies in the generation of enriched multimodal metadata, semantic annotations, and color-based attribute representations that extend the utility of the original dataset for advanced fashion AI research. Potential Applications: * Fashion Retrieval Systems * Multimodal Information Retrieval * Fashion Recommendation Systems * Fashion Compatibility Prediction * Cross-Modal Search * Vision-Language Learning * Fashion-Aware Large Language Models * Retrieval-Augmented Generation (RAG) * Attribute-Aware Ranking and Re-ranking * Multimodal Embedding Learning

Steps to reproduce

1. Download the original Polyvore Dataset from the publicly available repository. 2. Extract and organize fashion item images according to the original dataset structure. 3. Generate image captions for each fashion item using the BLIP image captioning model. 4. Refine the generated captions using the LLaMA large language model to enrich descriptions with fashion-specific semantic information, including style, occasion, and compatibility cues. 5. Extract dominant colors and representative color palettes from each image using K-Means clustering on image pixels. 6. Create structured metadata records containing category information, image references, BLIP captions, refined captions, dominant colors, color palettes, and outfit associations. 7. Generate a concatenated `text_input` field by combining the category name, BLIP caption, and refined caption. 8. Store metadata both as individual JSON files for each fashion item and as a consolidated JSON file containing metadata for the entire dataset.

Institutions

• Symbiosis International University

  Maharashtra, Pune

Categories

Licence