Dataset: AI Chat Models Performance in n8n Notification Workflows
Description
This dataset contains structured performance logs of four open-source AI chat models (Minicpm-o-2_6, Gemma-3-12B-IT, Pixtral-12B, and Qwen2-VL-7B-Instruct) deployed locally through LM Studio and integrated with n8n, an open-source workflow automation tool. The dataset was created to evaluate the efficiency of model deployment in automated notification workflows, focusing on both inference performance and workflow orchestration overhead. The dataset provides detailed information on: - System metrics: CPU usage (%), GPU usage (%), and memory usage (MB) logged per second. - Performance metrics: Model response time, workflow latency, and success status. - Scenario metadata: Test ID, scenario name, input type (text-only, text+image, image-only), task purpose, and initiation method (Telegram or Webhook). Companion files are included: - XLSX/CSV data files with all logs. - README.md documenting dataset content and scenarios. - n8n workflow (JSON import file) for replicating the experimental workflow design. This dataset supports benchmarking, reproducibility studies, and comparative analysis of local AI deployments, with potential applications in AI efficiency research, multimodal systems, workflow automation, and sustainable computing.
Files
Steps to reproduce
Steps to Reproduce the Dataset (Log_Template.xlsx) 1. System Setup A. Deploy LM Studio on a local workstation (e.g.: Ubuntu 22.04 LTS, Intel Core i7 14700k 20-core CPU 28 thread, NVIDIA RTX 4080 GPU Super with 16 GB VRAM, 32 GB RAM DDR5). B. Install and configure n8n as the workflow orchestration platform. 2. Model Deployment Load the following open-source AI chat models into LM Studio v.0.0.36 : - Minicpm-o-2_6 - Gemma-3-12B-IT - Pixtral-12B - Qwen2-VL-7B-Instruct 3. Workflow Design Build an n8n (v1.100.1) workflow with nodes: - Trigger Node: Telegram or Webhook input. - Processing Node: API call to LM Studio model. - Logging Node: Capture CPU, GPU, and memory usage per second; record timestamps. - Notification Node: Send response back to Telegram. 4. Scenario Execution A. Run tests across three input types: - Text-only (simple, medium, complex queries). - Text+Image (scene description, visual reasoning). - Image-only (object detection, captioning). B. Record response times, workflow times, and success status. 5. Data Logging Each test generates structured logs including: - Test ID, scenario name, purpose, input type, initiation method, and model used. - Model response time, workflow latency, and success status. - CPU/GPU/Memory usage arrays captured per second. 6. Data Compilation - Logs were exported into a consolidated file: Log_Template.xlsx (Sheet1). - A CSV version (Log_Template.csv) is also provided for easier import. 7. Validation - Checked logs for completeness (no missing usage arrays). - Ensured reproducibility by re-running multiple trials per scenario to confirm consistency.
Institutions
- Bina Nusantara University