A Decontextualized LLM-based Safeguard Technique for Automated Jailbreak Mitigation

Description

This repository contains the D-Shield Tool and all associated analyses, including: a Confusion Matrix Analysis (Jupyter notebook and data), a Fleiss' Kappa inter-rater agreement analysis (notebook and data), a Majority Vote Analysis (notebook and results), Expert Classification Results (comprising potentially non-compliant outputs alongside expert and LLM classifications), D-Shield performance results compared to a standard ChatGPT-4o baseline, and a Tool Comparison spreadsheet summarizing cross-tool evaluation outcomes.

Steps to reproduce

Readme inside the folder

Institutions

• Gran Sasso Science Institute

  Abruzzo, L’Aquila

Categories

Licence