Measuring Hallucination in Large Language Models for Cyber Threat Intelligence: An Exploratory Study
Description
In this paper, we presented a exploratory measurement study to assess hallucinations in Large Language Models (LLMs)—a critical challenge in the Cyber Threat Intelligence (CTI) domain. To facilitate this evaluation, we developed HalluVision, a framework that collects CTI articles, extracts relevant entities, and quantifies hallucinations using multiple similarity metrics. Using a dataset of 4,940 articles from Threatpost, we conducted a quantitative analysis based on the distribution of similarity scores, alongside a qualitative case study analysis that illustrates concrete examples of hallucinations.
Files
Steps to reproduce
All code is at https://github.com/SeungHo0422/HalluVision-official. 1. Clone the repository and create the environment: `conda env create -f environment.yml` (or `pip install -r requirements.txt`). 2. Download this dataset and lay it out as the analysis workspace: `python scripts/prepare_workspace.py --mendeley <download dir> --workspace workspace` `export HALLUVISION_WORKSPACE=$PWD/workspace` The script follows workspace\_map.json (included here) and derives the few files not shipped because they duplicate released data. 3. Offline re-evaluation (no API key, no GPU; minutes on a laptop). From the workspace: `scripts/run_analysis.sh` — entity answer matching for both tasks, failure cases `scripts/run_cosine_stats.sh --skip-embed` — paired bootstrap on the released per-document cosine values `SKIP_CYNER=1 scripts/run_llama_eval.sh` — Llama-3 vs GPT comparison on the 500-article subset These reproduce the contents of metrics\_per\_doc/ and stats/, and the entity-based tables of the paper. 4. Full regeneration, where needed: - LLM outputs: OpenAI API key; snapshots gpt-3.5-turbo-0125 and gpt-4o-mini-2024-07-18, temperature 0.0, top-p 1.0, seed 42; prompts in the paper appendix. - Entity extraction: the fine-tuned CyNER checkpoint, released at our huggingface repository; recipe in scripts/run\_cyner\_ner.py. Fine-tuning data and hyperparameters are in ner\_finetune/. - Cosine similarity: GPU with jina-embeddings-v3, CLS pooling (analyzer/cosine\_similarity.py). - The Hacker News article text is not redistributed; rebuild it from the URLs and row indices in subsets/thehackernews\_sample\_500.json and verify against the SHA-256 hashes there.
Institutions
- Chungbuk National UniversityNorth Chungcheong, Cheongju-si