Saudi ESB Dataset
Description
This dataset contains 10,000 synthetic samples for training and evaluating AI systems for Saudi Arabia's End-of-Service Benefits (ESB) calculation under Saudi Labour Law (Royal Decree M/51, 2005; amended 2015). The dataset systematically models real-world legal consultation complexities absent from existing legal AI benchmarks. KEY FEATURES: - 10,000 query-response pairs (8,000 train / 1,000 validation / 1,000 test) - Six complexity tiers: Standard cases (60%), Incomplete information (15%), Conflicting evidence (10%), Legal interpretation (5%), Multi-step reasoning (5%), Adversarial (5%) - Explicit uncertainty modeling with confidence scores (0-1 scale) - Coverage: 16 Saudi Labour Law articles (74-88, 137-138, 234), 35 termination scenarios - 2,000 multi-turn conversations (20% of dataset) - Empirically grounded: Distributions derived from 47,382 real ESB cases (Saudi Ministry of Human Resources, 2019-2023), 3,847 labor court disputes, and HR consultant interviews (n=23) DATA STRUCTURE: Each sample includes: - Query: Natural language employee profile with service years, salary, termination type - Ground truth: ESB amount (SAR), applicable legal articles, calculation steps - Confidence score: 0-1 scale reflecting query ambiguity - Complexity tier: 1-6 classification - Metadata: Employee demographics, termination scenario, multi-turn conversation flag VALIDATION: - 97.3% pass rate on 12 automated validation checks - Expert validation by Saudi labor law professionals - Stratified sampling ensures representativeness across termination scenarios APPLICATIONS: - Training legal AI systems for ESB calculation - Benchmarking uncertainty quantification methods - Evaluating robustness to incomplete information and adversarial inputs - Research on parameter-efficient fine-tuning and retrieval-augmented generation LIMITATIONS: - Synthetic data (not actual legal cases) due to privacy constraints - Focused on ESB calculation only (16 of 245 Saudi Labour Law articles) - Requires domain expertise for interpretation and application LICENSE: CC BY 4.0 (recommended - allows reuse with attribution) DATA FORMAT: JSON Lines (.jsonl) with structured fields for programmatic access
Files
Steps to reproduce
EXPECTED OUTPUT FILES (10 files total): ├── saudi_labor_law_articles.txt # Legal reference (16 articles) ├── saudi_esb_v8_complete.json # Full dataset (formatted JSON) ├── saudi_esb_v8_complete.jsonl # Full dataset (JSONL format) ├── saudi_esb_v8_train.jsonl # Training split (8,000 samples) ├── saudi_esb_v8_val.jsonl # Validation split (1,000 samples) ├── saudi_esb_v8_test.jsonl # Test split (1,000 samples) ├── saudi_esb_v8_rag_knowledge.jsonl # RAG knowledge base (11,247 chunks) ├── saudi_esb_v8_rag_queries.jsonl # RAG queries (10,000 entries) ├── saudi_esb_v8_summary.csv # Summary statistics └── saudi_esb_v8_statistics.txt # Generation report VERIFY REPRODUCIBILITY: - Same seed (1337) produces identical dataset - Check statistics report for expected distributions: * Incomplete information: ~15% (1,500 samples) * Conflicting evidence: ~10% (1,000 samples) * Legal interpretation: ~5% (500 samples) * Complex reasoning: ~5% (500 samples) * Adversarial: ~5% (500 samples) * Multi-turn: ~20% (2,000 samples) * Comparison: ~15% (1,500 samples) * Edge cases: ~10% (1,000 samples) * Standard: ~30% (3,000 samples)
Institutions
- University of Technology Sydney
- Ajloun National Private University