In view of the irregular trace distribution of rock discontinuities, rock mass appears as both a statistical distribution and a texture distribution in the spatial image. This paper proposes a new method on statistical texture analysis for automated demarcating the homogeneous domains of trace distribution within a rock mass. Grey-Level Co-occurrence Matrix (GLCM) is used to quantify the statistical texture features of trace distribution. Relativity, Inverse Difference Moment and Entropy are screened from ten texture parameters of GLCM using robustness analysis and using principal components analysis. The reliability of three screened texture parameters is verified by comparing the Chebyshev polynomials fitting of three screened texture parameters with Normal distribution, Fisher distribution, and Exponent distribution using χ2 testing. Automated demarcation of the homogeneous domains is implemented by means of classifying three texture parameters of Relativity, Inverse Difference Moment and Entropy in a moving window using the Iterative Self-Organizing Data Analysis Techniques Algorithm (ISODATA). The screening process of texture parameters and a case study indicates that texture parameters and automated demarcation method is so robust, reliable, and efficient that it could replace the traditional representation of the probability statistics in trace distribution and greatly save a lot of manual labor in a large-scale domain.