Tokenized Results of Comments on Bus Travel from Chinese Social Media Platforms
Description
The data comes from Chinese social media platforms — MicroBlog, Redbook, and TikTok — and reflects public evaluations of various aspects of bus travel. The dataset used here is the result of preprocessing the raw data, which involved two main steps: （1）Filtering Invalid Posts：To ensure data quality, invalid posts were removed. Systematic Text Cleaning and Transformation （2）The text was cleaned and transformed to facilitate subsequent analysis： This process included removing elements that might interfere with topic detection, performing tokenization and lemmatization to convert words to their base forms, and eliminating stopwords, excessively long words, and low-frequency words. After preprocessing, a total of 6,403 valid comments related to bus travel were extracted from the three platforms, covering the period from January 2023 to December 2024. Based on this dataset, seed word classification can be applied to assign each comment to a specific topic. After classification, TF-IDF can be performed on the dataset of a given topic to further extract the keywords from each comment, which can then serve as the foundation for topic modeling.
Files
Steps to reproduce
The data for this study were sourced from China's three major social media platforms: MicroBlog, Redbook, and TikTok. The data were first collected using the Octoparse tool, retrieving 32,678 original posts containing user IDs, timestamps, text content, and account type information from January 2023 to December 2024. To ensure data quality, the preprocessing phase was divided into two steps: (1) Filtering invalid data, which included retweets, content from non-personal accounts, posts with fewer than three characters, and posts mentioning “bus” but unrelated to the study topic; (2) Systematic cleaning and transformation, which involved removing interfering elements such as hyperlinks, hashtags, usernames, emojis, numbers, and punctuation, followed by tokenization and lemmatization, and further eliminating stopwords, excessively long words, and low-frequency words. Ultimately, 6,403 valid comments were obtained. In the data analysis phase, comments were first classified into four main topics—"Travel Experience," "Operational Efficiency," "Economics," and "Environmental Impact"—using seed word classification. The selection of seed words was based on high-frequency word analysis and literature review. If a text contained keywords associated with multiple topics, it was assigned to all relevant topics. To reduce omissions, synonym expansion was also applied, for example, expanding “crowded” to include synonyms such as “packed,” “too crowded,” and “overloaded.” After the initial classification, the TF-IDF algorithm was applied to extract up to 10 keywords from each comment. If fewer than 10 keywords were identified, the original tokenized results were retained. The weight of the keywords reflects their importance in distinguishing the topic of the document, and the results were visualized using word clouds. For further topic modeling, the GSDMM short-text topic model was used. Parameters such as the number of topics (K), α, β, and the number of iterations were adjusted, and the optimal number of topics was determined based on the C_V consistency index. To improve topic differentiation, the study also constructed a word co-occurrence matrix to reveal semantic relationships, and employed the Jaccard distance to evaluate distribution differences among multi-topic texts.
Institutions
- Zhejiang Sci-Tech University