https://jimmyhoulala.github.io/en/tags/chinese-word-segmentation/Chinese Word SegmentationHugo Blox Builder (https://hugoblox.com)en-usSat, 01 Feb 2025 00:00:00 +0000https://jimmyhoulala.github.io/media/icon_hu7729264130191091259.pnghttps://jimmyhoulala.github.io/en/tags/chinese-word-segmentation/https://jimmyhoulala.github.io/en/project/chineseseg/Sat, 01 Feb 2025 00:00:00 +0000https://jimmyhoulala.github.io/en/project/chineseseg/<p>This project implements a Chinese Word Segmentation (CWS) system based on a BiLSTM-CRF architecture, enhanced with several advanced features such as pretrained character embeddings, character-type embeddings, bigram embeddings, multi-corpus training, a customized CRF decoder with illegal transition constraints, loss visualization, and evaluation utilities. The system supports common CWS datasets including MSRA, PKU, and Souhu, and provides a complete workflow from data preprocessing → vocabulary construction → model training → evaluation → inference. Experiments show that the model achieves an F1 score of approximately 0.91 on the MSRA benchmark. The project is modular, extendable, and suitable for both engineering use and academic exploration.</p>