Underthesea
Underthesea - AI Assistant
Last verified:
What is Underthesea?
Underthesea is an open-source Vietnamese Natural Language Processing (NLP) toolkit designed to support research and development in Vietnamese language processing. It is a suite of Python modules, datasets, and tutorials that provides easy-to-use APIs for applying pre-trained NLP models to Vietnamese text. The toolkit was created in March 2017 to fill the gap for a complete, open-source, and easy-to-install Vietnamese NLP tool comparable to English counterparts like NLTK, spaCy, and Polyglot.
Key features include sentence segmentation, word segmentation (tokenization), part-of-speech (POS) tagging, chunking, dependency parsing, named entity recognition (NER), text classification, sentiment analysis, text normalization, and language detection. The toolkit supports both machine learning models and deep learning models, with some features using generative models. It also provides access to Vietnamese NLP resources including datasets like UTS2017-BANK, VNESES, VNTQ_BIG, VNTQ_SMALL, and VNTC. Upcoming features include machine translation, text-to-speech, and automatic speech recognition.
Underthesea is designed for researchers, data scientists, developers, and students working on Vietnamese NLP projects. It is particularly useful for those building applications requiring Vietnamese text processing, such as chatbots, content classification systems, sentiment analysis tools, and information extraction systems. The toolkit supports educational, research, and commercial purposes under the GNU General Public License v3.
Underthesea pricing
Pricing model: Freemium
Underthesea is completely free and open-source software published under the GNU General Public License v3.0 license. There is no paid tier or commercial pricing. Everyone can use it freely for educational, research, and commercial purposes. The toolkit is available via pip install underthesea and all features including pre-trained models are free to use.
Underthesea pros
- Open-source and free to use under GPL-3.0 license
- Simple pip installation with pip install underthesea
- Comprehensive Vietnamese NLP tasks in one toolkit
- Easy-to-use API for pre-trained models
- Supports both machine learning and deep learning models
- Includes sentence segmentation for Vietnamese text
- Word segmentation with compound word support
- Part-of-speech tagging for Vietnamese
- Named entity recognition for Vietnamese
- Dependency parsing with deep learning models
- Text classification with domain-specific models (general, bank)
- Sentiment analysis with domain-specific models
- Text normalization for Vietnamese
- Language detection capability
- Extensive documentation and comprehensive guides
- Active community support with quick issue responses
- Includes ready-to-use Vietnamese datasets
- Colab notebooks available for quick testing
- Live demo available at undertheseanlp.com
- Works with Python 3.6+
Underthesea cons
- Only supports Vietnamese language, not multilingual
- Some features still under development (machine translation, TTS, ASR)
- Requires downloading separate models for some tasks
- Deep learning models may require GPU for optimal performance
- Limited to Vietnamese NLP unlike spaCy which supports multiple languages
- Some dependency parsing features marked as work-in-progress
- Requires Cython and additional dependencies for installation
- May have slower performance compared to optimized commercial APIs
- Limited pre-trained models compared to large-language-model based tools
- No official cloud API service available
Frequently asked questions about Underthesea
What is Underthesea?
Underthesea is a toolkit that supports the research and development of Vietnamese natural language processing. It is a suite of open source Python modules, data sets and tutorials supporting research and development in Vietnamese Natural Language Processing.
What is Vietnamese NLP?
Vietnamese NLP (Natural Language Processing) is the field of artificial intelligence that focuses on enabling computers to understand, interpret, and generate Vietnamese language text. Underthesea provides tools for tasks like word segmentation, POS tagging, named entity recognition, and sentiment analysis specifically for Vietnamese.
How do I install Underthesea?
To install underthesea, simply run the command: pip install underthesea. The installation is straightforward and satisfaction is guaranteed. For some features like text classification and sentiment analysis, you may need to download additional models using underthesea download-model commands.
What NLP tasks does Underthesea support?
Underthesea supports sentence segmentation, word segmentation, POS tagging, chunking, dependency parsing, named entity recognition, text classification, sentiment analysis, text normalization, and language detection. Upcoming features include machine translation, text-to-speech, and automatic speech recognition.
Is Underthesea free to use for commercial purposes?
Yes, Underthesea is open-source software published under the GNU General Public License v3.0 license, allowing everyone to use it freely for educational, research, and commercial purposes.
What Python version is required?
Underthesea requires Python 3.6 or higher to run properly.
Does Underthesea support languages other than Vietnamese?
No, Underthesea is specifically designed for Vietnamese Natural Language Processing and does not support other languages.
How do I download models for text classification and sentiment analysis?
Use the commands: $ underthesea download-model TC_GENERAL and $ underthesea download-model TC_BANK for text classification, and $ underthesea download-model SA_GENERAL and $ underthesea download-model SA_BANK for sentiment analysis.
What datasets are available in Underthesea?
Underthesea includes Vietnamese NLP resources such as UTS2017-BANK (categorized, open license, 2017), VNESES (plaintext, open license, 2012), VNTQ_BIG (plaintext, open license, 2012), VNTQ_SMALL (plaintext, open license, 2012), and VNTC (categorized, open license, 2007). You can list them using $ underthesea list-data.
Where can I find documentation and tutorials?
Documentation is available at Underthesea Documentation website. Colab notebooks (latest and stable versions) are available for testing. The team also has a Facebook page (Underthesea Page) and YouTube channel (Underthesea NLP Channel) with tutorials. Live demo is available at undertheseanlp.com.