Botium Core
The Selenium for Chatbots - Bots Testing Bots
Last verified:
What is Botium Core?
Botium Core is an open-source testing framework designed specifically for conversational AI, often called
Botium Core pricing
Pricing model: Freemium
Botium Core is completely free and open source, available on GitHub as a Node.js module and Docker image. Botium Box offers paid plans: Mini (SaaS) is Free for community use; Professional (SaaS) costs €429/month billed monthly for chatbot professionals; Enterprise (Self-Hosting) costs €999/month for enterprise chatbots; Enterprise (SaaS) costs €1,098/month. Botium Platform is an SaaS offer with enterprise features including relational database support, multiple deployment options, monitoring, build pipeline integration, load balancing, and enterprise-grade support.
Botium Core pros
- Open source and free to use
- Supports 30+ chatbot connectors including Dialogflow, IBM Watson, Amazon Lex
- Automated conversation testing with simple BotiumScript DSL
- Integrated speech processing for voice bot testing
- NLP model testing with intent and entity validation
- Capture and Replay functionality for easy test creation
- Paraphrasing feature enhances test coverage automatically
- End-to-end testing using Selenium and Appium integration
- Load and stress testing capabilities for scalability
- Security testing and GDPR compliance testing included
- Seamless CI/CD integration with Jenkins, Azure DevOps, Bamboo
- Cross-channel testing for web chat, voice, SMS, messaging platforms
- Domain-specific and generic test datasets included
- Botium CLI provides command-line automation power
- Bring-Your-Own-Storage supports Git, Google Drive, protected folders
- No vendor lock-in works with any conversational AI platform
- Simulates human errors like typos for realistic testing
Botium Core cons
- No graphical user interface in core version requires CLI knowledge
- Steep learning curve for developers new to command-line tools
- Requires Node.js environment installation and setup
- Manual test case writing needed for complex scenarios
- Limited enterprise support compared to paid Botium Box
- No built-in reporting dashboard in open source version
- Voice testing requires separate audio file preparation
- Session handover between Selenium sessions not supported
- Configuration via botium.json can be complex for beginners
- No free trial for paid Botium Box plans
Frequently asked questions about Botium Core
What is Botium Core?
Botium Core is the heart and brain of the Botium Stack - a free, open-source testing framework for conversational AI. Just as Selenium is the standard for testing web applications and Appium for smartphone apps, Botium is the de-facto-standard for testing chatbots and voicebots. It provides core functionality for automating conversations with chatbots and virtual assistants.
Is Botium free to use?
Yes, Botium Core is completely free and open source, available on GitHub as a Node.js module and Docker image. The Botium Stack libraries are open source and free. Botium Box offers a Free Mini plan for the community, while Professional and Enterprise plans are paid subscriptions.
Which chatbot platforms does Botium support?
Botium supports 30+ chatbot connectors including Dialogflow, IBM Watson Assistant, Amazon Lex, Microsoft Bot Framework, Alexa Skills, Google Assistant, Kore.ai, Nuance, Chatlayer.ai, and custom LLM agents. It has adaptable generic connectors for custom Conversational AI APIs with no vendor lock-in.
How do I write test cases in Botium?
Test cases in Botium are called 'convos' and are written in BotiumScript using Text, Excel, or CSV files. A simple test case has the format with #me for user input and #bot for expected bot response. You create a botium.json configuration file, write convo files, then run tests using botium-cli run command.
Can Botium test voice bots?
Yes, Botium supports voice app testing for all technologies that can receive audio streams, respond with audio streams, or both. It includes integrated speech processing and supports over 30 different Botium Connectors. You prepare audio samples, specify a resource folder, and can even add voice effects to simulate real-life scenarios.
How does Botium integrate with CI/CD pipelines?
Botium seamlessly integrates with all common CI/CD products including Jenkins, Bamboo, Azure DevOps Pipelines, IBM Toolchain, Travis CI, and GitLab. The Botium CLI enables command-line automation, making it ideal for process and test automation in build pipelines without graphical interface impediments.
What is the difference between Botium Core and Botium Box?
Botium Core is the open-source command-line framework providing core testing functionality. Botium Box is the commercial SaaS product with a modern, responsive web-based graphical user interface to configure, control, and monitor Botium Core. Botium Box includes management and reporting features, while Core requires CLI usage.
Can Botium test NLP model accuracy?
Yes, Botium includes NLP testing capabilities that allow assessment of how well chatbots comprehend and respond to user inputs. Testers can define expected intents and entities, compare them with actual outcomes, and gauge NLP model accuracy. Botium NLP Viz provides continuous visualization of NLP performance metrics.
How do I get started with Botium?
To start with Botium Core, you need Node.js installed. Install Botium CLI globally with npm install -g botium-cli, then run botium-cli init. Create a botium.json configuration file with your chatbot connection details, write test case files (convos), and run tests with botium-cli run. Documentation is available at botium-docs.readthedocs.io.
Does Botium support enterprise deployments?
Yes, Botium Platform extends the open source Botium Stack with enterprise features including relational database support, multiple deployment options, monitoring, build pipeline integration, load balancing, security testing, performance testing, and enterprise-grade support. Enterprise plans are available as self-hosted (€999/month) or SaaS (€1,098/month).