Xlstm
Official repository of the xLSTM.
Last verified:
What is Xlstm?
xLSTM is a next-generation recurrent neural network architecture from NXAI, a frontier AI lab based in Linz, Austria. It is an advanced variant of the traditional Long Short-Term Memory (LSTM) network that matches Transformer performance at linear computational cost. The architecture introduces three core innovations: Exponential Gating for dynamic information control, Matrix Memory & Covariance Update for long-term storage, and Memory-Independent Update enabling true linear-time operation. xLSTM outperforms dominant Transformer architectures in state-tracking capabilities and is deployed across diverse domains including time series analysis, natural language processing, robotics, and biosciences.
The flagship application of xLSTM is TiRex, a foundation model for zero-shot forecasting and classification that speaks
Xlstm pricing
Pricing model: Freemium
xLSTM source code and the NXAI xLSTM 7B model are freely available for open research, small and medium-sized enterprises (SMEs), and open innovation under the NXAI Community License (based on Meta Llama 3 Community License). Organizations significantly benefiting from the technology in commercial products or services must fairly compensate NXAI for their research and development efforts. Custom deployment, finetuning, and domain-specific TiRex models require contacting NXAI for collaboration. The license is designed to support research and open innovation while ensuring large enterprises contribute back for commercial use.
Xlstm pros
- Matches Transformer performance at linear computational cost
- Faster inference than Transformers or SSM across all benchmarks
- Substantial energy savings often by an order of magnitude
- Linear compute growth instead of quadratic like Transformers
- Significantly fewer FLOPs required for same validation loss
- Superior state-tracking capabilities Transformers lack
- Compact models like TiRex with only 35 million parameters
- Zero-shot forecasting requires no additional training
- In-context learning enables use by non-experts
- Runs on edge devices including PLCs and embedded chips
- Excels at capturing long time span relationships
- Open-source code available via pip install xlstm
- xLSTM 7B model delivers fast inference speeds
- Top positions on globally recognized leaderboards with Google and Amazon
- Versatile across time series, language, robotics, and biosciences
- Lower latency for real-time industrial applications
- Memory-efficient for resource-constrained environments
- European-made architecture for digital sovereignty
Xlstm cons
- Relatively new architecture with less community adoption than Transformers
- Primarily focused on industrial applications rather than consumer AI
- Large enterprises must commercially compensate NXAI for technology use
- Limited documentation compared to established Transformer frameworks
- Requires collaboration contact for custom deployment support
- Fewer pre-trained models available compared to Transformer ecosystem
- Specialized for specific domains rather than general-purpose AI
- Edge deployment requires NXAI's proprietary edge lab infrastructure
Frequently asked questions about Xlstm
What is xLSTM and how does it differ from traditional LSTM?
xLSTM (Extended Long Short-Term Memory) is an advanced variant of traditional LSTM that incorporates three core architectural innovations: Exponential Gating for dynamic information control, Matrix Memory & Covariance Update for long-term storage, and Memory-Independent Update enabling true linear-time operation. These advances transform LSTM into a scalable architecture that is competitive with and often superior to Transformers, addressing inherent limitations of traditional LSTM models.
How does xLSTM compare to Transformer architectures?
xLSTM matches Transformer performance at linear computational cost instead of quadratic growth. It is faster than Transformers across all inference benchmarks, delivers substantial energy savings often by an order of magnitude, requires significantly fewer FLOPs for the same validation loss, and provides superior state-tracking capabilities that Transformers lack. xLSTM maintains or surpasses Transformer-level accuracy across benchmarks while being more cost-effective.
What is TiRex and what does it do?
TiRex is NXAI's foundation model for forecasting and classification based on xLSTM architecture. The name stands for 'time' and Latin 'Rex' (king). With only 35 million parameters, it is compact and memory-efficient, running on PLCs and smaller devices. TiRex enables zero-shot forecasting through in-context learning, meaning predictions for new time series require no additional training. It excels at both forecasting and classification in industrial applications like mechanical engineering, logistics, automation, and robotics.
Can I use xLSTM for my own projects?
Yes, xLSTM source code is open-source and available on GitHub. You can install it via pip with the command 'pip install xlstm'. The technology is freely available under the NXAI Community License for open research, development, SMEs, and open innovation. The repository includes documentation and links to research papers. For non-language applications use xLSTMBlockStack, for language modeling use xLSTMLMModel.
What domains does xLSTM work best for?
xLSTM demonstrates strengths across multiple domains with current focus on time series, robotics, and agentic AI workflows. It excels in time series analysis (TiRex), natural language processing (xLSTM 7B model), vision applications (Vision-xLSTM), biosciences (Bio-xLSTM), and robotics (Large Recurrent Action Model). The architecture is being applied in mechanical engineering, logistics, automation technology, process industries, automotive systems, energy, and finance.
Is xLSTM suitable for edge and embedded deployment?
Yes, xLSTM is specifically designed for edge and embedded applications. NXAI focuses on edge deployment through their own edge lab. TiRex, with only 35 million parameters, is compact enough to run on PLCs and even smaller devices. The architecture's memory efficiency and reduced computational overhead make it accessible for broader applications on edge and embedded systems in industrial settings.
Who is behind xLSTM and NXAI?
NXAI is a frontier AI lab founded in Linz, Austria, based in Europe. The Chief Scientist and co-founder is Prof. Dr. Sepp Hochreiter, one of the fathers of modern deep learning who created the original LSTM in the 1990s. The CEO is Albert Ortig. NXAI works closely with JKU Linz and other research institutions worldwide, combining elite AI research, digital product know-how, and industry expertise.
What are the energy efficiency benefits of xLSTM?
xLSTM delivers substantial energy savings often by an order of magnitude compared to Transformers. The architecture has been refined to ensure faster training times and reduced computational overhead. Due to linear compute growth instead of quadratic, xLSTM models are more energy-efficient across all inference benchmarks, making them more sustainable and cost-effective for large-scale deployments while maintaining superior performance.
How do I get support for deploying xLSTM or TiRex?
For collaboration, custom deployment, or finetuning TiRex in your environment, you need to contact NXAI directly through their 'Contact us' or 'Talk to a TiRex specialist' options. NXAI works with companies to make processes more competitive, efficient, and future-proof. They provide support for deploying or finetuning TiRex, building custom xLSTM models for companies, and developing domain-specific TiRex models for optimizing digital twins.
What research papers are available for xLSTM?
NXAI has published multiple research papers including: the main xLSTM Paper (research basis), Vision-xLSTM Paper (performance in vision tasks), xLSTM: A Large Recurrent Action Model for robotics, Tiled Flash Linear Attention (RNN kernels outperform Flash Attention), Bio-xLSTM (AI for bioscience), xLSTM Scaling Laws (showing xLSTM requires significantly fewer FLOPs), TiRex Paper (time series foundation model), and Classification Paper (TiRex compared to other models). These are accessible through their research section and GitHub repository.