Red Hat SummitRed Hat OpenShift AI
Red Hat® OpenShift® AI is a platform for managing the lifecycle of predictive and generative AI models, at scale, across hybrid cloud environments.
What is Red Hat OpenShift AI?
Built using open source technologies, OpenShift AI provides trusted, operationally consistent capabilities for teams to experiment, serve models, and deliver innovative applications.
OpenShift AI enables data acquisition and preparation, model training and fine-tuning, model serving and model monitoring, and hardware acceleration. With an open ecosystem of hardware and software partners, OpenShift AI delivers the flexibility you need for your specific use cases.
This video can't play due to privacy settings
To change your settings, select the "Cookie Preferences" link in the footer and opt in to "Advertising Cookies."
Bring AI-enabled applications to production faster
Combine the proven capabilities of Red Hat OpenShift AI and Red Hat OpenShift in a single enterprise-ready AI application platform that brings teams together. Data scientists, engineers, and app developers can collaborate in a single destination that promotes consistency, security, and scalability.
The latest release of OpenShift AI is designed to increase efficiency by providing access to smaller, pre-optimized models. Additionally, it helps manage costs of inferencing with distributed serving through a vLLM framework. To further reduce operational complexity, OpenShift AI offers advanced tooling to automate deployments and self-service access to models, tools, and resources.
Features & benefits
Less time managing AI infrastructure
Provide your teams with on-demand access to resources, so they can self-service and scale their model training and serving environments as needed. Additionally, reduce operational complexity by managing AI accelerators (GPUs) and workload resources across a scalable clustered environment.
Tested and supported AI/ML tooling
Red Hat tracks, integrates, tests, and supports common AI/ML tooling and model serving on our Red Hat OpenShift application platform, so you don’t have to. OpenShift AI draws from years of incubation in Red Hat’s Open Data Hub community project and open source projects like Kubeflow.
Flexibility across the hybrid cloud
Offered as either self-managed software or as a fully managed cloud service on top of OpenShift, Red Hat OpenShift AI provides a secure and flexible platform that gives you the choice of where you develop and deploy your models–whether on-premise, the public cloud, or even at the edge.
Leverage our best practices
Red Hat Consulting provides services that allow you to install, configure and use Red Hat OpenShift AI to its fullest extent.
Whether you’re looking to jumpstart your AI journey, upskill your ML platform team or need guidance on building your MLOps foundation, Red Hat Consulting will provide support and mentorship.
Red Hat Enterprise Linux AI
Red Hat® Enterprise Linux® AI is a foundation model platform used to seamlessly develop, test, and run Granite family large language models (LLMs) for enterprise applications.
Partnerships
Get more from the Red Hat OpenShift AI platform by extending it with other integrated services and products.
NVIDIA and Red Hat offer customers a scalable platform that accelerates a diverse range of AI use cases with unparalleled flexibility.
Intel® and Red Hat help organizations accelerate AI adoption and rapidly operationalize their AI/ML models.
IBM and Red Hat provide open source innovation to accelerate AI development, including through IBM watsonx.aiTM, an enterprise-ready AI studio for AI builders.
Starburst Enterprise and Red Hat support better and more timely insights through rapid data analysis across multiple disparate and distributed data platforms.
Scalable Kubernetes Infrastructure for AI platforms
Learn how to apply machine learning operations (MLOPs) principles and practices to build AI-powered applications.
Collaborating through model workbenches
Provide pre-built or customized cluster images to your data scientists to work with models using their preferred IDE, such as JupyterLab. Red Hat OpenShift AI tracks changes to Jupyter, TensorFlow, and PyTorch, and other open source AI technologies.
Scaling Model Serving with Red Hat OpenShift AI
Models can be served using an optimized version of vLLM (or other model servers of your choice) for integration into AI-enabled applications on-premise, in the public cloud or at the edge. These models can be rebuilt, redeployed, and monitored based on changes to the source notebook.
Solution Pattern
Red Hat AI applications with NVIDIA AI Enterprise
Create a RAG application
Red Hat OpenShift AI is a platform for building data science projects and serving AI-enabled applications. You can integrate all the tools you need to support retrieval-augmented generation (RAG), a method for getting AI answers from your own reference documents. When you connect OpenShift AI with NVIDIA AI Enterprise, you can experiment with large language models (LLMs) to find the optimal model for your application.
Build a pipeline for documents
To make use of RAG, you first need to ingest your documents into a vector database. In our example app, we embed a set of product documents in a Redis database. Since these documents change frequently, we can create a pipeline for this process that we’ll run periodically, so we always have the latest versions of the documents.
Browse the LLM catalog
NVIDIA AI Enterprise gives you access to a catalog of different LLMs, so you can try different choices and select the model that delivers the best results. The models are hosted in the NVIDIA API catalog. Once you’ve set up an API token, you can deploy a model using the NVIDIA NIM model serving platform directly from OpenShift AI.
Choose the right model
As you test different LLMs, your users can rate each generated response. You can set up a Grafana monitoring dashboard to compare the ratings, as well as latency and response time for each model. Then you can use that data to choose the best LLM to use in production.
Explore related resources
Case study
DenizBank empowers its data scientists with Red Hat OpenShift AI
Video
AI Guardrails for LLMs make sure that your generative AI models are safeguarded
Analyst material
Omdia report: Building a strong operational foundation for AI
Checklist
Top 5 ways to implement MLOps successfully in your organization
How to try Red Hat OpenShift AI
Developer Sandbox
For developers and data scientists who want to experiment with building AI-enabled applications in a preconfigured and flexible environment.
60-day trial
When your organization is ready to evaluate the full capabilities of OpenShift AI, explore them with a 60-day product trial. An existing Red Hat OpenShift cluster is required.
