Koray Oksay

Automatic scaling of worker nodes in your Kubernetes cluster is key to ensuring you don't waste $$$s with idle compute. There are 2 main approaches for this including Karpenter and the Cluster Auto Scaler (CAS). An example of Terraform to setup Karpenter is below. Kubernetes is great at using a pool of compute resources to run variable workloads. The problem is that most workloads are bursty. If you have a very consistent workload then node scaling isn't as important but most apps will not have this type of uniform demand. Using Infrastructure as Code (IaC) tools like Terraform can make setting up your environment much easier and repeatable. Altin Ukshini compares the CAS and Karpenter below and shows how to use Terraform to setup Karpenter. https://lnkd.in/ebz4iEJn

62

Let me simplify what is happening around the "𝗣𝗿𝗶𝘃𝗮𝘁𝗲 𝗔𝗜 𝗶𝗻𝗳𝗿𝗮𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲" landscape. Today, if an org wants to build an AI data center for GenAI applications at production scale, they have the following challenges to solve - • Infrastructure cost - Which compute (GPU/TPU/accelerator) to choose from? • Operational complexities - Which LLM and how to deploy? Size, tuning, compression, APIs, etc • Deployment constraints - How to have a similar runtime for LLMs across hybrid platforms? One option is to choose all the commoditised open-source components and stitch them together, but still, no guarantee of a 𝘤𝘰𝘯𝘴𝘪𝘴𝘵𝘦𝘯𝘵, 𝘧𝘢𝘴𝘵, 𝘢𝘯𝘥 𝘤𝘰𝘴𝘵-𝘦𝘧𝘧𝘪𝘤𝘪𝘦𝘯𝘵 𝘪𝘯𝘧𝘦𝘳𝘦𝘯𝘤𝘦 𝘢𝘵 𝘴𝘤𝘢𝘭𝘦? Here is when Red Hat is offering its AI services in its decade-old signature style, i.e., building a service around open-source components with Red Hat's guarantee & credibility 𝗣𝗶𝗰𝘁𝘂𝗿𝗲 𝟭 - shows three AI services for different use cases • RH AI inference server - Only serving ready LLMs, out-of-the-box (Consistent, fast, and cost-effective inference) • RH Enterprise Linux AI - Tuning and serving LLMs via a single machine i.e. (Single Linux Server + AI inference) • RH OpenShift AI - (AI inference + multi-machine OpenShift) with complete setup for GenAI and Predictive AI solution, including MLops. 𝗤𝘂𝗲𝘀𝘁𝗶𝗼𝗻: What is cooking inside Red Hat AI? 𝗣𝗶𝗰𝘁𝘂𝗿𝗲 𝟮 gives us the answer. RedHat acquired a firm in 2024 called 𝗡𝗲𝘂𝗿𝗮𝗹 𝗠𝗮𝗴𝗶𝗰--> Neural magic built 𝘃𝗟𝗟𝗠-->vLLM is the core of 𝗥𝗛 𝗔𝗜 "𝘃𝗟𝗟𝗠" - is a cheap and fast LLM serving engine that connects model creators to accelerated hardware providers In simple terms, vLLM is the open-source equivalent of AWS Sagemaker & Google Vertex AI platforms in the context of private cloud, but with the freedom to choose your own model & underlying infrastructure In cloud-native terminology, if - Hugging Face is the 𝘎𝘪𝘵𝘏𝘶𝘣 𝘰𝘧 𝘈𝘐 Ollama is the 𝘋𝘰𝘤𝘬𝘦𝘳 𝘰𝘧 𝘈𝘐, then vLLM is the 𝙆𝟪𝙨 𝙤𝙛 𝘼𝙄. That makes it possible to deploy any model on any infrastructure with a performance guarantee. 𝙎𝙤𝙪𝙧𝙘𝙚 - Official Red Hat sources. Link in comments

37

3 Yorum

🔥OpenShift for Telco 5G Workloads (CNFs vs VNFs) I found the OpenShift for Telco 5G Workloads (CNFs vs VNFs) document to be very insightful and easy to follow. It clearly explains the differences between VNFs and CNFs, breaking down their architectures, scaling models, and lifecycle management approaches in a simple comparison table. I especially liked how the document connects these concepts to 5G use cases such as URLLC, eMBB, and mMTC, showing why CNFs are more suited for modern telco environments. The explanation of OpenShift’s role in enabling carrier-grade Kubernetes, automation with Operators, and edge computing support was very practical. Real-world use cases like 5G core functions, RAN, and MEC made the content highly relatable. I also appreciated that the challenges—such as cultural shifts, performance tuning, and hybrid deployments—were discussed honestly instead of only highlighting advantages. Overall, this document provides a strong foundation for telecom professionals exploring CNF adoption on OpenShift in their 5G journey. 💬 Comment “CNFs vs VNFs” below if you’d like me to send you the OpenShift for Telco 5G Workloads (CNFs vs VNFs) PDF 📥 — it’s ready to share! #OpenShift #DevOps #Kubernetes #RedHat #CloudNative #5GNetwork #Learning #handsonLabs #Telco5G #NetworkFunctions #Network  

87

210 Yorum