Christian Ryan

The Context Engineering Framework is quickly becoming one of the most important tools for anyone building reliable LLM systems. Getting the model to respond is the easy part. The real challenge is: → What should the model know right now? → Where should that info come from? → How should it be structured, stored, retrieved, or compressed? That’s exactly what this framework solves. 🧠 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 Context engineering = designing dynamic systems that deliver the right info, in the right structure, at the right time, so models can reason, retrieve, and respond effectively. This matters most in agents, copilots, retrieval-augmented pipelines, and anything with memory or tools. ⚙️ 𝗜𝗻𝘀𝗶𝗱𝗲 𝘁𝗵𝗲 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 Here’s the 3-layer system I use when designing end-to-end LLM workflows 👇 1️⃣ Context Retrieval & Generation → Prompt Engineering & Context Generation → External Knowledge Retrieval → Dynamic Context Assembly 2️⃣ Context Processing → Long Sequence Processing → Self-Refinement & Adaptation → Structured + Relational Information Integration 3️⃣ Context Management → Fundamental Constraints (tokens, latency, structure) → Memory Hierarchies & Storage Architectures → Context Compression & Trimming 🧱 All of this feeds into the Context Engine, which handles: → User Prompts → Retrieved Info → Available Tools → Long-Term Memory This is what gives your system continuity, task awareness, and reasoning depth across steps. ⚙️ Tools I would recommend: → LangGraph for orchestration + memory → Fireworks AI for fast, open-weight inference → LlamaIndex for modular retrieval → Redis & Vector DBs for scoped memory recall → Claude/Mistral for summarization and compression If your system is hallucinating, drifting, or missing the mark, it’s likely a context failure, not a prompt failure. 📌 Save this framework. 📩 Share it with your team before your next agent or RAG deployment. 〰️〰️〰️ Follow me (Aishwarya Srinivasan) for real-world GenAI system breakdowns, and subscribe to my Substack for deep dives and weekly insights: https://lnkd.in/dpBNr6Jg

321

59 Comments

The complexity of PPO leads practitioners to avoid online RL in favor of RL-free or offline algorithms (e.g., DPO), but why not just use simpler versions of online RL? TL;DR: REINFORCE and RLOO have been shown to work well for training LLMs. And, they do not require a value model, which drastically reduces cost and complexity. Therefore, instead of completely avoiding online RL due to the overhead associated with PPO, we can just use algorithms that provide the benefits of online RL without the unnecessary complexity. Policy gradient: RL optimizers operate by computing the gradient of the RL objective (maximizing rewards) w.r.t. policy parameters—this is called the policy gradient—and performing gradient ascent. A basic policy gradient is easy to compute (see figure) but has high variance. So, we introduce some extra complexity to get stable gradients / training. Actor-critic: We can reduce variance by incorporating the advantage function (instead of the raw reward for a trajectory) into the policy gradient estimate. When we do this, the value function acts as a normalizer / baseline to the reward estimate for a state. PPO estimates advantage with an additional copy of the LLM—called a value model or critic—that is trained via an MSE loss to predict the final reward for every token in a sequence. The critic is on-policy (i.e., it depends on the current parameters of the policy during training) and is trained alongside the policy itself during RL. This is called an actor-critic framework. REINFORCE: Due to keeping and extra copy of the LLM in memory that is trained, actor-critic frameworks come with additional cost and complexity. As an alternative, we can use the much simpler REINFORCE algorithm, which estimates the value function via an average of rewards (either a moving average of all rewards or an average of rewards in a batch) throughout training. Estimating the value in this way is low cost and removes the need for a critic. REINFORCE does yield higher variance relative to actor-critic methods, but recent work has shown that this does not matter in the context of LLMs. Algorithms like PPO were developed in a prior generation of DeepRL research, where neural networks were primarily trained from scratch. In the LLM domain, we are finetuning pretrained models with a strong prior—unstable gradients are much less of a worry. For this reason, even simple online RL algorithms like REINFORCE work well for training LLMs despite their higher variance. RLOO: To reduce variance relative to REINFORCE, we can also use REINFORCE leave one out (RLOO). This algorithm reduces variance by sampling multiple completions for every prompt and estimating the value function as the average of rewards for other completions—excluding the completion for which the value is being estimated—to the same prompt.

217

6 Comments

Why Kimi K2 is the most important AI Model Released this year Not because of its eval scores — but because it quietly rewrites the blueprint. While others scale FLOPs, Kimi scales structure. The team at Moonshot built something different: → A second-order-inspired optimizer (Muon) that converges faster by respecting geometry — stabilized with a per-head clipping mechanism that prevents divergence mid-training. → A fully automated skill factory: rubrics, tool simulators, LLM-based verifiers — all stitched together to manufacture agentic reasoning at scale. → A recursive alignment loop: critics are trained, tested, and continuously re-calibrated against objective truth before they’re allowed to judge subjective quality. This isn’t a model. It’s a protocol. And it signals the end of the monolithic model era. The next platform wars will be won by systems that: Generate, verify, and adapt in feedback loops, Make behavior programmable, and hold alignment under pressure Full breakdown here below. For original formatting see this- https://lnkd.in/dyQhWPBh

29

2 Comments

GLM-4.5, an extremely powerful Chinese open-weights model, has been available for two weeks. A detailed paper is now out with fascinating technical insights. The pace of progress in open-weights AI development is nothing short of astounding 🤩 Paper: https://lnkd.in/eww-4FSy

88

11 Comments