From the course: LLM Evaluations and Grounding Techniques

Unlock the full course today

Join today to access over 25,000 courses taught by industry experts.

Human-in-the-loop systems

Human-in-the-loop systems

From the course: LLM Evaluations and Grounding Techniques

Start my 1-month free trial Buy for my team

Human-in-the-loop systems

“

- [Denys] For many tasks, human domain experts are needed to verify the correctness of outputs. This is why we need human-in-the-loop systems. This is especially critical in systems like healthcare, where the answers might be a life-or-death situation. When designing a human-in-the-loop system, there are three important considerations. Who's qualified to provide the responses? How quickly do you need to get the responses? And what is the right interface to provide the response? For our purpose, let's build a simple system. We'll assume that we're a developer that can assess responses fine, we can respond in real time, and we're okay using a command line interface. So let's head over to GitHub Codespaces and open up Branch>04_03. We can scroll down a little bit and see how we've changed the call_openai function. In this case, we've provided a system prompt. "Answer the question concisely and accurately." This is because we'll be asking some questions from this questions.txt file. We…

Contents