How to review third party paper with AI: 1. Identify which contract types you want to focus on first. A simple rule of thumb I find useful is to look at the highest volume contract that you redline the least. NDAs are an obvious bet and are ripe for automating away entirely. 2. Decide whether you are looking for automation or augmentation. Do you feel your team simply should not be wasting their time reviewing a particular kind of contract AT ALL? Or do you want to empower your team to be more consistent and efficient when reviewing third party paper? 3. Build your playbook before you build your AI workflow. The quality of your AI review is only as good as the standards you're asking it to enforce. Document your acceptable positions, fallback clauses, and hard lines. If your team can't articulate what "good" looks like, your AI certainly won't figure it out for you. (Note, Flank builds a playbook for you automatically based on your previously redlined documents). 4. Start with a pilot and measure real metrics. Pick 20-30 contracts and run them through your AI workflow alongside your normal process. Track turnaround time, revision quality, and how often AI flags match what your lawyers would have caught. Use this data to refine your prompts and build trust with your team. (If you'd like help setting up an evaluation framework, get in touch with me or Lorna Khemraz and we can help). 5. Design the human-AI handoff carefully. The most common anti-pattern isn't the AI missing issues, it's lawyers not trusting the output enough to act on it, or trusting it too much and missing edge cases. Be explicit about what requires human judgment and what doesn't. If the AI includes a supervision platform that keeps you in the loop EVEN when you're putting your reviews on autopilot, this is a bonus (Flank offers this). Finally, if you're an enterprise working at real scale with a fragmented, multi-region legal team, I always recommend working with an AI supplier who can offer a forward deployed team to help with the set up, evaluation, and ongoing maintenance, so you can rest easy and focus on the ROI.
Great breakdown! Especially the point about building the playbook first. AI can’t enforce standards your team hasn’t clearly defined.
Clear frames like this, is why we are now starting to see AI automate entire chunks of work away - not just shallow demos/POCs.
Excellent roadmap, plus building playbooks before focusing on the workflow is👌
More of this content for legal teams please! This is exactly what we need more of. A super useful breakdown they can apply!
Jake Jones such a good overview. And on (5), cherry on the cake is when you convert AI skeptics to champions to the point that they are very comfortable vouching for reliability of their agent's output in front of the business and already looking for the next contracting challenge to automate!
Min-Young Leclaire LL.M. oec
Helping in-house legal teams execute the simplest impactful change. Let's move fast and fix things.
2wThanks! I have a couple of builds. First, noting the teeny bandwidth of most in-house legal teams, even creating a playbook and having time to test 20-30 contracts may be overwhelming and end up being destroyed by distraction. Provider support can help with that (and I've no doubt your team are excellent), but may still take a bit of time to sign off. It will also reduce the sense of grip within the in-house team. One possible practical solution can be to plan and pilot first for a single aspect of the contract. Let's say governing law and dispute resolution for example. This creates fewer variables and makes it easier for the in-house team to see past edge case paralysis. Second, while everyone starts with NDAs for the good reasons you stated, it's important to think and be realistic about scalability outside of NDAs. For example, one angle to that is whether the project value and ROI can exist if you only succeed with NDAs. Another is thinking about how much of the contracting ecosystem has enough proximity in behaviour to NDAs. This comment is not intended to undermine what is a nice set of clean and clear steps. Thanks again for the post. And congratulations on the new feature!