AI Limitations Overview

BREAKING! The FDA just released this draft guidance, titled Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations, that aims to provide industry and FDA staff with a Total Product Life Cycle (TPLC) approach for developing, validating, and maintaining AI-enabled medical devices. The guidance is important even in its draft stage in providing more detailed, AI-specific instructions on what regulators expect in marketing submissions; and how developers can control AI bias. What’s new in it? 1) It requests clear explanations of how and why AI is used within the device. 2) It requires sponsors to provide adequate instructions, warnings, and limitations so that users understand the model’s outputs and scope (e.g., whether further tests or clinical judgment are needed). 3) Encourages sponsors to follow standard risk-management procedures; and stresses that misunderstanding or incorrect interpretation of the AI’s output is a major risk factor. 4) Recommends analyzing performance across subgroups to detect potential AI bias (e.g., different performance in underrepresented demographics). 5) Recommends robust testing (e.g., sensitivity, specificity, AUC, PPV/NPV) on datasets that match the intended clinical conditions. 6) Recognizes that AI performance may drift (e.g., as clinical practice changes), therefore sponsors are advised to maintain ongoing monitoring, identify performance deterioration, and enact timely mitigations. 7) Discusses AI-specific security threats (e.g., data poisoning, model inversion/stealing, adversarial inputs) and encourages sponsors to adopt threat modeling and testing (fuzz testing, penetration testing). 8) And proposed for public-facing FDA summaries (e.g., 510(k) Summaries, De Novo decision summaries) to foster user trust and better understanding of the model’s capabilities and limits.

The biggest shortcoming of AI resumes is simple. They fail to sell the human behind the document. Most AI-generated resumes follow the same pattern. They stack buzzwords. They inflate language. They sound polished. But they don’t communicate anything real. When you read enough resumes, you start to notice it fast. You see phrases like “results-driven,” “strategic thinker,” “cross-functional leader.” But you don’t see proof. You don’t see context and you don’t see the person. A resume is not supposed to sound impressive. It is supposed to be clear. When I review a resume, I’m looking for three things: 1. Why are you valuable? 2. What do you actually do? 3. How have you delivered results? If I can’t answer those in under 30 seconds, the resume failed. Strong resumes don’t hide behind language. They show specifics. Instead of saying you “led a team,” tell me how many people and what the outcome was. Instead of saying you “drove growth,” show me the numbers and the timeframe. Instead of saying you are “strategic,” show me a decision you made and what it changed. Clarity beats cleverness every time. AI can help you format a resume. It can help you clean up grammar. It can even help you organize your experience. But it cannot replace your thinking. If you rely on it to tell your story, you will end up with something that sounds good and says nothing. The best resumes feel human. They are specific. They are direct. They make it obvious why you matter. If someone finishes reading your resume and still has to guess what you’re good at, you didn’t do your job.

The Death of Originality The reason all corporate Generative AI strategies look the same… is because they are the same. Ever wonder why a zebra has stripes? It was a mystery for years. It's obviously not to camouflage themselves given they stick out like a sore thumb. It's so they can hide as a herd. So when they feel threatened they can crowd together and the individuals can blend together and hide in plain sight. This is the current predominant corporate Gen AI strategy. Virtually every large company follows the same exact playbook: Same consultants. Same vendors. Same emphasis on automation. Same short-term priorities. It's the same script to placate the board, appease the shareholders, and follow the competition. It's essentially a lay-off with a paint job. The Fortune 500: Use the same LLMs (OpenAI, Claude, Gemini) Work with the same consultants Deploy AI in the same domains first Track the same KPIs The Research Supports this trend: OpenAI, Anthropic, and DeepMind all report hallucination/error rates between 20%–70%, depending on task type (reasoning, factual accuracy, summarization).¹ Companies deploying AI in customer service and legal settings are already facing legal liability.² AI Reduces Trust A 2024 Pew study found that customer trust in AI-generated content declines in time.³ Research from Gartner shows brands using standard LLMs for content generation suffer from decreased perceived uniqueness.⁴ Organizations Are De-Skilling Their Talent MIT Sloan Management Review reports that heavy AI reliance in workflows has led to a decline in critical thinking.⁵ We must start with different questions: How do we focus on our revenue growth, differentiating ourselves and creating new sources of value? How do we use AI to strengthen, not dilute, our originality? How do we avoid vendor lock-in and preserve architectural control? How do we train our people to think, not just prompt? At some point someone will need to find the courage to stand apart from the crowd. Come out come out wherever you are. ******************************************************************************** The trick with technology is to avoid spreading darkness at the speed of light. Stephen Klein is Founder and CEO of Curiouser.AI, the only Generative AI platform to augment human intelligence, not automate it. He also teaches at UC Berkeley. To learn more visit curiouser.ai or connect on https://lnkd.in/gphSPv_e Footnotes & Sources: ¹ OpenAI, Anthropic, DeepMind: Technical docs, March 2023–March 2024 ² Mata v. Avianca Airlines (2023), SEC investigations of Gen AI usage in financial disclosures ³ Pew Research Center (2024). “AI Perception and Public Trust” ⁴ Gartner / Writer.com (2023). “Brand Differentiation and Language Models” ⁵ MIT Sloan Management Review (2024). “The Quiet Cost of Automating Strategic Thinking”

Why don't we regulate AI clinicians like clinicians? The FDA's current approach is to regulate AI as a device. But autonomous clinical AI — systems that make diagnoses, treatment recommendations, and triage decisions without per-case physician review — doesn't behave like a device. The device framework was built for static products with narrow indications. It's the wrong tool for this job. In JAMA, Alon Bergman, Robert Wachter, and I propose regulating autonomous clinical AI like a clinician. We propose a licensure framework that would require: ➡️ Standardized competency exams (USMLE performance at or above the median for human test takers) ➡️ A supervised deployment period analogous to residency ➡️ A defined scope of practice specifying what the AI can do ➡️ Time-limited certification with periodic reevaluation ➡️ Clear accountability — developers own model performance; institutions own implementation ➡️ Federal preemption to prevent 50 competing state licensing regimens The physician shortage is about to get significantly worse. Rural communities have already lost more than a thousand family physicians in just six years, leaving hundreds of counties with minimal or no primary care. Autonomous AI can help close that gap. But only if we have a regulatory framework capable of authorizing safe and effective AI clinical function. We think this is a major advance in thinking about AI regulation. What do you think? Read more in my latest piece in JAMA at the link in the comments. 

If your resume is too perfect, you might not get hired. Yes, you heard that right. I’ve reviewed over 100 resumes this month alone. And honestly, most of them looked great, with clean formatting, strong action words, and everything spelled correctly. Everything you’d expect from a top candidate. But guess what? > Almost all of them used the same templates. > Most achievements sounded… almost identical. > Very few actually made me pause. You could tell AI had cleaned things up, making the resumes neater… but also flatter. Less human. For example, a product analyst wrote, “Improved dashboard performance by 18%.” Not bad, right? But we changed it to: “Redesigned product dashboard after hearing repeated user complaints, which led to 18% faster decision-making for three internal teams.” Same number. A whole new story. Remember, recruiters don’t stop reading when something is perfect. They stop when something is forgettable. What sticks is the story, the context, the decisions behind those numbers, and the person behind the bullet point. So if you’re using AI to write your resume, that’s totally fine. But don’t stop there. Go back. Add your voice. Add the “why.” Bring in your personality, even if it’s just in small ways. Because at the end of the day, hiring managers aren’t just choosing a resume; they’re choosing a human. P.S. Have you looked at your resume recently and thought, “Wait… this doesn’t even sound like me”? If yes, it might be time for a rewrite. #ResumeTips #AIinHiring #JobSearch #InterviewCoach #cvtips

We've evaluated 50+ mental health apps for investments and I personally used 20+ products. None of them delivered meaningful results. The problem isn’t technology but the assumption that therapy is just pattern recognition & that AI can replace human presence. Here's a breakdown. #1 There is lack of emotional intelligence in the current AI models that are only designed to simulate conversation. But therapy is more than words. It’s about trust, micro-expressions, and intuitive emotional calibration—things AI still struggles with. Large Language Models (LLMs) like ChatGPT and others can generate empathetic responses, but they don’t “feel” anything. They simulate and patients know the difference. #2 There is a large cultural and linguistic barriers in Indian context. Most AI models are trained in western psychological frameworks. Mental health challenges in India are unlike other cultures -societal expectations, family structures, and stigma. An AI chatbot trained on CBT (Cognitive Behavioral Therapy) modules from Western research and university doesn't understand the pressure of an arranged marriage or parental expectations in an IIT-obsessed household. #3 Most mental health apps claim to be “personalized.” In reality, they offer generic suggestions based on symptom checklists. Real therapists go beyond checklists. They adapt in real time & modify their approach dynamically. AI, despite all its advancements, still struggles with this level of nuanced adaptability. Some thoughts how AI in Mental Health could actually work. #1 A good starting point is when AI supports therapists, not replace them. Imagine a model that helps analyze past therapy sessions and provide therapists with structured insights on patient progress. Instead of AI replacing human interaction, it should act as a co-pilot that enhances therapy effectiveness. #2 AI is fantastic at pattern recognition. We could use it to identify early warning signs of mental health deterioration by analyzing speech patterns, writing styles, and biometric data. Maybe hand over the interpretation and intervention to a real human therapist who can act on this data. #3 Mental health AI models need training data that reflect the Indian psyche. Solutions should be developed with inputs from Indian psychologists, using regional languages and culturally relevant therapy approaches. An AI that understands the shame associated with divorce in India or the anxiety induced by societal expectations will be far more effective. The future of AI in mental health isn’t about replacing therapists; it’s about amplifying human connection. Founders building in mental health should focus on integrating AI where it excels (data analytics, early diagnostics, and habit formation) while keeping humans where they matter most (trust, deep healing, and emotional intuition). AI in mental health isn’t about AI being human. It’s about AI making humans better. That’s where the real opportunity lies.

The act of writing is an act of thinking, making meaning, and cognitive and creative flourishing. Delegating work to AI is riskiest to those at the learning stage, because struggle is how skills get encoded. Whether searching for the right word or wrestling with a complex problem, AI producing the result faster can rob beginners of the opportunity to become experts. Experts who already have deep domain knowledge get the most out of AI, automating technical or admin tasks in order to focus their effort where originality lives. Beneath the surface, results are also often not as good for novices. An analysis of 370,000 college student essays found that human-written essays contain 8X more novel ideas than those generated by AI. Though AI works often contain more flowery language, story lines are more homogeneous and lack distinctive ideas. There are big implications for the guardrails we should put in place around AI in K-12 and higher education, as well as in entry-level work. Full article: https://lnkd.in/gAVpHuwg

AI Risk Is Becoming Uninsurable. Contracts Are Taking the Hit. Insurance has been quietly stepping away from meaningful AI coverage. Exclusions are expanding, sublimits are shrinking, and underwriting is getting tighter. Companies are still deploying AI at full speed, and the gap has to land somewhere. It is landing in contracts. Read the full article: https://lnkd.in/gRHtVEmp I wrote about this for Corporate Counsel because the shift is real and accelerating. We are watching contracts absorb functions that insurance used to perform. That change reshapes how indemnities work, how governance is drafted, and how responsibility is allocated across the AI lifecycle. Indemnities are narrowing. Broad, catch-all promises are being replaced by precise and limited obligations. The protection that many clients think they are getting often does not exist anymore. Governance obligations are expanding. They are moving upstream into how the system is built, validated, monitored, and supervised. Documentation and controls now influence liability in a way many teams have not expected. And, shared responsibility frameworks are becoming the norm because AI risk sits at the intersection of model behavior and human decisions. This is a structural shift. Contracts are functioning as underwriting instruments because the traditional backstop is pulling away. When the safety net is gone, the contract becomes the risk architecture. If you support procurement, sales, data partnerships, or AI deployments, this matters. Boilerplate AI language is no longer neutral. Internal processes now influence exposure. Many executives still assume their insurance covers AI-related risk when it does not. That disconnect shows up in negotiations every day. The article goes deeper into how these trends are playing out in real agreements and what in-house teams can do to respond with clarity and control. For more insights, check out the Contract Trust Report: https://lnkd.in/gJdXkUpJ — Olga V. Mack I build legal systems for real life.

Everyone assumes bigger AI models automatically mean better results. The data says otherwise. There's a hard ceiling in AI development called the compute-optimal frontier. Every model no matter how large hits a point. Where more investment stops producing meaningful improvement. Train a bigger model. Error rates drop. Then they plateau. Throw more compute at it. Same plateau. This isn't a temporary problem. It's a fundamental constraint in how AI learns. What does this mean for enterprise leaders? The race to buy the biggest, most expensive AI infrastructure is not a strategy. It's an assumption and an expensive one. The enterprises winning right now aren't the ones spending the most. They're the ones asking better questions before they spend: Is our data actually ready for AI to use? Are we measuring the right outcomes? Do we have governance structures that justify this investment to the board? Scaling compute without data readiness is like building a highway with no cars to drive on it. The next wave of enterprise AI won't be defined by model size. It will be defined by deploying the right model on clean, governed, AI-ready data. Bigger isn’t smarter. Structured is. Are your AI investments built on that principle? Video:- artificialintelligencetimes #EnterpriseAI #AIGovernance #DataReadiness #AIInfrastructure