Java and AI with LangChain4j: Jakarta EE gets AI
0 likes•101 views
Generative AI burst on to the public scene in November 2022, over ten years after Andrew Ng’s Machine Learning premiered on Coursera. Java developers have long felt like second class citizens, compared to the cool Python kids. LangChain4J changes the game. Java is cool with AI now! This lecture from 30-year industry veteran Ed Burns explores why it took so long for Java developers to have access to easy-to-use AI libraries, compared to Python developers. LangChain4J is the most popular of the Java libraries, but others exist. After a brief look at the landscape, we’ll take a deeper look at LangChain4J and how you use it to perform the most popular AI usage pattern: Retrieval Augmented Generation (RAG). RAG is basically a way to bring your own domain specific data to an existing AI model, and benefit from its power.
Java and AI with LangChain4j: Jakarta EE gets AI
- 1. Java and AI with LangChain4j Code the future with AI Ed Burns @edburns.bsky.social Principal Architect https://aka.ms/edburns/jugdarmstadt2025
- 2. Ed Burns professional biography Client NCSA Mosaic (1994) SGI Cosmo Web Authoring Sun Netscape 6 OJI Server J2EE JSF (2002) Oracle Java EE Servlet, JSF, Bean Validation, etc. Cloud Java Champion Microsoft Azure Cloud (2019) Jakarta EE 11 committer and release coordinator Books
- 3. Change in content from posted agenda Still 100% Java and AI with LangChain4j. Removed content: After a brief look at the landscape. How you use it to perform the most popular AI usage pattern: Retrieval Augmented Generation (RAG). Added content: langchain4j-microprofile-jakarta. If you want the RAG content, I recommend Easy RAG with LangChain4J and Docker by Julien Dubois.
- 4. JUG Tour Around JavaLand 2025
- 5. Breaking Jakarta EE news Jakarta EE 11 Web Profile released this past Friday. https://github.com/jakartaee/specifications/pull/802 https://deploy-preview-802--jakartaee- specifications.netlify.app/specifications/webprofile/11/ Many EE 12 component specifications in Plan Review stage. MicroProfile is discussing whether and how to host MicroProfile in Jakarta EE. The need for a unified Config solution for EE 12. Coincidentally timed soon after Evolving our middleware strategy blog post from Red Hat.
- 6. New to AI?
- 7. Don’t panic!
- 8. Ed’s fairly obvious observations Recognize the market forces that have always been in play. Vendors want adoption. They’ll make it easy to adopt. This is also true with AI.
- 9. James Gosling: I realized if I wanted bankers to use automatic garbage collection, I needed to give them curly braces rather than lots of irritating silly parentheses (LISP).
- 10. AI related career advice
- 11. Everyone I know admits: AI is moving faster than anything that has come before https://aka.ms/DeveloperCareerMasterplan/slides https://aka.ms/RockStarProgrammerSecrets
- 12. Scott Hanselman: Let’s all just chill From a trust perspective Very important to keep the “intelligence” in AI in perspective. LLMs cannot answer simple questions such as “Which episode of Gilligan’s Island was about mind reading?” The question that no LLM can answer and why it is important https://www.mindprison.cc/p/the-question-that-no-llm-can-answer
- 13. How to chill? Building some trust helps.
- 14. https://copilot.github.trust.page/faq I’m worried that my code is sent to Microsoft to do the suggestions. My concerns include: • Can someone snoop on my code using a man-in-the-middle attack? • My code is kept by Microsoft? • My code is being used by Microsoft to train the model? GitHub Copilot Trust Center Can I keep my code private?
- 16. Java meets AI Content from Emmanuel Hugonnet http://www.ehsavoie.com/ Tell the LangChain4j origin story Dmytro Liubarskyi
- 17. Components of LangChain4J Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 18. Quick look at LangChain4j examples https://github.com/langchain4j/langchain4j-examples
- 19. Jakarta EE meets AI smallrye-llm langchain4j- microprofile -jakarta Microprofile config Microprofile Fault Tolerance Microprofile OpenTelemetry CDI Content from Emmanuel Hugonnet http://www.ehsavoie.com/ Set up telling the value prop. A few slides from now
- 20. Value proposition for Langchain4j-microprofile-jakarta Bring the value of open standards to the rapidly evolving world of AI. AI-enable everything in Jakarta EE with all the power of LangChain4j. Without introducing any new AI standards. Builds on Jakarta Contexts and Dependency Injection MicroProfile Config MicroProfile Fault Tolerance MicroProfile Telemetry Delivered as Build Compatible CDI extension
- 21. Components of SmallRye LLM/LangChain4J Jakarta 1. AI Service Interface Processing: o Detects interfaces annotated with @RegisterAIService o Generates proxy implementations using LangChain4J o Makes them available as CDI beans 2. Plugin Bean Creation: o Reads configuration from properties with smallrye.llm.plugin.* prefix o Creates beans for LangChain4J components (models, embedding stores) o Makes these components available for injection 3. MicroProfile Config Integration: o Translates properties to LangChain4J configuration o Handles environment variable substitution (e.g., ${azure.openai.api.key}) 4. Fault Tolerance: o Provides integration with MicroProfile Fault Tolerance o Adds resilience to AI service calls 5. Telemetry: o Provides observability for AI service calls o Integrates with OpenTelemetry
- 22. Some Code
- 23. CDI Injection Injection.java @Inject @Named(value = "ollama") ChatLanguageModel chatModel; @Inject @Named(value = "embedding-store-retriever") ContentRetriever retriever; Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 24. AI Service SimpleAIService.java @RegisterAIService(chatLanguageModelName = "ollama", tools = {org.wildfly.ai.websocket.Weather.class,org.wildfly.ai.websocket.Cal culator.class}, scope = SessionScoped.class) public interface SimpleAIService { @SystemMessage(""" You are an AI named Bob answering general question. Your response must be polite, use the same language as the question, and be relevant to the question.""") String chat(@UserMessage String question); } Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 25. TokenStream instead of String
- 26. Demo Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 27. Another approach, with MCP
- 28. But what is MCP?
- 29. But what is MCP? https://modelcontextprotocol.io/
- 30. But what is MCP? https://github.com/modelcontextprotocol/servers
- 31. Demo Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 32. MCP Server MCP tools workflow MCP Server Web Application LLM MCP Server 1 2 3 4 MCP Server 5 6 Content from Emmanuel Hugonnet http://www.ehsavoie.com/ MCP Server Database Filesystem Message queue JavaFX
- 33. AI Service with MCP Tools SimpleAIService.java @RegisterAIService(streamingChatLanguageModelName= "streaming- ollama", toolProviderName = "mcp") public interface ChatAiService { @SystemMessage(""" You are an AI named Bob answering general question. Your response must be polite, use the same language as the question, and be relevant to the question.""") TokenStream streamingChat(@UserMessage String question); } Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 34. Demo Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 35. Conclusion Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 36. What we saw External configuration instead of code CDI Injection AI Service support Cross-vendors: Micronaute, OpenLiberty, Quarkus, WildFly Enterprise ready : OpenTelemetry / Fault Tolerance / MCP Makes LangChain4J even easier to use ! Content from Emmanuel Hugonnet http://www.ehsavoie.com/
- 37. Resources • LangChain4J: o https://docs.langchain4j.dev/ • Model Context Protocol: o https://spec.modelcontextprotocol.io/ • smallrye-llm: a cross vendor LangChain4J -CDI integration library o https://github.com/smallrye/smallrye-llm • WildFly AI Feature Pack: o https://github.com/wildfly-extras/wildfly-ai-feature-pack • WildFly MCP: to interact with WildFly servers through MCP o https://github.com/wildfly-extras/wildfly-mcp • Admin your WildFly with LLM and MCP o https://www.youtube.com/watch?v=wg1hAdOoe2w • Making WildFly Glow with Intelligence: o https://www.wildfly.org/news/2025/02/10/Glowing-with-AI/ • Playing with Generative AI with WildFly: o https://www.wildfly.org/news/2024/11/04/WildFly-playing-with-generative-ai/
- 38. Thank you Ed Burns @edburns on most of the things • Please fill out this survey • 5 winners will be chosen to win a $100 digital gift card • https://aka.ms/edburns/survey2025
- 39. 1.What is your key decision making process when it comes to app modernization? 2.What’s your key challenge when modernizing your Java workload? 3.Are you a Java only shop or multiple language shop? 4.What is the tech stack/framework for your Java apps? 5.When migrating, what is your primary destination on cloud? What platform are you choosing? VM? PaaS? Containers? Why? 6.How did you or your team make the above decisions?