Oraclus’ Post

Using GPT Preview to build code and websim to test.

Ran Llama 3.1 - 405B locally!! Seems to have out performed all other models.(GPT-4, Claude 3.5 sonnet) However GPT 4o stays on the top. Note- It is difficult to comment on model comparison as every model has its own advantage and drawbacks with the use. github - https://lnkd.in/gcmGnz86 #LLMs #llama3.1 #405B #ollama #streamlit #OpenSource

GPT-4 Turbo is out of preview. Supposedly, this model is a significant improvement over previous iterations, particularly with math problems. https://lnkd.in/grc49jdf

Did you know Custom GPTs now have "Blocks" and "State" you can set? - State seems to be like the new "Memory"—but for the user of your GPT. - Blocks are a bit like instructions for the different modules in GPT's system prompt (e.g. Code Interpreter, Dall-E)

OpenAI launches predicted API which suites well with predicted output like code. This will automate things like refactoring or code duplication Predicted Outputs use speculative decoding to efficiently update documents with minor edits. Unlike traditional GPT models that generate text word by word, speculative decoding skips over predictable content based on a reference string, focusing computation on sections needing updates. https://buff.ly/3O4q7V4

Successfully built a language model. What a rewarding experiment! SchorbGPT - a 124M parameter model trained on 10B tokens of Fineweb-edu. I finally managed to get myself hooked up to a powerful 8 x H100 training cluster. Can't recommend Nebius enough. It just works :) Lack of hardware has definitely been a barrier to entry in this space so I'm really happy that's been overcome at last. The responses from the model are a bit unpredictable, but coherent, as it's only been pre-trained but not fine-tuned yet. Although I'm going to do that as soon as possible. The training data is science leaning and the model reflects this, outperforming the original GPT-2 medium on science benchmarks, despite being less than half the size and 1/500th of the cost to train. Excited to see what else I can create now with this Nebius cluster to train on!

I used bolt.new to implement a GPT-2 language model within the browser container through the #xenovacom transformers library. Performance is poor. But I learned that it could be done. StackBlitz #LLM #GenerativeAI #React Model: Uses Xenova/gpt2 - a quantized (compressed) version of GPT-2 Runs entirely in the browser using WebAssembly No server required - all processing happens client-side Configuration: - Temperature: 0.7 (moderate creativity) - Top-p: 0.85 (nucleus sampling) - Max tokens: 100 per generation - Uses beam search (2 beams) for better coherence - Repetition penalty: 1.2 to reduce redundant text Features: - Progressive loading with visual feedback - Real-time generation progress display - Error handling and recovery - Sample prompts for easy testing Dark mode UI with responsive design - The model is optimized for browser performance, trading some capability for speed and size. It's best suited for: - Short text generation - Simple completions - Basic creative writing - Quick responses While not as powerful as server-based LLMs, it provides a good balance of functionality and accessibility by running directly in the browser.

Kraken Architecture: A powerful ML framework for dynamic text generation! Utilizes Hugging Face transformers to manage multiple causal language models (CLMs). 🧠 🔹 Dynamic Model Routing: Classifies and routes inputs to the best-suited model. 🔹 Multiple Language Models: Integrates various pre-trained CLMs. 🔹 Customizable Templates: Supports input formatting with predefined templates. 🔹 Extensible Configuration: Easily adaptable custom configuration setup. Explore more about this sophisticated framework! #KrakenArchitecture #MachineLearning #HuggingFace #AI #VidaeofAI https://lnkd.in/ezbQFvE2

Have had a go using US location VPN and it's very good. Not as many features as GPT4. Cannot import files into the prompt but, it's quick and image creation was very good at interpreting prompt inputs compared to DALLE (or maybe I was just lucky!). Not tried any coding yet but according to Matt Berman (trusted YT reviewer) it's very good at least on the Snake Code test. The standard benchmark scores look very impressive as well, beating just about everything aside from Claud Opus - which comes at a very high compute cost. What's really interesting is that this is free. If it stays free in a brave attempt to steal market share and then retain that footfall with proprietary features before releasing a next-gen paid version, they will have a very strong foothold. Questions are though - a) can they afford the compute if it's free b) will the next-gen GPT 4.5/5 be so far ahead that Llama 3 is soon obsolete?

Finishing up the week with another bit of research and resulting improvement to the Personal Knowledge Vault. Using Minio which is an s3 compliant open source based file store container, with the Menome Processor for text chunking and of course, Neo4j graph vector store, its now possible to add notes to the Personal Knowledge Vault. Notes can be added with text and an an image. The text is decomposed into a Graph Document structure using langchain semantic chunking in the same manner as other types of documents and files. More interesting though is the next step is to wire this API endpoint into the new OpenAI GPT4 turbo multi-model endpoint. This makes it possible to extract automatically text from images. Based on testing I have done it looks like this will do ocr of not only typical text type photos, but also handwritten notes, providing full descriptions of objects from pictures etc. So note taking becomes a radically more powerful thing - particularly when we consider the primary use case of helping someone with functional memory challenges remember things... So next step is to wire this new endpoint into my GPT directly...