Weekly Digest: 15 Minutes of Rapid Fire AI News with Alex Duffy

‍

Welcome to our Weekly Digest, where Alex Duffy summarizes the latest news you should know across the AI ecosystem, ComfyUI, and Salt AI in just 15 minutes. This week was absolutely jam-packed, with OpenAI debuting GPT4o, a wave of announcements at Google I/O, robotics breakthroughs, and Salt's own beta release.

‍

Check out the video above, and follow along with Alex's notes and links below:

‍

Big News

• OpenAI announced GPT4o, a truly multimodal new model that can understand images, sounds, and text
  • Some crazy demos that made me think that glasses were the obvious form factor
    • Multimodal models + glasses makes sense - could be more human, positive externalities
    • While not all wearables are good, glasses powered by Llama 3 seem to be popular
      • Won the Llama3 hackathon‍
  • Ilya Sutskever, one of the cofounders and their chief scientist (and the whole superalignment team) left OpenAI‍
  • OpenAI released Media Manager so content owners can manage how their works are used in AI and Model Spec ("our approach to shaping desired model behavior"), so trying to at least demonstrate how they’re approaching responsible AI

• At Google I/O, new LLMs, video generation, assistant, agents, integrating AI across their products (people were starting to do it themselves)
  • Actually also sneakily released a peek at glasses
  • Deepmind releases Alpha Fold 3
    • Hot off the heels of Meta and Google releasing medical models a couple weeks ago, we continue to see it as a focus. So exciting
      • They followed up with second paper and new medical models

• AI & Robotics exploding
  • Update to Tesla Robotics
    • Elon’s xAI nears $6B raise @  $18B valuation
      • The data Elon has access to holds huge value here - Two sources of data scale infinitely
  • Hugging Face launches LeRobot open source robotics code library
  • Robots balancing on exercise balls
  • Robots learning from the internet
  • Robot cleaning robots leveling up
  • Can now buy a humanoid robot for $16k
    • Could pair with your own humanoid parkour research

• Salt AI’s Beta Launch!
  • Blog Post
  • Perplexity example
  • Deploy workflows to be used as mini apps on any Discord

‍

New in LLMs

• Looking at Llama 3’s performance
  • May be not as great out of the box but anecdotally great with in context learning or few shot learning
• Long Context Prompting > Finetuning in many cases
  • But if it gets too long performance can suffer, need relevant information
  • Right now it can be used to help in specific scenarios like coding
  • Language models might go multi-token?
  • Or some new architecture may be released
  • Moral of the story, this is the worst these tools will ever be!
  • maybe less good for learning new things and better for honing in on a specific area
• Overview of Agents
  • Huggingface Agents - explains agents well
• Transcribe 1-hour videos in 20 SECONDS with Distil Whisper

‍

New in Generative AI

• Eleven Lab showed preview of ElevenLabs Music.
  • Rapping was.. really something
• Adobe project Neo, 3d content creation
  • Autodesk unveils Research Project Bernini for generative AI 3D shape creation
• Bytedance’s version of Sora
• Trending video
  • The Hardest Part made with Sora
• Tencent beats Stability to the punch with an SD3 architecture model
  • China is hot at the heels of American innovation

‍

New In ComfyUI

• Guidelines for organizing workflows
• New SDXL Controlnet models
• Light control implemented in IC Light

Bonus News

• Salt Release
  • AudioViz
  • Agents
  • Other examples
• How to build with LLM’s
  • LinkedIn’s learnings
  • Langchain
  • Instructions on how to train them from scratch
  • SLOP - unwanted AI generated content
    • Glitch tokens that can make them generate unwanted content
• Human preference prediction competition with $100,000 in prizes!
  • People LOVE benchmarks, we NEED them to filter the signal from the nosie
• Rumors of legal AI coming fast
• Google released 1.4 petabyte human brain connectome with 57k cells and 150M synapses in @ScienceMagazine
• NEW MODELS
  • Vision model LLAVA-NEXT released - demo
  • Chinese model DeepSeek-V2: The Cutting-Edge Open-Source MoE Model!
    • Yi just released an update on their model family - 6B, 9B, 34B - Apache 2.0 licensed
  • release from IBM : Fully transparent Granite Code models! 🚀 Comes in 3B, 8B, 20B and 34B
  • Hermes 2 Pro on Llama-3 8B! Nous Research's first Llama-3 based model is now available on HuggingFace
  • Refuel LLM-2 the world's best LLM for data labeling, cleaning, and enrichment
• NEW DATA
  • OpenDAC: an open dataset to train AI systems to help design compounds for the direct capture of CO2 from the air.
  • 1M ChatGPT Interaction Logs in the Wild
  • WebInstruct dataset, a dataset of 10M instructions examples harvested from Common Crawl
  • Fineweb now top 5

‍

That's all for this week! Head to our Discord to let us know what you think of this week's newsletter and what you'd like to see included in future editions.

‍