Weekly Digest: May 2, 2024

‍

Welcome to our Weekly Digest, where Alex Duffy summarizes the latest updates you should know across the AI ecosystem, ComfyUI, and Salt AI. This week, the US government eyes new AI regulations, GitHub announces Copilot Workspaces, two new healthcare-focused models from Meta and Google, a new AI world builder for agents, and a new model that can clone any voice and speak in many languages.

‍

Check out the video above, and follow along with Alex's notes and links below:

‍

Big News

• US government seemed to eye new AI regulation
  • Proposed in California met with angry outcry
  • Department of Homeland Security announced the establishment of the Artificial Intelligence Safety and Security Board
  • US @NIST publishes 1st draft of its "AI Risk Management Framework: Generative AI Profile

• GitHub announced waitlist for technical preview of Copilot Workspace‍
  • Quick summary on Twitter‍
  • It takes a while to build tools that work, that’s why we’re building Salt, but once they start to work, they’ll just keep getting better and can deliver real value
  • Next we need something like this for version control! Somewhere Salt solves problems but one others are looking at too

• Two new medical-focused models from Meta and Google‍
  • Meta announces Meditron a Llama LLM suite for low-resource medical settings‍
  • Google announces Med-Gemini, a family of Gemini models fine-tuned for medical tasks‍
  • MedGemini achieves SOTA on most of the tasks in these benchmarks and is preferred by clinician raters for the more challenging longform text tasks!
  • More and more evidence that LLM’s could really help democratize access to at least some healthcare - though of course not without risks

• MIT CSAIL and @myshell_ai researchers introduce OpenVoice V2 a text-to-speech model that can clone any voice and speak in many languages
  • Fully open source and allows for commercial use

‍

New in LLMs

• Llama3 continues to make an impact
  • Researchers make finetunes and push the context limit to 32K extremely effectively and to 1M+ with various rates of success
  • Long context lengths are helpful for models  because you can give them hundreds of examples you can do a LOT with prompting - few shot prompting is super powerfu

• AI-Town seeing a resurgence in popularity thanks to Llama3 and Pinokio‍
  • Doubled down with a town world editor
  • Solving dependencies is always an issue and this solves it for specific user group
  • Simulations with agents in a game are unassuming ways to push really complex functionality, especially if they go from just chatting to playing Among Us, for example

• HuggingFace released one of the best datasets for training LLMs ever
  • Data is very important and LLMs helping to make it is huge‍
  • The 45TB file was so popular it took down Huggingface which had intermittent outages for the next three days before becoming one of the top 10 most liked datasets on the platform

• StarCoder2-Instruct - Fully Transparent and Permissive Self-Alignment for Code Generation
  • The team uses the model to improve the past model, another data-point highlighting the promise of synthetic data
  • Comparable to other larger models

• Benchmarks
  • Start of a text to video benchmark‍
  • Function calling‍
  • Speed‍
    • Justine Tunney continues to push the envelope

‍

New in Generative AI

• The barrier to entry to VFX for video game and movie production is lowering and tools like ComfyUI and Salt are at the core of that
  • BlenderAlchemy - Editing 3D Graphics with Vision-Language Models - Stanford
  • Sewing and Generating Clothes from Text - Shanghai Tech Univ., UPenn
  • A creator using ComfyUI made visuals for a Coachella set

‍

New In ComfyUI

• Transitions with control
  • Matteo released new nodes that can do awesome animations from people to environments
  • Steerable Motion is also a similar node from Banodoco that is climbing in stars and improving in quality from 1.4 to 1.5
• Comfy in Blender!

‍

Bonus News

• LLMs might get better by planning ahead‍
• 31 years ago CERN released the source code for the World Wide Web for anyone to use

‍

That's all for this week! Head to our Discord to let us know what you think of this week's newsletter and what you'd like to see included in future editions.

‍