All tags
Topic: "optimizer"
not much happened this weekend
jamba-1.5 dream-machine-1.5 ideogram-v2 mistral-nemo-minitron-8b mistral-7b llama-3-8b nous-research cursor-ai gdm george-hotz agibot unitree eth-zurich disney uc-san-diego ai21-labs luma-labs ideogram nvidia mistral-ai meta-ai-fair distributed-ai optimizer inter-gpu-communication low-latency-training open-source humanoid-robots robotics physics-based-motion teleoperation multilingual-models long-context text-to-video text-to-image model-performance george-hotz adcock_brett aman
Nous Research announced DisTrO, a new optimizer that drastically reduces inter-GPU communication by 1000x to 10,000x enabling efficient training on slow networks, offering an alternative to GDM's DiLoCo. Cursor AI gained viral attention from an 8-year-old user and announced a new fundraise, with co-host Aman returning to their podcast. George Hotz launched tinybox for sale. In robotics, AGIBOT revealed 5 new humanoid robots with open-source plans, and Unitree showcased its G1 humanoid robot nearing mass production at $16,000. ETH Zurich and Disney developed an AI system for physics-based robot motion generation from text or images. UC San Diego released ACE, an open-source teleoperation system for controlling multiple robots. AI21 Labs unveiled Jamba 1.5, a multilingual model with 256k context length and permissive licensing. Luma Labs released Dream Machine 1.5 for improved text-to-video generation. Ideogram launched v2 of its text-to-image model with near-perfect text generation. Nvidia and Mistral released Mistral-NeMo-Minitron 8B, a small model outperforming Mistral-7B and llama-3-8b on the Open LLM leaderboard.
AdamW -> AaronD?
claude-3-opus llama-3 llama-3-300m bert-large stable-diffusion-1.5 wdxl openai hugging-face optimizer machine-learning-benchmarks vision time-series-forecasting image-generation prompt-injection policy-enforcement aaron-defazio
Aaron Defazio is gaining attention for proposing a potential tuning-free replacement of the long-standing Adam optimizer, showing promising experimental results across classic machine learning benchmarks like ImageNet ResNet-50 and CIFAR-10/100. On Reddit, Claude 3 Opus has surpassed all OpenAI models on the LMSys leaderboard, while a user pretrained a LLaMA-based 300M model outperforming bert-large on language modeling tasks with a modest budget. The new MambaMixer architecture demonstrates promising results in vision and time series forecasting. In image generation, Stable Diffusion 1.5 with LoRAs achieves realistic outputs, and the WDXL release showcases impressive capabilities. AI applications include an AI-generated Nike spec ad and a chatbot built with OpenAI models that may resist prompt injections. OpenAI is reportedly planning a ban wave targeting policy violators and jailbreak users. "The high alpha seems to come from Aaron Defazio," highlighting his impactful work in optimizer research.