All tags
Person: "lvwerra"
not much happened today
flux-3 flux-mimic qwen-audio-3.0-tts hugging-face black-forest-labs mimicrobotics alibaba open-datasets code-datasets distillation multimodality robotics video-modeling audio-generation tts model-training model-architecture anton_lozhkov loubnabenallal1 lvwerra eliebakouch gergelyorosz schmidhuberai suhail garrytan bfl_ai hila_chefer robrombach mimicrobotics generalistai alibaba_qwen
The Stack v3 is released as the largest open code dataset with 114 TB raw data, 224M repositories, and 5T deduplicated tokens, significantly expanding data for open code models and cyber-defense. The debate on distillation continues as a key ideological fault line, with calls for stronger investment in open-weight domestic models. Black Forest Labs launched FLUX 3, a unified multimodal model covering image, video, audio, and action prediction, with robotics transfer demonstrated by FLUX-mimic for general-purpose dexterity on a single GPU. Alibaba introduced Qwen-Audio-3.0-TTS supporting 16 languages and advanced control features, claiming the top spot on the Artificial Analysis TTS leaderboard.
not much happened today
fable-5 mythos anthropic model-performance trust data-retention benchmarking agentic-ai coding policy darioamodei natolambert martin_casado drfeifei antirez clementdelangue deanwball hlntnr _arohan_ dbahdanau gergelyorosz scaling01 dbreunig omarsar0 yacinemtb mchlhess jasonbotterill lvwerra lechmazur kimmonismus walden_yan hrishioa
Anthropic faced backlash for silently degrading AI research capabilities in its Fable/Mythos models without clear disclosure, raising concerns about trust, reproducibility, and enterprise data retention policies. Despite controversy, Fable 5 demonstrated strong benchmark performance, leading in agentic and coding tasks with high scores on Agent Arena, SimpleBench, CADGenBench, and PACT. Dario Amodei published a policy advocating stronger frontier AI oversight amid these tensions.
not much happened today
raev2 gated-deltanet-2 kda mamba-3 dclm nvidia openai nous-research representation-learning tokenization linear-attention long-context mechanistic-interpretability math data-filtering agent-infrastructure language-modeling commonsense-reasoning 1jaskiratsingh recatm sainingxie ahatamiz1 rasbt nousresearch tatsu_hashimoto goodfireai markchen90 wtgowers memecrashes cloneofsimo lvwerra
RAEv2 advances representation-first tokenization with >10x faster convergence and improved generation, tested on text-to-image and world models. NVIDIA's Gated DeltaNet-2 innovates linear attention with channel-wise gates, outperforming KDA and Mamba-3 at 1.3B parameters on language modeling and reasoning tasks. Studies on subword tokenization reveal only some benefits at scale, while data filtering research suggests that with enough compute, no filtering may be optimal at around 1e30 FLOPs. Mechanistic interpretability updates propose clustering features by joint firing patterns for better geometry understanding. OpenAI's AI-assisted breakthrough on an ErdΕs unit-distance math problem sparks debate on AI's role in mathematical research. Harnesses remain key for capability improvements in agent infrastructure.
not much happened today
claude-code composer-2 cursor openai anthropic langchain cognition reinforcement-learning developer-tooling agent-systems agent-runtimes security credential-management multi-agent-systems model-training benchmarking software-engineering enterprise-ai kimmonismus mntruell theo ellev3n11 amanrsanger charliermarsh gdb yuchenj_uw neilhtennek simonw yuvalinthedeep lvwerra hrishioa
Cursor launched Composer 2, a frontier-class coding model with major cost reductions and strong benchmark scores like 61.3 on CursorBench and 73.7 on SWE-bench Multilingual. The model was improved via a first continued pretraining run feeding into reinforcement learning, trained across 3β4 clusters worldwide by a ~40-person team. OpenAI acquired Astral, the team behind Python tools uv, ruff, and ty, strengthening its developer platform. Anthropic expanded Claude Code with messaging app channels for persistent developer workflows. The focus in AI agents is shifting from single agents to managed fleets and runtimes, with LangChain launching LangSmith Fleet for enterprise agent management emphasizing agent identity, credential management, and auditability. Other launches include Cognition's teams of Devins, AgentUI by lvwerra, and discussions on agent runtimes with features like checkpointing and rollback. Security and permissions are emerging as critical constraints in agent system design.
The Ultra-Scale Playbook: Training LLMs on GPU Clusters
deepseek-native-sparse-attention r1-1776 paligemma-2-mix muse baichuan-m1-14b stripedhyena-2 huggingface deepseek perplexity-ai google-deepmind microsoft baichuan stripedhyena gpu-training scaling multimodality vision model-training foundation-models medical-llm genome-modeling robotic-manipulation interactive-content eliebakouch nouamanetazi lvwerra thom-wolf proftomyeh alex-wang aravsrinivas _akhaliq _philschmid mervenoyann reach_vb arankomatsuzaki maximelabonne
Huggingface released "The Ultra-Scale Playbook: Training LLMs on GPU Clusters," an interactive blogpost based on 4000 scaling experiments on up to 512 GPUs, providing detailed insights into modern GPU training strategies. DeepSeek introduced the Native Sparse Attention (NSA) model, gaining significant community attention, while Perplexity AI launched R1-1776, an uncensored and unbiased version of DeepSeek's R1 model. Google DeepMind unveiled PaliGemma 2 Mix, a multi-task vision-language model available in 3B, 10B, and 28B sizes. Microsoft introduced Muse, a generative AI model trained on the game Bleeding Edge, and presented Magma, a foundation model for multimodal AI agents excelling in UI navigation and robotic manipulation. Baichuan-M1-14B was announced as a state-of-the-art medical LLM trained on 20T tokens, and a fully open-source 40B genome modeling model using StripedHyena 2 architecture was also released. "Making your own gaming experience is coming sooner than you'd think," noted in relation to Muse.