All tags
Topic: "model-parallelism"
not much happened today
kernelllm-8b gpt-4o deepseek-v3 mistral-medium-3 qwen3 blip3-o xgen-small anisora stable-audio-open-small alphaevolve meta-ai-fair mistral-ai qwen deepseek salesforce bilibili stability-ai google benchmarking model-performance multilinguality hardware-optimization multimodality image-generation video-generation text-to-audio model-parallelism chain-of-thought instruction-following reasoning mitigation-strategies reach_vb lmarena_ai theadimeline adcock_brett jxmnop dair_ai omarsar0
Meta released KernelLLM 8B, outperforming GPT-4o and DeepSeek V3 on KernelBench-Triton Level 1. Mistral Medium 3 debuted strongly in multiple benchmarks. Qwen3 models introduced a unified framework with multilingual support. DeepSeek-V3 features hardware-aware co-design. BLIP3-o family released for multimodal tasks using diffusion transformers. Salesforce launched xGen-Small models excelling in long-context and math benchmarks. Bilibili released AniSORA for anime video generation. Stability AI open-sourced Stable Audio Open Small optimized for Arm devices. Google’s AlphaEvolve coding agent improved Strassen's algorithm for the first time since 1969. Research shows chain-of-thought reasoning can harm instruction-following ability, with mitigation strategies like classifier-selective reasoning being most effective, but reasoning techniques show high variance and limited generalization. "Chain-of-thought (CoT) reasoning can harm a model’s ability to follow instructions" and "Mitigation strategies such as few-shot in-context learning, self-reflection, self-selective reasoning, and classifier-selective reasoning can counteract reasoning-induced failures".
1/4/2024: Jeff Bezos backs Perplexity's $520m Series B.
wizardcoder-33b-v1.1 mobilellama-1.4b-base shearedllama tinyllama mixtral-8x7b perplexity anthropic google nous-research mistral-ai hugging-face document-recall rnn-memory synthetic-data benchmarking multi-gpu-support context-length model-architecture sliding-window-attention model-parallelism gpu-optimization jeff-bezos
Perplexity announced their Series B funding round with notable investor Jeff Bezos, who previously invested in Google 25 years ago. Anthropic is raising $750 million, projecting at least $850 million in annualized revenue next year and implementing "brutal" changes to their Terms of Service. Discussions in Nous Research AI Discord cover topics such as document recall limits from gigabytes of data, RNN memory and compute trade-offs, synthetic datasets, and benchmarking of models like WizardCoder-33B-V1.1, MobileLLaMA-1.4B-Base, ShearedLLaMA, and TinyLLaMA. Other highlights include UnsLOTH optimizations for multi-GPU systems, AI rap voice models, context-extending code, and architectural innovations like applying Detectron/ViT backbones to LLMs, sliding window attention in Mistral, and parallelizing Mixtral 8x7b with FSDP and HF Accelerate.