All tags
Topic: "fp8"
OpenAI Titan XPU: 10GW of self-designed chips with Broadcom
llama-3-70b openai nvidia amd broadcom inferencemax asic inference compute-infrastructure chip-design fp8 reinforcement-learning ambient-agents custom-accelerators energy-consumption podcast gdb
OpenAI is finalizing a custom ASIC chip design to deploy 10GW of inference compute, complementing existing deals with NVIDIA (10GW) and AMD (6GW). This marks a significant scale-up from OpenAI's current 2GW compute, aiming for a roadmap of 250GW total, which is half the energy consumption of the US. Greg from OpenAI highlights the shift of ChatGPT from interactive use to always-on ambient agents requiring massive compute, emphasizing the challenge of building chips for billions of users. The in-house ASIC effort was driven by the need for tailored designs after limited success influencing external chip startups. Broadcom's stock surged 10% on the news. Additionally, InferenceMAX reports improved ROCm stability and nuanced performance comparisons between AMD MI300X and NVIDIA H100/H200 on llama-3-70b FP8 workloads, with RL training infrastructure updates noted.
not much happened today
gpt-4.1 o3 o4-mini grok-3 grok-3-mini o1 tpuv7 gb200 openai x-ai google nvidia samsung memory model-release hardware-accelerators fp8 hbm inference ai-conferences agent-collaboration robotics model-comparison performance power-consumption sama
OpenAI teased a Memory update in ChatGPT with limited technical details. Evidence suggests upcoming releases of o3 and o4-mini models, alongside a press leak about GPT-4.1. X.ai launched the Grok 3 and Grok 3 mini APIs, confirmed as o1 level models. Discussions compared Google's TPUv7 with Nvidia's GB200, highlighting TPUv7's specs like 4,614 TFLOP/s FP8 performance, 192 GB HBM, and 1.2 Tbps ICI bandwidth. TPUv7 may have pivoted from training to inference chip use. Key AI events include Google Cloud Next 2025 and Samsung's Gemini-powered Ballie robot. The community is invited to participate in the AI Engineer World's Fair 2025 and the 2025 State of AI Engineering survey.
not much happened today
deepseek-r1 gemma-3 gemma-3-27b openai nvidia deepseek hugging-face fp8 model-efficiency hardware-requirements quantization benchmarking model-deployment open-source sam-altman
DeepSeek R1 demonstrates significant efficiency using FP8 precision, outperforming Gemma 3 27B in benchmarks with a Chatbot Arena Elo Score of 1363 vs. 1338, requiring substantial hardware like 32 H100 GPUs and 2,560GB VRAM. OpenAI labels DeepSeek as "state-controlled" and calls for bans on "PRC-produced" models, sparking community backlash accusing OpenAI and Sam Altman of anti-competitive behavior. Discussions emphasize DeepSeek's openness and affordability compared to OpenAI, with users highlighting its local and Hugging Face deployment options. Meanwhile, Gemma 3 receives mixed community feedback on creativity and worldbuilding.
not much happened today
claude-3.7-sonnet claude-3.7 deepseek-r1 o3-mini deepseek-v3 gemini-2.0-pro gpt-4o qwen2.5-coder-32b-instruct anthropic perplexity-ai amazon google-cloud deepseek_ai coding reasoning model-benchmarking agentic-workflows context-window model-performance open-source moe model-training communication-libraries fp8 nvlink rdma cli-tools skirano omarsar0 reach_vb artificialanlys terryyuezhuo _akhaliq _philschmid catherineols goodside danielhanchen
Claude 3.7 Sonnet demonstrates exceptional coding and reasoning capabilities, outperforming models like DeepSeek R1, O3-mini, and GPT-4o on benchmarks such as SciCode and LiveCodeBench. It is available on platforms including Perplexity Pro, Anthropic, Amazon Bedrock, and Google Cloud, with pricing at $3/$15 per million tokens. Key features include a 64k token thinking mode, 200k context window, and the CLI-based coding assistant Claude Code. Meanwhile, DeepSeek released DeepEP, an open-source communication library optimized for MoE model training and inference with support for NVLink, RDMA, and FP8. These updates highlight advancements in coding AI and efficient model training infrastructure.