All tags
Person: "jubbaonjeans"
not much happened today
dflash nemo-automodel claude openai broadcom qualcomm modular nvidia skypilot modal anthropic hugging-face hardware inference performance-optimization model-training agent-ux security capability-based-security open-source fine-tuning infrastructure model-optimization gdb kimmonismus scaling01 clattner_llvm karpathy gallabytes dabit3 kentonvarda random_walker jubbaonjeans victormustar
OpenAI announced Jalapeño, its first custom AI chip for LLM inference, built with Broadcom, aiming to control more of the AI stack and improve compute economics with a fast 9-month design cycle. Community analysis suggests Jalapeño features 216GB HBM3E, ~7.1–7.4 TB/s bandwidth, and ~10 PFLOPS FP4 performance, signaling hyperscaler-style inference silicon as a new standard. Meanwhile, Qualcomm is acquiring Modular, with Mojo open-sourcing on track, indicating rising competition in vertically integrated inference stacks beyond NVIDIA/CUDA. On infrastructure, NVIDIA's NeMo AutoModel boosts training throughput for MoE models by 3.4–3.7x, and startups like SkyPilot and Modal advance unified and open-source inference solutions. Custom training of DFLASH models yields 30–50% decode gains. In UX, Anthropic's Slack-native Claude agent shifts agent interaction from tools to coworkers, raising new security and cost concerns around identity, permissions, and lock-in, with debates on capability-based security and attribution. Hugging Face responded with its self-hosted Slack coding agent Moon Bot.