Topic: "model-serving"

glm-5.3-flash gemini-omni-1.1-flash hugging-face pollen-robotics zhipu-ai togethercompute baseten databricks google-deepmind reinforcement-learning robotics open-source simulation quantization model-serving multimodality video-generation model-efficiency local-deployment clementdelangue thom_wolf yacinemtb gneubig theo unslothai danielhanchen zainhas yuchenj_uw

Microduck, a 25 cm open-source biped robot from Pollen Robotics and Hugging Face, priced at $399 and shipping before Christmas, features 15 actuators and a rich sensor suite including camera, LiDAR, NFC, Bluetooth, and Wi-Fi. It supports reinforcement-learning-based customization with an open simulator enabling transfer from simulation to real hardware, attracting strong community interest and rapid sales. The mystery model Ox Alpha was revealed as Z.ai / Zhipu's GLM-5.3-Flash, a 320B parameter model with 18B active parameters, 1M context window, and hybrid attention, notable for efficient local deployment with 3-bit and 4-bit quantization enabling practical use on consumer hardware. It demonstrates strong price/performance metrics, rivaling other models on benchmarks. Google released Gemini Omni 1.1 Flash, advancing the video generation race with multimodal capabilities.

Aug 04

not much happened today

qwen-3.8-max qwen-image-3.0-pro alpamayo-2-super shieldstral pokee-isaac-28b maple-preview deepseek-v4-flash alibaba nvidia mistral-ai pokee-ai deepgrove-ai nous-research clinepass vllm_project togethercompute cognition cursor_ai deepseek ollama epoch-ai-research multimodality vision long-context model-quantization model-efficiency inference routing model-serving moe training-systems open-source cost-reduction jensenhuang skalskip92 arena thsottiaux kimmonismus andrewcurran_ tomas_hk

Alibaba launched Qwen3.8-Max, enhancing multimodal capabilities and agent ecosystem integration. NVIDIA introduced Alpamayo 2 Super for autonomous vehicle reasoning, while Mistral AI released Shieldstral, a 3B parameter open-weights safety model for on-device moderation. Pokee AI unveiled Pokee-Isaac 28B with a 10M-token context and single-GPU deployability, and DeepGrove AI presented Maple-Preview, an open-source 20B ternary-weight reasoning model optimized for Mac Mini M4. Pricing shifts, notably with Luna and DeepSeek-V4-Flash, are influencing product design and serving economics. Routing innovations like Not Diamond Code and Devin Fusion are reducing costs significantly without quality loss. Infrastructure advances include Cursor AI's open-sourced MoK megakernel for MoE training.

Jun 19

not much happened today

glm-5.2 opus-4.8 gpt-5.5 nous-research hugging-face cloudflare open-weight-models coding agent-engineering agent-fan-out loop-engineering model-serving infrastructure software-engineering model-evaluation open-agent-stack session-compression patrick_toulme thomas_wolf andrew_ng meryem_arik banteg graham_neubig harrison_chase jared_from_cognition omar_sanseviero teknium

GLM-5.2 emerges as a leading open-weight coding model rivaling Opus 4.8 and GPT-5.5 in software engineering tasks, emphasizing the strategic importance of open models for provider competition, on-prem deployment, and fine-tuning rights. Experts like Patrick Toulme and Thomas Wolf highlight its frontier capabilities and structural impact on the AI ecosystem. The usability of GLM-5.2 heavily depends on serving infrastructure and agent harnesses, with tools like sglang cookbooks and deepagents code enhancing evaluation and deployment. In agent engineering, the focus shifts to orchestration patterns such as agent fan-out and loop engineering, with Hermes Agent v0.17.0 advancing as a robust open agent stack supported by community-driven deployments. Additionally, Cloudflare is becoming a significant player in agent infrastructure.

Jun 01

not much happened today

cosmos-3 nemotron-3-ultra minimax-m3 nvidia runway novita vercel cloudflare openclaude flowith omnimodal-models mixture-of-experts autoregressive-models diffusion-models structured-prompts fine-tuning open-weight-models multimodality agent-models benchmarking model-serving context-windows token-efficiency kimmonismus clementdelangue artificialanalysis scaling01 ctnzr caspar_br eliebakouch pbdtokenrouter rauchg gitlawb notjazii lostinlatencyx zhihufrontier

NVIDIA led open-source AI model releases with Cosmos 3, a comprehensive omnimodal world model unifying language, image, video, audio, and action using a Mixture-of-Transformers design, and Nemotron 3 Ultra, a 550B parameter open-weight model noted for high serving speed and strong evaluation performance. The Cosmos Coalition was launched to foster an open ecosystem for physical AI world models. Meanwhile, MiniMax M3 debuted as a multimodal agent/coding model with 1M context and strong benchmark scores, gaining rapid ecosystem support from vendors like Novita and Vercel AI Gateway. However, MiniMax M3 showed some inefficiencies such as high token consumption and verbose self-check loops. These developments highlight advances in open physical AI, multimodality, and agent models with significant community and infrastructure engagement.

May 15

not much happened today

openai-5.4 openai-5.5 cerebras openai inference model-serving compute-scarcity model-routing hardware-architecture trillion-parameter-models ishanit5 dee_bosa apoorv03 bob_komin

Cerebras made headlines with its IPO, marking a significant milestone for the company known for its contrarian hardware approach. The Cerebras CFO Bob Komin emphasized the company's capability to serve trillion-parameter models, including internal OpenAI 5.4 and 5.5 models, pushing back against the notion that Cerebras only supports small models. Investor Ishan N. Taneja praised Cerebras for its persistence and execution, calling their chip a "banger." The IPO is seen as a validation of Cerebras's long-term strategy in inference infrastructure, highlighting themes like compute scarcity, inference demand, and model routing.

Mar 11

not much happened today

nemotron-3-super gpt-oss-120b qwen3.5-122b-a10b nvidia perplexity replit base44 vllm llama.cpp ollama togethercompute baseten wandb langchain unsloth model-architecture model-optimization inference-speed kv-cache multi-token-prediction agent-infrastructure orchestration persistent-agents model-serving product-launches karpathy ctnzr bnjmn_marie artificialanlys

NVIDIA’s Nemotron 3 Super is a 120B parameter / ~12B active open model featuring a hybrid Mamba-Transformer / SSM Latent MoE architecture and 1M context window, delivering up to 2.2x faster inference than GPT-OSS-120B in FP4 with strong throughput gains. It supports agentic workloads and is unusually open with weights, data, and infrastructure details released. The model scored 36 on the AA Intelligence Index, outperforming GPT-OSS-120B but behind Qwen3.5-122B-A10B. Community and infrastructure support from projects like vLLM, llama.cpp, Ollama, Together, Baseten, W&B Inference, LangChain, and Unsloth GGUFs was immediate. Key technical innovations include native multi-token prediction (MTP) and a significant KV-cache efficiency advantage. On the product side, a shift towards persistent agent runtimes and orchestration layers is highlighted, with Andrej Karpathy advocating for a "bigger IDE" concept where agents replace files as the unit of work, enabling legible, forkable agentic organizations with real-time control. New launches fitting this vision include Perplexity’s Personal Computer, an always-on local/cloud hybrid running on Mac mini, and Computer for Enterprise orchestrating 20 specialized models and 400+ apps. Replit Agent 4 offers a collaborative, canvas-like workflow with parallel agents, while Base44 Superagents provide integrated solutions for nontechnical users. The engineering focus is increasingly on the orchestration harness rather than just the model.