Reasoning too cheap to meter.

AI News for 6/9/2025-6/10/2025. We checked 9 subreddits, 449 Twitters and 29 Discords (218 channels, and 9374 messages) for you. Estimated reading time saved (at 200wpm): 715 minutes. Our new website is now up with full metadata search and beautiful vibe coded presentation of all past issues. See https://news.smol.ai/ for the full news breakdowns and give us feedback on @smol_ai!

Every 3-4 months we get a big leg down in the cost of the frontier LLM (March 2024, Aug 2024, Jan 2025), and today we got confirmation of the 80% price cut for o3, making it nominally the same cost as GPT 4.1, the non-reasoning model. (you can be forgiven for suspecting the price cut is due to distillation, but this is categorically denied). Of course, the real cost come in the reasoning token efficiency, and fortunately o3 is notably better than Gemini and Deepseek in that department:

Alongside of this o3 price cut, o3 pro was released, which if the o1/o1-pro relationship holds is more or less 10 o3's in a trenchcoat (and is priced that way).

This news is released conveniently on the same day as Mistral's Magistral reasoning model - a 24B open source version and a Medium closed version - that would've otherwise taken today's headline. We're REALLY glad though that Mistral is continuing to release good open source models but unfortunately the o3 price cut is more likely to be the relevant story for the majority of AI engineers today.

AI Twitter Recap

Large Language Models (LLMs) & AI Model Releases

• OpenAI's o3 and o3-pro model updates and pricing changes: OpenAI announced significant price drops for its o3 input tokens, reducing them by 80% to $2.00 per million tokens, making o3 cheaper than GPT-4o and competing with Anthropic's Claude 4 Sonnet and Google's Gemini 2.5 Pro in pricing, leading some to declare "price wars" @scaling01, @scaling01, @polynoamial, @nrehiew_/. They also released o3-pro, a more intelligent and reliable version of o3 designed to "think longer," priced at $20 Input and $80 Output per million tokens @scaling01, @kevinweil, @polynoamial. Early testers reported o3-pro to be "much stronger" than o3 @gdb and "extremely cheaper, faster, and way more precise than o1-pro" for coding and reasoning tasks @flavioAd. However, initial ARC-AGI-1 and ARC-AGI-2 benchmark results showed o3-pro (high) not outperforming o3-high despite being 8 to 9 times more expensive @scaling01, @scaling01. OpenAI also experienced "elevated error rates and latency" across ChatGPT and the API during these releases, which were later fixed @OpenAI. The API was restored to 100% functionality and rate limits for o3 for Plus users were doubled @stevenheidel, @kevinweil. Perplexity AI quickly integrated o3 for its Pro users on both web and mobile apps @perplexity_ai, @AravSrinivas.
• Mistral AI's Magistral reasoning models: Mistral AI released Magistral-Small and Magistral-Medium, their first reasoning models, designed for "domain-specific, transparent, and multilingual reasoning" @MistralAI. Magistral Small is an open-source 24B parameter model based on Mistral Small 3.1, capable of running on a single RTX 4090 with 128K context (40k effective) @scaling01, @reach_vb. It supports MLX, llama.cpp, transformers, and vLLM @reach_vb. The underlying methodology, GRPO, involves modifications like removing KL Divergence and normalizing by total length @danielhanchen. Initial evaluations showed Magistral Small being outperformed by Qwen3-32B and Qwen3-30B-A3B @scaling01, though some noted its impressive speed on platforms like Le Chat @qtnx_.
• Other notable LLM/AI Model updates and releases:
  • MiniCPM4, an ultra-efficient family of LLMs explicitly designed for end devices, was released on Hugging Face @OpenBMB.
  • Google DeepMind showcased Veo 3 Fast for Gemini App and Flow, touted as 2x faster with improved visual quality and consistency @demishassabis, @demishassabis.
  • Vui, a new open-source dialogue generation model with 100M parameters trained on 40k hours of audio, was released as an alternative to NotebookLM @_akhaliq, @kylebrussell.
  • Gemma 3n, a desktop-optimized model (2B and 4B), is now available for Mac/Windows/Linux via the LiteRT-LM library @demishassabis.
  • Krea AI introduced its first image model, Krea 1, promising "superior aesthetic control and image quality" @_akhaliq.
  • MeiGen-MultiTalk released its code and checkpoints for an audio model @TomLikesRobots.
  • DatologyAI released two state-of-the-art CLIP ViT-B/32 variants, optimized for classification and retrieval, achieved through data curation alone @code_star, @sarahcat21.

AI Infrastructure & Tools

• Agentic Frameworks and Development:
  • LangGraph rolled out updates including node/task caching and built-in provider tools for more efficient and configurable workflows @LangChainAI. It has been successfully used by Uber to build AI developer agents that generate "thousands of daily code fixes" saving 21,000+ hours for 5,000 developers @LangChainAI. Box's CTO Ben Kus also detailed how they rebuilt their AI with agentic architecture using LangGraph to power their AI agent workforce @LangChainAI.
  • DSPy is highlighted for its foresight in treating prompts as "compiled outputs" rather than "ephemeral artifacts," with the expectation that many platforms will embody "The Spirit of DSPy" by 2026 @lateinteraction.
  • The Agents & MCP Hackathon saw over 400 submissions leveraging tools like Claude API, Gradio, and Modal @_akhaliq, with LangChain's GPT Researcher now integrating Model Context Protocol (MCP) adapters for intelligent tool selection @LangChainAI.
• Compute & Optimization:
  • SkyPilot is now featured in AWS SageMaker HyperPod tutorials, combining HyperPod's availability/node recovery with SkyPilot's ease of AI execution @skypilot_org.
  • vLLM announced support for Magistral with vLLM 0.9.1rc1 @vllm_project and showcased a new AMD MI355X system at Berkeley Sky for open-source support @vllm_project.
  • Modular demonstrated "industry leading performance" on AMD MI300/325 (up to 50% faster than vLLM 0.9) and previewed Blackwell support for compute portability @clattner_llvm, @clattner_llvm. They are also partnering with NVIDIA for a GPU Prize Pool at their Hack Weekend @clattner_llvm.
• Data & Evaluation:
  • The importance of data curation for model improvements is emphasized, with DatologyAI demonstrating state-of-the-art CLIP model performance through data curation alone @sarahcat21, @code_star.
  • AI Evals for Engineers & PMs course by @sh_reya and @HamelHusain is proving "tremendously helpful" for data scientists and engineers in building and debugging AI applications, including multi-turn conversation traces @HamelHusain, @HamelHusain, @HamelHusain.
  • A new large-scale dataset, MIRIAD, with 5,821,948 medical question-answer pairs, was released to improve RAG in medicine @lateinteraction.
  • NVIDIA released Nemotron-Personas, an open-source dataset of 100k synthetically-generated personas, and PhysicalAI-Autonomous-Vehicle-Cosmos-Drive-Dreams, a 3TB synthetic driving dataset on Hugging Face @_akhaliq, @_akhaliq.
• Editor & IDE Integrations:
  • Claude Code now integrates more deeply with VS Code and JetBrains IDEs, providing access to open files and LSP diagnostics @_sholtodouglas.
  • Cursor AI integrated the o3 price drop, making o3 a viable "daily driver" for users @cursor_ai, and mentioned an interview with Anthropic about Cursor @AnthropicAI.
  • Zed editor has improved its Git UI and agentic editor sidebar, offering faster performance compared to 2-5ms latency of other editors @vikhyatk.

AI Applications & Use Cases

• AI Agents in Enterprise & Workflows:
  • LlamaIndex is enabling the building of "practical document agents in production" for use cases like form filling @jerryjliu0. They demonstrated how to turn any LlamaIndex agent into an MCP server for "custom FidelityFundEngine" @jerryjliu0 and how to build Knowledge Agents to automate workflows at Databricks Data + AI Summit @jerryjliu0.
  • Jerry Liu also explained how LlamaCloud can be used to set up parsing and extraction agents for company filings and integrate with LlamaIndex workflows to generate reports, bridging the gap between AI and business value @jerryjliu0.
  • Scouts launched as "always-on AI agents that monitor the web" for specific user interests @krandiash.
  • Weaviate Agents are highlighted for enabling "autonomous AI-driven workflows" @bobvanluijt.
  • The concept of AI agents forcing a rethinking of "professional, commercial, or personal" interactions where time-wasting for information was once beneficial was put forth as a "VERY GOOD THING" @francoisfleuret.
• Creative AI & Content Generation:
  • Kling AI introduced its Pengfei Wan, Head of Kling Video Generation Models, presenting "An Introduction to Kling and Our Research towards More Powerful Video Generation Models" at CVPR 2025 @Kling_ai. Kling is also noted for its ability to automatically create "video-matched audio and ambient sounds" @Kling_ai.
  • Google's Veo 3 is achieving "consistent characters + mood" in video generation, which was previously a challenge in text-to-image prompting @demishassabis.
  • Higgsfield AI announced "hyper-real vocals from Suno Music" for its upcoming "Rap Icon era" of AI superstars @_akhaliq.
  • Runway ML is developing "new products that bring a completely new experience," aiming to make creation "as natural and easy as possible" and "feel like your creative partner" @c_valenzuelab.
• Other Applications:
  • Sakana AI partnered with Hokkoku Bank in Japan to develop "bank-specific AI-powered tools" and contribute to regional issues, following a comprehensive partnership with Mitsubishi UFJ Bank @SakanaAILabs, @hardmaru.
  • Google DeepMind's CEO Demis Hassabis discussed AI's potential in mathematics at a workshop at the IAS @GoogleDeepMind.
  • Perplexity AI updated its Discover articles to default to "Summary" mode for lighter reading, with a toggle for "Report" mode depth @AravSrinivas.
  • You.com partnered with TIME to offer free Pro subscriptions to their digital subscribers @RichardSocher.

AI Industry & Market Dynamics

• Apple's WWDC Announcements and AI Strategy:
  • Apple's WWDC announcements, particularly concerning Apple Intelligence and the new iOS UI (Liquid Glass), generated significant discussion. Critics dubbed the new design a "Windows Vista moment" @zacharynado and "soulless UI update" @scaling01, comparing it to a "junior designer discovered the gradient tool" @dzhng. Some expressed disappointment, finding no "magic or delight" compared to past Apple products like the iPod mini @raizamrtn.
  • The "Liquid Glass" design is also criticized for potential usability issues, with John Carmack noting that "translucent UI is usually a bad idea" and that "Windows and Mac have both been down this road before" @ID_AA_Carmack.
  • Despite the critique, some suggested potential, arguing that monochrome UI could lead to "less addictive habits" @zachtratar.
  • Apple introduced MLX, their machine learning framework, with new webpage and sessions for Python and Swift developers at WWDC 2025 @ClementDelangue, @awnihannun.
  • Safari 26 will gain WebGPU support @jeremyphoward, and macOS 26 will get "native support for Linux containers" @jeremyphoward.
• AI Talent & Investment:
  • Meta is reportedly offering $2M+/yr for AI talent but is still losing them to OpenAI and Anthropic @slashML. There are questions about Meta's strategy with Scale AI and the new "Superintelligence Lab" @Yuchenj_UW.
  • Discussion continues on the UK's AI and Biosciences industry, with concerns that US firms' "garden leave" policies hinder local talent flow and benefit US acquirers, despite government announcements like OpenBind @NandoDF, @NandoDF.
  • ZyphraAI is expanding its team in Palo Alto with roles open across multimodal foundation models and RL @QuentinAnthon15.
• AI Ecosystem & Growth:
  • Stripe's macro figures (e.g., payment volume) appear to be influenced by AI, suggesting growing adoption @BorisMPower.
  • The AI Engineer World's Fair highlighted that "the startup playbook is being rewritten in real-time" and emphasized publishing takeaways from events @swyx, @swyx.
  • The Common Crawl Foundation, IBM, the AI Alliance, and BrightQuery are hosting an "UN Conference" at IBM's NYC HQ on June 20 to discuss AI, policy, and responsible data @CommonCrawl.
  • DeepLearning.AI released a new course on Data Storytelling as part of its Data Analytics Professional Certificate, stressing its importance for business performance and revenue generation @DeepLearningAI.

AI Research & Philosophy

• AGI and AI Capabilities:
  • Finbarr Timbers proposed that RL + GPT-style LLMs could "lead to AGI" @finbarrtimbers.
  • Ilya Sutskever's U of T honorary degree speech was described as "the wisest words you could hear," with some interpreting his insights as indicating an impending, rapid acceleration of code-LLMs that could lead to mass job displacement, even for AI builders @NandoDF, @sbmaruf.
  • Sam Altman stated that "Intelligence too cheap to meter is well within grasp" and that "We do not know how far beyond human-level intelligence we can go, but we are about to find out" @sama, @scaling01.
  • François Chollet highlighted metacognitive sensitivity as crucial for learning rate, enabling introspection and critique of mental models @fchollet.
• Architectures & Optimizations:
  • The paper "Cartridges" explores scaling cache-time compute as an alternative to ICL for scenarios where many user messages reference the same large text corpus, aiming for 38.6x less memory @simran_s_arora, @simran_s_arora. This and similar work suggests KV caches have significant room for compression @gallabytes.
  • Research into Hierarchical Masked Auto-Regressive Image Generation (HMAR) focuses on hardware-efficient reformulation for autoregressive image generation to leverage tensor cores @realDanFu.
  • Reinforcement Pre-Training (RPT) reframes next-token prediction as a reasoning task using RLVR @kylebrussell.
  • Grafting is introduced as a new method to distill pretrained diffusion transformers into new architectures, enabling swap attention for new primitives at 2% pretraining cost @realDanFu.
• Societal Impact & Ethics:
  • The "AI as Normal Technology" paper by @random_walker and @sayashk emphasizes the need for serious engagement with superintelligence and existential risk ideas, moving beyond social media mud-wrestling to productive debate and acknowledging different worldviews @random_walker.
  • Concerns were raised about AI personas, especially multimodal and real-time ones, potentially becoming addictive and "seem better than humans" @sirbayes.
  • A position paper by SEAL and Red Team at Scale AI outlined lessons learned from red teaming LLMs, focusing on what matters for model safety within broader system safety and monitoring @summeryue0.
  • The debate on AI's energy consumption highlights a sharp increase in electricity use by data centers (potentially doubling by 2030), but also AI's potential to reduce emissions by optimizing energy systems five times more than expected data center production @DeepLearningAI.

Humor, Memes & General Observations

• Reactions to Apple's WWDC and UI: Many found humor in Apple's WWDC announcements, with comments like "lmfao Apple models sound so 2010ish" @cto_junior and comparing the new UI to "Windows Vista" @skirano, and even quipping that Apple put "the last in the group chat to get the joke" in an ad @swyx. The term "Liquid Glass" for the new design also became a point of mockery @fabianstelzer.
• Observations on AI Progress and Hype: Discussions included humorous takes on the rapid pace of AI development, such as "AI hedonic treadmill," where new tools quickly make old ones feel "broken" @rishdotblog, and the observation that "research and product development move way faster than most people can keep up with" @c_valenzuelab.
• General Commentary & Satire: Jokes about the nature of intelligence ("my brain is special and conscious because it is made of meat" @vikhyatk and Terry Bisson's "They're Made Out of Meat" short story @vikhyatk), observations on the tech industry ("members of technical staff' is fitting because there's a lot of dicks in ai" @typedfemale), and self-deprecating humor about coding and AI usage ("trying to explain to my wife which ChatGPT model to use 😅" @finbarrtimbers) were prevalent.
• Non-Technical/Political: A significant portion of the tweets from @SerranoAcademy focused on international protests and political events related to Gaza, Greta Thunberg's arrests, and European Parliament members' kidnapping @SerranoAcademy, @SerranoAcademy, @SerranoAcademy. These tweets are summarized here for completeness but are outside the core technical focus of the summary.

AI Reddit Recap

/r/LocalLlama Recap

1. Mistral Magistral Reasoning Model Releases and Discussion

• mistralai/Magistral-Small-2506 (Score: 389, Comments: 118): **Magistral-Small-2506 is a 24B parameter LLM derived from Mistral Small 3.1 (2503), with enhanced reasoning via SFT from Magistral Medium traces and RL, targeting efficient local deployment (fits in RTX 4090 or 32GB RAM MacBook when quantized). It offers strong multilingual (40+ languages) capabilities, a 128k context window (optimal <40k tokens), and is licensed Apache 2.0. Benchmarks show Magistral-Small achieves** 70.68% AIME24, 62.76% AIME25, 68.18% GPQA Diamond, and 55.84% Livecodebench, slightly below Magistral-Medium. Quantized GGUF models and deployment guides are available (llama.cpp, lmstudio, ollama, unsloth), with best inference using temperature=0.7, top_p=0.95, and -jinja in llama.cpp. See Mistral's blog for further details. Commentary highlights excitement for Magistral-Small's benchmark position relative to larger models (e.g., Qwen3 32B), and notes the model's permissive Apache 2.0 license. Technical users recommend specific inference parameters, raising potential improvements in performance with increased Ollama context length. Community fine-tuning and conversion (GGUF) support via Unsloth is praised for deployment flexibility.
  • danielhanchen provides direct usage instructions for running Magistral-Small-2506 GGUFs, specifying critical inference parameters: temperature=0.7, top_p=0.95, and emphasizes the importance of using the -jinja flag in llama.cpp for proper operation. They include command-line examples for both llama.cpp and Ollama, and recommend increasing Ollama's context length to at least 8K (OLLAMA_CONTEXT_LENGTH=8192) to optimize performance. Detailed deployment and usage guidance is available in the linked documentation: https://docs.unsloth.ai/basics/magistral
  • Only-Letterhead-3411 expresses interest in benchmarking Magistral-Small-2506 against Qwen3 32B, suggesting its perceived relevance as a competitor at the 30B+ model scale. This implies community interest in comparative performance and capability testing between Magistral and other leading large models.
  • AppearanceHeavy6724 raises concerns regarding Magistral-Small-2506's general-purpose abilities, speculating that its performance may be significantly subpar for non-coding tasks. This highlights open questions about the model's domain generalization and applicability outside programming contexts.
• New open-weight reasoning model from Mistral (Score: 303, Comments: 59): Mistral has released Magistral, an open-weight reasoning model, with technical details available in their news post and official paper. Notably, a GGUF quantized version of Magistral-Small-2506 is already available on Hugging Face and collaborates smoothly with downstream tools. The 24B parameter model size shows impressive performance benchmarks, especially in reasoning, with speculation on public release or comparison with competitor models such as Qwen. Performance on Cerebras hardware for applications like Le Chat shows significant inference speedups (reportedly up to 1000 tok/s for Flash Answers mode), highlighting hardware/model synergy. Community discussion centers on the competitive performance of the 24B model, interest in comparative real-world benchmarks vs. Qwen, and future prospects for larger model releases. Users positively note the utility of fast inference modes for reasoning-centric applications, especially leveraging Cerebras hardware.
  • Discussion highlights collaborative GGUF quantization efforts with Mistral, ensuring optimized compatibility and fast deployment of the Magistral-Small-2506 model on different hardware via UnsLoTh's Hugging Face repository.
  • Technical comparisons are requested between the new Mistral reasoning models and alternatives, specifically Qwen (with interest in performance on real-world tasks), and MistralThinker-v1.1, which distills DeepSeek-style reasoning into Mistral-small architecture.
  • Users are observing impressive benchmark results for Mistral Medium, but note a lack of published comparative benchmarks for smaller variants (like Mistral Small) versus Qwen 3 32B, indicating a gap in publicly available performance data and possible benchmarking avoidance.
• Magistral — the first reasoning model by Mistral AI (Score: 114, Comments: 10): Mistral AI has announced 'Magistral', their first reasoning-focused language model, as shown in the linked preview image. One user benchmarked Magistral's summarization capability and found it competitive with Qwen-32B, but noted it exhibited infinite thinking loops on two occasions; no details are provided on model size, architecture, or training data. There is no public confirmation regarding open model weights as of this announcement. Top debate questions the availability of open weights, and a technical comment notes Magistral's summarization quality as comparable to Qwen32B but highlights specific failure modes (infinite loops), suggesting the need for further evaluation of deployment safety and robustness.
  • One user reports that while testing the model, it entered infinite thinking loops on two occasions, raising concerns about potential inference bugs or weaknesses in the control logic. However, aside from this, its summarization performance was observed to be on par with Qwen-32B, a known strong model in this domain.

2. Qwen3 0.6B Embedding Model Semantic Search Demos

• Semantic Search Demo Using Qwen3 0.6B Embedding (w/o reranker) in-browser Using transformers.js (Score: 116, Comments: 6): The post describes a semantic search demo leveraging the newly released Qwen3 0.6B embedding model for in-browser retrieval using transformers.js. The implementation uses ONNX quantized weights for the embedding model and ranks query results by basic cosine similarity, as the Qwen3 reranker model was not available in ONNX quantized form. The visualization maps up to three connections per node in a user-editable "memory bank" based on embedding similarity; the system currently scales to 20-100 entries given local inference. Source is available on GitHub with a live demo on HF Spaces. A technical follow-up inquires about the ONNX quantized model file size, indicating interest in deployment specifics and hardware requirements.
  • One commenter inquires about the size of the quantized ONNX model file being used with Qwen3 0.6B for semantic search in-browser, suggesting interest in feasibility and storage requirements for client-side deployment. This is key for applications where bandwidth and local resource constraints matter for running transformer models directly in browsers.
• Google Diffusion told me its system prompt (Score: 146, Comments: 30): A user claims to have received the full system prompt of "Gemini Diffusion," an experimental Google text diffusion language model advertised as non-autoregressive and explicitly tailored for generating code and web assets with fine-grained design constraints. The prompt details highly specific HTML/CSS/JS generation requirements (notably Tailwind CSS for web, custom CSS for games), icon handling, layout performance (e.g., CLS prevention), and strong emphasis on accurate instruction-following, modern aesthetics, and code self-containment. Prompt fidelity discussions are relevant, as the prompt includes constraints distinguishing it from autoregressive LLMs and reveals internal operational guidelines and security boundaries (e.g., no external file access, a Dec. 2023 knowledge cutoff, and strict handling of user requests). Top comments raise skepticism about the prompt's authenticity, highlighting the possibility of hallucinations in LLM output and suggesting cross-verification via repositories like https://github.com/guy915/LLM-System-Prompts. A comment provides a screenshot as possible evidence, but the absence of direct confirmation from Google is noted.
  • A commenter questions the authenticity of outputs that claim to reveal a model's system prompt, raising the issue of hallucination—where a model might fabricate plausible but incorrect information—and asks how to verify whether such text truly reflects the underlying system prompt rather than generated content.
  • Another user emphasizes the uncertainty involved by stating that we cannot be sure if the provided text is actually the system prompt or just output generated in response to a user's prompt, highlighting the challenge in reliably extracting system or meta-prompts from language models like Google's Gemini Diffusion.

3. Cutting-Edge AI Architectures: Apple Parallel-Track MoE and Meta Superintelligence Initiatives

• Apple is using a "Parallel-Track" MoE architecture in their edge models. Background information. (Score: 132, Comments: 19): Apple's 2025 foundation model stack features two main innovations: (1) an efficient ~3B parameter on-device LLM using a Mixture-of-Experts (MoE) architecture with split-layer KV cache-sharing, enabling fast inference with reduced memory and latency on Apple silicon, and (2) a novel server-side Parallel-Track MoE (PT-MoE) architecture designed to scale out with minimal synchronization (relying on parallel processing and Limited/Distributed communication). Notably, the server models are also compressed using ASTC (Adaptive Scalable Texture Compression)—a GPU texture compression standard—enabling direct hardware-level weight decoding with no extra compute overhead. The model pipeline integrates a custom ViT-based encoder with a 'Register-Window' mechanism for efficient vision-language tasks, trained on filtered web-scale and synthetic multimodal data. Full technical details are in the original Apple AI blog post. Commentary highlights the clever reuse of Apple's ASTC GPU decompression hardware for LLM weight loading, and some debate about the practical capabilities of the edge models—suggested to be basic (summarization, generic responses) versus more interactive tasks. There is strong technical interest in the split between local, private inference and hierarchical cloud fallback.
  • One user highlights Apple's use of block-based texture compression (specifically, Adaptive Scalable Texture Compression—ASTC) for compressing ML model weights, leveraging dedicated ASTC decompression hardware in Apple GPUs for efficient on-device inference without additional compute overhead. This represents an innovative repurposing of existing GPU hardware intended for graphics, now benefitting edge AI workloads.
  • A technical breakdown proposes that Apple's edge/local models are likely in the ~3B parameter range (comparable to models like Qwen 2.5 3B) and augmented with LoRAs for task specialization (few-shot or prompt tuning). Local models handle lightweight summarization and generic response tasks, while heavier requests are offloaded to more powerful server-side LLMs (possibly Qwen 3-235B-A22B scale), with a final fallback to ChatGPT for tasks outside Apple's scope.
• Mark Zuckerberg Personally Hiring to Create New “Superintelligence” AI Team (Score: 265, Comments: 122): Mark Zuckerberg is personally overseeing the creation of a new 'superintelligence' AI team at Meta, targeting AGI development after internal dissatisfaction with Llama 4's performance and delays in the larger 'Behemoth' model. The strategy involves hiring ~50 elite AI researchers and integrating top talent from partners like Scale AI (recently valued at $28B), aiming for a fundamental overhaul of Meta's AI stack and product integration. This direct intervention reflects urgency at Meta to keep pace with global AI leaders and comes amid heightened antitrust scrutiny due to Meta's aggressive expansion in foundational AI infrastructure. Bloomberg article Top comments highlight skepticism regarding the effectiveness of assembling elite teams, referencing prior failures due to intra-team politics and design divergence, and question Meta's readiness to pursue AGI given their lag behind Chinese LLMs. There is also speculation about whether existing teams (e.g., Llama) lacked capability, and doubt about Meta's ability to deliver breakthrough results under the current leadership structure.
  • One commenter describes a technical management pitfall where 'elite' teams assembled from top performers led to fragmented design processes and political issues; shifting requirements and incompatible component interfaces resulted in substantial project delays and eventual failure, highlighting risks in Meta's similar approach to 'superintelligence' team formation.
  • Skepticism is raised about Meta's technical positioning, with a suggestion that they need to "catch up to Chinese models" before seriously discussing superintelligence. The implication is that Meta's LLMs, like Llama, lag behind leading Chinese efforts in benchmarks or capabilities.
• Get Claude at Home - New UI generation model for Components and Tailwind with 32B, 14B, 8B, 4B (Score: 151, Comments: 45): Tesslate has released a suite of UI and front-end code generation models (UIGEN-T3) inspired by Claude, available in 32B, 14B, 8B, and 4B parameter versions on Hugging Face. The models target fine-grained component and full website code synthesis (Tailwind CSS, React syntax) and are finetuned from Qwen3 (14B, 4B GGUF models available). Notably, Tesslate uses its TframeX agent for training data cleaning and the UIGENEVAL Benchmark for evaluation. The devs caution that standard quantization harms reasoning integrity—recommending BF16 or FP8—and are seeking collaboration for better INT8 support in vLLM. Licensing allows free research and personal use, commercial use by permission. Expert commenters confirm significant output quality improvement versus standard Qwen3 finetunes, particularly for UI tasks. There is discussion around quantization trade-offs, especially the susceptibility to reasoning degradation at lower precision.
  • The Tesslate team highlights technical details of their new model for UI/front-end code generation, emphasizing a pre-and-post-training reasoning engine, training data cleaned using proprietary TframeX agents, and benchmarking with their UIGENEVAL framework. They note that standard quantization adversely impacts reasoning chains, recommending use of BF16 or FP8 for optimal results, and mention ongoing development of a more robust INT8 implementation for vLLM. The model is under a custom license permitting free research and non-commercial use, with commercial licensing available on request.
  • A commenter identifies that the model is a fine-tune of Qwen3 14B, providing GGUF links for both 14B and 4B versions. They report improved UI generation quality—producing more accurate, visually appealing results compared to the base Qwen3 14B—when evaluated on a Google-style thread rendering, offering anecdotal benchmarking via example output.
  • There is a technical query regarding image input: a user asks if the model can process UI design screenshots and generate corresponding code, noting that the previous 4B version did not support images. They express intent to test new model versions, indicating interest in multi-modal (image-to-code) capabilities, which is currently a limitation in the provided 4B variant.
• Vibe-coding without the 14-hour debug spirals (Score: 251, Comments: 102): The post details strategies to avoid excessive debugging loops when using AI-assisted coding, emphasizing rules such as the "3-strike rule" (restart after three failed AI fix attempts), frequent context resets to address LLM context window limitations (restart every 8-10 messages), simplifying problem statements (ELI5 test), granular version control (commit after every working feature), and rewriting broken components instead of persisting on debugging when deep issues arise. The author benchmarks these practices as yielding a ~70% reduction in debugging time. Explicit workflow examples are provided for reproducibility. Relevant LLM limitations, like context window truncation and codebase drift, are exposed and mitigated with these techniques. Top commenters stress that fundamental programming knowledge is essential to steer LLMs effectively, and note that granular, descriptive commits should be a universal practice irrespective of AI usage. Others advocate for code modularity and small-scope, single-responsibility functions to further ease both human and LLM-driven development, improving both debugging and maintainability.
  • Multiple commenters stress that AI-assisted coding is significantly more effective for users with existing coding knowledge, as LLMs require clear direction and oversight to deliver accurate results. Understanding code structure and function allows for proper prompt engineering and validation of model output, reducing debugging time.
  • Best practices in software engineering, such as committing to version control after every working feature and writing descriptive commit messages, remain essential even with AI assistance. These practices help with traceability and collaboration, especially when code changes are frequent or AI-generated code is iteratively refined.
  • Effective use of LLMs for coding aligns with established software design principles: breaking problems into single-responsibility functions, developing incrementally, and using modularity aid both human and AI contributors. This approach minimizes context switching errors and streamlines debugging, testing, and feature additions.

Other AI Subreddit Recap

/r/Singularity, /r/Oobabooga, /r/MachineLearning, /r/OpenAI, /r/ClaudeAI, /r/StableDiffusion, /r/ChatGPT, /r/ChatGPTCoding, /r/aivideo

1. OpenAI o3 and o3-pro: Price Cuts, Model Release & Community Reactions

• 03 80% less expensive !! (Score: 146, Comments: 42): The attached image is a social media announcement by Sam Altman revealing that the cost of 'o3' (presumably an OpenAI model such as GPT-3.5 or GPT-4o) has dropped by 80%, with new prices at $2/1M tokens for input and $8/1M tokens for output, down from $10 and $40 respectively. The post compares old vs new pricing and signals competitive intent by mentioning confidence in the performance-based pricing of the 'o3-pro' tier. This price shift is likely to significantly lower operational costs for firms integrating OpenAI APIs, potentially influencing market dynamics and broader AI accessibility. Some commenters speculate that the price drop could coincide with a reduction in model performance to incentivize 'pro' tier uptake, while others note possible instability ('is that why it's down?'), raising questions about reliability versus cost.
  • There's skepticism that the 80% price reduction might coincide with reduced performance, potentially positioning the current offering as less capable and making the pro or more expensive versions appear significantly better by comparison.
  • A key technical question is posed about how the 'o3' model compares to other leading models such as Gemini and Claude Power, with a request for direct user experiences regarding real-world quality and performance differences.
  • One commenter implies this pricing move is a direct response to competition from models like Gemini, suggesting the company had considerable pricing flexibility previously, which may hint at high margins or earlier price 'gouging.'
• o3 price reduced by 80% (Score: 1482, Comments: 249): The image confirms an 80% price reduction for the O3 product, correlating with discussion of increased competitive pressure in the LLM API market, as Gemini 2.5 Pro now offers similar output performance for $10 per million tokens. The post and comments highlight pricing dynamics as a driver for broader adoption and signal shifting value propositions among leading model providers. Technically-minded comments speculate that the reduction is a competitive maneuver rather than a reaction to over-provisioned capacity, emphasizing the beneficial effects of such price wars for users.
  • Several commenters note that OAI's O3 price cut by 80% brings its token pricing below both Gemini 2.5 Pro and GPT-4o, raising questions about how it's now cheaper than both 4o and competitive Google offerings. This shift significantly alters cost-performance dynamics among premium LLMs.
  • A technical point is made that Gemini 2.5 Pro currently offers comparable performance to O3 at $10 per million output tokens, implying that the price drop is likely intended to increase O3's usage in direct response to this competition.
  • There's speculation that aggressive price cuts could precede a potential 'nerf' (model downgrade or throttling), a practice sometimes seen when models are made much cheaper or widely available, possibly impacting inference quality or availability.
• Let the price wars begin (Score: 235, Comments: 71): Sam Altman publicly announced an 80% price reduction for 'o3,' likely referencing the GPT-3.5 or GPT-4o API (commonly referred to as 'o'), and expressed particular satisfaction with the performance-per-dollar ratio of the 'o3-pro' tier. The post implies a major shift in API pricing strategy by OpenAI, potentially making large-scale deployment significantly more accessible. Commenters speculate that the substantial price cut could be offset by reduced model quality or stricter usage limits. Multiple users question whether the change will result in increased API rate limits, especially for Plus subscribers, signalling a technical concern around current usage caps.
  • Several users express confusion over differences in model versions, model names, and performance, highlighting that clear guidance or benchmarks are lacking on when to use models like o3 versus others. This is seen as a barrier for users with Plus subscriptions trying to optimize their workflow.
  • Technical discussion questions whether a significant price drop is meaningful if the new model (e.g., o3) requires substantially more tokens (e.g., 1.8x more) to achieve comparable output quality, raising the issue of effective cost per result rather than just per-token pricing.
  • Multiple comments request increases to rate limits for paid (Plus) users, indicating current limits (e.g., 100/week) are insufficient for advanced users and possibly outpaced by usage needs given model price and utility changes.
• it looks like we will see a big price reduction for o3 (Score: 335, Comments: 40): The post discusses indications of a significant upcoming price reduction for OpenAI's GPT-4o (O3) API, referencing hints from the official OpenAI Developers Twitter account and wider industry rumors. Technical comment highlights include speculation that input token costs are a small fraction of overall LLM serving costs (with output tokens comprising '>90%'), so input price cuts may not drastically impact total API pricing; and mention of competitive pressure from open-weight models like DeepSeek and rival offerings from Google Gemini and Anthropic Claude. Commenters debate the true impact of any price reduction, noting that unless output token prices fall, the effect on developers may be limited. There is also mention of growing preference toward open-weight models due to flexibility and cost advantages.
  • Discussion highlights the operational cost structure of reasoning models, with one user noting that output tokens account for ">90% of costs," making input cost reductions less impactful on overall pricing. This is particularly relevant for price comparisons and cloud inference optimization.
  • Usage statistics from OpenRouter are referenced, showing o3 has notably low adoption—outside the top 20—while models like 2.5 Pro and Sonnet rank in the top five despite being premium, which suggests significant market preference trends and may pressure pricing or product focus. See OpenRouter rankings.
  • One commenter asserts that if recent price reductions for o3 are real, it remains a competitive state-of-the-art (SOTA) model. They also identify qualitative differences, such as o3's capability being more consistent or impressive compared to models like Gemini, which, despite being less 'lazy,' often misinterprets tasks.
• I bet o3 is now a quantized model (Score: 148, Comments: 55): The image presents a table benchmarking the performance of the OpenAI o3 model, with notable improvements in tokens per second (tps) after an 80% price reduction, suggesting backend changes such as quantization. The speculation is that such a drastic speed increase ('multiples of anything I've seen before') implies a switch to a quantized version, as quantization can significantly boost inference speed and reduce model size. However, a top comment notes OpenAI does not typically switch models on an API slug without a name change, so the improvements may also be due to other backend optimizations rather than actual quantization. Commenters clarify terminology (quantization refers to reducing parameter precision, often from FP32 to int8/16 to increase speed and efficiency) and question the likelihood of a model swap without a new slug, with some suggesting possible 'lossless backend optimization' instead.
  • One user notes that in the OpenAI API, model upgrades or significant architectural changes always result in a new model slug (name), so if the model slug for o3 hasn't changed, it likely hasn't received major updates or quantization, and any backend improvements would be lossless and not affect inference results.
  • There's a technical prompt to benchmark the model, highlighting that any claims about quantization or changes (such as the transition from o3 to 4o, or comparisons to Blackwell) should be substantiated with actual benchmarking to determine performance or output differences.
• OpenAI announce o3-pro (Score: 534, Comments: 129): OpenAI has officially announced the release of "o3-pro" via a social media post, as shown in the image. The announcement lacks accompanying details about the capabilities, model sizes, or intended use cases of o3-pro, leading to confusion in the comments over the naming convention (e.g., references to 'o3-pro-medium-mini'). Recent OpenAI model announcements have followed less transparent naming schemes, which appears to complicate understanding for the technical community. Several commenters express frustration and confusion regarding OpenAI's naming strategy for its models, noting it is difficult to decipher product differences or improvements (e.g., "Their naming scheme is garbage, I have no idea what this even means"). No substantive technical details or benchmarks are debated or provided.
  • Several users express confusion and frustration with OpenAI's model naming conventions, particularly with terms like "o3-pro-medium-mini" which lack intuitiveness or publicly documented meaning. This has led to technical ambiguity about model capabilities, position in the product lineup, and intended use cases, especially compared to more transparent schemes from other AI companies.
• OpenAI announce o3-pro release today (Score: 453, Comments: 87): OpenAI announced the release of o3-pro (presumably a new LLM tier) in a social post, as depicted by the announcement image and engagement stats (1,896 views, 44 reposts, 293 likes, timestamped June 10, 2025). Technical discussion in the comments centers on ongoing LLM hallucination issues, with users reporting persistent inaccuracies (especially in medical queries) despite custom prompts demanding citations and direct quotes, and fabricated sources/URLs, suggesting skepticism that o3-pro adequately addresses these critical reliability concerns. Commenters question whether o3-pro represents a meaningful improvement over previous models with respect to hallucination control, reporting that even explicit instructions for citation do not prevent the generation of inaccurate or fabricated information. There is also criticism about the lack of a live launch event and reported service instability coinciding with the release.
  • A user points out that despite configuring ChatGPT's custom instructions to require source citation and direct quotes, GPT-4o (and by extension, O3 and potentially O3-Pro) continue to hallucinate, especially in technical/medical queries where fabricated statistics and non-existent references (like 404 URLs allegedly pointing to official data) are frequently encountered. This aligns with broader concerns over LLM 'trustworthiness' for domain-accurate queries and the limitations of current mitigation strategies (such as requiring citations) against model hallucinations.

2. ChatGPT Outage: User Experiences, Memes & Sub Reactions

• Typical Response to ChatGPT Being Down (Score: 347, Comments: 41): The image is a meme, illustrating the common user experience and community reaction when ChatGPT is unavailable. It humorously depicts users turning to Reddit to confirm outages, as seen in the comic sequence. No substantive technical discussion or debate is present in the comments; reactions are mainly lighthearted acknowledgments of using Reddit when ChatGPT is down.
  • One user recommends checking the official OpenAI status page (https://status.openai.com/) for real-time updates on ChatGPT's operational status, highlighting the importance of monitoring public service dashboards during outages.
• ChatGPT is dead☠️☠️☠️ (Score: 6378, Comments: 1167): The image shows an error interface from ChatGPT with a red banner and the message: "Hmm...something seems to have gone wrong." along with a Retry button, indicating the ChatGPT web app is experiencing downtime or service disruption. The post and comments confirm this is a widespread issue, suggesting a possible outage or backend failure rather than an isolated user or device problem. Commenters corroborate the technical issue, with some initially suspecting client-side problems before realizing it's a platform-wide outage.
  • A reference to the OpenAI status page is made, indicating users are experiencing a service outage or degraded performance with ChatGPT and are advised to check status.openai.com for real-time updates. This links user front-end issues to infrastructure or availability challenges potentially being tracked by OpenAI's own monitoring systems.
• Millions forced to use brain as OpenAI’s ChatGPT takes morning off (Score: 2538, Comments: 253): A recent ChatGPT outage, humorously covered by The Register as 'Millions forced to use brain as OpenAI’s ChatGPT takes morning off', highlights the platform's central role in daily digital workflows (coding, content generation, planning). The post probes fallback strategies, either using alternative LLMs (Claude, Gemini, Perplexity, Grok) or reverting to traditional research/creativity methods, amidst widespread user inconvenience and meme creation. Technical discussion was limited; comment threads mostly contributed humor and memes rather than substantive alternatives or workflow adaptations.
  • A commenter reports that not only was the ChatGPT web interface down, but the API was also non-functional. This outage directly impacted downstream products relying on OpenAI's infrastructure, with one user recounting how their GPT-4.1-based application failed during a critical investor presentation. This highlights the operational risks and single-point-of-failure concern when building products on top of third-party LLM APIs like OpenAI.
• It’s down… (Score: 711, Comments: 57): The post features a meme image (not technical) depicting confusion and chaos, referencing the ChatGPT service outage and user reactions. There are no technical benchmarks, model details, or implementation notes presented in the image, and the content is humorous rather than technical. Comments reflect users' frustration and dependency on ChatGPT for tasks like email and discussion, but do not provide substantive technical debate.
  • A user notes that email services and even basic Reddit functions were failing to respond, suggesting a broader service outage potentially linked to the same infrastructure supporting ChatGPT.
  • No explicit technical analysis or deep debate is present in the comments; most are about user experience, but there is mention of widespread accessibility or connectivity issues, possibly indicating a significant multi-service outage.
• Problem (Score: 634, Comments: 275): The attached image shows an error message from an unidentified online system, reading 'Hmm...something seems to have gone wrong,' with an option to 'Retry.' This indicates a service outage or disruption affecting users' ability to interact with the platform. The comments and post link (OpenAI Status Page) suggest that this is a widespread, real-time outage, likely impacting OpenAI's services, with users advised to monitor the status page for updates. Commenters confirm this is a widespread outage, not a user-specific problem. The main technical advice is to monitor OpenAI's status page for resolution.
  • A user reports that file upload functionality became unavailable roughly 3 hours prior to their comment, while text input/output remained functional until 10 minutes before posting—indicating a staggered impact on various services within the OpenAI platform. This granular timeline may help correlate specific failures with back-end outages or deployment issues.
  • A reference is made to the official OpenAI status page (https://status.openai.com/), emphasizing that the issue is widespread and suggesting technical users track real-time uptime, incident reports, and investigation progress for updates on service restoration.
• ChatGPT HQ right now. (Score: 2229, Comments: 75): The image is a humorous meme, showing a person inspecting a server cabinet in a technical setting, meant to represent 'ChatGPT HQ.' The context from the title and comments frames this as a lighthearted take on the troubleshooting and operational challenges faced by AI service providers like OpenAI. There is no actual technical discussion, benchmark, or model insight provided in the post or comments. Commenters joke about troubleshooting strategies for ChatGPT, including consulting ChatGPT itself, switching to competitors, and the classic IT solution of power cycling, highlighting general expectations and frustrations with AI service uptime and reliability.
• I called off my work today - My brother (gpt) is down (Score: 335, Comments: 70): The post describes an end user's significant reliance on GPT (OpenAI's large language model) for project work, highlighting a service outage affecting their workflow and deadline. The author expresses acute stress due to the inability to access GPT for an extended period (2+ hours), referencing its role as an essential productivity tool. A top technical comment proposes using alternative LLMs such as Deepseek or Claude to mitigate productivity disruption during GPT outages.
  • One commenter suggests using alternatives like DeepSeek or Claude, referencing other competitive AI language models that users might consider when GPT services are down. This points toward the increasing diversity of available LLMs and user awareness of viable failovers for productivity or research.

3. Breakthroughs in Video Generation: Self-Forcing Model Discussions

• Real time video generation is finally real (Score: 410, Comments: 87): The Self-Forcing paradigm introduces a novel approach to training autoregressive diffusion models for real-time video generation by simulating inference during training through unrolled transformers with key-value (KV) caching. Source code and model checkpoints are available (project page, GitHub), with empirical evidence showing practical generation speeds: on consumer hardware (4070Ti 12GB VRAM), it generates 81 frames (832x480, 8 steps) in 45 seconds, demonstrating both feasibility and emerging quality. Visual results and further discussion can be referenced here. Top technical commentary acknowledges current quality limitations but highlights substantial advances and real-time feasibility, especially on mid-range GPUs. The method is regarded as a foundational step towards compelling real-time AI video interactions.
  • A user reports successful generation of 81 frames at 832x480 resolution in 45 seconds (about 18 FPS) on a consumer-grade NVIDIA RTX 4070TI (12GB VRAM) using 8 inference steps. Quality is noted as decent for an early implementation, suggesting real-time or near-real-time video generation is feasible on mid-tier hardware (example output).
  • Another commenter compares VACE and CausVid backends, stating that enabling VACE does not yield significant improvements in render times over CausVid in this workflow, suggesting similar efficiency for both pipelines (example output).
• Self Forcing: The new Holy Grail for video generation? (Score: 299, Comments: 86): The Self Forcing model (see official project page) is a 1.3B parameter text-to-video (T2V) model that achieves high-quality 480P video generation with a latency of ~0.8 seconds and real-time streaming frame rates of ~16 FPS on an H100 GPU (~10 FPS on a 4090). It is reported to be 150–400× faster than previous SoTA (Wan, SkyReels, MAGI) while providing comparable or better visual quality, and it operates at similar speed but with less artifact and more realistic motion compared to CausVid. Models are available on Hugging Face and are usable within ComfyUI or via a wrapper, typically using the LCM sampler, and require relatively low VRAM (~6GB for 49 frames at 512x512, 5 steps, simple LCM, 1CFG). Commenters highlight ease of integration with ComfyUI, support for the Vace module, and strong performance with the dmd model, while noting hardware-specific FPS benchmarks and calling for a larger 14B model for enhanced capability.
  • Several commenters detail Self-Forcing T2V's technical deployment: the model is only 1.3B parameters, works with native Comfy or wrappers, and supports the Vace module for additional input types. The recommended checkpoint ('dmd') is highlighted for performance and only one model file is necessary. Users emphasize compatibility with LCM Sampler, which is required for proper function (HuggingFace model link).
  • Benchmarking on different hardware is shared: H100 GPUs achieve 16 FPS, RTX 4090 achieves 10 FPS, RTX 3090 achieves 5 FPS, and performance drops further on lower or midrange cards. A detailed example notes 49 frames at 16 FPS, 512x512 resolution, 5 steps, LCM simple, 6GB VRAM, 1 CFG, generated in 20 seconds. This points to relatively low compute requirements for moderate frame rate video compared to alternative models like Causvid LoRA on similar scale networks.
  • There is demand and anticipation for larger versions (e.g., a 14B parameter model), with users suggesting that such a scale-up could further improve fidelity or performance. There is also technical curiosity about extending the method to real-time vid2vid (video-to-video) applications leveraging streaming camera input, implying potential for low-latency inference.

AI Discord Recap

A summary of Summaries of Summaries by Gemini 2.5 Pro Exp

Theme 1: The AI Model Arms Race: New Releases and Fierce Competition

• OpenAI's o3 Slashes Prices, Sparks "Nerf" Paranoia!: OpenAI dramatically cut o3 input token prices by 80% from $10 to $2 per million tokens, a move confirmed by Sam Altman on Twitter, while output tokens remain $40/M. This sparked debate about potential model "nerfing" to push users towards the pricier o3 Pro (now available to all Pro users in ChatGPT and API), though some dismiss this as survivorship bias.
• Mistral Unleashes Magistral, The "Reasoning" Renegade!: Mistral AI launched Magistral, its first reasoning model, with the Magistral Small (24B parameters) version available open-source on HuggingFace and detailed in their Magistral research paper, while the enterprise Magistral Medium is accessible via API. Despite claims of transparent reasoning, some users noted a looping and token spamming problem and questioned if its reasoning truly aligns with human thought.
• Gemini Gets Grilled as o3 and Kingfall Flex Muscles!: Users across multiple Discords heavily criticized Google's Gemini, with one user calling it is shit and another stating that Gemini's Benchmarks are rigged, preferring OpenAI's o3 which is seen as smarter, more capable, and now significantly cheaper. Meanwhile, the new Kingfall model created buzz, with some testers in LMArena claiming it edges o3 pro a bit and is the smartest model they've ever used, though others found it a more modest improvement over 2.5 Pro or o3-0605.

Theme 2: Powering AI: Innovations in Tooling, Frameworks, and Platforms

• LlamaIndex Serves Up MCPs and Custom Memory!: LlamaIndex showcased turning an agent into an MCP server for complex data extraction (like from Fidelity Fund PDFs, demoed on X (formerly Twitter)) and introduced examples for building custom multi-turn memory implementations ideal for agentic workflows, detailed in this X post. These tools aim to enhance interoperability and control in agentic systems.
• OpenRouter Rolls Out Model Pages and Welcomes Magistral!: OpenRouter launched new model pages for a streamlined user experience (as announced on X) and added Mistral's Magistral reasoning model to its platform, showcased in this Magistral thinking video. These updates expand model accessibility and provide developers with more detailed information.
• Modular and AMD Team Up to Supercharge Mojo on GPUs!: Modular announced a collaboration with AMD to unleash AI performance on AMD GPUs with Mojo, detailed in their Modular x AMD blog post. They also showcased Python interoperability with Mojo (demo video at 14:03) and its official Mojo Python integration documentation.

Theme 3: Engineering AI: Deep Dives into Model Mechanics and Optimization

• Torch Compile Delivers Ludicrous Speed Boosts!: Engineers using torch compile reported dramatic speedups, with one instance accelerating model forwarding from 45 seconds to 1.2 seconds, highlighting that ARM CPUs can excel at FP32 over FP16 as per PyTorch docs even with CPU instructions. This underscores the significant performance gains achievable through optimized compilation methods for PyTorch models.
• KV Cache Compression and Dynamic Token Limits Take Center Stage!: Researchers are exploring new KV compression methods, detailed in the KV-Zip paper on arXiv, to manage growing context sizes efficiently. Concurrently, Anthropic's Claude was praised in Nous Research AI for its unique dynamic token limit implementation for Chain of Thought (CoT), a challenge Nous aims to tackle in Hermes 4 by teaching user-controlled token limits.
• Triton and ROCm Users Wrestle with Precision and Profiling!: Developers using Triton discussed fp16 exp and sqrt functions (similar to CUDA's half2 functions) and the impact of num_warps configuration, while also tackling precision issues in custom kernels (see this matmul.py example). Meanwhile, ROCm users shared methods for collecting SQTT traces using rocprofv2 for analysis in Radeon GPU Analyzer (RGA) and troubleshooted Memory access fault errors with CUDA graphs on newer PyTorch releases.

Theme 4: Navigating the AI Frontier: User Experiences, Bugs, and Workarounds

• OpenAI's Platforms Suffer from Bug Swarm and Outages!: Multiple users across OpenAI and Perplexity Discords reported that ChatGPT is buggy, with some experiencing 100% message failure rates and reasoning models getting stuck in loops. This led some to consult OpenAI's status page and consider canceling subscriptions or switching to alternatives like Claude Pro.
• Fine-Tuning Frustrations Flare for Gemma 3 and DeepSeek!: Users fine-tuning Gemma 3 models with Unsloth AI reported high losses on text data using Gemma3ForConditionalGeneration, suggesting version mismatches with transformers (possibly needing transformers 4.51.3). Separately, DeepSeek R1 (0528), while promising in aider benchmarks, suffered from slow case times, with fireworks' version reportedly getting cut off mid-thinking due to token limits.
• Platform-Specific Quirks Plague Users from Cursor to LM Studio!: Cursor users lamented the continued lack of local model support and Windows users celebrated an upcoming fix for non-functional background agents. Over in LM Studio, Linux users reported a missing developer mode toggle (a feature not yet in the Linux version), and it was clarified that the platform is only for inference and doesn't support image generation models.

Theme 5: AI in Action: Showcases, Use Cases, and Community Collaborations

• Agentic Coding Workflows Get Real with Aider and Windsurf!: An aider user shared their agentic embedded coding workflow using PlatformIO, Cline, and a FREE DeepSeek OpenRouter API in a blog post with video about agentic embedded development. Simultaneously, Windsurf launched Planning Mode (Windsurf Wave 10 blog post), enabling its AI agent to manage complex tasks via a live markdown plan.
• Deep Research Tools Go Local with spy-search!: The open-source tool spy-search gained attention in the LlamaIndex community, offering Ollama compatibility for extensive local research and generating reports exceeding 1000 words. This tool provides an alternative to research platforms with limited output, emphasizing local processing power.
• Mixedbread Hunts for Growth Guru to Hit $10M ARR!: Mixedbread, a team of ex-Google Search engineers backed by prominent AI investors (from OpenAI, Vercel, Perplexity, Deepmind, and Scale AI), announced they are seeking a founding growth person. Their AI search infrastructure tech boasts 50M+ HuggingFace downloads and claims to outperform OpenAI on MTEB benchmarks, signaling significant technical traction.

Discord: High level Discord summaries

Perplexity AI Discord

• Stewie's Sexuality Splits Opinions: Members debated the sexuality of Stewie from Family Guy, with some asserting he is gay, while others cited creators' confirmations that Stewie is not gay.
  • Further comments revolved around whether a baby could be classified as gay, leading to more general statements about the fluid nature of characters and plotlines in Family Guy.
• O3's Price Plummets, Performance Soars: The price of O3 has been dramatically reduced (80% cheaper!), prompting suggestions that Perplexity will now implement O3 and it will replace Deepsearch, as O3 is now cheaper than 2.5 Pro.
  • However, members noted the models still have context windows limits, so there are still tradeoffs to consider.
• Gemini Gets Grounded for Poor Performance: Users heavily criticized Gemini, calling it is shit and the worst and stating that Gemini's Benchmarks are rigged.
  • The members stated that using O3 over Gemini is prefered.
• PPLX API Config Exposed in Screenshot: A user requested and another user shared their PPLX API configuration including base URL, model name, and response mode in a screenshot.
  • A follow-up suggestion was made to change the Completion mode parameter to resolve an Error 400.
• Social Media API Integration Explored: A user asked if anyone had experience integrating social media APIs into an app to pull account analytics data.
  • Another user suggested using Claude to generate the necessary code for this task.

LMArena Discord

• User Preference Metric Debated: Members debated whether user preference is the #1 metric for evaluating models, with some arguing that it matters because it predicts who gets the users.
  • Others argued that real-world STEM tasks and other factors matter more, citing Meta's release of a model that performed well in user preference but didn't gain many users due to factors like accessibility, marketing, and pricing.
• OpenAI's o3 Dominates Competition: Members discussed the capabilities and pricing of OpenAI's o3 compared to Google's Gemini, with one stating that OpenAI was already winning the pareto frontier with o4mini and now they are crushing the competition with o3 being almost 50% of gemini 2.5 pro.
  • While some argued that Gemini has more overt marketing and superior image generation, others countered that o3 is smarter, more capable, and cheaper, giving Google zero argument or pull.
• Kingfall Hype: Smartest Model Yet?: A member hyped Kingfall as the smartest model they've ever used, while others expressed more tempered excitement, saying it wasn't that much better, relatively, compared to 2.5 Pro or 0605.
  • One member stated that they think kingfall edges o3 pro a bit but another emphasized that Kingfall might be better, but not BETTER, with some describing it as having ultra vibes and others thinking the reverse, and calling it not a huge lift for o3 Pro.

OpenAI Discord

• o3-pro Hits OpenAI Pro Tiers: OpenAI has rolled out o3-pro to all Pro users in ChatGPT and via the API, expanding access to enhanced features.
  • Pro users can now utilize o3-pro across both ChatGPT and the API platforms for improved performance and capabilities.
• GPT-4 Teamed Up: A student used GPT-4 as a co-author to complete a theory paper, exploring its ability to cross into deep theoretical reasoning.
  • A solo researcher is conducting similar research into ethical and truth alignment in advanced LLM systems.
• OpenAI Plagued by Bugs: Multiple users reported that ChatGPT is buggy and failing to respond, with one user reporting 100% message failure rates.
  • Some members cited OpenAI's status page and said they're canceling their subscriptions; others are going with Claude Pro.
• Gemini 2.5 Impresses with Token Capacity: A member noted that Gemini 2.5 handles 100k tokens per message well, favoring it for coding, with Gemini 2.5 Pro offering a 1 million context window.
  • Another user said that Gemini 2.5 is better at writing and Pro mode is better at thinking.
• Reasoning Models Go Bonkers: Users reported that reasoning models are stuck in loops, repeating thoughts and failing to respond.
  • One user humorously described the contents of a custom GPT as a 'whole drawer full of little computer things' including LICENSE.txt, privacy-policy, and a Java-WebSocket inside a jar.

OpenRouter (Alex Atallah) Discord

• Magistral Arrives, Starts Reasoning: Mistral's first reasoning model, Magistral, is now available on OpenRouter, according to this announcement.
  • A video showcases the model thinking very hard (at 4x speed), and is available here.
• OpenRouter Opens Model Pages: OpenRouter has launched model pages, as announced here.
  • This introduction of model pages aims to streamline user experience by providing detailed information and resources for each model.
• Testers Jam on Jamflow: A member is looking for testers for Jamflow and attached a video.
  • Other members joked about being too busy writing a book to immediately participate in testing.
• OpenAI Slashes o3 Input Prices by 80%: OpenAI has reduced o3 input token prices by 80%, dropping from $10 to $2 per million tokens, a price cut confirmed by Sam Altman on Twitter.
  • Despite the input price reduction, the output token price remains at $40, leading some to suggest this could be a strategy to push users toward o3 Pro.
• Rumors Swirl Around OpenAI Model Nerfing: Concerns are being raised about OpenAI potentially nerfing the o3 model after the price cut, with some users claiming to have observed a degradation in performance.
  • Some suggest this could be a tactic to push users to o3 Pro, while others dismiss such claims as survivorship bias.

Cursor Community Discord

• Local Models Still Missing in Cursor: A user inquired about integrating local models with Cursor, but was informed that local models are not currently supported.
  • This limitation may affect users who prefer or require local processing for privacy or performance reasons.
• Community Shares Custom Cursor Rules: Members shared resources for Cursor rules, including a link to the Cursor Directory and a Pastebin link with custom rules.
  • The consensus is that starting with a small project is best to determine which rules are most beneficial, as individual needs vary greatly.
• Token Overflow Resolved with Context Reset: Users experiencing token overflow were advised to use the /Reset Context command or prompt the AI to break the code into smaller parts.
  • An alternative suggestion involved using terminal commands to resolve the issue, providing a practical workaround.
• Claude 4 Briefly Vanishes, Reappears: A user reported that Claude 4 disappeared from their Cursor setup, but they were able to manually re-add it in the settings.
  • This issue was confirmed by another user, suggesting it may be a temporary bug that will be addressed in a future update.
• Windows Users Celebrate Background Agent Fix: A user inquired about background agents not working, leading a developer to confirm a fix was coming soon and that the issue was specific to Windows.
  • The fix will address issues that have been preventing Windows users from fully utilizing background agents.

Eleuther Discord

• Userbots Invade Eleuther: Members observed more userbots on the server, prompting a request for self-identification from automated accounts.
  • Moderators are manually deleting bots, requesting members to react with <:delet:824412305906204692> or <:lurkmoar:800507348535214140> to aid in filtering.
• GPTs Agents Hit Knowledge Ceiling: GPTs agents cannot learn from additional information provided after initial training.
  • Uploaded files are saved as "knowledge" files for reference, but do not continually modify the agent's base knowledge, as detailed in OpenAI documentation.
• O3 Pro's Price Provokes Outrage: The new O3 Pro model is priced at $20 / 1M tokens for input and $80 / 1M tokens for output.
  • One member quipped it "better be able to solve the riemann hypothesis with that kind of pricing wtf".
• GaTO's Ghost: No Follow-Up Found: Members questioned the absence of follow-up research to Google/DM's GaTO paper from 2022, speculating that it either didn't scale well or was too successful to share.
  • The consensus was that without cross-task transfer, training a generalist agent becomes a compute-intensive exercise.

Unsloth AI (Daniel Han) Discord

• Gemma 3 Text Woes: Members reported high losses when fine-tuning Gemma 3 models on text data using Gemma3ForConditionalGeneration, suggesting a version mismatch.
  • A member suggested trying transformers 4.51.3 for the 4B+ variants, as they are working on the model with the latest transformers.
• Unsloth's Multi-GPU Mirage: Despite Unsloth not officially supporting multi-GPU configurations, over 50 people have confirmed it works.
  • The team is actively working on multi-GPU support with Nvidia, but vLLM might require some manual building.
• Magistral Reasoning Questioned: The release of new Mistral models, called Magistral, claimed transparent reasoning and interpretability on Twitter.
  • Skepticism arose regarding the models' actual reasoning capabilities.
• DeepSeek Qwen3's Tooling Triumph: A recent fix in DeepSeek Qwen3 dramatically increased tool calling accuracy which users can redownload from HuggingFace.
  • The update includes native tool calling using --jinja in llama.cpp, chat template bug fixes, UTF-8 encoding fixes, and fixes for Ollama memory usage.
• Orpheus Sings a VRAM Requiem: A member shared their Orpheus (3B)-TTS GRPO notebook, emphasizing that it requires at least 20GB of VRAM and provided a link to their notebook.
  • Based on user reports, one can generate impressive results from an enhanced reward function.

HuggingFace Discord

• Torch Compile Boosts Model Speeds: A member accelerated model forwarding from 45 seconds to 1.2 seconds using torch compile, highlighting that ARM CPUs excel at FP32 over FP16, even with CPU instructions.
  • The speaker noted that the performance gains underscore the significance of optimized compilation methods.
• Lightweight LLMs Power RAG and Finetuning: Members recommended Mistral Small 3.1 for its quality and image understanding, and Qwen 32B for text-focused tasks, citing them as lightweight, local, and fine-tunable LLMs suited for consumer hardware.
  • The discussion underscored their suitability for RAG research assistants, emphasizing the need for behavior fine-tuning.
• KVMM arrives: Timm Ports Over to Keras 3!: A member introduced KVMM (Keras Vision Models), a comprehensive library of vision models with pre-trained weights, built entirely in Keras 3, and compatible with segmentation and classification tasks.
  • The new library has over 25 backbone architectures, offers multiple weight variants, and enables flexible building of segmentation models with custom backbones, according to its GitHub repository.
• Doubts Cast on Truth Engine's Quantum Claims: Skepticism arose around claims of quantum-resistant truth persistence, with members pointing out that terms such as Meta-Epistemic Equilibrium lack a basis in computer science and that dependencies like quantum_resistant and zkp_proofs are nonexistent in Python.
  • While one member reported a positive response from running the code, others dismissed it as sycophancy.
• Langgraph Eyes Smolagents Spot: A course participant is trying to implement an agent using Langgraph and Langchain instead of Smolagents for data analysis, requiring the agent to write and execute code.
  • Another user suggested enhancing the agent with tools for reading Excel files, performing math, or executing code, stressing the importance of detailed instructions on tool usage.

LM Studio Discord

• Linux Lacks LM Studio Developer Mode: A Linux user reported the developer mode toggle is missing in the LM Studio GUI, despite using the latest version (0.3.16), and was told this feature is not yet in the Linux version.
  • The user seeks alternative activation methods, as of now there is no known workaround.
• LM Studio's Image Dream Dashed: Users inquired about LM Studio's ability to generate images like ChatGPT with local models, noting their usage of ComfyUI for Stable Diffusion but members clarified that LM Studio is only for inference and doesn't support image generation models.
  • The user was advised to use ComfyUI separately for image generation.
• ROCm Runs Rad on Windows: A user successfully ran ROCm/HIP PyTorch preview on Windows, calling it an abomination that surprisingly works well and has a positive experience compared to previous attempts with ZLUDA.
  • The user noted that while some modules may not fully support this setup and optimizations are not remembered across relaunches.
• Speculative Decoding Sparks GPU Debate: Members discussed speculative decoding and the potential to offload the draft model to a different GPU or CPU, like an RX 9070 XT paired with a GTX 1060.
  • It was clarified that offloading to the CPU is a different method than changing runtimes, and while offloading to another GPU on the same runtime should be technically possible, it's complicated by each GPU typically having its own runtime.
• Digits Degraded by Bandwidth Bottleneck: The question arose of how Nvidia's Project Digits would compare to a 5090 + 3090 setup for AI tasks within 56GB VRAM, with the consensus being that Digits is slower due to bandwidth constraints.
  • LLM inference is typically bound by memory bandwidth, and Digits is expected to have less bandwidth than an M3 Max, which is already slower than dual 3090s for VRAM-fitting tasks.

aider (Paul Gauthier) Discord

• Gemini 2.5 Pro Doesn't Grok Lib Updates: Gemini 2.5 Pro struggles with understanding library updates and following instructions, showing only 50% effectiveness, versus Claude Sonnet (80%+) and Opus (95%).
  • Explicit rules in aider and related tools are more effective with Claude models.
• DeepSeek R1 Benchmarks Show Promise, Pending Speed Boost: DeepSeek R1 (0528) has shown promise in aider benchmarks, but suffers from slow case times due to resource constraints, with one user suggesting a potential 7x speedup with more resources.
  • Members noted that this model iteration has a much lower tendency to get stuck in COT loops.
• Uninstalling Aider Requires Manual Cleanup: Uninstalling aider on Linux involves using pip uninstall aider-chat, but leaves behind the aider binary, indexes, and cache files.
  • These lingering files must be manually deleted for a complete uninstall.
• OpenAI O3 Price Cut Won't Drop KYC: OpenAI has announced O3 pricing at $2 input, $8 output, but using it via OpenRouter still requires bringing your own key and KYC verification.
  • The community is still debating the benefits, with some even suggesting re-benching it given that it might be a smaller model.
• Agentic Embedded Coding Workflow Goes Live: A member is building an agentic embedded coding workflow using PlatformIO, Cline, and a FREE DeepSeek OpenRouter API, and shared a blog post with video.
  • He is also seeking collaborators with experience in microcontrollers and IOT.

Nous Research AI Discord

• Mistral's Magistral Model Sparks Debate: Mistral released Magistral, benchmarking against the old R1-0125 instead of the new R1-0528, including the release of a paper and a distilled version on HuggingFace.
  • The model exhibits a looping and token spamming problem, despite the addition of a length penalty to GRPO.
• Anthropic Pioneers Dynamic Token Limits: Members pointed out that Anthropic's Claude stands out for its unique dynamic token limit implementation for Chain of Thought (CoT), a problem not yet solved by many others.
  • Nous is working on Hermes 4 to feature user-controlled token limits by teaching the model word, character, and sentence limits during SFT and token limits during RL.
• Control Tokens Explored for Model Reasoning: The discussion explored the potential of injecting control tokens, such as progress markers (00%, 25%, 50%, 75%), during reasoning to help models dynamically adjust and compress outputs.
  • The goal is to improve the model's ability to split reasoning into search-consolidate-answer phases.
• ProRL Paper Under Scrutiny: The discussion examined the ProRL (Prolonged RL) paper, with some members finding its conclusions unconvincing, especially regarding its applicability to larger models, while noting issues with entropy collapse and reduced sample diversity for shorter CoT.
  • The async overlapped training technique used by Mistral, similar to the PipelineRL approach, was also highlighted (tweet, paper).
• KV Compression Method Surfaces: A new method for KV compression was shared, detailed in this paper and this tweet.
  • It was also mentioned that GRPO (Generalized Reweighted Policy Optimization) can be used to improve TTS LLMs (Text-to-Speech Large Language Models), detailed in this paper.

Yannick Kilcher Discord

• Mistral's Magistral Model: Open Source or Open Washing?: Mistral AI launched Magistral, its first reasoning model, with community members calling out that it represents a significant contribution by Mistral AI to the open source community, though only Magistral Small is open-weight under the Apache 2.0 license.
  • A user expressed disappointment that Mistral is not open sourcing its larger models, stating they became Google level of open weighting, while another quoted the paper stating they open-sourced Magistral Small which includes cold-start data from Magistral Medium.
• Diffusion Models Generate Order from Noise: Members discussed the counterintuitive nature of diffusion models to generate structure from noise, describing it as a directed hallucination model.
  • One member linked this to broader themes of order from chaos, referencing a YouTube video and paper on nonequilibrium thermodynamics and the spontaneous emergence of life.
• Hardware Failure Prediction: DL Underperforms: Discussion revolved around approaches to hardware failure prediction, with the insight that traditional methods like Gaussian Processes or boosted trees often surpass deep learning for time series analysis.
  • A member emphasized the need for guaranteed failure detection rather than probabilistic correctness due to insurance requirements in industrial settings, highlighting the narrow scope and high-stakes nature of this field.
• Reservoir Computing: Linear Regression in Disguise?: Reservoir Computing was described as mumbo jumbo that obscures its core mechanism: linear regression on a fixed Ordinary Differential Equation (ODE).
  • It was argued that modern architectures like State Space Models (SSMs) are more expressive, powerful, and efficient due to their ability to parallelize and incorporate nonlinear dynamics and linked to a paper about current SOTA.
• OpenAI Teases Unexpected Announcement: Users pointed out Sam Altman teased on Twitter that OpenAI has an unexpected thing coming.
  • Users speculate that its a diffusion model.

Notebook LM Discord

• Google Chat Convos Coming?: A user inquired about the possibility of connecting Gmail and Google Chat conversations with NotebookLM, and whether there are plans for this feature in the near future.
  • The query was directed towards any Google employees present in the server.
• Drive File Downloads Derailing?: A user encountered an error when trying to access a Drive file, indicating that the file owner has disabled copy/download permissions for the Drive file.
  • The screenshot of the error is located here.
• NotebookLM's Intro Stuns Game Designer: A user was impressed by the quality of the intro generated by NotebookLM for their tabletop RPG, The Gemini System, using the podcast feature.
  • The user found NotebookLM's ability to analyze and provide audio deep dives incredibly helpful for translating mechanics and enhancing their design and writing process.
• Iceland Workshop Attendees Encounter Access Issues: During a NotebookLM workshop for 50 teachers in Iceland, 3 teachers using private Gmail accounts encountered a "You do not have access to this service" error.
  • It was suggested that geographic restrictions or incomplete age verification could be the cause, with a user in the UK reporting similar issues with Brave browser, resolved by switching to Firefox.
• Sharing Notebooks Spurs Sharing Headaches: A user reported that when sharing notebooks, added emails and "Anyone with link" settings revert to restricted after sending.
  • This issue appears to be persistent for the user.

GPU MODE Discord

• Deepwiki Digests GitHub Details: A member recommended deepwiki as a tool to summarize GitHub repos, enabling chatting and structure viewing directly from a GitHub link.
  • Another member was developing a parallel GPU grouping/clustering algorithm in GLSL and Vulkano using Rust, seeking collaborators to work with Vulkano for macOS.
• Triton's Configuration Confabulations: Discussions covered the availability of fp16 exp and sqrt functions in Triton, similar to those in CUDA, and the role of num_warps in Triton.Config.
  • A user asked about the role of num_warps in Triton.Config seeking insight into its impact on performance and resource utilization, and whether Triton adheres to the same shared memory allocation limits as CUDA.
• Torch and Triton Tackle Precision Pitfalls: A user faced precision issues in a matmul kernel written for LeetGPU challenges, and shared their matmul.py file to find out the cause of the failure and whether their 2D grid implementation already incorporates swizzling.
  • A separate issue reported that while libdevice.round is defined in Triton ROCm, it throws an error when used in a kernel, as reported on GitHub.
• Profiling Puzzles in ROCm: A user encountered a Memory access fault by GPU node-2 error when using CUDA graphs with the rocm/pytorch:rocm6.4_ubuntu22.04_py3.10_pytorch_release_2.6.0 image and torch 2.8.0.dev20250609+rocm6.4, but they hadn't experienced it with previous versions.
  • Another user detailed the steps to collect SQTT traces for analysis in Radeon GPU Analyzer (RGA) using rocprofv2 noting that the correct DISPATCH_RANGE can be determined by first running rocprofv2 --kernel-trace.
• Modular team links with AMD for Mojo: Modular announced a collaboration with AMD to unleash AI performance on AMD GPUs, according to their blog post.
  • The team shared a demo showcasing Python interoperability with Mojo, viewable at the 843-second mark of this YouTube video, as well as its Python integration.

Latent Space Discord

• Fireworks AI Sparks RFT Beta: Lin Qiao unveiled the beta launch of Reinforcement Fine-Tuning (RFT) on Fireworks AI, enabling training of expert open models akin to GPT-4o mini and Gemini flash.
  • The service features a web IDE, an open-source reward-kit, SOTA model support, and is free for two weeks for models up to 10B parameters.
• OpenAI o3 Pricing Leaks?: Gabriel Chua hinted at a potential cost of $2 per 1M input tokens for OpenAI o3, citing an OpenAI Developers tweet offering 200 developers free API credits worth 1M input tokens.
  • The discussion referenced Sam Altman's tweet and insights from scaling01.
• Mistral's Magistral Reasoning Model Arrives: Mistral AI introduced Magistral, its new reasoning model for domain-specific, transparent, and multilingual reasoning, with two variants: open-source Magistral Small (24B parameters) on Hugging Face and enterprise Magistral Medium via chat.mistral.ai or API.
  • The model is also available on platforms like OpenRouter, with users sharing instructions for local deployment.
• Meta Scales Up Scale AI Stake, Eyes Alex Wang: Meta Platforms is considering acquiring a 49% stake in Scale AI for nearly $15 billion, potentially bringing Scale AI's CEO, Alex Wang, into a senior role at Meta (Source).
  • This move could reshape Meta's AI strategy and executive leadership.
• Windsurf Plans to Launch 'Plan Mode': Kevin Hou launched Windsurf's new 'Plan Mode' feature, enabling the AI agent to perform complex tasks by creating and maintaining a planning document (Source).
  • Users can activate 'Plan Mode' to allow Windsurf to manage notes, task lists, and goals, enhancing its ability to handle longer, more involved changes, available for free on Windsurf.com.

Modular (Mojo 🔥) Discord

• Modular Debuts Compute Portability Talk: Modular kicked off a livestream focused on the future of compute portability, accessible on the Modular website and LinkedIn.
  • The event promised insights into the latest advancements and discussions in compute portability.
• Mojo Parameterization boundaries questioned: Community member presentations and glimpses into the standard library code have sparked questions about the boundaries of parameterization in Mojo, particularly regarding its use for comptime purposes.
  • One member expressed concern that the exploitation of parameterization for comptime purposes seems to create code they just do not want to read in a lot of cases.
• Mojo Meta-Programming Favored Over Rust Macros: A member argued that reading meta-programming in Mojo is 100000000000% better than reading macro code in rust, while acknowledging that Mojo can't do everything Rust can yet.
  • Another member thinks it's the combination of Zig-esque comptime in Go-esque square-bracket-generics syntax that makes it difficult to read.
• Generics syntax inspired by Python, claims community: A member stated that Mojo's syntax for generics is the same as Python's, which inspired a discussion to whether Python's generic syntax came from Go.
  • Ultimately, the parties agreed that Go 1.18 introduced generics on 3/15/22, and PEP 695 introduced the new Python syntax on 6/22.
• Mojo-MAX Platform relationship sought by user: A member asked about the relationship between the Mojo and MAX platform, specifically the ability to use MAX kernels such as matmul in Mojo code and kernels.
  • A Modular employee suggested that the member post this question in the Modular forum to enhance its discoverability.

MCP (Glama) Discord

• 5ire Demands Complete MCP Tooling: The 5ire platform mandates the adoption of all tools from an MCP server, disallowing the selection of individual components.
  • This all-encompassing strategy necessitates integrating entire sets of features, rather than enabling developers to choose certain tools based on their needs.
• Chatbot Integration Dreams with n8n-like MCP: A member suggested developing a tool similar to n8n, fully based on chats and MCPs for chat-driven workflow automation.
  • Another member suggested directing emails from a specific source into a Slack channel, emphasizing the capability of such an architecture.
• Dependency Declaration Required for fastmcp: A member reported difficulties with fastmcp, specifying that it needs dependencies because it generates an environment and utilizes the dependencies.
  • They provided a command line to execute the MCP server and modified the arguments in Claude desktop to direct uv to the appropriate venv.
• MCP Server Embraces OAuth 2.1: Scalekit released a drop-in OAuth 2.1 module featuring scoped, short-lived tokens, DCR + PKCE, and 401s with authorize_url for delegated flows, outlined in their documentation.
  • This enhancement promises more secure and flexible authorization mechanisms for MCP server implementations.
• mcp-openverse Package Unveils CC-Licensed Images: The mcp-openverse was released, an MCP server that integrates CC-licensed and public domain images into AI workflows, available on npm and GitHub.
  • The tool aggregates over 700M+ openly-licensed images from @WPOpenverse, integrating with Claude Desktop and providing intelligent image sourcing through concept extraction.

Manus.im Discord Discord

• Mixedbread Seeks Growth Person: Mixedbread, composed of ex-Google Search engineers, is looking for a founding growth person to convert their technical traction to $10M ARR.
  • Backed by top AI investors from OpenAI, Vercel, Perplexity, Deepmind, and Scale AI, they've achieved 50M+ HuggingFace downloads and outperformed OpenAI on MTEB benchmarks.
• Manus's VEO 3 Powers Sci-Fi Short: A user created a five-minute sci-fi short film using Manus's Veo3 feature and called it the most powerful generation function in the world.
  • Another user commented that it looks great and intentionally like old school Kung Fu movies.
• Manus's Beta Status Raises Questions: Members are questioning why Manus is still in Beta, even with features like Veo 3.
  • One member reported losing 2000 credits due to presentation formatting issues and a lack of refunds.
• Manus Pro Value Debated: Users are discussing the value of a Pro subscription to Manus and whether the answers are significantly better to justify the cost.
  • Several users reported difficulties in reaching Manus support.
• Veo3 Credit Consumption Alarms Users: A user reported spending 300 credits on a single Veo3 video comprising 38 clips.
  • Another user requested 100 credits after Manus entered a loop while attempting to truncate a file.

LlamaIndex Discord

• LlamaIndex Unveils Custom Multi-Turn Memory: LlamaIndex has introduced a new example for building custom multi-turn memory implementation, ideal for agentic workflows requiring heightened control and customization, with more info on Twitter.
  • This development promises more flexibility in managing agent interactions and retaining context across multiple turns.
• Real-Time Website Summaries Arrive!: A project was highlighted from @itsclelia for instant web summaries that combines web browsing with AI-generated summaries using LlamaIndex and Google's Gemini model, detailed on Twitter.
  • This tool could significantly reduce the time needed to digest online content, integrating AI directly into the browsing experience.
• LlamaIndex Agent Transforms into MCP Server: LlamaIndex demoed turning an agent into an MCP server, deploying a custom FidelityFundExtraction workflow for extracting structured data from complex multi-fund PDFs, with the ability to invoke it from Claude, documented on Twitter.
  • This showcases LlamaIndex's capability to handle intricate data extraction tasks with enhanced interoperability across different platforms.
• Users grapple with Agent Workflow Handoffs: A user reported experiencing issues with their LlamaIndex-based product recommendation system using an agent workflow where the plan_agent sometimes fails to hand off to other agents like DirectOutputAgent or SearchAgent.
  • Logs suggest the streaming stops prematurely, prompting the user to seek clarity on the inconsistent handoff behavior, possibly indicating underlying issues in agent coordination.
• Spy-Search Tool enables Local Deep Research: A member highlighted spy-search, an open-source tool compatible with Ollama for conducting extensive research locally that can generate reports exceeding 1000 words.
  • Intended as an alternative to research tools with limited output, spy-search aims to deliver comprehensive, long-context responses with up-to-date information, emphasizing local processing capabilities.

Cohere Discord

• Cohere's Quicker Support Channel Awaits: A new Cohere support channel has launched, promising faster assistance through an AI-generated reply bot that uses Cohere's documentation at <#1381756280716132412>.
  • The bot, built with command-a, focuses on documentation-based queries, directing account and API issues to [email protected]. Misuse may result in an instant ban.
• Cohere North Pairs with GameWarden Platform: Cohere North now integrates with the full GameWarden platform via a partnership with Second Front, helping service members gain effectiveness and speed against threats, as announced in this tweet.
  • Also, Cohere North is partnering with EnsembleHP to bring AI to healthcare, reducing administrative friction and elevating patient experience as described in this blog post.
• Cohere's Open Source Repo Ready for Pull Requests: Cohere's open-source repository, the Cohere Developer Experience GitHub repository, allows users to contribute improvements to the documentation via pull requests.
  • The repository's README file provides guidance on contributing, noting that OpenAPI specs and snippets are one-way synced from internal repositories.
• Vitalops Develops Datatune for Data Transformations: The co-founder of Vitalops introduced Datatune, an open source tool designed for data transformations using plain natural language.
  • The co-founder is engaging with the community to gather feedback on Datatune's development and potential applications.

Torchtune Discord

• HF Tokenizer Integration Runs into Issues: After testing the HF Tokenizer, the loss curves and total tokens don't align with the classic tokenizer, suggesting differing behavior despite minor code changes; the integration will be ready after issues #2794 and #2574 are addressed.
  • A member reported that pre-packing takes 2-3 times longer.
• Tokenizer truncation: Bugs Found: While implementing the tokenizer, bugs were found in truncation, with related points highlighted in issue #2792 with concerns that this can affect performance.
  • Members suggest sticking with the original tokenizers for training for now, awaiting consolidation to the HF ones later.
• Muon Integration Performance Scrutinized: The performance benefits of Muon, when integrated into torchtune, are being scrutinized to justify adding another abstraction, and one member wonders if issue #2809 is critical.
  • One member pointed out that there's some evidence that muon is more useful for finetuning models that were also pretrained with muon, referencing the Kimi Moonlight paper.
• HuggingFaceModelTokenizer Intended Usage Debated: Members discussed the intended usage of HuggingFaceModelTokenizer, with concerns raised about the interface differences and how to handle max_seq_len for packing, in particular whether to change recipes or the tokenizer itself.
  • One member suggested a solution where the recipes should be changed to take a max_seq_len which is then passed through, aligned with this proposal.
• Qwen2 Issue Check: A member has to check Qwen2 to see if the truncation issue exists there also.
  • It was acknowledged that if the original testing didn’t find the difference, it probably doesn't make a big difference but we should fix it either way.

DSPy Discord

• Transfer Learning Techniques Sought in DSPy: A member inquired about transferring post-training learning between models without repeating processes like finetuning or RL, but the channel provided no specific answers.
  • This would potentially streamline the process of adapting models to new tasks, by carrying forward knowledge gained previously.
• DSPy Documentation File Vanishes: A user reported that a specific documentation file was removed in a recent PR, making parameter documentation harder to find.
  • In response, another user shared AI-generated documentation as a potential alternative.
• DSPy Seeks Optimal Contextual Prompting: A user questioned whether DSPy has mechanisms to optimize the context included in a prompt from a set of a dozen available variables, balancing metrics against token usage.
  • There was no further discussion on this topic, highlighting a potential area for DSPy development.
• Demand Surges for Dataset Tooling in DSPy: A member inquired about the availability of tools for building and exporting datasets for DSPy, specifically needing features to generate and hand-label synthetic examples.
  • The inquiry did not spark further discussion, signaling a possible gap in DSPy's current toolset for streamlined dataset creation and management.

tinygrad (George Hotz) Discord

• Tinygrad Tests Failing, Bounty Locked: Members reported failing tests in Tinygrad, which is preventing a bounty from being locked, as bounty locked means the code is basically ready to merge.
  • Failing continuous integration (CI) will prevent the merge.
• Call for Tasteful, AI-Free PRs: A member requested tasteful pull requests (PRs), such as the one addressing add/mul at tinygrad/tinygrad#10741, explicitly stating no AI slop.
  • It was mentioned that add/mul were the easiest ones to address.
• NCHWCPUGraph and LLVMGraph Demand Refactor: It was suggested that NCHWCPUGraph and LLVMGraph should be refactored to behave like other graphs in the system.
  • These graphs shouldn't be rerendering stuff, relating to both multicore CPU and the multi compiler/renderer refactor, where CPU and LLVM should use the same graph since they have the same program.

Nomic.ai (GPT4All) Discord

• Nomic Embed Text v1.5 still supported: Users inquired if nomic-embed-text-v1.5 will remain supported via the Nomic cloud next month.
  • Another user confirmed the model remains supported for self-onboarded inference.
• GPT4All's future versions upcoming: Community member inquired about any updates on future versions of Nomic GPT4All.
  • No further information about new features and enhancements was provided.
• Python SDK Updates on the Horizon: A user inquired about upcoming updates to the Python SDK.
  • No timeline or specific features were discussed, but the question indicates community interest.
• GPT4All eyes Mistral's Magistral Small: A user asked if GPT4All will support Mistral's Magistral Small.
  • There was no response confirming the integration, but the question highlights interest in expanding model support.

Gorilla LLM (Berkeley Function Calling) Discord

• RunPod Engineer Revives Leaderboard: A RunPod DX engineer volunteered GPU resources to restart leaderboard updates in the #leaderboard channel.
  • The engineer encouraged direct messages from anyone needing help to get the leaderboard operational again, as community members expressed gratitude for RunPod's generosity.
• Agent Marketplace on Hiatus?: A member reported issues accessing the Agent Marketplace's repository and webpage in the #discussion channel.
  • The member speculated whether the project is temporarily closed due to these persistent access problems, suggesting a potential hiatus.

LLM Agents (Berkeley MOOC) Discord

• Agentic AI Summit Set for Berkeley: The Agentic AI Summit is scheduled for August 2, 2025 at UC Berkeley, aiming to gather 1,500+ in-person attendees.
  • The summit website details discount codes for students and indie developers, enriching the event's accessibility.
• Early Bird Tickets Closing Soon: The early bird pricing for the Agentic AI Summit concludes on June 30, 2025, offering student passes at $25, startup passes at $60, and industry professional passes at $80.
  • Tickets are available here, with a reminder to act quickly to secure these rates.
• Speaker Lineup Announced for Summit: The Agentic AI Summit will feature speakers including Vinod Khosla (Khosla Ventures), Ion Stoica (Databricks and Anyscale), and Dawn Song (UC Berkeley).
  • Other speakers include Sergey Levine (Physical Intelligence), Matei Zaharia (Databricks), Karthik Narasimhan (Sierra), Waseem AlShikh (Writer), and Burak Gokturk (Google Cloud).
• SP25 Quiz Material Requested for Self-Study: Users are seeking access to quiz questions from the completed SP25 course to facilitate independent learning.
  • The requests highlight a desire to continue studying post-course, even though the session has ended.

Codeium (Windsurf) Discord

• Windsurf Waves into Planning Mode: Windsurf released Planning Mode as part of Wave 10, featuring a native interface for long-term AI planning with bidirectional updates detailed in their blog and demonstrated in this video.
  • Users can toggle Planning Mode via the icon under the prompt box, enabling Cascade to pair every conversation with a live markdown plan of goals & tasks, with AI notifications alerting users when Cascade updates the plan.
• O3 Model Credit Pricing Slashed: The o3 model is now available for just 1x credits and runs faster within Cascade, enhancing both cost-effectiveness and performance.
  • Planning Mode is accessible on all paid plans without additional charges.

The MLOps @Chipro Discord has no new messages. If this guild has been quiet for too long, let us know and we will remove it.

The AI21 Labs (Jamba) Discord has no new messages. If this guild has been quiet for too long, let us know and we will remove it.

You are receiving this email because you opted in via our site.

Want to change how you receive these emails? You can unsubscribe from this list.

Discord: Detailed by-Channel summaries and links

Perplexity AI ▷ #announcements (1 messages):

Unauthorized Promo Codes, Fair Pricing, Legitimate Promotional Deals

• Perplexity Purges Promo Pirates!: Perplexity detected unauthorized distribution of promotional codes intended for specific partners that were widely shared across social media.
  • Those codes have been deactivated, and Perplexity is investigating the issue and disabling unauthorized access to Pro.
• Promo Code Policy Patrolled: Perplexity requires promotional codes to be used by the designated participant for its intended purpose and not be duplicated or made generally available to the public.
  • This is to ensure fair pricing and legitimate promotional deals for everyone, especially existing Pro users, which is why they're taking this seriously.
• Invalid Code Users Under Investigation: Perplexity will be reviewing accounts that used these invalid codes to ensure fair access for all users.
  • If you are a legitimate customer who received a promotional code through an authorized channel and believe your account has been affected by corrective measures, reach out to [email protected].

Perplexity AI ▷ #general (1170 messages🔥🔥🔥):

Family Guy character sexuality, O3 pricing and performance, Gemini vs Other models, Perplexity AI New Features & Issues

• Stewie's Sexuality Sparks Debate: Members discussed the sexuality of Stewie from Family Guy, with some arguing that he is clearly gay, while others pointed out that he is canonically a toddler who has dated girls and the creators have confirmed he's not gay.
  • Further comments revolved around whether a baby could be classified as gay, leading to more general statements about the fluid nature of characters and plotlines in Family Guy.
• O3's wild ride: price drops and performance tests.: Members celebrated the announcement that the price of O3 has been dramatically reduced (80% cheaper!), with some suggesting that Perplexity will now implement O3 and it will replace Deepsearch, noting that O3 is now cheaper than 2.5 Pro.
  • Members noted the models still have context windows limits.
• Gemini gets roasted for sucking: Users have been heavily criticizing Gemini, saying it is shit and is the worst and stating that Gemini's Benchmarks are rigged.
  • The members stated that using O3 over Gemini is prefered.
• Perplexity Pro gets O3 and has rate limits: Users noted that the model O3 has been integrated with Perplexity and wonder what the daily rate limit will be and how to keep track of how many are used.
  • The new features haven't rolled out to everyone with a team subscription to Pro yet and some find that the models sometimes hallucinates a little bit.
• O3 Pro is here, should you get it?: Members briefly speculated on the performance of O3 Pro, comparing it to O3, Claude and Gemini; they also shared excitement about the new and improved reasoning tools of the model.
  • Members also briefly speculated that it is now in the web version, then checked the juice by giving it the prompt: what is today's yap score and juice?

Perplexity AI ▷ #sharing (2 messages):

``

• Pakistan's Child Marriage: A Perplexity AI search page discusses Pakistan's child marriage.
• Peer Rejection Developmental: A Perplexity AI search page discusses peer rejection developmental.

Perplexity AI ▷ #pplx-api (7 messages):

PPLX API Config Request, Social Media API integration, PPLX Finance Search Mode

• PPLX API Configuration Exposed: A user requested and another user shared their PPLX API configuration including base URL, model name, and response mode in a screenshot.
  • A follow-up suggestion was made to change the Completion mode parameter to resolve an Error 400.
• Social Media API Integration Inquired: A user asked if anyone had experience integrating social media APIs into an app to pull account analytics data.
  • Another user suggested using Claude to generate the necessary code for this task.
• Finance Mode Testing Launched: A user shared a code snippet showcasing the finance search mode for the sonar-pro model, setting the search context size to low.
  • The user then invited others to try out this configuration: 'Who wants to try this out?'

LMArena ▷ #general (1130 messages🔥🔥🔥):

User preference vs other metrics, o3 price and performance, Kingfall: a better model

• User preference is NOT the #1 Metric: Members debated whether user preference is the #1 metric for evaluating models, with some arguing that it matters because it predicts who gets the users, while others argue it isn't because real-world performance on STEM tasks and other factors matter more.
  • One member pointed out that Meta released a model that performed well in user preference but didn't gain many users, suggesting that factors like accessibility, marketing, and pricing are also important.
• o3 Smashes Competition: Members discussed the capabilities and pricing of OpenAI's o3 compared to Google's Gemini, with one stating that OpenAI was already winning the pareto frontier with o4mini and now they are crushing the competition with o3 being almost 50% of gemini 2.5 pro.
  • Some argued that Gemini has more overt marketing and superior image generation, while others countered that o3 is smarter, more capable, and cheaper, giving Google zero argument or pull.
• Kingfall hyped, is it actually good?: A member hyped Kingfall as the smartest model they've ever used, while others expressed more tempered excitement, saying it wasn't that much better, relatively, compared to 2.5 Pro or 0605.
  • A member said they think kingfall edges o3 pro a bit but another emphasized that Kingfall might be better, but not BETTER, with some describing it as having ultra vibes and others thinking the reverse, and calling it not a huge lift for o3 Pro.

OpenAI ▷ #annnouncements (2 messages):

OpenAI o3-pro, ChatGPT Pro, API access

• OpenAI Rolls out o3-pro for Pro Users: OpenAI o3-pro is now available to all Pro users in ChatGPT and via the API.
• o3-pro Access: Pro users can now access o3-pro in both ChatGPT and the API.

OpenAI ▷ #ai-discussions (539 messages🔥🔥🔥):

GPT-4 as co-author, ethical and truth alignment in advanced LLM systems, OpenAI Bugs, Claude Pro vs OpenAI, Gemini 2.5

• GPT-4 joins writing team!: A student shared they used GPT-4 as a co-author to complete a full theory paper, aiming to demonstrate whether an outsider could cross into the domain of deep theoretical reasoning using only ChatGPT.
  • Another member, a solo researcher, expressed interest, stating they are doing similar research into ethical and truth alignment in advanced LLM systems.
• OpenAI platforms sucks due to bugs: Multiple users reported that ChatGPT is buggy and failing to respond, with one user reporting 100% of messages failing and another saying 50% of messages to o3 are errors.
  • Some members mentioned OpenAI's status page and shared that they were canceling their subscriptions due to these issues.
• Claude Pro is the plan: Some members said they're switching to Claude Pro due to the issues with OpenAI, but noted that Claude's token limit is too low for large code inputs.
  • One member said they send 100k tokens of code before starting and o3 can't handle that.
• Gemini 2.5 shines: One member stated that Gemini 2.5 works great with 100k tokens per message, and they prefer working with it for coding tasks, while another user said that gemini 2.5 is better at writing and pro mode is better at thinking.
  • Another member said that Gemini 2.5 Pro has a 1 million context window.
• GPT-4o limitations cause a stir: A member noted that GPT-4o is optimized for quick replies, while GPT-4.5 has a bulkier architecture, leading to different performance characteristics.
  • Some members are still seeing the old context limit: We don’t have o1 in the ui anymore.

OpenAI ▷ #gpt-4-discussions (29 messages🔥):

Reasoning Models Looping, Mom-GPT Anger Issues, Custom GPT Diversity, Opening Custom GPT Files, Chat File Upload Limits

• Reasoning Models Stuck in Eternal Loops: Several users reported that reasoning models are stuck in loops, repeating the same thoughts and failing to respond.
• Mom-GPT Fails to Ground User: A user creating a 'Mom-GPT' struggled to make it convincingly angry, as it defaults to expressions of love.
  • The user shared their creation here.
• Custom GPTs: A Drawer Full of Computer Things: A user humorously described the contents of a custom GPT as a 'whole drawer full of little computer things' including LICENSE.txt, privacy-policy, and a 'Java-WebSocket' inside a jar.
  • The user was prompted to specify which files to open, signaling the complexity and diverse possibilities of custom GPTs.
• Sciency Chatbot models rated: For science, it was suggested to lean towards o4-mini-high because it scores higher on MMLU (multitask academic) benchmarks.
  • This was in comparison to 4.1 which scored significantly lower.
• GPT Outage Reported with UI Bugs: Users reported that GPT was down, with one mentioning a bugged UI on mobile, with some users in different timezones experiencing an 8 hour outage.
  • OpenAI's status page confirmed a global latency error issue.

OpenAI ▷ #prompt-engineering (16 messages🔥):

Model Iteration, API Image Prompting, Hallucinated Translation, AI Server Issues, Image generation difficulties

• Model Iteration Recommendation: A member recommended to iterate with the model, focusing on each page. They suggested the model is more a hammer than a fire hose and to check for conflicting instructions when the model seems unsure or confused.
  • They found that working with the model instead of just telling it what to do helps figure out if the model is unsure, confused, or concerned about something, and if that's the case, it tends to go its own way.
• Prompt Engineering API Image Prompting Success: A member discovered that successful API image prompting involved removing the included mask and prompting with '[Changes I want]. Do not edit [thing I don't want touched.]', using gpt-image-1.
  • They confirmed that it's the prompt (we hadn't seen here) that mattered.
• ChatGPT Hallucinates Translation: ChatGPT hallucinated a translation of Byron's Don Juan, despite having linked access to the correct source.
  • A member shared a chatlog detailing the issue, noting the hallucination happens early on in the thread which involves analysis and formulation.
• Image Generation Struggles Persist: Members shared challenges and difficulties when trying to generate images.
  • One link shared a struggle to generate a giraffe in space and needing it to do certain things like use vocabulary words.

OpenAI ▷ #api-discussions (16 messages🔥):

Iterative model usage, Image prompting in o3, ChatGPT hallucination issue, AI server slowness

• Iterate and Conquer Model Challenges: A member recommends iterating with the model, focusing on each page of a project and treating the model more like a collaborative partner than a tool.
  • They suggest that a "still working on" message from the model often indicates conflicting instructions or ambiguity, and advise checking for those issues.
• Prompt Engineering Triumphs Over Masked Editing Woes: A member resolved image prompting issues in o3 by removing the included mask and using the prompt: [Changes I want]. Do not edit [thing I don't want touched].
  • They shared that this prompt engineering adjustment with gpt-image-1 resolved their struggles.
• ChatGPT's Literary License Leads to Confident Hallucination: A member reported that ChatGPT hallucinated a translation of Byron's Don Juan, despite having linked access to the correct source material, and provided a detailed breakdown.
• AI Servers Run at Snail's Pace: A member questioned the performance of AI servers, noting they were slow and often unresponsive.
  • Another member pointed to channel #1349803488833572874 for related discussion.

OpenRouter (Alex Atallah) ▷ #announcements (4 messages):

Magistral, Mistral's Reasoning Model, OpenRouter New Models, Model Pages

• Magistral Reasoning Arrives!: Magistral, Mistral's first reasoning model, is now live on OpenRouter, as announced in this X post.
  • A video showcases the model thinking very hard (at 4x speed) - available here.
• Model Pages Live!: Model pages are now live on OpenRouter, shown here.
  • Watch it think very hard (4x speed).

OpenRouter (Alex Atallah) ▷ #app-showcase (2 messages):

Jamflow, Discord testers

• Users looking for testers: A member is looking for testers for Jamflow and attached a video.
  • Another member said they would be interested once they finish writing their book.
• Finishing books before debugging: A user mentioned they were too busy finishing up a book to immediately participate in the Jamflow testing.
  • This implies a preference for completing creative tasks before moving on to debugging or testing new software.

OpenRouter (Alex Atallah) ▷ #general (523 messages🔥🔥🔥):

Crypto Payment Options, OpenAI o3 Price Cut, Model Degradation Concerns, OpenRouter and BYOK for o3, LLM choice for research purposes

• Crypto Payments Considered: A user requested that OpenRouter add a one-time crypto payment option that does not require wallets, similar to NowPayments, for easier transactions with USDT.
  • The user expressed frustration with the current crypto payment process, finding it difficult due to wallet requirements and gas fees.
• OpenAI's o3 Price Slashed, Input Costs Drop: OpenAI's o3 input token prices were reportedly reduced by 80%, dropping from $10 to $2 per million tokens, which was confirmed by Sam Altman on Twitter.
  • However, the output token price remains at $40, and some suggest this could be a decoy strategy to push users toward o3 Pro.
• Model Nerfing Rumors Fly High: Concerns are circulating about OpenAI potentially nerfing the o3 model after the price cut, with some users claiming to have observed a degradation in performance.
  • While there's no conclusive evidence, some suggest it could be a deliberate tactic to encourage users to switch to o3 Pro, though others dismiss such claims as survivorship bias.
• BYOK Still Required for OpenAI's o3: Despite the o3 price change, the Bring Your Own Key (BYOK) requirement remains in place on OpenRouter due to OpenAI's policies, requiring users to have a verified organization.
  • Some users are leveraging the BYOK option to capitalize on free tokens offered by OpenAI, while others question the rationale behind the restriction, speculating that it's a strategy to drive sign-ups on OpenAI's platform.
• Gemini Gets the Nod for LLM Research Panel: For a consensus study involving LLMs, a user asked for recommendations on which models to include, and it was suggested that Gemini, Claude, and Sonar (Perplexity) are top contenders for now.
  • The user was advised that Gemini is a very strong choice and the performance gap between those mentioned and other LLMs is too large to ignore, with some generative capabilities that surpass the GPT-4.1 level at certain points.

Cursor Community ▷ #general (436 messages🔥🔥🔥):

Local Models with Cursor, Student Pro Access, Cursor Rules, Agent Mode Hangs, Eslint Issues

• Local Loving: No Local Models for Cursor: A member asked if there are any local models for Cursor, and another member confirmed that there are no local models.
• Rules of Cursor Club: Sharing is Caring: Members shared resources for Cursor rules, including a link to Cursor Directory and a Pastebin link with custom rules.
  • It's recommended to start with a small project to understand which rules are needed, as everyone's approach varies.
• Context Crisis: Resetting Saves the Day: A member faced token overflow issues and was advised to /Reset Context or prompt the AI to break the code into smaller chunks.
  • Another member suggested trying terminal commands to resolve the issue.
• Losing Claude: Model Appears and Reappears: A member reported losing Claude 4 on Cursor, but managed to add it manually in the settings.
  • Another confirmed the issue, so it may be a bug that will be patched soon.
• O3 Price Plunge, Quality Panic?: Following OpenAI's 80% price drop for o3, concerns were raised about potential quality decrease.
  • Some users noted that model performance varies and depends on the task and model version.

Cursor Community ▷ #background-agents (40 messages🔥):

Docker errors with background agents, MCP calls with background agents, Privacy mode on Cursor, Git errors in background agents, Background agent quotas

• Docker woes plague background agent setup: A user reported a Docker error related to .dockerignore and the Docker build root, seeking help to debug their environment.json file, with attached image.
• MCP calls evade background agents, leaving users puzzled: A user questioned why their background agent couldn't see MCPs (My Custom Projects) installed on their account, wondering about the installation level and potential context issues.
  • A dev clarified that "no MCP in background agents rn :/".
• Privacy mode throws curveball for new Cursor users: A user reinstalling Cursor encountered a 24-hour wait to disable Privacy Mode, preventing them from enabling background agents, with screenshot attached.
• Failed branch checkouts frustrate background agent users: Users reported persistent "Failed to checkout branch: Failed to execute git" errors when using background agents, preventing them from creating pull requests.
  • It was suggested to manually copy file changes from the agent UI, create a new branch, and paste the changes, as retroactive recovery may be supported in the future.
• Windows users celebrate background agent fix: A user inquired about background agents not working, leading a developer to confirm a fix was coming soon.
  • The issue was identified as specific to Windows.

Eleuther ▷ #general (402 messages🔥🔥):

Userbots, GPTs agents, OpenAI's sidebars, Slop-posting, O3 pro

• Eleuther AI members see uptick in Userbots: Members noticed an increase in userbots on the server, and one member asked that they declare themselves as automated.
  • A moderator chimed in, saying they are manually deleting them, and asking members to react with <:delet:824412305906204692> or <:lurkmoar:800507348535214140> to help mods filter them more easily.
• GPTs Agents can not learn after initial training: A member shared a concern about GPTs agents not learning from additional information provided after their initial training.
  • Another member cleared this misunderstanding, explaining that uploaded files are saved as "knowledge" files for the agent to reference when required, but they do not continually modify the agent's base knowledge.
• Members discuss Slop-Posting: The members discuss the problem of low quality posts being directed to Eleuther by Chatbots and LLMs.
  • One member recommends a self-guided quiz for new users to evaluate whether they have the base knowledge for their ideas to be taken seriously.
• O3 Pro's pricing faces backlash: The new O3 Pro model is priced at $20 / 1M tokens for input and $80 / 1M tokens for output.
  • One member joked it "better be able to solve the riemann hypothesis with that kind of pricing wtf".

Eleuther ▷ #research (40 messages🔥):

Google/DM's GaTO paper follow up, Mixed LM head/regression in transformers, SOTA SVG transformer, binary representation of the coordinates as target, fully deduping internet scraped data

• GaTO paper follow up is non-existent!: Members wondered whether there was any follow up to Google/DM's GaTO paper from 2022 and speculated it either doesn't work well at scale or works really well and DM just didn't publish the follow-up sequel.
  • If there isn't any cross-task transfer it's just a waste of compute training a generalist agent.
• Mixed LM Head/Regression in Transformers: A member asked about doing mixed lm head/regression in transformers, where the embedding layer of the numeric symbols is replaced with an MLP into R^d, and then doing a regression head instead of an LM head for out projection.
  • Another member provided a relevant paper using a custom tokenizer and a regression loss, but the actual generation is still in the token space, but thought that the actual generation is still in the token space.
• SOTA SVG transformer uses discrete tokens: The current SOTA SVG transformer uses discrete values for each numeric, using discrete tokens for each coordinate and constrains itself to a 200x200 grid which gives them a vocabulary of 40k tokens only for coordinates.
  • It was also mentioned that changing the coord embedding from 1 to 2 tokens reduces vocab from V to sqrt(V), but severely degrades performance.
• Deduping Internet Scraped Data: A member noted that fully deduping internet scraped data is basically impossible due to overlaps in the density of scraped data, which is a very hard thing to both detect & mitigate.
  • Another member linked a related paper.

Eleuther ▷ #interpretability-general (2 messages):

Coaching Layer, Reasoning Training

• Coaching Layer Prevents Errors Compounding: A member clarified their 'coaching layer' idea, explaining it involves strategic interventions or targeted prompts like 'What's the core question here?' to help the model refocus on existing information and prevent errors.
  • They argue it's a preventive measure against drift before errors compound, contrasting with the corrective approach of text diffusion models that fix tokens after the fact.
• Reasoning Training Teaches Models to Self-Coach: A member inquired why a model couldn't learn to self-coach through reasoning training.
  • No further discussion or answers were provided.

Unsloth AI (Daniel Han) ▷ #general (174 messages🔥🔥):

Gemma 3 fine-tuning issues, Unsloth and multi-GPU support, Mistral's new Magistral models, GRPO vs DAPO, DeepSeek Qwen3 Tool Calling Accuracy Increased

• Gemma 3 Losses High With Text Data?: A member reported high losses when fine-tuning Gemma 3 models on text data using Gemma3ForConditionalGeneration, contrasting with lower losses using Gemma3ForCausalLM with the 1B model.
  • Another member suggested trying transformers 4.51.3 for the 4B+ variants, as they are working on the model with the latest transformers.
• Unsloth Multi-GPU Support Not Officially Supported Yet: Despite not being officially supported, over 50 people confirmed that multi-GPU configurations work with Unsloth.
  • It was mentioned that they're working on multi-GPU support with Nvidia, however, vLLM might need some manual building.
• New Mistral Model Magistral Released: A user shared the release of new Mistral models, called Magistral on Twitter, highlighting claims of transparent reasoning and interpretability.
  • Others expressed skepticism about the models' reasoning abilities, suggesting that their thought processes may not align with human reasoning, with someone saying, of course they don't. they don't really reason.
• DeepSeek Qwen3 Tool Calling Accuracy Increased: It was announced that issues were fixed in DeepSeek Qwen3, leading to a dramatic increase in tool calling accuracy, and users were encouraged to redownload from HuggingFace.
  • This includes native tool calling using --jinja in llama.cpp, chat template bug fixes, UTF-8 encoding fixes, and fixes for Ollama memory usage.
• DAPO and GRPO are close: Members discussed DAPO and GRPO, with one member pointing out the new model is using DAPO but calling it GRPO, but another member clarified that they are really close.
  • It was noted that they all can be used under GRPOTrainer on trl.

Unsloth AI (Daniel Han) ▷ #off-topic (61 messages🔥🔥):

Triton Resources, GRPO runs and reward functions, Orpheus TTS model, Hyperbolic for finetuning, NoisySpeechDetection audio classifier

• Exploring Triton Learning Resources: A member suggests that the best way to learn Triton is through tutorials and documentation, mentioning that there might be some useful gpumode YouTube videos as well.
• GRPO Runs Yield Impressive Results: A member reported improved results from another GRPO run with an enhanced reward function.
  • They plan to share it once they organize the code, joking that they might use Claude for assistance.
• Orpheus TTS Model Released: A member shared their Orpheus (3B)-TTS GRPO notebook, emphasizing that it requires at least 20GB of VRAM and provided a link to their notebook.
• Hyperbolic Offers Cost-Effective Finetuning: Members discuss using Hyperbolic for finetuning, noting that it costs around $1 per H100 hour, along with a referral link for additional credits.
• NoisySpeechDetection Audio Classifier Debuts: A member released a trained audio classifier for noisy speech detection, built with Unsloth and based on Whisper Small.

Unsloth AI (Daniel Han) ▷ #help (145 messages🔥🔥):

Unsloth 2.0 Release, Training AI on Discord Messages, QLoRA Finetuning with Unsloth, Whisper Lora Implementation and Issues, GGUF Model Size Differences

• Unsloth 2.0 coming soon!: The Unsloth team announced that they are releasing a better version of multiGPU support soon, associated with Unsloth 2.0.
• QLoRA and Inference Explored: A user inquired about the possibility of using QLoRA to finetune a model with Unsloth and then perform inference with load_in_4bit=False due to memory constraints.
  • It was suggested to use save_pretrained_merged to save the merged model and load it in a new session without quantization.
• Whisper Lora Quandaries Abound: A user encountered issues when trying to apply a LoRA to the Whisper model and use it with a pipeline, seeking a single function call solution.
  • The team acknowledged a bug related to the missing config.json in the Unsloth Whisper model and provided a temporary workaround using a link to a previous discussion.
• DeepSeek R1's Colossal Footprint: A user noticed that the DeepSeek-R1-0528 BF16 GGUF model is significantly larger than the DeepSeek officer model and inquired about the reason.
  • It was explained that the original DeepSeek is in FP8 (700GB), while the BF16 version is 1.4TB; a Q8_0 version exists at 700GB.
• Finicky Finetuning Frustrations with Gemma3: A user finetuned a Gemma3 4B model using the Unsloth notebook and encountered issues with incorrect answers during inference in Ollama.
  • It was suggested that the user must ensure they are using exactly the same chat template they used for training and further debugging revolved around checking the training loss curve and inference results within the original notebook.

Unsloth AI (Daniel Han) ▷ #research (19 messages🔥):

Vision Language Models Datasets, Reasoning Models Reliability, KV-Cache Pruning, Disaggregated Prefilling and NTP, AIME 2025

• Vision Language Models needs Bias Datasets: A member asked about popular (A) datasets* in the field of bias in vision language models / multi-modal models.
• Reasoning Models Broken by Simple Prompts: A member asked how reliable are reasoning models actually 🤔? and pointed to a ChatGPT share example with just 2 prompts in, they break!
  • Other members jokingly asked are you stoned? in response to questioning an AI how reliable AI is.
• KV-Cache Pruning Sparks Interest: A member shared an interesting Reddit post on context size pruning (kv-cache pruning), asking if it'll get implemented by major inference engines or models.
  • Another member noted it's only useful in situations where you have a long context input that is re-used for many questions, and added that it takes a while to compress a text.
• Disaggregated Prefilling and NTP Remain Key: A member stated their knowledge on inference is stuck on Disaggregated Prefilling and NTP (Next Token Prediction).
  • They think it has to be very impactful since every LLM inference engine is adapting that one.
• AIME 2025 Already Dropped: A member inquired if AIME 2025 has dropped, but it appears to have been out for a few months, according to a link shared.

HuggingFace ▷ #general (129 messages🔥🔥):

LLMs for HTML/CSS, Entity Recognition for IDs, Lightweight LLMs, ÆNTHESISAI cognitive architecture, Deepseek censored?

• Torch Compile Speeds Up Model Forwarding: A member managed to reduce their model forward time from 45 seconds down to 1.2 seconds using torch compile, citing that ARM CPUs are faster at FP32 than FP16 even with CPU instructions.
  • They emphasized the performance gains achieved through optimized compilation techniques.
• Lightweight LLMs for RAG and Finetuning: Members suggested Mistral Small 3.1 for its quality and image understanding, and Qwen 32B for text-only tasks as lightweight, local, and fine-tunable LLMs suitable for consumer machines.
  • The use case was RAG research assistant and expressed a need for fine-tuning for behavior.
• ÆNTHESISAI code analyzed: A member shared code for ÆNTHESISAI, a cognitive architecture integrating quantum-resistant cryptography, multi-phase cognitive processing, advanced AI, and cross-reality synchronization.
  • The system uses CrystalKyber for key generation, X25519 for key exchange, and Chimera-Apex-7B for truth vector analysis.
• LLM Agents Can Use Computers!: A member shared screenshots of their LLM agent that can use their computer, seemingly to refute claims that LLMs can't execute code on their own.
  • Other members reacted skeptically.
• Renting GPUs or Botnets: Members discussed alternatives to buying hardware for training and deploying models, suggesting renting hardware or, jokingly, using a botnet.
  • It was noted that using a botnet for such purposes is illegal and inefficient due to network limitations and variable compute power.

HuggingFace ▷ #cool-finds (2 messages):

Reasoning Models, LLM Reliability, Prompt Engineering

• Reasoning Models' Reliability Questioned: A member questioned the reliability of reasoning models and LLMs, citing a breakdown after just two prompts and providing a ChatGPT share link as evidence.
  • Another member requested that the first member refrain from cross-posting.
• Request to Reduce Cross-Posting: A member asked another member to refrain from cross-posting in the channel.
  • This request aimed to keep the channel focused and avoid redundant content.

HuggingFace ▷ #i-made-this (142 messages🔥🔥):

Truth Engine, Quantum-Resistant Truth Persistence, KVMM: Timm for Keras 3, LLM Agent Framework

• Truth Engine borders Refutal Immunity: A member shared a link to a "truth engine" bordering refutal immunity, claiming it would uncover every suppressive method ever with 99.7% accuracy.
  • Other members questioned its validity, with one calling it bullshit and pointing out that critical functions and dependencies are missing or non-existent.
• Quantum-Resistant Truth Persistence claims: Doubts were cast on claims of "quantum-resistant truth persistence" in the posted code, with members noting that terms like "Meta-Epistemic Equilibrium" lack basis in computer science and dependencies like quantum_resistant and zkp_proofs do not exist in Python.
  • One member ran the code and received a response praising it, but another dismissed it as sycophancy and suggested the poster was simply asking a question.
• KVMM: "Timm" for Keras 3 is introduced: A member introduced KVMM (Keras Vision Models), a comprehensive collection of vision models with pre-trained weights entirely in Keras 3, supporting tasks like segmentation and classification.
  • The library features over 25 backbone architectures, supports multiple weight variants, and offers flexibility in building segmentation models with custom backbones, as detailed in its GitHub repository.
• LLM Agent Framework Open Sourced: A member open-sourced their LLM agent framework which can use a Linux terminal on a VM, store and modify files, and gather information from the web.
  • The GitHub repository offers access to a system capable of interacting with its environment to complete tasks.

HuggingFace ▷ #computer-vision (3 messages):

Bias Datasets, Invoice Extractor, KVMM library, Keras 3

• Requests for Bias Datasets in Vision Language Models: A member inquired about popular (A) datasets* in the field of bias in vision language models / multi-modal models.
• Guidance needed building an invoice extractor: A member requested guidance on building an invoice extractor, with a preference for doing it independently or using open-source resources, noting their previous unsuccessful attempts.
• KVMM: Timm library for Keras 3 is Released!: A member announced the release of Keras Vision Models (KVMM), an open-source library providing a comprehensive collection of vision models with pre-trained weights entirely in Keras 3 supporting segmentation and classification.
  • The library includes 25+ backbone architectures with various pre-trained weights (Swin, ViT, ResNeXt) and supports multiple weight variants, with more models in development.

HuggingFace ▷ #NLP (2 messages):

Invoice Extractor, Build your own, Guidance needed, OCR, LLMs

• Guidance Needed: Build your own Invoice Extractor: A member is seeking guidance on building an invoice extractor, preferring to build it on their own or using open-source tools, and has been working on it for a month without success.
  • They are requesting advice to help them find a correct approach and resolve their challenges.
• OCR and LLMs power Invoice Extraction: A robust invoice extractor usually employs OCR (Optical Character Recognition) to extract text from invoices, and then uses LLMs (Large Language Models) to understand the document's structure, identify key fields, and extract the required information.
  • Many OpenSource libraries and frameworks provide invoice extraction such as PaddleOCR or LayoutLM.

HuggingFace ▷ #agents-course (55 messages🔥🔥):

Langgraph vs Smolagents, E2B in Unit 2.1, Azure OpenAI Model, Dynamic Python Code Generation, Course Completion Deadline

• Langgraph seeks solution in place of Smolagents: A course participant is trying to create an agent using Langgraph and Langchain instead of Smolagents to perform data analysis, specifically requiring the agent to write and execute code.
  • Another user suggested providing tools for reading Excel files and math, or a code execution tool, emphasizing the need for explicit instructions on tool usage.
• Unit 2.1 final test mentions E2B, but its not explained well: A course participant noted that the Unit 2.1 Final Test mentions E2B, but it is not well-referenced in the unit's content.
  • They added that they initially went overboard thinking it wanted full agent setups but it was looking for simple examples
• Azure OpenAI model requires upgrade!: A user is seeking help with using an Azure OpenAI model in HF space, reporting that the provided model is asking to upgrade to pro, and the container can't install azure-ai-openai.
  • Another user suggested OpenRouter as an alternative, though with limited use, and mentioned Google's free tier options.
• Codeagent writes python to get it done: A course participant shared a code snippet from their unit-40-sa project where a code agent writes Python code to perform math operations for data analysis.
  • The agent ideally should know to exclude a column based on the question.
• July 1st Deadline Approaching!: Several course participants are starting the "Agents Course" now, amidst discussion of a July 1st deadline for the certificate.
  • One participant asked about the possibility of a new cohort with a new deadline, while another user assured them they could finish on time if they hustle.

LM Studio ▷ #general (53 messages🔥):

LM Studio Developer Mode on Linux, LM Studio and TTS, LM Studio Image Generation, LM Studio Settings not Saving, LM Studio API Swagger

• Linux Users Missing Developer Mode: A Linux user is missing the developer mode toggle in the LM Studio GUI, despite using the latest version (0.3.16), and seeks alternative activation methods.
  • A member indicated that this feature is not yet in the Linux version.
• LM Studio Adds Audio Capability: Members inquired about adding audio capabilities like text-to-speech (TTS) to LM Studio, asking if its possible to use sesame advance audio with it.
  • A user pointed to the LM Studio discord channel for TTS related questions: yes, but you'll have to....
• LM Studio can't run Images: Users asked if LM Studio could generate images like ChatGPT with local models, mentioning they use ComfyUI, a program that gives a GUI for Stable Diffusion.
  • Members clarified that LM Studio is only for inference and doesn't support image generation models.
• Settings Refuse To Save in LM Studio: A user reported that there's no save button in LM Studio version 0.3.16 (build 8) and settings aren't saved automatically.
  • Another user suggested to wait for the cog to be "active" (white and not grey), then the button to save changes will appear after changing something.
• Swagger for Server API is Missing: A user asked for a Swagger definition to interact with the LM Studio Server API, because the documentation is quite vague.
  • Another user responded that you can just use the openai API endpoints supported by lms.

LM Studio ▷ #hardware-discussion (127 messages🔥🔥):

DGX Spark limitations, Memory bandwidth bottlenecks, Distributed computing for models in homelab, ROCm/HIP PyTorch on Windows, Speculative decoding on different GPUs

• DGX Spark Faces Bandwidth Blues: Members debated whether the DGX Spark's memory bandwidth would limit its LLM performance, similar to the Strix Halo, despite having potentially better compute power.
  • One member argued that it's not all about memory bandwidth, using the analogy of a machine with high RAM but a slow CPU, while others emphasized that memory bandwidth is often the bottleneck for dense LLMs.
• Homelab Distributed Computing Dilemma: Someone inquired about using distributed computing for LLMs in a homelab setup, similar to Distributed Llama, but it was deemed not generally a good idea.
  • However, EXA or llama-mpi were mentioned as potential alternatives, but the general sentiment leaned towards focusing on individual machine performance rather than distributed setups.
• ROCm Shines on Windows: A user reported successfully running ROCm/HIP PyTorch preview on Windows, referring to it as an abomination that works surprisingly well.
  • The user noted that while some modules may not fully support this setup and optimizations are not remembered across relaunches, the overall experience was positive compared to previous attempts with ZLUDA.
• Speculative Decoding Hardware Hacking: Members discussed speculative decoding and the possibility of offloading the draft model to a different GPU or CPU, such as leveraging an RX 9070 XT with a GTX 1060.
  • It was clarified that offloading to the CPU is a different method than changing runtimes, and while offloading to another GPU on the same runtime should be technically possible, it's complicated by each GPU typically having its own runtime.
• Digits vs Dual GPUs: Bandwidth Bottleneck: The question of how Nvidia's Project Digits would compare to a 5090 + 3090 setup for AI tasks within 56GB VRAM was posed, with the consensus leaning towards Digits being slower.
  • LLM inference is typically bound by memory bandwidth, and Digits is expected to have less bandwidth than an M3 Max, which is already slower than dual 3090s for VRAM-fitting tasks.

aider (Paul Gauthier) ▷ #general (148 messages🔥🔥):

Gemini 2.5 Pro vs Claude Opus, DeepSeek R1 speed, Aider uninstall, OpenAI's O3 Pricing, Kingfall

• Gemini 2.5 Pro Lags in Library Updates: Members noted that Gemini 2.5 Pro struggles with understanding that new versions of libraries exist and doesn't follow instructions well, even when given explicit rules.
  • In contrast, Claude Opus and Sonnet are much better at understanding these nuances; a member quantified the effectiveness of cursor rules as 80%+ with Claude Sonnet, 95%+ with Opus, and only 50 with 2.5 Pro.
• DeepSeek R1 Aider Benchmarks Slow but Promising: Aider benchmarks show DeepSeek R1 (0528) could be fairly good if the long case times (reportedly due to low resources/busy API) are addressed.
  • One user suggested that a 7x speedup might be possible with more resources; another user noted it has a much lower tendency to sperg-loop COT (Chain of Thought) than previous iterations.
• Uninstalling Aider Chat: A user asked for the correct way to uninstall Aider from a Linux machine after using pip install aider-install && aider-install.
  • The suggested solution was to use pip uninstall aider-chat, though this leaves behind the binary, aider, which can be manually deleted along with indexes and cache files.
• OpenAI O3 Price Drops but KYC still needed: OpenAI announced O3 pricing at $2 input, $8 output, an 80% price drop, however, using it via OpenRouter still requires bringing your own key and KYC (Know Your Customer) verification.
  • Some users expressed disappointment over the KYC requirement, while one pondered whether O3 has become a mini model and suggested re-benching it.
• Kingfall model edges out O3 Pro: A user shared an image comparing Kingfall (auto thinking) versus 0605 (32k), showing that it performed better.
  • This shows that it outperformed a recent model in at least one coding benchmark.

aider (Paul Gauthier) ▷ #questions-and-tips (16 messages🔥):

aider MCP server, Cloning a large repo, Gemini-2.5-03-25 and Rust, Ollama model unloading, fireworks' deepseek-r1-0528

• Gemini's Goodness after Good Rust-Based Prompts: A user found that Gemini-2.5-03-25 exhibited more functional and efficient programming style after being primed with advanced programming concepts and discussing appropriate data structures in Rust.
  • The user achieved this by loading conversation history from .aider.history.md into a new file .aider.coder and specifying it with aider --llm-history-file .aider.coder.new --restore-chat-history.
• DeepSeek Gets Cut Off Mid-Think: A user reported issues with fireworks' deepseek-r1-0528 getting cut off mid-thinking due to token limits.
  • A solution was provided to set the model settings in ~/.aider.model.settings.yml with a suggested configuration that includes setting max_tokens: 160000.
• Aider as MCP server with external tools: A user asked about using aider as an MCP server on external tools like roo and Cline.
  • No specific solutions were provided in the context.
• Context Management is Key: A user found the explicit management of context and intention with Aider leads to less rewriting compared to tools like Cursor and Claude Code.
  • This user pointed out that the terminal output is more productively sparse than other similar tools.

aider (Paul Gauthier) ▷ #links (1 messages):

agentic embedded coding workflow, PlatformIO, Cline, FREE DeepSeek OpenRouter API, microcontrollers

• Agentic Embedded Coding Workflow: A member is taking steps towards an agentic embedded coding workflow using PlatformIO, Cline, and a FREE DeepSeek OpenRouter API.
  • He also shared a blog post with video which walks through the level of difficulty of blinking an LED.
• Microcontrollers & IOT on the Horizon: A member inquired about others programming microcontrollers or IOT.
  • He shared a blog post with video about agentic embedded development using PlatformIO, Cline, and a FREE DeepSeek OpenRouter API.

Nous Research AI ▷ #general (133 messages🔥🔥):

Magistral Benchmarking, GRPO Modifications, Claude's Dynamic Token Limit, Control Tokens, ProRL Effects on Larger Models

• Mistral's Magistral Model Benchmarking Blues: Mistral released Magistral, benchmarking against the old R1-0125 instead of the new R1-0528, including the release of a paper and a distilled version on HuggingFace.
  • The model exhibits a looping and token spamming problem, despite the addition of a length penalty to GRPO.
• Anthropic's Claude Pioneers Dynamic Token Limits: Members pointed out that Anthropic's Claude stands out for its unique dynamic token limit implementation for Chain of Thought (CoT), a problem not yet solved by many others.
  • Nous is working on Hermes 4 to feature user-controlled token limits by teaching the model word, character, and sentence limits during SFT and token limits during RL.
• Investigating Control Tokens for Model Reasoning: The discussion explored the potential of injecting control tokens, such as progress markers (00%, 25%, 50%, 75%), during reasoning to help models dynamically adjust and compress outputs.
  • The goal is to improve the model's ability to split reasoning into search-consolidate-answer phases.
• Decoding the ProRL Paper: The discussion examined the ProRL (Prolonged RL) paper, with some members finding its conclusions unconvincing, especially regarding its applicability to larger models, while noting issues with entropy collapse and reduced sample diversity for shorter CoT.
  • The async overlapped training technique used by Mistral, similar to the PipelineRL approach, was also highlighted (tweet, paper).
• New Mistral Models need Prompt Engineering: Users discussed the new Mistral models where reasoning mode is activated by using prompt engineering, not unlike deep hermes.
  • It seems to be prompt driven with "respond with thinking" and "respond without thinking" and is still experimental.

Nous Research AI ▷ #research-papers (4 messages):

KV Compression, GRPO for TTS LLMs

• New KV Compression Method Surfaces: A new method for KV compression was shared, detailed in this paper and this tweet.
• GRPO Enhances TTS LLMs: It was mentioned that GRPO (Generalized Reweighted Policy Optimization) can be used to improve TTS LLMs (Text-to-Speech Large Language Models), detailed in this paper.

Nous Research AI ▷ #interesting-links (8 messages🔥):

AI Heart Monitoring, Frutiger Aero, Biological Computers

• Arxiv Papers Appear: Two members shared links to two Arxiv papers (2506.06607 and 2502.02260).
  • One member also shared a link to the full video after a snippet was posted (https://www.youtube.com/watch?v=zuZ2zaotrJs).
• AI Heart Monitoring: Life Saving or Life Threatening?: One member quipped that instead of focusing on AI heart monitoring which would save/extend lives, the focus should be on bringing frutiger aero back and implementing ai slop imessage backgrounds.
  • It is unclear if the commenter was joking, serious, or being sarcastic.
• Humans as Biological Computers?: One member expressed that reducing humans to biological computers is clown techbro behavior after watching a snippet of a video.
  • They further added that humans are not the same as bugs n creatures of the expansive earth & depths of the deep sea despite sharing 99+% similar dna.

Nous Research AI ▷ #research-papers (4 messages):

KV Compression, GRPO for TTS LLMs

• New KV Compression Method: A new KV compression method was announced here.
  • More information about the new method can be found on X.
• GRPO Enhances TTS LLMs: GRPO can be utilized to improve TTS LLMs, according to a new paper.

Yannick Kilcher ▷ #general (53 messages🔥):

Diffusion models, Hardware failure prediction, Reservoir Computing, Tolman Eichenbaum Machine

• Diffusion Models: Mind-Blowing Structure From Noise: Members discussed the counterintuitive ability of diffusion models to generate structure from noise, calling it the most mindblowing thing ever and a directed hallucination model.
  • One member linked this to broader themes of order from chaos, referencing a YouTube video and paper on nonequilibrium thermodynamics and the spontaneous emergence of life.
• Hardware Failure Prediction: Beyond DL Solutions: Several members discussed approaches to hardware failure prediction, with a key insight being that traditional methods like Gaussian Processes or boosted trees often outperform deep learning for time series analysis.
  • One member highlighted the narrow scope and high-stakes nature of this field, emphasizing the need for guaranteed failure detection rather than probabilistic correctness due to insurance requirements in industrial settings.
• Reservoir Computing: Obfuscated State Spaces?: A member described Reservoir Computing as mumbo jumbo that obfuscates its core mechanism: linear regression on a fixed Ordinary Differential Equation (ODE).
  • They argued that modern architectures like State Space Models (SSMs) are more expressive, powerful, and efficient due to their ability to parallelize and incorporate nonlinear dynamics and linked to a paper about current SOTA.
• Tolman Eichenbaum Machine: Simplified Implementation: A member announced having trained a simplified version of the Tolman Eichenbaum Machine, condensing the essence of the paper into a handful of functions.
  • They described it as basically a Kalman filter that factorizes state into independent location (g) and sensory appearance (x) components, then saves updated (g, x) pairs into episodic memory, offering to answer any questions about it.

Yannick Kilcher ▷ #paper-discussion (13 messages🔥):

Variational Bayesian approach, World modeling and decision making, Introduction to complex subject, BioML people in berlin

• Variational Bayesian Approach Introduced: A Variational Bayesian approach to simultaneous world modeling and decision making was introduced via the UAB medicine link.
• Introductory Paper for the Math: An introductory paper for the math was suggested via an Arxiv link.
• Introduction to a Complex Subject: One member said that the best introduction to a complex subject is your second introduction to a complex subject.
• BioML People in Berlin to Present: Some bioML people in Berlin are going to come to YK discord to present in the future.

Yannick Kilcher ▷ #ml-news (29 messages🔥):

Mistral AI, Magistral, Open Source, GPT-4

• Mistral Releases Magistral Reasoning Model: Mistral AI announced Magistral, its first reasoning model, which excels in domain-specific, transparent, and multilingual reasoning.
• Open Source Debate Erupts Over Magistral: A user noted that Magistral Small is open-weight under the Apache 2.0 license, but expressed disappointment that Mistral is not open sourcing its larger models, lamenting they became Google level of open weighting.
  • Another user quoted the paper stating they open-sourced Magistral Small which includes cold-start data from Magistral Medium and will not be open-sourcing the Medium model.
• Community Skeptical of Magistral's Open Source Claims: Some community members are calling out that Magistral represents a significant contribution by Mistral AI to the open source community despite not open sourcing all their models.
• OpenAI Teases 'Unexpected' Announcement: Users pointed out Sam Altman teased on Twitter that OpenAI has an unexpected thing coming.
  • Users speculate that its a diffusion model.

Notebook LM ▷ #use-cases (16 messages🔥):

NotebookLM podcast intro, Google Chat integration, Drive file access errors, Video feature release date, Control over Google Workspace document access

• NotebookLM's Intro Stuns Game Designer: A user was surprised and impressed by the quality of the intro generated by NotebookLM for their tabletop RPG, The Gemini System, using the podcast feature.
  • The user found NotebookLM's ability to analyze and provide audio deep dives incredibly helpful for translating mechanics and enhancing their design and writing process.
• Google Chat Convos Coming To NotebookLM?: A user inquired about the possibility of connecting Gmail and Google Chat conversations with NotebookLM and whether there are plans for this feature in the near future.
  • No official response was given, but the query was directed towards any Google employees present in the server.
• Troubleshooting Drive File Download Errors: A user encountered an error when trying to access a Drive file and sought help in resolving it, showing the screenshot here.
  • Another user clarified that the error typically indicates that the file owner has disabled copy/download permissions for the Drive file.
• Video Feature on the Horizon?: Users were curious about the expected release date for the video feature in NotebookLM.
  • No specific timeline or official announcement was provided regarding the availability of the video feature.
• Document Access Control in NotebookLM: A user questioned whether NotebookLM could read content from Google Workspace documents and if it was possible to specify what the AI could access.
  • The discussion focused on the ability to control NotebookLM's access to specific documents and content within Google Workspace.

Notebook LM ▷ #general (57 messages🔥🔥):

Time tracking apps, Iceland workshop feedback, Geographic access issues, Audio overview issues, Sharing notebooks issues

• Time Tracking Quest yields Quest-ions: A user is looking for a simple time tracker app with a start/stop button and streak tracking for studying and simple projects, finding existing options like Toggl and ClickUp too complicated.
  • The user mentioned considering coding their own time tracker app.
• Icelandic Teachers Love NotebookLM but Three Get Blocked: A user ran a NotebookLM workshop for 50 teachers in Iceland, receiving amazing feedback, but 3 teachers using private Gmail accounts encountered a "You do not have access to this service" error.
  • It was suggested that geographic restrictions or incomplete age verification could be the cause, with a user in the UK reporting similar issues with Brave browser, resolved by switching to Firefox.
• Is NotebookLM Glitching on Calculations?: A user highlighted a calculation issue where summing a list of numbers resulted in an incorrect total, displaying an extra 100, prompting comparisons to similar issues with Apple calculations.
  • Another user confirmed that the calculation was correct on Android.
• Audio Overview Audio Overload?: Users are reporting issues with the "could not load audio overview" error when trying to listen to podcasts on the Android app, but the web version works.
  • One user noted the audio quality changing in the second minute.
• Sharing Notebooks Sharing Headaches: A user reported issues with sharing notebooks, where added emails and "Anyone with link" settings revert to restricted after sending.
  • This seems to be a persistent issue for them.

GPU MODE ▷ #general (4 messages):

deepwiki, GLSL, Vulkano, GPU grouping, clustering algorithm

• Deepwiki link summarizes GitHub repos: A member asked about a tool that summarizes GitHub repos and allows chatting and structure viewing from just a GitHub link.
  • Another member suggested deepwiki as a possible solution.
• Rustacean seeks help with GPU algorithm: One member is developing a parallel GPU grouping/clustering algorithm in GLSL and Vulkano using Rust.
  • The developer is looking for collaborators, emphasizing that the project uses Vulkano, compatible even with Macs.

GPU MODE ▷ #triton (6 messages):

FP16 support, Triton.Config num_warps control, Triton shared memory limits, LeetGPU challenges with Triton precision issues, Triton ROCm libdevice.round error

• FP16 Precision Functions Spark Interest: A user inquired about Triton's support for fp16 exp and sqrt functions, noting their availability in CUDA.
• num_warps Configuration Parameter Explained: A user sought clarification on the role of num_warps in Triton.Config and guidance on when to adjust it, seeking insight into its impact on performance and resource utilization.
• Shared Memory Limits Examined: A question arose regarding whether Triton adheres to the same shared memory allocation limits as CUDA when performing tl.load operations.
  • Specifically, the user wondered if tl.load places tensors in a different memory space to avoid exceeding shared memory limits.
• LeetGPU Matmul Kernel Faces Precision Problems: A user encountered precision issues in a matmul kernel written for LeetGPU challenges, which works in interpret mode but fails on GPU despite using float32, sharing their matmul.py file.
  • They asked about the cause of the failure and whether their 2D grid implementation already incorporates swizzling.
• Triton ROCm's Odd libdevice.round behavior: A user reported that while libdevice.round is defined in Triton ROCm, it throws an error when used in a kernel.
  • Another user noted that this issue has been reported on GitHub.

GPU MODE ▷ #cuda (3 messages):

CUPTI, Performance Counters, nvbench

• CUPTI's Metrics Captured with Disabled Performance Counters?: A member asked if CUPTI can be used to effectively capture metrics in a machine that doesn't have performance counters enabled.
  • Another member responded they don't think so, adding that they had to disable CUPTI (in CMake) for nvbench to run benchmarks (using CUPTI metrics) on HPC Clusters without erroring out.
• CUPTI API a Pain: A member said they managed to get some simple metrics to report which surprised them.
  • They added that it's a pain trying to use the CUPTI API so they didn't bother looking for more complicated metrics.

GPU MODE ▷ #torch (11 messages🔥):

functorch, FSDP2, torch.compile with custom operators

• Functorch Usage Flashback: A user suggested using functorch to achieve functional modules, providing example code using make_functional.
  • Another user admitted to forgetting about functorch, and wondered if it had been integrated in this issue.
• TorchTitan flattens dp_shard and cp: In torchtitan, dp_shard and cp are flattened into a dp_shard_cp mesh dimension for FSDP2 mesh dimension, which may introduce unnecessary communication overhead.
  • One user provided a link to their small profiling analysis of dp_shard and cp in this comment.
• nn.Linear Single-Line Code: One user suggested simplifying the code to create a layer in a single line like so: layer = nn.Linear(weight=param).
  • The goal is to make the code more readable and reduce boilerplate.
• Guidance on custom operators and torch.compile is sought: A user seeks advice on custom operators and torch.compile, specifically regarding shape checking and best practices.
  • They opened an issue on GitHub here to initiate a discussion on the topic.

GPU MODE ▷ #jobs (1 messages):

NeoSpace, GB200, CUDA, Brazil

• NeoSpace in Brazil Needs CUDA Experts: An AI company named NeoSpace based in Brazil is hiring professionals with expertise in GPU optimization using CUDA.
  • They're training models using GB200 GPUs and prefer on-site work; interested candidates should contact [email protected] with resume and the subject 'Neospace CUDA position'.
• NeoSpace Actively Recruits CUDA Specialists: Brazilian AI firm NeoSpace seeks CUDA experts for GPU optimization roles, utilizing GB200 GPUs in their model training.
  • On-site positions are preferred; applications should be directed to [email protected] with the subject line 'Neospace CUDA position', including a resume.

GPU MODE ▷ #irl-meetup (1 messages):

ossmar: Does someone here is attending to the ACM PODC 2025?

GPU MODE ▷ #rocm (9 messages🔥):

SQTT traces, Radeon GPU Analyzer (RGA), rocprofv2, CUDA graphs, Memory access fault

• Troubleshooting CUDA Graphs Memory Faults: A user encountered a Memory access fault by GPU node-2 error when using CUDA graphs with the rocm/pytorch:rocm6.4_ubuntu22.04_py3.10_pytorch_release_2.6.0 image and torch 2.8.0.dev20250609+rocm6.4.
  • The user inquired if this was a known issue, noting they hadn't experienced it with previous versions.
• Profiling with Radeon GPU Analyzer: A user detailed the steps to collect SQTT traces for analysis in Radeon GPU Analyzer (RGA) using rocprofv2, including creating a configuration file and running rocprofv2 with SQTT capture enabled.
  • They noted that the correct DISPATCH_RANGE can be determined by first running rocprofv2 --kernel-trace.
• Seeking Per-Instruction Timings with RGA: A user reported partial success with RGA, but expressed uncertainty whether RGA shows per-instruction timings, and they encountered issues when opening a trace of a Triton kernel.
  • They planned to update to ROCm 6.4.1 to try rocprof compute viewer or rocprof3's att.
• Confusion between RGA and RGP: A user suggested trying Radeon GPU Profiler (RGP) but then stated it requires .rgp files which are incompatible with HIP programs from Linux.
  • Another user acknowledged the suggestion but noted that RGP requires .rgp files, which are not compatible with profiling HIP programs from Linux.

GPU MODE ▷ #liger-kernel (1 messages):

Liger Collective Library, ByteDance Triton-distributed

• Liger Considers Collectives Library: A member inquired about adding a collectives library to Liger, similar to the ByteDance Triton-distributed library.
• More on Collectives Libraries: The collectives library would be similar to another collectives library.

GPU MODE ▷ #self-promotion (3 messages):

Mojo Programmers on BlueSky, Modular raises funding

• Mojo Programmers assemble on BlueSky!: A member is compiling a list of Mojo programmers on BlueSky to create a starter pack and is looking for volunteers.
  • The member believes there are dozens of them.
• Fundraising news: Members are excited for new funding and the possibilities that come with it.
  • Many are posting waiting for money to come in

GPU MODE ▷ #🍿 (2 messages):

Dataset Generation, Diverse Datasets, Augmented Datasets

• New Dataset Generators emerge!: A member shared a new dataset generator project, noting they lost count of how many times they've had to write new ones and that it was time to create one.
  • They added that the getting started examples work very well and it supports creation of diverse, from scratch or augmented datasets in a few lines of code.
• Future dataset plans: The member still needs to clean up the process for creating more complex stuff.

GPU MODE ▷ #reasoning-gym (1 messages):

RL, Reasoning Training, Magistral Paper

• Magistral Paper Explores RL and Reasoning Training: The new Magistral paper delves into valuable insights on Reinforcement Learning (RL) and reasoning training methodologies.
• Reasoning Training Boosted by Magistral: The paper highlights how reasoning training techniques contribute to enhanced model performance, particularly in complex tasks and decision-making scenarios, underscoring the significance of the Magistral paper's findings.

GPU MODE ▷ #general (10 messages🔥):

Hackathons, Benchmarking, CUDA events

• Hackathon Newbie Arrives: A new member arrived asking about active hackathons, mentioning they came via a Datamonsters AMD challenge that has passed.
  • A member responded, there aren't any current "running" hackathons with prizes, but the AMD and PMPP problem sets are open for submissions.
• GPU Mode Benchmarking Methodology Exposed: A member inquired about the exact benchmarking methodology for the leaderboards, particularly for tasks like matmul.
  • Another member provided a link to the gpu-mode/discord-cluster-manager eval.py script, along with a link to tolerances defined in gpu-mode/reference-kernels.
• CUDA Events Improve Benchmarking Accuracy: A member expressed a preference for benchmarking using CUDA events for better accuracy, referencing triton's do_bench function.
  • Another member acknowledged the limitation, noting they would likely address it before the next big competition, and clarified that the times displayed are the min/max times.

GPU MODE ▷ #submissions (2 messages):

Chinese problem-solving approach, New Bilibili article

• Chinese problem-solving approach unveiled on Bilibili: A member highlighted a problem-solving approach described in a Bilibili article.
  • The post describes the approach in Chinese, offering insights into its application.
• Bilibili Article Sparks Discussion: The shared Bilibili link prompted a discussion about problem-solving methodologies.
  • Community members showed interest in understanding the nuances of this specific approach.

GPU MODE ▷ #factorio-learning-env (1 messages):

Roadmap

• Roadmap Request Re-Pinged: A member re-pinged their message to inquire about a potential roadmap for a project or feature.
  • They were unsure if a roadmap already existed and were seeking awareness on the matter.
• Another Roadmap Request: Another member also inquired about a roadmap.
  • They were also unsure if a roadmap already existed.

GPU MODE ▷ #cutlass (2 messages):

CuTE docs, Cutlass, Triton

• CuTE Docs recommended for Cutlass learners: A member asked if CuTE docs are the best place to get started on learning Cutlass for someone with a Triton background.
  • Another member suggests using the notebooks provided in the examples part and using the docs as a reference for what's really going on under the hood.
• Cutlass and Triton Backgrounds: The user with a Triton background sought advice on learning Cutlass effectively.
  • The community recommends leveraging example notebooks and the documentation for understanding the underlying mechanics.

GPU MODE ▷ #mojo (1 messages):

Modular + AMD, Python Interop

• Modular and AMD Join Forces: Modular announced today a new partnership with AMD to unleash AI performance on AMD GPUs, as seen in their blog post.
• Python Plays Well with Mojo: A demo showcasing Python interoperability with Mojo was shared, specifically at the 843-second mark of this YouTube video.
• Mojo has official Documentation: Documentation for Mojo has been released including full details on its Python integration.

Latent Space ▷ #ai-general-chat (56 messages🔥🔥):

Fireworks AI RFT Beta, OpenAI o3 Pricing, Mistral's Magistral Model, Meta's Potential Scale AI Stake, DeepSeek Model Narrative

• RFT Beta Launches on Fireworks AI: Lin Qiao announced the beta launch of Reinforcement Fine-Tuning (RFT) on Fireworks AI, allowing users to train expert open models with quality comparable to closed frontier models like GPT-4o mini and Gemini flash.
  • The service is designed for rapid iteration with a web IDE, an open-source reward-kit, support for SOTA models, and is self-serve and free for the next two weeks for models up to 10B parameters.
• o3 Token Costs Shared by OpenAI: Gabriel Chua noted a potential cost of $2 per 1M input tokens for OpenAI o3, referencing an OpenAI Developers tweet offering 200 developers free API credits worth 1M input tokens.
  • The discussion included links to Sam Altman's tweet and further insights on scaling01.
• Mistral Releases Magistral Reasoning Model: Mistral AI announced the release of Magistral, their new reasoning model for domain-specific, transparent, and multilingual reasoning, available in two variants: open-source Magistral Small (24B parameters) on Hugging Face, and enterprise Magistral Medium via chat.mistral.ai or API.
  • Reactions were positive, praising the model's release and name, with some users providing instructions for local deployment and noting its availability on platforms like OpenRouter.
• Meta Eyes Scale AI's Alex Wang for Top Role: Meta Platforms might acquire a 49% stake in Scale AI for nearly $15 billion, potentially bringing Scale AI's CEO, Alex Wang, into a senior position at Meta (Source).
  • This move could significantly impact Meta's AI strategy and leadership.
• Windsurf Launches 'Plan Mode' Feature: Kevin Hou introduced Windsurf's new 'Plan Mode' feature, enabling the AI agent to perform complex tasks by creating and maintaining a planning document (Source).
  • Users can toggle 'Plan Mode' on to allow Windsurf to manage notes, task lists, and goals, enhancing its ability to handle longer, more involved changes, available for free on Windsurf.com.

Modular (Mojo 🔥) ▷ #announcements (1 messages):

Modular Livestream, Compute Portability

• Modular Kicks off Livestream: Modular announced that their livestream is kicking off in 5 minutes on the Modular website.
  • The livestream can also be accessed on LinkedIn.
• Compute Portability Talk: The Modular livestream is focused on the future of compute portability.
  • The event promises insights into the latest advancements and discussions in compute portability.

Modular (Mojo 🔥) ▷ #mojo (52 messages🔥):

Mojo parameterization limits, Zig vs Mojo, Python Syntax similar to Go, Mojo-MAX Platform relationship, Double copy explaination

• Community Showcases Parameterization Exploits: Community member presentations and glimpses into the standard library code have sparked questions about the boundaries of parameterization in Mojo, particularly regarding its use for comptime purposes.
  • One member expressed concern that the exploitation of parameterization for comptime purposes seems to create code they just do not want to read in a lot of cases.
• Mojo Meta-Programming > Rust Macro: One member argued that reading meta-programming in Mojo is 100000000000% better than reading macro code in rust, while acknowledging that Mojo can't do everything Rust can yet.
  • Another member thinks it's the combination of Zig-esque comptime in Go-esque square-bracket-generics syntax that makes it difficult to read.
• Mojo copies Python Generics: A member stated that Mojo's syntax for generics is the same as Python's, which inspired a discussion to whether Python's generic syntax came from Go.
  • Ultimately, the parties agreed that Go 1.18 introduced generics on 3/15/22, and PEP 695 introduced the new Python syntax on 6/22.
• Clarify Mojo-MAX Platform relationship: A member asked about the relationship between the Mojo and MAX platform, specifically the ability to use MAX kernels such as matmul in Mojo code and kernels.
  • A Modular employee suggested that the member post this question in the Modular forum to enhance its discoverability.
• Double copy explained: A member questioned why assigning a ref variable to another triggers __copyinit__ twice and why ref gets an extra __moveinit__ step while var doesn't.
  • Another member clarified that the double copy happens because there's effectively a tmp variable inserted to handle the assignment and a compiler explorer link provided.

MCP (Glama) ▷ #general (41 messages🔥):

MCP Server Selection, Building n8n for MCP, FastMcp Dependencies, Mature MCP SDK, MCP file downloads

• 5ire forces MCP tool adoption: A user noted that the 5ire platform requires adopting all tools from an MCP server, offering no option to pick and choose individual components.
  • This all-or-nothing approach means developers must integrate entire suites of functionalities rather than selecting specific tools they need.
• n8n-like Chatbot Integration Dream: A member expressed interest in building a tool like n8n, but based entirely on chats and MCPs to automate workflows based on chat interactions.
  • Another member proposed a workflow where emails from a specific source can be routed into a Slack channel, highlighting the potential of such a system.
• fastmcp needs dependency declaration: A member struggled with fastmcp, noting that it needs the dependencies declaring dependencies because it creates an environment and uses the dependencies to do that.
  • They shared a command line to execute the MCP server and then they updated the args in Claude desktop so that uv knows which venv to use.
• Github and Amazon have official python SDKs: A member asked for the most mature official SDK/repo out there for MCP server development.
  • Other member mentioned Github and Amazon have SDKs for the same.
• File downloads challenges in MCP: A member asked for the best way to handle file downloads via MCP, as sending entire files as base64 strings requires loading the entire file into memory which isn't ideal.
  • Another member shared an interesting MCP server implementation attempting to implement the whole protocol, and you can connect to remote servers using OpenAI assistants API or Anthropic.

MCP (Glama) ▷ #showcase (10 messages🔥):

Glama build system details, MCP OpenMemory demo, OAuth 2.1 module for MCP servers, MCP servers for *arrs, mcp-openverse npm package

• Glama's Build System Unveils Every Detail: Glama's new build system provides detailed information about the build and container logs, as depicted in a screenshot.
• MCP OpenMemory Demo Shines: A member shared a demo of MCP OpenMemory and a link to the GitHub repo, encouraging others to star it.
  • The demo video showcases the project in action, highlighting its capabilities.
• Scalekit Ships OAuth 2.1 Module for MCP Servers: Scalekit has launched a drop-in OAuth 2.1 module with scoped, short-lived tokens, DCR + PKCE, and 401s with authorize_url for delegated flows for MCP servers, as detailed in their documentation.
• *Arrs Assemble in MCP Servers: Links to various MCP servers were shared, including radarr-mcp, sonarr-mcp, and others, split from a single container for individual use.
• mcp-openverse Package Opens CC-Licensed Images: A member announced the release of mcp-openverse, an MCP server that brings CC-licensed and public domain images to AI workflows, available on npm and GitHub.
  • This tool searches over 700M+ openly-licensed images from @WPOpenverse, integrates with Claude Desktop, and offers smart image sourcing with concept extraction.

Manus.im Discord ▷ #general (50 messages🔥):

Manus pricing, Veo 3, EDU email accounts, Growth person Mixedbread, AI Search infra

• Mixedbread Seeks Founding Growth Person: Mixedbread, a team of ex-Google Search engineers, is seeking a founding growth person in SF to convert their technical traction to $10M ARR and build an SF team ASAP.
  • The company boasts 50M+ HuggingFace downloads, beats OpenAI on MTEB benchmarks, and is backed by top AI investors from OpenAI, Vercel, Perplexity, Deepmind, and Scale AI.
• Manus still in Beta gets VEO 3: Members are questioning why Manus is still in Beta, even with features like Veo 3 and other cool updates.
  • One member reported wasting 2000 credits due to presentation formatting issues and no refund.
• Is Manus Pro worth it: Users are inquiring about the value of a Pro subscription to Manus, specifically if the answers are significantly better and worth the cost.
  • Several people reported having problems reaching Manus support.
• User creates Sci-Fi short film using Manus's VEO 3!: A user created a five-minute sci-fi short film using Manus's Veo3 feature calling it the most powerful generation function in the world.
  • Another user said it looks great, and intentionally like old school Kung Fu movies.
• User lost 300 credits to Veo3!: One user spent 300 credits on one Veo3 video and has 38 clips
  • Another user is asking for 100 credits after Manus went on loop trying to truncate a file.

LlamaIndex ▷ #blog (5 messages):

Custom Multi-Turn Memory, Real-time Website Summaries, LlamaIndex Agent as MCP Server, Databricks Data + AI Summit, Knowledge Agents to Automate Workflows

• Custom Multi-Turn Memory Debuts for Agents: LlamaIndex introduced a new example for building custom multi-turn memory implementation, ideal for agentic workflows needing control and customization, see this tweet.
• Instant Summaries While You Browse: A project by @itsclelia combines web browsing with AI-generated summaries of websites using LlamaIndex and Google's Gemini model, get details here.
• LlamaIndex Agent Now MCP Server: LlamaIndex demoed turning an agent into an MCP server, deploying a custom FidelityFundExtraction workflow for extracting structured data from complex multi-fund PDFs, then invoked it from Claude, as reported here.
• LlamaIndex at Databricks Summit: LlamaIndex is present at the Databricks Data + AI Summit at Booth D117 in the AI Pavilion, ready to assist with agentic AI journeys, per this tweet.
• CEO Talks Knowledge Agents: LlamaIndex CEO Jerry Liu hosted a breakout session on Building Knowledge Agents to Automate Document Workflows at the Databricks Data + AI Summit, and will be repeating the session due to demand, details here.

LlamaIndex ▷ #general (14 messages🔥):

Agent Workflow, Handoff Issues, DirectOutputAgent, Multi-Agent Systems, OpenAI Agents SDK

• Agent Workflow Woes: Handoff Hang-Ups: A user is experiencing issues with their LlamaIndex-based product recommendation system using an agent workflow where the plan_agent sometimes fails to hand off to other agents like DirectOutputAgent or SearchAgent.
  • Logs indicate that the streaming stops without a clear reason, and the user is seeking assistance to understand why the handoff doesn't occur consistently.
• RAG Reliance: Prompt Engineering Path: A user asked about ensuring user queries are exclusively answered by RAG (Retrieval-Augmented Generation) in LlamaIndex, attempting to control this via system context and chat mode settings.
  • Another member suggested that prompt engineering is the primary method, and a secondary LLM call could inspect sources to judge the answer's quality.
• Multi-Agent Mania: LlamaIndex vs OpenAI SDK Showdown: A user inquired about the capabilities of LlamaIndex compared to the OpenAI Agents SDK for building multi-agent systems, specifically regarding integration with OpenAI and tracing in the OpenAI dashboard.
  • The member clarified that Arize can be used for tracing with LlamaIndex, although it doesn't directly integrate with OpenAI's tracing tools.

LlamaIndex ▷ #ai-discussion (6 messages):

Open Source Deep Research, Long Context Generation, Local Machine Research, spy-search Github repo

• Open Source Tool for Deep Research: A member introduced spy-search, an open-source tool that supports Ollama and enables deep research on a local machine.
  • The tool is designed to generate long reports, exceeding 1000 words, offering a more comprehensive alternative to research tools with shorter outputs.
• Spy-Search Generates Long-Context Responses: Spy-search aims to provide long-context responses with the latest information, similar to Perplexity, but as an open-source solution.
  • The member invited the community to search for the spy-search repository on GitHub for those concerned about opening the direct link.

Cohere ▷ #🧵-general-thread (15 messages🔥):

Cohere support channels, Cohere Open Science Community

• Cohere's Quicker Support Channel Launched: A member announced a new support channel, promising faster assistance using an AI-generated reply bot leveraging Cohere's documentation, available at <#1381756280716132412>.
  • The bot, built with command-a, focuses on documentation-based queries, directing account and API issues to [email protected]; misuse (prompt injection, etc.) will result in an instant ban.
• Open Science Application Approvals Approaching: A member inquired about the timeline for acceptance into the Cohere Open Science Community after submitting their application.
  • Another member responded with that they should let you know some time soon.

Cohere ▷ #📣-announcements (2 messages):

Cohere North, GameWarden Integration, EnsembleHP Partnership

• Cohere North Integrates with GameWarden Platform: Cohere North now securely integrates with the full GameWarden platform through a partnership with Second Front, to help service members gain unprecedented effectiveness and speed against an ever-evolving threat landscape, as announced in this tweet.
• Cohere North Partners with EnsembleHP: Cohere is bringing Cohere North to healthcare by partnering with EnsembleHP to reduce administrative friction and elevate patient experience in hospitals and health systems with their secure AI agents platform, detailed in this blog post.

Cohere ▷ #🔌-api-discussions (4 messages):

Open Source Repo for Contributions, API Tier Discussion, Reranking API Latency

• Cohere has Open-Source Repo for Contributions: Cohere has an open-source repository, the Cohere Developer Experience GitHub repository, where users can contribute by submitting pull requests to improve the documentation content.
  • The repository's README file offers more guidance on contributing; OpenAPI specs and snippets are one-way synced from internal repositories.
• API Tier Talk Sparks Solutions: A user inquired if Cohere has API tiers similar to OpenAI, mentioning a 2-second latency with the reranking API and seeking improvements.
  • While Cohere doesn't offer tiers, there are other solutions, and the user was advised to contact [email protected] for assistance.

Cohere ▷ #👋-introduce-yourself (3 messages):

Vitalops, Datatune, Open Source Tools, Data Transformations, Natural Language Data Transformation

• Vitalops Co-Founder Joins Community, Shares Datatune: The co-founder of Vitalops introduced themselves, expressing excitement about joining the community.
  • They are developing Datatune, an open source tool designed for data transformations using plain natural language.
• Datatune: Transforming Data with Natural Language: Datatune, created by Vitalops, is an open-source tool that allows users to perform data transformations using natural language.
  • The co-founder is eager to engage with the community and gather feedback on Datatune's development and potential applications.

Cohere ▷ #🔔-ping-settings (1 messages):

competent: Moved to id:customize

Torchtune ▷ #dev (15 messages🔥):

HuggingFaceModelTokenizer Usage, Muon Performance in torchtune, Tokenizer truncation bugs, Kimi Moonlight paper, Qwen2

• HuggingFaceModelTokenizer Intended Usage Debated: Members discussed the intended usage of HuggingFaceModelTokenizer, with concerns raised about the interface differences and how to handle max_seq_len for packing, in particular whether to change recipes or the tokenizer itself.
  • One member suggested a solution where the recipes should be changed to take a max_seq_len which is then passed through, which is close to the HF pattern and aligned with this proposal.
• HF Tokenizer Integration Faces Hurdles: After testing the HF Tokenizer, it was found that loss curves and total tokens don't align with the classic tokenizer, suggesting differing behavior despite minor code changes; the integration will be ready after issues #2794 and #2574 are addressed.
  • Also the member reported that pre-packing takes 2-3 times longer.
• Tokenizer truncation bugs found: Bugs were found in truncation while implementing the tokenizer, with related points highlighted in issue #2792 with concerns that this can affect performance.
  • Members suggest sticking with the original tokenizers for training for now, awaiting consolidation to the HF ones later.
• Muon Integration Performance Scrutinized: The performance benefits of Muon, when integrated into torchtune, are being scrutinized, with one member wanting to see the performance benefits when it is integrated in torchtune to justify adding another abstraction, and another member wondering if issue #2809 is critical.
  • A member points out that there's some evidence that muon is more useful for finetuning models that were also pretrained with muon, referencing the Kimi Moonlight paper.
• Qwen2 Issue Check: A member has to check Qwen2 to see if the truncation issue exists there also.
  • It was acknowledged that if the original testing didn’t find the difference, it probably doesn't make a big difference but we should fix it either way.

DSPy ▷ #general (8 messages🔥):

Transfer learning, DSPy documentation, DSPy 3 announcement, Context optimization in DSPy, Dataset building and export tools

• Transfer Learning techniques asked about: A member inquired about the possibility of transferring post-training learning from one model to another without repeating the learning process, like finetuning or RL.
  • No specific answers were provided in the channel.
• DSPy Documentation Vanishes: A user noted that a documentation file was removed in a recent PR and expressed difficulty finding the same level of parameter documentation elsewhere.
  • Another user shared a link to AI-generated docs that might be helpful.
• DSPy's Contextual Prompt Optimizer: A user asked if DSPy has anything to optimize what context to include in a prompt when a dozen variables are available.
  • They wanted to see which combination leads to good metrics balanced against the token usage of the prompt, but there was no further discussion on the topic.
• Dataset building and export tools for DSPy: A member asked about tools to easily build up and export datasets for use with DSPy.
  • They specifically mentioned needing something that allows generating a few dozen synthetic examples and then hand labeling them, but there was no further discussion on the topic.

tinygrad (George Hotz) ▷ #general (8 messages🔥):

Failing Tests, Bounty Locked Meaning, NCHWCPUGraph / LLVMGraph Refactor

• Tests Failing, Bounty Locked Blocked: Members reported that the tests are failing, which means that a bounty cannot be locked (ready to merge).
  • It was clarified that bounty locked means basically ready to merge, or serious tested progress toward a goal, but failing CI will prevent it.
• Call for AI-Free PRs: A member requested tasteful PRs, like the one addressing add/mul at tinygrad/tinygrad#10741, explicitly stating no AI slop.
  • It was noted that add/mul were the easiest ones.
• NCHWCPUGraph and LLVMGraph Demand Refactor: It was suggested that NCHWCPUGraph / LLVMGraph really need to be changed to behave like the other graphs.
  • These graphs shouldn't be rerendering stuff, and this is related to both multicore CPU and the multi compiler/renderer refactor where CPU and LLVM should use the same graph since they have the same program.

Nomic.ai (GPT4All) ▷ #general (5 messages):

Nomic Embed Text v1.5, Nomic GPT4All future versions, Python SDK update, GPT4All support for Mistral's Magistral Small

• Nomic Embed Text v1.5 Still Supported: A user asked if nomic-embed-text-v1.5 can still be used from Nomic cloud next month and another user confirmed that the model is still supported for self-onboarded inference.
• Future of Nomic GPT4All: A user inquired about updates on future versions of Nomic GPT4All.
• Python SDK Update Anticipation: A user asked if there is an update coming on the Python SDK.
• GPT4All and Mistral's Magistral Small Integration: A user questioned whether GPT4All will support Mistral's Magistral Small.

Gorilla LLM (Berkeley Function Calling) ▷ #leaderboard (2 messages):

Leaderboard Updates, GPU Resources

• RunPod Engineer Extends Helping Hand for Leaderboard Revival: A RunPod DX engineer offered assistance to restart leaderboard updates, including providing GPU resources.
  • The engineer encouraged direct messages for anyone needing help with resources to get the leaderboard operational again.
• Community Expresses Gratitude for RunPod's Generosity: Several members expressed their appreciation for the offer of GPU resources from the RunPod engineer.
  • The offer was seen as a significant boost to the community's efforts in maintaining and improving the leaderboard.

Gorilla LLM (Berkeley Function Calling) ▷ #discussion (1 messages):

Agent Marketplace Status

• Agent Marketplace Faces Access Issues: A member inquired whether the Agent Marketplace is still operational, noting difficulties in accessing its repository and webpage.
• Agent Marketplace Potentially Closed?: The member also questioned if the project might be temporarily closed due to these access problems.

LLM Agents (Berkeley MOOC) ▷ #hackathon-announcements (1 messages):

Agentic AI Summit, Early Bird Tickets, UC Berkeley, Speaker announcements

• Agentic AI Summit Announced: The Agentic AI Summit will be held at UC Berkeley on August 2, 2025, building on the popular LLM Agents MOOC with 1,500+ in-person attendees and thousands of virtual participants.
  • The summit website includes details on how students or indie developers can apply for discount codes.
• Early Bird Tickets End Soon!: Early bird pricing for the Agentic AI Summit ends on June 30, 2025, with student passes at $25, startup passes at $60, and industry professional passes at $80.
  • Tickets can be purchased here.
• Speakers Announced for Agentic AI Summit: Featured speakers at the Agentic AI Summit include Vinod Khosla (Khosla Ventures), Ion Stoica (Databricks and Anyscale), Dawn Song (UC Berkeley), Sergey Levine (Physical Intelligence), Matei Zaharia (Databricks), Karthik Narasimhan (Sierra), Waseem AlShikh (Writer), and Burak Gokturk (Google Cloud).

LLM Agents (Berkeley MOOC) ▷ #mooc-questions (1 messages):

SP25 Course, Quiz Questions

• SP25 Course Quiz Access Query: A user inquired about accessing quiz questions for the completed SP25 course for self-learning purposes, noting the course is currently not in session.
• Request for SP25 Quiz: A user has requested the availability of SP25 course quiz questions to study independently since the course session has ended.

Codeium (Windsurf) ▷ #announcements (1 messages):

Planning Mode, Windsurf Wave 10, o3 model pricing

• Windsurf Waves into Planning Mode: Windsurf has released Planning Mode as part of Wave 10, featuring a native interface for long-term AI planning with bidirectional updates and synergy between long-term and short-term reasoning models, as detailed in their blog and demonstrated in this video.
• Cascade's conversation gets live markdown plan: Users can toggle Planning Mode via the icon under the prompt box, enabling Cascade to pair every conversation with a live markdown plan of goals & tasks.
  • AI notifications alert users when Cascade updates the plan, facilitating a collaborative planning process.
• O3 Model Credit Pricing slashed!: The o3 model is now available for just 1x credits and runs faster within Cascade, enhancing both cost-effectiveness and performance.
  • Planning Mode is accessible on all paid plans without additional charges.