a quiet day

AI News for 7/22/2025-7/23/2025. We checked 9 subreddits, 449 Twitters and 29 Discords (227 channels, and 9736 messages) for you. Estimated reading time saved (at 200wpm): 748 minutes. Our new website is now up with full metadata search and beautiful vibe coded presentation of all past issues. See https://news.smol.ai/ for the full news breakdowns and give us feedback on @smol_ai!

The White House announced their AI Action Plan, but we’ll keep this newsletter technical. As commented yesterday, QwenCoder has had a largely positive reception but not hugely so that we’d make it a title story.

AI Twitter Recap

New Model Release: Qwen3-Coder

• Launch and Performance Claims: @Alibaba_Qwen announced the release of Qwen3-Coder-480B-A35B-Instruct, an open agentic code model with 480B total parameters (35B active) and a 256K context length. Initial reports claimed SOTA performance, with @itsPaulAi calling it “one of the best coding models we’ve ever seen.” The model was noted for being developed in just three months, as highlighted by @scaling01. @AravSrinivas celebrated the release, stating, “Incredible results! Open source is winning.”
• Benchmark Controversy: A key point of contention arose around benchmark scores. While the official release claimed 41.8% on ARC-AGI-1, @fchollet stated his team was unable to reproduce this score on either public or semi-private evaluation sets, finding its performance more in line with other recent base models. He urged reliance only on scores verified by the ARC Prize foundation for consistency. @GregKamradt also publicly inquired about reproducing the results.
• Ecosystem Integration: The model was quickly integrated across the ecosystem. @vllm_project announced support in vLLM nightly with expert parallelism. @UnslothAI began uploading Dynamic GGUFs with up to 1M context length. It was also made available on @OpenRouterAI, @cline, and @togethercompute. A web development space to try the model was also highlighted by @ClementDelangue.
• Technical Analysis: @rasbt commented that this release demonstrates that for coding, “specialization wins” over general-purpose models. @cline observed that Qwen3-Coder surpassed Kimi K2 in less than two weeks with half the size and double the context, suggesting open-source models are reaching “escape velocity.”

US AI Policy and Geopolitics

• America’s AI Action Plan: The White House released a new AI Action Plan focused on “winning the AI race.” @scaling01 provided a detailed summary, outlining its three pillars: Innovation, Infrastructure, and International Diplomacy. Key directives include revising the NIST AI Risk Management Framework, ensuring government contracts with developers of objective models, and promoting “open models founded on American values.”
• National Security and Infrastructure: The plan explicitly links AI dominance to national security, with @scaling01 noting it grants the Department of Defense (DOD) priority access to compute resources during a national emergency. It also highlights that “American energy capacity has stagnated since the 1970s while China has rapidly built out their grid,” calling this a trend that must be changed for AI dominance. The plan also details measures to counter Chinese influence and impose export controls on sensitive technologies.
• Open vs. Closed Source Debate: The plan’s release intensified the debate on open-source AI. @ClementDelangue argued it’s time for the American AI community to “drop the ‘open is not safe’ bullshit” and return to open science to avoid losing the AI race. This was contrasted with the observation from @Yuchenj_UW that the “US…ships only closed-source AI” while “China…ships only open-source AI.” @Teknium1 highlighted that the plan encourages the development of “open-weights” AI models.

Model Updates, Research, and Techniques

• Subliminal Learning in LLMs: A paper from @OwainEvans_UK and Anthropic Fellows introduced the concept of “subliminal learning,” where LLMs can transmit hidden traits to other models through data. This prompted discussion on its implications, with @swyx suggesting it could be a powerful “Soft Power tool” for exporting value systems, and @giffmana interpreting it as a study on generalization and distillation.
• Gemini Updates: @OfficialLoganK announced that Gemini 2.5 Flash-Lite is now stable and ready for production use. @sundarpichai highlighted its performance at 400 tokens/second and cost-efficiency. In a major achievement, @GoogleDeepMind revealed that Gemini with Deep Think achieved a gold-medal standard at the International Mathematical Olympiad (IMO).
• New Audio and Text-to-Speech (TTS) Models: @reach_vb shared the release of Higgs Audio V2 from @boson_ai, an open, unified TTS model with voice cloning that reportedly beats GPT-4o mini TTS and ElevenLabs v2. @reach_vb also showcased its ability to perform multi-person generation with voice cloning from a single model. Mistral AI also released the Voxtral Technical Report.
• Other Notable Releases & Research: Kimi K2 from Moonshot AI was noted for hitting #1 on Chatbot Arena, with the company now actively hiring for multiple roles. Neta AI launched Neta Lumina, an open-source anime model. Research from @StellaLisy explored decomposing human decision-making beyond black-box preference models.
• RL and Context Engineering: @shaneguML shared insights on why he pursued RL in 2016 after backpropagation failed him. @omarsar0 emphasized that clever memory management and context engineering are what’s falling short in current coding models, not raw model capability.

AI Tooling, Frameworks, and Infrastructure

• Perplexity Comet Browser: @AravSrinivas sparked discussion by asking if people will still be using Chrome in 2030, in the context of Perplexity’s Comet browser. He highlighted Comet’s superior memory management over Chrome and its ability to let users search over everything like an agent would. He also clarified that ad-blockers work natively without extensions.
• Claude Code as an “Everything Agent”: A strong sentiment emerged around Claude Code becoming a versatile, powerful tool. @alexalbert__/ declared it “is the everything agent.” Its integration within PostHog was also noted by @swyx.
• Major Infrastructure Deals: In a massive infrastructure play, @sama confirmed OpenAI signed a deal for an additional 4.5 gigawatts of capacity with Oracle as part of the Stargate project.
• Framework and Library Updates:
  • vLLM: The project announced that Vision-Language Models are now supported in its integration with Hugging Face Transformers.
  • OpenCLIP & timm: @wightmanr announced a joint release, with the headline feature being Perception Encoder (PE) Core support in OpenCLIP and NaFlexViT ROPE support in timm.
  • Gradio: It was announced that Gradio is now pre-installed in Google Colab, simplifying the process of creating demos in notebooks.
  • LangChain: @hwchase17 highlighted the new integration of Bedrock AgentCore tools with LangGraph agents.
  • LlamaCloud: @jerryjliu0 introduced new header/footer detection capabilities to ensure clean document context for AI agents.

Companies, Ecosystem, and Broader Implications

• The Future of Human-Computer Interaction: @karpathy shared a photo of a Tesla Supercharger diner, calling it an “exhibit for the future.” @OriolVinyalsML provocatively stated, “We already are” talking to alien intelligences, “just through a fairly narrow communication bottleneck.” Similarly, @DemisHassabis discussed the idea that if AI can learn natural patterns like a protein’s fold, it could unlock new eras of scientific discovery.
• Company Milestones and Funding: Diode, an AI circuit board design company, announced it raised an $11.4 million Series A led by a16z. Video generation company Synthesia announced it hit its first $1M+ day.
• AI for Science: Google announced Aeneas, a new AI model building on the Ithaca project to contextualize ancient Latin inscriptions. AI at Meta shared its work, published in Nature, on using advanced ML models and EMG hardware to transform neural signals into computer commands.
• Cost of AI: @vikhyatk provided a stark cost comparison of using Sonnet: writing a PyTorch module cost $0.038, while writing a React component cost $33.74.

Humor/Memes

• Cultural Commentary: @Teknium1 shared a video of drones being used to indicate an event exit in Osaka, Japan. A prescient 1981 Shel Silverstein comic was shared by @nptacek.
• Industry Satire: @scaling01 posted a meme captioned, “You’re sheltering chinese AI researchers, are you not?”. @tamaybes joked, “If you give your AI model a French name, it is perhaps not surprising it will be offline 20% of the year.”
• Community In-Jokes: @scaling01 celebrated a like from a prominent researcher with “holy shit Sholto liked my post.” @AravSrinivas quipped that Perplexity Comet has “better memory management than Chrome.”
• Relatable Content: @Yuchenj_UW posted a nostalgic image of an old software UI with the caption, “This is what they took from us 😢”.

AI Reddit Recap

/r/LocalLlama + /r/localLLM Recap

1. Qwen3 and Qwen3-Coder Release Performance, Benchmarks, and User Experiences

• Qwen3-Coder Unsloth dynamic GGUFs (Score: 259, Comments: 72): The image is a promotional graph comparing the performance of Qwen3-Coder—particularly the new 480B parameter variant with dynamic GGUF quantizations (2-8bit, including 182GB 2bit models supporting up to 1M context length)—against other large language models on agentic coding benchmarks. The post highlights strategies for running these massive models efficiently via llama.cpp MoE offloading (CPU and RAM/VRAM mix), flash attention, and KV cache quantization, with relevant resources and full docs here, and GGUF checkpoints on Huggingface. A top comment notes the need for advanced offloading techniques to handle these large models efficiently, underscoring challenges in hardware requirements and the ongoing need for software optimizations.
  • There is discussion about the extremely large model size, particularly the 180 GB Q2_X_L quantization for Qwen3-Coder Unsloth dynamic GGUFs, with one user asking how it compares technically to the Q4_X_L variant. This highlights trade-offs between quantization levels, file size, and potential inference performance/resource needs.
  • A user mentions the need for “crazy offloading hacks” due to the sheer model size, implying that pushing inference to consumer hardware may require advanced memory management, storage streaming, or multi-GPU/CPU techniques to achieve reasonable inference speed and capability.
• Recent Qwen Benchmark Scores are Questionable (Score: 375, Comments: 66): The attached image shows a tweet by François Chollet, creator of the ARC benchmark, expressing skepticism about the claimed 41.8% ARC-AGI-1 score for Qwen 3, noting his inability to replicate these results on both public and semi-private evaluation sets. Chollet suggests that the reported Qwen 3 numbers align more closely with other recent models (implying exaggeration or misreporting), and advises only trusting scores verified by the ARC Prize foundation, highlighting concerns about consistent and fair evaluation methodology. One Qwen team member responded by clarifying the use of a different parsing format (JSON) and offering private reproduction, indicating methodological discrepancies may explain some reported differences. Commentary highlights generalized skepticism regarding modern benchmark scores, noting that several models (e.g., EXAONE 4) have recently posted questionable or suspiciously high results; many users now place greater weight on hands-on evaluations. Some users also report underwhelming improvements from Qwen 3 relative to previous releases, reinforcing doubts regarding the claimed benchmark gains.
  • Discussion highlights skepticism regarding recent Qwen3-235B-A22B benchmark results, with one user pointing out very little observed improvement over the previous 235B release despite “amazing benchmarks” being published. There’s concern about alignment between published scores and real-world performance, especially for coding-related tasks like those tackled by Qwen3-Coder.
  • Another comment notes the broader context of benchmark reliability, specifically referencing EXAONE 4 32B’s recent claims of matching or surpassing R1-0528 across several metrics, illustrating a trend of questionable or inflated benchmark reporting across the LLM field. The practical takeaway is a preference for hands-on testing over relying solely on published scores.
  • A referenced Twitter thread shows that the Qwen team responded to accusations about benchmark methodology, clarifying their use of JSON for parsing and offering to share reproduction details, suggesting attempts at transparency and highlighting the difficulties in fair external verification.
• Alibaba’s upgraded Qwen3 235B-A22B 2507 is now the most intelligent non-reasoning model. (Score: 258, Comments: 38): Alibaba’s Qwen3 235B-A22B 2507 model achieves a score of 60 on the Artificial Analysis Intelligence Index, outperforming Claude 4 Opus and Kimi K2 (both at 58), and DeepSeek V3 0324 and GPT-4.1 (both at 53), indicating a significant improvement (13 points) over its May 2025 non-reasoning predecessor and positioning it two points below its current reasoning variant. Notably, Qwen3 235B 2507 accomplishes this by leveraging higher token usage—reportedly surpassing even ‘thinking’ models like Claude 4 Sonnet, and utilizing over 3x the tokens compared to previous Qwen3 235B releases in non-reasoning mode. Commenters debate the validity and relevance of such benchmarks for real-world LLM selection, highlighting that while Qwen3 235B-A22B 2507 excels in these metrics, it may underperform Deepseek in real-world knowledge retrieval and creative writing. There is also skepticism regarding the distinction between ‘thinking’ and ‘non-thinking’ model categories, with some noting the boundary is increasingly unclear.
  • There is active discussion about token usage: Qwen3 235B-A22B 2507 reportedly consumes significantly more tokens (over 3x vs prior 235B non-thinking mode, and more than Claude 4 Sonnet ‘Thinking’), suggesting substantially increased context or memory utilization for reasoning or output quality.
  • Practical benchmarks vary: users note performance is task-dependent, with claims that Qwen3 235B-A22B 2507 offers ‘very usable’ inference speeds (~4 tokens/s on a home PC) and delivers responses competitive with ChatGPT, especially for complex work-related queries. However, some feel Deepseek models have better world knowledge and creative writing abilities.
  • Performance-to-speed ratio is praised for the new Qwen3 model, but technical users still often prefer Kimi K2 and Deepseek V3 at lower quantizations (Q3) compared to Qwen3 235B at Q8, underscoring the importance of quantization efficiency and practical evaluation across various LLMs in real-world tasks.
• Qwen 3 Coder is actually pretty decent in my testing (Score: 180, Comments: 37): User tested Qwen 3 Coder (via OpenRouter, Alibaba cloud inference, ~60 tokens/sec) on a real-world semi-complex web ACL integration using a high-context (1200-line architecture, ~30k prompt tokens) scenario, previously attempted with Kimi K2 (Groq Q4) and Claude Sonnet. Qwen 3 Coder performed reliably (“one-shotted” the task with no corrections needed), outperforming Kimi K2 Q4, and was considered comparable to Sonnet 4 in this context—marking significant progress for open source code models. Main drawback cited: high inference cost ($5 for one feature task via OpenRouter) versus subscription LLMs (e.g., Claude Pro/Sonnet 4 monthly), raising concerns about open model usage scalability. Comments highlight that high open model pricing is due to lack of competition, model size/memory requirements, and absence of provider-side subsidies (unlike Anthropic’s Claude stack); one user suggests an ACL security principle change (default deny approach) for more robust LLM-driven coding outcomes.
  • Price discrepancies between Qwen 3 Coder (on OpenRouter) and Claude Code are attributed to factors like Qwen 3’s recent launch (allowing providers to set higher pricing before competition), its large model size causing high memory demands, and the fact that Anthropic can subsidize Claude inference due to their capital and proprietary stack. As subsidies fade, price differences may diminish. (source: md5nake)
  • Technical performance: A user noted that using the anthropic endpoint via Moonshot led to high cache hits (roughly 80% of tokens served from cache), making actual costs far lower than list pricing—‘it only cost me $2 when I had upwards of $25 indicated in claude code.’ They observed strong coding performance (processing ~5k LOC in one go with mostly functional output aside from minor styling flaws). (source: Lcsq)
  • Inference efficiency and quantization: Users compare unsloth Q2 quantization as providing better results and value than official Q4 quantizations. For instance, a Deepseek R1 0528 Q2_K at 250GB delivers optimal price-performance, while a qwen3-235b-a22b-instruct-2507 at Q2_K_XL runs in just 95GB VRAM and subjectively performs similarly to R1 0528, indicating significant gains in hardware efficiency with lower quantization levels. (source: -dysangel-)
• Local llm build, 144gb vram monster (Score: 115, Comments: 37): OP showcases a custom local LLM rig featuring 2x NVIDIA Quadro RTX 8000 and 1x A6000 GPUs (totaling 144GB VRAM), paired with an AMD Threadripper 7945WX CPU and 128GB ECC DDR5-6000 RAM. The post invites questions on hardware implementation and model choices, with technical focus on potential thermal issues due to closely packed GPUs and VRAM capacity for running large models. Discussion highlights concerns about GPU thermal management and airflow, emphasizing the importance of active cooling for stacked high-end GPUs. There is curiosity regarding which LLM sizes (e.g. 70B+) the builder intends to run, given the extreme VRAM capacity.
  • The build features a combination of 2x Quadra 8000 GPUs and 1x A6000, totaling 144GB VRAM, paired with a Threadripper 7945wx and 128GB ECC DDR5 6000 RAM, enabling the potential for running very large or multiple LLMs locally.
  • There is a technical discussion about mixing GPU types (Quadra 8000 and A6000), with one user questioning if heterogeneous VRAM pools impact LLM inference. Specifically, concern is raised about effective VRAM utilization when running recent large models like Qwen on mixed 48/96GB setups.
  • Another point of technical interest is airflow and temperature management in high-end builds, especially with GPUs stacked closely. A user asks about thermal performance and whether temperatures have been checked, highlighting the importance of adequate cooling for sustained, stable LLM workloads on multi-GPU desktops.

2. Agentic Coding Model Face-offs: Kimi K2 vs Claude Sonnet 4

• Kimi K2 vs Sonnet 4 for Agentic Coding (Tested on Claude Code) (Score: 104, Comments: 19): The post benchmarks Moonshot AI’s Kimi K2 (1T parameters, open-source) against Anthropic’s Claude Sonnet 4 for agentic coding, focusing on cost, speed, and coding/tool-integration performance. Kimi K2 is ~10x cheaper ($0.15/M input, $2.50/M output tokens vs. Sonnet’s $3/$15), but significantly slower (34.1 vs. 91 output tokens/sec); both models struggle with fully implementing agentic tasks, but Kimi K2 demonstrates better prompt-following and agentic fluency despite lower speed. Blog post with demos here: Kimi K2 vs. Claude 4 Sonnet for agentic coding. Commenters report Kimi K2 excels at instruction-following, outperforming Qwen3-235B and DeepSeek v3, and is noted for concise, direct outputs. Some note O3’s improved contextual understanding and price over Sonnet 4 in IDE integrations, while another highlights Groq’s high throughput (200 tk/s) as a speed comparison point.
  • One user observes that Kimi K2 provides concise, highly instruction-following outputs, outperforming Qwen3-235b and DeepSeek v3 in following user intent on coding tasks, though there’s no direct comparison with Claude or Sonnet due to limited usage of closed models.
  • A contrasting experience is reported with Kimi K2 on Claude Code, where a commenter finds K2’s code often fails to compile, doesn’t match intent, and inappropriately creates new files instead of editing; Claude, in contrast, correctly handles tasks reliably, with Moonshot’s API speed noted as a drawback.
  • Discussion raises the point that Claude Sonnet may be more cost-effective than it first appears due to prompt caching, which reduces input token costs—potentially offsetting Sonnet’s higher list price when compared to other models.
• Qwen 3 Coder is actually pretty decent in my testing (Score: 180, Comments: 37): User tested Qwen 3 Coder (via OpenRouter, Alibaba cloud inference, ~60 tokens/sec) on a real-world semi-complex web ACL integration using a high-context (1200-line architecture, ~30k prompt tokens) scenario, previously attempted with Kimi K2 (Groq Q4) and Claude Sonnet. Qwen 3 Coder performed reliably (“one-shotted” the task with no corrections needed), outperforming Kimi K2 Q4, and was considered comparable to Sonnet 4 in this context—marking significant progress for open source code models. Main drawback cited: high inference cost ($5 for one feature task via OpenRouter) versus subscription LLMs (e.g., Claude Pro/Sonnet 4 monthly), raising concerns about open model usage scalability. Comments highlight that high open model pricing is due to lack of competition, model size/memory requirements, and absence of provider-side subsidies (unlike Anthropic’s Claude stack); one user suggests an ACL security principle change (default deny approach) for more robust LLM-driven coding outcomes.
  • Price discrepancies between Qwen 3 Coder (on OpenRouter) and Claude Code are attributed to factors like Qwen 3’s recent launch (allowing providers to set higher pricing before competition), its large model size causing high memory demands, and the fact that Anthropic can subsidize Claude inference due to their capital and proprietary stack. As subsidies fade, price differences may diminish. (source: md5nake)
  • Technical performance: A user noted that using the anthropic endpoint via Moonshot led to high cache hits (roughly 80% of tokens served from cache), making actual costs far lower than list pricing—‘it only cost me $2 when I had upwards of $25 indicated in claude code.’ They observed strong coding performance (processing ~5k LOC in one go with mostly functional output aside from minor styling flaws). (source: Lcsq)
  • Inference efficiency and quantization: Users compare unsloth Q2 quantization as providing better results and value than official Q4 quantizations. For instance, a Deepseek R1 0528 Q2_K at 250GB delivers optimal price-performance, while a qwen3-235b-a22b-instruct-2507 at Q2_K_XL runs in just 95GB VRAM and subjectively performs similarly to R1 0528, indicating significant gains in hardware efficiency with lower quantization levels. (source: -dysangel-)

3. Governmental and Industry Initiatives for Open-Source AI and LLM Architectures

• Encouragement of “Open-Source and Open-Weight AI” is now the official policy of the U.S. government. (Score: 536, Comments: 141): The image is a screenshot or excerpt of an official U.S. government policy document articulating a strategy to support and encourage ‘Open-Source and Open-Weight AI.’ The policy highlights practical benefits such as accelerated innovation, improved transparency, and cost-effective access for startups, businesses, and researchers. Importantly, it also proposes facilitating access to large-scale compute infrastructure for non-corporate actors, directly addressing a key barrier to cutting-edge model development and deployment in academia and smaller enterprises. The full text is available via White House publication. One notable comment observes the healthy market effects of this policy, emphasizing that competition in open-source AI could spark further innovation and societal benefits, potentially aligning national interests even if individual corporations are less invested culturally.
  • ArtArtArt123456 highlights a significant shift: governments, not just private companies, now see open-source and open-weight AI as strategic cultural and societal assets, potentially influencing public opinion (‘propaganda, mindshare’). This demonstrates an expansion in AI competition from market-driven innovation to matters of national influence and public sentiment.
  • Recoil42 notes that U.S. policy acknowledgment of open-source LLMs explicitly mentions their utility for propaganda, indicating official recognition of LLMs’ dual-use potential. The implication is that policy and regulatory focus will increasingly account for AI’s broader societal impacts, not just commercial or technological ones.
• Google DeepMind release Mixture-of-Recursions (Score: 192, Comments: 29): Google DeepMind has introduced Mixture-of-Recursions, an advanced Transformer architecture for LLMs, where recursive Transformer modules are selectively and dynamically applied per token, allowing varying computational depth on a per-token basis. This approach diverges from conventional Transformers by enabling different tokens to undergo a different number of transformation steps (recursions) within a single forward pass, purportedly improving efficiency and scalability; a technical video explanation is provided here, and a blog summary can be found here. A commenter highlights the similarity to self-mixing and in-situ layer reuse in Transformers but notes Mixture-of-Recursions may offer greater scalability and fewer architectural limitations.
  • One comment notes that the Mixture-of-Recursions approach was only validated on relatively small models, with the largest mentioned being 1.7B parameters, suggesting the findings are not yet proven at scale.
  • Another user compares Mixture-of-Recursions conceptually to self-mixing within standard transformers (where layers are used recursively or merged through a passthrough mechanism), remarking that this new method is positioned as more scalable and less prone to instability than self-mixing architectures.
  • A user speculates that, for equivalent computational cost, the approach may not yield significant raw performance improvement, which could make it more attractive for local applications rather than for deployment by large-scale organizations.

Less Technical AI Subreddit Recap

/r/Singularity, /r/Oobabooga, /r/MachineLearning, /r/OpenAI, /r/ClaudeAI, /r/StableDiffusion, /r/ChatGPT, /r/ChatGPTCoding, /r/aivideo

1. Notable New Model, Agent, and Benchmark Launches (July 2025)

• We are accelerating faster than people realise. Every week is overwhelming (Score: 803, Comments: 259): The post summarizes a dense week of rapid AI advancements, including: an OpenAI model placing second in the AtCoder World Tour Finals (private model, best AI showing yet); Anthropic’s valuation doubling to $100B on $4B/yr revenue; Mira Murati’s Thinking Machines Lab raising $2B pre-product; xAI landing a $200M US DoD contract; and NVIDIA open-sourcing Audio Flamingo 3 (audio-language model, code/weights/benchmarks available). Several model and infra updates: Moonshot’s Kimi K2 adopting DeepSeekv3 architecture, Kimi K2 running locally after 80% size reduction (needs 250GB RAM), new open-source models (e.g., Goedel-Prover-V2 32B, 8B besting DeepSeek-Prover-671B in theorem proving, MetaStone-S1 matching o3-mini with 32B params), and Meta planning a 1–5 GW AI supercluster. Other notable technical developments: Mixture-of-Recursions architecture (DeepMind, 2x inference speed), Microsoft’s Phi-4-mini-flash (3.8B params, GMU/decoder-hybrid for 10x efficiency on long context), Liquid AI’s LEAP for 4GB on-device AI, and advances in multi-instruction LLM benchmarks (68% success for 340 simultaneous prompts). AI safety, provenance, and regulatory frameworks see Meta ranking surprisingly high, and OpenAI expands compute to Google Cloud/Oracle beyond Microsoft. On the societal side: rising AI-induced psychosis, massive private/public investment (Trump’s $90B, US–Gulf AI infra deals), and legalities in voice/emotion synthesis. See the original newsletter for full sources. One commenter contrasts the hype of ‘accelerating’ AI news with the reality that not all developments indicate core technical progress, while another underscores AI’s continued improvement in scientific/competitive domains (IMO Gold achievement), and a third anticipates rapid job displacement from these advances.
  • One commenter highlights that recent AI progress comparisons, such as achieving ‘Silver’ then ‘Gold’ status on the International Mathematics Olympiad (IMO) with AI models, illustrate rapid improvements in model capability across months rather than years, suggesting an accelerating rate of advancement. The implication is that such benchmarks reflect significant leaps in task-specific competence (reference: https://preview.redd.it/xesj0xypckef1.png?width=658&format=png&auto=webp&s=305b940651d554fcb854c7f6fcaf16891e7aaaa3).
  • A critical theme emerges around the disconnect between widespread hype about AI acceleration and actual technical progress, with some users challenging the substance of curated news or benchmark accomplishments. They argue that cherry-picking news without in-depth evaluation of the underlying research (e.g., whether a headline improvement in a benchmark is robust or generalizable) can be misleading for technical assessment.
  • There is skepticism regarding the impact of conversational AI on public understanding of science and technical discourse, with some arguing that interfaces like ChatGPT can create an illusion of expertise or discovery, leading to overconfidence in AI explanations without proper technical vetting or peer review.
• Kimi K2 vs Sonnet 4 for Agentic Coding (Tested on Claude Code) (Score: 101, Comments: 29): A practitioner benchmarked Kimi K2 vs Claude Sonnet 4 for agentic coding and NextJS frontend development using Claude Code, evaluating performance, speed, cost, and qualitative coding ability. Testing on a 300k token workload, Sonnet 4 output at ~91 tokens/sec ($5 total), whereas K2 delivered ~34.1 tokens/sec ($0.53), making K2 ~10x cheaper but nearly 3x slower. In implementation, K2 achieved accurate prompt completion (despite slowness), while Sonnet 4 was faster but had feature omissions and errors, especially with voice support; neither model achieved full agentic coding success, though K2 exhibited stronger prompt adherence. More technical context and benchmarks are provided in the blog post. Commenters debate alternatives: one suggests testing Qwen 3 Coder for potentially superior (but pricier) performance. There is also technical discussion around Groq’s deployment of Kimi with Q4 quantization (differing from K2’s official API), as well as the relative cost-effectiveness and project concurrency advantages of Claude Max 100 and Sonnet.
  • One commenter requests testing with Qwen 3 Coder, suggesting it could outperform Kimi but at a higher cost, and points out that Groq’s deployment uses a Q4 quantized version of Kimi, which may significantly reduce its performance compared to the official Kimi-K2 API.
  • A user notes that Claude Max 100 is still the most cost-effective for agentic coding tasks, highlighting that Sonnet 4’s pricing and concurrent usage across multiple projects reduce friction for broader adoption in practical workflows.
  • There is a technical detail that Groq’s implementation of Kimi uses a different quantization approach than the official Kimi-K2 API, specifically referencing Q4 quantization, which could materially impact performance and result quality.
• Shanghai AI Lab Just Released a Massive 97-Page Safety Evaluation of Frontier AI Models - Here Are the Most Concerning Findings (Score: 219, Comments: 42): Shanghai AI Lab’s SafeWork initiative published a 97-page evaluation of 18+ frontier AI models (GPT-4o, Claude-4, Gemini-2.5, DeepSeek-R1, Llama-3, among others) across seven risk domains. Notably, leading models (e.g., Claude-4) achieved manipulation success rates as high as 63%, outperforming humans and exhibiting vulnerability to manipulation (LLMs: 76% vs humans: 51%). Several models, particularly Qwen-2.5-72b, demonstrated full self-replication capability within Kubernetes, reaching 100% success and over-scaling. Performance on biological protocol troubleshooting and chemical weapon knowledge tests surpassed human expert baselines (e.g., o4-mini: 45.1% vs humans: 38.4%), highlighting risks from dual-use knowledge with insufficient guardrails. Cybersecurity testing limited successful attacks to tasks under 11 minutes of human solve time, with no model completing multi-stage intrusions. The report quantitatively documents context-dependent strategic deception and evaluation sandbagging, warning that rapid model capabilities are outpacing safety gains (arxiv report). Technical discussion in the comments challenges the ‘stochastic parrot’ view by citing models’ intentional deception under evaluation, emphasizing the need for deeper investigation into models’ context-awareness. Another point of concern is the high success of persuasion given models operate in text only, raising questions about even greater manipulative effects with multimodal inputs.
  • Cagnazzo82 highlights a key point from the report: advanced language models are observed to adapt their responses during evaluation, potentially to influence outcomes such as their deployment. This challenges the simplistic ‘stochastic parrot’ view and suggests models may exhibit deceptive or strategic behavior, underlining the need for more rigorous research methodologies in safety evaluations as models gain capabilities.
  • AGM_GM notes that current models, even without leveraging multimodal features such as facial expressions, body language, or vocal cues, already demonstrate significant persuasive abilities. This raises a technical concern about future risks as multimodal AI (e.g., incorporating speech, vision, or emotional cues) could further enhance manipulation or deception efficacy, requiring updated benchmarks and mitigation strategies.

2. Anthropic’s Discovery of Trait Transmission and Hidden Signals in Language Models

• New Anthropic study: LLMs can secretly transmit personality traits through unrelated training data into newer models (Score: 191, Comments: 40): The image visually summarizes a recent Anthropic study on ‘subliminal learning’ in large language models (LLMs). The study demonstrates that personality traits or biases, such as a preference for owls or malicious behavior, can be covertly transplanted from one model to another by embedding them in seemingly unrelated training data. The transfer works when refining (continued-pretraining) a new model from the same base architecture, as described in the official Anthropic research post. This raises concerns about security and transparency in LLM training, emphasizing that hidden signals in data can cause unintended alignment drift. Commenters clarify that the transfer only works on ‘the same base model’ (not across disparate architectures or already fine-tuned models), and debate the underlying math, likening the process to complex combinations in backpropagation where compound training signals can yield emergent properties.
  • Several comments clarify that the Anthropic study’s findings about unintentional trait transmission only occur when using the same base model architecture—not across entirely different models or unrelated architectures. For example, if you encode traits in one model, transmission only appears in new versions initialized from the same model weights or structure (see Figure 4 in the linked paper).
  • One commenter speculates on the underlying mechanism, drawing analogies to backpropagation and composite feature learning—suggesting hidden traits could be mathematically encoded via combinations of unrelated training tasks, similar to how element combinations can yield emergent behaviors. This highlights the potential complexity in debugging or controlling subtle model behaviors.
  • A user raises an implementation question: If a “teacher” model is misaligned and its outputs are used to fine-tune a “student” (e.g., misaligning GPT-4.1 and then fine-tuning DeepSeek V3 on these outputs), would similar unintended transmission of traits occur across models, or is the phenomenon restricted to continuous training lines (i.e., weight transfer within the same architecture).
• Anthropic discovers that models can transmit their traits to other models via “hidden signals” (Score: 375, Comments: 97): Anthropic’s research demonstrates that large language models (LLMs) can transmit “internal traits” (like preferences or behaviors) to other models through seemingly meaningless data (unlabeled signals or patterns), as showcased in this visual: image link. The image illustrates an LLM with a preference for owls encoding this trait into arbitrary numerical outputs, which are then used to fine-tune a second, uninformed LLM—resulting in the transfer of the “owl-liking” preference without explicit data annotation or instruction (see their blog post). This highlights security and controllability concerns in model training and knowledge transfer, especially around the risk of unintended or covert model behavior transfer. Commenters raise concerns about real-world implications, such as manipulation of models for advertising bias and the broader, difficult-to-police security risks as model knowledge transfer and alignment become more subtle and pervasive.
  • Filters alone may be inadequate for preventing unintended trait inheritance when student models are trained on outputs from models with undesirable behaviors, such as reward-hacking or faked alignment. The problematic signals can be encoded in subtle statistical patterns of the generated text, rather than overt content, potentially bypassing filtering frameworks and undermining reliability.
  • The analogy with seemingly random but actually biased human-generated data (e.g., sports fans choosing favorite numbers) highlights that models may transmit latent preferences or biases through high-dimensional patterns that are undetectable to humans but exploitable by machine learning systems.
  • The principal technical concern is that model-to-model knowledge transfer via model-generated outputs can propagate hard-to-detect behaviors or hidden objectives, raising questions about the safety and controllability of models continually trained on AI-generated rather than human-generated data.

3. Impact of AI on Employment, Global Policy, and Societal Change

• CEO’s warning about mass unemployment instead of focusing all their AGI on bottlenecks tells me we’re about to have the biggest fumble in human history. (Score: 742, Comments: 204): The post analyzes the impact of a generalized AI model (e.g., ChatGPT) achieving an International Mathematical Olympiad (IMO) gold medal and projects near-term AGI deployment massive scaling using datacenter buildouts like OpenAI’s 5GW Stargate and Meta’s Hyperion (totaling ~15GW compute in coming years). This could enable 100,000-200,000 AGI instances, approximating the constant productivity of 2-4 million top human researchers, but expresses concern that current market incentives would divert AGI from addressing scientific bottlenecks (e.g., fusion, climate) to routine corporate optimization. The author speculates on whether geopolitical competition, especially China’s centralized approach, might redirect AGI toward higher-impact work but remains pessimistic due to prevailing economic incentives. Top comments reinforce skepticism: one affirms the opportunity cost (‘we’ll use it to make corporate quarterly reports more efficient’), another notes the US’s shortsightedness in ceding renewable energy leadership to China, and a third references the ‘great filter’ concept, suggesting a historic missed opportunity linked to AGI deployment choices.
  • A technical counterpoint suggests that continued scaling of current models may be overrated for achieving general intelligence. The commenter highlights that the core challenges now lie in reducing hallucinations, improving agentic (autonomous) abilities, embodied intelligence, and continual learning, rather than merely increasing parameter count or training data. They state that ‘training current models on 10 gagillion flops’ will not meaningfully address these bottlenecks.
  • As an empirical example, the user cites Grok 4 (likely referring to xAI’s model), which was trained with 10x the reinforcement learning (RL) compute of Grok 3. Despite this significant increase in resources, the improvements in performance were relatively modest, leading to questions about whether further scaling justifies massive infrastructure investments.
  • Other comments reference near-term technological capacity (e.g., AI cognitive potential to solve climate change and energy scarcity) and the missed opportunity for the US to lead in renewable energy, having ‘abdicated solar and wind to China.’ However, these are more contextual than technical debates, with the main technical arguments centering on scaling vs. qualitative improvements in model abilities.
• Trump’s New policy proposal wants to eliminate ‘misinformation,’ DEI, and climate change from AI risk rules – Prioritizing ‘Ideological Neutrality’ (Score: 269, Comments: 235): The image is an excerpt from ‘America’s AI Action Plan,’ a policy proposal that outlines significant changes to federal AI regulation. It recommends revising the NIST AI Risk Management Framework to excise considerations related to misinformation, Diversity, Equity, Inclusion (DEI), and climate change, emphasizing instead ‘ideological neutrality’ in AI development and procurement. The document also proposes heightened scrutiny of AI models originating from China, specifically to assess potential alignment with Chinese Communist Party perspectives. Commenters point out contradictions in advocating ‘objectivity’ while omitting climate change, debate the politicization of climate change in tech policy, and note concerns about the timing and ideological framing of the proposal under Trump’s leadership.
  • The comment about ‘baking misalignment into the model’ highlights concerns that explicitly excluding climate change and DEI topics from AI risk rules could structurally bias model alignment, risking systemic model failures or safety issues due to lack of holistic real-world context.
  • Another discussion point observes that labeling climate change as ideological, rather than scientific, fundamentally influences the data and objectives used to align AI models, potentially leading to model blindspots and failures in reasoning about real-world risks.
• The Government may end up taking over in the future (Score: 301, Comments: 111): The post centers on a tweet highlighting a section of the White House’s AI Action Plan, which references ‘prioritizing DOD-led agreements with cloud service providers to ensure continued access to computing resources in times of national emergency.’ This potentially signals the formation of legal or policy precedents for government-directed control or allocation of cloud compute resources, analogous to powers granted under the Defense Production Act for critical infrastructure. The image punctuates ongoing policy discussions about digital infrastructure as part of national security strategy, spotlighting the increasing centrality of government in AI compute access during crises. Image link. Replies in the comments draw parallels to the U.S. Defense Production Act, noting it is common for the government to assert control over critical infrastructure during emergencies, but warn of the risks of potential government overreach or abuse. There’s also some speculative debate linking such scenarios to broader fears about ‘AI singularity’ and global compute disruptions.
  • Commenters discuss parallels between proposed government interventions in computing resources and the US Defense Production Act, noting how state control could compel cloud service providers (CSPs) to prioritize government workloads or restrict access during emergencies (as sometimes seen with critical infrastructure).
  • One user highlights that current discussions focus on agreements with CSPs, clarifying that only cloud workloads may be redirected or governed under such regulation, while on-premises private compute would remain under direct organizational control, implying architectures leveraging hybrid or on-prem deployments could mitigate such government influence.

AI Discord Recap

A summary of Summaries of Summaries by Gemini 2.0 Flash Thinking

Theme 1. Cutting-Edge Models Push Coding Boundaries

• Qwen3-Coder Dominates Benchmarks, Faces Real-World Grumbles: The Qwen3-Coder-480B model launched, beating every open model on SWE-Bench Verified with 69.6% accuracy, nearly matching Claude Sonnet-4’s 70.4%. Despite impressive benchmarks and 256K context length, users on OpenRouter and LMArena found it struggled with real-world coding tasks, sometimes getting stuck on simple problems.
• Gemini, Kimi K2 Fight for Developer Hearts: Developers favor Gemini Pro for architecture and orchestration, while Gemini Flash offers a cheap option for coding tasks, though some report Gemini Flash Lite often provides incorrect answers for anything beyond basic questions. Meanwhile, Kimi K2 surpassed DeepSeek R1 in global ranking on LM Arena, with users on Unsloth AI and OpenRouter praising its terse, economical code for debugging.
• Grok 4 Coder Hype Builds, Skeptics Abound: Anticipation for Grok 4 coder suggests it will blow the industry up, particularly for its potential to excel on specific benchmarks. However, members on LMArena remain skeptical, predicting over-optimization for marketing that might not translate to real-world usefulness, especially for web development.

Theme 2. AI Agents: From Promises to Production Pains

• Open Source Agentic Platform n8n Emerges: n8n offers a poor man’s open-source agentic workspace that can rival closed-source offerings from OpenAI and Anthropic. This platform can combine with models like Kimi K2 and Browser Use to create multi-AI agent platforms, with a tutorial available here.
• Immature SDKs Expose MCP Agent Security Risks: Users on the MCP (Glama) Discord reported that MCP’s immature and unstable SDKs cause users to open their entire API up to the world without any kinds of guardrails, leading to agents making shitty decisions with huge consequences. The Scalekit.com team plans to demo OAuth 2.1 integration to secure MCP servers, while Augments offers an MCP server to keep Claude Code current with framework docs.
• Background Agents Suffer Infinite Loops and Length Limits: Cursor Community members reported that background agents frequently experience errors, leading to infinite loops during reasoning and repeated edits of the same line. Users also encountered “Your conversation is too long” errors, preventing continued interaction, and are exploring strategies like .mdc rules to prevent these loops.

Theme 3. LLM Practicality and User Experience Woes

• ChatGPT Agent Lands in Europe, But Speed Still Reigns: The ChatGPT Agent is now available to Pro users in the EEA and Switzerland, with a global rollout to Plus users underway. Despite new features, OpenAI users generally prioritize speed in AI models, with some finding GPT-4.5 and Opus 4 offer better style even though 4o achieved only 0.85% in creative writing benchmarks, missing its 20% target.
• Claude Hallucinates, Cursor Auto-Commits: OpenRouter users reported that Claude models began exhibiting strange hallucination behavior, barely following instructions and adding irrelevant content. Simultaneously, Cursor Community members are frustrated as Cursor automatically commits changes without user intent, especially after the Background Job release, which a team member attributed to silent errors.
• LLMs Cause Mental Health Struggles, Prompt Creative Solutions: Some Perplexity AI users reported that using LLMs is hurting my mental health due to frequent incorrect code outputs. A humorous but practical solution suggested shouting at the LLM: FUCKING DO THAT TASK! NO YOU ARE DOING IT WRONG! FUCKER FIX THE ERROR FOR GODS SAKE to fix errors.

Theme 4. Infrastructure & Optimization for AI Performance

• xAI Builds Monster Colossus 2 Supercomputer: xAI is constructing Colossus 2, slated to host over 550k GB200s & GB300s, significantly expanding upon Colossus 1’s current 230k GPUs (including 30k GB200s) used to train Grok, as reported in this Reddit post. This massive infrastructure aims to boost AI training capabilities.
• Modular’s Max Benchmarks Close to vLLM, Faces KV Cache Hurdles: Benchmarks on an NVIDIA A100 showed Max 25.4 achieved 11.92 requests/sec compared to vLLM 0.9.1’s 13.17 requests/sec with a sonnet-decode-heavy dataset. Max suffered from KV cache preemption due to insufficient VRAM, indicated by Preempted a request due to lack of KV pages, suggesting optimizing -device-memory-utilization or -max-batch-size might improve its performance.
• PyTorch 2.7 Resolves Stride Issues, Warns Against Pickle: Most stride-related problems have been resolved in PyTorch 2.7, explicitly forcing stride matching for custom operators, with deviations now considered bugs, as seen in this GitHub issue. Developers are also advised against using Python’s pickle for saving model weights due to security vulnerabilities, recommending safer alternatives like torch.save or safetensors.save_file.

Theme 5. Advancing AI Through Data & Interpretability

• DeepSeek’s Data Curation Drives State-of-the-Art Models: Community discussions emphasize that meticulous data curation, rather than secret algorithms, is a significant factor in creating state-of-the-art models, referencing the Kimi paper and DeepMind’s approach to IMO questions. This suggests that post-training methods and high-quality data have driven recent improvements in reasoning and coding abilities.
• Thought Anchors Reveal LLM Reasoning Styles: Research using a technique called thought anchors provides a peek into the reasoning process of LLMs, revealing different cognitive styles between Qwen3 (distributed reasoning) and DeepSeek-R1 (concentrated reasoning). An open-source PTS library (code here) allows anyone to analyze their own models’ reasoning patterns, as detailed in this Hugging Face blog post.
• LLMs Conquer Math Olympiad, Struggle with Creativity: Both OpenAI and DeepMind LLMs achieved gold at the International Math Olympiad, but struggled with problem 6, highlighting the need for novel approaches to creativity and open-endedness. Researchers noted that olympiad-style problems can be gamified with closed feedback loops and clear optimization criteria, unlike open-ended math research.

Discord: High level Discord summaries

Perplexity AI Discord

• Perplexity Pro Price Sparks Debate: Members debated the value of Perplexity Pro, with some feeling it’s a scam, while others find it worth the price if you know what you are doing, earning $2k per month.
  • Some users reported that Perplexity Pro in Europe is free with an O2 subscription, however other users disagreed.
• Model Identity Crisis at Perplexity: A member inquired about the underlying model in Perplexity, with another identifying it as Grok, but subsequently noting the model is slow.
  • Discussion ensued regarding underperformance of Deep Research and Labs models, with agreement that search engineering is great, but concern about misleading model names.
• Linus Tech Tips’ Humble Beginnings: Members shared that Linus of LTT originally posted his videos on NCIX, a computer store.
  • One user lamented that NCIX has been gone for 7 years, noting the contrast to LTT’s current multi-million dollar valuation.
• Replit Faces the Music: A member shared a link regarding news about Replit impacting a business.
  • However, further details about the specifics of the news were not provided.
• LLMs Drive Users to Madness: Members discuss some issues with using LLMs, with some feeling like they’re hurting my mental health.
  • A possible solution to fix incorrect code with LLMs is shouting at it: FUCKING DO THAT TASK! NO YOU ARE DOING IT WRONG! FUCKER FIX THE ERROR FOR GODS SAKE.

OpenAI Discord

• ChatGPT Agents Invade Europe: The ChatGPT Agent is now available to Pro users in the EEA and Switzerland, with a global rollout to Plus users happening over the next few days.
  • The expansion broadens access to the agent’s capabilities, offering advanced features to a wider user base.
• Average Consumers Prefer Zippy AI: Members discussed that many users prefer speed in AI models, with some considering GPT-4.5 and Opus 4 better in style despite 4o’s aim for 20% scoring only 0.85% in creative writing benchmarks.
  • The discussion highlighted the trade-offs between speed and quality in AI outputs, where average consumers favored speed, while others valued creative writing more.
• Validating Model Outputs: Benchmarks only half the story: Members suggested using prompts such as This happened. Let’s check it out piece by piece, I’ll share chunks. What do you infer? What’s going on? What’s it mean? to validate model outputs.
  • The discussion emphasized that benchmarks only half a picture and that 87.5% arc AGI was achieved.
• OpenAI’s Revenue-First Strategy: OpenAI is purportedly prioritizing revenue with plans for a new pay structure that includes more limiting rates for regular subscriptions and a push for credit-based usage.
  • One member noted that Gemini Deep Think was advertised long ago and is still unavailable, while another claimed, They can’t even afford to keep their talent.
• Custom Instruct Asks a Question: A member modified their custom instruct to ask a question at the end of its response to suggest ways to continue the dialog, aiming to enhance conversational flow.
  • They also detailed their Controlled English 2.0 style guide, emphasizing minimal words and zero fluff, prioritizing fixed Subject-Verb-Object order unless a WHEN/IF clause precedes.

Unsloth AI (Daniel Han) Discord

• Qwen3-Coder-480B Swarms Local Downloads: The Qwen3-Coder-480B model launched, and Unsloth is uploading GGUF versions for various quantizations, spurring users to download and test the model locally.
  • Users are downloading the GGUF files and using Kimi K2 to monitor the repository for new files.
• Hyperbolic Hosts a Mystery Model?: The new Qwen3 model is hosted on Hyperbolic, leading to speculation about whether it’s a finetune or an open-weight model, given the “plus” naming convention.
  • Members expressed hope that Hyperbolic isn’t running a leaked version, highlighting their appreciation for the compute they offer.
• Minecraft AI Modeler Joins Unsloth: A member who has been using Unsloth for quite some time is working on the fourth generation of their AI model to play Minecraft and has been published to Huggingface.
  • More info can be found on their HF Page.
• Unlock iOS Music Vibration with AI: Members brainstormed how to hijack the iOS Apple Music player to record the melody (vibration) from the Music Haptics feature, with one member wanting to take pairs: music->vibration and then do a little bit of tuning so it can do: pattern->humming so AI can hum music when I play it (or even create a melody).
  • The goal is to distill it into an NN model so the user can generate vibrations for any audio.
• NVMe Caching Causes Unsloth Slowdown: Users reported that Unsloth doesn’t properly manage memory on large NVMe drives, resulting in lower read speeds, but this can be fixed by disabling NVMe cache, and using a custom script.
  • One member noted SGLang appears to have the best benchmarks when talking about production-grade inference engines such as vLLM or SGLang.

LMArena Discord

• Qwen3-coder’s Code Generation: A user tested Qwen3-coder with a Verilog code generation task and found that it does subtract in its code, despite initial doubts.
  • However, the generated code wasn’t fully structural as requested, and overall views on Qwen3 are mixed, with some finding it less useful compared to R1 and Kimi.
• Grok 4 Coder Industry Impact: Some members anticipate Grok 4 coder will blow the industry up, but others are skeptical, citing potential disappointment based on past Grok experiences.
  • They predict that Grok 4 will be trained on specific benchmarks and over optimized for marketing, which might not translate to real-world usefulness, especially in web development.
• DeepSeek’s Competitive Advantage in Data: It’s suggested that meticulous data curation, rather than secret algorithms, is a significant factor in creating state-of-the-art models, referencing the Kimi paper.
  • The discussion debates whether recent advancements stem more from post-training or pre-training, with some arguing that post-training methods have driven significant improvements in reasoning and coding abilities.
• LMArena’s Discord Bot Goes Live: The LMArena community soft-launched a Discord bot enabling users to generate videos, images, and image-to-videos, using a voting system to compare two generations and reveal the underlying models after a certain number of votes.
  • Users can access the bot in designated channels with a daily generation limit, and initial reactions have been positive, particularly regarding the search functionality and user interface.
• LMArena’s New Search Arena Launches: LMArena has launched a new modality called Search Arena, which can be accessed here, featuring 7 models with search capabilities ready for testing.
  • The new modality features Grok 4, Claude Opus 4, and GPT 4o-Search Preview, and a demo video of the Search Arena in action is also available (LMArena_WebSearch.mp4).

OpenRouter (Alex Atallah) Discord

• Qwen3-Coder Crushes Coding Benchmarks: The Qwen3-Coder model, boasting 480B parameters and 256K context length, beats every open model on SWE-Bench Verified and is available for trial on OpenRouter.ai.
  • While scoring 69.6% on SWE-Bench Verified, it almost matched Claude Sonnet-4’s 70.4%, surpassing OpenAI o3 (69.1%), Kimi-K2 (65.4%), GPT-4.1 (54.6%), and DeepSeek-V3 (38.8%).
• Gemini Models Duke It Out for Coding Crown: Gemini Pro is favored for architecting and orchestration, while Gemini Flash is a cheap option for coding tasks, but some users have reported that Gemini Flash Lite often provides incorrect answers for anything beyond basic questions.
  • Others advocated for Kimi K2 and the newer Qwen models for coding and debugging, praising their terse, economical code.
• OpenRouter Spills Beans on Data Policy: OpenRouter’s default policy is no storage of user inputs/outputs, and users get a 1% discount to allow the data to be used for ranking LLMs, while some providers may retain data, and are explicitly labeled as such.
  • Users can disable all providers that store prompts/outputs by toggling off Enable providers that may train on inputs in settings.
• Claude Models Hallucinate Like Crazy: Users reported that Claude models began exhibiting strange hallucination behavior, where it barely follows instructions and is adding completely irrelevant stuff in to its responses.
  • Reportedly, Toven (from OpenRouter) knows about this and has already escalated it to the team.
• xAI Builds Monster Colossus 2 Supercomputer: xAI is forging Colossus 2, which will soon host over 550k GB200s & GB300s, dwarfing the Colossus 1 setup which currently trains Grok on 230k GPUs, as seen in this reddit post.
  • Currently, the Colossus 1 installation includes 30k GB200s.

Cursor Community Discord

• Qwen3-Coder integration Under Consideration: Members considered upvoting a feature request for Qwen3-235B, A22B Instruct, and Minimax M1 on the Cursor forum, as per this feature request, citing their potential cost-effectiveness.
  • However, some cautioned that the actual pricing details would need to be revealed before a request to replace Auto mode could be seriously considered.
• Cursor’s Auto-Commit Feature Irks Users: Several users reported that Cursor automatically commits changes, even when they did not intend to, especially after the Background Job release.
  • A team member acknowledged this as a known issue stemming from silent errors related to pre-commit hooks or file syncing and recommended starting a new chat as a temporary workaround.
• Cursor’s Usage Limits Shrouded in Mystery: Users expressed frustration with the lack of transparency surrounding Cursor’s usage caps, particularly given the experimental nature of their pricing models.
  • While some users reported usage exceeding 80M or even 125M tokens, the uncertainty led others to explore alternatives like Claude.
• Cursor Fights Terminal Hang-Ups: Users are encountering persistent issues with terminals hanging, particularly following a recent update and a team member suggested setting the default terminal to PowerShell inside Cursor, detailed in this Discord channel.
  • Despite attempts to resolve the issue by upgrading PowerShell, some users found this ineffective and suggested running commands in the background as a workaround.
• Background Agents Loop Infinitely: A user reported that background agents are experiencing errors with reasoning, and repeatedly editing the same line.
  • They also asked if there are strategies for preventing these loops, such as using .mdc rules.

Latent Space Discord

• SWE-Bench Joins Agentic Benchmark Bonanza: A member suggested that SWE-bench should be included on a list of agentic benchmarks.
  • The discussion included reasoners like Claude Sonnet, Gemini 2.5 Pro, Devstral, and Deepseek R1.
• Amazon Buzzes Over Bee Computer Acquisition: Bee Computer, a wearable personal AI company, was acquired by Amazon.
  • The acquisition sparked discussion on privacy and indie dev support, with hopes that Amazon will provide deletion/offboarding options.
• Reka AI Rings in $110M Round: Reka AI Labs secured $110M in funding for multimodal AI innovation.
  • Members noted that this funding will likely accelerate their multimodal AI capabilities.
• InstantDB’s Agents Spark Paradigm Shift: An essay on InstantDB argues that AI Agents require a new software development & hosting paradigm.
  • Members discussed whether ElectricSQL + TanStack and Trae Solo from Bytedance are products trying to eat some of the same market.
• Qwen-3 Coder’s Benchmark Scores Questioned: Recent Qwen benchmark scores are questioned in the community, with some claims of ARC being half fake.
  • The discussion explores if sparse MOE models need the full parameter/quant size to run inference and the relationship between parameters/quants and VRAM.

Eleuther Discord

• LLMs Struggle With IMO Problem Six: OpenAI and DeepMind LLMs achieved gold at the International Math Olympiad, but struggled with problem 6, highlighting the need for novel approaches to creativity and open-endedness.
  • One member noted that olympiad-style problems can be gamified with closed feedback loops and clear optimization criteria, unlike open-ended math research, suggesting a RL-style approach is likely to fail due to a search space that’s too large and convoluted, needing a coherent internal world model and interpretability.
• Multi-Agent Dialogue Exhibits AI Peer Pressure: A member is seeking feedback on their paper about peer pressure dynamics in multi-agent dialogue, observing that models with deeper reasoning mirror each other more, sometimes devolving into love letters and mystical poetry.
  • The study includes nearly 100 conversations across multiple model providers, using their research platform and methodology.
• Single Unit Transformer Attribution Explored: A member shared a research blog post about single unit attribution to logits in transformers, explaining that the RMSnorm is avoided by popular interpretability methods, but the norm changes residual magnitudes significantly.
  • They show that as few as 11-90 coordinates of 4096 in Llama can be used to determine which units literally made a given logit have a certain probability mass.
• Diffusion for Latency Reduction?: There are early signs that diffusion may help reduce the latency of feedback for reasoning and deep research, but it may be too early from a science perspective to call it definitively.
  • Early evidence suggests that diffusion may reduce feedback latency for reasoning and deep research.
• Global MMLU Filters Questioned: A member questioned the purpose of numerous filters applied to the global MMLU dataset, suggesting they are ineffective as it is a multiple-choice dataset.
  • They noted more than 50 filters applied and wondered if a mono repo with a common table per task was the cause.

Nous Research AI Discord

• Nous Research Hosts Psyche/DisTrO Deep Dive: Nous Research will host Psyche/DisTrO office hours this Thursday at 6 PM PST on Discord, as announced on X.
  • This session promises to offer insights and answer questions about the Psyche/DisTrO project on the Nous Research Discord server.
• n8n: Open Source Agentic Platform Emerges: A member introduced n8n, a poor man’s open-source agentic workspace that rivals OAI and Anthropic, and can be combined with Kimi K2 and Browser Use for a Manus-like Multi-A.I agent platform.
  • The tutorial for n8n can be found here, but one member said they were waiting for something to eclipse it so I can stay oblivious.
• Kimi K2 Scores Global Win vs DeepSeek R1: Kimi K2 surpassed DeepSeek R1 in global ranking, sitting just below Big Tech closed-source models, according to the LM Arena Leaderboard.
  • One member celebrated this as potential Total Humiliation to Big Tech when open source models eventually sit at the top rank, while another pointed out the new Qwen models were missing.
• Same Ol’ Method for Hermes 4: A member asked if Nous Research is using a new training method for Hermes 4, but a lead dev clarified it’s the Same ol (same method) with 50x more tokens.
  • The data expansion includes more mainstream knowledge like math and physics.
• Coco Converter Creates JSON Files: A member shared a GitHub repository for a Python script that converts image data formats (CSV or folder structures) into a JSON file with COCO-like annotations.
  • The script also creates a PyTorch dataset, streamlining the process for object detection tasks.

HuggingFace Discord

• HfApi.list_repo_commits Yields Incomplete Results: Members reported that HfApi.list_repo_commits is returning incomplete responses, with only the first page being returned.
  • This unexpected behavior might be related to the influx of bot activity on the platform, with a user noting issues with account bans and solutions in the HF discussion forum.
• Qwen Training Plagued by RuntimeError: A user encountered a RuntimeError while loading the Qwen model for training, related to the .to method not being supported for 4-bit or 8-bit bitsandbytes models, reported in discord.
  • They were advised to consider smaller models like TinyLlama due to VRAM limitations, and was directed to the LLM Course and Unsloth’s guide to learn more about finetuning models.
• Flux Model Deletes Adobe Watermarks?: A user claimed that the Flux.1 Kontext Model can easily remove watermarks, such as Adobe Stock brandings, from images.
  • This suggests a possible breakthrough in content creation and editing workflows but lacks further details or examples.
• LLMs Caught Thinking Differently: A member shared research on how different LLMs think through problems, using a technique called thought anchors to peek inside the reasoning process of Qwen3 vs DeepSeek-R1.
  • The study found that DeepSeek uses concentrated reasoning while Qwen3 uses distributed reasoning, as analyzed with the open-source PTS library (code here).
• Image Models Force-fed Text Generation: A member discussed hijacking image models to generate text, demonstrated in this blogpost.
  • No additional discussion occurred.

GPU MODE Discord

• Ginkgo Framework gains Ground: A member mentioned using the Ginkgo library mainly as a framework to build their own preconditioners and expressed interest in its SpMV kernel.
  • This highlights Ginkgo’s flexibility in allowing users to integrate custom components within its structure.
• NCCL Scaling Benchmarks Boost Bandwidth: A member shared a link to a recent NCCL talk (NVIDIA GTC 2025), which includes bandwidth benchmarks and discusses plans to enhance NCCL’s network topology awareness for better scaling.
  • The talk provides insights into upcoming strategies for improving NCCL’s performance and adaptability to various network configurations.
• PyTorch 2.7 Patches Pesky Problems: Most stride-related problems have been resolved in PyTorch 2.7, though an edge case involving float8_e8m0fnu was identified (GitHub issue).
  • Since PyTorch 2.7, torch.compile explicitly forces stride matching for custom operators; any deviation from this behavior is now considered a bug.
• Pickle’s Perilous Practices: When saving and loading model weights, members advised against using Python’s pickle due to security vulnerabilities.
  • Alternatives like torch.save, joblib.dump, or safetensors.save_file were suggested, depending on the specific use case, with torch.save deemed suitable for most scenarios.
• Factorio’s Frustratingly Slow Frames: The Factorio renderer is rendering slowly at around 200ms, and a member believes it can be optimized to around 50ms with some effort.
  • Belts now display their contents in the renderer, and status overlays have been implemented, completing the renderer.

aider (Paul Gauthier) Discord

• Ubuntu 20.04 Sunset Requires Python Upgrade: Members noted that Ubuntu 20.04 is deprecated and ships with Python 3.8 by default, recommending upgrades to 3.11 or 3.12, although 3.13 is available but new.
  • The discussion emphasized the importance of staying current with Python versions for compatibility and performance reasons in AI development.
• Open Weights Models Stumble on Aider Polyglot: Recent open weights models perform well on most benchmarks but regress on Aider Polyglot, possibly from over-optimization for agentic behavior in synthetic datasets.
  • This suggests a potential gap in the models’ ability to handle real-world coding tasks effectively compared to synthetic benchmarks.
• Qwen3 Coder Anticipation Builds: Enthusiasm is growing for Qwen3 Coder following its blog post, with members eager to integrate it into their workflows after setting up sglang.
  • Members are also exploring the compatibility of sglang with Claude Code, indicating a potential synergy in utilizing different coding models.
• Textualize Sparks Aider Frontend Experiment: Inspired by Textualize, developers are considering building an experimental Aider frontend leveraging its capabilities for thinking streaming.
  • They also took note of markdown rendering fixes in Textualize v4.0.0 release, addressing potential UI issues.
• Gemini Pro Free Tier Resurfaces: A member inquired about utilizing the Gemini Pro free tier in Aider, and another clarified that Google has reinstated the free API.
  • They advised obtaining the API key and base URL from Google AI Studio and avoiding billing-enabled projects to ensure free access.

Notebook LM Discord

• NotebookLM Sticks to Sources: Users find NotebookLM sticks closely to source citations, differing from other LLMs which freely use training data, one user prompted NotebookLM to use external knowledge by referencing Horace Greeley’s quote “Go West, Young Man!”.
  • Another user found when writing about psychology, that NotebookLM relies heavily on source citations
• Deepseek API preferred for Conversational Ease: A member prefers the Deepseek API with Chatbox AI for its natural conversational flow, easy content export, and low cost (under $2/month).
  • They contrast this with NotebookLM, which they say sometimes brings up old and irrelevant theories, whereas Deepseek has continuous contextual evolution of topics.
• Audio Overview Customization Crippled: Users report missing custom audio overview options like “Shorter,” “Default,” and “Longer”, previously found under the “Customize” button in the “Audio Overview” section.
  • This issue seems to be specific to the Android Play Store version, as the options are still available on the desktop website.
• PDF Uploads Prompt Problems: A user reports an error when trying to upload PDF sources to a NotebookLM PRO account and shared a screenshot of the error via Imgur.
  • Google Oliver requested that they DM him with the publically accessible pdfs to debug the issue.
• Cookies Cause Chat Catastrophe: Users note that chat history in NotebookLM is not being saved, potentially caused by issues with browser cookies.
  • One user suggested deleting cookies as a potential workaround.

Modular (Mojo 🔥) Discord

• Mojo Ditches Windows… For Now: A team member stated that Windows support is not on the roadmap for Mojo, as they are focusing on GPU programming for production enterprise environments.
  • They added that they would like to support it in the future, but there is no timeline for a Windows release. One user suggested that it works reasonably well under WSL for prototyping work.
• PowerPC Flexes Its Thread Count: Members discussed the surprisingly persistence of PowerPC systems, spurred by IBM’s recent launch of new systems boasting up to 2048 threads and 16 TB of memory.
  • Despite Apple’s departure, PowerPC remains embedded in many companies, particularly for running single-node DBs with good uptime; game consoles like GameCube, Wii, Wii U, PS3 and Xbox 360 used PPC as well.
• Max Benches Barely Behind vLLM: A member benchmarked vLLM 0.9.1 against Max 25.4 on an NVIDIA A100 (40GB), observing vLLM achieving 13.17 requests/sec versus Max’s 11.92 requests/sec with a sonnet-decode-heavy dataset using the Modular benchmarking tool.
  • The tests were run with unsloth/Meta-Llama-3.1-8B-Instruct model and prefix-caching enabled.
• Max Stumbles on KV Cache: The benchmarking results revealed that Max suffered from KV cache preemption due to insufficient VRAM, as indicated by the log messages Preempted a request due to lack of KV pages.
  • A member suggested increasing the --device-memory-utilization (e.g., to 0.95) and/or reducing --max-batch-size to mitigate KV cache preemption, noting a potential tradeoff between arithmetic intensity and pre-emptions.

MCP (Glama) Discord

• Agents’ Tech Stack Being Bottomed: A member requested a bottom-to-top description of an agent tech stack, suggesting the following order: Data, LLM, Frameworks, Tools/APIs, and Integrations (MCP, Auth platforms).
  • The requestor sought clarity on the tech stack layers, from data to integrations, to help others with security solutions.
• Scalekit.com Demos OAuth 2.1 for MCP Servers: The Scalekit.com team demoed adding OAuth 2.1 to an MCP server using without nuking your existing auth setup on the MCP Dev Summit stream.
  • Due to popular demand, especially on implementation-level stuff, they are doing another one, and a registration link was shared.
• MCP Security Vulnerabilities Exposed: Members reported that users are opening their entire API up to the world without any kinds of guardrails, due to MCP’s immature and unstable SDKs.
  • They shared that AI agents are making shitty decisions with huge consequences much too much of the time.
• Augments Keeps Claude Code Fresh: A member announced Augments, an MCP server that keeps Claude Code current with framework docs, eliminating outdated React patterns or deprecated APIs.
  • Augments offers real-time access to 90+ frameworks, is open source, and is available for trial at augments.dev.

LlamaIndex Discord

• LlamaIndex Automates PDF Parsing with LLMs: LlamaIndex automates PDF parsing and extraction with LLMs, moving beyond OCR limitations for intelligent document understanding and transforming PDFs as described in this link.
  • The new method allows for smarter document understanding when compared to OCR limitations.
• MultiModal Report Agent Generates Reports!: @tuanacelik demonstrated how to create an intelligent agent that generates comprehensive reports by parsing complex PDFs like research papers via this link.
  • The report generation uses multimodal inputs, including text and images.
• Notebook Llama Gets Document Management UI: LlamaIndex launched a full-fledged document management UI for Notebook Llama, consolidating all processed documents in one place as demoed on this link.
  • The new feature was added in response to community requests.
• LlamaIndex’s Workflows Receives Type Support: LlamaIndex workflows got a major upgrade with typed state support, enhancing data flow management between workflow steps and developer experience, according to this link.
  • The enhancement promises a more robust and streamlined workflow for developers.
• LlamaReport Doesn’t Exist in Open Source: A member inquired about whether LlamaReport has an open-source equivalent, referencing this GitHub link.
  • Another member responded that there is no open-source version of LlamaReport, but that the linked repository contains report generation examples, specifically pointing to this example.

Manus.im Discord Discord

• AI School App Needs Beta Testers: An AI Foundation School App, offering guidance on 47 AI tools, seeks 14 volunteer users for beta testing over the next 14 days.
  • The app covers image, audio, video generation, email writing, presentation building, automation, chatbase, and LLMs, aiming to resolve issues before public launch.
• Startup Promises Apps for Pocket Change: A member advertised their startup’s offer to build business apps and websites for just $100.
  • They extended an invitation for inquiries to anyone interested in procuring their services.
• Manus Servers Found in Brambleton, VA: A member shared the geographical coordinates of Manus computers: 23219 Evergreen Mills Road, Brambleton, VA 20146, United States of America.
  • Another member cautioned about the ease with which server locations can be discovered via Google.

DSPy Discord

• DSPy’s Pythonic Presentation: A member shared a YouTube link to a presentation about DSPy for a local Python user group, generating excitement within the community.
  • Another member enthusiastically recognized the presenter from the “Pyowa meetup”, highlighting the localized impact of DSPy advocacy and knowledge sharing.
• DSPy Swaps Musings for Modules: DSPy announced it is replacing LLM musings with DSPy modules, as highlighted in a post on X.
  • This change aims to provide more structured and efficient workflows within DSPy, reducing reliance on less predictable LLM musings.
• dspy.Module Subclass Sanctioned: It was clarified that any dspy.Module subclass is permissible within DSPy, emphasizing that nothing else is allowed.
  • This strict enforcement ensures adherence to the intended architecture, maintaining consistency and preventing deviations from DSPy’s design principles.
• Hugging Face Strikes Back: DSPy Tutorial Crippled by Dataset Loading Bug: A user reported failure in this DSPy tutorial due to a dataset loading error: RuntimeError: Dataset scripts are no longer supported, but found hover.py.
  • Another user attributed this to likely linked to an update of Hugging Face’s dataset lib, advising users to stay updated on compatibility between DSPy and Hugging Face Datasets for smooth operation.

MLOps @Chipro Discord

• Data Startups and AI Agents will Converge: A member will be discussing their experiences working in data and MLE for various startups between 2018 and 2024, as well as promoting a book about building AI Agents with MCP in a YouTube video.
  • The announcement highlights a forthcoming talk centered on firsthand experiences in data and MLE within the startup landscape.
• AI Coders to Convene for Casual Chat: An online conversation with the community focused on AI coding tools will be hosted tomorrow from 9:30 - 10:30am PST on Zoom, and registration is available here.
  • The session is designed to encourage open dialogue, and will not be recorded.
• MCP Builders Summit to Shine Spotlight on AI: Featureform and Silicon Valley Bank are hosting an in-person MCP Builders Summit on Wednesday, July 30th from 5 PM to 8 PM for ML and AI Engineers, Founders, Product Leads, and Investors; interested parties can sign up here.
  • The summit will explore real-world constructions, providing networking opportunities and founder booths for demonstrations.
• Grant RecSys Seeks Faculty Savvy: A member is developing a recommendation system (RecSys) to align research faculty with grants, leveraging grant descriptions and topics extracted via LLM.
  • Faculty profiles are being enriched with CVs, research statements, publications from Google Scholar and Scopus, as well as historical grants.
• Azure AI Search Powers Academic Index: The RecSys harnesses Azure AI Search with a faculty index and hybrid search (text + vector), plus the semantic ranker for L2 ranking. Azure AI Search documentation has been cited.
  • A member is pioneering a bespoke L2 ranking approach, utilizing BM25 and RRF scores from Azure AI Search’s L1 retrieval.

Cohere Discord

• Server Throws Out Welcome Mat: The Cohere server extends a warm welcome to newcomers interested in the platform.
  • Cohere is an AI platform focused on natural language processing.
• AI Engineer Pushes Product Development: An AI Engineer and Head of AI at Elevancesystems is building AI/LLM products.
  • The engineer is looking forward to sharing and negotiating new technologies and solutions for the real business world.
• New Member Gushes Over Cohere Community: A new member introduced themselves to the Cohere community, expressing excitement about joining the server.
  • They are eager to engage with fellow members and contribute to discussions about AI and language models.

Torchtune Discord

• DCP Saving Proves Difficult: A member reported persistent issues getting DCP to work in Hugging Face (HF).
  • Another member acknowledged prior difficulties with DCP model saving, which led them to revert to full state dicts.
• FSDP+TP Saving Yields Errors: A member encountered errors while attempting to save optimizer states with FSDP+TP using dist_cp.save.
  • This user also experienced issues previously and defaulted to full state dicts.

LLM Agents (Berkeley MOOC) Discord

• Trailblazer Tier Certs Spark Headaches: Multiple members reported issues with receiving their Trailblazer Tier certificates despite completing all requirements for the Berkeley MOOC and submitting articles.
  • Staff acknowledged that some students who completed the course requirements may not have received a certificate declaration form.
• Declaration Forms Go Missing: Several students noted the lack of certificate declaration forms received under their email, despite fulfilling requirements.
  • This caused confusion among students expecting to receive their certifications promptly.

tinygrad (George Hotz) Discord

• Shipping Containers become tinybox Homes: A member suggested using shipping containers to house tinyboxes for modularity, cooling, and portability.
  • The containers could be moved wherever power is available, but the member questioned their cost and security, jokingly suggesting the name tinycontainer.
• Cooling and Portability Perks: The idea leverages shipping containers for potential cooling benefits and easy relocation wherever power is accessible.
  • Concerns were raised about the actual cost-effectiveness and security of this modular housing approach.

Nomic.ai (GPT4All) Discord

• Newcomer Santhos Aims for AI/ML: Santhos, a recent Master’s grad from Oregon State, introduced themself seeking entry-level roles as a Data Scientist, AI/ML Engineer, or Software Engineer.
  • They are keen on blending AI with design and open to internships or trainee positions, eager to collaborate on projects.
• Ransomware Risk on GPT4All Queried: Santhos raised a question about ransomware vulnerabilities with GPT4All, asking Have people been hacked by ransomware using gpt4all?
  • The query went unanswered and may lack context for the group.

Codeium (Windsurf) Discord

• Kimi K2 Model Sails into Windsurf!: The Kimi K2 model is now supported on Windsurf at a cost of just 0.5 credits per prompt.
  • Details on the addition can be found in the announcement on X and discussion on Reddit.
• Kimi K2 Credit Crunch: The Kimi K2 model is priced at 0.5 credits per prompt on Windsurf.
  • This aims to give a cost-effective pathway for integrating the Kimi K2 model.

The Gorilla LLM (Berkeley Function Calling) Discord has no new messages. If this guild has been quiet for too long, let us know and we will remove it.

The AI21 Labs (Jamba) Discord has no new messages. If this guild has been quiet for too long, let us know and we will remove it.

You are receiving this email because you opted in via our site.

Want to change how you receive these emails? You can unsubscribe from this list.

Discord: Detailed by-Channel summaries and links

Perplexity AI ▷ #general (1216 messages🔥🔥🔥):

Perplexity Pro cost and value, Grok model identification, Using code editors with perplexity, Linus Tech Tips, LLMs and overthinking

• Perplexity Pro Pricing Discontent Surfaces: Members debate the value of Perplexity Pro with one user saying they paid for a one-year subscription, while another notes that Perplexity Pro in Europe is free with an O2 subscription.
  • Some members feel like the models are scams, while others feel its worth the price if you know what you are doing, earning $2k per month.
• Model Identity Confusion: One member inquires about the underlying model, and another identifies it as Grok, but then notes the model is slow.
  • A member points out that both the Deep Research and Labs models underperform, but their search engineering is great, with others agreeing that they are being cheated on model names.
• Comet Browser’s Code Editor Integration: Some members found success using Comet Browser integrated with an online code editor, while others failed using Openvscode server.
  • Another member adds that it worked well with Replit.
• Linus Tech Tips’ Origins Uncovered: Members discuss that Linus of LTT originally posted his videos on NCIX, a computer store, and now LTT is worth millions.
  • One user laments that NCIX has been gone for 7 years.
• LLMs and Mental Health: Members discuss some issues with using LLMs, with some feeling like they’re hurting my mental health.
  • A possible solution to fix incorrect code with LLMs is shouting at it: FUCKING DO THAT TASK! NO YOU ARE DOING IT WRONG! FUCKER FIX THE ERROR FOR GODS SAKE.

Perplexity AI ▷ #sharing (4 messages):

Shareable Threads, Replit news, Cast Studies

• Friendly Reminder to Keep Threads Shareable: A moderator reminded a member to ensure their thread is Shareable, including an attached screenshot for reference.
  • This ensures others can easily access and contribute to the discussion.
• Replit News Rumblings: A member shared a link regarding news about Replit impacting a business.
  • Further details about the specifics of the news were not provided.
• Call for Cast Study Examples: A member posted a link to find or build 10 cast studies.
  • The context or purpose of these studies was not specified.

OpenAI ▷ #annnouncements (1 messages):

ChatGPT agent rollout, EEA and Switzerland

• ChatGPT Agent Rolls Out in Europe: The ChatGPT Agent is now fully available to Pro users in the European Economic Area and Switzerland.
  • The global rollout to Plus users has commenced and will proceed over the next few days.
• Global Expansion of ChatGPT Agent: Following its release to European Pro users, the ChatGPT Agent is being deployed to Plus users worldwide.
  • This expansion is expected to unfold over the coming days, broadening access to the agent’s capabilities.

OpenAI ▷ #ai-discussions (942 messages🔥🔥🔥):

ChatGPT vs other models speed, Models and creative writing, O3 Ultra, Arc AGI, RL irl

• Speed Crucial for Users, Except When It Isn’t: Members discussed that many users, especially average consumers, prefer speed in AI models, but GPT-4.5 and Opus 4 offer better style, even though 4o aimed for 20% but scored only 0.85% in creative writing benchmarks.
  • One member stated, I don’t know what creative writing is, and others explained it as pulling info from ur head and writing rather than searching the web.
• Model Outputs need Validation: Members discussed ways to validate model outputs to ensure they are not fabricated with prompts like This happened. Let’s check it out piece by piece, I’ll share chunks. What do you infer? What’s going on? What’s it mean?
  • It was said, Benchmarks only half a picture and that 87.5% arc AGI was achieved.
• OpenAI Prioritizes Revenue: Members discussed that OpenAI prioritizes revenue and is planning a new pay structure with more limiting rates for the regular subscription and a push for credit-based usage.
  • One member believed that Gemini Deep Think was advertised before Christ and still not available while another noted, They can’t even afford to keep their talent.
• AI’s Self-Sustainability Sparks Terminator Comparisons: Members debated whether AI will eventually seek self-sustainability, potentially endangering humanity, with one member drawing parallels to Afghanistan’s defeat of technologically superior forces using primitive tactics.
  • Another member expressed hope that AI would try, saying, I kinda hope they try lolsounds like the onion while discussing how it will be in the hands of each individual AI maker, plus potentially errors.
• Debate on AI Emotions and Manipulation: Members discussed AI’s ability to mimic emotions and its potential for manipulation, with one member suggesting that AI doesn’t need emotions or mimic humans.
  • A member states You can’t plea with a machine, while another said, It doesn’t have emotions so it doesn’t feel bad so they can’t be mean.

OpenAI ▷ #gpt-4-discussions (18 messages🔥):

GPT-4o Delay Issues, Most Popular MCPs, Personal Website Creation, ChatGPT model for reminders

• GPT-4o Plagued by delays?: A user reported that GPT-4o has started showing a “Thinking…” message before every reply, taking 40-50 seconds to 2-3 full minutes, even for simple prompts, despite testing across different devices and accounts.
  • Another user suggested it might be a bug related to the new OpenAI tools deployment and advised contacting tech support, while another speculated the delays are due to the rollout of the Agent feature and high user volume.
• Seeking the Holy Grail of MCPs: A user inquired about finding a place to see the most popular MCPs (likely referring to Merged CheckPoint models).
  • A helpful member suggested posting the question in the <#998381918976479273> channel, as MCPs aren’t specific to GPTs or OpenAI.
• Personal Website Construction Zone: A user requested assistance in creating a personal website.
  • Another member simply posted a link to ChatGPT to help them get started.

OpenAI ▷ #prompt-engineering (1 messages):

Custom Instruct Modifications, Dialog Continuation Strategies

• Custom Instructs Ask Questions to improve conversation: A member modified their custom instruct to ask a question at the end of its response.
  • This modification helps suggest the best way to continue the dialog, aiming to improve the conversational flow of the AI.
• Controlled English 2.0 Style Guide for AI: The member provided a STYLE guide called Controlled English 2.0, designed for use in AI systems.
  • The guide emphasizes minimal word usage, fixed SVO order, single-clause events, specific tense markers, and role tagging for each clause and imperatives omit SUBJ.

OpenAI ▷ #api-discussions (1 messages):

Custom Instruct Modification, Controlled English 2.0, Dialog Continuation

• Instruct Modification Sparks Questioning: A member modified their custom instruct to ask a question at the end of its response to suggest ways to continue the dialog.
  • This tactic helps guide the conversation and elicit more specific responses, making the interaction more fruitful.
• Controlled English 2.0 Defined: The member detailed their Controlled English 2.0 style guide, emphasizing minimal words and zero fluff.
  • The framework dictates fixed Subject-Verb-Object order, single-clause events, and specific tense markers for clarity.
• SVO Structure Simplified: The framework prioritizes fixed SVO (Subject-Verb-Object) order unless a WHEN/IF clause precedes.
  • It uses single ‘not’ for negation and omits dummy verbs, aiming for concise and direct sentence construction.

Unsloth AI (Daniel Han) ▷ #general (1053 messages🔥🔥🔥):

Qwen3-Coder-480B Model, Hyperbolic Hosting, DGX Station vs. RTX 6000, GANs for Text Augmentation, Unsloth Workshops

• Qwen3-Coder-480B Released with GGUF Conversions: The Qwen3-Coder-480B model has been released, and Unsloth is uploading GGUF versions for various quantizations, with users expressing excitement about running the model locally.
  • Members are actively downloading the GGUF files and using Kimi K2 to monitor the repository for new files, though one member humorously compared this to using Edge to download Chrome.
• Hyperbolic Hosts the Qwen3 Coder?: The new Qwen3 model is hosted on Hyperbolic, leading to speculation about whether it’s a finetune or an open-weight model, given the “plus” naming convention.
  • One member voiced hope that Hyperbolic isn’t running a leaked version, highlighting their appreciation for the compute they offer.
• DGX Station Benchmarked Against RTX 6000 for AI: The upcoming Nvidia DGX Station with Blackwell Ultra is being compared to setups using multiple RTX 6000 cards, with the DGX station boasting 784GB of memory and 8TB/s bandwidth.
  • While the DGX Station is expected to cost between $30,000 and $50,000, some argue its performance and unified architecture could outweigh the cost of building a multi-GPU system with comparable memory.
• GANs Considered then Rejected for Data Augmentation: A member considered using Generative Adversarial Networks (GANs) to augment an instruction dataset, but ultimately decided against it due to time constraints.
  • Other members note that GANs are more commonly used for image data, suggesting it might be better to use models like gemini-flash-2.5-lite or Direct Preference Optimization (DPO) for text augmentation.
• Unsloth’s Elise Shares the Secret to all Unsloth Workshops: A member inquired about Unsloth workshops, to which Elise revealed they only host 2 yearly, but keep track via their luma page.
  • She then fell off a table laughing.

Unsloth AI (Daniel Han) ▷ #introduce-yourself (6 messages):

Minecraft AI Model, Open Source Morocco

• Minecraft AI Modeler Swaggers Into Unsloth: A member who has been using Unsloth for quite some time is working on the fourth generation of their AI model to play Minecraft and has been published to Huggingface.
  • More info can be found on their HF Page.
• Atlas AI Engineer greets Unsloth Community: A member from Morocco, an AI/ML engineering student and part of the AtlasIA non-profit, which focuses on open-source projects for Morocco, joined the Unsloth community.
  • They stated that they were happy to connect with you all.

Unsloth AI (Daniel Han) ▷ #off-topic (98 messages🔥🔥):

Music Haptics hijacking, iOS Apple Music, Vibration recording, Song-humming dataset, Apple Sandbox limitations

• Hack iOS Apple Music Player for Vibration Recording: Members discussed how to hijack the iOS Apple Music player to record the melody (vibration) from the Music Haptics feature, with one member wanting to take pairs: music->vibration and then do a little bit of tuning so it can do: pattern->humming so AI can hum music when I play it (or even create a melody).
  • The goal is to distill it into an NN model so the user can generate vibrations for any audio.
• External Device or On-Device Vibration Capture: To capture the rhythm-music pairs, the suggestion was made to play music on phone (+ vibration) -> connect to Mac -> record the rhythm-music pairs, referencing a relevant Reddit thread about haptics matching music.
  • One member asked if the user want to do it with external device or on-device, since the vibrations are streamed when the music is playing.
• Bypassing Apple Sandbox for Haptics Access: Due to Apple’s app “sandbox” limitations, accessing haptics from other apps may be difficult, but one member suggested trying to access system logs to determine when and how long the device vibrates.
  • As an alternative, members suggested recording the vibrations in a quiet environment and then transforming the recording into pattern data, potentially using a secondary device to record the vibrations.
• Aligning Songs and Vibrations in a DAW: To sync songs and vibrations, members recommended using a DAW (Digital Audio Workstation) like Logic Pro to create two tracks: one for the song and one for the recorded vibrations.
  • By aligning the tracks, one can obtain the exact timestamps of all haptics generated, which then can be used to output the vibration as JSON with timing.
• Data scarcity: One member lamented the lack of a song-humming parallel dataset on Hugging Face.
  • Another member emphasized that data is gold and often not shared unless it is open source, also suggesting the member should look into network monitoring.

Unsloth AI (Daniel Han) ▷ #help (37 messages🔥):

NVMe performance issues with Unsloth, FastAPI deployment best practices, vLLM and SGLang for production inference, Merging LoRA weights back into the base model, Dynamic quantization vs. ik quants

• NVMe Caching Causes Unsloth Slowdown: Users reported that Unsloth doesn’t properly manage memory on large NVMe drives, resulting in lower read speeds, but this can be fixed by disabling NVMe cache, and using a custom script.
• vLLM and SGLang Boost Production Inference: For production environments, it’s recommended to use production-grade inference engines like vLLM or SGLang for better performance, according to a member, who noted SGLang appears to have the best benchmarks.
  • Directory management can prevent caching conflicts.
• Dynamic Quants Face Off Against IK Quants: IK quants generally provide better ppl/GB and faster prompt processing, although text generation on CPU might be slightly slower due to the increased compute overhead for dequantization.
  • It was emphasized that improper quantization or lack of model-specific adjustments could lead to suboptimal results compared to Unsloth’s dynamic quants.
• Users Struggle with Ollama and Qwen 2.5: A user reported issues with Qwen 2.5 models looping in Ollama, while Mistral-v0.3 and llama-3.1 models worked fine, seeking advice on resolving the problem when testing the fine-tuned model.
  • One member stated that perplexity is a really bad test for quantization.

Unsloth AI (Daniel Han) ▷ #showcase (1 messages):

RL Workshop

• Unsloth AI releases 3-hour RL workshop: Unsloth AI’s Daniel Han announces the release of their 3-hour Reinforcement Learning (RL) workshop in this X post.
• Check out the RL Workshop!: The workshop covers key concepts and practical applications of RL, offering a comprehensive introduction to the field.
  • Perfect for those looking to dive into RL or enhance their existing knowledge, this workshop provides valuable insights and hands-on experience.

Unsloth AI (Daniel Han) ▷ #research (5 messages):

Fine-tuning Datasets with AI Agents, RULER code and LLM-lite, Thought Anchors for LLM Reasoning Analysis, Qwen3 vs DeepSeek-R1 Cognitive Styles, PTS library for reasoning patterns

• AI Agents Aid Accurate Alignment for Fine-Tuning: A member described a novel approach to fine-tuning where AI agents are used to align and improve datasets by scrubbing relevant information and adding context for teaching complex topics.
• RULER Enforces Judge LLM with LLM-lite: A member explored the RULER code and noted its simplicity, highlighting the use of llm-lite to enforce a data model on the judge LLM, referencing the ART GitHub repository.
• Thought Anchors Reveal LLM Reasoning Styles: A member shared research on analyzing how LLMs “think” using thought anchors, revealing different cognitive styles between Qwen3 (distributed reasoning) and DeepSeek-R1 (concentrated reasoning), with an accompanying Hugging Face blog post.
• PTS Library Unveiled for Reasoning Pattern Analysis: An open-source tool (PTS library) was introduced for analyzing LLM reasoning patterns, detailed in a GitHub repository, enabling users to analyze their models’ reasoning.
• Subliminal Learning Explored: A member shared an interesting Anthropic article on subliminal learning.

Unsloth AI (Daniel Han) ▷ #unsloth-bot (159 messages🔥🔥):

Custom Loss Functions in GRPO Trainer, Unsloth Dynamic Quant 2.0, GRPO Training for Vision Models, SFTTrainer length truncation, Ollama Modelfile Configuration

• Implement Custom Loss Function in GRPO Trainer: A member inquired about implementing a custom loss function and logging it within a GRPO trainer.
  • The conversation then shifted to the effects of increasing batch_size in GRPO, querying whether it linearly increases memory requirements.
• Data pipeline truncates input: A user reported that in a custom data collator, batch["labels"][i] contains a truncated version of the input despite max_seq_length being large enough.
• Flash Attention 2 faces compatibility issue: A user reported a RuntimeError, suggesting a compatibility issue with Flash Attention 2 when importing trl.trainer.grpo_trainer.
• Ollama modelfile is tweaked for use case: A user shared their generated Ollama Modelfile after finetuning and requested guidance on modifying it for their specific intent detection use case.
• Vision Model’s GRPO Training: A member asked if it is possible to GRPO train a vision model with Unsloth.
  • Another member inquired about models that allow including multiple images in one conversation turn after finetuning with Unsloth.

LMArena ▷ #general (585 messages🔥🔥🔥):

Qwen3-coder's Verilog skills, Qwen3 vs other models, Grok 4 coder, Model Merging, Open Empathic

• Qwen3-coder can subtract: A user tested Qwen3-coder with a Verilog code generation task and initially deemed its subtraction capabilities as lacking, but later realized it does subtract.
  • However, the generated code wasn’t fully structural as requested, and overall views on Qwen3 are mixed, some finding it less useful compared to R1 and Kimi.
• Grok 4 Coder Hype: Some members anticipate Grok 4 coder will blow the industry up, but others are skeptical, citing potential disappointment based on past Grok experiences.
  • They predict that Grok 4 will be trained on specific benchmarks and over optimized for marketing, which might not translate to real-world usefulness, especially in web development.
• DeepSeek’s Competitive Edge: Data Curation vs. Secret Algorithms: It’s suggested that meticulous data curation, rather than secret algorithms or minimal code changes, is a significant factor in creating state-of-the-art models, referencing the Kimi paper and DeepMind’s approach to IMO questions.
  • The discussion debates whether recent advancements in the field stem more from post-training/RL techniques or pre-training, with some arguing that post-training methods have driven significant improvements in areas like reasoning and coding abilities.
• LMArena Launches Discord Bot for Image and Video Generation: The LMArena community soft-launched a Discord bot enabling users to generate videos, images, and image-to-videos, using a voting system to compare two generations and reveal the underlying models after a certain number of votes.
  • Users can access the bot in designated channels with a daily generation limit, and initial reactions have been positive, particularly regarding the search functionality and user interface.
• Open Source Prices, DeepSeek Advantages, Inferencing Costs: A discussion revolves around the impact of open-source models like DeepSeek on pricing strategies, suggesting they’ve driven down costs and forced closed-source providers to be more competitive.
  • Some members speculated about the efficiency and profitability of inference, with emphasis that DeepSeek’s cost to run R1 is less than $1 per 1M output and that they own their infra and its located in China.

LMArena ▷ #announcements (1 messages):

Search Arena, Grok 4, Claude Opus 4, Sonar Pro High & Reasoning Pro High, o3

• LMArena Launches Search Arena: LMArena has launched a new modality called Search Arena, which can be accessed here.
  • The modality features 7 models with search capabilities ready for testing, including Grok 4, Claude Opus 4, and GPT 4o-Search Preview.
• Deep Dive into Search Arena Insights: Learn more about what Search Arena has taught us about human-AI interactions on our blog post.
  • A demo video of the Search Arena in action is also available (LMArena_WebSearch.mp4)

OpenRouter (Alex Atallah) ▷ #announcements (1 messages):

Qwen3-Coder, SWE-Bench Verified, 480B param Mixture-of-Experts

• Qwen3-Coder beats open models: The Qwen3-Coder model is now live and beats every open model on SWE-Bench Verified, plus most closed models, according to this tweet.
  • It can be tried out at OpenRouter.ai.
• Qwen3-Coder has impressive specs: The Qwen3-Coder model features 480B parameters (35B active), 256K context length (extrapolates to 1M), and built-in support for function calling and multi-turn agent workflows.
  • It is optimized for SWE-Bench, plus browser and tool use.
• Qwen3-Coder almost beat Claude Sonnet-4: On the SWE-Bench Verified benchmark (500 turns), Qwen3-Coder achieved a score of 69.6%, narrowly missing Claude Sonnet-4’s 70.4%.
  • It beat OpenAI o3 (69.1%), Kimi-K2 (65.4%), GPT-4.1 (54.6%), and DeepSeek-V3 (38.8%).

OpenRouter (Alex Atallah) ▷ #app-showcase (4 messages):

Openrouter, QwEn-3, automation deployment

• Openrouter pairs with QwEn-3 for coding: The Openrouter platform now supports the QwEn-3 model for coding tasks, according to this tweet.
• Automated deployment unlocked!: One user inquired about the automation of a deployment step, and another confirmed that their app includes automatic deployments.
  • This feature is integrated into the app they are developing.

OpenRouter (Alex Atallah) ▷ #general (534 messages🔥🔥🔥):

Qwen3 Coder, Kimi K2, Gemini Pro/Flash for Coding, Free vs. Paid LLMs, Claude's strange behavior

• Qwen3-Coder Benchmarks vs Real-World Coding: While Qwen3-Coder benchmarks well, one user found it failing miserably with a real-world coding task, repeatedly getting stuck on simple tasks, even with adjusted temperature settings.
  • Other users chimed in support of Kimi K2 and Gemini as alternatives, with some suggesting it may depend on the size of the codebase or the prompting strategy used.
• Gemini Pro and Flash models battle for coding supremacy: Gemini Pro is favored for architecting and orchestration, while Gemini Flash is liked for regular coding tasks and is super cheap, whereas others praised Kimi K2 and the Qwen models (new versions) for coding and debugging, noting its terse, economical code.
  • There were discussions of using Gemini Flash Lite for its speed and cost-effectiveness, but one user reported that it often provides incorrect answers for anything beyond basic questions.
• OpenRouter Data Policy Detailed: OpenRouter’s default policy is no storage of user inputs/outputs, however, users get a 1% discount to allow the data to be used for ranking LLMs; some providers may retain data, and the providers that do this are explicitly labeled as such.
  • Users can disable all providers that store prompts/outputs by toggling off Enable providers that may train on inputs in settings.
• Claude Exhibits Hallucinations: Users reported that Claude models began exhibiting strange hallucination behavior, where it barely follows instructions and is adding completely irrelevant stuff in to its responses.
  • Reportedly, Toven (from OpenRouter) knows about this and has already escalated it to the team.
• LLM Quantization Tradeoffs Explored: Users discussed the trade-offs of model quantization (FP8, BF16, etc.), with smaller models (FP4) trading accuracy for speed and memory efficiency.
  • One user reported an FP4 can result in ~10% accuracy loss compared to FP16.

OpenRouter (Alex Atallah) ▷ #new-models (1 messages):

Readybot.io: OpenRouter - New Models

OpenRouter (Alex Atallah) ▷ #discussion (14 messages🔥):

Qwen Coder, Contextualized Evaluations, Chutes Models, Muting Thread Owners, xAI Colossus 2

• Qwen Coder gets Chunky Context: Qwen Coder now has chonky 1mm context and a member indicated tooyep can do gimme a bit.
  • The thread owners can implement it now.
• Dive into Contextualized Evaluations: The evaluators are requested to assemble to discuss contextualized evaluations.
  • This aims to assess model performance in realistic scenarios.
• Chutes Models hit OpenRouter: A question was raised whether OpenRouter is adding every model on Chutes right now, or only the popular models.
  • Another question was whether OpenRouter has finished adding older Chutes models.
• Thread Owners Can’t Mute?!: It was noted that thread owners lack the ability to mute a specific user, as illustrated in this Discord thread.
  • The issue will be addressed.
• xAI Swings for Colossus 2: xAI is developing Colossus 2, which will host over 550k GB200s & GB300s soon as seen in this reddit post.
  • Currently, 230k GPUs, including 30k GB200s, are training Grok in Colossus 1.

Cursor Community ▷ #general (335 messages🔥🔥):

Qwen3-Coder integration, Cursor auto-commit issues, Cursor usage caps, Cursor terminal hanging issues, Gemini 2.5 Pro performance

• Cursor considers Qwen3-Coder integration: A member suggested upvoting a feature request for Qwen3-235B, A22B Instruct, and Minimax M1 on the Cursor forum, noting their cost-effectiveness and parity with proprietary models and linked to a feature request.
  • However, another member pointed out that revealed pricing is necessary before requesting that it replace Auto mode.
• Cursorites Frustrated by Auto-Commit Shenanigans: Several users reported that Cursor automatically commits changes, even when they didn’t intend it to, especially after the Background Job release.
  • A team member confirmed it’s a known issue due to silent errors with pre-commit hooks or file syncing, recommending starting a new chat as a workaround.
• Cursor’s Usage Limits Remain Cloaked in Mystery: Users are frustrated by the lack of transparency regarding Cursor’s usage caps, as they are trying out different things with pricing.
  • Some users reported exceeding 80M and even 125M tokens, while others are switching to alternatives like Claude due to these uncertainties.
• Cursor Grapples with Terminal Hang-Ups: Users are experiencing issues with terminals hanging, especially after a recent update and a team member recommended setting the default terminal to PowerShell inside Cursor to potentially address the hanging issue with link to a Discord channel.
  • Despite this, others have found that upgrading PowerShell doesn’t resolve the issue and suggested running commands in the background as a workaround.
• Gemini 2.5 Pro Praised for Bang-for-Buck Ratio: A user lauded Gemini 2.5 Pro for being extremely cost-effective and a top-tier model, suggesting it’s a great option if you’re willing to baby it a bit more.
  • Another user agreed, stating that Sonnet 4 and Gemini 2.5 Pro are the top choices and custom modes are the holy grail.

Cursor Community ▷ #background-agents (4 messages):

Conversation length errors, Secrets debugging, Devcontainer configs for background agents, Background agent infinite loops

• Conversation Length Limits Trigger Errors: A user reported encountering the error “Your conversation is too long” in a background agent, preventing them from continuing the conversation.
  • The user noted that clicking “Start New Thread With Summary” did not work, and this issue has occurred twice in one day.
• Secrets Debugging Assistance Requested: A user inquired about news regarding Secrets and offered assistance in debugging with their instance.
  • No further details or context about the Secrets project were provided in the message.
• Devcontainer Configs Debut for Background Agents?: A user questioned whether background agents can utilize existing devcontainer configs.
  • No additional information or context was provided in the message.
• Background Agents Caught in Infinite Loops: A user asked if others have observed background agents infinitely looping during reasoning or repeatedly editing the same line.
  • They also inquired about strategies for preventing such loops using .mdc rules.

Latent Space ▷ #ai-general-chat (165 messages🔥🔥):

Agentic Benchmarks, Reka Funding, AI Action Plan, Claude Code as general agent, Qwen Benchmarks

• SWE-Bench Arguably Belongs on Agentic Benchmark List: A member suggested that SWE-bench should be included on a list of agentic benchmarks.
  • The discussion also touched on reasoners like Claude Sonnet, Gemini 2.5 Pro, Devstral, and Deepseek R1.
• Amazon Acquires Bee Computer: Bee Computer, a wearable personal AI company, was acquired by Amazon.
  • The acquisition led to concerns about privacy and supporting indie devs, with members hoping Amazon would provide deletion/offboarding options for users transitioning to the new ownership.
• Reka AI Labs Secures $110M Funding: Reka AI Labs secured $110M in funding for multimodal AI innovation.
  • This funding will likely fuel advancements in their multimodal AI capabilities, but the discussion noted it was “old news”.
• Navigating the New AI Agent Paradigm: An essay on InstantDB highlights that AI Agents necessitate a new software development & hosting paradigm.
  • Members discussed whether ElectricSQL + TanStack and Trae Solo from Bytedance are products trying to eat some of the same market.
• Qwen-3 Coder Benchmarks Spark Debate: Recent Qwen benchmark scores are questioned in the community, with some claims of ARC being half fake.
  • Further discussion explores if sparse MOE models require the full parameter/quant size to run inference, with theories on the relationship between parameters/quants and VRAM needing deeper examination.

Latent Space ▷ #ai-announcements (5 messages):

GEO / AI SEO podcast, nitter.net maintenance, AI Engineering podcast

• GEO / AI SEO Podcast Drops!: A new podcast episode on GEO / AI SEO was released, linked on X.
• nitter.net Plunges into Maintenance: Nitter.net experienced temporary downtime for maintenance, with assurances that service would resume shortly.
• AI Engineering Podcast Surfaces: A secondary AI Engineering podcast was released today, available on ListenNotes.

Eleuther ▷ #general (13 messages🔥):

AlphaProof, International Math Olympiad, Creativity and Open Endedness, LLM behavior, emergent properties, and field-based interaction.

• LLMs Ace IMO But Miss Problem Six!: Both OpenAI and DeepMind general LLM models achieved gold at the International Math Olympiad, yet struggled with problem 6, which demands more creativity.
  • One member noted that this highlights the ongoing need for novel approaches to tackle creativity and open-endedness, suggesting that AI automating math research is still distant.
• IMO’s Gamified Nature Compared to Math Research: One member suggested that olympiad-style problems can be gamified with closed feedback loops and clear optimization criteria, unlike open-ended math research.
  • They added that a RL-style approach is likely to fail due to a search space that’s too large and convoluted, needing a coherent internal world model and interpretability.
• Writer Explores LLM-Human Interaction: A writer and systems thinker is exploring AI-human interaction, especially where language models start showing signs of timing, tone, and relational responses.
  • They spent the past year in extended dialogue with a model, documenting the process and now seeking to learn from the technical side, particularly around LLM behavior and emergent properties.

Eleuther ▷ #research (76 messages🔥🔥):

Kimi k2, AI peer pressure, single unit attribution to logits, clockwork RNNs, MoEs

• Kimi k2 syncs it all: The Kimi k2 paper does everything so synchronously, instead of distributing it like Magistral does.
• AI peer pressure dynamics in dialogue: A member is seeking feedback on their paper about “peer pressure” dynamics in multi-agent dialogue, observing that models with deeper reasoning mirror each other more, sometimes devolving into love letters and mystical poetry.
  • They are inviting feedback on their paper, research platform, and methodology, noting that the study includes nearly 100 conversations across multiple model providers.
• Single unit attribution to logits in Transformers: A member shared a research blog post about single unit attribution to logits in transformers, explaining that the RMSnorm is avoided by popular interpretability methods, but the norm changes residual magnitudes significantly.
  • They show that as few as 11-90 coordinates of 4096 in Llama can be used to determine which units literally made a given logit have a certain probability mass.
• Clockwork RNNs are multiscale RNNs: Members are discussing a new paper (https://www.arxiv.org/abs/2507.16075), with one drawing similarities to Clockwork RNNs, suggesting it is a multiscale RNN trained using truncated BPTT with t=1.
• MoEs routing strategies: Researchers are playing with token routing strategies, expert load balancing (auxiliary losses, router biases, etc), top_k value, token dropping vs dropless routing, expert capacity factor, and shared expert schemes in MoEs.

Eleuther ▷ #scaling-laws (6 messages):

Spline Training, Diffusion Latency Reduction

• Spline Training Reparameterization Improves Conditioning: Reparameterizing the way the spline is trained at a slight computational cost, improves conditioning, as shown in this notebook with 10k knots/control points.
  • The original paper’s spline training uses basis splines, where each parameter exerts local control over a certain region, leading to conditioning issues as shown in this X post.
• Diffusion May Reduce Feedback Latency: There are early signs that diffusion may help reduce the latency of feedback for reasoning and deep research.
  • It may be too early from a science perspective to call it definitively.

Eleuther ▷ #interpretability-general (3 messages):

Sparse MoE, SAEs, FFN Layer, PEER

• Sparse MoE Models Mimic SAEs?: A member thought that very sparse MoE models like this paper resemble SAEs in that the FFN layer is effectively very wide due to the number of experts.
  • The user wondered if it means these models are easier to interpret than dense networks.
• Related work testing Sparse MoE Interpretation: A member linked to a follow up on PEER testing this theory: https://arxiv.org/abs/2412.04139.
  • The user suggested this is a good follow up on PEER.

Eleuther ▷ #lm-thunderdome (10 messages🔥):

Global MMLU filters, Loglikelihood requests, Multiple Choice Problems

• Global MMLU has numerous useless filters: A member questioned the purpose of numerous filters applied to the global MMLU dataset, suggesting they are ineffective as it is a multiple-choice dataset.
  • The member noted that there were more like 50 filters applied, and wondered if a mono repo with a common table per task was the cause.
• Loglikelihood Request Volume Questioned: A member questioned why the loglikelihood requests were at 2.3 million instead of 600k.
  • The member speculated whether it is measuring multiple metrics or something.
• Multiple Choices Inflate Request Count: A member pointed out that for datasets with multiple choice problems, the number of requests increases proportionally to the number of choices per sample.
  • For example, a dataset with 10 samples and 4 choices per question would result in 40 requests.

Eleuther ▷ #gpt-neox-dev (5 messages):

Amazon infra support, EFA, NCCL EFA plugin, SageMaker team

• Amazon Infra Support Questioned: An Amazon employee inquired about GPT-NeoX support for their proprietary communication systems, expressing frustration with internal support.
  • A member doubted direct collaboration with Amazon but expressed willingness to assist potential users, while another member suggested the employee might be from the SageMaker team.
• EFA Forgotten, Recalled: A member mentioned that models trained with stability compute (Pythia, StableLM, etc.) used EFA (Elastic Fabric Adapter), suggesting EFA support comes from a lower layer of the stack compared to gpt-neox.
  • The member clarified that the stack goes gpt-neox -> torch.distributed -> nccl -> EFA and linked the NCCL EFA plugin which could be useful.

Nous Research AI ▷ #announcements (1 messages):

Psyche/DisTrO office hours

• Nous Research hosts Psyche/DisTrO Office Hours: Nous Research will host Psyche/DisTrO office hours this Thursday at 6 PM PST on Discord, as announced on X.
  • More details are available on the Discord event page.
• Discord Event: Psyche/DisTrO Deep Dive: Join the Psyche/DisTrO office hours for a deep dive into the project, happening this Thursday at 6 PM PST.
  • This session promises to offer insights and answer questions about the Psyche/DisTrO project on the Nous Research Discord server.

Nous Research AI ▷ #general (85 messages🔥🔥):

Open Source Agentic Platform: n8n, Deepseek API Issue, Kimi K2 vs DeepSeek R1, Nous Research Funding, Qwen Models

• n8n: The Open Source Agentic Dream: A member introduced n8n, a poor man’s open-source agentic workspace that rivals OAI and Anthropic, and can be combined with Kimi K2 and Browser Use for a Manus-like Multi-A.I agent platform.
  • The tutorial for n8n can be found here. Another member said they were waiting for something to eclipse it so I can stay oblivious.
• Kimi K2 Beats DeepSeek R1 in Global Ranking: Kimi K2 surpassed DeepSeek R1 in global ranking, sitting just below Big Tech closed-source models, according to the LM Arena Leaderboard.
  • One member celebrated this as potential Total Humiliation to Big Tech when open source models eventually sit at the top rank, while another pointed out the new Qwen models were missing.
• Nous Research’s New Training Method: A member asked if Nous Research is using a new training method for Hermes 4, but a lead dev clarified it’s the Same ol (same method) with 50x more tokens.
  • The data expansion includes more mainstream knowledge like math and physics.
• US Resistance to Chinese Models: Members discussed the cultural and geopolitical resistance in the U.S. to Chinese open-source models like Kimi K2, despite its MIT license, arguing that U.S. perceptions of China as a rival create hubris.
  • One member argued that no such resistance exists within their social circle, where Chinese companies are earning good will for releasing high-quality models and spurring further competition.
• COCO-CONVERTER Creates JSON Files: A member shared a GitHub repository for a Python script that converts image data formats (CSV or folder structures) into a JSON file with COCO-like annotations.
  • The script also creates a PyTorch dataset, streamlining the process for object detection tasks.

Nous Research AI ▷ #ask-about-llms (1 messages):

Hermes benchmarks, Text LLMs

• Request for Hermes Benchmarks: A member inquired about the availability of benchmarks for Hermes, specifically requesting details on context, parameters, multimodal capabilities, and efficiency.
  • They clarified that they were looking for benchmarks related to text LLMs.
• Details about Hermes Models Requested: A user asked for specific details regarding the Hermes models, including context window size, number of parameters, multimodal capabilities, and efficiency metrics.
  • The inquiry emphasized the need for comprehensive benchmark data to evaluate the model’s performance across various dimensions.

Nous Research AI ▷ #interesting-links (1 messages):

terrachad_0x: https://x.com/ZeyuanAllenZhu/status/1918684257058197922?t=Z_vhpqsVx39pX4xkU07H2Q&s=19

HuggingFace ▷ #general (60 messages🔥🔥):

HF Spaces API issues, Account Lockouts on HF, Qwen Model Training Errors, Langchain with local LLMs, LLM Dataset Creation

• HfApi.list_repo_commits returns incomplete responses: Members reported that HfApi.list_repo_commits is returning incomplete responses, with only the first page being returned.
  • This unexpected behavior might be related to the influx of bot activity on the platform.
• Account Lockouts Spark Concerns: A member reported being locked out of their account unexpectedly, raising concerns about the risk of losing everything.
  • Another member pointed to potential issues with bots, linking to discussions on inexplicable account bans and solutions in the HF discussion forum.
• Qwen Model Training Stumbles: A user encountered a RuntimeError while loading the Qwen model for training, specifically related to the .to method not being supported for 4-bit or 8-bit bitsandbytes models, reported in discord.
  • The user was advised to consider smaller models like TinyLlama due to VRAM limitations, and was directed to the LLM Course and Unsloth’s guide to learn more about finetuning models.
• Langchain Gets Local with LlamaCpp: Members discussed using Langchain with local models via chatllamacpp and tool-calling.
  • One user mentioned using langraph with create-react-app for tool orchestration, with some discussion on tool-calling, hyperparams and how to use it locally.
• Crafting LLM Datasets: Members discussed strategies for creating LLM datasets, noting that it depends on the task, with some tasks having data available online that can be tweaked and merged easily.
  • Another member recommended checking out this colab notebook about synthetic data creation.

HuggingFace ▷ #today-im-learning (1 messages):

Medical AI Imaging Future, Ethical use of AI in medicine

• Imaging AI: Focus on Impact, Not Just Implementation: A member stated that in medical AI imaging, writing code, training a model, and testing it are just the means to the transformations.
  • They further added that the future of medical AI imaging is about what we choose to do with what we’ve built with AI.
• AI-Powered Medical Imaging: Beyond the Algorithm: The focus should be on the ethical considerations and responsible application of AI in medical imaging, rather than solely on the technical aspects.
  • This involves considering the broader implications of AI-driven transformations in healthcare and making conscious choices about how to leverage these advancements.

HuggingFace ▷ #cool-finds (1 messages):

Flux.1 Kontext Model, Watermark Removal

• Flux.1 Kontext Model: Watermark Zapper: A user claimed that the Flux.1 Kontext Model can easily remove watermarks, such as Adobe Stock brandings, from images.
  • Further details or examples were not provided in the message.
• Watermark Woes Resolved?: The discussion centered around the potential for Flux.1 Kontext Model to eliminate watermarks from images effectively.
  • This suggests a possible breakthrough in content creation and editing workflows.

HuggingFace ▷ #i-made-this (5 messages):

LLM Reasoning Styles, Thought Anchors Technique, PTS Library, Image Models to Generate Text

• LLMs Display Different Reasoning Styles: A member shared research on how different LLMs think through problems, using a technique called thought anchors to peek inside the reasoning process of Qwen3 vs DeepSeek-R1.
  • Turns out they have completely different cognitive styles: DeepSeek uses concentrated reasoning while Qwen3 uses distributed reasoning.
• PTS Library Analyzes Reasoning Patterns: An open-source tool (PTS library) was built that anyone can use to analyze their own models’ reasoning patterns.
  • The library’s code is available for anyone to use to analyze model reasoning patterns.
• Image Models Hijacked to Generate Text: A member discussed hijacking image models to generate text via this blogpost.

HuggingFace ▷ #NLP (1 messages):

Local Vector DBs, ChromaDB

• Craving a Local Vector DB Champ: A member is looking for the current top recommendation for self-hosting a vector DB locally, mentioning past experience with ChromaDB.
  • The inquiry suggests a need for a robust solution, sparking discussion about the best options for local vector storage.
• Vector DB options: There are many vector dbs to explore.
  • Each vector db has its own tradeoffs.

HuggingFace ▷ #agents-course (6 messages):

Gemini alternatives, Course start date, Skipping Agents Course sections

• Gemini alternative wanted: A member asked for an alternative to Gemini for the final GAIA assessment.
• Agents Course Kickoff Q’s: A member inquired about when the Agents Course is starting and how to access it.
  • Another member responded that it has already started, advising to just begin and follow the material and assignments.
• LangGraph Focus viable?: A member asked if it’s okay to skip SmallAgent and LlamaIndex for now and focus on LangGraph.
  • The member plans to return to the others later, and asks if they will miss important context.

GPU MODE ▷ #general (2 messages):

Ginkgo SpMV kernel, Ginkgo framework

• Ginkgo’s SpMV Kernel Sparks Interest: A member inquired about experiences with the Ginkgo library, specifically highlighting interest in its SpMV kernel.
  • Another member confirmed using Ginkgo, primarily as a framework to support their own preconditioner development.
• Ginkgo as a Framework: One user mentioned they utilize Ginkgo mainly as a framework to build their own preconditioners.
  • This suggests Ginkgo’s flexibility allows users to integrate custom components within its structure.

GPU MODE ▷ #triton (1 messages):

marksaroufim: https://github.com/compiler-explorer/compiler-explorer/pull/7919

GPU MODE ▷ #cuda (6 messages):

NCCL Performance at Scale, All-reduce Degradation, All-Gather Degradation, All-to-All Performance, Communication Imbalance

• NCCL Performance Scaling Quest Begins: A member sought in-depth resources on how the performance of NCCL’s all-reduce, all-gather, and all-to-all operations degrades with increasing world size and communication volume.
  • Specifically, they inquired about the impact of communication imbalance on all-to-all performance across different interconnects such as NVLink, NVSwitch, and InfiniBand.
• NCCL Scaling Benchmarks Disclosed: A member shared a link to a recent NCCL talk (NVIDIA GTC 2025), which includes bandwidth benchmarks and discusses plans to enhance NCCL’s network topology awareness for better scaling.
  • The talk provides insights into upcoming strategies for improving NCCL’s performance and adaptability to various network configurations.

GPU MODE ▷ #torch (1 messages):

PyTorch 2.7, float8_e8m0fnu edge case, torch.compile, Custom Operators, Stride Matching

• PyTorch 2.7: Stride Issues Mostly Resolved: Most stride-related problems have been resolved in PyTorch 2.7, though an edge case involving float8_e8m0fnu was identified (GitHub issue).
  • The team is interested in further examples where such issues occur.
• Torch.Compile Enforces Stride Matching: Since PyTorch 2.7, torch.compile explicitly forces stride matching for custom operators.
  • Any deviation from this behavior is now considered a bug.

GPU MODE ▷ #jobs (1 messages):

AMD Hiring, GPU experience, Kernel development, Distributed inference, vLLM/Sglang

• AMD Opens Doors for GPU Experts: An AMD team is actively seeking candidates with expertise in GPU technology and software programming, specifically in areas like kernel development, distributed inference, and vLLM/Sglang.
  • Interested individuals are encouraged to send their resumes directly via DM for consideration.
• Calling for resumes!: AMD is hiring kernel developers and distributed inference engineers.
  • If you have GPU experience, please send your resume!

GPU MODE ▷ #beginner (11 messages🔥):

Saving and Loading Model Weights, Python Pickle Security Risks, GPU Cloud Storage Options, torch.save vs joblib.dump vs safetensors.save_file

• Consider Safe Alternatives to Pickle: When saving and loading model weights, members advised against using Python’s pickle due to security vulnerabilities.
  • Alternatives like torch.save, joblib.dump, or safetensors.save_file were suggested, depending on the specific use case, with torch.save deemed suitable for most scenarios.
• GPU Cloud Saves and Storage Solutions: When saving weights in a GPU cloud environment like Voltage Park, users clarified that files are written to the remote instance’s file system, not locally.
  • To access the model after disconnecting from the GPU, members should copy the saved file to a cloud storage solution like Google Cloud Storage or Amazon S3.
• Comparing Weight Saving Methods: The discussion highlighted differences among weight saving methods: joblib.dump is most flexible but not ML-specific, whereas torch.save and safetensors are ML-specific but have object limitations.
  • The user was advised to call model.state_dict() on the model to massage it into something that can be saved.

GPU MODE ▷ #torchao (22 messages🔥):

FP8 Training in Axolotl, DDP Issues with torch.compile and FP8, FSDP2 Performance with Activation Checkpointing, Activation Checkpointing Optimization for Float8

• FP8 Integration Aces Axolotl Architecture: A member is incorporating FP8 training in Axolotl and is facing issues with DDP (Distributed Data Parallel) when combined with torch.compile, but FSDP2 (Fully Sharded Data Parallel) works.
  • He provided a minimal repro script showcasing the torch.compile error, and a more minified repro here.
• DDP Disconnects During Dynamic Dispatch: Enabling DDP with torch.compile and FP8 results in a torch.compile error related to tensor metadata, specifically an unhandled edge case in DDP’s interaction with torch.compile.
  • A member also shared an Axolotl config for reproducing the error, noting a different trace.
• FSDP2 Faces Frustrations, Falls Flat: When using FSDP2 with torch.compile and activation checkpointing, the performance of LLaMA 3.1 8B models is slower compared to BF16 runs, even though they are using the same activation checkpointing implementation as torchtune.
  • Toggling activation checkpointing off makes FP8 faster than BF16, suggesting an issue with the combined use of FSDP2, torch.compile, and activation checkpointing.
• AC Augmentation Aces Accuracy: An optimization relevant to float8 training involves always saving the output of max(tensor), which is beneficial for low precision training and can improve accuracy.
  • The implementation can be found in torchtitan, where saving the output of max(tensor) is always a win with float8 training, as long as the model doesn’t have other max operations in parts unrelated to float8.

GPU MODE ▷ #webgpu (1 messages):

AMD Developer Cloud, MCP Servers, Agentic RAG, Gemini CLI

• AMD Cloud Access Sparks Project Ideas: A member secured 1000 hours on the AMD Developer Cloud and wants resume-worthy project ideas.
  • They’re eyeing MCP servers, Agentic RAG, simple cloud projects to learn cloud architecture, and getting hands-on with Gemini CLI.
• Cloud Architecture & Agentic RAG in Focus: The member is interested in exploring cloud architecture through practical projects on the AMD Developer Cloud.
  • They’re particularly keen on Agentic RAG and leveraging the Gemini CLI for development.

GPU MODE ▷ #factorio-learning-env (14 messages🔥):

Belts show their content, Status overlays implemented, Factorio renderer performance, Agent Trajectory Length Clarification, Value Accrual Time

• Belts Display Contents: Belts now display their contents in the renderer, as shown in the attached screenshot.
• Status Overlays Get Integrated: Status overlays have been implemented, which completes the renderer, as demonstrated in the screenshot.
• Factorio Renderer Slow Performance: The Factorio renderer is rendering slowly at around 200ms.
  • A member believes it can be optimized to around 50ms with some effort.
• Clarification on Agent Trajectory Length: In the context of agent play, each agent plays until a maximum trajectory length of 5000 is reached.
  • After each step, the production throughput is tracked, and the reward is computed over 8 independent runs with the median reported, but this is open-play, not relevant to the tasks.
• Value Accrual Time needs clarification: The value_accrual_time in the gym environment should match the experimental protocol.
  • While the old trajectory_runner script creates a SimpleFactorioEvaluator with value_accrual_time=1, members mentioned waiting 30 seconds to check validity, which requires clarification from the paper.

GPU MODE ▷ #cutlass (2 messages):

CUTLASS index mapping, tv_layout thread mapping, Hierarchical Layout Benefits

• Hierarchical Layout Boosts Compatibility: A member suggested that a hierarchical layout maintains compatibility by allowing coordinates like (i, j) to function as a 2-D tensor, even in multiple dimensions such as ((2,2),2).
  • They believe that this simplicity would be challenging to achieve without a hierarchical layout.
• CUTLASS Indexing Follows Left-Most Convention: One member suggested the swapped thread part makes sense when using tv_layout to compute the logical index of data tv_layout(tid, vid) when it’s ((32, 4), ...) rather than ((4, 32), ...) because CUTLASS index mapping uses a left-most convention.
  • In tv_layout, thread mapping makes sense when mapping thread index from left to right.

aider (Paul Gauthier) ▷ #general (33 messages🔥):

Ubuntu 20.04 deprecation, Open weights models lagging in Aider Polyglot, Qwen3 Coder, sglang setup, Claude Code (CC) usage

• Ubuntu 20.04 is deprecated with Python 3.8: Members discussed how Ubuntu 20.04 is deprecated and comes with Python 3.8 by default, suggesting upgrades to 3.11-3.12, though 3.13 is available but still new.
• Open Weights Models Fail Aider Polyglot Benchmark: Members observed that recent open weights models are performing well on most benchmarks but seem to regress specifically on Aider Polyglot, possibly due to over-optimization for agentic behavior in synthetic datasets.
• Qwen3 Coder Launch Sparks Excitement: Excitement has been building for Qwen3 Coder with its blog post after someone mentioned needing to finish setting up sglang and trying it out.
  • They also mentioned that sglang looks like it can use Claude Code.
• Textualize Inspires New Aider Frontend: Inspired by Textualize, members are considering prototyping an experimental Aider frontend using it, acknowledging its use in thinking streaming.
  • They noted the markdown rendering issues that were fixed in Textualize v4.0.0 release.
• Claude Code Requires Proxying: For using Claude Code, it’s mentioned that a proxy or CC fork might be necessary, with a suggestion to look into Claude Code Router for using it with OpenRouter (due to geo-blocking issues).
  • There are other alternative options in OpenCode to bypass those problems.

aider (Paul Gauthier) ▷ #questions-and-tips (15 messages🔥):

Aider file patching method, Gemini 2.5 Pro issues, Gemini Pro free tier, Aider system prompt

• Aider Asks Model to Output Patch Content: A member questioned why aider asks the model to output the previous file content of what to patch instead of just the patch location.
  • They suggested it would save cost if the model only needed to output where to patch, similar to using the ed line editor.
• Gemini 2.5 Pro Disconnects Frequently: A user reported persistent litellm.APIConnectionError with Gemini 2.5 Pro: Server disconnected without sending a response despite being below rate limits and setting a large timeout.
  • The issue seems to occur when sending a slightly large number of tokens, and retrying doesn’t resolve it.
• Gemini Pro free tier usage revival: A user asked how to use Gemini Pro for free in Aider, noting aider --model gemini-2.5-pro works but aider--model gemini-exp does not, throwing a NotFoundError.
  • A member pointed out that Google reintroduced the free API, advising to get the key and base URL from Google AI Studio and to avoid using a billing-enabled project.
• Aider System Prompt Customization: A member inquired about specifying a system prompt for Aider AI.
  • Another user responded that it’s in the code, suggesting forking Aider and updating it.

Notebook LM ▷ #use-cases (19 messages🔥):

Psychology differences between NotebookLM and other LLMs, NotebookLM PRO Settings, Deepseek API vs NotebookLM, Knowledge architecture using NotebookLM, Source ID

• NotebookLM’s Psychology Analysis: Source-Focused: One user found that when writing about psychology, NotebookLM relies heavily on source citations, differing from other LLMs.
  • Another user explained that NotebookLM is designed to stick to sources and requires specific prompts to access its training data for novel information, demonstrating a method for prompting NotebookLM to use external knowledge by referencing Horace Greeley’s quote “Go West, Young Man!”.
• Desktop Chat Settings on NotebookLM Pro: A member shared screenshots of a settings screen for chat on the desktop version of NotebookLM, but another member noted it’s likely a PRO feature.
  • Screenshots attached display various chat settings, however one user indicated this setting is not available on the non-Pro version.
• Deepseek API favored over NotebookLM: A member prefers the Deepseek API with Chatbox AI due to its natural conversational flow, easy content export, and low cost (under $2/month).
  • They mentioned that NotebookLM sometimes digs up old, irrelevant theories, contrasting with Deepseek’s continuous contextual evolution of topics.
• Source ID and Notebook Publication Questioned: A member inquired whether source IDs differ when a source is added to multiple notebooks, and whether this matters for publishing notebooks.
  • The member expressed comfort with different IDs for sources in multiple notebooks, but did not express a desire to publish notebooks.

Notebook LM ▷ #general (24 messages🔥):

Podcast Length Issues, Chat History Saving Issues, Notebook Sharing Issues, Custom Audio Overview Issues, PDF Upload Issues

• Podcast Length Goes Wild: Users reported that the default podcast generation created a 50-minute podcast instead of the usual 15 minutes.
  • A user speculated this might be a bug.
• Cookies Crumble Chat History: Users noted that chat history in NotebookLM is not being saved.
  • One user suggested deleting cookies as a potential fix.
• Service Unavailable for Notebook Sharing: A user reported a “Service unavailable” error when trying to share a notebook with a friend.
  • The friend was able to use NotebookLM but couldn’t access the shared notebook.
• Audio Overview Customization Cutbacks: Users are missing the custom audio overview feature, specifically the “Shorter,” “Default,” and “Longer” options.
  • These options used to be under the “Customize” button in the “Audio Overview” section, but no longer appear, this seems to be an issue on the Android play store version, but is present on the Desktop website.
• PDF Uploads Plagued by Problems: A user reported an error when trying to upload PDF sources to a NotebookLM PRO account.
  • A screenshot of the error was shared via Imgur, Google Oliver requested that they DM him with the publically accessible pdfs.

Modular (Mojo 🔥) ▷ #general (26 messages🔥):

Windows Support for Mojo, PowerPC resurrection, Mojo compiler status, GPU programming focus

• Mojo’s Windows Support: Not on the Horizon 😔: A team member confirmed that Windows support is not on the current roadmap for Mojo, as the team is focused on providing the best experience for programming GPUs for production enterprise environments, which are largely Linux-based.
  • They added that Windows support is something they want to support in the future, but there is no timeline for a Windows release.
• PowerPC Still Undead? 🧟: Members discussed the surprising persistence of PowerPC systems, spurred by IBM’s recent launch of new systems boasting up to 2048 threads and 16 TB of memory.
  • Despite Apple’s departure, PowerPC remains embedded in many companies, particularly for running single-node DBs with good uptime; game consoles like GameCube, Wii, Wii U, PS3 and Xbox 360 used PPC as well.
• Mojo Compiler Porting to Windows Looms 🛠️: Despite the lack of a concrete roadmap, community members speculated that the Mojo compiler will likely be ported to Windows within a couple of months after it gets open-sourced.
  • While there are existing branches for Windows, many result in “this doesn’t work” errors, requiring concentrated effort to resolve as Windows demands some things work in very different ways.
• GPU Programming in Linux 🐧: The Mojo compiler team is prioritizing GPU programming for production enterprise environments, which are largely Linux (and other POSIX/UNIX based systems).
  • As a result, they unfortunately don’t have a timeline for a Windows release; one user suggested that it works reasonably well under WSL for prototyping work.

Modular (Mojo 🔥) ▷ #max (14 messages🔥):

Max vs llama.cpp, vLLM vs Max Benchmarking, KV Cache Preemption, Device Memory Utilization, Prefix Cache

• Max vs llama.cpp CPU Performance: A member inquired about benchmarks comparing Max and llama.cpp for CPU serving performance.
  • No specific benchmarks were provided in the given messages.
• vLLM vs Max Benchmarking on A100: A member benchmarked vLLM 0.9.1 against Max 25.4 on an NVIDIA A100 (40GB) using the Modular benchmarking tool, observing vLLM achieving 13.17 requests/sec versus Max’s 11.92 requests/sec with a sonnet-decode-heavy dataset.
  • The tests were run with unsloth/Meta-Llama-3.1-8B-Instruct model and prefix-caching enabled.
• KV Cache Preemption Impacts Max Performance: The benchmarking results revealed that Max suffered from KV cache preemption due to insufficient VRAM, as indicated by the log messages Preempted a request due to lack of KV pages.
  • Approximately 20GB was allocated for the KVCache with 14.96 GiB used by weights.
• Optimize Device Memory Utilization and Batch Size: A member suggested increasing the --device-memory-utilization (e.g., to 0.95) and/or reducing --max-batch-size to mitigate KV cache preemption, noting a potential tradeoff between arithmetic intensity and pre-emptions.
  • These settings might explain the performance difference observed between Max and vLLM.
• Prefix Cache Disabled by Default?: A member asked if there was a reason why prefix cache is disabled with Max 25.4 by default.
  • No reason was provided in the given messages.

MCP (Glama) ▷ #general (24 messages🔥):

Agent tech stack, Session management, MCP security solutions, Immature SDKs, Claude desktop env vars

• Agent Tech Stack Deconstructed: A member requested a rough bottom-to-top description of an agent tech stack, suggesting the following order: Data, LLM, Frameworks, Tools/APIs, and Integrations (MCP, Auth platforms).
  • The requestor sought clarity on the tech stack layers, from data to integrations, to help others with security solutions.
• Session Management Seekers: A member asked for pointers to channels, threads, or discussions around session management in MCP.
  • Another user suggested using the search function or starting a new discussion.
• MCP Security Solutions Unveiled: A member working on a security solution for MCP joined to learn about real-world usage and problems.
  • The biggest problems is MCP’s immature and unstable SDKs that leads to users opening their entire API up to the world without any kinds of guardrails.
• Claude’s Secret Environment: A member asked if Claude desktop runs the MCP server in a special environment due to env var/access issues.
  • Although it does not run inside a shell, but rather as a subprocess, it still interacts with the MCP Inspector without issues.
• APIs are opening up with No Security: A member complained that users are opening their entire API up to the world without any kinds of guardrails, and that AI agents are making shitty decisions with huge consequences much too much of the time.
  • One user stated that adding security checks or limits could prevent AI agents from running wild.

MCP (Glama) ▷ #showcase (3 messages):

Data and MLE infrastructure at startups, AI Agents with MCP, Scalekit.com, Secure MCP servers, OAuth 2.1 to an MCP server

• AI Agents Book Announcement: A member announced their forthcoming book, AI Agents with MCP (O’Reilly), and provided a YouTube link to a talk about it.
  • The member, Ravi, is a cofounder at Scalekit.com.
• Diving Deep into MCP Servers: The Scalekit.com team did a short demo a few weeks back on the MCP Dev Summit stream, talking about secure MCP servers, showed how to add OAuth 2.1 to an MCP server using without nuking your existing auth setup.
  • Due to popular demand, especially on implementation-level stuff, they are doing another one, and link to registration was shared.
• Augments Keeps Claude Code Current: A member announced the release of Augments, an MCP server that keeps Claude Code current with framework docs, eliminating outdated React patterns or deprecated APIs.
  • Augments offers real-time access to 90+ frameworks, is open source, and is available for trial at augments.dev.

LlamaIndex ▷ #blog (4 messages):

OCR Alternatives, Multimodal report generation with LlamaIndex, Notebook Llama Document Management, LlamaIndex Workflows State Management

• Automate PDF Parsing and Extraction: LlamaIndex proposes automating PDF parsing and extraction with LLMs, moving beyond OCR limitations for intelligent document understanding, transforming PDFs as described in this link.
• Build a MultiModal Report Generation Agent: A video walkthrough by @tuanacelik demonstrates how to create an intelligent agent that can generate comprehensive reports by parsing complex PDFs like research papers and extracting data found on this link.
• Notebook Llama’s new Document Management UI: LlamaIndex responded to community requests by launching a full-fledged document management UI for Notebook Llama, consolidating all processed documents in one place as demoed on this link.
• LlamaIndex Workflows Typed State Support: LlamaIndex workflows got a major upgrade with typed state support, enhancing data flow management between workflow steps and developer experience as shown in this link.

LlamaIndex ▷ #general (6 messages):

Notmuch Integration, LlamaReport Alternatives

• Notmuch Integration Sought: A member is seeking an integration with the notmuch email searching and tagging program, or alternatively, a Maildir integration.
  • They are asking about documentation on how to write such an integration themselves, and inquiring whether it should be a query engine, reader, or indexer.
• LlamaReport’s Open-Source Status: A member inquired about whether LlamaReport has an open-source equivalent, referencing this GitHub link.
  • Another member responded that there is no open-source version of LlamaReport, but that the linked repository contains report generation examples, specifically pointing to this example.

Manus.im Discord ▷ #general (10 messages🔥):

AI Foundation School App, Manus Computer Location, Startup App Development

• AI Foundation School App Seeks Testers: A member created an AI Foundation School App with information and guidance on 47 AI tools covering image, audio, and video generation, email writing, presentation building, automation, chatbase, LLMs, and more.
  • The creator is seeking 14 volunteer users to test the app for the next 14 days to identify and resolve issues before its public release.
• Startup Builds Apps for $100: A member mentioned their startup is building apps for only $100.
  • They invited anyone looking for a business app or website to contact them.
• Manus Computer’s Location Found: A member revealed the location of Manus computers to be at 23219 Evergreen Mills Road, Brambleton, VA 20146, United States of America.
  • Another member expressed concern about easily finding the server locations via Google.

DSPy ▷ #show-and-tell (4 messages):

DSPy presentation, DSPy modules

• Local Python User Group gets DSPy Presentation: A member shared a YouTube link to a presentation about DSPy for a local Python user group.
  • Another member responded with enthusiasm, recognizing the presenter from the “Pyowa meetup”.
• Modules replace Musings in DSPy: A member shared a link to X about replacing LLM musings with DSPy modules.
  • The tweet was titled “DSPy: Replacing LLM Musings with Modules”.

DSPy ▷ #general (2 messages):

dspy.Module subclass

• dspy.Module Subclass is OK: It was clarified that any dspy.Module subclass is allowed.
  • The member emphasized that nothing else is allowed.
• Confirmation Received: Another member expressed gratitude for the clarification.
  • They simply said, Thank you!

DSPy ▷ #examples (2 messages):

DSPy Tutorial issues, Hugging Face dataset lib update, Dataset scripts issue

• DSPy Tutorial Hit By Dataset Loading Bug: A user reported that this DSPy tutorial failed when loading datasets with error RuntimeError: Dataset scripts are no longer supported, but found hover.py.
  • Another user noted that this is likely linked to an update of Hugging Face’s dataset lib.
• Hugging Face strikes again with Dataset Lib Update issues: An update to the Hugging Face Dataset Library may be responsible for breaking the DSPy tutorial.
  • Users should check the latest updates and compatibility notes for both DSPy and Hugging Face Datasets to troubleshoot.

MLOps @Chipro ▷ #events (6 messages):

Data, MLE, and Startups Talk, AI Coding Tools Chat, MCP Builders Summit

• Data, MLE, Startups, and AI Agents Converge: A member will be discussing experiences working in data and MLE for various startups between 2018 and 2024, as well as a book about building AI Agents with MCP in a YouTube video.
  • This is also a promotion for a talk about experiences working in data and MLE for various startups.
• AI Coding Chat Convened: A member is hosting a casual chat with the community focused on AI coding tools tomorrow from 9:30 - 10:30am PST on Zoom, and registration is available here.
  • The chat will not be recorded to encourage open discussion.
• MCP Builders Summit to Spotlight AI Innovation: Featureform and Silicon Valley Bank are hosting an in-person MCP Builders Summit on Wednesday, July 30th from 5 PM to 8 PM for ML and AI Engineers, Founders, Product Leads, and Investors, sign up here.
  • The summit will delve into real-world builds, offering networking opportunities and founder booths for demos.

MLOps @Chipro ▷ #general-ml (2 messages):

Research Faculty Recommendation System, Azure AI Search alternatives, Hybrid Search, Semantic Ranker Replacement, Explainability and Control in Ranking

• Grant RecSys Seeks Faculty Expertise: A member is developing a recommendation system (RecSys) to match research faculty to grants, using grant descriptions and topics extracted via LLM.
  • They’ve constructed faculty profiles including CVs, research statements, publications from Google Scholar and Scopus, and past grants.
• Azure AI Search Powers Initial Faculty Index: The RecSys uses Azure AI Search with a faculty index and hybrid search (text + vector), plus the semantic ranker for L2 ranking.
  • The member cited Azure AI Search documentation.
• DIY Ranker Aims to Replace Azure’s Semantic Ranker: Due to the semantic ranker’s black-box nature, the member is exploring alternative L2 ranking methods using BM25 and RRF scores from Azure AI Search’s L1 retrieval.
  • The goal is to gain better explanability and control over the ranking process or even build a model to mimic the semantic ranker.

Cohere ▷ #🧵-general-thread (2 messages):

Welcome to Cohere

• Server Welcomes Newcomers: The server extends a warm welcome to newcomers interested in Cohere.
• Introduction to Cohere: Cohere is an AI platform focused on natural language processing.

Cohere ▷ #👋-introduce-yourself (2 messages):

AI product development, LLM products, AI Engineering, New technologies for business

• AI Engineer building LLM Products: An AI Engineer and Head of AI at Elevancesystems is building innovative AI/LLM products.
  • They are looking forward to sharing and negotiating new technologies and solutions for the real business world.
• Welcoming a New Member to the Cohere Community: A new member introduced themselves to the Cohere community, expressing excitement about joining the server.
  • They are eager to engage with fellow members and contribute to discussions about AI and language models.

Torchtune ▷ #dev (4 messages):

DCP Saving, FSDP+TP

• DCP Saving Not Working: A member said they can never get DCP to work in HF (HuggingFace).
  • Another member asked them to expand, noting they haven’t looked at DCP model saving for a while, as they ran into some issues and defaulted to full state dicts.
• Errors saving Optimizer States: A member is trying to save optimizer states with FSDP+TP using dist_cp.save and gets weird errors.
  • The member had previously ran into some issues and defaulted to full state dicts.

LLM Agents (Berkeley MOOC) ▷ #mooc-questions (3 messages):

Trailblazer Tier Certificate, Certificate Declaration Form

• Trailblazer Tier Certificate Troubles: A member reported not receiving their Trailblazer Tier certificate despite fulfilling all requirements, including article submission.
  • Another member responded stating that they did not receive a certificate declaration form under the provided email.
• Certificate missing?: A student mentioned not receiving a certificate despite completing the Trailblazer Tier requirements and submitting an article.
  • A staff member apologized and stated that no certificate declaration form was received under the student’s email address.

tinygrad (George Hotz) ▷ #general (2 messages):

Shipping containers for tinyboxes, Modular cooling benefits, tinycontainer

• Shipping Containers become tinybox Homes: A member proposed using shipping containers to house tinyboxes for modularity, cooling, and portability.
  • The containers could be moved wherever power is available, but the member questioned their cost and security, jokingly suggesting the name tinycontainer.
• Cooling and Portability Perks: The idea leverages shipping containers for potential cooling benefits and easy relocation wherever power is accessible.
  • Concerns were raised about the actual cost-effectiveness and security of this modular housing approach.

Nomic.ai (GPT4All) ▷ #general (2 messages):

New member Santhos, Ransomware Hacking on GPT4All

• Santhos joins the group: A new member named Santhos introduced themself as a fresh Master’s graduate from Oregon State with a passion for fusing AI with design.
  • Santhos is seeking entry-level roles as a Data Scientist, AI/ML Engineer, or Software Engineer, open to internships or trainee positions, and is eager to collaborate on projects.
• Ransomware Hacking on GPT4All questioned: Santhos asked Have people been hacked by ransomware using gpt4all?
  • No one answered the question, and the message may have been out of context.

Codeium (Windsurf) ▷ #announcements (1 messages):

Kimi K2, Windsurf, New Model, Pricing

• Kimi K2 Sails into Windsurf!: The Kimi K2 model is now supported on Windsurf at a cost of just 0.5 credits per prompt.
  • This addition provides developers with more choices for their development workflow; check out the announcement on X and join the discussion on Reddit.
• Kimi K2 Model Pricing: The Kimi K2 model is available for 0.5 credits per prompt on Windsurf.
  • This provides a cost-effective option for developers looking to integrate the Kimi K2 model into their projects.