All tags
Company: "zep"
Creating a LLM-as-a-Judge
claude-3.5-sonnet claude-3.5 notebooklm simpleqa recraft-v3 anthropic openai deepmind apple zep perplexity-ai github critique-shadowing llm-judging domain-experts dataset-creation prompt-engineering error-analysis temporal-knowledge-graphs memory-layer ai-agent-memory hallucination-reduction integration hamel-husain swyx
Anthropic released details on Claude 3.5 SWEBench+SWEAgent, while OpenAI introduced SimpleQA and DeepMind launched NotebookLM. Apple announced new M4 Macbooks, and a new SOTA image model, Recraft v3, emerged. Hamel Husain presented a detailed 6,000-word treatise on creating LLM judges using a method called critique shadowing to align LLMs with domain experts, addressing the problem of untrusted and unused data in AI teams. The workflow involves expert-reviewed datasets and iterative prompt refinement. Additionally, Zep introduced a temporal knowledge graph memory layer to improve AI agent memory and reduce hallucinations. Anthropic also integrated Claude 3.5 Sonnet with GitHub Copilot, expanding access to Copilot Chat users.
Claude 3.5 Sonnet (New) gets Computer Use
claude-3.5-sonnet claude-3.5-haiku llama-3.1 nemotron anthropic zep nvidia coding benchmarks computer-use vision multimodal-memory model-updates ai-integration philschmid swyx
Anthropic announced new Claude 3.5 models: 3.5 Sonnet and 3.5 Haiku, improving coding performance significantly, with Sonnet topping several coding benchmarks like Aider and Vectara. The new Computer Use API enables controlling computers via vision, scoring notably higher than other AI systems, showcasing progress in AI-driven computer interaction. Zep launched a cloud edition for AI agents memory management, highlighting challenges in multimodal memory. The update also mentions Llama 3.1 and Nemotron models from NVIDIA.
Did Nvidia's Nemotron 70B train on test?
nemotron-70b llama-3.1-70b llama-3.1 ministral-3b ministral-8b gpt-4o claude-3.5-sonnet claude-3.5 nvidia mistral-ai hugging-face zep benchmarking reinforcement-learning reward-models temporal-knowledge-graphs memory-layers context-windows model-releases open-source reach_vb philschmid swyx
NVIDIA's Nemotron-70B model has drawn scrutiny despite strong benchmark performances on Arena Hard, AlpacaEval, and MT-Bench, with some standard benchmarks like GPQA and MMLU Pro showing no improvement over the base Llama-3.1-70B. The new HelpSteer2-Preference dataset improves some benchmarks with minimal losses elsewhere. Meanwhile, Mistral released Ministral 3B and 8B models featuring 128k context length and outperforming Llama-3.1 and GPT-4o on various benchmarks under the Mistral Commercial License. NVIDIA's Nemotron 70B also surpasses GPT-4o and Claude-3.5-Sonnet on key benchmarks using RLHF (REINFORCE) training. Additionally, Zep introduced Graphiti, an open-source temporal knowledge graph memory layer for AI agents, built on Neo4j.
The AI Nobel Prize
claude-3.5-sonnet reka-flash got openai anthropic reka-ai zep artificial-neural-networks nobel-prize knowledge-graphs memory-layers real-time-voice-api vision fine-tuning prompt-caching multimodality function-calling ocr open-source single-sign-on software-testing ai-assisted-coding ai-ethics geoff-hinton john-hopfield philschmid alexalbert mervenoyann clementdelangue svpino bindureddy ylecun rohanpaul_ai
Geoff Hinton and John Hopfield won the Nobel Prize in Physics for their work on Artificial Neural Networks. The award citation spans 14 pages highlighting their contributions. Zep released a new community edition of their low-latency memory layer for AI agents, emphasizing knowledge graphs for memory. At OpenAI's DevDay, new features like real-time voice API, vision model fine-tuning, and prompt caching with a 50% discount on reused tokens were introduced. Anthropic's Claude 3.5 Sonnet was recognized as the best model currently. Reka AI Labs updated their Reka Flash model with enhanced multimodal and function calling capabilities. The GOT (Generic OCR Transformer) achieved 98.79% accuracy on OCR benchmarks. Discussions on open-source AI models highlighted their role in fostering competition and decentralization. Software development insights included the importance of Single Sign-On (SSO), thorough testing, and AI-assisted coding workflows. Ethical and societal topics covered critiques of tax policies and the appointment of France's first Minister of AI.