All tags
Topic: "rnn"
not much happened this weekend
o1-preview claude-3.5-sonnet 21b-flash-model openai meta-ai-fair reka langchainai entropix prompting-techniques finetuning entropy-based-sampling temporal-understanding native-audio tool-use instruction-chaining multimodality retrieval-augmented-generation synthetic-data-generation rnn parallel-training biologically-inspired-ai-safety text-to-video-generation video-editing lex-fridman imrat jjitsev giffmana _philschmid karpathy rasbt adcock_brett glennko rohanpaul_ai labenz
AI news from 10/4/2024 to 10/7/2024 highlights several developments: OpenAI's o1-preview shows strong performance on complex tasks but struggles with simpler ones, while Claude 3.5 Sonnet can match its reasoning through advanced prompting techniques. Meta introduced Movie Gen, a cutting-edge media foundation model for text-to-video generation and editing. Reka updated their 21B Flash Model with temporal video understanding, native audio, and tool use capabilities. Interest grows in "open o1" reproductions focusing on prompting and finetuning, with Entropix exploring entropy-based sampling. LangChainAI demonstrated a Retrieval Agent for complex Q&A, and synthetic data generation research surveyed 417 models. A resurgence in RNNs shows efficient parallel training making them competitive with Transformers. Biologically-inspired AI safety approaches were also noted. "A quiet weekend and air conditioning is all you need."
Test-Time Training, MobileLLM, Lilian Weng on Hallucination (Plus: Turbopuffer)
llama-2-7b codegeex4-all-9b mamba facebook-research meta-ai-fair tsinghua-university hallucination-detection anti-hallucination-methods on-device-ai model-architecture rnn long-context-modeling model-scaling expressive-hidden-states code-generation lilian-weng yann-lecun
Lilian Weng released a comprehensive literature review on hallucination detection and anti-hallucination methods including techniques like FactualityPrompt, SelfCheckGPT, and WebGPT. Facebook AI Research (FAIR) published MobileLLM, a sub-billion parameter on-device language model architecture achieving performance comparable to llama-2-7b with innovations like thin and deep models and shared weights. A new RNN-based LLM architecture with expressive hidden states was introduced, replacing attention mechanisms and scaling better than Mamba and Transformer models for long-context modeling. Additionally, Tsinghua University open sourced CodeGeeX4-ALL-9B, a multilingual code generation model excelling in code assistance.