Marktechpost AI Newsletter: Mistral AI Releases Codestral-22B + Samba-1-Turbo + MAP-Neo + SleepFM and many others...

Want to get in front of 1.5 Million AI enthusiasts? Work with us here

Featured Research..

Mistral AI Releases Codestral-22B: An Open-Weight Generative AI Model for Code Generation Tasks and Trained on 80+ Programming Languages, Including Python

The Mistral AI Team has announced the release of its groundbreaking code generation model, Codestral-22B. Codestral empowers developers by enhancing their coding capabilities and streamlining the development process. Codestral is an open-weight generative AI model explicitly crafted for code generation tasks. It supports over 80 programming languages, including popular ones like Python, Java, C, C++, JavaScript, and Bash, as well as more specialized languages like Swift and Fortran. This extensive language base ensures that Codestral can be an invaluable tool across diverse coding environments and projects. The model assists developers by completing coding functions, writing tests, and filling in partial code, significantly reducing the risk of errors and bugs.

Codestral is available for download under the Mistral AI Non-Production License for research and testing purposes and can be accessed via HuggingFace. The release also includes a dedicated endpoint, codestral.mistral.ai, optimized for IDE integrations and accessible through a personal API key. This endpoint is free during an 8-week beta period, managed via a waitlist to ensure quality service.

 Editor’s Picks…

SambaNova Systems Breaks Records with Samba-1-Turbo: Transforming AI Processing with Unmatched Speed and Innovation

In an era where the demand for rapid and efficient AI model processing is skyrocketing, SambaNova Systems has shattered records with the release of Samba-1-Turbo. This groundbreaking technology achieves a world record of processing 1000 tokens per second at 16-bit precision, powered by the SN40L chip and running the advanced Llama-3 Instruct (8B) model. The Centre of Samba-1-Turbo’s performance is the Reconfigurable Dataflow Unit (RDU), a revolutionary piece of technology that sets it apart from traditional GPU-based systems.

Their limited on-chip memory capacity often hampered GPUs, necessitating frequent data transfers between GPU and system memory. This back-and-forth data movement leads to significant underutilization of the GPU’s compute units, especially when dealing with large models that can only fit partially on-chip. SambaNova’s RDU, however, boasts a massive pool of distributed on-chip memory through its Pattern Memory Units (PMUs). Positioned close to the compute units, these PMUs minimize the need for data movement, thus vastly improving efficiency.

ADVERTISEMENT

MaxAI.me - Do more Faster with 1-Click AI

MaxAI.me lets you chat with GPT-4, Claude 3, Gemini 1.5. You can also perfect your writing anywhere, save 90% of your reading & watching time with AI summary, and reply 10x faster on email & social media.

Join 1M+ users on MaxAI.me to do more faster!

MAP-Neo: A Fully Open-Source and Transparent Bilingual LLM Suite that Achieves Superior Performance to Close the Gap with Closed-Source Models

Researchers from M-A-P, University of Waterloo, Wuhan AI Research, and 01.AI have released MAP-Neo, a highly capable and transparent bilingual language model with 7 billion parameters, trained on 4.5 trillion high-quality tokens. This model, fully open-sourced, matches the performance of leading closed-source LLMs. The release includes the cleaned pre-training corpus, data cleaning pipeline, checkpoints, and an optimized training and evaluation framework. The comprehensive documentation covers data curation, model architecture, training processes, evaluation codes, and insights into building LLMs, aiming to support and inspire the global research community, especially in non-English regions.

The advancement of open-source LLMs is crucial for AI research and applications. Recent efforts focus on enhancing both performance and transparency. MAP-Neo-7B stands out by integrating intermediate checkpoints, a comprehensive data cleaning process, accessible pre-training corpus, and reproduction code, unlike Mistral, LLaMA3, Pythia, Amber, and OLMo models. MAP-Neo-7B excels in benchmarks for Chinese and English understanding (C-EVAL, MMLU), mathematical ability (GSM8K), and coding (HumanEval). It achieves high scores across all tests and sets a new standard for transparency and performance, promoting trustworthiness and collaboration in the research community.

Researchers at Stanford Propose SleepFM: A New Multi-Modal Foundation Model for Sleep Analysis

Researchers from Stanford University and the Technical University of Denmark introduced SleepFM, a groundbreaking multi-modal foundation model for sleep analysis. This model leverages a vast dataset of multi-modal sleep recordings from over 14,000 participants, totaling more than 100,000 hours of sleep data collected between 1999 and 2020 at the Stanford Sleep Clinic. SleepFM utilizes a contrastive learning approach to integrate brain activity, ECG, and respiratory signals. This integration enables the model to capture comprehensive physiological representations, significantly enhancing the accuracy of sleep analysis.

SleepFM employs three 1D convolutional neural networks (CNNs) to generate embeddings from each modality (BAS, ECG, and respiratory signals). The architecture of these models is based on a 1D CNN developed for classifying ECG measurements. Each CNN is tailored to handle the specific characteristics of its respective modality: 10 channels for BAS, 2 for ECG, and 7 for respiratory channels. A novel leave-one-out contrastive learning technique is introduced, significantly outperforming the standard pairwise contrastive learning in capturing the synergy between different physiological signals.

Trending AI Social Media Posts

• Nature (the magazine) comments about Science (the body of knowledge) and science (the activity), triggered by my recent exchange with @elonmusk.

• ✨Launching today:✨ LlamaParse can now handle spreadsheets! Parse even your gnarliest Excel file into clean, LLM-friendly tables and get great results from your RAG pipeline! Handles all kinds of spreadsheets including Numbers!

• InternLM Research Group Releases InternLM2-Math-Plus: A Series of Math-Focused LLMs in Sizes 1.8B, 7B, 20B, and 8x22B with Enhanced Chain-of-Thought, Code Interpretation, and LEAN 4 Reasoning

• Excited to introduce Contextual Self-Supervised Learning (ContextSSL) for vision, an approach to learning a general representation that can adapt to be invariant or equivariant to task-specific symmetries given a few examples as the context.

Upcoming June Events

• Webinar: Using Open-Source CopilotKit for Personalized Banking Applications [June 3 2024]

• IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024

• IEEE World Congress on Computational Intelligence (IEEE WCCI 2024)

  and many others…