A German research consortium has released Soofi S, an open-source language model designed to excel in both English and German. Trained on Deutsche Telekom's AI cloud infrastructure, this model has been made available to the public under a permissive license.
Soofi S is a 30 billion parameter model that employs a resource-efficient hybrid architecture. This design activates only 3.2 billion of its parameters per token, allowing it to maintain consistent processing speeds even with lengthy inputs. The model has demonstrated superior performance in benchmarks when compared to other fully open models like Olmo 3 32B and Apertus 70B, specifically in the domains of German, English, and programming tasks.
The technical underpinnings of Soofi S include a mixture-of-experts (MoE) architecture, which is different from traditional dense models. This approach allows for an efficient parameter activation strategy, optimizing the model's performance across different tasks. The consortium overcame a data contamination issue by removing the GPQA benchmark from its evaluation, ensuring fair and accurate comparisons with other models. Subsequent evaluations confirmed the model's robust performance, and fine-tuned variants are currently in beta testing.
Soofi S is aimed at developers, researchers, and enterprises seeking advanced language processing capabilities. It can be utilized for a variety of applications, including machine translation, content generation, and programming assistance.
Work implications: Soofi S could enhance productivity for developers and researchers by offering improved language understanding and generation capabilities, potentially streamlining workflows in multilingual environments.
Originally reported by The Decoder