#llm/audio

Because vision, audio, and text inputs share the exact same weights, you no longer need to co-tune separate frozen encoders. Downstream adapter fine-tuning (such as LoRA) or full fine-tuning naturally updates the entire multimodal token loop in a single pass.

While many AI open source model providers are pursuing larger and more powerful models, Google is still giving attention to the smaller, more local side of the market. Today, the tech giant released Gemma 4 12B, an 11.95-billion-parameter open-weights model with permissive Apache 2.0 license optimized to execute locally on a standard enterprise laptop using just 16GB of VRAM or unified memory.

Traditional multimodal systems typically utilize discrete, separate encoders to translate audio waveforms and visual data into representations that the core language model can process.

This conventional approach inherently increases both inference latency and total memory consumption.

Gemma 4 12B radically alters this pipeline by functioning entirely without these secondary encoders. Instead, visual patches and raw audio waveforms are projected directly into the core large language model's embedding space through lightweight linear layers.

The vision encoder is replaced by a 35-million-parameter module utilizing a single matrix multiplication, while the audio encoder is eliminated entirely.

For enterprise engineering teams, this unified architecture delivers distinct operational advantages: lower latency for multimodal tasks, reduced VRAM requirements (down to 16GB — typical for laptops), and the ability to fine-tune the entire multimodal system in a single, cohesive pass.

Google has ensured that Gemma 4 12B is not an isolated experiment; it is ready for production. Weights are available on Hugging Face and Kaggle, and the model integrates seamlessly with industry-standard deployment frameworks such as vLLM, SGLang, MLX, and llama.cpp.

Powering the best enterprises, creators, and developers. From ElevenAgents for customer experience, ElevenCreative for content creation, to the leading AI voice generator.

One of Deepgram’s goals for the upcoming year is to pass the Audio Turing Test, which assesses how realistic and human-like AI-generated audio sounds.

Voxtral Realtime - A 4 billion parameter model aimed at live transcription, achieving “state of the art” transcription with 480ms latency across 13 languages. It can be configurable down to sub-200ms latency.

Performance on the FLEURS benchmark shows that Voxtral Mini Transcribe V2 performs competitively against models from Gemini and OpenAI, with the lowest diarization error rate.

“The body cam software and the AI report writing software picked up on the movie that was playing in the background, which happened to be ‘The Princess and the Frog,’” a Heber City sergeant told FOX 13 News. “That’s when we learned the importance of correcting these AI-generated reports.”

Chinese open source LLM company