This release adds the LiquidAI LFM3 family (text, vision, audio and omni) together with a new experimental streaming-audio API in mtmd, and wires realtime audio in and out of the server through the new /v1/realtime and /tts endpoints. A batch of new architectures landed: DeepSeek V4.1 Flash, SenseNova U1, Beam, Kolibri-1, Flint, Muse Spark, Granite Switch Decision, plus decision/embedding models from LiquidAI, Baidu and NVIDIA. The core got sparse-attention (QSA/DSA) reworked onto a fused masking op, a 50/50 SLRU cache for host-resident MoE experts, and cleaner no_alloc memory handling; the server got router memory-management and tool-calling improvements; the UI can now browse and download models straight from HF Hub, search conversations and compact them. Finally, ggml was bumped to v0.28.0, bringing GGML_OP_TOP_K_MASK, -1 expert-index semantics and large backend performance work (CUDA chunked graph launches, SYCL graph replay, OpenCL Q2_K/Q3_K, Hexagon HTP rework, KleidiAI SVE kernels).

I’m interested to try the performance from this change:

MoE experts kept in host memory now use a 50/50 SLRU cache instead of a plain LRU cache llama/30637

  • BeefAndPoultry@lemmus.orgOPEnglish
    3·
    12 days ago

    for home users they would be more palatable if they didn’t require any extra VRAM, if it was just a different “mode” for your normal LLM

    • [object Object]@lemmy.caEnglish
      3·
      12 days ago

      That’s one thing I like about unified memory, but yeah.

      I know some people turn qwens into decision models, but to get them calibrated you still have to fine tune the weights. And then it’s not the same weights anymore.