llama.cpp v1.0.0 : A stronger foundation for the local AI ecosystem
huggingface.co
A Blog post by ggml-org on Hugging Face

From their official post on Reddit:

Hey folks, it’s Merve from Hugging Face. Today, llama.cpp releases v1!

Moving forward, we’ll keep covering every platform and every model type, while keeping llama.cpp robust and lean for downstream applications that build on it.

While I’m here, want to announce that we’ve been shipping Llama desktop apps to lower the barrier of entry to local AI to few clicks, built-on llama.cpp. I will leave link to the details for both v1 and apps in comments.

Many thanks for all the love you’ve shown to llama.cpp for years 🦙❤️ Cheers!

  • BeefAndPoultry@lemmus.orgEnglish
    9·
    19 hours ago

    That blog post doesn’t seem to talk about what changed in this release since v0.9.0 https://github.com/ggml-org/llama.cpp/releases/tag/v1.0.0

    llama.cpp v1.0.0 removes the deprecated public API and promotes the NextN/speculative API to llama.h, adds OCP FP8 KV caches, a host-memory MoE expert cache with a configurable CPU share, and loading of external PLE shards. It adds three text architectures (Kimi-K3.1-Pro, Ripple and xing4_0), drama-3 TTS, and MageVL-Cowork and Kimi-K3.1 vision, hardens llama-server (SSRF, CORS, DNS rebinding, MCP tool cancellation) and adds a /v1/audio/speech endpoint, ships a redesigned web UI with chat groups and a markdown editor, and updates ggml to v1.0.0.

    Some highlights:

    • llama.h: add moe_cache_size and moe_cache_cpu_share to llama_context_params (#30966)
    • FP8 KV caches (-ctk/-ctv f8_e4m3) with optional static per-tensor scales (#28392)
    • MoE cache: CPU share for misses, LRFU/ClassLRU replacement, no CPU wait on cached experts (#30966, #31233, #31888)
    • external PLE/engram loading from separate GGUF shards via --shard/–ple (#31185)
    • chat : refactor gemma4 to build loose schema rules and clean up (#31915) This one includes a compatibility break: I also took this as an opportunity to clean up the old workaround and template. The official Gemma 4 templates have addressed the issues those solved and have been out for quite a while. If anyone hasn’t updated yet, it’s on them! You can download the newer template from here https://huggingface.co/google/gemma-4-31B-it/blob/main/chat_template.jinja

    I hope they start doing a separate bleeding edge branch and stable branch. Like how Linux has rolling distros and LTS distros.

  • Kissaki@programming.devEnglish
    8·
    20 hours ago

    Great news, great project. Projects staying below 1.0 for a long time are always tragic. No confidence in stability, and consequences of that for users.

    I don’t know if they were still iterating with such big changes or feature/model support gaps? Anyway, great to see this stabilization. Always feels better to use a stable >=1.0.

  • Kissaki@programming.devEnglish
    3·
    20 hours ago

    They embedded videos to blend-in-animation their graphs. But I have autoplay off. So all I see is grey, with no video controls until hover. Quite confusing. And so unnecessary for a blend in animation. Could have been images. Or - probably not on huggingfaces there - a simple CSS transition.

    I was confused and assumed content/embed was not loading. Even when scrolling up and down.