llama.cpp support

[knetrix]

Hi. Can you add support for locally running models with llama.cpp? I use Macbook and LLamaBarn. I can access the models via Cherry Studio with API Host: http://localhost:2276/. But I would like this to be added to Kilo Code as I want to write code as well. Devstral 2 support was added to LLamaBarn. Thanks.

[anthonyalayo]

Just tried kilo code today, surprised support isn't there. LM Studio has optimization issues that llama.cpp fixes, so some prefer to not use LM Studio.

[kaotika]

@anthonyalayo llama.cpp is still integrated.

• settings
• providers
• add a Custom provider with url as, e.g. http://localhost:8080/v1 (or any other domain/ip, if it's not hosted locally)
• add some provider id, e.g. anthonys-laptop-llamacpp
• add all models or just specific one from the list below
• works like a charme :-)

[anthonyalayo]

Ah, custom provider, thanks!