#inference

A llama.cpp-based app for running local models.

A great open source alternative that I used for running llms locally without having to use llama.cpp directly.