Replies: 4 comments 8 replies
-
|
One question: does the scope of this potentially include making |
Beta Was this translation helpful? Give feedback.
This comment was marked as spam.
This comment was marked as spam.
-
|
It is reassuring to hear that llama-completion will be a permanent option going forward. It was moderately traumatic to have the entirety of the way I interact with llama.cpp abruptly removed yesterday. I suspect you are underestimating how many of us utilise raw completions (or like to experiment with non-default chat templates) in our workflow. I also default to outputting to a file and this is no longer possible with the new chat CLI experience. I’m no expert (obviously) but from a user perspective I struggle to understand the logic of essentially duplicating the existing webui/server chat-based combo. I’ve always appreciated that there was, for want of a better word, a “pure” experience available in main as part of this project. It would be worth updating the documentation to notify users of the existence of llama-completion. |
Beta Was this translation helpful? Give feedback.
-
|
I use raw completions all the time but usually use a custom CLI tool with llama-server as the backend in order to have finer control over how it runs. This is especially helpful for co-writing. If I am not using the CLI tool, I am using the legacy webui interface. Sometimes I do still use llama-cli. The thing that has been annoying with completions is having to format using the chat template manually. If there was an easy way to assist with this, it would be helpful. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
We are planning to improve the UX of
llama-cli, more details can be found in this issue.Important
For people coming here to complain about this breaking your workflow:
llama-completionis there and we won't remove itThe plan is to migrate the code base of
llama-cliinto allama-server-based client. This will effectively allow CLI to inherit all of the features available on server, including:llama-clifails in certain cases)The current
llama-cliwill be moved to a new example calledllama-completion, and the code will be kept simple to serve as a learning example. If you are already usingllama-cliin a deterministic way in your pipeline, please consider usingllama-completionif you encouters any problems.The new
llama-cliwill have enhanced features (as mentioned above) and improved user experience.This discussion is added so that users can discuss issues and workarounds if needed.
Beta Was this translation helpful? Give feedback.
All reactions