提示词模板格式是什么，如何避免生成结果混乱？

[linonetwo]

我目前使用这样的模板

template: '{{systemPrompt}}\n{{history}}model:{{completion}}\nuser:',

结果生成结果非常不好，请问 qwen1.5 gguf 需要使用什么样的提示词模板呢，文档里暂时没看到，是否有 Jinja 模板可以参考？例如

template: '<|im_start|>system\n{{systemPrompt}}<|im_end|>\n{{history}}<|im_start|>model:{{completion}}<|im_end|>\nuser:',

[linonetwo]

use https://github.com/chujiezheng/chat_templates/blob/main/chat_templates/openchat.jinja (which is said to compatable with qwen1.5 https://github.com/chujiezheng/chat_templates/blob/main/generation_configs/qwen2-chat.json ) ???

[jklj077]

The chat template is embedded in the official GGUF files. They are also provided in tokenizer.json as the standard practice in transformers. For example, https://huggingface.co/Qwen/Qwen1.5-72B-Chat/blob/main/tokenizer_config.json#L31.

[linonetwo]

I'm using llama.cpp https://github.com/withcatai/node-llama-cpp , how to use the embedding template?

chat template is embedded in the official GGUF files.

---

It will use embedding template if you not passing a custom template.

And that is what I did, and I get messy result. So it mighe be caused by tokenlizer? I'm not sure, and I am going to try that.

[jklj077]

If you're not using llama.cpp but node-llama-cpp, it is most likely a problem of node-llama-cpp. Please report to them.