Microsoft
MicrosoftHi, I just rerun the example using localai and the phi-4 library, and got a different result. How can this be happening?
The only difference with the example is I did not give the model a set of specific answers. I want the model to achieve the same answer as shown.
🤔
Hi!
It's a bit of an "art" to get the models to behave the way you want for a particular use case.
To get a bit more predictable answer from a model you most probably need to:
1. always use the same model (check potential release dates / producers of the model)
2. use a very low temperature, perhaps investigate top_p and top_k
3. same systeminstruction all the time
4. examplify how the model should respond in the systemprompt / userprompt - https://www.promptingguide.ai/techniques/fewshot
It's also beneficial to "test" and fine-tune the settings for the AI-implementation... not meaning you are training the model. I mean have a good set of queries you test with and measure the result and then tweak the settings accordingly. Perhaps ask a more performant AI with web-search capabilities on how to do it? :)
