It’s important to understand the difference between Prompt-Engineering, Fine-Tuning, and Retrieval Augmentation as they are all great methods to incorporate domain knowledge into your Generative AI application. 

Prompt-Engineering is all about in-context learning. This is particularly good when you have a static situation and you can iterate on it until you get the answer you want. https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/advanced-prompt-engineering 

Fine-Tuning is good to learn skills or new jargon, or ways to produce responses and are pernament to your LLM. To Fine Tune or Not Fine Tune? That is the question (youtube.com) 

Retrieval-Augmentation is good for learning new facts by grounding your prompt with the relevant information it needs on the fly to answer a question. RAG and generative AI - Azure AI Search | Microsoft Learn 

To your question, storing user queries in a vector database is a strategy that can significantly enhance the model's performance over time if they are relevant to the task the LLM is trying to solve. If the user queries are NOT relevant, and you’re leveraging them as history to maintain in your user prompt, you’re likely just providing the LLM noise.  

Hope this helps!