How are people estimating LLM API costs before putting an AI application into production?

That is the million dollar question, is it not? Estimating tokens can be tricky because output length varies so much. A spreadsheet is your best friend here. I would map expected requests against average input and output token usage to get a realistic estimate. The outcome usually gives you a solid ballpark for monthly spend. Just remember to factor in conversation history, because that can grow surprisingly quickly. It is not perfect, but it is certainly better than guessing. I would also use the provider's pricing page as your baseline.