Building Custom Chat AI: A Comprehensive Guide for Developers
In today's rapidly evolving digital landscape, the integration of artificial intelligence (AI) into business operations has become a pivotal strategy for companies aiming to enhance their customer engagement and streamline their processes. This article delves into the foundational steps and considerations for developers embarking on the journey of building a custom chat AI for their company website. From understanding the core concepts of AI to selecting the right models and implementing effective prompt engineering techniques, this guide provides a comprehensive overview to help developers navigate the complexities of AI development. Whether you are a beginner or have some experience in the field, the insights shared here will equip you with the knowledge and tools needed to create a robust and efficient chat AI tailored to your business needs. A discussion will be held with https://www.linkedin.com/in/nityan/, Senior Cloud Advocate at Microsoft specializing in AI, and https://mvp.microsoft.com/en-US/MVP/profile/fe4dbe00-cdb0-ec11-983f-000d3a1017e3, a Chinese AI MVP, to delve into these critical topics. What are the first steps a developer should take when starting to build a custom chat AI for their company website? Nitya: If you are new to AI, start by familiarizing yourself with the core concepts and usage of AI models. A course like https://aka.ms/genai-beginnerscan be a great starting point. Next, get hands-on experience with models by trying out GitHub Models, which are free to use with just a GitHub account. This will help you build your intuition for model selection and prompt engineering. If you already have some experience, the initial steps to building a custom chat AI are as follows: Identify the use case and requirements (e.g., typical questions asked and valid responses). Choose a model to start prototyping (test the question with various models and compare results). If your chat AI is grounded in your data, identify the data sources and formats (where and what). Select an AI app template to jumpstart development and customize it with your model and data choices. How does understanding model choice impact the development of a custom chat AI? Nitya: Understanding model choice is crucial for developing a custom chat AI. It involves evaluating models based on three key factors: cost, customization, and performance. Customization: Start by identifying the task you want to execute (e.g., chat, image, embeddings, agents). Filter models that support this capability and validate them with a test prompt to ensure they fit your requirements. This process will narrow down your options from thousands to a few suitable models. Cost: Consider whether the model supports serverless deployments (pay-as-you-go, per token) or managed deployments (subscription-based, per VM). Evaluate costs not just for usage (chat completion) but also for end-to-end development (evaluations, iterative ideation). Performance: Assess models based on latency (e.g., chat completions vs. reasoning models) and the quality and safety of responses. Understand default model characteristics (model card) and perform custom evaluations to ensure quality for your desired prompts dataset. Can you explain the concept of prompt engineering and how it can be applied using GitHub models? Nitya: Prompt engineering involves guiding the model on how to process questions and generate responses to improve quality. Think of developers as teachers and models as students being taught to answer exam questions. Prompt engineering provides a rubric to guide models in giving relevant answers. This includes providing examples, creating personas (e.g., "answer politely using formal language"), defining output formats (e.g., "answer in 1-2 sentences", "reply with results in JSON format"), and configuring model parameters (e.g., temperature, stop-words, top-p, max tokens). When working with GitHub models, you can configure models using the https://github.com/marketplace/models or move to an IDE with the https://learn.microsoft.com/en-us/rest/api/aifoundry/modelinference/API, offering both low-code and code-first options for prompt engineering. What is retrieval augmented generation (RAG), and how does it enhance the ability to chat with data? Wei: https://learn.microsoft.com/en-us/azure/search/retrieval-augmented-generation-overview?tabs=docs involves grounding user questions in retrieved knowledge from private data sources to ensure responses are relevant to the application scenario. It works by wrapping the initial user prompt in a prompt template to create the final model prompt sent to the model. Thehttps://learn.microsoft.com/en-us/azure/search/retrieval-augmented-generation-overview?tabs=docs workflow includes retrieval of knowledge, augmentation of the prompt, and generation of the response. This dynamic process provides relevant grounding data and instructions to contextualize user questions for app-required responses. What are some practical tips for developers to streamline their end-to-end journey from catalog to cloud? Nitya: Here are three tips to get started: Model Selection: Use GitHub Models with diverse test prompts to build intuition for prompt engineering and model capabilities. Compare models side-by-side. Copilot Development: Start with an https://aka.ms/ai-apps. Deploy it to understand the application and its architecture before customizing it to your needs. Validate your development environment and get familiar with tools. Safety & Evaluation: Explore built-in content safety filters and evaluators in the Azure AI platform to understand metrics and effectiveness of your prompt engineering or RAG strategy. Use tracing and App Insights to monitor performance and cost. What are some common challenges developers might face when building a custom chat AI, and how can they overcome them? Nitya and Wei: There are many challenges we can think of - here are three that are important: App Architecture: Understand the app architecture for your scenario (e.g., RAG, multi-agent). Explore existing AI app templates to build intuition and customize one that fits your requirements. Model Choice: Choose models based on cost, quota availability, and flexibility for future configuration. Use the Azure AI model inference API to abstract provider-specific SDKs and decouple your code from your choice, allowing for easier model swaps later. Observability: Debug issues in app development or execution performance. Use platforms and tools that bring observability to the end-to-end workflow. Activate App Insights and use tracing tools to generate telemetry for insights locally or in production. What resources and samples are available for further exploration into this subject? Wei: Explore https://aka.ms/ai-apps, https://aka.ms/genai-beginners, https://aka.ms/rag/azure-ai-foundry, https://aka.ms/aitour/reposand https://microsoft.github.io/generative-ai-for-beginners For more workshops and talks, visit https://aka.ms/aitour/repos. Feel free to check out opensource projects like https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fautogen&data=05%7C02%7CRochelle.Sonnenberg%40microsoft.com%7C564dabbda6454d889d8308dd867510be%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638814559548370151%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=ihWSr5jedQIQKe6F%2Bk0Cwm5edG6Xc62jvhSQ9I0w6S0%3D&reserved=0 https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.llamaindex.ai%2Fen%2Fstable%2F&data=05%7C02%7CRochelle.Sonnenberg%40microsoft.com%7C564dabbda6454d889d8308dd867510be%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638814559548398018%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=04lUnuTY9g13jf3HvtYgjrRcDycipl2Qq%2BQ55kXVBZI%3D&reserved=0, https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpython.langchain.com%2Fdocs%2Fintroduction%2F&data=05%7C02%7CRochelle.Sonnenberg%40microsoft.com%7C564dabbda6454d889d8308dd867510be%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638814559548436587%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=4Wt%2FQavECvoZ%2BVfgn0qhflrCHKWVRmkzpxxfNL4jQQU%3D&reserved=0, and https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcamel-ai%2Fcamel&data=05%7C02%7CRochelle.Sonnenberg%40microsoft.com%7C564dabbda6454d889d8308dd867510be%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638814559548424270%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=8MwQd1vUnaADU3Nkl2mApD1%2FpgSzfWp7McLoyC5WooQ%3D&reserved=0 documentation. As we conclude this exploration into building a custom chat AI for your company website, it's clear that the journey is both challenging and rewarding. By understanding the core concepts of AI, selecting the right models, and mastering prompt engineering, developers can create a powerful tool that enhances customer engagement and streamlines business operations. The insights and practical tips shared in this article provide a solid foundation for embarking on this journey. Remember, the key to success lies in continuous learning and adaptation. As AI technology evolves, you should also adapt your approach to developing and refining your chat AI. Stay curious, stay innovative, and most importantly, stay committed to delivering the best possible experience for your users.
Enterprise (LSP) Partner Community Q&A Call - ASIA - Chinese
We are pleased to invite you to our Microsoft Partner Operations: Community Q&A call for Enterprise (LSP) Partners. These sessions are dedicated to assist Enterprise Partners with questions related to launches and upcoming changes. For your convenience, community collections are made available ahead of time in the partner readiness gallery where you can review launch material that will be discussed on the Q&A call. Our goal is to help drive a smoother business relationship with Microsoft. We look forward to having you in one of our upcoming sessions. Register today! Thank you, Microsoft Partner OperationsSS33: smart pip mode - better tiktok integration (or other video social/websites) picture-in-picture
Suggestion (SS): 33 Classification: TIKTOK + PIP / picture-in-picture + audio control PRIORITY IN MY OPINION: 4-6 on a scale from 1 (low) to 10 (high) EDIT: i just found this picture in picture https://link.ws/pipdiscussion1 discussion in the feed too. 1) i don't see this directly is annoying i need to activate it via flags 2) pip doesn't work, even if i activate it goal like 360 chrome (or even better, see 3) or https://link.ws/vivopepip with better video control (sugg.17.) and pip buttons (sugg.18.), but avoid to put it over the downloader... (like the picture) 3) smart player with zoomed features like this on the right 4) remember tiktok specific size 5) keep always on top over all mac apps (i haven't tested this with edge, because i don't see pip) 6) please go to next tiktok automatically 7) allow extension in pip mode, so i can download tiktok videos in pip mode with the overlay button PS: about 6) i already told to them inside my 50+ suggestions to tiktok too, but maybe edge is able to fix it too. i hope auto for both pip mode and normal web browsing. 😎add possibility to add multiple url inside a list of pip mode, so a sort of playlist pip mode. 9) maybe like 8 with collections or bookmarks folders, so you pip mode randomly url 10) add to bookmarks inside pip mode directly, especially if you implement 6) 11) same like 10 for collections 12) maybe a separate history for pipmode visited urls (if 6. is implemented) EDIT: sorry i can activate it via 1 ... there is no button over the video for pip (or below, near the video). if i do that, i see there is no mute option too. 13) we cannot change the size, we can only change position. yes pip is over all mac apps (i see it now after enabling with 1. ) 14) remember to add even the option "keep behind mac app too". so both option should be available. 15) loop mode support for other website here the max size possible 16) better audio control panel, where we see all tabs with audio active and we can choose where to keep audio on or off. for example if 300 tabs are opened, but 20 with audio, in this panel we see only the 20 tabs. 17,18) see above 19) ability to quick copy url link or share the video (in pip mode) 20) screenshot feature integrated in pip mode. 21) loop from x to y https://www.tiktok.com/@buketalican11.12.16/video/6682465643712744709 example from 0 to 0.22 in this video, by skipping the rest 22) continue to loop, even if i launch another website, in this case you have 2 audios, 1 with pip, 1 without. would be cool the loop function for normal browsing too (is possible via extension...) 23) remember the dual mode suggestion: pip and full mode at the same time https://link.ws/edgess25 about 24) it depends how you implement all that. if you do like 360chrome, where each video has separate pip then allow both option "deactivate audio in dock mode" or "keep audio in dock mode" (not available in 360). what i like is that i can pip my videos in the dock. so if you use a player, maybe a way to pip the player in the dock is cool too, with both option "audio on" "audio off". 25) if there is no idea for 23) then a way to switch from pip to full faster, and from full to pip too about 26) i know this get against tiktok policy, but maybe useful for other websites**. a download button. **at the end is not possible to download it at all, since it doesn't work. plus the fact that tiktok download is not directly a feature a browser need to support and extension can do it perfectly. so the point is to add it for other websites, that allow you to download videos without copyright problem... 27) spotify player should be different from video player. so we can have spotify and pip at the same time. 28) 2x click switch size for example with opera max size is then with a 2x click we go to so a small size, which is not related to the full size video - pip mode switch suggestion that i wrote before. in this case the max size could be 1/2 of display, full mode is 1/1 29) right click options would be cool to see suggestions or skip ads or stop auto suggestions with right clickEdge Dev PDF reader with wrong title encoding
I'm a Chinese user and this is what a get from the newest release of Edge Dev. If you open a pdf file with a name containing Chinese, it shows title in a wrong encoding while the link itself and its pages are correct. Other titles are correct, like "百度一下，你就知道".
