Tuesday Prompt Day | 6W + E Practical Experiment #3 — From AI Output to Business Decision
In our previous discussion, we explored an important idea: Prompt → Output → Evaluate → Refine → Better Output Today, let's take the next step. What happens when the goal is not simply to get a better answer from Copilot, but to get an answer that helps someone make a better decision? Let's look at a practical enterprise scenario. BASIC PROMPT "Review this project update and tell me if we are on track." It looks simple. But what does "on track" actually mean? On track against what? Who needs the answer? What decision are they trying to make? What evidence should Copilot consider? This is where 6W + E becomes useful. 6W + E PROMPT "Act as an enterprise program advisor. Review the project status information provided below and prepare an assessment for the project steering committee. WHY: The purpose is to determine whether the project is on track and whether leadership intervention is required. WHAT: Assess progress, major risks, dependencies, issues and upcoming milestones. WHO: The audience is senior business and IT leadership. WITH: Use only the information provided in the project status material. Do not invent missing facts. WAY: Present the response using these sections: Overall status Evidence supporting the status Key risks and their business impact Critical dependencies Decisions or actions required from leadership WIN: The output should allow a steering committee member to understand the situation quickly and identify where action is required. EVALUATE: Before finalizing the response, check whether each conclusion is supported by the source material. Clearly distinguish facts, observations and assumptions." Notice what changed. The prompt is not simply longer. The problem has become clearer. IMPROVED OUTPUT Instead of simply saying: "The project appears to be on track, although there are some risks." Copilot can be guided toward something more useful: Overall status: Amber - progress is continuing, but a dependency may affect the next milestone. Evidence: Current delivery remains aligned with the planned milestone. A key dependency is still unresolved. The available information does not confirm whether the dependency will be resolved before the milestone. Business impact: If the dependency remains unresolved, the next milestone may be delayed. Leadership action: Confirm ownership and resolution date for the dependency. Information gap: The source material does not provide a confirmed resolution date. That is a very different outcome. The AI is no longer just summarizing information. It is helping structure the information around a business decision. NOW EVALUATE Before accepting this output, ask: Are the conclusions supported by evidence? Did Copilot confuse an assumption with a fact? Is the business impact clear? Is the recommended action actually supported by the information? Can a decision-maker understand the situation quickly? What information is still missing? This is where EVALUATE becomes more than a final proofreading step. It becomes a quality-control mechanism. REFINE Suppose our evaluation identifies one problem: The response identifies the dependency, but the leadership action is still too generic. We can refine the instruction: "Refine the leadership action. Do not simply recommend monitoring the dependency. Identify the specific decision, owner or escalation required based only on the available information. If the source material does not provide enough information to identify an owner or decision, explicitly state what information is missing." Now we have another cycle: Prompt → Output → Evaluate → Refine → Better Output And this leads to a broader question. Are we really trying to teach people how to write better prompts? Or are we trying to teach people how to work effectively with AI? I believe there is an important difference. Prompt engineering may start with the prompt. But effective AI collaboration continues through evaluation, judgment and refinement. YOUR TURN Think about a Copilot interaction you use in your day-to-day work. Ask yourself: What decision is the output supposed to support? What evidence should Copilot use? What would make the answer genuinely useful? How would you evaluate the first response? What would you refine if the answer was only almost right? Share your experience without including confidential information. I'm especially interested in examples where Copilot produced a technically correct answer but the answer was not useful for the actual business decision. Those examples can teach us more than perfect prompts. This discussion continues the 6W + E practical experiment series. Please see the Resources section for the previous experiments and the original 6W + E framework. The goal of this series is not simply to create better prompts. It is to explore whether 6W + E can become a repeatable method for working with AI in real-world scenarios. What would you evaluate first in your next Copilot response?Tuesday Prompt Day 🚀 | 6W + E Practical Experiment #2
In our previous practical experiment, we took a simple Copilot request and transformed it using the Six W + E framework. Today, let's focus on the part that can make the biggest difference: E = EVALUATE A common assumption is: Prompt → Copilot → Answer But in real-world enterprise work, I believe the process should be: Prompt → Output → Evaluate → Refine → Better Output Let's continue with the same scenario. 🔹 BASIC PROMPT "Create a summary of our cloud migration project." The response may be reasonable. But before accepting it, let's evaluate it. 🔹 EVALUATE Ask yourself: Did Copilot understand the intended audience? Did it focus on the business objective? Did it distinguish facts from assumptions? Did it surface the risks that actually matter? Can the intended audience act on the result? Suppose the answer is: "Mostly good, but the risks are too generic and the executive summary contains too much technical detail." That feedback is valuable. We now know what needs to change. 🔹 REFINE Instead of starting over, we refine the instruction: "Refine the previous response for senior business and IT leadership. Reduce technical implementation details. Prioritize the most significant business risks. For each risk, provide: Risk • Business impact • Current mitigation • Decision or action required Keep the executive summary concise. Do not introduce information that is not supported by the source material. Clearly identify any information that is unavailable." Now the interaction has changed. We are no longer simply asking Copilot for an answer. We are using the first answer to improve the next instruction. 🔹 IMPROVED OUTPUT The objective is not necessarily to make the prompt longer. The objective is to make the next interaction more precise. That distinction matters. A good prompt can produce a useful first response. But a good evaluation process helps us systematically improve the result. This is why I see EVALUATE as an important part of Six W + E. It creates a feedback loop: Think → Prompt → Output → Evaluate → Refine And this raises an interesting question for enterprise AI adoption: Should we teach people only how to write better prompts? Or should we teach them how to evaluate AI output and refine their interaction with AI? I believe the second capability is just as important. 💡 YOUR TURN Take one prompt you use with Copilot. Run it once. Then evaluate the response before rewriting the prompt. Share: What you originally asked What was missing or incorrect in the response What you changed in your prompt Whether the second result was actually better Please avoid sharing confidential or sensitive information. I'm particularly interested in examples where the first Copilot response looked correct but wasn't actually useful for the business problem. Those are often the most interesting examples. 🔗 This discussion continues our Six W + E journey. Start with the original framework discussion and then explore the practical experiment series from there. I'll use the strongest examples from this series to explore how Six W + E can evolve from a prompting framework into a practical method for working with AI.From AI PoC to Production: 7 Architecture Decisions Every Enterprise Must Get Right
Architecture Deep Dive Moving from an AI Proof of Concept to an enterprise-ready production workload requires much more than selecting the right model. AI is no longer just an experimentation topic. Across enterprises, teams are building copilots, RAG applications, AI agents, intelligent automation and domain-specific AI solutions. But there is a significant difference between making an AI Proof of Concept work and making an AI solution production-ready. A PoC asks: “Can we make AI do this?” Production asks much harder questions: “Can we make it secure, reliable, scalable, observable, governed and financially sustainable?” That is where architecture becomes critical. Microsoft’s Azure Well-Architected guidance for AI workloads highlights that AI systems introduce architectural considerations beyond traditional applications, including nondeterministic behavior, grounding data, model operations, testing, responsible AI and continuous evaluation. Here are seven architecture decisions I believe every enterprise should consider before moving an AI workload from PoC to production. 1. Start With the Business Outcome — Not the Model One of the most common mistakes is starting with: “Which AI model should we use?” The better question is: “What business problem are we solving?” Before selecting a model or Azure service, define: • The business outcome • The users • The expected experience • The measurable success criteria • Regulatory and compliance requirements • Data sensitivity • Expected scale For example: Instead of saying: “We want to build an enterprise chatbot.” Define the outcome: “We want employees to find accurate information from 500,000 internal documents in less than 5 seconds while respecting existing access permissions.” That single statement changes the architecture conversation completely. 2. Design the Data and Grounding Architecture First Enterprise AI is only as useful as the information it can access and trust. For many enterprise scenarios, the challenge isn't simply selecting a powerful model. The challenge is providing the model with the right context. This is where grounding and RAG architectures become important. A typical flow looks like: User → Application → Orchestration → Knowledge/Retrieval → Model → Response But an enterprise implementation also needs to consider: • Data ingestion • Chunking and enrichment • Metadata • Indexing • Access control • Data freshness • Source attribution • Retrieval quality • Auditability Microsoft's current AI architecture guidance explicitly treats the knowledge layer as a core architectural component and emphasizes enforcing data access policies and authorization within that layer. The key architectural question is therefore not: “Can the model answer the question?” It is: “Can the model answer the question using authorized, relevant and trustworthy enterprise data?” 3. Separate Intelligence, Inference, Knowledge and Tools AI applications are becoming more sophisticated. Modern architectures may involve models, agents, orchestration, enterprise data and external tools. Putting everything into one application layer quickly becomes difficult to secure, scale and operate. A better approach is to establish clear architectural boundaries. A useful conceptual model is: Client Layer ↓ Intelligence / Orchestration Layer ↓ Inference Layer ↓ Knowledge Layer ↓ Tools / Business APIs Each layer can have its own: • Identity • Security policies • Scaling strategy • Monitoring • Caching • Failure handling This separation becomes particularly important when moving from a simple chatbot to agentic AI applications. Microsoft's AI application design guidance recommends distinct client, intelligence, inference, knowledge and tools layers for intelligent applications. 4. Treat Security as an Architecture Principle — Not a Checklist AI introduces new security considerations. You need to think beyond traditional application security. Ask: • Who can access the AI application? • What data can the user retrieve? • Can the model access information the user cannot? • How are identities propagated across components? • How are prompts and responses protected? • How are AI tools authorized? • How are sensitive outputs detected? • How are activities audited? One particularly important principle is: The AI system should not become an alternative path around existing enterprise authorization. If an employee cannot access a document directly, the AI assistant should not expose that document through a generated response. Security therefore needs to exist across the entire AI architecture: Identity → Data → Retrieval → Model → Tools → Output 5. Design for Scale and Reliability Before You Need It A PoC might have: 10 users 100 documents 1 model 1 environment Production might have: 100,000 users Millions of documents Multiple models Multiple business applications Continuous availability requirements The architecture must therefore consider: • Horizontal scaling • Availability Zones • Regional resiliency • Load balancing • Model availability • Rate limiting • Failover • Caching • Capacity planning AI workloads also have unique infrastructure considerations. Inference capacity can become a bottleneck, and GPU-based workloads can introduce significant infrastructure costs. Microsoft's current Azure AI architecture guidance recommends designing for scalability and availability across the intelligence, orchestration, inference and knowledge layers. 6. Cost Must Be Designed Into the Architecture AI can create unexpected cost growth. A solution may work perfectly from a technical perspective and still fail the business case because of: • Token consumption • Model selection • GPU utilization • Storage • Data processing • Retrieval infrastructure • Logging • Network traffic • High-frequency inference Therefore, ask: “What is the expected cost per transaction?” Then model: Users × Requests × Tokens × Model Cost But don't stop there. Also evaluate: • Caching opportunities • Model routing • Smaller models for simpler tasks • Batch processing • GPU utilization • Resource scaling • Storage optimization Microsoft's Well-Architected guidance specifically highlights monitoring utilization and avoiding unnecessary AI infrastructure costs. The cheapest architecture isn't necessarily the best architecture. The goal is: Maximum business value per unit of AI spend. 7. Production Requires Continuous Evaluation and Observability Traditional applications usually monitor: CPU Memory Latency Errors Availability AI applications need more. You also need to understand: • Response quality • Grounding accuracy • Retrieval relevance • Hallucination rate • Model performance • Prompt effectiveness • Safety violations • User feedback • Token consumption • Cost per interaction AI is nondeterministic. The same input may not always produce exactly the same output. That means testing cannot simply end when the application goes live. Production evaluation becomes part of the architecture. Microsoft's guidance recommends extending observability to AI-specific quality metrics and supporting testing and evaluation with real production inputs. The Architecture Mindset Shift The biggest transition from PoC to production is not necessarily choosing a better model. It is changing the questions we ask. PoC thinking: “Can AI do it?” Production thinking: “Can the enterprise operate it safely and economically at scale?” That leads to a different architecture conversation: Business Outcome ↓ Data & Grounding ↓ Security & Identity ↓ AI Application Architecture ↓ Infrastructure & Scalability ↓ Observability & Governance ↓ Cost Optimization ↓ Continuous Evaluation My 7-Question Production Readiness Test Before approving an enterprise AI workload for production, I would ask: 1. What measurable business outcome are we delivering? 2. Can we trust and govern the data being used? 3. Can the AI respect existing identity and authorization boundaries? 4. Can every major component scale and recover from failure? 5. Can we measure AI quality—not just infrastructure health? 6. Do we understand the cost at production scale? 7. Can we continuously evaluate, improve and govern the solution? If the answer to several of these is “not yet”, the solution may still be a PoC. And that's perfectly fine. The objective isn't to rush an AI PoC into production. The objective is to build the architecture that makes production possible. Final Thought AI architecture is becoming less about: “Which model should we use?” And increasingly about: “How do we build an AI system that the enterprise can trust?” That is the real journey: PoC → Architecture → Production → Scale → Business Value The organizations that get this architecture right will be in a much stronger position to move from AI experimentation to sustainable enterprise AI adoption. What do you think is the biggest challenge when moving an enterprise AI solution from PoC to production — security, data, scalability, cost, or something else?🧠 What is Retrieval-Augmented Generation (RAG)?
Have you ever wondered how AI tools answer questions using your company's documents instead of making things up? That's where Retrieval-Augmented Generation (RAG) comes in. Instead of relying only on what the AI learned during training, RAG first searches trusted sources—such as PDFs, SharePoint libraries, knowledge bases, or internal documentation—and then uses that information to generate a response. Why organizations use RAG ✅ Reduces hallucinations ✅ Uses the latest company knowledge ✅ Keeps responses grounded in trusted data ✅ Improves enterprise AI accuracy Common Microsoft stack Azure AI Search Azure OpenAI Microsoft Copilot SharePoint Microsoft Fabric RAG is one of the key building blocks behind modern enterprise AI assistants. 💬 Discussion: Have you implemented a RAG solution in your organization, or are you planning one?