Running Phi-4 Locally with Microsoft Foundry Local: A Step-by-Step Guide
In our previous post, we explored how Phi-4 represents a new frontier in AI efficiency that delivers performance comparable to models 5x its size while being small enough to run on your laptop. Today, we're taking the next step: getting Phi-4 up and running locally on your machine using Microsoft Foundry Local. Whether you're a developer building AI-powered applications, an educator exploring AI capabilities, or simply curious about running state-of-the-art models without relying on cloud APIs, this guide will walk you through the entire process. Microsoft Foundry Local brings the power of Azure AI Foundry to your local device without requiring an Azure subscription, making local AI development more accessible than ever. So why do you want to run Phi-4 Locally? Before we dive into the setup, let's quickly recap why running models locally matters: Privacy and Control: Your data never leaves your machine. This is crucial for sensitive applications in healthcare, finance, or education where data privacy is paramount. Cost Efficiency: No API costs, no rate limits. Once you have the model downloaded, inference is completely free. Speed and Reliability: No network latency or dependency on external services. Your AI applications work even when you're offline. Learning and Experimentation: Full control over model parameters, prompts, and fine-tuning opportunities without restrictions. With Phi-4's compact size, these benefits are now accessible to anyone with a modern laptop—no expensive GPU required. What You'll Need Before we begin, make sure you have: Operating System: Windows 10/11, macOS (Intel or Apple Silicon), or Linux RAM: Minimum 16GB (32GB recommended for optimal performance) Storage: At least 5 - 10GB of free disk space Processor: Any modern CPU (GPU optional but provides faster inference) Note: Phi-4 works remarkably well even on consumer hardware 😀. Step 1: Installing Microsoft Foundry Local Microsoft Foundry Local is designed to make running AI models locally as simple as possible. It handles model downloads, manages memory efficiently, provides OpenAI-compatible APIs, and automatically optimizes for your hardware. For Windows Users: Open PowerShell or Command Prompt and run: winget install Microsoft.FoundryLocal For macOS Users (Apple Silicon): Open Terminal and run: brew install microsoft/foundrylocal/foundrylocal Verify Installation: Open your terminal and type. This should return the Microsoft Foundry Local version, confirming installation: foundry --version Step 2: Downloading Phi-4-Mini For this tutorial, we'll use Phi-4-mini, the lightweight 3.8 billion parameter version that's perfect for learning and experimentation. Open your terminal and run: foundry model run phi-4-mini You should see your download begin and something similar to the image below Available Phi Models on Foundry Local While we're using phi-4-mini for this guide, Foundry Local offers several Phi model variants and other open-source models optimized for different hardware and use cases: Model Hardware Type Size Best For phi-4-mini GPU chat-completion 3.72 GB Learning, fast responses, resource-constrained environments with GPU phi-4-mini CPU chat-completion 4.80 GB Learning, fast responses, CPU-only systems phi-4-mini-reasoning GPU chat-completion 3.15 GB Reasoning tasks with GPU acceleration phi-4-mini-reasoning CPU chat-completion 4.52 GB Mathematical proofs, logic puzzles with lower resource requirements phi-4 GPU chat-completion 8.37 GB Maximum reasoning performance, complex tasks with GPU phi-4 CPU chat-completion 10.16 GB Maximum reasoning performance, CPU-only systems phi-3.5-mini GPU chat-completion 2.16 GB Most lightweight option with GPU support phi-3.5-mini CPU chat-completion 2.53 GB Most lightweight option, CPU-optimized phi-3-mini-128k GPU chat-completion 2.13 GB Extended context (128k tokens), GPU-optimized phi-3-mini-128k CPU chat-completion 2.54 GB Extended context (128k tokens), CPU-optimized phi-3-mini-4k GPU chat-completion 2.13 GB Standard context (4k tokens), GPU-optimized phi-3-mini-4k CPU chat-completion 2.53 GB Standard context (4k tokens), CPU-optimized Note: Foundry Local automatically selects the best variant for your hardware. If you have an NVIDIA GPU, it will use the GPU-optimized version. Otherwise, it will use the CPU-optimized version. run the command below to see full list of models foundry model list Step 3: Test It Out Once the download completes, an interactive session will begin. Let's test Phi-4-mini's capabilities with a few different prompts: Example 1: Explanation Phi-4-mini provides a thorough, well-structured explanation! It starts with the basic definition, explains the process in biological systems, gives real-world examples (plant cells, human blood cells). The response is detailed yet accessible. Example 2: Mathematical Problem Solving Excellent step-by-step solution! Phi-4-mini breaks down the problem methodically: 1. Distributes on the left side 2. Isolates the variable terms 3. Simplifies progressively 4. Arrives at the final answer: x = 11 The model shows its work clearly, making it easy to follow the logic and ideal for educational purposes Example 3: Code Generation The model provides a concise Python function using string slicing ([::-1]) - the most Pythonic approach to reversing a string. It includes clear documentation with a docstring explaining the function's purpose, provides example usage demonstrating the output, and even explains how the slicing notation works under the hood. The response shows that the model understands not just how to write the code, but why this approach is preferred - noting that the [::-1] slice notation means "start at the end of the string and end at position 0, move with the step -1, negative one, which means one step backwards." This showcases the model's ability to generate production-ready code with proper documentation while being educational about Python idioms. To exit the interactive session, type `/bye` Step 4: Extending Phi-4 with Real-Time Tools Understanding Phi-4's Knowledge Cutoff Like all language models, Phi-4 has a knowledge cutoff date from its training data (typically several months old). This means it won't know about very recent events, current prices, or breaking news. For example, if you ask "Who won the 2024 NBA championship?" it might not have the answer. The good thing is, there's a powerful work-around. While Phi-4 is incredibly capable, connecting it to external tools like web search, databases, or APIs transforms it from a static knowledge base into a dynamic reasoning engine. This is where Microsoft Foundry's REST API comes in. Microsoft Foundry provides a simple API that lets you integrate Phi-4 into Python applications and connect it to real-time data sources. Here's a practical example: building a web-enhanced AI assistant. Web-Enhanced AI Assistant This simple application combines Phi-4's reasoning with real-time web search, allowing it to answer current questions accurately. Prerequisites: pip install foundry-local-sdk requests ddgs Create phi4_web_assistant.py: import requests from foundry_local import FoundryLocalManager from ddgs import DDGS import json def search_web(query): """Search the web and return top results""" try: results = list(DDGS().text(query, max_results=3)) if not results: return "No search results found." search_summary = "\n\n".join([ f"[Source {i+1}] {r['title']}\n{r['body'][:500]}" for i, r in enumerate(results) ]) return search_summary except Exception as e: return f"Search failed: {e}" def ask_phi4(endpoint, model_id, prompt): """Send a prompt to Phi-4 and stream response""" response = requests.post( f"{endpoint}/chat/completions", json={ "model": model_id, "messages": [{"role": "user", "content": prompt}], "stream": True }, stream=True, timeout=180 ) full_response = "" for line in response.iter_lines(): if line: line_text = line.decode('utf-8') if line_text.startswith('data: '): line_text = line_text[6:] # Remove 'data: ' prefix if line_text.strip() == '[DONE]': break try: data = json.loads(line_text) if 'choices' in data and len(data['choices']) > 0: delta = data['choices'][0].get('delta', {}) if 'content' in delta: chunk = delta['content'] print(chunk, end="", flush=True) full_response += chunk except json.JSONDecodeError: continue print() return full_response def web_enhanced_query(question): """Combine web search with Phi-4 reasoning""" # By using an alias, the most suitable model will be downloaded # to your device automatically alias = "phi-4-mini" # Create a FoundryLocalManager instance. This will start the Foundry # Local service if it is not already running and load the specified model. manager = FoundryLocalManager(alias) model_info = manager.get_model_info(alias) print("🔍 Searching the web...\n") search_results = search_web(question) prompt = f"""Here are recent search results: {search_results} Question: {question} Using only the information above, give a clear answer with specific details.""" print("🤖 Phi-4 Answer:\n") return ask_phi4(manager.endpoint, model_info.id, prompt) if __name__ == "__main__": # Try different questions question = "Who won the 2024 NBA championship?" # question = "What is the latest iPhone model released in 2024?" # question = "What is the current price of Bitcoin?" print(f"Question: {question}\n") print("=" * 60 + "\n") web_enhanced_query(question) print("\n" + "=" * 60) Run It: python phi4_web_assistant.py What Makes This Powerful By connecting Phi-4 to external tools, you create an intelligent system that: Accesses Real-Time Information: Get news, weather, sports scores, and breaking developments Verifies Facts: Cross-reference information with multiple sources Extends Capabilities: Connect to databases, APIs, file systems, or any other tool Enables Complex Applications: Build research assistants, customer support bots, educational tutors, and personal assistants This same pattern can be applied to connect Phi-4 to: Databases: Query your company's internal data APIs: Weather services, stock prices, translation services File Systems: Analyze documents and spreadsheets IoT Devices: Control smart home systems The possibilities are endless when you combine local AI reasoning with real-world data access. Troubleshooting Common Issues Service not running: Make sure Foundry Local is properly installed and the service is running. Try restarting with foundry --version to verify installation. Model downloads slowly: Check your internet connection and ensure you have enough disk space (5-10GB per model). Out of memory: Close other applications or try using a smaller model variant like phi-3.5-mini instead of the full phi-4. Connection issues: Verify that no other services are using the same ports. Foundry Local typically runs on http://localhost:5272. Model not found: Run foundry model list to see available models, then use foundry model run <model-name> to download and run a specific model. Your Next Steps with Foundry Local Congratulations! You now have Phi-4 running locally through Microsoft Foundry Local and understand how to extend it with external tools like web search. This combination of local AI reasoning with real-time data access opens up countless possibilities for building intelligent applications. Coming in Future Posts In the coming weeks, we'll explore advanced topics using Hugging Face: Fine-tuning Phi models on your own data for domain-specific applications Phi-4-multimodal: Analyze images, process audio, and combine multiple data types Advanced deployment patterns: RAG systems and multi-agent orchestration Resources to Explore EdgeAI for Beginners Course: Comprehensive 36-45 hour course covering Edge AI fundamentals, optimization, and production deployment Phi-4 Technical Report: Deep dive into architecture and benchmarks Phi Cookbook on GitHub: Practical examples and recipes Foundry Local Documentation: Complete technical documentation and API reference Module 08: Foundry Local Toolkit: 10 comprehensive samples including RAG applications and multi-agent systems Keep experimenting with Foundry Local, and stay tuned as we unlock the full potential of Edge AI! What will you build with Phi-4? Share your ideas and projects in the comments below!Two Public Sector Roundtables at PPCC25
Heading to the Power Platform Community Conference in Las Vegas? Don’t miss two sessions designed for the public sector. The Microsoft Sovereign Cloud Architect Panel gives U.S. GCC, GCC High, and DoD customers direct access to Microsoft experts for roadmap and security insights. Meanwhile, Powering Public Impact brings together global government and education leaders to share strategies for governance, AI adoption, and scaling low‑code innovation. Join these conversations to learn, connect, and shape the future of digital transformation in government.The Role of Storytelling in Community Building
On building community. "Stories are the single most powerful weapon in a leader’s arsenal." – Dr. Howard Gardner, Harvard University Storytelling has played a crucial role in human connection since the inception of civilization. It shapes identities, preserves historical records, and fosters a sense of belonging within communities. Whether conveyed through spoken word, literature, or digital media, storytelling possesses the ability to unify individuals, create shared experiences, and enhance communal bonds. The Power of Storytelling in Community Building Fostering Shared Identity Stories help communities define who they are and what they stand for. They preserve cultural traditions and values, passing them on to future generations. Encouraging Empathy and Connection Personal narratives allow people to understand each other’s experiences and struggles. Stories create emotional connections that transcend differences, building unity. Inspiring Action and Change A compelling story can mobilize people around a cause or mission. Communities often use storytelling to advocate for social change and inspire collective action. Strengthening Engagement and Participation Shared narratives encourage individuals to take active roles within their communities. Engaging stories help retain interest and commitment to communal efforts. Preserving History and Legacy Oral and written storytelling ensures that a community’s history remains alive. Stories highlight past challenges and triumphs, shaping future aspirations. I love a bookstore and find that they are great places to gather. I have many in cities around the world that I love to dip into over and over and that have wonderful schedules of events. Libraries as well, and most have programs where they sell books at a big discount. Librarians are also some of the smartest people you will meet and who know everything about the cities they serve. It has been interesting learning that "Barnes & Noble is planning to open at least 60 new stores in 2025 — a stunning turnaround for a bookseller that just a few years ago closed hundreds of locations and appeared doomed to follow in the footsteps of its shuttered competitors. Barnes & Noble opened 57 new locations in 2024 and operates around 600 stores in total, making it the largest bookseller in the US. In recent years, the bookseller has also leaned into the popularity of book content on TikTok and using #BookTok as the hashtag - and the post-pandemic thirst for "third spaces" to meet and socialize." - Business Insider To that end - our stories matter. Another favorite quote from Doctor Who, "We are all stories in the end, just make is a good one." Stories have the power to inspire, connect, and bring people together. Start by sharing your personal experiences, whether through conversations, writing, or digital platforms. Encourage others to do the same and contribute to a culture of storytelling that strengthens your local community, this community - the best community in tech. What story will you share today? And where and how will you share it? #ThePowerofCommunity #Storytelling #CommunityBuilding #BelongingMatters #CuppaCommuniTea #MicrosoftCommunity 🍵 Today I am drinking a cuppa Throat Coat by Traditional Medicines - Organic Throat Coa tea supports throat health with renowned slippery elm, used in Native American herbal medicine for hundreds of years.The Heart of Community – How Friendship Strengthens Our Social Circles
On building community. “Social connections are one of the most important factors in determining our well-being and longevity. Strong friendships benefit both individuals and the entire community. By prioritizing these relationships, we can build stronger, healthier communities." Psychologist Dr. Julianne Holt-Lunstad Friendship is vital for strong communities, providing emotional and social benefits. It fosters connection, trust, and belonging. Close friendships enhance personal well-being, reducing loneliness and boosting happiness. Research from the American Psychological Association shows that people with strong social connections are 50% more likely to live longer. Friendship's value goes beyond personal health; it positively impacts communities. Connected individuals are more likely to engage in social causes, collaborate on initiatives, and support each other, creating collective responsibility. This applies whether it is your local community, those that are part of your technical community, or hobbies you love like a book club or board game nights/groups. Host a community gathering – Organize a small event or gathering in your neighborhood to bring people together and foster new connections. Host a small meeting online for a quarterly check-in with friends. (I have a standing meeting with some college friends every other Sunday, sometimes there are two of us, sometimes six, sometimes we cancel, but having it on the calendar keeps it going. I find this time grounding and many times support just when I need it). Support local businesses – Frequent local shops and restaurants and engage with the owners and staff to build relationships within your community. (I recently debuted my DJing house music skills at a neighborhood benefit we did for our Los Angeles animal rescues for the wildfires.) Get to know your local business owners, chamber, rotary they are your neighbors! Volunteer for local causes – Dedicate some time to volunteer for community projects or organizations that resonate with you, helping to strengthen communal ties. One organization could benefit from even 1 hour of time and volunteering a way to make new acquaintances and friends. In the Microsoft Community space, a big shout out to those who produce both free and paid technical skilling community events and who run user groups - you are the purveyors of belonging and friendship. Be sure to check out all events in the Microsoft Community ecosystem on Communitydays.org. If you are a producer who creates events around Microsoft products, please list your event on the site! As we continue to spotlight and celebrate the individuals and initiatives that make our communities vibrant and connected, let us remember that each small act of connection and kindness contributes to the greater whole. By fostering friendships and creating inclusive spaces, we not only enhance our own lives but also build resilient, supportive communities. Let's all take a moment to reach out, engage, and uplift those around us, knowing that every gesture matters in the tapestry of community life. After twenty-four years in this space, I feel grateful to call so many of you friends of mine as well as colleagues. What is one way you can reach out to a friend today? #TheHeartofCommunity #BelongingMatters #CuppaCommuniTea #MicrosoftCommunity 🍵 Today I am drinking a cuppa Lemon Ginger tea from Bigelow Family Tea Blenders since 1945 - Fairfield, Connecticut. A probiotic tea that is a delicious blend of zesty lemon and ginger combined with BC30™ probiotic that supports healthy digestion.
Need to Recover Deleted Files from Local OneDrive Folder on Linux
I mistakenly synced my local OneDrive folder to the remote OneDrive using rclone, intending to update the remote with my local files. However, the sync went the wrong way, and now my local files are gone. Unfortunately, there’s nothing in the remote OneDrive Recycle Bin because all the files were originally on my local machine. I’m working on a Linux system and need to recover the deleted files from my local OneDrive folder. Is there a way to recover these files from the local Recycle Bin or any other method to restore them? Any advice or tools recommended for file recovery on Linux would be greatly appreciated! Thank you!
Shared folder sync not possible anymore?! Workaround?
Hi guys We are using the OneDrive for Business. (Work) Around 6 months ago, I made a OD share with my colleague from the same organization. He was able to sync this share on his machine with the local OneDrive client. My colleague had a breake for 6 months, during this time he was not working on that share. Today he starts again with the work. We realized that it's not possible anymore, to sync the shared folder (where Im the owner of the share) with his local OneDrive Client. Accessing to the share over web is no problem. Why is that so? Is there a workaround? If not, this fact would make the cooperation work over OneDrive really useless. After research I did find this information: Source: https://support.office.com/en-us/article/add-and-sync-shared-folders-to-onedrive-8a63cd47-1526-4cd8-bd09-ee3f9bfc1504 Thank you in advance.
