What broke, how I tested each fix, what Clarity Copilot told me, and what I had solved.
I run the website for a small foundation in Malaysia. We do food aid, school supplies for kids from low-income families, and practical help when a family hits a rough patch. The site is in English and Bahasa Melayu, and volunteers write our stories in a small admin area.
A few weeks ago I set myself a simple goal: when someone in Malaysia searches for help, or for a way to give, they should find us. I expected a weekend of tweaking titles. Instead I spent a lot longer finding problems that were invisible in a browser and very visible to a crawler.
This post walks through the whole thing as a set of test cases. Each one has what I expected, what I actually found, what I changed, and how I proved the fix worked. Most of the evidence comes from Microsoft tools: Bing Webmaster Tools, Microsoft Clarity with Copilot, Azure AI Translator and IndexNow. I've kept the parts that are still broken too, because I think those are the most useful.
The Microsoft side of the setup
I won't go into the rest of our stack. It's a modern web framework with a CDN in front, and that's all that matters for this post. The interesting part is what sits around it.
Bing Webmaster Tools is where I check indexing, run SEO scans, submit the sitemap and look at keyword data. Microsoft Clarity records how real visitors use the site, and its built-in Copilot answers questions about that data in plain English. Azure AI Translator keeps the English and Malay versions of every story in step, so we never publish in one language only. And IndexNow, the open protocol Bing supports, tells search engines the moment a page changes. All four are free or close to it, which matters a lot when you're a charity.
Our home page. It looked fine to me, which was the whole problem.
Test case 1: does Bing actually like our home page?
My first stop was URL Inspection in Bing Webmaster Tools. It has two tabs, and the difference matters. Bing Index shows the page as Bing last stored it, while Live URL fetches it right now. Together they make a perfect before and after tool.
The stored copy of our English home page, from 31 October, had two problems:
Bing Index view: "Title too long" (error) and "Alt attribute for images is missing" (notice).
The title problem was everywhere, not just on the home page. Every title had the full foundation name appended, and plenty of them ran past 70 characters, so Bing and Google cut them off halfway through a sentence. I wrote a small function that fits each title into 70 characters. It tries the full name first, then our short initials, and drops the brand completely when the page title is long enough on its own. Unit tests keep it honest.
The alt text problem was funnier. I expected the logo and team photos to be missing descriptions. I didn't expect a pile of story images whose alt text was literally "BlockNote image", the default label from our rich text editor. Those now fall back to the story title, and every other image has a real description.
Then I ran the Live URL test on the same page:
Live URL view, same page, same day: "No SEO/GEO issues found".
| Test case 1: home page title and alt text | |
|---|---|
| Tool | Bing Webmaster Tools, URL Inspection |
| Expected | No SEO issues on the English home page |
| Found | Title too long (error) and missing alt text (notice) |
| Fix | 70 character title budget with a short brand fallback; real alt text on every image |
| Result | Live URL test shows no SEO/GEO issues |
The unit tests that guard the title rules and the story checklist.
Test case 2: is our structured data telling the truth?
This one was a bit embarrassing. Every page carries a block of JSON-LD that tells search engines "this is an NGO, here's the address, here's how to reach us." Ours had an address in a city we don't operate from, a placeholder phone number, a founding year nobody could confirm and social links someone had guessed. All of it came with a template, and none of it came from us.
For a charity, trust is the product, so I deleted every invented value. The structured data now reads from a settings page we actually maintain, and an empty field is left out instead of guessed. Stories carry NewsArticle and breadcrumb markup, and the Contact page has a short FAQ with matching FAQPage markup.
Our live NGO markup. Every value comes from settings (contact details hidden for this post).
URL Inspection reports "1 Markup type found" on the stored copy, so Bing is reading it. If you do one thing after this post, open that section for your own home page. It's the quickest way to see who Bing thinks you are.
| Test case 2: structured data | |
|---|---|
| Tool | Bing Webmaster Tools, URL Inspection (Markup) |
| Expected | Organisation details that match reality |
| Found | Invented address, placeholder phone number and guessed social links from a template |
| Fix | Markup now reads from our own settings; empty fields are left out |
| Result | Bing reads the markup and every value in it is real |
Test case 3: are the Malay pages really Malay?
Our Malay pages had English headings, English buttons and sometimes whole English paragraphs, and some links on them dropped visitors onto the English version. To a crawler that looks like one confused page rather than two languages, and a lot of Malaysians search in Malay.
Bing leans on the language a page declares more than on hreflang, so I made sure every Malay page declares itself as Malay and every English page as English. I added hreflang with an x-default for Google, and fixed the links so you stay in the language you started in.
Translation is where Azure came in. Nobody on our team has time to write every story twice, so the story editor has a "Translate whole story" button that calls Azure AI Translator. Out of the box it mangled some of our words, like asnaf, zakat, B40 and our programme names. I added a small glossary that protects those terms before the text goes to the API and puts them back afterwards, and the glossary has its own tests.
The Malay home page, Malay all the way through, buttons included.
The safety net is an audit script I run against the live site. It visits every URL in the sitemap and fails if a page's language doesn't match its address, if the main text reads like the other language, if a link loses its language prefix, or if an image is broken.
Today's run against production: 18 pages and 21 images, PASS.
| Test case 3: language consistency | |
|---|---|
| Tool | Production audit script and Bing URL Inspection |
| Expected | Malay pages fully in Malay, English pages fully in English |
| Found | Mixed languages on Malay pages and links that switched language |
| Fix | Finished translations, declared language per page, hreflang, Azure AI Translator with a protected glossary |
| Result | Audit passes on 18 pages and 21 images, and it has already caught one regression |
Test case 4: can crawlers even render the page?
Our robots.txt blocked the folder that serves the site's styles and scripts. Crawlers could read the text but couldn't render the page the way a visitor sees it, and because everything looked perfect in a browser, nobody had noticed.
Removing that rule was the main fix. While I was there, I made every page declare the www address as its canonical home, marked donation receipts and the 404 page as noindex, and rebuilt the sitemap so it only lists www addresses. Then I resubmitted it in Bing Webmaster Tools.
Bing Webmaster Tools > Sitemaps: 1 sitemap, 0 errors, 0 warnings, 18 URLs discovered.
I did hit a setback here. My first Site Scan was pointed at the bare domain rather than www. It scanned exactly one page, hit an HTTP 4xx error and stopped. Scan the address you actually serve.
My first Site Scan, on the bare domain: 1 page scanned, 1 error (Http 400-499).
While writing this post I also noticed that the bare domain redirects to www with a temporary redirect. For a permanent move you want a permanent redirect, so search engines hand everything over to the www address with confidence. That one is on my list.
| Test case 4: crawlability and sitemap | |
|---|---|
| Tool | Bing Webmaster Tools, Sitemaps and Site Scan |
| Expected | Crawlers can render every page and find every URL |
| Found | robots.txt blocked our styles and scripts; the first Site Scan stopped after one page |
| Fix | Unblocked the folder, one canonical www address, a clean sitemap resubmitted |
| Result | 1 sitemap, 0 errors, 18 URLs discovered |
Test case 5: what are real visitors doing?
Our admin had fields for analytics IDs that someone had filled in long ago. They never saved. There was no error and no warning, the values just quietly disappeared, so we thought we were measuring visitors and we weren't. I rebuilt that settings page, then opened the live page source to check every value really rendered. After a few surprises I wasn't trusting anything I hadn't seen with my own eyes.
Then I added Microsoft Clarity, and it changed how I think about this whole project. Setup is one script tag, and I linked it to Bing Webmaster Tools so the two share data. Here's the dashboard for the last three days:
Clarity dashboard, last 3 days: 61 sessions (2 bot sessions excluded), 1 page per session, 77.25% scroll depth, 21 seconds active time, 6.56% dead clicks, 0% rage clicks.
Three numbers jumped out. People view one page per visit and leave, which tells me nothing is pulling them to a second page, and that's a content and linking problem rather than a technical one. Scroll depth is 77%, so when people arrive, they actually read. And 6.56% of sessions had a dead click, where someone clicked something that did nothing. On a site this small, that's worth chasing.
Asking Clarity Copilot
I don't have time to watch every recording, so I opened Copilot in the Clarity dashboard and asked it straight out: "Where are visitors getting stuck or frustrated on the site, and what should I fix first?"
Clarity Copilot's actual answer. It lists dead clicks by page and recommends starting with the Contact pages.
Copilot worked through the sessions and came back with dead clicks on our Contact pages in both languages and on a couple of blog pages, no rage clicks and no significant JavaScript errors. Its advice was to start with the Contact pages. I liked that it didn't stop at a number. It told me where to look.
Checking Copilot's answer with a heatmap
I wanted to see it for myself, so I opened the click heatmap for the Malay Contact page:
Click heatmap for /ms/contact. The most clicked element is our office address, which wasn't a link.
There it is. The most clicked thing on the page was our office address, with 2 of the 6 clicks, and it was plain text. People expect to tap an address and get a map, especially on a phone. We had a separate "Open in Google Maps" link underneath, but nobody went for it, so I made the whole address block the map link and shipped it the same day. Clarity also masks text by default (you can see the numbers and email hidden in the heatmap), which made me much more comfortable using it on a charity site.
| Test case 5: dead clicks on the Contact page | |
|---|---|
| Tool | Microsoft Clarity dashboard, Copilot and click heatmap |
| Expected | Few or no dead clicks |
| Found | 6.56% of sessions had dead clicks, mostly on the Contact pages, where visitors clicked the plain-text address |
| Fix | The whole address block is now the map link |
| Result | Live; I'll recheck the same heatmap after a week |
Clarity also caught my own mess
This one I didn't see coming. The Heatmaps page lists your most visited pages, and two of the top entries were addresses on localhost. That was me, testing on my own laptop, because the analytics script loaded everywhere and my development sessions were mixed in with real visitors.
Popular pages in Clarity. Rows 2 and 9 are my local machine, not real visitors.
Our IndexNow code already refused to run anywhere except the live site, so I gave analytics the same rule. It was a one-line change. The same list also showed our 404 page with the foundation name twice in its title, so that got its own proper title too. Two bugs found for free.
Test case 6: my fix broke the contact form
This is the one that taught me to test outside the SEO bubble. When I made www the canonical address, our contact form started rejecting submissions. Its security check only accepted requests from the address it already knew about, and after my change everyone was arriving on www.
The fix was small, and I added tests so it can't quietly break again. Funnily enough, Clarity Copilot's first recommendation was also to test the contact forms and buttons. Two different tools pointed at the same page. If you change your URLs, test your forms.
| Test case 6: contact form after the canonical change | |
|---|---|
| Tool | Manual form tests and Clarity Copilot |
| Expected | The contact form works on the www address |
| Found | Submissions were rejected after the canonical change |
| Fix | Security check updated and covered by automated tests |
| Result | The form works and the tests guard against a repeat |
Test case 7: can new pages reach Bing quickly?
Our site is small and Bing doesn't crawl it often, so waiting days for a new story to show up felt silly. I wired up IndexNow. Whenever a story or programme is published, edited, unpublished or deleted, the site sends both language versions to the IndexNow endpoint. Scheduled posts send theirs when they go live, and there's a "Send all pages now" button for big batches. The request itself is tiny:
POST https://api.indexnow.org/indexnow
{
"host": "www.example.org",
"key": "<your key>",
"keyLocation": "https://www.example.org/<your key>.txt",
"urlList": ["https://www.example.org/en/blog/...", "https://www.example.org/ms/blog/..."]
}
I hit two gotchas. The key file has to be served as plain text, not wrapped in an HTML page. And only the live site should send pings, otherwise test builds tell Bing about addresses that don't exist.
| Test case 7: getting new pages to Bing | |
|---|---|
| Tool | IndexNow and Bing Webmaster Tools |
| Expected | New and changed stories reach Bing within hours |
| Found | A small site that gets crawled rarely, so new stories took days to appear |
| Fix | Automatic IndexNow pings on publish, edit, unpublish and delete, in both languages |
| Result | Every change is submitted the moment it happens |
Test case 8: what's happening in AI search?
People ask Copilot and ChatGPT about charities now, and I wanted those answers to be right about us. I added llms.txt and llms-full.txt, plain summaries built from our live content, and a robots.txt that explicitly welcomes the main AI crawlers.
Bing Webmaster Tools has a new AI Performance report (still in beta) that shows how often your pages are cited as sources in Microsoft Copilot answers. Ours is honest and humbling:
Bing Webmaster Tools > AI Performance (beta): 0 citations, 0 cited pages over 3 months.
Zero. Then, while grabbing screenshots for this post, I found part of the reason other AI tools can't see us. Our CDN had quietly added its own managed block to the top of our robots.txt that disallows GPTBot, ClaudeBot, Google-Extended and others. Further down, my own rules allow the same bots, so the file argues with itself.
Our live robots.txt. The managed block at the top disallows AI crawlers (red); my rules further down allow them (green).
It isn't only the file, either. When I sent requests pretending to be those crawlers, some of them were turned away before they ever reached the site:
403 means blocked at the edge. GPTBot and ClaudeBot are refused; PerplexityBot and Bingbot get through.
Bingbot gets through, so Copilot can read us, and our zero citations are more likely down to having only a handful of stories so far. ChatGPT and Claude can't read us at all right now, though, which makes the llms.txt work pointless for them. This is a real decision rather than a bug. I'm leaning towards letting AI search and answer bots in while still blocking crawlers that only collect content for training.
Clarity has an AI Visibility section as well, which tracks AI bot activity and citations. Mine says "almost ready" because the domain needs verifying through Bing Webmaster Tools first, which is my next five-minute job.
Clarity > AI Visibility: waiting for domain verification via Bing Webmaster Tools or Google Search Console.
| Test case 8: AI crawler access | |
|---|---|
| Tool | Bing AI Performance, Clarity AI Visibility and crawler user-agent tests |
| Expected | AI assistants can read the site and llms.txt |
| Found | 0 Copilot citations so far; some AI crawlers are blocked at the CDN and robots.txt contradicts itself |
| Fix | Waiting on a decision about which AI crawlers to allow |
| Result | Open |
Helping our writers do the right thing
Everything above is plumbing. People find you through content, and our stories are written by volunteers who aren't SEO people. Handing them a style guide was never going to work, so the guidance lives inside the story editor. As they write, a 13-point checklist scores the story: is the focus phrase in the title, is the title the right length, is there a real search description, does every photo have a description, is it long enough, and does it link to another page. A preview shows the result as it will appear in search, using the same title rules as the live site. One click adds Malaysian search phrases in English or Malay, and "Translate whole story" uses Azure AI Translator so every story exists in both languages.
The story editor's "Search and sharing" panel. This story sits at 70% and tells you exactly what's missing.
Bing's Keyword Research taught me something too: Malay keyword data is thin. "Bantuan asnaf" (help for asnaf families) is a phrase people really use, but Bing doesn't have enough data to show a trend, and the top results are government and zakat bodies. We're not going to win that head term. We can win the long tail, with specific stories about specific programmes in specific places.
Bing Keyword Research for "bantuan asnaf": not enough data for a trend, and the top 10 are government and zakat sites.
Finally, I built a small Search insights page in our admin that uses the Bing Webmaster API and Google Search Console to show both engines side by side. It highlights phrases where we rank between 8 and 20, close to page one, and page-one results that nobody clicks. Bing is still collecting data for our www address, so that page will get more useful over the next few weeks.
Conclusion
When I started, I thought SEO was about keywords. For a small site like ours it turned out to be mostly about trust and hygiene: telling search engines the truth, making sure they can actually read the page, and making sure both languages are really there. Every fix in this post came from looking at the site the way a crawler does, not the way I do.
The Microsoft tools did most of the work. Bing Webmaster Tools gave me the clearest to-do list I had, and the Bing Index and Live URL tabs in URL Inspection gave me a clean before and after for every fix. Clarity showed me what real visitors do, and Copilot turned a pile of sessions into one clear instruction: start with the Contact page. Azure AI Translator keeps both languages in step, and IndexNow gets new stories in front of Bing within hours. None of it needed a budget.
Writing this post paid off too. Collecting the screenshots turned up three more bugs: my own laptop counting as a visitor, a doubled title on the 404 page, and an address people kept tapping because they expected a map. All three are fixed and live. Two things are still on my list, deciding how much access AI crawlers get and making the bare domain redirect permanent. After that comes the slow part, writing more stories and linking them together so visitors read a second page.
If you're doing this for a small nonprofit, look before you add anything, write your checks down as tests, and find out what sits in front of your site. And if you've already worked through the AI crawler question, I'd really like to hear how you handled it in the comments.