Looking for Guidance: Improving Playwright MCP Runbook Replay Reliability

hi shobhit-vishwakarma​ here is my 2 cents, 90% replay success is a good start. From my understanding, the main challenge here is that the ref generated from a snapshot is tied to that particular snapshot, so I wouldn't treat it as a permanent identifier for the element.

I would consider storing the ref as the first option, but also keep multiple ways to identify the element, such as:

• Role + accessible name
• Visible text
• Stable attributes or test IDs, where available
• href or other unique properties
• Parent/ancestor context
• Page URL or a normalized URL

During replay, you could try the original ref first. If it no longer exists, fall back to the strongest available locator and re-resolve the element from a fresh snapshot.

I'd also add explicit waits and verification between important steps. For example, instead of simply replaying click → type, use something closer to:

Navigate → Wait for expected state → Resolve element → Interact → Verify result

If the element can't be resolved, rather than failing the entire runbook, you could take a new snapshot and use a lightweight recovery step to identify the intended element based on the stored metadata. If successful, you could even update the runbook with the new locator information.

I think the key is to treat the runbook as self-healing rather than static replay. The ref is useful for fast execution, but the runbook should retain enough context to recover when the DOM or accessibility tree changes.

It may also be worth categorizing your remaining 10% failures timing, dynamic content, authentication/session state, element resolution, or navigation state. You might find that most failures are coming from one or two specific areas.

For a production target of 99%+, I'd probably go with a hybrid approach: deterministic replay for stable steps, with targeted re-resolution and recovery only when a step fails. That should preserve most of the latency and token savings you're getting while giving the automation a way to recover from dynamic pages.