Trying to Build Truly Non-Persistent Machines

I'm trying to test a setup that will be used for public machines in a library setting through AVD and thin clients. To do this, I've built an AVD pool, which (according to documentation) is supposed to be completely non-persistent. However, I've noticed that if a user logs in and makes changes (creates files, changes settings, etc.) and then logs out, if they connect to the same virtual machine the next time they log in, their changes remain.

This isn't non-persistent as advertised, and is definitely a problem. Since the machines this is meant to be testing for will have potentially hundreds of users logging in, changes cannot remain when they log off. Does anyone know of a way to actually make these machines non-persistent?

An important note: We won't be able to utilize 3rd-party software like Nerdio (which I see a lot of people recommend, and which I wish I could use) because we're a public/government organization, and there's just no budget for a tool like that.