different session hosts in the same hostpool?

You basically deploy one kind of base image (with pre-configured settings and software) to a single host-pool. So if the hostserver needs to be different for those two users groups and you cannot force this difference via Group Policy or AppAttach/AppMasking then you will need to deploy two host-pools with a different base pre-configured image for both. You'd also need two different hostpools if you don't want the users to mingle on the same server since you attach your FullDesktop Application Group on a hostpool level and not a server level.

Look up FSLogix App Masking if you want to cut costs by only deploying a few servers that are shared by everyone but access to the installed software needs to be restricted for some (active directory) groups. That way you only need to manage/update one central image instead of separate ones for each user group.
https://docs.microsoft.com/en-us/fslogix/implement-application-masking-tutorial