There is a setting in AADConnect (and also a powershell that can be run) to enable this (it is called hybrid azure ad joining". Basically tells Azure that these computers exist in your On-Prem AD
The place we look is in Azure Portal, Home > Microsoft Intune > Azure AD Devices
You can also get there at Azure Portal > Azure Active Directory > Devices
You'll see two entries for each computer if you've done it right under Join Type, 1 for the "Azure AD Registered", and 1 for the "Hybrid Azure AD Joined". We've never been able to figure out how to merge them into the same entry.