Is the description "on-prem" technically/strictly referring to when we have our own servers in-house? Or can that also apply for example when running a SP farm in a hosted VM? And I assume all that's different than O365 (which is I guess only referring to hosted by Microsoft).
These configuration definitions could be helpful when reading how-to articles online for example, especially if on-prem examples are actually applicable to hosted situations where you're still in control of some part of the setup/config.
When reading how-to articles, it can get tricky, Some articles from SP 2010 are still applicable to SPonline, but many are not, It really depends on the specific topic. In general most of the articles about installation and configuration of a SP farm are not applicable to SPonline. If you compare the SPO admin center with SP Central Admin, you can get a sense of the overlapping controls (i.e., search properties can be configured in both but search crawl schedule cannot be managed in SPO)