We currently use Office 365 Video and SharePoint document libraries for a key business process around video surveys. The surveys have two components a Video that is uploaded to office 365 Video and Document / Images that are uploaded to SharePoint Document Libraries. Users are sent a notification email that includes an embedded URL which performs a delve search and retrieves documents, images and videos from within Delve.
We are working with premier support to understand our transition path but I note in the FAQ that Stream Videos showing up in Delve is scheduled for 2018.
Unfortunately, Stream doesn't have an API available yet. And since Stream videos are indeed not added to the OfficeGraph/Delve yet, the only option that I currently see is what you are already doing.
The Stream API has been informally announced for this year though, and I know making videos searchable from SharePoint/Delve is on the to do list of the product team as well. My guess is that you won't be forced into a migration until functionality like this is available, but that is just that, a guess.