Sep 28 2017 03:26 AM
I've got a Cloud PBX subscription with a few users enabled for testing. We also have Cloud Connector Edition installed, and according to our Skype for Business Online admin portal, it is reporting in as a on premisis PSTN equipment that is running.
We've double checked all of the configuration settings against the variety of config guides availalbe and it looks like we've got everything configured correctly. However, when any of our test users make a call from their Skybe for Business Client to an outside line, the call results in a fast busy or a "Number not in service" message (iPhone client).
I've sniffed traffic to/from our edge server and I'm not seeing ANY inbound traffic at the time of the test calls on the SIP port or the media ports. I confirmed that the prots are open on our firewall, and also tried connecting through the firewall with a telnet client to ensure the ports were open and I see those connection attempts with the packet captures.
Any help in troubleshooting why calls aren't routing correctly from the Cloud PBX to the edge server would be appreciated.
Sep 29 2017 06:46 AM
Some additional info... I ended up unregsitering and uninstalling our CCE, and then rerunning the install and registration process... I noticed at the end a warning that read
Warning: The network configuration has not been set for the tenant.
Sep 30 2017 09:06 AM
Possible to share your SFB client logs as well as Gateway's log file to determine the root cause of the issue?
Sep 30 2017 12:50 PM
UNfortunately, there are no logs to share!! I followed the instructions here: https://technet.microsoft.com/en-us/library/mt605228.aspx
And as I mentioned at the end of the Install-CcAppliance script, it gives me a warning that says "Network configuration has not been set for this Tenant"
When I try and place a call from a skype client to an external PSTN line, the call goes fast busy immediately. I used snooper to look at the logs for the skype client and there's an ms-diagnostics error 1003: USer not found. So for some reason, Skype Online is not determining that the phone number is external to my Tenant.
When sniffing traffic going to/from my edge server on the external switch connected to the Internet, I see NO TRAFFIC coming in from the outside when trying to place calls. I've confirmed that the appropriate ports ARE OPEN on the firewall.
So this appears to be an issue inside the tenant and I can't track down what to look at next - is it a dial plan, voice routing, some combination of the two? I've confirmed that all the settings described in the technet article above are correct
Sep 30 2017 03:55 PM
It looks like you haven't run Hybrid command yet?
Ref: https://technet.microsoft.com/en-us/library/mt740651.asp
Run this command through SFB Online PowerShell.
Set-CsTenantHybridConfiguration -PeerDestination <External Access Edge FQDN> -UseOnPremDialPlan $false Set-CsTenantFederationConfiguration -SharedSipAddressSpace $True
Also login to your SFB Online through PowerShell and run the following command, replace user with the SFB Online user you provisioned Enterprise Voice.
get-csusersession -User USER@YOURDOMAIN.COM -StartTime (Get-Date).AddHours(-4)
Run the following command and paste output here
Get-CsTenantHybridConfiguration
Oct 01 2017 05:28 AM - edited Oct 01 2017 05:31 AM
So I'd actually run both those hybrid commands during the initial configuration and installation of cloud connector, so obviously they didn't help the problem.
When I searched for user sessions, there was nothing found in the time period desipte several attempts at test calls.
The output of the Get-CsTenantHybridConfiguration is below.
Identity : Global HybridPSTNSites : {Index=1;Name=RAC;EdgeFQDN=ap.rehobothambulance.org;EnableAutoUpdate=True;LastTopologyUpdateTime=09/30/2017 19:36:58;BitsUpdateTimeWindowList=BitsUpdateDefault;OsUpdateTimeWindowList=OsUpdateDefault} HybridPSTNAppliances : {Identity=8eb5905f-e506-4cb9-8cca-1a30a129c409;Name=RACHYPERV;SiteIndex=1;MediationServerIPAddress=192.168.3.212;MediationServerFqdn=MedServer.rac.local;Mediation ServerGruu=sip:MedServer.rac.local@rehobothambulance.org;gruu;opaque=srvr:MediationServer:Ff5N1ODPQ1C7cKYyZpotzwAA;MaintenanceMode=False;ConfigurationReplicatedOn =09/30/2017 22:56:26;ConfigurationSnapshot={"EnableBypass":false,"EnableReferSupport":true,"EnableFastFailoverTimer":true,"ForwardCallHistory":false,"ForwardPAI": false};ConfigurationSnapshotUpdatedOn=09/30/2017 22:56:38;RegistrationStatus=RegisteredAndDeployed;RegistrationAction=Register;RunningVersion=2.0.1;RunningStatus=Running;RunningError=;OsUpdatedOn=10/01/2017 04:54:47;DeployedOn=09/30/2017 22:45:22;StatusUpdatedOn=10/01/2017 04:54:47;DeploymentVersion=2.0.1;DeploymentStatus=Deployed;DeploymentError=;DeploymentStartTime=09/30/2017 19:39:16;OsUpdateStatus=None;OsUpdateError=;OsUpdateStartTime=10/01/2017 04:11:35} TenantUpdateTimeWindows : {Name=BitsUpdateDefault;Type=Weekly;StartTime=00:00:00;Duration=1.00:00:00;DayOfMonth=;WeeksOfMonth=;DaysOfWeek=1, Name=OsUpdateDefault;Type=Monthly;StartTime=00:00:00;Duration=1.00:00:00;DayOfMonth=;WeeksOfMonth=1;DaysOfWeek=1} PeerDestination : ap.rehobothambulance.org HybridConfigServiceInternalUrl : HybridConfigServiceExternalUrl : UseOnPremDialPlan : False
One other thing I noticed when I used Get-CSOnlineUser to look at my user configuration is the users are all using InternationalCallsDisallowed, which is OK since we don't need international calling. When I look at the voice routing policy for InternationalCallsDisallowed, I noticed the following:
Identity : Tag:InternationalCallsDisallowed PstnUsages : {} Description : Name : InternationalCallsDisallowed AllowInternationalCalls : False HybridPSTNSiteIndex : 0
And when I looked at Get-CsHybridPstnSite, I see the following:
RunspaceId : 605d99cc-f1be-47ef-9f35-7b08ef10d734 Identity : RAC EdgeFQDN : ap.rehobothambulance.org Index : 1 EnableAutoUpdate : True LastTopologyUpdateTime : 9/30/2017 7:36:58 PM BitsUpdateTimeWindowList : {BitsUpdateDefault} OsUpdateTimeWindowList : {OsUpdateDefault}
It looks like the PSTN Site Index has a mismatch... The voicerouting policy is matching to index 0, but the PSTN site shows that it's index is 1...
The PstnUsages is empty... Is this potentially why calls aren't routing out to the PSTN? The voice routing policy doens't know what to use it for..
Oct 01 2017 06:13 AM
Some more information. Here's the error report information from using Snooper to examine the client logs of a Skybe for Business online client:
- <reportError xmlns="http://schemas.microsoft.com/2006/09/sip/error-reporting"> - <error toUri="sip:+1##########@SIPDOMAIN.org;user=phone" callId="8e7b229fe4d342c691cc22f1e0c8389f" fromTag="02c2a3c5f3" toTag="CDA1DBB01E51BD29E26FCFE674ED47CD" contentType="application/sdp;call-type=audiovideo" responseCode="404" requestType="INVITE"> <diagHeader>1003;reason="User does not exist";domain="SIPDOMAIN.org";source="sipfed2a.online.lync.com";OriginalPresenceState="0";CurrentPresenceState="0";MeInsideUser="No";ConversationInitiatedBy="0";SourceNetwork="0";RemotePartyCanDoIM="No"</diagHeader> <progressReports/> - </error> </reportError>
the actual phone number dialed has been replaced with ##########, but it is a valid 10 digit US phone number. Also replace domain with SIPDOMAIN.org
Oct 01 2017 07:45 AM
Oct 01 2017 03:46 PM
Mine looks identical to yours.
Oct 01 2017 09:16 PM
Oct 02 2017 03:55 AM
We haven't been worrying about incoming calls since we forward our main number into an Auto Attendant. I'll have to unforward that and do an inbound call test at some point.
I'll be looking into the event logs on the edge and mediation servers.
Unified messaging is disabled on users.
Problem seems to be across all users.
And here's a sanitized Get-CSOnlineUser. Also, the PSTN Settings for the user include the sitename and the edge server FQDN so thats correct as well...
RunspaceId : 8fb97b90-b8e7-49e1-a518-4677947de9af UserAccountControl : PasswordNotRequired, NormalAccount Id : CN=92e1bd1e-bbb7-453f-bea6-a1dec1dad6b6,OU=88c87be0-cf12-4a16-a8cc-ef7fff55d35d,OU=OCS Tenants,DC=lync2a001,DC=local CountryAbbreviation : US CountryOrRegionDisplayName : United States Company : Department : Description : {Administrator} Fax : HomePhone : IPPhone : City : Rehoboth Manager : MobilePhone : aaaaaaaaaa OriginatorSid : OtherTelephone : {} Office : PostalCode : 02769 PreferredLanguage : en-US Puid : 10037FFEA1CE11F9 ServiceInfo : {} StateOrProvince : MA Street : StreetAddress : street address ThumbnailPhoto : Title : Phone : bbb-bbb-bbbb x2 WebPage : AdminDescription : AssignedPlan : {<XmlValueAssignedPlan xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <Plan SubscribedPlanId="3afef5df-3756-40e7-87c7-ffa126e720a3" ServiceInstance="MicrosoftCommunicationsOnline/NOAM-2A-S6" CapabilityStatus="Enabled" AssignedTimestamp="2017-09-15T22:50:50Z" ServicePlanId="3e26ee1f-8a5f-4d52-aee2-b81ce45c8f40" xmlns="http://schemas.microsoft.com/online/directoryservices/change/2008/11"> <Capability> <Capability Plan="MCOMEETADD" xmlns="http://schemas.microsoft.com/online/MCO/2009/01" /> </Capability> </Plan> </XmlValueAssignedPlan>, <XmlValueAssignedPlan xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <Plan SubscribedPlanId="8a2a79ba-5fde-403a-9ebf-8ce97dd5a6f1" ServiceInstance="MicrosoftCommunicationsOnline/NOAM-2A-S6" CapabilityStatus="Enabled" AssignedTimestamp="2017-09-15T22:50:50Z" ServicePlanId="4828c8ec-dc2e-4779-b502-87ac9ce28ab7" xmlns="http://schemas.microsoft.com/online/directoryservices/change/2008/11"> <Capability> <Capability Plan="MCOEV" xmlns="http://schemas.microsoft.com/online/MCO/2009/01" /> </Capability> </Plan> </XmlValueAssignedPlan>, <XmlValueAssignedPlan xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <Plan SubscribedPlanId="b6af42c0-5a7f-4398-85b2-35fff291e98d" ServiceInstance="MicrosoftCommunicationsOnline/NOAM-2A-S6" CapabilityStatus="Enabled" AssignedTimestamp="2017-09-15T22:50:50Z" ServicePlanId="0feaeb32-d00e-4d66-bd5a-43b5b83db82c" xmlns="http://schemas.microsoft.com/online/directoryservices/change/2008/11"> <Capability> <Capability Plan="MCOProfessional" xmlns="http://schemas.microsoft.com/online/MCO/2009/01" /> </Capability> </Plan> </XmlValueAssignedPlan>, <XmlValueAssignedPlan xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <Plan SubscribedPlanId="98236811-e581-4903-b9ef-4e4af7e278fd" ServiceInstance="MicrosoftCommunicationsOnline/NOAM-2A-S6" CapabilityStatus="Deleted" AssignedTimestamp="2017-09-15T15:47:19Z" ServicePlanId="4ed3ff63-69d7-4fb7-b984-5aec7f605ca8" xmlns="http://schemas.microsoft.com/online/directoryservices/change/2008/11"> <Capability> <Capability Plan="MCOPSTN1" xmlns="http://schemas.microsoft.com/online/MCO/2009/01" /> </Capability> </Plan> </XmlValueAssignedPlan>} InterpretedUserType : PureOnline Alias : rfischman BaseSimpleUrl : DirSyncEnabled : ObjectId : 92e1bd1e-bbb7-453f-bea6-a1dec1dad6b6 UsageLocation : US HideFromAddressLists : False OnPremHideFromAddressLists : False ProvisionedPlan : {<XmlValueProvisionedPlan xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <Plan SubscribedPlanId="8a2a79ba-5fde-403a-9ebf-8ce97dd5a6f1" ServiceInstance="MicrosoftCommunicationsOnline/NOAM-2A-S6" CapabilityStatus="Enabled" AssignedTimestamp="2017-09-15T22:50:50Z" ProvisioningStatus="Success" ProvisionedTimestamp="2017-09-15T23:57:50.0375131Z" xmlns="http://schemas.microsoft.com/online/directoryservices/change/2008/11" /> </XmlValueProvisionedPlan>, <XmlValueProvisionedPlan xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <Plan SubscribedPlanId="3afef5df-3756-40e7-87c7-ffa126e720a3" ServiceInstance="MicrosoftCommunicationsOnline/NOAM-2A-S6" CapabilityStatus="Enabled" AssignedTimestamp="2017-09-15T22:50:50Z" ProvisioningStatus="Success" ProvisionedTimestamp="2017-09-15T23:57:50.0375131Z" xmlns="http://schemas.microsoft.com/online/directoryservices/change/2008/11" /> </XmlValueProvisionedPlan>, <XmlValueProvisionedPlan xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <Plan SubscribedPlanId="b6af42c0-5a7f-4398-85b2-35fff291e98d" ServiceInstance="MicrosoftCommunicationsOnline/NOAM-2A-S6" CapabilityStatus="Enabled" AssignedTimestamp="2017-09-15T22:50:50Z" ProvisioningStatus="Success" ProvisionedTimestamp="2017-09-15T23:55:24.6156401Z" xmlns="http://schemas.microsoft.com/online/directoryservices/change/2008/11" /> </XmlValueProvisionedPlan>, <XmlValueProvisionedPlan xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <Plan SubscribedPlanId="98236811-e581-4903-b9ef-4e4af7e278fd" ServiceInstance="MicrosoftCommunicationsOnline/NOAM-2A-S6" CapabilityStatus="Deleted" AssignedTimestamp="2017-09-15T15:47:19Z" ProvisioningStatus="Success" ProvisionedTimestamp="2017-09-15T16:13:40.9297247Z" xmlns="http://schemas.microsoft.com/online/directoryservices/change/2008/11" /> </XmlValueProvisionedPlan>} ProvisioningStamp : SubProvisioningStamp : UpgradeRetryCounter : 0 SyncingCounter : ProvisioningCounter : 0 SubProvisioningCounter : 0 PublishingStamp : PublishingCounter : 0 OnPremHostingProvider : OnPremOptionFlags : 0 OnPremEnterpriseVoiceEnabled : False OnPremSIPEnabled : OnPremSipAddress : OnPremLineURI : tel:+1bbbbbbbbbb MCOValidationError : {} ShadowProxyAddresses : {} SipProxyAddress : sip:rfischman@SIPDOMAIN.org ServiceInstance : microsoftcommunicationsonline/noam-2a-s6 StsRefreshTokensValidFrom : 6/4/2017 4:55:27 PM LicenseRemovalTimestamp : LastSyncTimeStamp : 9/15/2017 7:54:56 PM LastProvisionTimeStamp : 9/15/2017 7:55:24 PM LastPublishTimeStamp : 9/29/2017 6:26:48 AM LastSubProvisionTimeStamp : 9/29/2017 6:26:34 AM ExchUserHoldPolicies : {} OwnerUrn : ContactOptionFlags : OnlineDialinConferencingPolicy : ServiceAllowed OverridePreferredDataLocation : PreferredDataLocation : SubProvisionLineType : UserProvisonType : NormalProvision SamAccountName : $CP9LB3-B185JLBIMFPI UserPrincipalName : rfischman@SIPDOMAIN.org FirstName : Firstname LastName : Lastname WindowsEmailAddress : rfischman@SIPDOMAIN.org Sid : S-1-5-21-782800828-1820537485-3938890916-112895788 LineServerURI : SoftDeletionTimestamp : PendingDeletion : False AudioVideoDisabled : False IPPBXSoftPhoneRoutingEnabled : False RemoteCallControlTelephonyEnabled : False PrivateLine : AcpInfo : {<acpInformation default="true"> <tollNumber>15085566648</tollNumber> <participantPassCode>0</participantPassCode> <domain>resources.lync.com</domain> <name>Microsoft</name> <url>https://dialin.lync.com/6f92fa88-330f-4102-8d4a-06783f547f47</url> </acpInformation>} OriginalPreferredDataLocation : HostedVoiceMail : True DisplayName : First Last ProxyAddresses : {SMTP:rfischman@SIPDOMAIN.org, sip:rfischman@SIPDOMAIN.org} HomeServer : TargetServerIfMoving : EnterpriseVoiceEnabled : True EnabledForRichPresence : True ExchangeArchivingPolicy : Uninitialized NonPrimaryResource : False MNCReady : True OnPremLineURIManuallySet : True OptionFlags : 196992 LineURI : tel:+1bbbbbbbbbb SipAddress : sip:rfischman@SIPDOMAIN.org Enabled : True TenantId : <TENANT ID> UserRoutingGroupId : 9ced8589-94e4-593d-8988-4a09dd202495 TargetRegistrarPool : VoicePolicy : HybridVoice CallerIdPolicy : CallingLineIdentity : MobilityPolicy : MobilityEnableOutsideVoice ConferencingPolicy : BposSAllModality BroadcastMeetingPolicy : CloudMeetingPolicy : CloudMeetingOpsPolicy : TeamsMeetingPolicy : TeamsCallingPolicy : TeamsInteropPolicy : ClientUpdatePolicy : ClientUpdateOverridePolicy : OnlineVoicemailPolicy : PresencePolicy : VoiceRoutingPolicy : InternationalCallsDisallowed1 RegistrarPool : sippooldm12a05.infra.lync.com DialPlan : US TenantDialPlan : IPPhonePolicy : LocationPolicy : ClientPolicy : ClientVersionPolicy : ArchivingPolicy : LegalInterceptPolicy : PinPolicy : CallViaWorkPolicy : GraphPolicy : ExternalAccessPolicy : FederationAndPICDefault HostedVoicemailPolicy : BusinessVoice PersistentChatPolicy : UserServicesPolicy : ExperiencePolicy : W15DefaultUX XForestMovePolicy : PreferredDataLocationOverwritePolicy : AddressBookPolicy : SmsServicePolicy : ExternalUserCommunicationPolicy : ThirdPartyVideoSystemPolicy : CloudVideoInteropPolicy : OnlineDialOutPolicy : HostingProvider : sipfed.online.lync.com ExUmEnabled : False Name : 92e1bd1e-bbb7-453f-bea6-a1dec1dad6b6 DistinguishedName : CN=92e1bd1e-bbb7-453f-bea6-a1dec1dad6b6,OU=88c87be0-cf12-4a16-a8cc-ef7fff55d35d,OU=OCS Tenants,DC=lync2a001,DC=local Identity : CN=92e1bd1e-bbb7-453f-bea6-a1dec1dad6b6,OU=88c87be0-cf12-4a16-a8cc-ef7fff55d35d,OU=OCS Tenants,DC=lync2a001,DC=local Guid : e971e425-7b04-4464-b807-f1efd2b377f9 ObjectCategory : CN=Person,CN=Schema,CN=Configuration,DC=lync2a001,DC=local ObjectClass : {top, person, organizationalPerson, user} WhenChanged : 10/2/2017 6:07:20 AM WhenCreated : 6/5/2017 8:58:26 PM OriginatingServer : BLU2A00ADS01.lync2a001.local IsByPassValidation : True IsValid : True ObjectState : Unchanged
Oct 02 2017 04:33 AM
Oct 02 2017 04:44 AM
We've migrated to online only. I ran AZ AD Sync with our on-prem AD to migrate our users online. Once that was done, we retired the on-prem AD. THe only AD On-prem now is the one associated with CCE.
Oct 02 2017 04:59 AM
Oct 02 2017 05:08 AM
Oct 02 2017 05:16 AM
Oct 02 2017 05:21 AM
Oct 02 2017 06:59 AM
Here's the output of the user I'm testing with. Happens to be my account..
ExtensionData : System.Runtime.Serialization.ExtensionDataObject AlternateEmailAddresses : {ALTEMAIL} AlternateMobilePhones : {} AlternativeSecurityIds : {} BlockCredential : False City : CITY CloudExchangeRecipientDisplayType : 1073741824 Country : United States Department : DirSyncProvisioningErrors : {} DisplayName : DISPLAY NAME Errors : Fax : FirstName : FIRST NAME ImmutableId : i9Zr9uuYH0ugtQvi2vE3WQ== IndirectLicenseErrors : {} IsBlackberryUser : False IsLicensed : True LastDirSyncTime : LastName : LAST NAME LastPasswordChangeTimestamp : 6/4/2017 8:55:27 PM LicenseReconciliationNeeded : False Licenses : {rfischmanrehobothambulance:ENTERPRISEPREMIUM, rfischmanrehobothambulance:EMS} LiveId : 10037FFEA1CE11F9 MSExchRecipientTypeDetails : MobilePhone : MOBILEPHONE ObjectId : 92e1bd1e-bbb7-453f-bea6-a1dec1dad6b6 Office : OverallProvisioningStatus : PendingInput PasswordNeverExpires : XXXXXXX PasswordResetNotRequiredDuringActivate : PhoneNumber : OFFICE PHONE PortalSettings : PortalSettings PostalCode : ZIP PreferredDataLocation : PreferredLanguage : en-US ProxyAddresses : {SMTP:EMAIL ADDRESS} ReleaseTrack : ServiceInformation : {} SignInName : SIGNIN NAME SoftDeletionTimestamp : State : STATE StreetAddress : STREET ADDRESS StrongAuthenticationMethods : {Microsoft.Online.Administration.StrongAuthenticationMethod, Microsoft.Online.Administration.StrongAuthenticationMethod} StrongAuthenticationPhoneAppDetails : {} StrongAuthenticationProofupTime : StrongAuthenticationRequirements : {} StrongAuthenticationUserDetails : Microsoft.Online.Administration.StrongAuthenticationUserDetails StrongPasswordRequired : True StsRefreshTokensValidFrom : 6/4/2017 8:55:27 PM Title : UsageLocation : US UserLandingPageIdentifierForO365Shell : UserPrincipalName : USERPRINCIAPLNAME UserThemeIdentifierForO365Shell : UserType : Member ValidationStatus : Healthy WhenCreated : 5/28/2017 1:34:44 PM
And I noticed provisoning status shows pending input. Here's the status of the assigned products:
ServicePlan ProvisioningStatus ----------- ------------------ FORMS_PLAN_E5 Success STREAM_O365_E5 Success THREAT_INTELLIGENCE Success Deskless Success FLOW_O365_P3 Success POWERAPPS_O365_P3 Success TEAMS1 Success ADALLOM_S_O365 PendingInput EQUIVIO_ANALYTICS Success LOCKBOX_ENTERPRISE Success EXCHANGE_ANALYTICS Success SWAY Success ATP_ENTERPRISE PendingProvisioning MCOEV Success MCOMEETADV Success BI_AZURE_P2 Success INTUNE_O365 PendingActivation PROJECTWORKMANAGEMENT Success RMS_S_ENTERPRISE Success YAMMER_ENTERPRISE Success OFFICESUBSCRIPTION Success MCOSTANDARD Success EXCHANGE_S_ENTERPRISE Success SHAREPOINTENTERPRISE Success SHAREPOINTWAC Success
Oct 02 2017 07:37 AM
It's confirmed that Microsoft Office365 portal is still seeing all your users as "Synced with on-premise Active Directory"
Root Cause: Microsoft Office365 portal is still seeing all your users as "Synced with on-premise Active Directory" after you migrated your AD users to Office365.
This has been a long strech of finding out what's going on. Finally, I managed to pin down the "Root Cause of the Issue" in conclusive manner. :D
As you can see, the ImmutableID is still showing GUID which means Office365 portal is still seeing all your users as "Synced with on-premise Active Directory"
ImmutableId : i9Zr9uuYH0ugtQvi2vE3WQ==
If it is a pure Cloud user, you shold see "blank" in ImmutableID paramter.
Let me see how to fix/workaround over it.
Oct 02 2017 08:13 AM
SOLUTION
Set-MsolDirSyncEnabled –EnableDirSync $false
(Get-MSOLCompanyInformation).DirectorySynchronizationEnabled