08-12-2016 11:21 AM - edited 08-12-2016 11:23 AM
08-12-2016 11:21 AM - edited 08-12-2016 11:23 AM
Microsoft is pleased to announce the preview availability of a new security analytics service called the Office 365 Secure Score. The Secure Score is a security analytics tool that will help you understand what you have done to reduce the risk to your data in Office 365, and show you what you can do to further reduce that risk. We think of it as a credit score for security. Our approach to this experience was very simple. First, we created a full inventory of all the security configurations and behaviors that our customers can do to mitigate risks to their data in Office 365 (there are about 77 total things that we identified). Then, we evaluated the extent to which each of those controls mitigated a specific set of risks and awarded the control some points. More points means a more effective control for that risk. Lastly, we measure the extent to which your service has adopted the recommended controls, add up your points, and present it as a single score.
The core idea is that it is useful to rationalize and contextualize all of your cloud security configuration and behavioral options into one simple, analytical framework, and to make it very easy for you to take incremental action to improve your score over time. Rather than constructing a model with findings slotted into critical, moderate, or low severity, we wanted to give you a non-reactive way to evaluate your risk and make incremental changes over time that add up to a very effective risk mitigation plan.
The Office 365 Secure Score is a preview experience, so you may find issues, and you will note that not all of the controls are being measured. Please share any issues on the Office Network Group for Security. You can access the Secure Score at https://securescore.office.com.
The Secure Score does not express an absolute measure of how likely you are to get breached. It expresses the extent to which you have adopted controls which can offset the risk of being breached. No service can guarantee that you will not be breached, and the Secure Score should not be interpreted as a guarantee in any way.
Your Secure Score Summary
The first, most important piece of the Secure Score experience is the Score Summary. This panel gives you your current Secure Score, and the total number of points that are available to you, given your subscription level, the date that your score was measured, as well as a simple pie chart of your score. The denominator of your score is not intended to be a goal number to achieve. The full set of controls includes several that are very aggressive and will potentially have an adverse impact on your users’ productivity. Your goal should be to optimize your action to take every possible risk mitigating action while preserving your users’ productivity.
As mentioned, the Office 365 Secure Score is in a preview release. Over the coming months you will see us continue to add new controls, new measurements, and improvements to the remediation experiences. If you like what you see, please share with your network. If you see something we can improve, please share it with us on the Office Network Group for Security. We’re looking forward to seeing your scores go up, and making the Secure Score experience as useful, simple, and easy as it can be.
08-12-2016 12:23 PM
Wasn't it in Preview already? Or was that Private? Cause the links were publicly accessible and I've even showcased it on a local user group session... whoops? :)
Anyway, the idea is awesome, the tool was lacking some stuff last time I tried it though. Will give it a spin over the weekend and report back :)
08-12-2016 03:53 PM
Yep, it was in an alpha preview and was based on a manual collection of data via powershell. The new site is a non-alpha preview, and the collection is all automated. We've done a lot of work to refine the consumption experience, and have around 30 controls' worth of data collection automated. Goal is to have all 77 controls automatically collected, and to have all controls with a two-click remediation. Feedback is greatly appreciated!
08-14-2016 06:27 AM
Would be great if this tool was integrated in the Office 365 partner-site, so we can check the health of all our Office 365 customers in a single overview!
08-14-2016 12:28 PM
Well, I do miss the PowerShell bit - always good to know what exactly a tool is doing. In this regard I do think you should update the consent part, for example make sure that the publisher is listed as Microsoft, and provide a clear explanation why you need the type of permissions ("write directory data" can be a hard sell to some organizations as it can easily be taken out of context).
Other that that - it's cool. Obviously needs some more work, lots of UI glitches, lots of missing/inactive controls, but overall I like it. A "rescan" button might be handy? Also some sort of filter/template per industry or per security standard, so that people can easily check where they stand in terms of meeting compliance for their particular needs.
Will do a more detailed review/blog post in the next days and send some additional feedback your way. Lots has changed in the service since the last time I spammed email@example.com :)
08-15-2016 08:13 PM
I'd like to see alerting for score changes. If I do the work to improve security, and then another global admin undoes some of that work maliciously or through error, being notified of a score change would be useful. It would also be helpful to be notified of new items when they are added to the tool.
Just to clarify, the [Not Scored] items such as reviewing reports, is the intention to score them eventually? E.g. if I click through that item and review the report, does Secure Score see that and add points to the score?
Also will Secure Score facilitate the regular reviews? E.g. by emailing/notifying me when a review item is due for another review? Or will I need to self-maintain that via a calendar item or similar mechanism?
08-16-2016 10:20 AM
Thanks for the suggestion! Adding it to the list.
08-16-2016 10:27 AM
08-16-2016 06:33 PM
08-22-2016 11:58 AM
Thanks for the feedback. That is definitely our intention. API access is on the docket for the near future.
08-23-2016 02:06 AM
I have been impressed with Secure Score, I think it has a lot of potential, already it seems very useful. Incidentally, I have put some feedback in a post on my blog -
08-25-2016 05:40 AM
08-25-2016 10:40 AM
Sorry for the trouble. The most likely cause is that the acocunt you are using has not been assigned the global administrator role. The Secure Score requires that privilege level at the moment.
08-25-2016 10:43 AM
Thanks for the reply, however I'm using my account and I am a global administrator.
09-01-2016 02:02 PM
Was looking at my admins, and notice I have a Dirsync Admin, how do I setup the Dirysync admin with MFA?
I also have an admin my vendor setup as the initial admin, and said we did not need to use this but it had to be there MFA, is this true?
09-01-2016 11:10 PM
Newer versions of AADConnect support MFA, using a GA account with MFA enabled should not be a problem.
09-02-2016 12:21 PM
Vasil's got it right. I should also note that the Secure Score gives MFA credit to organizations that have MFA enabled in dirsync'ed on-prem directories, so long as the federated directory configuration includes the 'SupportsMFA' flag in AAD.
09-15-2016 04:35 AM
I assume that this will also eventually be at least linked from the Security & Compliance portal so everything is in one place?
09-15-2016 11:19 AM
Thanks for the feedback. It is our intention to integrate the experience into the Security and Compliance Center once we can prove that the concept is an effective way to drive improvements to customer risk postures. Please let your peers know about the Score, and let us know if you have any feedback about the experience.
09-16-2016 12:39 AM
Could I suggest integration with OMS, Security and Audit. Maybe also for alerting?
09-16-2016 09:53 AM
Thanks for the feedback! You are reading our minds. We are working on publishing the secure score data through API end points that will live in the same place as the Management Activity API endpoints (which provide activity and audit data), so that our third party ISV partners can /super/ easily include this data in their stores. OMS is one of those key partners.
09-21-2016 05:53 AM
when I try to access my demo tenant, I do not get any current scoring.
The box "Your Secure Score is:" is missing. Instead I see only a target score.
Do I need a valid subscription in Azure first to setup Scure Score?
09-22-2016 09:22 AM
Thanks for reaching out. I am able to confirm that your company does have results in the Secure Score database. Can you email firstname.lastname@example.org so we can connect offline and troubleshoot?
10-12-2016 07:44 AM
Interested to see Secure Score has been promoted this week via the Message center, also there have been further advancements -
"We have introduced the first fully remediated workflow under the control "Designate less than 5 tenant admins". Click "Learn more" after expanding the widget"
The Get Advice section that points to this community is going to the old address (https://network.office.com/t5/Security-Privacy-Compliance/bd-p/security_privacy), by the way, which generates a certificate mismatch.
10-13-2016 08:52 AM
I made some updates to improve my score.
When will I see my score updated?
10-15-2016 07:48 AM
My client is not using Exchange Online. It would be helpful if there was some way to exclude the actions related to exchange from the Secure Score recommendations
10-15-2016 12:38 PM
Thanks for reaching out. The Secure Score is calculated once per day (at 1am PST). Please note that not all of the controls are instrumented, so some actions might not reflect an improvement in your score yet (those controls are labeled [Not Scored]). Otherwise, you should see your score reflect your actions within a maximum of 24 hours!
10-15-2016 12:41 PM
10-18-2016 01:28 AM
@Brandon Koeller wrote:
... It is surprisingly hard to find the source data in the ecosystem, ...
Well, at last! Someone from Microsoft acknowledging this. Perhaps you could also raise the visibility of some of the audit issues - like missing data from the audit reports.
Also perhaps you could get someone to finally deal with the issue of trying to identify which users have not used the system recently (e.g. have not logged in in the last 90d). This appears to still be virtually impossible, especially when users are not using Exchange Online.
These issues are causing no end of problems.
I recently tried to identify people not using the system in order to recover licenses. I used the audit reports for the last 180d thinking that at the very least all active users must have changed their password in that time and that should have been audited. Needless to say that resulted in nearly 10% of identified users that were actively using the system.
10-18-2016 01:19 PM
10-19-2016 08:06 AM
It is nice tool and this is what we really need, we couldn't expect to teach everyone about security issue and threats and they just want to press one key to see what is going on and we are the one in background working to make sure that key is working fine.
10-20-2016 05:33 AM
Many thanks Brandon. I've been tracking these issues for some while but I've struggled to pin down actual evidence.
Having just revisited the issues that I'm having. I now have hard evidence from the get-msoluser and the combined audit log that something is very badly wrong. At least with our tenancy if not something wider.
Two definitive issues: One is that get-msoluser consistently reports some users with PasswordNeverExpires set to TRUE which should never happen.
The second is even more serious. I have found a user who is currently logged into the system but according to the Get-MsoUser data hasn't changed her password for 181 days (our tenant is set to require password change after 90d). Here is some relevant information:
BlockCredential : False
IsLicensed : True
LastPasswordChangeTimestamp : 2016-04-22 11:27:22
LicenseReconciliationNeeded : False
OverallProvisioningStatus : Success
PasswordNeverExpires : False
StrongPasswordRequired : True
StsRefreshTokensValidFrom : 2016-04-22 11:27:22
ValidationStatus : Healthy
WhenCreated : 2013-05-07 10:11:03
Checking the combined audit log I can see that it agrees that the user last changed their password on the 22nd April but they are still logging in. They should not have been able to log in after July 21st. However, the audit log has recorded 23 logins since then.
Previously, I'd been assuming that some data was missing from the audit logs but it appears that there may be a more serious issue.
10-20-2016 05:38 AM
10-20-2016 05:41 AM
No problem Dean.
I don't believe that it is isolated for us. It's just that this is the first time I've actually been able to prove it happening.
10-20-2016 05:52 AM
Also, thanks for the pointer to the Investigation Tooling Github. I've run the script to check for users not logged in in the last 90d but the first entry that it reports is one that I know is used daily because the person sits behind me in the office! They are a very heavy Office 365 user as they helped my set up our tenant.
10-20-2016 09:09 AM
Great service and it's really helped secure and understand Office 365!
One of the items counts the number of Exchange mailboxes with auditing enabled and the action is to enable auditing. The mailbox count and number of mailboxes with auditing enabled varies has not matched my mailbox count and their audit status. We're new to Office 365 so we may be overlooking a system database or some other setting. We're simply using get-mailbox and also with the -SoftDeletedMailbox and -Migration switches and check the AuditEnabled status.
10-20-2016 11:53 AM
I've raised a couple of issues in the github log. I think the reason it thought my colleague hadn't logged in is that it only returns 5k records. That's nowhere near enough for a 90d review of logins for 8k users. I'll update the issue with a new script when I've finished it or I can do a pull if you prefer, let me know in the issue (I am TotallyInformation on GitHub).
10-24-2016 09:19 AM
It turns out that the actual script needed to validate unused logins is significantly more complex than the example. Even reworking the example to deliver the maximum 50k entries is nowhere near enough to cope with the logins from an active 8k seat tenancy for 90d.
So I had to turn the script around so that the combined audit log is queried by user. Even then a few users exceeded 5k logins in 90d, one admin user had nearly 20k logins in that period.
The new example script is on GitHub here:
10-24-2016 09:21 AM - edited 10-24-2016 09:21 AM
Oh, and it takes around 10 hours to run!!
10-28-2016 02:26 AM
What are the roles other than the admin of Tenant who can access the Secure O365 Score function?
Thank you very much for your help and feedback.
10-28-2016 03:13 AM - edited 10-28-2016 03:16 AM
Currently, it is Global Admins only. I attended an O365 Deep Dive webinar a few days ago and we were told that more roles are planned. They are currently gathering feedback so that they can determine what those roles should be.
@Brandon Koeller was the presenter and he should be able to provide more details
10-28-2016 01:05 PM
11-04-2016 01:16 PM
I really appreciate your feedback and information you share.
Following your answer, i discuss with my customer ans i am waiting information from him about role he would like to Add to grant access for Secure Score experience.
As soon as i receive information, i share.
Thank you very much for your feedback.
11-10-2016 08:51 AM
12-15-2016 05:47 AMSolution
Another issue with Secure Score.
"You should require that all of your users reset their password at least every 60 days"
This is no longer current best practice where strong passphrases and 2FA are used since more rapid enforced change of passwords leads to the use of weaker ones.
12-15-2016 09:19 AM
12-15-2016 11:42 AM
That password recommendations document contains a lot of good info. Can you get it copied from the Research org over into some public places, such as docs.microsoft.com and support.office.com?