Aug 12 2016
- last edited on
May 24 2021
Microsoft is pleased to announce the preview availability of a new security analytics service called the Office 365 Secure Score. The Secure Score is a security analytics tool that will help you understand what you have done to reduce the risk to your data in Office 365, and show you what you can do to further reduce that risk. We think of it as a credit score for security. Our approach to this experience was very simple. First, we created a full inventory of all the security configurations and behaviors that our customers can do to mitigate risks to their data in Office 365 (there are about 77 total things that we identified). Then, we evaluated the extent to which each of those controls mitigated a specific set of risks and awarded the control some points. More points means a more effective control for that risk. Lastly, we measure the extent to which your service has adopted the recommended controls, add up your points, and present it as a single score.
The core idea is that it is useful to rationalize and contextualize all of your cloud security configuration and behavioral options into one simple, analytical framework, and to make it very easy for you to take incremental action to improve your score over time. Rather than constructing a model with findings slotted into critical, moderate, or low severity, we wanted to give you a non-reactive way to evaluate your risk and make incremental changes over time that add up to a very effective risk mitigation plan.
The Office 365 Secure Score is a preview experience, so you may find issues, and you will note that not all of the controls are being measured. Please share any issues on the Office Network Group for Security. You can access the Secure Score at https://securescore.office.com.
The Secure Score does not express an absolute measure of how likely you are to get breached. It expresses the extent to which you have adopted controls which can offset the risk of being breached. No service can guarantee that you will not be breached, and the Secure Score should not be interpreted as a guarantee in any way.
Your Secure Score Summary
The first, most important piece of the Secure Score experience is the Score Summary. This panel gives you your current Secure Score, and the total number of points that are available to you, given your subscription level, the date that your score was measured, as well as a simple pie chart of your score. The denominator of your score is not intended to be a goal number to achieve. The full set of controls includes several that are very aggressive and will potentially have an adverse impact on your users’ productivity. Your goal should be to optimize your action to take every possible risk mitigating action while preserving your users’ productivity.
As mentioned, the Office 365 Secure Score is in a preview release. Over the coming months you will see us continue to add new controls, new measurements, and improvements to the remediation experiences. If you like what you see, please share with your network. If you see something we can improve, please share it with us on the Office Network Group for Security. We’re looking forward to seeing your scores go up, and making the Secure Score experience as useful, simple, and easy as it can be.
Sep 16 2016 09:53 AM
Thanks for the feedback! You are reading our minds. We are working on publishing the secure score data through API end points that will live in the same place as the Management Activity API endpoints (which provide activity and audit data), so that our third party ISV partners can /super/ easily include this data in their stores. OMS is one of those key partners.
Sep 21 2016 05:53 AM
when I try to access my demo tenant, I do not get any current scoring.
The box "Your Secure Score is:" is missing. Instead I see only a target score.
Do I need a valid subscription in Azure first to setup Scure Score?
Sep 22 2016 09:22 AM
Thanks for reaching out. I am able to confirm that your company does have results in the Secure Score database. Can you email firstname.lastname@example.org so we can connect offline and troubleshoot?
Oct 12 2016 07:44 AM
Interested to see Secure Score has been promoted this week via the Message center, also there have been further advancements -
"We have introduced the first fully remediated workflow under the control "Designate less than 5 tenant admins". Click "Learn more" after expanding the widget"
The Get Advice section that points to this community is going to the old address (https://network.office.com/t5/Security-Privacy-Compliance/bd-p/security_privacy), by the way, which generates a certificate mismatch.
Oct 13 2016 08:52 AM
I made some updates to improve my score.
When will I see my score updated?
Oct 15 2016 07:48 AM
My client is not using Exchange Online. It would be helpful if there was some way to exclude the actions related to exchange from the Secure Score recommendations
Oct 15 2016 12:38 PM
Thanks for reaching out. The Secure Score is calculated once per day (at 1am PST). Please note that not all of the controls are instrumented, so some actions might not reflect an improvement in your score yet (those controls are labeled [Not Scored]). Otherwise, you should see your score reflect your actions within a maximum of 24 hours!
Oct 15 2016 12:41 PM
Oct 18 2016 01:28 AM
@Brandon Koeller wrote:
... It is surprisingly hard to find the source data in the ecosystem, ...
Well, at last! Someone from Microsoft acknowledging this. Perhaps you could also raise the visibility of some of the audit issues - like missing data from the audit reports.
Also perhaps you could get someone to finally deal with the issue of trying to identify which users have not used the system recently (e.g. have not logged in in the last 90d). This appears to still be virtually impossible, especially when users are not using Exchange Online.
These issues are causing no end of problems.
I recently tried to identify people not using the system in order to recover licenses. I used the audit reports for the last 180d thinking that at the very least all active users must have changed their password in that time and that should have been audited. Needless to say that resulted in nearly 10% of identified users that were actively using the system.
Oct 18 2016 01:19 PM
Oct 19 2016 08:06 AM
It is nice tool and this is what we really need, we couldn't expect to teach everyone about security issue and threats and they just want to press one key to see what is going on and we are the one in background working to make sure that key is working fine.
Oct 20 2016 05:33 AM
Many thanks Brandon. I've been tracking these issues for some while but I've struggled to pin down actual evidence.
Having just revisited the issues that I'm having. I now have hard evidence from the get-msoluser and the combined audit log that something is very badly wrong. At least with our tenancy if not something wider.
Two definitive issues: One is that get-msoluser consistently reports some users with PasswordNeverExpires set to TRUE which should never happen.
The second is even more serious. I have found a user who is currently logged into the system but according to the Get-MsoUser data hasn't changed her password for 181 days (our tenant is set to require password change after 90d). Here is some relevant information:
BlockCredential : False
IsLicensed : True
LastPasswordChangeTimestamp : 2016-04-22 11:27:22
LicenseReconciliationNeeded : False
OverallProvisioningStatus : Success
PasswordNeverExpires : False
StrongPasswordRequired : True
StsRefreshTokensValidFrom : 2016-04-22 11:27:22
ValidationStatus : Healthy
WhenCreated : 2013-05-07 10:11:03
Checking the combined audit log I can see that it agrees that the user last changed their password on the 22nd April but they are still logging in. They should not have been able to log in after July 21st. However, the audit log has recorded 23 logins since then.
Previously, I'd been assuming that some data was missing from the audit logs but it appears that there may be a more serious issue.
Oct 20 2016 05:38 AM
Oct 20 2016 05:41 AM
No problem Dean.
I don't believe that it is isolated for us. It's just that this is the first time I've actually been able to prove it happening.
Oct 20 2016 05:52 AM
Also, thanks for the pointer to the Investigation Tooling Github. I've run the script to check for users not logged in in the last 90d but the first entry that it reports is one that I know is used daily because the person sits behind me in the office! They are a very heavy Office 365 user as they helped my set up our tenant.
Oct 20 2016 09:09 AM
Great service and it's really helped secure and understand Office 365!
One of the items counts the number of Exchange mailboxes with auditing enabled and the action is to enable auditing. The mailbox count and number of mailboxes with auditing enabled varies has not matched my mailbox count and their audit status. We're new to Office 365 so we may be overlooking a system database or some other setting. We're simply using get-mailbox and also with the -SoftDeletedMailbox and -Migration switches and check the AuditEnabled status.
Oct 20 2016 11:53 AM
I've raised a couple of issues in the github log. I think the reason it thought my colleague hadn't logged in is that it only returns 5k records. That's nowhere near enough for a 90d review of logins for 8k users. I'll update the issue with a new script when I've finished it or I can do a pull if you prefer, let me know in the issue (I am TotallyInformation on GitHub).
Oct 24 2016 09:19 AM
It turns out that the actual script needed to validate unused logins is significantly more complex than the example. Even reworking the example to deliver the maximum 50k entries is nowhere near enough to cope with the logins from an active 8k seat tenancy for 90d.
So I had to turn the script around so that the combined audit log is queried by user. Even then a few users exceeded 5k logins in 90d, one admin user had nearly 20k logins in that period.
The new example script is on GitHub here:
Oct 24 2016 09:21 AM - edited Oct 24 2016 09:21 AM
Oh, and it takes around 10 hours to run!!