Secure Score not Scoring....

[James Oxton]

Hi there,

I've been using Secure Score for a few months now to test the security baseline and see how customers can get the most out of it.  However every Tuesday I do the weekly reports which for the last two weeks are not showing up on my score.  You can see at the beginging of last month that it works and all the weekly reports are showing, however a few weeks ago even though I keep spending some 45 mins doing the weekly reports I dont see any score being registered.

Is there something I'm missing, I'm going directly to the report am I surposed to only go to the report via the Secure Score webpage?  Any help would be most welcome, I know that it can take upto 48 hours for things to show up but for it to never show up for weeks on end something must be wrong?

Thanks

James

[#2]

Anthony, what is the difference between Review Non-Global Administrators and Review Role Changes?  They both point to the same place.  You are reviewing the same data from what I can tell...

[#3]

Hi Zeff,

Thanks for bringing this to my attention.  You have a good point.

The spirit of the "Review role changes weekly" report was to have you see if someone was made a global admin, while the "Review non-global administrators weekly" report focuses on roles like billing admin, user admin, etc.  The "Review role changes weekly report" does not correctly communicate the spirit.  In talking with the engineering team they are going to make a change to merge the two actions into one and update the text to say that you need review changes to your global and non-global admin roles weekly.  No ETA yet on when that will get updated.

Thanks again for bringing this to my attention!

[#4]

75 days later since we opened our ticket, here is our answer as to why reviewing reports as a compliance administrator, security reader, or as any other role besides global admin fails to give credit and increase the secure score:

"The issue is this: in order to get points for these controls, the user reviewing these reports must be a Global Administrator. A user set as a Compliance Administrator isn't able to raise their securescore by reviewing reports, it must be a Global Admin.”

One of the securescore recommendation is to designate five or fewer global admins. Now we have to designate one of those five just to look at reports and increase our score? What about separation of duties, and principle of least privilege? I, as a security professional, do not want access to functions I will never use (but a hacker certainly would, should my account become compromised) just so that I can read reports and increase our score. The actual global admin doesn't want to spend her day reading security reports - that is my job. She wants to focus on turning on the new features that secure score recommends be implemented in our tenant.

[#5]

@Christopher Borders- I feel like a standing ovation is in order for your statement you just made! As an IT Security, Risk and Compliance professional I whole heatedly agree with your entire statement. O365 available roles are not being appropriately allocated to match the real world roles that actually align with IT compliance guidelines that must be kept. One also must be a global admin in O365 to access, read and use items from ATP and TI which is not the role of a Global Amin which is why they are listed under the Security and Compliance area of O365.

[#6]

I am having exactly the same problem, I have enabled data loss prevention polies.  They have been applied for months.  I have tried turning them on and off to kickstart the points however It just isn't being picked up at the moment.  Someone please help me?

[#7]

same here, we have been reviewing sign-ins after multiple failure report weekly, however, we haven't got any score on the secure score.

[#8]

I am having the same issues, MS security scorecard not updating after 72 hours. Please can you help.

[#9]

Hi James and Juan,

I have alerted the development team that the score has not updated recently.  For the reports issue please make sure that you are accessing the report from Secure Score by pressing the "review" button.  If you are not using the Secure Score user interface and going directly to the report, no points will be provided as the underlying report does not have any telemetry on if you viewed it.

If you are using the review button, please use the feedback link in the bottom right of any Secure Score page to tell us.  This way we can get your tenant information which helps us in our investigation.

[#10]

Having the same issue, have also hit the review button. Using a GA account to perform all tasks but does nothing for the score.

[#11]

Same here. Same as all the rest. Score doesn't seem to change after doing as suggested.

I am a little confused about the "Enforce MFA for All Users" option. I had assumed that if each user opted to set up MFA on their own, this would show as enabled? It didn't.
I think it is the way I read the word "enforced". I was afraid that some people might not understand the necessity for it. I opted to tell them all to use the link provided. If everyone is now using MFA then it shouldn't be a problem for me to "Enforce" their use?
I don't want to surprise everyone with a "new request" for input of additional phone numbers.

[#12]

I can't comment on the reporting, since I haven't really looked.  But you are correct that the issue with MFA in your environment is brought about by the word "Enforce".  Secure Score doesn't check every user in the environment to see that they are using MFA.  In a lot of environments that would require looking at thousands or tens of thousands of users.  That would be a huge performance drain.  All it checks is whether you have turned on the setting that requires all users to use MFA.  So even if all the users are using MFA, but you aren't requiring them to do it in settings then MFA isn't Enforced.

[#13]

Hi Anthony, 

it has been a long time now since this threat started. I have seen changes happening where a score was given and then suddenly weeks/months later it was removed, without changes, no real hint on what caused the loss of the score.

In one of my tenants I see for example "Turn on audit data recording [Not Scored]" with 15/15 points, but some other items with [Not Scored], that were completed, are really not scored, getting 0 points. This is confusing.

I am responsible for multiple Office 365 tenants (over 30) and I can see changes and differences from one to the other with respect to Secure Score.

Will this Secure Score system be "fixed" in the near future? I would really like to be able to rely on the score. And, if changes are applied by Microsoft that break a previously good working security configuration, then it would be very helpful to receive/see details about it, so that it can be put back in place quickly. It would be very helpful to be able to apply a security configuration, really get the score and then be able to rely on it.

@Anthony Smith (A.J.) 

[#14]

Hi @geekworld, recently there was at least one improvement action where the data we needed for scoring was no longer going to be accessible and thus we were forced to switch the item to Not Scored. The story behind this is long and complex but we hope to bring it back to a "Scored" state in the future. We apologize for the inconvenience and will work to try an ensure this type of issue doesn't reoccur in the future. 

Regarding the  the following comment I want to make sure I'm interpreting it correctly and not making any incorrect assumptions:

"In one of my tenants I see for example "Turn on audit data recording [Not Scored]" with 15/15 points, but some other items with [Not Scored], that were completed, are really not scored, getting 0 points."

For "Turn on audit data recording [Not Scored]" with 15/15 point" it sound like you're saying the item is now automatically getting scoring data and that we need to remove the "[Not Scored]" text from the title. Correct?

For "some other items with [Not Scored], that were completed, are really not scored, getting 0 points." it sounds like you are saying you used the Resolved through third-party" option and you didn't get the points added to your score. Correct?

Thanks,

Chris Hallum

[#15]

Would it be possible to add a filter to filter out the scores that are not scored?

[#16]

Hi @Chris Hallum ,

thank you for your response.

I confirmed with 4 different tenants that "Turn on audit data recording" is scored with 15/15. So, it seems that you can remove the [Not Scored] notation. 

Overall, it seems that more than one improvement action was demoted to [Not Scored] from being scored before.

There is a cool feature, when I click on a completed secure score item that is a "Review" task, it tells me "Action completed by name on date". This is very helpful with "Review" Improvement Actions, like "Review mailbox access by non-owners bi-weekly", etc. But, it seems to have accuracy issues. In one tenant I noticed that "Review malware detections report weekly" was completed on Jun 1, 2019, which is 9 days ago, yet the scoring system gives a 5/5 score for this, which should have been put to 0/5 at least 2 days ago. With other tenants this appears to work fine and the score goes down to 0/5 after the required review period, but apparently not with all of them.

Additionally, getting a completed date for every other completed and scored improvement action would be very helpful.

Such "Review" items have a Review button, but instead of always going to the final destination of the review item, they may launch into a dashboard, from where one has to figure out where to go for the actual review. Linking the Review button to the final review destination would be very helpful.

I never use the "Resolved through third party" or "Ignore" for any of the tenants I manage.

Also, I want to mention that in at least one tenant, for a limited time several months ago, there were buttons to actually apply/implement certain improvement actions automatically. Unfortunately, this great feature was quietly removed at some point in time. I would love to get this back.

Thanks,

Ralph

[#17]

Secure score never update. i think we should not relay on this