BUG: "Disable user" action missing after creation of a custom detection rule

In the security portal you can use advanced hunting to create a custom detection rule. In this rule you can define actions on the user entities that the query outputs. However what I noticed is is that the new "disable user" action from MDI is only visible the first time you create the detection rule. 

After it has been created this option disappears from the settings page when you go an edit the detection rule. The only option that's left on the edit page is the "Mark user as compromised" option.

I had to delete the rule completely and recreate it in order to see the option again.