Microsoft Secure Tech Accelerator
Apr 03 2024, 07:00 AM - 11:00 AM (PDT)
Microsoft Tech Community
SOLVED

ATA Services not starting

Copper Contributor

Hi,

 

We have had Microsoft ATA v1.7 running for around a year now, but recently the services have stopped and will not start. I also noticed that an optional update has been installed to upgrade to v1.8.

 

The service and windows logs state "The Microsoft Advanced Threat Analytics Center service terminated unexpectedly.  It has done this 15655 time(s).  The following corrective action will be taken in 5000 milliseconds: Restart the service." - which is obviously not helpful at all.

 

Within the ATA Errors logs, i am receiving the following:

 

"2018-01-11 15:41:26.9913 1552 6 00000000-0000-0000-0000-000000000000 Error [CenterConfigurationManager+<GetConfigurationAsync>d__7] System.NullReferenceException: Object reference not set to an instance of an object.
at async Microsoft.Tri.Center.Service.CenterConfigurationManager.GetConfigurationAsync(?)
at async Microsoft.Tri.Infrastructure.Framework.ConfigurationManager`2.UpdateConfigurationAsync[](?)
at async Microsoft.Tri.Infrastructure.Framework.ConfigurationManager`2.OnInitializeAsync[](?)
at async Microsoft.Tri.Center.Service.CenterConfigurationManager.OnInitializeAsync(?)
at async Microsoft.Tri.Infrastructure.Framework.Module.InitializeAsync(?)
at async Microsoft.Tri.Infrastructure.Framework.ModuleManager.OnInitializeAsync(?)
at async Microsoft.Tri.Infrastructure.Framework.Module.InitializeAsync(?)
at async Microsoft.Tri.Infrastructure.Framework.Service.OnStartAsync(?)
at Microsoft.Tri.Infrastructure.Framework.Service.OnStart(String[] args)"

 

I have tried to roll the ATA config back to a known state, which hasn't worked, i also thought that the optional update may have corrupted it, but i cannot see a way of rolling the update back.

 

I wonder if i uninstall the ATA Centre on the server, will it uninstall Mongo and lose all of the data? I don't really want to do this i know i will have wait for ATA to learn all of the patterns etc again, which isn't an option for our Security Team :) 

 

Any ideas would be much appreciated.

 

TIA

6 Replies

Please run on the center machine from mongo's bin folder:

Mongo.exe ATA --eval "var collectionNames = db.getCollectionNames(), indexes = [];collectionNames.forEach(function (name) {printjson(name);printjson(db[name].getIndexes());print('-------------------------------------');});" > indexes.txt

And paste the output int he text file here.

Hi,

 

Thanks for your reply, please see output below:

 

MongoDB shell version v3.4.2
connecting to: mongodb://127.0.0.1:27017/ATA
MongoDB server version: 3.4.2
"DirectoryServicesActivity"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.DirectoryServicesActivity"
	}
]
-------------------------------------
"Dns_20171011061153"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.Dns_20171011061153"
	}
]
-------------------------------------
"GroupMembershipChangeEvent_20171011144300"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.GroupMembershipChangeEvent_20171011144300"
	}
]
-------------------------------------
"KerberosAp_20171011013137"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.KerberosAp_20171011013137"
	}
]
-------------------------------------
"KerberosAs_20160722202708"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.KerberosAs_20160722202708"
	}
]
-------------------------------------
"KerberosTgs_20160722202708"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.KerberosTgs_20160722202708"
	}
]
-------------------------------------
"LsaRpc_20171011072820"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.LsaRpc_20171011072820"
	}
]
-------------------------------------
"MonitoringAlert"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.MonitoringAlert"
	}
]
-------------------------------------
"Notification"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.Notification"
	}
]
-------------------------------------
"NtlmEvent_20160722202706"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.NtlmEvent_20160722202706"
	}
]
-------------------------------------
"Ntlm_20160722202710"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.Ntlm_20160722202710"
	}
]
-------------------------------------
"Samr_20171011052414"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.Samr_20171011052414"
	}
]
-------------------------------------
"ServiceControl_20171011012750"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.ServiceControl_20171011012750"
	}
]
-------------------------------------
"SuspiciousActivity"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.SuspiciousActivity"
	}
]
-------------------------------------
"SuspiciousActivityActivity"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.SuspiciousActivityActivity"
	}
]
-------------------------------------
"SystemProfile"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.SystemProfile"
	}
]
-------------------------------------
"Telemetry"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.Telemetry"
	}
]
-------------------------------------
"UniqueEntity"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.UniqueEntity"
	}
]
-------------------------------------
"UniqueEntityProfile"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.UniqueEntityProfile"
	}
]
-------------------------------------
"UserPhoto"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.UserPhoto"
	}
]
-------------------------------------
"Wmi_20171011061616"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.Wmi_20171011061616"
	}
]
-------------------------------------

Thanks

best response confirmed by Martin Kerr (Copper Contributor)
Solution

Sorry to say but this confirmed you are a victim of a mongo bug that causes a DB wipe...

(Fixed for vNext, as we embed a new version of mongo which should have a fix for it)

Please follow this procedure for Center recovery:

https://docs.microsoft.com/en-us/advanced-threat-analytics/disaster-recovery

 

Given that you have a backup of the json file as described in the article, you won't have to reinstall the Gateways, and you can be back up an running in a few minutes.

Hi,

 

Looks like I might have the same issue.  What specifically in the output did you see that confirmed the bug?

 

Thanks

Martyn

All the collections have an index only for the id field and nothing else. most collections should have more indexes.

Thanks for the quick reply.  Looks like its DR for me too then :(

 

Martyn

1 best response

Accepted Solutions
best response confirmed by Martin Kerr (Copper Contributor)
Solution

Sorry to say but this confirmed you are a victim of a mongo bug that causes a DB wipe...

(Fixed for vNext, as we embed a new version of mongo which should have a fix for it)

Please follow this procedure for Center recovery:

https://docs.microsoft.com/en-us/advanced-threat-analytics/disaster-recovery

 

Given that you have a backup of the json file as described in the article, you won't have to reinstall the Gateways, and you can be back up an running in a few minutes.

View solution in original post