ATA Services not starting

Hi,

We have had Microsoft ATA v1.7 running for around a year now, but recently the services have stopped and will not start. I also noticed that an optional update has been installed to upgrade to v1.8.

The service and windows logs state "The Microsoft Advanced Threat Analytics Center service terminated unexpectedly. It has done this 15655 time(s). The following corrective action will be taken in 5000 milliseconds: Restart the service." - which is obviously not helpful at all.

Within the ATA Errors logs, i am receiving the following:

"2018-01-11 15:41:26.9913 1552 6 00000000-0000-0000-0000-000000000000 Error [CenterConfigurationManager+<GetConfigurationAsync>d__7] System.NullReferenceException: Object reference not set to an instance of an object.
at async Microsoft.Tri.Center.Service.CenterConfigurationManager.GetConfigurationAsync(?)
at async Microsoft.Tri.Infrastructure.Framework.ConfigurationManager`2.UpdateConfigurationAsync[](?)
at async Microsoft.Tri.Infrastructure.Framework.ConfigurationManager`2.OnInitializeAsync[](?)
at async Microsoft.Tri.Center.Service.CenterConfigurationManager.OnInitializeAsync(?)
at async Microsoft.Tri.Infrastructure.Framework.Module.InitializeAsync(?)
at async Microsoft.Tri.Infrastructure.Framework.ModuleManager.OnInitializeAsync(?)
at async Microsoft.Tri.Infrastructure.Framework.Module.InitializeAsync(?)
at async Microsoft.Tri.Infrastructure.Framework.Service.OnStartAsync(?)
at Microsoft.Tri.Infrastructure.Framework.Service.OnStart(String[] args)"

I have tried to roll the ATA config back to a known state, which hasn't worked, i also thought that the optional update may have corrupted it, but i cannot see a way of rolling the update back.

I wonder if i uninstall the ATA Centre on the server, will it uninstall Mongo and lose all of the data? I don't really want to do this i know i will have wait for ATA to learn all of the patterns etc again, which isn't an option for our Security Team 🙂

Any ideas would be much appreciated.

TIA

EliOfek
Jan 15, 2018
Sorry to say but this confirmed you are a victim of a mongo bug that causes a DB wipe...

(Fixed for vNext, as we embed a new version of mongo which should have a fix for it)

Please follow this procedure for Center recovery:

https://docs.microsoft.com/en-us/advanced-threat-analytics/disaster-recovery

Given that you have a backup of the json file as described in the article, you won't have to reinstall the Gateways, and you can be back up an running in a few minutes.

6 Replies

EliOfek

Microsoft

Jan 12, 2018

Please run on the center machine from mongo's bin folder:

Mongo.exe ATA --eval "var collectionNames = db.getCollectionNames(), indexes = [];collectionNames.forEach(function (name) {printjson(name);printjson(db[name].getIndexes());print('-------------------------------------');});" > indexes.txt

And paste the output int he text file here.

Martin Kerr

Copper Contributor

Jan 15, 2018

Hi,

Thanks for your reply, please see output below:

MongoDB shell version v3.4.2
connecting to: mongodb://127.0.0.1:27017/ATA
MongoDB server version: 3.4.2
"DirectoryServicesActivity"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.DirectoryServicesActivity"
	}
]
-------------------------------------
"Dns_20171011061153"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.Dns_20171011061153"
	}
]
-------------------------------------
"GroupMembershipChangeEvent_20171011144300"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.GroupMembershipChangeEvent_20171011144300"
	}
]
-------------------------------------
"KerberosAp_20171011013137"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.KerberosAp_20171011013137"
	}
]
-------------------------------------
"KerberosAs_20160722202708"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.KerberosAs_20160722202708"
	}
]
-------------------------------------
"KerberosTgs_20160722202708"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.KerberosTgs_20160722202708"
	}
]
-------------------------------------
"LsaRpc_20171011072820"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.LsaRpc_20171011072820"
	}
]
-------------------------------------
"MonitoringAlert"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.MonitoringAlert"
	}
]
-------------------------------------
"Notification"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.Notification"
	}
]
-------------------------------------
"NtlmEvent_20160722202706"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.NtlmEvent_20160722202706"
	}
]
-------------------------------------
"Ntlm_20160722202710"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.Ntlm_20160722202710"
	}
]
-------------------------------------
"Samr_20171011052414"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.Samr_20171011052414"
	}
]
-------------------------------------
"ServiceControl_20171011012750"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.ServiceControl_20171011012750"
	}
]
-------------------------------------
"SuspiciousActivity"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.SuspiciousActivity"
	}
]
-------------------------------------
"SuspiciousActivityActivity"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.SuspiciousActivityActivity"
	}
]
-------------------------------------
"SystemProfile"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.SystemProfile"
	}
]
-------------------------------------
"Telemetry"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.Telemetry"
	}
]
-------------------------------------
"UniqueEntity"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.UniqueEntity"
	}
]
-------------------------------------
"UniqueEntityProfile"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.UniqueEntityProfile"
	}
]
-------------------------------------
"UserPhoto"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.UserPhoto"
	}
]
-------------------------------------
"Wmi_20171011061616"
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "ATA.Wmi_20171011061616"
	}
]
-------------------------------------

Thanks

EliOfek
Microsoft
Jan 15, 2018
Sorry to say but this confirmed you are a victim of a mongo bug that causes a DB wipe...

(Fixed for vNext, as we embed a new version of mongo which should have a fix for it)

Please follow this procedure for Center recovery:

https://docs.microsoft.com/en-us/advanced-threat-analytics/disaster-recovery

Given that you have a backup of the json file as described in the article, you won't have to reinstall the Gateways, and you can be back up an running in a few minutes.

Forum Discussion

ATA Services not starting

6 Replies

Resources