Settings app takes so much time for loading

At first the settings page will open completely fine and work nice and smooth but after a couple of restarts it breaks and will then take ages to load and while it does it actually causes a slowdown of the system.