Performance issues on an extremely large table

%3CLINGO-SUB%20id%3D%22lingo-sub-1591749%22%20slang%3D%22en-US%22%3EPerformance%20issues%20on%20an%20extremely%20large%20table%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1591749%22%20slang%3D%22en-US%22%3E%3CP%3EA%20table%20in%20a%20database%20has%20a%20size%20of%20nearly%202%20TB.%20Even%20though%20the%20server%20is%20running%20with%208%20CPUs%2C%2080%20GB%20RAM%20and%20very%20fast%20Flash%20disks%2C%20performance%20is%20bad.%20Is%20this%20just%20what%20to%20expect%20with%20a%20table%20as%20big%20as%20this%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EFor%20some%20reasons%2C%20the%20server%20never%20been%20set%20up%20with%20any%20maintenance%20scripts.%20No%20Index%20defragmentation%2C%20no%20update%20statistics.%20But%20trying%20to%20run%20this%20takes%20ages%20and%20makes%20the%20server%20run%20even%20worse.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThis%20big%20table%20has%20some%20indexes%20setup.%20One%20for%20primary%20key%2C%20and%20when%20I%20run%20a%20script%20like%20this%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fbasitaalishan.com%2F2012%2F07%2F06%2Ffind-the-size-of-index-in-sql-server%2F%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fbasitaalishan.com%2F2012%2F07%2F06%2Ffind-the-size-of-index-in-sql-server%2F%3C%2FA%3E%20to%20tell%20me%20the%20size%20of%20the%20indexes%2C%20I%20am%20told%20that%20the%20PK-index%20has%20a%20size%20of%201.8%20TB.%20I%20really%20do%20not%20understand%20how%20this%20can%20be%20as%20the%20disk%20on%20the%20server%20is%20less%20than%204%20TB%20and%20also%20carries%20other%20databases.%20The%20index%20and%20the%20table%20itself%20will%20take%20up%20more%20than%20the%20disk%20available.%20So%20how%20to%20understand%20Index%20size%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EListing%20fragmentation%20on%20the%20index%20tells%20me%20that%20the%20PK%20index%20only%20has%20a%20fragmentation%20of%201%25%20-%20this%20is%20probably%20because%20data%20are%20never%20deleted.%20Then%20I%20assume%20that%20index%20optimization%20will%20not%20help.%20But%20how%20about%20update%20statistics.%20What%20does%20this%20do%2C%20and%20can%20this%20benefit%20performance%20in%20my%20case%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ERegards%2C%20Lars.%3C%2FP%3E%3C%2FLINGO-BODY%3E
New Contributor

A table in a database has a size of nearly 2 TB. Even though the server is running with 8 CPUs, 80 GB RAM and very fast Flash disks, performance is bad. Is this just what to expect with a table as big as this?

 

For some reasons, the server never been set up with any maintenance scripts. No Index defragmentation, no update statistics. But trying to run this takes ages and makes the server run even worse.

 

This big table has some indexes setup. One for primary key, and when I run a script like this https://basitaalishan.com/2012/07/06/find-the-size-of-index-in-sql-server/ to tell me the size of the indexes, I am told that the PK-index has a size of 1.8 TB. I really do not understand how this can be as the disk on the server is less than 4 TB and also carries other databases. The index and the table itself will take up more than the disk available. So how to understand Index size?

 

Listing fragmentation on the index tells me that the PK index only has a fragmentation of 1% - this is probably because data are never deleted. Then I assume that index optimization will not help. But how about update statistics. What does this do, and can this benefit performance in my case?

 

Regards, Lars.

1 Reply
I know this is an old post but I thought I'd necro it.

The "PK_Index" that they're talking about is very likely the Clustered Index. The Clustered Index is where the data for the "table" lives at and that explains the size.

1% fragmentation on that Clustered Index is nothing to be concerned about UNLESS the lead column of the index is of the UNIQUEIDENTIFIER type and it's not defaulted to NEWSEQUENTIALID().

If performance is bad on your large table, it's not because the table is large. It's because you don't have the correct indexes to support the necessary queries or it's because someone wrote some performance challenged code that needs to be fixed or a combination of both. The latter of those two is usually the case and can also include such nuances with datatype mismatches and "Non-SARGable" predicates in the JOINs or the WHERE Clauses.

You need to search the Microsoft documentation on what statistics do... they're uber important and teaching yourself through the MS documentation and some of the blog posts of heavy-hitters will do you a world of good Based on your questions (and no... I'm not being snarky here!) you need to learn a lot more about indexes, especially what a Clustered Index and a Non-Clustered Index actually is. It's the only way you and the Developers are going to learn how to tame and query that size table.