Forum Discussion
Technical_Architect5
Jan 31, 2025Copper Contributor
Store Procedure to capture changes to tables/column
I need help to develop a stored procedure that can capture changes to all tables/columns in a CDC-enabled SQL database (all the tables are also CDC-enabled). The database is on SQL 2016. Store these operational changes (INSERTS, UPDATES, DELETES) into another table. The procedure should also capture old value, new value, and the date/time the changes took place.
- Technical_Architect5Copper Contributor
Thanks. Well appreciated.
- olafhelperBronze Contributor
I need help to develop a stored procedure that can capture changes to all tables/columns
And now you are expecting a "universal-I-can-do-anything" solution without giving any usefull informations about your database/tables??
Impossible!!!
You have to develop an own solution depending on your database/tables design.
- Mks_1973Iron Contributor
Enable CDC on the Database and Tables
If your database is not yet CDC–enabled, run the following on your database (replace YourDatabase with your name):
USE YourDatabase;
GO
EXEC sys.sp_cdc_enable_db;
GO
-------------------
Then, for each table that you wish to capture, run (adjusting the schema and table name):
EXEC sys.sp_cdc_enable_table
@source_schema = N'dbo',
@source_name = N'YourTable',
@role_name = NULL; -- or specify a role if needed
GO
*Repeat for all tables you want audited.
------------------------------
Create an Audit TableSee the below example, we create a table named AuditLog that will store:
The source table (including schema)
The type of operation (INSERT, UPDATE, DELETE)
The date/time the change was captured
The “old” values (for deletes and update–before images)
The “new” values (for inserts and update–after images)
CREATE TABLE dbo.AuditLog (
AuditLogID INT IDENTITY(1,1) PRIMARY KEY,
SourceTable NVARCHAR(128),
Operation NVARCHAR(10),
ChangedDateTime DATETIME2,
OldValues XML NULL,
NewValues XML NULL
);
GO
------------------------------------------
Create the Stored Procedure to Capture CDC Changes
The following stored procedure (named dbo.CaptureAllCDCChanges) accepts two parameters: a starting LSN (@from_lsn) and an ending LSN (@to_lsn). (Typically you would record the last processed LSN after each run and use that as the next run’s starting LSN.) The procedure then uses a cursor to loop over all CDC–enabled tables (information available from the system table cdc.change_tables). For each table it builds and executes a dynamic SQL statement that calls the appropriate CDC function (for example, if the capture instance is named YourTable, then the function is called cdc.fn_cdc_get_all_changes_YourTable).CREATE PROCEDURE dbo.CaptureAllCDCChanges
@from_lsn varbinary(10),
to_lsn varbinary(10)
AS
BEGIN
SET NOCOUNT ON;-- Declare variables to hold the schema name, table name, and capture instance.
DECLARE @schema_name sysname,
@table_name sysname,
@capture_instance sysname,
@SQL NVARCHAR(MAX);-- Get a list of all CDC-enabled tables from the CDC system table.
DECLARE cdc_cursor CURSOR FAST_FORWARD FOR
SELECT sch.name AS SchemaName,
tab.name AS TableName,
ct.capture_instance
FROM cdc.change_tables ct
INNER JOIN sys.tables tab ON ct.[object_id] = tab.[object_id]
INNER JOIN sys.schemas sch ON tab.[schema_id] = sch.[schema_id];OPEN cdc_cursor;
FETCH NEXT FROM cdc_cursor INTO @schema_name, @table_name, @capture_instance;
WHILE @@FETCH_STATUS = 0
BEGIN
--
-- Build a dynamic SQL statement that:
-- 1. Calls the CDC function for this table/capture instance for changes between @from_lsn and to_lsn.
-- 2. Inserts the results into dbo.AuditLog.
--
-- For CDC, the __$operation column returns:
-- 1 = DELETE (the row’s “before” image)
-- 2 = INSERT (the row’s “after” image)
-- 3 = UPDATE (the row’s “before” image)
-- 4 = UPDATE (the row’s “after” image)
--
-- In our simple example, we log:
-- - For INSERT: NewValues (the full inserted row)
-- - For DELETE: OldValues (the full deleted row)
-- - For UPDATE: We log one row for the before image (operation=3) and one for the after image (operation=4)
-- and mark both as 'UPDATE'. (A more advanced solution would pair these rows.)
--
SET @SQL = N'
INSERT INTO dbo.AuditLog (SourceTable, Operation, ChangedDateTime, OldValues, NewValues)
SELECT
''' + @schema_name + '.' + @table_name + ''',
CASE
WHEN __$operation = 1 THEN ''DELETE''
WHEN __$operation = 2 THEN ''INSERT''
WHEN __$operation IN (3,4) THEN ''UPDATE''
ELSE ''UNKNOWN''
END AS Operation,
GETDATE() AS ChangedDateTime,
CASE
WHEN __$operation = 3 THEN (SELECT t.* FOR XML AUTO, ELEMENTS)
WHEN __$operation = 1 THEN (SELECT t.* FOR XML AUTO, ELEMENTS)
ELSE NULL
END AS OldValues,
CASE
WHEN __$operation = 2 THEN (SELECT t.* FOR XML AUTO, ELEMENTS)
WHEN __$operation = 4 THEN (SELECT t.* FOR XML AUTO, ELEMENTS)
ELSE NULL
END AS NewValues
FROM cdc.fn_cdc_get_all_changes_' + QUOTENAME(@capture_instance) +
'(@from_lsn, to_lsn, ''all'') AS t;
';-- Execute the dynamic SQL with parameter substitution.
EXEC sp_executesql
@SQL,
N'@from_lsn varbinary(10), to_lsn varbinary(10)',
@from_lsn = @from_lsn,
to_lsn = to_lsn;FETCH NEXT FROM cdc_cursor INTO @schema_name, @table_name, @capture_instance;
END;CLOSE cdc_cursor;
DEALLOCATE cdc_cursor;
END;
GO
NOTE: (LSN parameters) The CDC functions expect a “from LSN” and “to LSN” of type varbinary(10). In production you would record (in a separate table or variable) the last LSN processed so that each run only captures new changes. For example, you might call
DECLARE @startLSN varbinary(10) = (SELECT LastLSN FROM dbo.CDCStatus);
DECLARE @endLSN varbinary(10) = sys.fn_cdc_get_max_lsn();
EXEC dbo.CaptureAllCDCChanges @from_lsn = @startLSN, to_lsn = @endLSN;
-- Then update dbo.CDCStatus with @endLSN.
-----------------------------
CDC records updates as two rows (one with operation = 3 [before image] and one with operation = 4 [after image]).
In the above sample we simply log each CDC row as a separate audit row. If you need to combine (pair) the before and after images for an update, you would need to write additional logic (for example, using a common table expression or temporary table keyed on the CDC sequence value) so that each update appears as a single record with both old and new values.
Running the Procedure
You would call the procedure by supplying the appropriate LSN range.DECLARE @from_lsn varbinary(10) = sys.fn_cdc_get_min_lsn('YourCaptureInstance');
DECLARE to_lsn varbinary(10) = sys.fn_cdc_get_max_lsn();EXEC dbo.CaptureAllCDCChanges @from_lsn, to_lsn;
In a production scenario you would:
Store the “last processed” LSN (for example, in a table such as dbo.CDCStatus).
Use that LSN as the starting point for the next run.
Schedule the procedure (for example, via SQL Server Agent) to run periodically.
(This solution provides a generic framework that you can further refine (for example, to log only the changed columns) to meet your requirements.)- Technical_Architect5Copper Contributor
Thanks Mks_1973. The framework you provided has given me a good starting point.
- Massimiliano_BuschiCopper Contributor
Here's a stored procedure example to capture changes in all tables/columns in a CDC-enabled SQL database and store those changes into another table.
CREATE PROCEDURE CaptureChanges
AS
BEGIN
DECLARE @tableName NVARCHAR(256);
DECLARE @schemaName NVARCHAR(256);
DECLARE @cdcQuery NVARCHAR(MAX);
DECLARE @changeTable NVARCHAR(256);DECLARE table_cursor CURSOR FOR
SELECT schema_name(schema_id) AS schemaName, name AS tableName
FROM sys.tables
WHERE is_tracked_by_cdc = 1;OPEN table_cursor;
FETCH NEXT FROM table_cursor INTO @schemaName, @tableName;WHILE @@FETCH_STATUS = 0
BEGIN
SET @cdcQuery = '
SELECT
''' + @schemaName + '.' + @tableName + ''' AS TableName,
CASE
WHEN __$operation = 2 THEN ''INSERT''
WHEN __$operation = 3 THEN ''DELETE''
WHEN __$operation = 4 THEN ''UPDATE''
END AS Operation,
__$start_lsn AS LSN,
__$seqval AS SeqVal,
__$update_mask AS UpdateMask,
' + @tableName + '.*,
CURRENT_TIMESTAMP AS ChangeDate
FROM cdc.fn_cdc_get_all_changes_' + @tableName + '(sys.fn_cdc_get_min_lsn(''' + @schemaName + '_' + @tableName + '''), sys.fn_cdc_get_max_lsn(), ''all'')';SET @changeTable = @schemaName + '_' + @tableName + '_Changes';
IF NOT EXISTS (SELECT * FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = @schemaName AND TABLE_NAME = @changeTable)
BEGIN
EXEC('CREATE TABLE ' + @changeTable + ' (
ChangeID INT IDENTITY(1,1) PRIMARY KEY,
TableName NVARCHAR(256),
Operation NVARCHAR(10),
LSN VARBINARY(10),
SeqVal VARBINARY(10),
UpdateMask VARBINARY(10),
ChangeDate DATETIME,
' + @tableName + '.*
)');
ENDEXEC('INSERT INTO ' + @changeTable + ' ' + @cdcQuery);
FETCH NEXT FROM table_cursor INTO @schemaName, @tableName;
ENDCLOSE table_cursor;
DEALLOCATE table_cursor;
END;This stored procedure does the following:
- It loops through all the CDC-enabled tables.
- Constructs a query to get all changes for each table.
- Checks if a change table exists for each table, and creates it if it doesn't.
- Inserts the changes into the respective change table.
You'll need to adapt this procedure based on your specific requirements and the structure of your tables.