SQL Timestamp Incremental Load Issue
An incremental load is implemented based on timestamp, using a view to update a fact table. • The fact table contains multiple KPIs coming from different source tables. • Two main KPI tables are included directly in the view and drive the incremental logic (their timestamps change). • Additional KPIs come from other source tables, which are LEFT JOINed to the view. During an incremental run: • Main source tables are updated → timestamp changes • Other source tables are NOT updated → timestamp unchanged • Because of the LEFT JOIN, missing values from these tables are returned as NULL • These NULLs overwrite existing values in the fact table Example Fact table (before load) id app_number score 1 333 5 Source tables • source1, source2 → timestamps updated (drive incremental load) • source3 → timestamp unchanged Stored procedure logic MERGE fact_table tgt USING ( SELECT app_number, ISNULL(score, 0) AS score FROM vw_main_kpis v LEFT JOIN source3 s3 ON v.app_number = s3.app_number ) src ON tgt.app_number = src.app_number WHEN MATCHED THEN UPDATE SET tgt.score = src.score; Result (incorrect) id app_number score 1 333 0 Existing data is lost.SQL Server 2017 – CLR was loaded in an unsupported manner (All SSIS jobs failed)
Hi, We are facing a critical issue in our SQL Server 2017 instance. When trying to use a built-in CLR function or running SSIS-related jobs, we are getting the below error: The Common Language Runtime (CLR) was loaded in an unsupported manner. This can occur if an extended stored procedure or OLE Automation object running in SQL Server calls into managed code before the CLR integration runtime host loads the CLR. You need to restart SQL Server to use CLR integration features. Steps tried so far: Restarted SQL Server service Restarted the entire Windows Server Verified .NET Framework version (4.7.03062 installed) Confirmed CLR integration is enabled (sp_configure 'clr enabled', 1) All SSIS jobs are failing due to this issue. Any suggestions, please?
Pet project on SQL Server 2022 platform
Hello, world! I would like to share my pet project on SQL Server 2022 platform. I have created a DWH solution that includes many MS's best practices and interesting features such us: ETL process with data cleansing and MDM that easy expand Documentation CI/CD Functional ETL test Ready analytical templates Time intelligence New & returning customers Cluster customers based on spending volume Product ABC classification Basket analysis Events in progress https://dev.azure.com/zinykov/NorthwindBI Unfortunately in SQL Server 2025 will be no DQS & MDS...
Insert & Update both implemented wrong semantically
Insert means to introduce something, adding/entering something new Update means to change something, change and existing value. Semantically and logically speaking, Insert should only be used to enter data into an empty or partially empty record/table. Update should only be used to change existing values within a record/table. Using a 5 column table as an example where column 1 is an email address Example1 INSERT INTO table (Column1,Column2,Column3) VALUES (value1,value2,value3) Or Example2 INSERT INTO table VALUES (value1,value2,value3,value4,value5) With example 1, 2 columns are left empty Semantically, insert should be used to populate columns for the first time INSERT INTO table (column4, column5) VALUES (value4,value5) where column1 = "EmailAddress" Then for example, UPDATE table SET COLUMN2 = "newValue" WHERE COLUMN1 = "EmailAddress" Updating (setting) an empty column should cause an error, cannot update an empty column SQL, a great idea, pity its implementation is tainted by bad design. SQL works, but its design/implementation of Insert & Update is semantically wrong. IMO As the saying goes, if it isn't broken, don't fix it. Who cares about semantics or logic.SQL Server does not reduce the size of MDF and LDF
I am a SQL Server 2017 user, I deleted a table that had too many records, but after the delete process, the size of the MDF and LDF files did not decrease, but increased. I have shrunk the file but the file size remains the same. What should I do to change the file size?
Having difficulties to paste data to excel ?
Hi, I need to provide data in excel format. One of the field is having varchar - I am using CAST(AP.IDINVC AS VARCHAR) as [INVOICE_ID] But I am missing the leading zero's when I copy the data from MS SQL Server Result to excel file. No of rows is 147474. How to keep the leading zeros when copying the Invoice ID from MS SQL Server result to Excel.How can I optimize this query for better performance
Hi , I have this query and it is taking a long time. If there are more than 10k rows, it takes more than 5 minutes. Is there another way to speed up the process? SELECT ROW_NUMBER() OVER (ORDER BY CreationDate DESC) AS RowId, Id [Id], transactionsId [TransactionsId], amount [Amount], AccountId [AccountId], dbo.Account_FirstBalance(CreationDate, AccountId, 161, CompanyId) [FirstBalance] FROM p_Ledger WHERE CreationDate >= '2024-11-01' AND CreationDate <= '2025-02-11' AND CompanyId = 117 AND branchId = 161 ALTER FUNCTION [dbo].[Account_FirstBalance]( @TransactionsDate DATETIME, @AccountId BIGINT, @BranchId INT, @CompanyId BIGINT ) RETURNS FLOAT AS BEGIN DECLARE @credit FLOAT; SELECT @credit = SUM(CASE WHEN T.transactionStatusId = 1 THEN T.amount ELSE -T.amount END) FROM dbo.Transactions T WHERE T.Approval = 1 AND T.CompanyId = @CompanyId AND T.AccountsId = @AccountId AND T.IsDeleted = 0 AND T.transactionsDate < @TransactionsDate AND (@BranchId = 0 OR T.branchId = @BranchId); RETURN ROUND(COALESCE(@credit, 0), 2); END;one column refer to 2 columns
Hello there global table is dbo.Employee(EmployeeId int, sex char(10)) Another global table is dbo.GradeEmployee(PolicyEmployeeId int (FK of Employee-EmployeeId), FaultEmployeeId int (FK of Employee-EmployeeId), grade char(5) Another is my temp table called #TmpEmployee(EmployeeId int, employeeName char(50), quantity) GradeEmployee- PolicyEmployeeId is derived from Employee-EmployeeId GradeEmployee- FaultEmployeeId is also derived from Employee-EmployeeId Now, the task is I want to get the data from #TmpEmployee table, there I want to Display in the below format PolicyEmployeeId, FaultEmployeeId, employeeNameSum(quantity) Sum(quantity) is total quantity of a pair FaultEmployeeId, PolicyEmployeeId, note in #TmpEmployee table sometime the Quantity of FaultEmployeeId wont present; sometime the Quantity of PolicyEmployeeId wont present but, in the result it has to display. How to write the select query?
