Forum Discussion
SQL Timestamp Incremental Load Issue
Your incremental load fires because one table changed, but the other tables you LEFT JOIN to didn’t. Since they didn’t change, they return NULL, and your MERGE/update blindly writes those NULLs over perfectly good existing values. Nothing’s wrong with timestamps or SQL here it’s the load logic. You need to either not update a column when the source value is NULL, or stop driving the whole load off a single timestamp when multiple sources are involved. Been there, it’s an easy trap to fall into.
Thanks for your feedback. I attempted to retrieve data only when the source value is not null; however, this approach delays the load of the main KPIs. I also considered separating the updates by source, with a dedicated MERGE statement per source, but I don’t believe this is the most efficient solution.