Recent Discussions
Best Practices for Connecting Internal SQL Server Financial Systems to Online Payment Platforms
I currently have an internal enterprise system used for purchasing, payments, and finance operations. The system runs on an on-premises database using Microsoft SQL Server and stores all financial transactions and internal workflow data. We now have a new requirement to enable online payment services for customers. These services will be exposed externally (likely in the cloud) and must interact with the same financial system so that transactions are reflected in our internal database. My main concerns are related to architecture, security, and data synchronization. Key points about the current setup: The core system and database are hosted internally (on-premises). The database contains sensitive finance and payment data. Internal processes depend on the current database structure and workflows. The new requirements: Develop an online payment service accessible over the internet. Ensure transactions from the online service update the internal system. Maintain data integrity and security. Avoid performance issues for the internal system. I’m evaluating a few possible approaches but I’m unsure which is best practice: Allow the cloud payment service to connect directly to the internal SQL Server database through secure networking. Maintain a replicated or read/write copy of the database in the cloud. Use SQL Server replication (transactional or snapshot) between on-prem and cloud. Introduce an API or middleware layer that handles all transactions and updates the internal database. Maintain separate databases and synchronize transactions asynchronously. My main questions: Is it recommended to expose the internal SQL Server database directly to cloud services? Should I use replication, a secondary database, or a service/API layer? What architecture pattern is commonly used for integrating on-prem financial systems with online payment platforms? How can we ensure consistency between internal transactions and online payments? Are there recommended SQL Server features or patterns for this scenario (replication, service broker, CDC, etc.)? Any advice on best practices, architecture patterns, or real-world implementations would be greatly appreciated.
Best architecture to integrate internal SQL Server system with cloud-based online payment services
I currently have an internal enterprise ERP system also need to be integrated with online payments, and finance operations. The system runs on an on-premises database using Microsoft SQL Server 2022 and stores all financial transactions and internal workflow data. We now have a new requirement to enable online payment services for customers. These services will be exposed externally (likely in the cloud) and must interact with the same financial system so that transactions are reflected in our internal database. My main concerns are related to architecture, security, and data synchronization. Key points about the current setup: The core system and database are hosted internally (on-premises). The database contains sensitive finance and payment data. Internal processes depend on the current database structure and workflows. The new requirements: Develop an online payment service accessible over the internet. Ensure transactions from the online service update the internal system. Maintain data integrity and security. Avoid performance issues for the internal system. I’m evaluating a few possible approaches but I’m unsure which is best practice: Allow the cloud payment service to connect directly to the internal SQL Server database through secure networking. Maintain a replicated or read/write copy of the database in the cloud. Use SQL Server replication (transactional or snapshot) between on-prem and cloud. Introduce an API or middleware layer that handles all transactions and updates the internal database. Maintain separate databases and synchronize transactions asynchronously. My main questions: Is it recommended to expose the internal SQL Server database directly to cloud services? Should I use replication, a secondary database, or a service/API layer? What architecture pattern is commonly used for integrating on-prem financial systems with online payment platforms? How can we ensure consistency between internal transactions and online payments? Are there recommended SQL Server features or patterns for this scenario (replication, service broker, CDC, etc.)? Any advice on best practices, architecture patterns, or real-world implementations would be greatly appreciated.
SSRS 2016 Browsing/API very slow
Since moving an existing SSRS server to an Azure VM we get extremely slow performance when opening to browse reports. Devtools shows 2 calls to the API are taking 25s+ to respond - ServiceState and Me. When we call the API direct for those 2 (e.g. <servername>/reports/api/v1.0/ServiceState) they are both the same - very slow. They return a 200 response, so are working ok, and once you've logged in browsing around is fine. The reports themselves run well, and if we skip the API and just browse to ReportServer it's fast. The only thing that has changed on the server is the IP when it moved to Azure - there are no other new firewall rules/ACLs/etc in place. Has anyone had this issue or can shed some light on this?
SQL Server issue but don't know what - please help!
Hi, I'm facing a SQL Server focused issue that I don't understand why it's occurring and would like your help to identify and resolve, please. I will provide an in-depth breakdown of the scenario. Two years ago, I created a Azure Data Factory (ADF) Pipeline to take data from Azure Synapse to Azure SQL Server, and two other Pipelines to take data from an On-Premises Sage server to the same Azure SQL Server. These Pipelines were working perfectly up until two days ago (11/03/26) when the Pipelines still always complete successfully but the duration have increased greatly. The below screenshot shows the log of these three Pipeline for the past week. Note how prior to the 12/03/26 the Synapse Pipeline took about 6 minutes to complete and the two Sage Pipelines took around 25 seconds. Also note how on the 12/03/26 the Synapse Pipeline took over 3 hours but the two Sage Pipelines continued with their normal 25 seconds. Notice today (13/03/26) the Synapse Pipeline was still slow but now the two Sage Pipelines are taking over an hour. I'll note here that the Pipelines still complete successfully (so the config must be correct as it has been for two years). Each Pipeline contains a single (for Sage) or several (for Synapse) 'Copy data' objects. The objects have their source configuration to simply extract from the source (so either Synapse or Sage) and then the sink configuration has a 'Pre-Copy script' which simply Truncates the target SQL Server table, before loading to the SQL Server table. The screenshot below is one example - each Pre-Copy script is populated the same but just with the tables being different. When I look at the log for the Pipelines, I see a common theme for each and that is the 'Pre-Copy script' (the Truncate) is consuming 99% of the time. The screenshot below shows this common theme. So at this point, I ask the question why, after two years of all working well and completing very quickly, are the Pipelines taking so long to complete? This also seems to be an intermittent problem as I have performed several manual executions which will take a long while, then revert back to the quick several minutes again (so good), then revert once more to taking a long while again. It is intermittent. See the screenshot below. Notice how the same Pipeline have different durations. It's intermittent. The reason why I think it is a SQL Server focused issue for the following reasons: 1) Synapse - Performing simple commands (Select Top) returns data within seconds. 2) SQL Server - Performing simple commands (Select Top) usually takes seconds but during the past two days is often taking half hour. 3) SQL Server - When amending a SQL View it would take a split second to complete. During the past two days, it goes over ten minutes and still doesn't complete (I was amending the View for testing purposes). 4) SQL Server - When looking at the Views node under a database, during the past two days, it intermittently doesn't show the View. Sometimes, it will work if I log out and then back into SQL Server. 5) Power BI - Refreshing a Power BI report, whether from the Power BI Service or Desktop, call SQL Server Views. These SQL Server Views read from several SQL Server tables. These report refreshes are failing due to an IDbCommand interface error. These Power BI report data refreshes simply read from a SQL Server table via a SQL View. They don't reference or consider at all the any ADF Pipelines. The Pipelines execute between 2am and 3am each morning. The Power BI report refresh their data around 12pm to 5pm. As SQL Server seems to be problematic at the point of the 'Pre-Copy script', which is one of the end to end process, and then problematic at the very other end where Power BI reports consumes SQL Views - leads me to believe the issue is with SQL Server. I am a report developer and not an ADF expert. I've built the ADF process on the side. I haven't changed any development/configuration/etc... between this all working and failing. Our I.T. department have advised they have made no changes to contribute to this issue. Please can someone advise on what's happening here and why this issue has arose when for two years all was fine? Thanks.
SSMS 22 - Not seeing the Database name information in the Status Bar and Title bar is gone.
I installed SSMS 22 yesterday and am trying to get used to the features. I am not a fan of the new layout. I might be in the future, but at the moment, not so much. Status Bar settings are all set to [True] But nothing is showing in the Status Bar. There is no Title Bar anymore. WHY Would they take away the Title Bar? That is insane. So, my question to the Forum is this. How do we get the Database Name information in the Status Bar? As stated, everything is set to TRUE in settings. Also, will the Status Bar show the title of the Editor file you are working on? (I miss the old design, to be honest. )
SSMS 21/22 Error Upload BACPAC file to Azure Storage
Hello All In my SSMS 20, I can use "Export Data-tier Application" to export an BACPAC file of Azure SQL database and upload to Azure storage in the same machine, the SSMS 21 gives error message when doing the same export, it created the BACPAC files but failed on the last step, "Uploading BACPAC file to Microsoft Azure Storage", The error message is "Could not load file or assembly 'System.IO.Hashing, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The system cannot find the file specified. (Azure.Storage.Blobs)" I tried the fresh installation of SSMS 21 in a brand-new machine (Windows 11), same issue, Can anyone advice? ThanksAccess to SQL Jobs
Hi, I have several jobs in SQL Server Agent. In the department there are 3 people who need administration access to the jobs. What kind of security access must they have? They already have SQL Agent Operator, SQL Agent Reader and SQL Agent User. As far as I know you can only edit jobs where you are the Owner. But, all 3 people must be able to edit all jobs. How to solve that issue?
Synchronize SQL database between two servers
Questions (to make right the synchronize SQL database between two servers) 1. Is required to have SQL Enterprise in both servers, for to be able the synchronization? Or will be enough to have SQL Enterprise in server1. And in server2 with only SQL Express, please? 2. Which requirements are required to fulfil or prepare for to be able synchronize SQL database between two servers, please? 3. Which possibilities (when are more possible solutions) have existed for synchronize SQL database (totally awaiting between four servers), please? 4. Exist a manual for reading an synchronization of SQL server - settings and steps? Thanks in advance.
SQL Server AG Failover - Automatic Failover
Hello, I am looking for a straight and definitive answer that I was hoping someone could answer for me. I want to trust what Copilot says, but I really need to hear it from Microsoft and I can't find any documentation from Microsoft confirming my question. My Environment: 2 replicas in datacenter 1 1 replica in datacenter 2 All three (3) replicas are set to synchronous-commit mode with automatic failover. I tested the failover manually between all three (3) replicas without issue. When I test the automatic failover - I take down both replicas in datacenter 1 at the same time to simulate a datacenter outage. I look at the replica in datacenter 2 and it is just says (Resolving...) next to the replica name. The replica does not come online and the DB is not moved. When I was searching I couldn't find out why. So I turned to Copilot not solve the issue, but to see if it could point me in the right direction. I tell Copilot my setup and what happened. Copilot responded stating that by design from Microsoft you cannot have more than two (2) replicas set to synchronous-commit mode with automatic failover in a SQL Server AG instance. That if more than two (2) are set for automatic failover. The SQL Server AG will use the first two (2) replicas it sees in its metadata and ignore the rest. Copilot went into detail about why this is designed this way, but the amount of information would make this post longer than it already is. If this is true - then when I took down both replicas in datacenter 1, SQL Server AG only saw those two (2) replicas in datacenter 1 as the available replicas to use for an automatic failover and thus why the replica in datacenter 2 did not come online and the DB not being moved So let's do a test. I brought back up the two (2) replicas in datacenter 1. Then I made a change in the AG proprieties. I set the 2nd replica in datacenter 1 to manual. So 1 replica is set to automatic failover and 1 replica is set to manual failover in datacenter 1. The replica in datacenter 2 is set to automatic failover I then take down both replicas in datacenter 1 again to simulate the "outage" and the replica in datacenter 2 comes online and the DB is moved. So is Copilot right? Can there only be two (2) replicas allowed to have/use automatic failover? I cannot find a definitive answer confirming this. Or is my configuration wrong/missing something and if it is, could you please point me in the right direction on how to get this resolved?
Cannot connect Azure OpenAI Embeddings model to SQL Server 2025
On SQL Server 2025, I am trying to vectorize a table. To set up the ability for SQL Server 2025 to communicate with Azure OpenAI embeddings model, I first created a master key for encryption. CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'Secret'; GO Then I set up a database scoped credential. CREATE DATABASE SCOPED CREDENTIAL [MyAzureOpenAICredential] WITH IDENTITY = 'HTTPEndpointHeaders', SECRET = '{"api-key":"secret"}'; Then I created an external model. CREATE EXTERNAL MODEL AzureOpenAIEmbeddingsModel WITH ( LOCATION = 'https://{secret}-eastus2.cognitiveservices.azure.com/openai/deployments/text-embedding-3-small/embeddings?api-version=2023-05-15', API_FORMAT = 'Azure OpenAI', MODEL_TYPE = EMBEDDINGS, MODEL = 'text-embedding-3-small', CREDENTIAL = [MyAzureOpenAICredential] ); However, when I run this simple script: DECLARE @text NVARCHAR(MAX) = N'SQL Server 2025 enables AI-powered applications'; DECLARE @embedding VECTOR(1536) = AI_GENERATE_EMBEDDINGS(@text USE MODEL AzureOpenAIEmbeddingsModel); I get this error. The database scoped credential 'MyAzureOpenAICredential' cannot be used to invoke an external rest endpoint. I have read through https://learn.microsoft.com/en-us/training/modules/build-ai-solutions-sql-server/4-integrate-ai-models pertaining to this task. As well as SQL Server 2025 docs for creating a model. I have also read SQL Server 2025 docs for creating https://learn.microsoft.com/en-us/sql/t-sql/statements/create-database-scoped-credential-transact-sql?view=sql-server-ver17. I have not found any answers.Polybase - Enforce TCP/IP Protocol
Hello, We know Polybase service in SQL server by default makes use of shared memory protocol to connect SQL server (local). I would like to know if there is any way we can change or force the connections to make use of TCP / IP protocol. Any help would be appreciated.SQL Timestamp Incremental Load Issue
An incremental load is implemented based on timestamp, using a view to update a fact table. • The fact table contains multiple KPIs coming from different source tables. • Two main KPI tables are included directly in the view and drive the incremental logic (their timestamps change). • Additional KPIs come from other source tables, which are LEFT JOINed to the view. During an incremental run: • Main source tables are updated → timestamp changes • Other source tables are NOT updated → timestamp unchanged • Because of the LEFT JOIN, missing values from these tables are returned as NULL • These NULLs overwrite existing values in the fact table Example Fact table (before load) id app_number score 1 333 5 Source tables • source1, source2 → timestamps updated (drive incremental load) • source3 → timestamp unchanged Stored procedure logic MERGE fact_table tgt USING ( SELECT app_number, ISNULL(score, 0) AS score FROM vw_main_kpis v LEFT JOIN source3 s3 ON v.app_number = s3.app_number ) src ON tgt.app_number = src.app_number WHEN MATCHED THEN UPDATE SET tgt.score = src.score; Result (incorrect) id app_number score 1 333 0 Existing data is lost.Service Broker on Ubuntu 20.04 (Docker) cryptography error
Hi, Has anyone successfully created a multi‑instance / multi‑host SQL Server Service Broker implementation on Linux, specifically on distributions using OpenSSL 3? I have a fully working Service Broker environment on SQL Server 2017, running in Docker on Ubuntu 18.04 (OpenSSL 1.1.1). My environment has 6 containers, arranged to simulate two datacenters: DC1: Primary AG replica + synchronous replica + async replica DC2: Same setup 7 databases in the Availability Group Service Broker dialogs between databases on the same SQL instance work perfectly. This confirms: Database‑level dialog security works Certificate‑based authentication between database principals is correct DMKs are auto-opened and protected by SMK All dialogs use BEGIN DIALOG with certificates valid and present Cross-instance messages work correctly. I have: Broker endpoints using certificate‑based endpoint authentication Proper certificate exchange (private key on owning instance, public key on the peer) Routes defined correctly Remote Service Bindings used to force dialog security Certificates created dynamically with correct validity date ranges DMKs created, opened, and re-encrypted with SMK at runtime Scripts that fully automate certificate creation, export, import, user creation, RSB creation, etc. Everything here works fine as long as I stay on Ubuntu 18.04 (OpenSSL 1.1.1). The problem: If I switch only the base Linux image—to Ubuntu 20.04, Ubuntu 22.04, or RHEL 8—and change nothing else, then all cross‑instance Service Broker dialogs fail with: An exception occurred while enqueueing a message in the target queue. Error: 9641, State: 122. A cryptographic operation failed. This error indicates a serious problem with SQL Server. This happens on SQL Server 2019 and 2022, both CU up-to-date, both using OpenSSL 3. Key points: Intra‑instance dialog security still works Endpoint transport security still works (i.e., endpoints authenticate and connect) The failure occurs only when creating session keys for cross‑instance dialog security The failure happens even when using: encryption = off fresh DMKs fresh certificates certificates stored in master or user DB key length 2048 or 3072 different validity periods So far I have: Confirmed DMKs exist on both instances Confirmed DMKs are encrypted by SMK Confirmed DMKs auto-open (is_master_key_encrypted_by_server = 1) Manually opened DMKs before creating certificates Recreated SMK/DMK on clean containers Everything behaves as expected. Repeatedly rebuilt all of the following: Endpoint authentication certificates Dialog security certificates Remote Service Bindings Database principals Routes Service Broker services, queues, and message types I’ve verified: All certificate subjects match All public keys export/import correctly Private keys exist where they should exist Thumbprints match across both sides No old certificates remain in master or user DBs Modified the OpenSSL behaviour to legacy (1.1.1) behaviour, e.g. [openssl_init] providers = provider_sect ssl_conf = ssl_sect alg_section = evp_properties [evp_properties] rh-allow-sha1-signatures = yes fips_mode = no [provider_sect] default = default_sect legacy = legacy_sect [default_sect] activate = 1 [legacy_sect] activate = 1 [ssl_sect] system_default = system_default_sect [all_policy] rsa_pkcs1_padding_check = 0 [system_default_sect] # Lowest policy to allow legacy algorithms (including SHA-1, 1024-bit RSA) CipherString = DEFAULT:@SECLEVEL=0 To rule out network/transport issues, I validated: Endpoints authenticate each other Connections appear in sys.dm_broker_connections STATE = OPEN Transport security appears to be working on OpenSSL 3. I confirmed: Messages flow when RSB is removed and encryption=off is used Messages fail only when RSB is enabled, which requires certificate‑based dialog security The target instance claims “private key missing” even though it is present and readable Has anyone successfully created a multi‑instance / multi‑host SQL Server Service Broker implementation on Linux, specifically on distributions using OpenSSL 3? If so, any guidance on how to resolve the issues i am facing? Many thanks, Andrew
SQL Server 2025 VECTOR functions accepting JSON array strings
Hello! Playing with new VECTOR type and functions, I can read following https://learn.microsoft.com/en-us/sql/t-sql/functions/vector-distance-transact-sql?view=sql-server-ver17 for vector parameters: An expression that evaluates to vector data type. To me this means that any expression (including character strings as JSON-arrays) can be used as parameter. Since INSERT statement accepts to convert a JSON-array string to a VECTOR, I would expect that these function also accept this conversion. However, it appears that we are forced to cast the JSON-array to VECTOR. Any chance to improve this? Here some T-SQL example: declare v1 as VECTOR(3) = '[-1,1,0]' declare S1 as VARCHAR(50) = '[-1,1,0]' drop table tab1; create table tab1 (pkey int not null primary key, emb vector(3)); insert into tab1 values ( 101, v1 ); insert into tab1 values ( 102, S1 ); select * from tab1 order by pkey; select vector_distance('cosine',emb,@v1) from tab1; select vector_distance('cosine',emb,@s1) from tab1; -- fails Seb
SQL Server 2005 (compatibility level 90)
Hello, I’m testing the behavior described in the SQL Server documentation for **compatibility level 90** regarding the special attributes `xsi:nil` and `xsi:type`: > “The special attributes `xsi:nil` and `xsi:type` can't be queried or modified by data manipulation language statements. This means that `/e/@xsi:nil` fails while `/e/@*` ignores the `xsi:nil` and `xsi:type` attributes. However, `/e` returns the `xsi:nil` and `xsi:type` attributes for consistency with `SELECT xmlCol`, even if `xsi:nil = "false"`. ” But on **SQL Server 2005**, I can successfully query `@xsi:nil` and it returns the expected value. I’m trying to reproduce the documented “`/e/@xsi:nil` fails” behavior, but I can’t. ### Environment - Product: **Microsoft SQL Server 2005** - Database compatibility level: **90** --- ## ✅ Repro script ```sql IF EXISTS (SELECT * FROM sys.xml_schema_collections WHERE name = 'MyTestSchema') DROP XML SCHEMA COLLECTION MyTestSchema; GO CREATE XML SCHEMA COLLECTION MyTestSchema AS N' <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <xsd:element name="root"> <xsd:complexType> <xsd:sequence> <xsd:element name="element" nillable="true" type="xsd:string" /> </xsd:sequence> </xsd:complexType> </xsd:element> </xsd:schema>'; GO DECLARE @xmlData XML(MyTestSchema) = N' <root xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <element xsi:nil="true" /> </root>'; ;WITH XMLNAMESPACES ('http://www.w3.org/2001/XMLSchema-instance' as xsi) SELECT @xmlData.query('<result> { /root/element/@xsi:nil } </result>') AS Typed_Result; ``` ### Actual result `Typed_Result` contains `xsi:nil="true"` under `<result>...`. ### Expected result (based on docs) I expected `/root/element/@xsi:nil` to fail, or not return `xsi:nil`. --- ## Questions 1. In the documentation, does “data manipulation language statements” mean only **XML DML** (i.e., `.modify()`), not XQuery used in `SELECT` with `.query()` / `.value()`? 2. Does the “`/e/@xsi:nil` fails” behavior apply only when the XML is stored in a **table column**, not when using an **XML variable**? 3. Is the behavior different between **typed XML** (with an XML schema collection) vs **untyped XML**? 4. Can someone provide a minimal reproduction in SQL Server 2005 where `/e/@xsi:nil` fails as described? Thank you. ---Compat level 90: XML string-to-datetime UDF
Hello, I’m testing a behavior described in SQL Server documentation for **database compatibility level 90**. The docs state that a user-defined function that converts an XML constant string value to a SQL Server date/time type is marked as **deterministic**. On **SQL Server 2005**, I’m seeing the opposite: the function is marked as **non-deterministic** (`IsDeterministic = 0`). I’m trying to understand whether I’m missing a requirement/constraint or whether this is a doc mismatch / version-specific behavior. ### Environment - Product: **Microsoft SQL Server 2005** - Database compatibility level: **90** --- ## ✅ Repro script ```sql IF OBJECT_ID('dbo.fn_ParamXmlToDatetime', 'FN') IS NOT NULL DROP FUNCTION dbo.fn_ParamXmlToDatetime; GO CREATE FUNCTION dbo.fn_ParamXmlToDatetime (@xml XML) RETURNS DATETIME WITH SCHEMABINDING AS BEGIN DECLARE @y DATETIME; -- Convert an XML value to DATETIME SET @y = CONVERT(DATETIME, @xml.value('(/r)[1]', 'datetime')); RETURN @y; END GO SELECT OBJECTPROPERTY(OBJECT_ID('dbo.fn_ParamXmlToDatetime'), 'IsDeterministic') AS IsDeterministic, OBJECTPROPERTY(OBJECT_ID('dbo.fn_ParamXmlToDatetime'), 'IsPrecise') AS IsPrecise; GO ``` ### Actual result `IsDeterministic = 0` (non-deterministic) ### Expected result (based on docs) `IsDeterministic = 1` (deterministic) for this pattern under compat level 90. --- ## Questions 1. Are there additional conditions required for SQL Server to mark this UDF as deterministic (for example, specific XQuery usage, avoiding `CONVERT`, using `CAST`, using `datetime2` doesn’t exist in 2005, etc.)? 2. Does the determinism rule apply only when converting from an **XML literal constant** inside the function, rather than an XML parameter value? 3. Is this behavior different for **typed XML** (XML schema collections) vs **untyped XML**? 4. Is this a known difference/bug in SQL Server 2005 where the UDF is functionally deterministic but still reported as non-deterministic by `OBJECTPROPERTY`? Thank you for any clarification. ---
Recent Blogs
- Bulk data import using operations like BULK INSERT and OPENROWSET(BULK…) BULK INSERT (Transact-SQL) - SQL Server | Microsoft Learn is fundamental to ETL and data ingestion workflows. On SQL Server ru...Mar 19, 2026
- This document helps to Troubleshoot SQL Server Backup Failures to Azure Blob Storage. Backing up SQL Server databases directly to Azure Blob Storage using BACKUP TO URL is a common and recommended ...Mar 18, 2026
