Build fast, scalable data system on Azure SQL Database Hyperscale | Clearent

Published 05-04-2021 01:35 PM 1,274 Views
Contributor

How to build a fast, scalable data system on Azure SQL Database Hyperscale. Hyperscale’s flexible architecture scales with the pace of your business to process large amounts of data with a small amount of compute in just minutes, and allows you to back up data almost instantaneously.

 

Screen Shot 2021-05-04 at 4.15.45 PM.png

 

Zach Fransen, VP of data and AI at Xplor, joins Jeremy Chapman to share how credit card processing firm, Clearent by Xplor, built a fast, scalable merchant transaction reporting system on Azure SQL Database Hyperscale. Take a deep dive on their Hyperscale implementation, from their approach with micro-batching to continuously bring in billions of rows of transactional data, from their on-premises payment fulfillment system at scale, as well as their optimizations for near real-time query performance using clustered column store indexing for data aggregation.

 

 

QUICK LINKS:

 

 

Link References:

 

Unfamiliar with Microsoft Mechanics?

 

Keep getting this insider knowledge, join us on social:


Video Transcript:

 

%3CLINGO-SUB%20id%3D%22lingo-sub-2323616%22%20slang%3D%22en-US%22%3EBuild%20fast%2C%20scalable%20data%20system%20on%20Azure%20SQL%20Database%20Hyperscale%20%7C%20Clearent%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2323616%22%20slang%3D%22en-US%22%3E%3CP%3E%3CSPAN%3EHow%20to%20build%20a%20fast%2C%20scalable%20data%20system%20on%20Azure%20SQL%20Database%20Hyperscale.%20Hyperscale%E2%80%99s%20flexible%20architecture%20scales%20with%20the%20pace%20of%20your%20business%20to%20process%20large%20amounts%20of%20data%20with%20a%20small%20amount%20of%20compute%20in%20just%20minutes%2C%20and%20allows%20you%20to%20back%20up%20data%20almost%20instantaneously.%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Screen%20Shot%202021-05-04%20at%204.15.45%20PM.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F278017i8BF3E3F9E7019278%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Screen%20Shot%202021-05-04%20at%204.15.45%20PM.png%22%20alt%3D%22Screen%20Shot%202021-05-04%20at%204.15.45%20PM.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%3EZach%20Fransen%2C%20VP%20of%20data%20and%20AI%20at%20Xplor%2C%20joins%26nbsp%3B%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Ftwitter.com%2Fdeployjeremy%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3EJeremy%26nbsp%3BChapman%3C%2FA%3E%26nbsp%3Bto%20share%20how%20credit%20card%20processing%20firm%2C%20Clearent%20by%20Xplor%2C%20built%20a%20fast%2C%20scalable%20merchant%20transaction%20reporting%20system%20on%20Azure%20SQL%20Database%20Hyperscale.%20Take%20a%20deep%20dive%20on%20their%20Hyperscale%20implementation%2C%20from%20their%20approach%20with%20micro-batching%20to%20continuously%20bring%20in%20billions%20of%20rows%20of%20transactional%20data%2C%20from%20their%20on-premises%20payment%20fulfillment%20system%20at%20scale%2C%20as%20well%20as%20their%20optimizations%20for%20near%20real-time%20query%20performance%20using%20clustered%20column%20store%20indexing%20for%20data%20aggregation.%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3C%2FP%3E%3CDIV%20class%3D%22video-embed-center%20video-embed%22%3E%3CIFRAME%20class%3D%22embedly-embed%22%20src%3D%22https%3A%2F%2Fcdn.embedly.com%2Fwidgets%2Fmedia.html%3Fsrc%3Dhttps%253A%252F%252Fwww.youtube.com%252Fembed%252FDUV9wop86Tw%253Ffeature%253Doembed%26amp%3Bdisplay_name%3DYouTube%26amp%3Burl%3Dhttps%253A%252F%252Fwww.youtube.com%252Fwatch%253Fv%253DDUV9wop86Tw%26amp%3Bimage%3Dhttps%253A%252F%252Fi.ytimg.com%252Fvi%252FDUV9wop86Tw%252Fhqdefault.jpg%26amp%3Bkey%3Db0d40caa4f094c68be7c29880b16f56e%26amp%3Btype%3Dtext%252Fhtml%26amp%3Bschema%3Dyoutube%22%20width%3D%22600%22%20height%3D%22337%22%20scrolling%3D%22no%22%20title%3D%22YouTube%20embed%22%20frameborder%3D%220%22%20allow%3D%22autoplay%3B%20fullscreen%22%20allowfullscreen%3D%22true%22%3E%3C%2FIFRAME%3E%3C%2FDIV%3E%3CP%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%2236d0%22%20class%3D%22ln%20in%20gs%20ba%20io%20lo%20lp%20lq%20is%20lr%20ls%20lt%20iw%20lu%20lv%20lw%20ja%20lx%20ly%20lz%20je%20ma%20mb%20mc%20ji%20md%20ho%22%20data-selectable-paragraph%3D%22%22%20id%3D%22toc-hId--469762181%22%20id%3D%22toc-hId--469762183%22%3EQUICK%20LINKS%3A%3C%2FH2%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20jo%20ir%20jp%20jq%20jr%20iv%20js%20jt%20ju%20jv%20jw%20jx%20jy%20jz%20ka%20kb%20kc%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DDUV9wop86Tw%26amp%3Bt%3D35s%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3E00%3A35%3C%2FA%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%E2%80%94%20Intro%20to%20Clearent%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DDUV9wop86Tw%26amp%3Bt%3D93s%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3E01%3A33%3C%2FA%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%E2%80%94%20Starting%20point%20and%20challenges%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DDUV9wop86Tw%26amp%3Bt%3D192s%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3E03%3A12%3C%2FA%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%E2%80%94%20Clearant%E2%80%99s%20shift%20to%20Hyperscale%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DDUV9wop86Tw%26amp%3Bt%3D293s%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3E04%3A53%3C%2FA%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%E2%80%94%20Near%20real-time%20reporting%2Fmicro-batching%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DDUV9wop86Tw%26amp%3Bt%3D385s%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3E06%3A25%3C%2FA%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%E2%80%94%20See%20it%20in%20action%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DDUV9wop86Tw%26amp%3Bt%3D508s%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3E08%3A28%3C%2FA%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%E2%80%94%20Processing%20large%20amounts%20of%20data%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DDUV9wop86Tw%26amp%3Bt%3D582s%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3E09%3A42%3C%2FA%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%E2%80%94%20Namd%20replicas%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DDUV9wop86Tw%26amp%3Bt%3D634s%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3E10%3A34%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3C%2FA%3E%E2%80%94%20Query%20speed%20ups%20%E2%80%94%20clustered%20column%20store%20indexing%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DDUV9wop86Tw%26amp%3Bt%3D705s%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3E11%3A45%3C%2FA%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%E2%80%94%20What%E2%80%99s%20next%20for%20Clearent%20by%20Xplor%3F%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DDUV9wop86Tw%26amp%3Bt%3D746s%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3E12%3A26%3C%2FA%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%E2%80%94%20Wrap%20up%3C%2FP%3E%0A%3CH2%20class%3D%22ln%20in%20gs%20ba%20io%20lo%20lp%20lq%20is%20lr%20ls%20lt%20iw%20lu%20lv%20lw%20ja%20lx%20ly%20lz%20je%20ma%20mb%20mc%20ji%20md%20ho%22%20data-selectable-paragraph%3D%22%22%20id%3D%22toc-hId-2017750652%22%20id%3D%22toc-hId-2017750650%22%3E%26nbsp%3B%3C%2FH2%3E%0A%3CH2%20id%3D%22c9cc%22%20class%3D%22ln%20in%20gs%20ba%20io%20lo%20lp%20lq%20is%20lr%20ls%20lt%20iw%20lu%20lv%20lw%20ja%20lx%20ly%20lz%20je%20ma%20mb%20mc%20ji%20md%20ho%22%20data-selectable-paragraph%3D%22%22%20id%3D%22toc-hId-210296189%22%20id%3D%22toc-hId-210296187%22%3ELink%20References%3A%3C%2FH2%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20jo%20ir%20jp%20jq%20jr%20iv%20js%20jt%20ju%20jv%20jw%20jx%20jy%20jz%20ka%20kb%20kc%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3ELearn%20more%20about%20Clearent%20by%20Xplor%20and%20what%20they%E2%80%99re%20doing%20with%20Hyperscale%20at%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Faka.ms%2FClearentMechanics%22%20rel%3D%22noopener%20noreferrer%22%20target%3D%22_blank%22%3Ehttps%3A%2F%2Faka.ms%2FClearentMechanics%3C%2FA%3E%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20jo%20ir%20jp%20jq%20jr%20iv%20js%20jt%20ju%20jv%20jw%20jx%20jy%20jz%20ka%20kb%20kc%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3EFor%20more%20guidance%20on%20implementing%20Azure%20SQL%20Database%20Hyperscale%2C%20check%20out%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Faka.ms%2FMechanicsHyperscale%22%20rel%3D%22noopener%20noreferrer%22%20target%3D%22_blank%22%3Ehttps%3A%2F%2Faka.ms%2FMechanicsHyperscale%3C%2FA%3E%3C%2FP%3E%0A%3CH2%20class%3D%22ln%20in%20gs%20ba%20io%20lo%20lp%20lq%20is%20lr%20ls%20lt%20iw%20lu%20lv%20lw%20ja%20lx%20ly%20lz%20je%20ma%20mb%20mc%20ji%20md%20ho%22%20data-selectable-paragraph%3D%22%22%20id%3D%22toc-hId--1597158274%22%20id%3D%22toc-hId--1597158276%22%3E%26nbsp%3B%3C%2FH2%3E%0A%3CH2%20id%3D%220a6d%22%20class%3D%22ln%20in%20gs%20ba%20io%20lo%20lp%20lq%20is%20lr%20ls%20lt%20iw%20lu%20lv%20lw%20ja%20lx%20ly%20lz%20je%20ma%20mb%20mc%20ji%20md%20ho%22%20data-selectable-paragraph%3D%22%22%20id%3D%22toc-hId-890354559%22%20id%3D%22toc-hId-890354557%22%3EUnfamiliar%20with%20Microsoft%20Mechanics%3F%3C%2FH2%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20jo%20ir%20jp%20jq%20jr%20iv%20js%20jt%20ju%20jv%20jw%20jx%20jy%20jz%20ka%20kb%20kc%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3EWe%20are%20Microsoft%E2%80%99s%20official%20video%20series%20for%20IT.%20You%20can%20watch%20and%20share%20valuable%20content%20and%20demos%20of%20current%20and%20upcoming%20tech%20from%20the%20people%20who%20build%20it%20at%20%23Microsoft.%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20jo%20ir%20jp%20jq%20jr%20iv%20js%20jt%20ju%20jv%20jw%20jx%20jy%20jz%20ka%20kb%20kc%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CUL%20class%3D%22%22%3E%0A%3CLI%20id%3D%22a4ea%22%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20me%20mf%20mg%20ho%22%20data-selectable-paragraph%3D%22%22%3ESubscribe%20to%20our%20YouTube%3A%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Fwww.youtube.com%2Fc%2FMicrosoftMechanicsSeries%3Fsub_confirmation%3D1%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Fwww.youtube.com%2Fc%2FMicrosoftMechanicsSeries%3Fsub_confirmation%3D1%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%20id%3D%229ad9%22%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20mh%20ir%20jp%20jq%20mi%20iv%20js%20jt%20mj%20jv%20jw%20jx%20mk%20jz%20ka%20kb%20ml%20kd%20ke%20kf%20me%20mf%20mg%20ho%22%20data-selectable-paragraph%3D%22%22%3EJoin%20us%20on%20the%20Microsoft%20Tech%20Community%3A%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fmicrosoft-mechanics-blog%2Fbg-p%2FMicrosoftMechanicsBlog%22%20target%3D%22_blank%22%3Ehttps%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fmicrosoft-mechanics-blog%2Fbg-p%2FMicrosoftMechanicsBlog%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%20id%3D%2213a1%22%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20mh%20ir%20jp%20jq%20mi%20iv%20js%20jt%20mj%20jv%20jw%20jx%20mk%20jz%20ka%20kb%20ml%20kd%20ke%20kf%20me%20mf%20mg%20ho%22%20data-selectable-paragraph%3D%22%22%3EWatch%20or%20listen%20via%20podcast%20here%3A%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Fmicrosoftmechanics.libsyn.com%2Fwebsite%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Fmicrosoftmechanics.libsyn.com%2Fwebsite%3C%2FA%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CH2%20class%3D%22ln%20in%20gs%20ba%20io%20lo%20lp%20lq%20is%20lr%20ls%20lt%20iw%20lu%20lv%20lw%20ja%20lx%20ly%20lz%20je%20ma%20mb%20mc%20ji%20md%20ho%22%20data-selectable-paragraph%3D%22%22%20id%3D%22toc-hId--917099904%22%20id%3D%22toc-hId--917099906%22%3E%26nbsp%3B%3C%2FH2%3E%0A%3CH2%20id%3D%225a55%22%20class%3D%22ln%20in%20gs%20ba%20io%20lo%20lp%20lq%20is%20lr%20ls%20lt%20iw%20lu%20lv%20lw%20ja%20lx%20ly%20lz%20je%20ma%20mb%20mc%20ji%20md%20ho%22%20data-selectable-paragraph%3D%22%22%20id%3D%22toc-hId-1570412929%22%20id%3D%22toc-hId-1570412927%22%3EKeep%20getting%20this%20insider%20knowledge%2C%20join%20us%20on%20social%3A%3C%2FH2%3E%0A%3CUL%20class%3D%22%22%3E%0A%3CLI%20id%3D%22dd60%22%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20jo%20ir%20jp%20jq%20jr%20iv%20js%20jt%20ju%20jv%20jw%20jx%20jy%20jz%20ka%20kb%20kc%20kd%20ke%20kf%20me%20mf%20mg%20ho%22%20data-selectable-paragraph%3D%22%22%3EFollow%20us%20on%20Twitter%3A%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Ftwitter.com%2FMSFTMechanics%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Ftwitter.com%2FMSFTMechanics%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%20id%3D%229bd0%22%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20mh%20ir%20jp%20jq%20mi%20iv%20js%20jt%20mj%20jv%20jw%20jx%20mk%20jz%20ka%20kb%20ml%20kd%20ke%20kf%20me%20mf%20mg%20ho%22%20data-selectable-paragraph%3D%22%22%3EFollow%20us%20on%20LinkedIn%3A%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fmicrosoft-mechanics%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fmicrosoft-mechanics%2F%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%20id%3D%229358%22%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20mh%20ir%20jp%20jq%20mi%20iv%20js%20jt%20mj%20jv%20jw%20jx%20mk%20jz%20ka%20kb%20ml%20kd%20ke%20kf%20me%20mf%20mg%20ho%22%20data-selectable-paragraph%3D%22%22%3EFollow%20us%20on%20Facebook%3A%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3CA%20class%3D%22bv%20ll%22%20href%3D%22https%3A%2F%2Ffacebook.com%2Fmicrosoftmechanics%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Ffacebook.com%2Fmicrosoftmechanics%2F%3C%2FA%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CHR%20%2F%3E%0A%3CH2%20id%3D%22d0fd%22%20class%3D%22ln%20in%20gs%20ba%20io%20lo%20lp%20lq%20is%20lr%20ls%20lt%20iw%20lu%20lv%20lw%20ja%20lx%20ly%20lz%20je%20ma%20mb%20mc%20ji%20md%20ho%22%20data-selectable-paragraph%3D%22%22%20id%3D%22toc-hId--237041534%22%20id%3D%22toc-hId--237041536%22%3EVideo%20Transcript%3A%3C%2FH2%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20jo%20ir%20jp%20jq%20jr%20iv%20js%20jt%20ju%20jv%20jw%20jx%20jy%20jz%20ka%20kb%20kc%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20Up%20next%2C%20we%20meet%20with%20credit%20card%20processing%20firm%20Clearent%20by%20Xplor%20to%20see%20how%20they%20built%20a%20fast%2C%20scalable%20merchant%20transaction%20reporting%20system%20on%20Azure%20SQL%20Database%20Hyperscale.%20From%20their%20approach%20with%20micro-batching%20to%20continuously%20bring%20in%20billions%20of%20rows%20of%20transactional%20data%20from%20their%20on-premises%20payment%20fulfillment%20system%20at%20scale%2C%20as%20well%20as%20their%20optimizations%20for%20near%20real-time%20query%20performance%20using%20clustered%20columnstore%20indexing%20for%20data%20aggregation%20and%20much%20more.%20So%20today%2C%20I%E2%80%99m%20joined%20by%20Zach%20Fransen%2C%20who%E2%80%99s%20the%20VP%20of%20data%20and%20AI%20at%20Xplor.%20So%20welcome%20to%20Microsoft%20Mechanics.%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20jo%20ir%20jp%20jq%20jr%20iv%20js%20jt%20ju%20jv%20jw%20jx%20jy%20jz%20ka%20kb%20kc%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20Thanks.%20It%E2%80%99s%20great%20to%20be%20on%20the%20show.%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20Thanks%20so%20much%20for%20joining%20us%20today.%20It%E2%80%99s%20really%20great%20to%20have%20you%20on.%20So%20if%20you%E2%80%99re%20new%20to%20Clearent%20by%20Xplor%2C%20they%E2%80%99re%20a%20US-based%20payment%20solution%20provider%20that%20enables%20merchants%20and%20retailers%20to%20accept%20credit%20card%20payments%20from%20anywhere.%20They%20process%20over%20500%20million%20transactions%20per%20year.%20And%20since%20their%20inception%20in%202005%2C%20they%E2%80%99ve%20built%20several%20intelligent%20solutions%20to%20reduce%20credit%20card%20processing%20fees%20and%20significantly%20speed%20up%20the%20fulfillment%20of%20credit%20card%20payments%20with%20next%20day%20funding.%20Now%20key%20to%20this%20has%20been%20connecting%20their%20back-end%20payment%20fulfillment%20system%20with%20their%20transaction%20reporting%20system.%20In%20fact%2C%20they%E2%80%99ve%20recently%20completed%20an%20app%20and%20data%20modernization%20effort%20using%20Azure%20SQL%20Database%20Hyperscale.%20Their%20new%20Compass%20Online%20Reporting%20Suite%20gives%20their%20customers%20unprecedented%20and%20concurrent%20access%20to%20sales%20transaction%20data%20stored%20on%20Azure%20SQL%20Hyperscale%20for%20near%20real-time%20views%20of%20revenue%2C%20sales%20trends%2C%20and%20much%20more.%20So%20Zach%2C%20to%20put%20this%20effort%20into%20context%2C%20can%20you%20help%20us%20understand%20where%20you%20were%20coming%20from%3F%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20Sure.%20So%20when%20we%20started%20this%20effort%2C%20we%20were%20fully%20on-premises.%20We%20had%20two%20monolithic%20legacy%20apps%2C%20Horizon%20and%20Compass%2C%20running%20off%20the%20same%20Clearent%20database.%20Payments%20from%20merchants%20came%20in%20from%20a%20legacy%20third-party%20mainframe%20payment%20gateway%20and%20landed%20in%20Horizon%20as%20flat%20files.%20Horizon%20takes%20care%20of%20qualification%2C%20billing%2C%20and%20settling%20merchant%20payments.%20Part%20of%20our%20value%20add%20is%20to%20pay%20our%20merchant%20customers%20ahead%20of%20the%20credit%20agencies.%20When%20merchants%20batch%20in%20their%20sales%20at%20the%20end%20of%20the%20day%2C%20the%20system%20qualifies%20the%20nature%20of%20the%20transaction%20by%20checking%20things%20like%20if%20the%20transaction%20was%20made%20online%20or%20from%20a%20card%20reader%20via%20a%20chip%2C%20magnetic%20strip%2C%20or%20a%20rewards%20card.%20These%20are%20all%20things%20that%20determine%20the%20cost%20of%20the%20transaction%20and%20the%20fees%20charged%20by%20Visa%2C%20Mastercard%2C%20Amex%20and%20other%20credit%20agencies.%20We%20also%20charge%20a%20derivative%20fee%20based%20on%20that%2C%20so%20this%20is%20an%20intricate%20process%20to%20get%20right.%20In%20parallel%2C%20our%20Compass%20reporting%20system%20running%20on%20IIS%20had%20a%20SQL%20Server%20always%20on%20cluster%20on%20the%20same%20Clearent%20database%20back-end%20that%20would%20pull%20data%20from%20Horizon%20using%20a%20stored%20procedure%20to%20transform%20the%20data%20into%20reporting%20schemas%20for%20merchants%20to%20get%20visibility%20into%20their%20transactions.%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20Okay%2C%20so%20how%20often%20were%20you%20able%20to%20pull%20the%20data%20for%20reporting%3F%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20Well%2C%20we%20did%20a%20two-day%20lookback%20once%20a%20day%2C%20so%20we%20were%20having%20to%20process%20two%20full%20days%20of%20data%20every%20day%20for%20reporting%20and%20we%20were%20always%20a%20day%20behind.%20There%20was%20also%20contention%20because%20we%20were%20using%20the%20same%20database%20for%20processing%20transactions%20and%20reporting.%20And%20as%20our%20data%20and%20customers%20grew%2C%20our%20run%20times%20for%20processing%20the%20data%20coming%20into%20Horizon%20took%20longer.%20Also%2C%20concurrent%20customer%20queries%20in%20Compass%20would%20sometimes%20fail.%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20That%E2%80%99s%20not%20too%20surprising.%20Large%20database%20operations%20like%20these%20are%20prone%20to%20performance%20issues%20and%20storage%20limitations.%20Plus%2C%20you%20had%20to%20find%20a%20better%20way%20to%20aggregate%20the%20data%20and%20make%20it%20more%20available%20for%20reporting%20as%20well.%20So%20what%20approach%20then%20did%20you%20take%20to%20remediate%20some%20of%20these%20issues%3F%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20Well%2C%20we%20adopted%20a%20hybrid%20approach%20which%20allowed%20us%20to%20keep%20our%20data%20on-premises%20but%20migrate%20the%20data%20that%20we%20needed%20for%20reporting%20into%20the%20Cloud.%20As%20our%20foundation%20was%20SQL%20Server%2C%20we%20wanted%20to%20continue%20to%20use%20that%20skill%20set%20and%20choose%20an%20option%20where%20we%20didn%E2%80%99t%20have%20to%20worry%20about%20storage%20limits%20or%20syntax%20changes.%20We%20went%20with%20the%20Azure%20SQL%20Database%20Hyperscale%20service%20tier%20that%20scales%20to%20up%20to%20100TB%20of%20storage%20as%20the%20reporting%20back-end%20for%20our%20new%20Compass%20Online%20Reporting%20Suite.%20We%20also%20modernized%20the%20Compass%20App%2C%20taking%20an%20extensible%20microservices-based%20approach.%20We%20built%20an%20angular%20app%20that%20calls%20dozens%20of%20services%20that%20sit%20both%20on-prem%20and%20in%20the%20Cloud%2C%20depending%20on%20what%20it%E2%80%99s%20trying%20to%20do.%20And%20we%20also%20fragmented%20our%20data%20model%20to%20match%20our%20service%20topology.%20Then%2C%20to%20more%20consistency%20bring%20data%20in%2C%20we%20built%20our%20own%20payment%20gateway%20called%20Quest%2C%20which%20comprises%20a%20consistent%20set%20of%20APIs%20to%20pull%20in%20merchant%20data%20from%20physical%20credit%20card%20terminals%2C%20e-commerce%2C%20or%20integrated%20payment%20systems%20in%20use%20by%20our%20merchant%20customers.%20In%20fact%2C%20as%20we%20made%20it%20easier%20to%20bring%20data%20in%20and%20broke%20the%20Compass%20app%20and%20data%20monolith%20apart%2C%20Hyperscale%E2%80%99s%20role%20in%20providing%20a%20single%20hub%20pulling%20all%20the%20data%20together%20became%20even%20more%20important.%20Additionally%2C%20we%20only%20keep%20active%20data%20on%20premises%2C%20so%20Hyperscale%20is%20critical%20in%20keeping%20a%20persistent%20record%20of%20all%20our%20historical%20data%20and%20to%20maintaining%20a%20lower%20data%20footprint%20on-prem%20in%20those%20transactional%20systems.%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20Okay%2C%20so%20now%20you%E2%80%99ve%20got%20a%20more%20scalable%20and%20agile%20app%20and%20data%20layer%2C%20but%20with%20so%20many%20transactions%20that%20are%20happening%20at%20any%20one%20time%2C%20how%20would%20you%20solve%20for%20getting%20the%20data%20then%20into%20Hyperscale%20fast%20enough%20to%20support%20near%20real-time%20reporting%20for%20your%20merchants%3F%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20So%20the%20game%20changer%20for%20us%20was%20adopting%20low-latency%2C%20intraday%20batch%20processing%2C%20and%20data%20streaming.%20We%E2%80%99ve%20split%20our%20Hyperscale%20instance%20into%20two%20logical%20layers%2C%20comprising%20an%20Operational%20Data%20Store%20and%20an%20Enterprise%20Data%20Warehouse.%20And%20we%20shifted%20from%20once-a-day%20batch%20processing%20to%20micro-batch%20processing%20where%20we%20use%20SQL%20Server%E2%80%99s%20Change%20Data%20Capture%20feature%20to%20poll%20our%20disparate%20systems%20for%20the%20latest%20change%20in%20data%20every%20two%20minutes.%20We%20have%20data%20extractors%20running%20on%20our%20on-premises%20databases%20reading%20our%20change%20capture%20tables%2C%20and%20we%20have%20a%20set%20of%20configuration%20tables%20sitting%20in%20our%20operational%20data%20store%20that%20tells%20it%20what%20columns%20to%20read.%20The%20raw%20data%20along%20with%20the%20change%20history%20for%20every%20table%20is%20then%20loaded%20into%20Hyperscale%20over%20Express%20Route%20into%20our%20ODS.%20And%20we%20only%20extract%20and%20load%20the%20delta%20tables%20in%20our%20Enterprise%20Data%20Warehouse%20for%20reporting%20and%20analytics.%20At%20the%20same%20time%2C%20as%20a%20credit%20card%20transaction%20is%20made%2C%20Quest%20immediately%20places%20it%20on%20a%20queue%2C%20and%20we%20use%20the%20SQL%20Server%20JSON%20parsing%20capability%20to%20extract%20out%20the%20pieces%20of%20information%20from%20the%20message%20that%20we%20are%20interested%20in%2C%20such%20as%20the%20date%2C%20time%2C%20and%20amount%20of%20the%20transaction%2C%20and%20then%20stream%20it%20directly%20into%20our%20star%20schemas%20in%20Hyperscale%20for%20reporting%20so%20that%20the%20data%20is%20available%20to%20merchants%20right%20away.%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20This%20really%20sounds%20like%20a%20great%20solution.%20Can%20we%20see%20it%20in%20action%3F%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20Sure.%20So%20here%20is%20the%20Compass%20Online%20Reporting%20Suite.%20On%20the%20homepage%2C%20you%20get%20a%20quick%20summary%20of%20all%20of%20your%20activity%20from%20the%20last%20recent%20deposit%20through%20to%20trends%20in%20sales%20volume.%20From%20there%2C%20you%20can%20drill%20in%20to%20see%20the%20detail.%20I%E2%80%99ll%20click%20into%20one%20of%20my%20deposits.%20For%20example%2C%20one%20thing%20that%20is%20useful%20for%20merchants%20is%20being%20able%20to%20deconstruct%20a%20deposit%20that%20may%20just%20show%20up%20as%20one%20line%20item%20on%20their%20bank%20account%20statement.%20In%20Compass%2C%20they%20can%20drill%20in%20to%20see%20the%20detail%20behind%20the%20deposit%20whether%20that%E2%80%99s%20multiple%20terminals%20or%20multiple%20batches%20because%20Hyperscale%20is%20bringing%20all%20that%20data%20together.%20Now%20let%20me%20show%20you%20what%20happens%20as%20soon%20as%20the%20transaction%20is%20made.%20We%E2%80%99re%20in%20the%20transaction%20view%20within%20a%20batch.%20I%20can%20see%20all%20my%20recent%20transactions%20in%20this%20batch.%20Now%20I%E2%80%99ll%20show%20the%20process%20of%20running%20a%20transaction%20along%20with%20where%20we%20can%20view%20it.%20I%E2%80%99m%20in%20the%20virtual%20terminal%20built%20on%20Quest%2C%20but%20this%20could%20be%20a%20physical%20terminal%20or%20another%20integrated%20payment%20system.%20I%E2%80%99ll%20make%20a%20credit%20card%20transaction.%20And%20you%E2%80%99ll%20see%20it%20succeeds%20in%20the%20virtual%20terminal%20in%20under%20a%20second.%20Now%20I%E2%80%99ll%20hop%20over%20to%20Compass%20into%20our%20batch%20view%2C%20hit%20refresh%2C%20and%20you%E2%80%99ll%20see%20the%20payment%20is%20pending%20right%20away.%20That%E2%80%99s%20made%20possible%20by%20Hyperscale%20instantaneously%20consuming%20the%20data%20from%20our%20messaging%20queue.%20Finally%2C%20I%E2%80%99ll%20hop%20over%20to%20my%20query%20in%20SQL%20Server%20Management%20Studio%20and%20I%E2%80%99ll%20query%20this%20batch%20in%20our%20Hyperscale%20database%2C%20and%20you%E2%80%99ll%20see%20the%20results%20are%20also%20instantly%20available.%20Next%2C%20if%20we%20look%20into%20our%20list%20of%20settled%20transactions%2C%20you%E2%80%99ll%20see%20that%20once%20the%20merchant%20batches%20in%20at%20the%20end%20of%20the%20day%20if%20they%20click%20into%20a%20deposit%2C%20it%20shows%20the%20amounts%20and%20fees%20along%20with%20the%20net%20deposit%20amount%20and%20when%20payment%20will%20be%20posted.%20I%E2%80%99ll%20click%20in%20to%20see%20transactions%20in%20this%20batch.%20Transaction%20data%20is%20also%20enriched%20by%20the%20data%20from%20our%20micro-batch%20processing%2C%20as%20you%20can%20see%20here%20by%20optional%20column%20views%20available.%20I%E2%80%99ll%20select%20Product%20and%20POS%20Entry%20Mode.%20You%E2%80%99ll%20see%20that%20information%20show%20up%20in%20our%20view%20and%20these%20are%20critical%20elements%20that%20determine%20the%20fees%20for%20the%20transaction.%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20Okay%2C%20so%20this%20is%20an%20example%20then%20of%20just%20one%20transaction%2C%20but%20what%20if%20you%20have%20maybe%2050%2C000%20or%20more%20merchants%20that%20are%20running%20millions%20of%20transactions%20a%20day%3F%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20Yeah.%20We%20process%20around%201%20billion%20rows%20of%20data%20per%20month%20on%20our%20Hyperscale%20implementation%20right%20now.%20As%20you%20can%20see%20here%20in%20the%20portal%2C%20this%20has%20amounted%20to%20over%2028TB.%20And%20a%20big%20advantage%20of%20Hyperscale%20is%20that%20the%20compute%20is%20independent%20to%20the%20size%20of%20your%20data.%20Let%20me%20give%20you%20another%20view%20from%20our%20monitoring%20tool.%20Here%2C%20the%20green%20chart%20represents%20Hyperscale.%20It%E2%80%99s%20processing%20a%20large%20quantity%20of%20billing%20information%20as%20the%20micro-batch%20comes%20in%20at%20intervals.%20It%E2%80%99s%20actually%20spiking%20up%20to%20almost%20100%25%20utilization%2C%20which%20is%20around%20200%2C000%20IOPS%2C%20and%20incidentally%20this%20is%20only%20using%208%20cores%20right%20now%20which%20is%20a%20relatively%20small%20amount%20of%20compute.%20We%20can%20elastically%20scale%20compute%20when%20we%20need%20to%20process%20a%20large%20amount%20of%20data%20or%20build%20an%20index%20on%20a%20giant%20table%20with%20all%20the%20history%20and%20to%20scale%20it%20up.%20It%20only%20takes%20a%20few%20minutes.%20Also%20in%20Hyperscale%2C%20backups%20are%20near%20instantaneous.%20It%E2%80%99s%20constantly%20taking%20snapshots%20of%20your%20data%2C%20which%20helps%20significantly%20with%20data%20restore.%20Through%20PowerShell%2C%20we%E2%80%99ve%20automated%20restore%20when%20we%20do%20full-scale%20load%20testing%2C%20and%20that%20only%20takes%20like%20an%20hour%2C%20even%20though%20it%E2%80%99s%20a%2028TB%20database.%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20Okay%2C%20so%20how%20else%20then%20have%20you%20configured%20your%20Hyperscale%20implementation%3F%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20Secondary%20replicas.%20Not%20just%20read-only%20replicas%2C%20but%20named%20replicas%20have%20been%20game%20changing%20for%20us.%20This%20allowed%20us%20to%20keep%20our%20primary%20instance%20for%20our%20compute%20intensive%20ETL%20and%20data%20processing.%20We%20can%20then%20assign%20name%20replicas%20for%20specific%20purposes%20like%20analytics%20and%20reporting.%20These%20tap%20into%20a%20copy%20of%20the%20same%20data%2C%20but%20have%20their%20own%20dedicated%20compute.%20If%20you%20recall%2C%20when%20I%20showed%20you%20the%20slider%20screen%2C%20we%20had%20a%20secondary%20replica%20configured%20for%20customer%20traffic%2C%20which%20is%20part%20of%20a%20scalable%20pool%20of%20replicas%20for%20load%20balancing.%20If%20I%20switch%20tabs%2C%20you%20can%20see%20that%20we%20also%20have%20a%20named%20replica%20configured%20to%20handle%20analytics%20traffic.%20Routing%20traffic%20to%20these%20replicas%20is%20as%20simple%20as%20changing%20the%20connection%20string.%20This%20means%20that%20critical%20customer%20workloads%20are%20not%20interrupted%20by%20our%20compute%20intensive%20analytic%20operations.%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20So%20with%20such%20massive%20amounts%20of%20data%20that%20are%20coming%20in%2C%20beyond%20assigning%20the%20right%20amount%20of%20compute%2C%20are%20you%20doing%20anything%20special%20then%20to%20speed%20up%20query%20operations%3F%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20Yes%2C%20we%20do%20a%20few%20things.%20For%20example%2C%20one%20of%20the%20things%20we%E2%80%99ve%20really%20taken%20advantage%20of%20is%20clustered%20columnstore%20index%20for%20data%20aggregation.%20A%20big%20advantage%20of%20Azure%20SQL%20Hyperscale%20is%20that%20it%E2%80%99s%20very%20flexible.%20It%20can%20handle%20a%20variety%20of%20different%20types%20of%20traffic%20and%20does%20really%20rapid%20aggregation.%20Here%2C%20we%20have%20an%20example%20of%20an%20analytical%20workload%20where%20we%20are%20going%20to%20take%20a%20very%20large%20number%20of%20records%20and%20produce%20some%20aggregates.%20In%20this%20particular%20table%2C%20we%20have%20a%20little%20over%203%20billion%20rows%20of%20data.%20To%20see%20some%20trending%2C%20I%20have%20a%20query%20that%20will%20produce%20a%20time%20series%20by%20collecting%20data%20from%20different%20months%2C%20which%20in%20this%20case%2C%20are%20in%20different%20partitions%20in%20a%20clustered%20columnstore%20index.%20When%20I%20run%20it%2C%20you%20can%20see%20it%20is%20able%20to%20aggregate%20records%20across%20three%20months%20in%202021%2C%20and%20it%E2%80%99s%20almost%20instantaneous.%20Another%20way%20to%20look%20at%20the%20data%20is%20to%20look%20at%20a%20cross-section%20using%20dimensional%20slices%2C%20like%20a%20state.%20And%20here%2C%20we%20see%20the%20total%20number%20of%20transactions%20in%20New%20York%20and%20Washington.%20So%20we%E2%80%99re%20able%20to%20look%20across%20billions%20of%20rows%20of%20data%20and%20pull%20these%20aggregates%20together%20in%20fractions%20of%20a%20second.%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20That%E2%80%99s%20super%20impressive.%20And%20now%20you%E2%80%99ve%20transformed%20then%20your%20app%20and%20data%20stack%20for%20reporting%20with%20Azure%20SQL%20Database%20Hyperscale%2C%20but%20what%E2%80%99s%20next%20then%20for%20Clearent%20by%20Xplor%3F%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20So%20given%20our%20recent%20merger%20with%20the%20transaction%20services%20group%20to%20form%20Xplor%2C%20we%20are%20taking%20our%20Clearent%20platform%20international%20to%20deliver%20a%20service%20called%20Xplor%20Pay%20with%20Azure%20SQL%20Database%20Hyperscale%20as%20our%20reporting%20back-end.%20And%20now%20that%20we%E2%80%99ve%20got%20our%20data%20infrastructure%20to%20this%20level%20of%20maturity%2C%20the%20next%20logical%20step%20is%20to%20leverage%20Azure%20Machine%20Learning.%20This%20will%20help%20offer%20additional%20value-added%20services%20to%20our%20customers%2C%20such%20as%20helping%20them%20to%20understand%20when%20to%20run%20specific%20promotions%20based%20on%20existing%20sales.%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22jk%20jl%20gs%20jm%20b%20jn%20lg%20ir%20jp%20jq%20lh%20iv%20js%20jt%20li%20jv%20jw%20jx%20lj%20jz%20ka%20kb%20lk%20kd%20ke%20kf%20dp%20ho%22%20data-selectable-paragraph%3D%22%22%3E-%20Thanks%20so%20much%2C%20Zach%2C%20for%20joining%20us%20today%20and%20also%20giving%20us%20the%20deep%20dive%20on%20your%20Hyperscale%20implementation.%20In%20fact%2C%20to%20learn%20more%20about%20Clearent%20by%20Xplor%20and%20what%20they%E2%80%99re%20doing%20with%20Hyperscale%2C%20check%20out%2C%20aka.ms%2FClearentMechanics.%20And%20for%20more%20guidance%20on%20implementing%20Azure%20SQL%20Database%20Hyperscale%2C%20check%20out%20aka.ms%2FMechanicsHyperscale.%20Of%20course%2C%20don%E2%80%99t%20forget%20to%20subscribe%20to%20our%20channel%20if%20you%20haven%E2%80%99t%20already.%20Thanks%20so%20much%20for%20watching.%20We%E2%80%99ll%20see%20you%20next%20time.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-2323616%22%20slang%3D%22en-US%22%3E%3CP%3E%3CSPAN%3EHow%20to%20build%20a%20fast%2C%20scalable%20data%20system%20on%20Azure%20SQL%20Database%20Hyperscale.%26nbsp%3B%3C%2FSPAN%3E%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-2323616%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAzure%20SQL%20Databse%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Version history
Last update:
‎May 04 2021 01:40 PM
Updated by: