%3CLINGO-SUB%20id%3D%22lingo-sub-1063741%22%20slang%3D%22en-US%22%3EDecember%202019%20unified%20Azure%20SDK%20Release%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1063741%22%20slang%3D%22en-US%22%3E%3CP%3EWelcome%20to%20another%20release%20announcement%20for%20the%20unified%20Azure%20SDK.%26nbsp%3B%20This%20month%2C%20we%20have%20expanded%20our%20service%20support%20to%20include%20a%20preview%20of%20the%20Azure%20Storage%20DataLake%20SDK.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EGenerally%20available%20releases%3A%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3EAzure%20Storage%20Blobs%20(with%20batch%20support)%3C%2FLI%3E%0A%3CLI%3EAzure%20Storage%20Queues%3C%2FLI%3E%0A%3CLI%3EAzure%20Storage%20Files%20(SMB%20and%20other%20File%20shares)%3C%2FLI%3E%0A%3CLI%3EAzure%20Key%20Vault%20Secrets%20and%20Keys%3C%2FLI%3E%0A%3CLI%3EAzure%20Identity%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3EThese%20are%20ready%20to%20use%20in%20your%20production%20applications.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EPreview%20releases%3A%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3EAzure%20Storage%20DataLake%20Files%3C%2FLI%3E%0A%3CLI%3EAzure%20Key%20Vault%20Certificates%3C%2FLI%3E%0A%3CLI%3EAzure%20Event%20Hubs%2C%20including%20a%20simplified%20event%20processor%20module%3C%2FLI%3E%0A%3CLI%3EAzure%20App%20Configuration%3C%2FLI%3E%0A%3CLI%3EAzure%20Cosmos%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3EWe%20believe%20these%20are%20ready%20for%20your%20use%2C%20but%20not%20yet%20ready%20for%20production.%26nbsp%3B%20Between%20now%20and%20the%20GA%20release%2C%20these%20libraries%20may%20undergo%20API%20changes.%26nbsp%3B%20We'd%20love%20your%20feedback!%26nbsp%3B%20If%20you%20use%20these%20libraries%20and%20like%20what%20you%20see%2C%20or%20you%20want%20to%20see%20changes%2C%20let%20us%20know%20in%20the%20GitHub%20issues%20for%20the%20appropriate%20language.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId--1439458184%22%20id%3D%22toc-hId--1439458177%22%3E%3CSTRONG%3E%3CFONT%20size%3D%225%22%3EGetting%20Started%3C%2FFONT%3E%3C%2FSTRONG%3E%3C%2FH2%3E%0A%3CP%3EUse%20the%20links%20below%20to%20get%20started%20with%20your%20language%20of%20choice.%26nbsp%3B%20You%20will%20notice%20that%20all%20the%20preview%20libraries%20are%20tagged%20with%20%22preview%22.%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fazure.github.io%2Fazure-sdk%2Freleases%2F2019-12%2Fdotnet.html%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3E.NET%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fazure.github.io%2Fazure-sdk%2Freleases%2F2019-12%2Fjava.html%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3EJava%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fazure.github.io%2Fazure-sdk%2Freleases%2F2019-12%2Fjs.html%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3EJavaScript%20%2F%20TypeScript%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fazure.github.io%2Fazure-sdk%2Freleases%2F2019-12%2Fpython.html%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3EPython%3C%2FA%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%3CFONT%20size%3D%224%22%3EFor%20those%20of%20you%20who%20want%20to%20dive%20deep%20into%20the%20content%2C%20the%20release%20notes%20linked%20above%20and%20the%20change%20logs%20they%20point%20to%20give%20more%20details%20on%20what%20has%20changed.%3C%2FFONT%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId-1048054649%22%20id%3D%22toc-hId-1048054656%22%3E%3CSTRONG%3E%3CFONT%20size%3D%225%22%3EEvent%20Hubs%20Processor%3C%2FFONT%3E%3C%2FSTRONG%3E%3C%2FH2%3E%0A%3CP%3EEvent%20Hubs%20is%20a%20message%20passing%20service.%26nbsp%3B%20One%20side%20(the%20producer)%20sends%20events%20to%20the%20hub%20and%20another%20(the%20consumer)%20receives%20the%20events.%26nbsp%3B%20We've%20refactored%20the%20base%20Event%20Hubs%20library%20to%20better%20reflect%20this%20-%20there%20is%20now%20an%20EventHubConsumerClient%20and%20an%20EventHubProducerClient%20within%20the%20library.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EFor%20me%2C%20the%20more%20exciting%20change%20is%20the%20introduction%20of%20the%20Event%20Processor.%26nbsp%3B%20Writing%20scalable%20services%20that%20consume%20events%20is%20hard.%26nbsp%3B%20It%20needs%20to%20scale%20to%20multiple%20instances%2C%20and%20each%20instance%20must%20understand%20which%20events%20have%20been%20consumed%20already%20and%20which%20need%20to%20be%20consumed.%26nbsp%3B%20We've%20wrapped%20all%20the%20best%20practices%20for%20implementing%20scalable%20event%20handlers%20into%20a%20new%20client.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ELet's%20take%20a%20typical%20event%20processing%20service%20that%20needs%20to%20consume%20events%20coming%20in%20from%20an%20Event%20Hub.%26nbsp%3B%20In%20the%20past%2C%20you%20would%20have%20to%20set%20up%20a%20loop%20that%20received%20a%20batch%20of%20an%20appropriate%20size%2C%20then%20processed%20that%20batch.%26nbsp%3B%20At%20the%20end%20of%20the%20batch%2C%20it%20would%20write%20a%20checkpoint%20somewhere%2C%20then%20go%20on%20to%20receive%20the%20next%20batch.%26nbsp%3B%20If%20another%20service%20was%20started%2C%20it%20could%20assert%20control%20over%20the%20event%20queue%2C%20read%20the%20checkpoint%2C%20and%20continue%20processing.%26nbsp%3B%20There%20were%20several%20issues%20you%20need%20to%20resolve%2C%20and%20a%20lot%20of%20boiler-plate%20code.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EWith%20the%20new%20event%20processor%20library%2C%20we've%20solved%20most%20of%20this%2C%20allowing%20you%20to%20concentrate%20on%20just%20two%20things%3A%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3EWhat%20do%20I%20need%20to%20do%20to%20process%20and%20event%3F%3C%2FLI%3E%0A%3CLI%3EWhen%20should%20I%20write%20a%20checkpoint%3F%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3EYou%20can%20write%20a%20checkpoint%20after%20each%20event%20or%20a%20pre-determined%20time.%26nbsp%3B%20Let's%20look%20at%20the%20code%20for%20a%20typical%20event%20processor%20with%20the%20new%20library%20(this%20service%20is%20written%20in%20JavaScript)%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-javascript%22%3E%3CCODE%3Eimport%20%7B%20EventHubConsumerClient%2C%20CheckpointStore%20%7D%20from%20%22%40azure%2Fevent-hubs%22%3B%0Aimport%20%7B%20ContainerClient%20%7D%20from%20%22%40azure%2Fstorage-blob%22%3B%0Aimport%20%7B%20BlobCheckpointStore%20%7D%20from%20%22%40azure%2Feventhubs-checkpointstore-blob%22%3B%0A%0Aconst%20containerClient%20%3D%20new%20ContainerClient(storageConnectionString%2C%20storageContainerName)%3B%0Aconst%20checkpointStore%20%3A%20CheckpointStore%20%3D%20new%20BlobCheckpointStore(containerClient)%3B%0Aconst%20eventHubConsumerClient%20%3D%20new%20EventHubConsumerClient(consumerGroupName%2C%20ehConnectionString%2C%20eventHubName)%3B%0A%0Aconst%20subscription%20%3D%20eventHubConsumerClient.subscribe(%0A%20%20partitionId%2C%20%7B%0A%20%20%20%20%2F%2F%20In%20V5%20we%20deliver%20events%20in%20batches%2C%20rather%20than%20a%20single%20message%20at%20a%20time.%0A%20%20%20%20%2F%2F%20You%20can%20control%20the%20batch%20size%20via%20the%20options%20passed%20to%20the%20client.%0A%20%20%20%20%2F%2F%0A%20%20%20%20%2F%2F%20If%20your%20callback%20is%20an%20async%20function%20or%20returns%20a%20promise%2C%20it%20will%20be%20awaited%20before%20the%0A%20%20%20%20%2F%2F%20callback%20is%20called%20for%20the%20next%20batch%20of%20events.%20%0A%20%20%20%20processEvents%3A%20(events%2C%20context)%20%3D%26gt%3B%20%7B%20%2F**%20your%20code%20here%20**%2F%20%7D%2C%0A%0A%20%20%20%20%2F%2F%20Prior%20to%20V5%20errors%20were%20handled%20by%20separate%20callbacks%20depending%20%0A%20%20%20%20%2F%2F%20on%20where%20they%20were%20thrown%20i.e%20when%20managing%20different%20partitions%20vs%20receiving%20from%20each%20partition.%0A%20%20%20%20%2F%2F%20%0A%20%20%20%20%2F%2F%20In%20V5%20you%20only%20need%20a%20single%20error%20handler%20for%20all%20of%20those%20cases.%0A%20%20%20%20processError%3A%20(error%2C%20context)%20%3D%26gt%3B%20%7B%20%0A%20%20%20%20%20%20if%20(context.partitionId)%20%7B%0A%20%20%20%20%20%20%20%20console.log(%22Error%20when%20receiving%20events%20from%20partition%20%25s%3A%20%25O%22%2C%20context.partitionId%2C%20error)%0A%20%20%20%20%20%20%7D%20else%20%7B%0A%20%20%20%20%20%20%20%20console.log(%22Error%20from%20the%20consumer%20client%3A%20%25O%22%2C%20error)%3B%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%7D)%3B%0A%20%20%0Aawait%20subscription.close()%3B%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EIf%20you%20use%20the%20EventProcessorHost%20previously%2C%20check%20out%20the%20%3CA%20href%3D%22https%3A%2F%2Fgithub.com%2FAzure%2Fazure-sdk-for-js%2Fblob%2Fmaster%2Fsdk%2Feventhub%2Fevent-hubs%2Fmigrationguide.md%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EMigration%20Guide%3C%2FA%3E%20for%20converting%20your%20code%20to%20the%20new%20event%20proecssor.%26nbsp%3B%20Of%20course%2C%20if%20you%20need%20even%20more%20fine-tuned%20control%2C%20the%20lower%20level%20library%20is%20available%20for%20this.%26nbsp%3B%20We're%20still%20working%20on%20this%20preview%20library%2C%20so%20please%20do%20give%20us%20feedback%20on%20how%20you%20like%20(or%20don't%20like)%20this%20approach.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId--759399814%22%20id%3D%22toc-hId--759399807%22%3EWorking%20with%20us%20and%20giving%20Feedback%3C%2FH2%3E%0A%3CP%3ESo%20far%2C%20the%20community%20has%20filed%20hundreds%20of%20issues%20against%20these%20new%20SDKs%20with%20feedback%20ranging%20from%20documentation%20issues%20to%20API%20surface%20area%20change%20requests%20to%20pointing%20out%20failure%20cases.%20Please%20keep%20that%20coming.%20We%20work%20in%20the%20open%20on%20GitHub%20and%20you%20can%20submit%20issues%20here%3A%3C%2FP%3E%0A%3CUL%20style%3D%22margin-left%3A%20.375in%3B%20direction%3A%20ltr%3B%20unicode-bidi%3A%20embed%3B%20margin-top%3A%200in%3B%20margin-bottom%3A%200in%3B%22%20type%3D%22disc%22%3E%0A%3CLI%20style%3D%22margin-top%3A%200%3B%20margin-bottom%3A%200%3B%20vertical-align%3A%20middle%3B%22%3E%3CA%20href%3D%22https%3A%2F%2Fgithub.com%2FAzure%2Fazure-sdk%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EAPI%20design%20Guidelines%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%20style%3D%22margin-top%3A%200%3B%20margin-bottom%3A%200%3B%20vertical-align%3A%20middle%3B%22%3E%3CA%20href%3D%22https%3A%2F%2Fgithub.com%2FAzure%2Fazure-sdk-for-net%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3E.NET%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%20style%3D%22margin-top%3A%200%3B%20margin-bottom%3A%200%3B%20vertical-align%3A%20middle%3B%22%3E%3CA%20href%3D%22https%3A%2F%2Fgithub.com%2FAzure%2Fazure-sdk-for-java%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EJava%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%20style%3D%22margin-top%3A%200%3B%20margin-bottom%3A%200%3B%20vertical-align%3A%20middle%3B%22%3E%3CA%20href%3D%22https%3A%2F%2Fgithub.com%2FAzure%2Fazure-sdk-for-js%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EJavaScript%2FTypeScript%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%20style%3D%22margin-top%3A%200%3B%20margin-bottom%3A%200%3B%20vertical-align%3A%20middle%3B%22%3E%3CA%20href%3D%22https%3A%2F%2Fgithub.com%2FAzure%2Fazure-sdk-for-python%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EPython%3C%2FA%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3EFinally%2C%20make%20sure%20to%20follow%20us%20on%20twitter%3A%20%3CA%20href%3D%22https%3A%2F%2Ftwitter.com%2Fazuresdk%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3E%40azuresdk%3C%2FA%3E%20where%20you%20can%20find%20the%20latest%20news%20and%20announcements%20and%20interact%20directly%20to%20the%20Azure%20SDKs%20team%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-1063741%22%20slang%3D%22en-US%22%3E%3CP%3EWelcome%20to%20another%20release%20announcement%20for%20the%20unified%20Azure%20SDK.%26nbsp%3B%20This%20month%2C%20we%20have%20expanded%20our%20service%20support%20to%20include%20a%20preview%20of%20the%20Azure%20Storage%20DataLake%20SDK.%3C%2FP%3E%3C%2FLINGO-TEASER%3E
Microsoft

Welcome to another release announcement for the unified Azure SDK.  This month, we have expanded our service support to include a preview of the Azure Storage DataLake SDK.

 

Generally available releases:

  • Azure Storage Blobs (with batch support)
  • Azure Storage Queues
  • Azure Storage Files (SMB and other File shares)
  • Azure Key Vault Secrets and Keys
  • Azure Identity

These are ready to use in your production applications.

 

Preview releases:

  • Azure Storage DataLake Files
  • Azure Key Vault Certificates
  • Azure Event Hubs, including a simplified event processor module
  • Azure App Configuration
  • Azure Cosmos

We believe these are ready for your use, but not yet ready for production.  Between now and the GA release, these libraries may undergo API changes.  We'd love your feedback!  If you use these libraries and like what you see, or you want to see changes, let us know in the GitHub issues for the appropriate language. 

 

Getting Started

Use the links below to get started with your language of choice.  You will notice that all the preview libraries are tagged with "preview".

For those of you who want to dive deep into the content, the release notes linked above and the change logs they point to give more details on what has changed.

 

Event Hubs Processor

Event Hubs is a message passing service.  One side (the producer) sends events to the hub and another (the consumer) receives the events.  We've refactored the base Event Hubs library to better reflect this - there is now an EventHubConsumerClient and an EventHubProducerClient within the library.

 

For me, the more exciting change is the introduction of the Event Processor.  Writing scalable services that consume events is hard.  It needs to scale to multiple instances, and each instance must understand which events have been consumed already and which need to be consumed.  We've wrapped all the best practices for implementing scalable event handlers into a new client.

 

Let's take a typical event processing service that needs to consume events coming in from an Event Hub.  In the past, you would have to set up a loop that received a batch of an appropriate size, then processed that batch.  At the end of the batch, it would write a checkpoint somewhere, then go on to receive the next batch.  If another service was started, it could assert control over the event queue, read the checkpoint, and continue processing.  There were several issues you need to resolve, and a lot of boiler-plate code.

 

With the new event processor library, we've solved most of this, allowing you to concentrate on just two things:

  • What do I need to do to process and event?
  • When should I write a checkpoint?

You can write a checkpoint after each event or a pre-determined time.  Let's look at the code for a typical event processor with the new library (this service is written in JavaScript):

 

import { EventHubConsumerClient, CheckpointStore } from "@azure/event-hubs";
import { ContainerClient } from "@azure/storage-blob";
import { BlobCheckpointStore } from "@azure/eventhubs-checkpointstore-blob";

const containerClient = new ContainerClient(storageConnectionString, storageContainerName);
const checkpointStore : CheckpointStore = new BlobCheckpointStore(containerClient);
const eventHubConsumerClient = new EventHubConsumerClient(consumerGroupName, ehConnectionString, eventHubName);

const subscription = eventHubConsumerClient.subscribe(
  partitionId, {
    // In V5 we deliver events in batches, rather than a single message at a time.
    // You can control the batch size via the options passed to the client.
    //
    // If your callback is an async function or returns a promise, it will be awaited before the
    // callback is called for the next batch of events. 
    processEvents: (events, context) => { /** your code here **/ },

    // Prior to V5 errors were handled by separate callbacks depending 
    // on where they were thrown i.e when managing different partitions vs receiving from each partition.
    // 
    // In V5 you only need a single error handler for all of those cases.
    processError: (error, context) => { 
      if (context.partitionId) {
        console.log("Error when receiving events from partition %s: %O", context.partitionId, error)
      } else {
        console.log("Error from the consumer client: %O", error);
      }
    }
});
  
await subscription.close();

 

If you use the EventProcessorHost previously, check out the Migration Guide for converting your code to the new event proecssor.  Of course, if you need even more fine-tuned control, the lower level library is available for this.  We're still working on this preview library, so please do give us feedback on how you like (or don't like) this approach.

 

Working with us and giving Feedback

So far, the community has filed hundreds of issues against these new SDKs with feedback ranging from documentation issues to API surface area change requests to pointing out failure cases. Please keep that coming. We work in the open on GitHub and you can submit issues here:

Finally, make sure to follow us on twitter: @azuresdk where you can find the latest news and announcements and interact directly to the Azure SDKs team