Featured Blog

[Guest Blog] HBCU Student Ambassador Voices: My Journey in Trusting the Process

justtrent — Fri, 4 Sep 2020 14:38:29 PDT

This article was written by Microsoft Ignite 2020 Humans of IT student ambassador Trent Dalcourt, a Computer Science junior from Dillard University. This is the fourth article in a series featuring this year's student ambassadors. To learn more about our student ambassadors hailing from five Historically Black Colleges and Universities (HBCUs) this year, click here. In this article, Trent shares about his personal experiences, inspiration drivers, and journey as a computer science major and #FutureTechnologist.

If I could leave ONE quote behind as a lesson for someone else reading this, it would be: “You have to learn to crawl before you learn to walk”. As simple as it sounds, this quote can be applied figuratively to several situations in life. It enables one to be conscious of the progress towards success while learning humility. It also allows you to learn from yourself while knowing that everything happens for a reason. In fact, this quote has instilled hope and motivation, leading me to strive to be the first in my family to successfully attain a college degree.

For as long as I can remember, I have been on the stride towards success. In my household of 7, my parents have always emphasized the value in education. Thanks to them, I've followed that path over the years. My mother was a program coordinator for a nonprofit organization called “Save The Children”. This same program became an extremely beneficial after-school program at my own school - it made me hone my reading and writing skills. It also encouraged students to develop skills in all areas of STEM. In addition to my interest in video games, going to this program for several school years sparked my interest and passion for technology, which, in turn, led to me dreaming of one day going to college and graduating as a first-generation college student.

Before being accepted into one of the first HBCUs in Louisiana, Dillard University, I had a hard time choosing what exactly I wanted to major in. No one in my family had graduated from college, and I was reluctant to just choose something that I would not be interested in. Also knowing that knowledge is power, I wanted to major in something that I knew would pay the bills and could eventually help me support myself and my family. This led me to major in computer science, with aspirations of one day becoming an IT professional.

In my first year of college, I got to attend my first hackathon and learn new programming languages. I also joined IT-related organizations on campus and made some new friends. My second year consisted of me working towards my goal of becoming a better student, and leader. I improved my grade point average which earned me a spot on the Dean’s List, and I got my first internship by the end of the year. I even traveled to New York for the very first time to attend a conference hosted for tech students! My third year, although I'm only just starting, is already off to a great start. College has given me such a genuinely rich experience that I will be forever grateful for.

I have taken many classes that have helped shape my perspective and knowledge as a student and young adult. I am definitely relishing where I am now in life, and appreciate working hard while letting my work ethic speak for itself. While my undergraduate career has not been easy in the slightest, having a growth mindset helps in knowing that you have to remain consistently resilient and not give up, or else all of the hard work you put in goes to waste. This brings me back to the quote mentioned earlier, about learning to crawl before you walk.

As a Microsoft Human of IT student ambassador, I am super excited to see where I go next in my path towards becoming an IT Professional. In this blog post, I could have listed more failures or doubts of mine. I could have even listed all of the times that I considered given up on my own future. However, I want to make it clear that it is important to know that everything you say about your destiny, your legacy, has meaning and power behind it. If I were to focus on all the shortcomings I had, I would certainly not be where I am now.

Just imagine the people that are looking up to you as a role model, and how much impact you could have in inspiring them to keep pushing on. No matter who you are, where you are from or how tough the journey is - keep taking it step by step, and day by day. Keep working towards your goals, chipping away at it diligently and who knows, maybe one day it will all be worth it.

I know that I have a long way to go, but I also know that I WILL make it. To all of you out there reading this, remember to trust the process. You've got this.

#HumansofIT

#StudentAmbassadors

Coming soon: Video Hub

Anna Chu — Fri, 4 Sep 2020 14:17:16 PDT

I am excited to share some information about a new resource coming to the Microsoft Tech Community. We have been working to create a home for new videos and interactive guides to help you work effectively with our products, and I am pleased to announce that this new resource is almost ready!

On Sept, 22^nd we will launch the Microsoft Tech Community Video Hub, initially with content to support Microsoft Ignite, but it will be the future home for Tech Community videos to help you learn everything you need to know about our major products.

Mock up of the New Video Hub

The new Video Hub will be the place to view videos, click through our interactive guides (a click-through demo experience) and join conversation about this content with fellow community members. We will also show, where available, who the subject matter experts are and, if this video is one of a series, we will provide a link to the next video in that series.

We would love for you to tell us how much you enjoyed the video using the ‘rate this video’ function on the right hand side. We will also work with our teams to show a list of video resources which connect you to other places to continue your learning or just next steps.

When the Video Hub goes live, on Sept 22^nd, you will be able to access it here: https://aka.ms/VideoHub.

Azure Data Explorer Kafka Connector - new features with version 1.x

Anagha-Microsoft — Fri, 4 Sep 2020 13:25:24 PDT

Azure Data Explorer has a mature, open source Kafka Connect sink connector that is in use in production by multiple customers. The connector was *gold* certified in September 2020 by Confluent and is now listed in Confluent Connector Hub. The certification involved stringent testing by Confluent for quality, standards, best practices, feature completeness, delivery guarantees and performance. To meet the certification requirements, we added a number of new features to our connector, renamed some, and removed support for some with version 1.x. This article details the same.

For comprehensive information about our connector, visit our Git repo.

Improved delivery guarantees

Our connector now offers "At least once" delivery guarantees (no data loss).

What's new

1. Comprehensive upfront validation

The connector validates for all required and dependent sink properties, access and grants and shuts down upfront with appropriate error messages.

2. Configurable behavior on error

A new sink property with possible values of fail (shut down connector task), ignore (ignore and continue processing), and log (log errors and continue processing).

behavior.on.error

3. *New* configurable retries

A set of new sink properties-

errors.retry.max.time.ms
errors.retry.backoff.time.ms

errors.retry.max.time.ms: Period of time in milliseconds to retry for transient errors

errors.retry.backoff.time.ms: Period of time in milliseconds to backoff before retry for transient errors

4. Additional converters supported

We have added support for the following converters-

org.apache.kafka.connect.json.JsonConverter
io.confluent.connect.avro.AvroConverter
io.confluent.connect.json.JsonSchemaConverter

JsonSchemaConverter is for Json with schema.

4. Schema registry support

The connector now supports schema registry with the property-

value.converter.schema.registry.url

We support schema registry for avro and json, protobuf is on the roadmap.

5. Schema with message

The connector now supports messages with schema with the following sink property that accepts a boolean-

value.converter.schemas.enable

6. Kafka Connect deadletterqueue

The optional sink properties below can be provided for Kafka Connect to redirect messages that failed deserialization-

errors.deadletterqueue.bootstrap.servers
errors.deadletterqueue.topic.name
errors.deadletterqueue.security.protocol
errors.deadletterqueue.sasl.mechanism
errors.deadletterqueue.sasl.jaas.config
errors.deadletterqueue.sasl.kerberos.service.name

The last 4 sink properties are required for secure clusters and*kerberos* property is specific to kerberized Kafka clusters.

7. Miscellaneous deadletterqueue

The optional sink properties below can be provided for the connector to redirect messages that failed besides deserialization, e.g. transient failures beyond retry period -

misc.deadletterqueue.bootstrap.servers
misc.deadletterqueue.topic.name
misc.deadletterqueue.security.protocol
misc.deadletterqueue.sasl.mechanism
misc.deadletterqueue.sasl.jaas.config
misc.deadletterqueue.sasl.kerberos.service.name

The last 4 sink properties are required for secure clusters and*kerberos* property is specific to kerberized Kafka clusters.

What's renamed

For the purpose of standardization and consistency, we have renamed some properties-

Previous name	New name
kusto.auth.authority	aad.auth.authority
kusto.auth.appid	aad.auth.appid
kusto.auth.appkey	aad.auth.appkey
kusto.tables.topics_mapping	kusto.tables.topics.mapping
kusto.sink.tempdir	tempdir.path
kusto.sink.flush_interval_ms	flush.interval.ms
kusto.sink.flush_size	flush.size.bytes

What's removed

The following properties are no longer supported with version 1.x-

Property name	Rationale
kusto.auth.username	Customers typically only use application IDs/service principals; We therefore removed support for user principals
kusto.auth.password	Customers typically only use application IDs/service principals; We therefore removed support for user principals
eventDataCompression	We now compress all data shipped to Azure Data Explorer, to improve performance

Connector documentation

Visit out Git repo

Connector download

Visit out Git repo release page

Confluent Connector Hub

https://www.confluent.io/hub/microsoftcorporation/kafka-sink-azure-kusto

Need support?

Ran into a bug? Need hep getting started? Have a feature request? Open an issue.

Want to contribute?

Read our guidelines and get involved. We welcome community contribution.

SQL Server IAM page

Liwei — Fri, 4 Sep 2020 12:43:12 PDT

Hi Guys, welcome to SQL Server Allocation Series, hope you enjoy the post1 and post2.

Today, I’m going to show you the detail of IAM page.

An Index Allocation Map (IAM) page maps the extents in a 4-GB part of a database file used by an allocation unit. An allocation unit is one of three types:

IN_ROW_DATA
Holds a partition of a heap or index.
LOB_DATA
Holds large object (LOB) data types, such as XML, VARBINARY(max), and VARCHAR(max).
ROW_OVERFLOW_DATA
Holds variable length data stored in VARCHAR, NVARCHAR, VARBINARY, or SQL_VARIANT columns that exceed the 8,060 byte row size limit.

Each partition of a heap or index contains at least an IN_ROW_DATA allocation unit. It may also contain a LOB_DATA or ROW_OVERFLOW_DATA allocation unit, depending on the heap or index schema.

An IAM page covers a 4-GB range in a file and is the same coverage as a GAM or SGAM page. If the allocation unit contains extents from more than one file, or more than one 4-GB range of a file, there will be multiple IAM pages linked in an IAM chain. Therefore, each allocation unit has at least one IAM page for each file on which it has extents. There may also be more than one IAM page on a file, if the range of the extents on the file allocated to the allocation unit exceeds the range that a single IAM page can record.

IAM pages linked in a chain per allocation unit An IAM page has a header that indicates the starting extent of the range of extents mapped by the IAM page. The IAM page also has a large bitmap in which each bit represents one extent. The first bit in the map represents the first extent in the range, the second bit represents the second extent, and so on. If a bit is 0, the extent it represents is not allocated to the allocation unit owning the IAM. If the bit is 1, the extent it represents is allocated to the allocation unit owning the IAM page.

When the SQL Server Database Engine has to insert a new row and no space is available in the current page, it uses the IAM and PFS pages to find a page to allocate, or, for a heap or a Text/Image page, a page with sufficient space to hold the row. The SQL Server Database Engine uses the IAM pages to find the extents allocated to the allocation unit. For each extent, the SQL Server Database Engine searches the PFS pages to see if there is a page that can be used. Each IAM and PFS page covers lots of data pages, so there are few IAM and PFS pages in a database. This means that the IAM and PFS pages are generally in memory in the SQL Server buffer pool, so they can be searched quickly. For indexes, the insertion point of a new row is set by the index key, but when a new page is needed, the previously described process occurs.

In post 1, I created a table with 8 pages, all these 8 pages are in mixed extent. I’m going to insert more rows into the table to increase the pages. As you expected, these new rows/pages will be allocated to uniform extents.

I’m going to walk you through how SQL Server save the info in IAM pages.

Data manipulation:

T-SQL:

insert heaptable1 values(15,REPLICATE('o',8000)) --page 9

insert heaptable1 values(16,REPLICATE('p',8000)) --page 10

insert heaptable1 values(17,REPLICATE('q',8000)) --page 11

insert heaptable1 values(18,REPLICATE('r',8000)) --page 12

insert heaptable1 values(19,REPLICATE('s',8000)) --page 13

insert heaptable1 values(20,REPLICATE('t',8000)) --page 14

insert heaptable1 values(21,REPLICATE('u',8000)) --page 15

insert heaptable1 values(22,REPLICATE('v',8000)) --page 16

insert heaptable1 values(23,REPLICATE('w',8000)) --page 17

The table heaptable1 has 17 pages in four extents, two mixed extents and two uniform extents.

select allocated_page_file_id as [FileID],allocated_page_page_id as [PageID],page_type_desc,extent_page_id/8 as ExtentID,

is_mixed_page_allocation,extent_page_id as [First Page in Extent],extent_page_id+7 as [LastPage in Extent],is_allocated From

sys.dm_db_database_page_allocations(db_id(),object_id('dbo.heaptable1'),null,null,'detailed') order by allocated_page_page_id

Page 246 is the IAM page.

Page 245,247,328,329,330,331,332 and 333 are in mixed extent.

Starting from page (1:352), all the pages are allocated in uniform extent.

Page(1:352)~(1:359) are in extent 44,

Page(1:360)~(1:367) are in extent 45.

Let me run DBCC PAGE of IAM page to show you how the info recorded.

The first 8 pages are listed under ‘Single page allocations’, which means they are in mixed extent.

(1:352)~(1:360) are in uniform extent, the result matches the DMF sys.dm_db_database_page_allocations

(1:352) is the first page of extent 44

(1:360) is the first page of extent 45

Let’s dive deep into the IAM page with parameter 1:

1.There are two slots in IAM page,

The first slot(slot 0) are for pages in Mixed extent.
The second slot(slot 1) are for pages in Uniform extent.

2.Let me interpret the slot 0.

1)Each string in rectangle stands for one page, the format is PageId:FileId.

2)The first 8 characters are page id, the last four part is for Fileid.

Here is the interpretation table:

Original string->	PId:FileId->	FId:PId->	FId:PId->	FId:PId(Decimal)
f50000000100	f5000000:0100	0100: f5000000	0001:000000f5	1:245
f70000000100	f7000000:0100	0100:f7000000	0001:000000f7	1:247
480100000100	48010000:0100	0100:48010000	0001:00000148	1:328
490100000100	49010000:0100	0100:49010000	0001:00000149	1:329
4a0100000100	4a010000:0100	0100:4a010000	0001:0000014a	1:330
4b0100000100	4b010000:0100	0100:4b010000	0001:0000014b	1:331
4c0100000100	4c010000:0100	0100:4c010000	0001:0000014c	1:332
4d0100000100	4d010000:0100	0100:4d010000	0001:0000014d	1:333

3.I have demonstrate how the IAM page stores the first 8 pages in mixed extent, now let’s move on to the uniform extent.

4.The uniform extents info is stored in the second slot(slot 1).

1)’0000381f’ is reserved, we don’t need to worry about it.

2)The string I cared about is ‘00000000 0030’

Following table interprets the string.

3)Each bit in the table stands for one extent.

4)0 means the extent is not occupied/allocated by this table of the IAM.

5)1 means the extent is occupied/allocated by this table of IAM, and it’s uniform extent.

6)Here is more detail

Which exactly as same as the result we got from DBCC Page with parameter 3 before.

You may download the dbtest20200904.zip and give it a try.

Video Tutorial: Install Application Integration - Application Deployment Part 17

Yvette O'Meally — Fri, 4 Sep 2020 12:01:00 PDT

Hello everyone, here is part 17 of a series focusing on Application Deployment in Configuration Manager. This series is recorded by @Steve Rachui, a Microsoft principal premier field engineer.

This session focuses on the Install Application option for rapidly deploying applications to device(s). The tutorial walks through what this component is, how to configure it and demonstrates its use.

Posts in the series

Go straight to the playlist