SOLVED

Auto Create SQL table from Imported CSV in ADF?

%3CLINGO-SUB%20id%3D%22lingo-sub-2641411%22%20slang%3D%22en-US%22%3EAuto%20Create%20SQL%20table%20from%20Imported%20CSV%20in%20ADF%3F%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2641411%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20All%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EWondering%20if%20its%20possible%20to%20automatically%20create%20a%20sql%20table%20from%20an%20imported%20CSV%20in%20Data%20Factory%3F%26nbsp%3B%20To%20make%20things%20a%20little%20more%20complicated%20not%20all%20csv's%20will%20have%20the%20same%20headings%20and%20differing%20number%20of%20headings.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EIf%20so%2C%20can%20anyone%20point%20me%20in%20the%20direction%20of%20a%20how%20to%20guide%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ECase%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI've%20been%20ingesting%20csv's%20using%20a%20HTTP%20connector%20in%20ADF%20then%20storing%20CSV%20data%20into%20a%20SQL%20table%20(manually%20created)%20and%20then%20transforming%20and%20cleaning%20said%20data%20into%20a%20Datastore%20SQL%20table%20that%20is%20also%20manually%20created.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20know%20I'm%20a%20little%20slow%20to%20the%20party%20but%20I've%20been%20looking%20at%20using%20parameters%20and%20was%20wondering%20if%20I%20pulled%20the%20csv's%20into%20blob%20storage%20and%20then%20transformed%20from%20there.%26nbsp%3B%20Then%20I'd%20only%20need%20to%20create%20one%20SQL%20table%20and%20if%20I%20could%20automate%20the%20entire%20process%20in%20ADF%20that%20would%20save%20future%20me%20a%20lot%20of%20time.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20have%20another%20issue%20but%20I'll%20post%20that%20separate%20as%20its%20a%20slightly%20different%20topic.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThanks%20for%20Reading%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EHope%20someone%20can%20point%20me%20in%20the%20right%20direction.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EJohn%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-2641411%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAzure%20Data%20Factory%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EAzure%20ETL%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ECopy%20Activity%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EMapping%20Data%20Flows%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2653091%22%20slang%3D%22en-US%22%3ERe%3A%20Auto%20Create%20SQL%20table%20from%20Imported%20CSV%20in%20ADF%3F%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2653091%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F239814%22%20target%3D%22_blank%22%3E%40John%20Dorrian%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EWell%20that%20was%20easier%20than%20I%20thought%2C%20see%20link.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fanswers%2Fquestions%2F35570%2Fcreate-a-target-table-on-the-fly-in-data-factory.html%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fanswers%2Fquestions%2F35570%2Fcreate-a-target-table-on-the-fly-in-data-factory.html%3C%2FA%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E
Occasional Contributor

Hi All

 

Wondering if its possible to automatically create a sql table from an imported CSV in Data Factory?  To make things a little more complicated not all csv's will have the same headings and differing number of headings.

 

If so, can anyone point me in the direction of a how to guide?

 

Case 

 

I've been ingesting csv's using a HTTP connector in ADF then storing CSV data into a SQL table (manually created) and then transforming and cleaning said data into a Datastore SQL table that is also manually created.

 

I know I'm a little slow to the party but I've been looking at using parameters and was wondering if I pulled the csv's into blob storage and then transformed from there.  Then I'd only need to create one SQL table and if I could automate the entire process in ADF that would save future me a lot of time.

 

I have another issue but I'll post that separate as its a slightly different topic.

 

Thanks for Reading

 

Hope someone can point me in the right direction.

 

John

1 Reply
best response confirmed by John Dorrian (Occasional Contributor)