Forum Discussion
Azure Devops and Data Factory
I have started a new job and taken over ADF. I know how to use Devops to integrate and deploy when everything is up and running. The problem is, it's all out of sync. I need to learn ADO/ADF as they work together so I can fix this. Any recommendations on where to start? Everything on YouTube is starting with a fresh environment which I'd be fine with. I'm not new to ADO, but I've never been the setup guy before. And I'm strong on ADO management, just using it.
Here are some of the problems I have:
- A lot of work has been done directly in the DEV branch rather than creating feature branches. Setting up a pull request from DEV to PROD wants to pull everything. Even in-progress or abandoned code changes.
- Some changes were made in the PROD branch directly, so I'll need to pull those changes back to DEV. We have valid changes in both DEV and PROD.
- I'm having trouble cherry-picking. It only lets me select one commit, then says I need to use command-line. It doesn't tell me the error. I don't know what tool to use for the command line.
- I've tried using Visual Studio, and I can pull in the Data Factory code, but have all the same problems there.
I'm not looking for an answer to the questions, but how to find the answer to these questions. Is this Data Factory, or should I be looking at Devops? I'm having no trouble managing the database code or Power BI in Devops, but I created that fresh.
Thanks for any help!
Hi!
It sounds like the main problem here is to clean a chaotic git repository. So you should look up how to address certain problems using the options git offers.
Some solutions you could look up further:For 1.: Create a new branch from dev, clean the in-progress or abandoned code changes using an IDE like vscode and merge the new branch into prod.
For 2.: If the main branch should be prod, I would concentrate to get prod clean. As you only want to merge from dev to prod, there should be no need to do anything. You only need to take action if you want to remove made changes from prod.
For 3.: Try using the command line, and look into the git commands (in general very useful). You can use git bash, PowerShell, or the built-in terminal in VS Code. If you need multiple commits, try:
git cherry-pick <commit-hash1> <commit-hash2>
If that fails, running "git status" or checking the error message can give clues on what’s wrong.
For 4.: Vscode is a great editor imo, I would use it to address the problems.
Hope that helps, best regards and luck!
4 Replies
Regarding cherry-picking, using Github Desktop helps (Azure Devops is very limited in this regard).
Hi!
It sounds like the main problem here is to clean a chaotic git repository. So you should look up how to address certain problems using the options git offers.
Some solutions you could look up further:For 1.: Create a new branch from dev, clean the in-progress or abandoned code changes using an IDE like vscode and merge the new branch into prod.
For 2.: If the main branch should be prod, I would concentrate to get prod clean. As you only want to merge from dev to prod, there should be no need to do anything. You only need to take action if you want to remove made changes from prod.
For 3.: Try using the command line, and look into the git commands (in general very useful). You can use git bash, PowerShell, or the built-in terminal in VS Code. If you need multiple commits, try:
git cherry-pick <commit-hash1> <commit-hash2>
If that fails, running "git status" or checking the error message can give clues on what’s wrong.
For 4.: Vscode is a great editor imo, I would use it to address the problems.
Hope that helps, best regards and luck!
Thank you. You are correct that I'm trying to clean a chaotic repository. I have both VSCode and git bash installed. Would learning the git commands allow me to do everything I need? It feels like the command line will be easier to learn. I get so many errors whenever I try to do anything from previous mistakes that I'm having trouble navigating VSCode.
I was able to speak with the previous data factory developer and he's been doing all his deployments outside Devops. He downloads the code to his local file system in ADF, then uploads it to prod. A lot of the conflicts are probably nearly identical files. What I'm thinking is that I'll download dev and copy it off somewhere so I don't accidentally lose changes. Then I can push prod over to dev and force prod to be the version used. I know basically what has not been deployed yet and I can then add that back in from my copy.
Happy to help!
And yes, (from what I know) learning the git commands and understanding how that works should be sufficient to do everything you need.
From my experience, I would stick to using the command line, there I had the fewest problems with.
For comparing files, you can use vscode to compare files or even branches (with the gitlens extension). Alternatively you can use difftools (there are many of them), which highlight the difference (also possible on on folder level with some tools).And for the rest - I would use a local copy of the repository and switch to the branches to make changes in the files, and commit & push them. Also merge and push is possible using the command line in your local repository.
Best regards!
