Hello Everyone! Today I would like to share with you a very important security topic. Did you ever push password or secrets creds to GitHub by accident? Did you ever wish if there is a way to block your commit or warning you that there are sensitive creds in your code?
Today many developers and Open-Source communities use GitHub to collaborate and store their code. There are many security best practice articles already published and we can find it easy by any search engine. I am not going to talk about best practice today.
I would like to talk about a specific case scenario. Assuming as a developer and you structure your .gitignore and your code and everything looks great however maybe by an accident an important file with Azure creds was saved or placed in your repo and You are not aware of the file is there. You may think it is not real scenario or it is difficult to happen. In this case you maybe interested to read this article
Now since the .gitignore file does not know about it, this sensitive file will sneak into your public repo during the commit and push process. Or Maybe to you missed structuring .gitignore and now your local.settings.json, .env that contains hardcoded secrets creds will be pushed to your rep. Would be nice if git warns you before you do commit?
I would introduce git-secrets, It is an open-source project that helps to prevent you from committing passwords and other sensitive information into git repo. The plugin supports Azure, AWS, and GCP.
git-secrets scans commit, to prevent adding secrets into your git repositories. If a commit, commit message, or any commit in a --no-ff merge history matches one of your configured prohibited regular expression patterns, then the commit is rejected.
Installing git-secrets
git-secrets must be placed somewhere in your PATH so that it is picked up by git when running git secrets.
You can use the install target of the provided Makefile to install git secrets and the man page. You can customize the install path using the PREFIX and MANPREFIX variables.
make install
PS > ./install.ps1
brew install git-secrets
Installing git hooks
You MUST install the git hooks for every repo that you wish to use with git secrets --install.
Here's a quick example of how to register Azure provider:
cd /path/to/my/repo
git secrets --install
git secrets --register-azure
You can also install another provider like AWS and GPC
cd /path/to/my/repo
git secrets --install
git secrets --register-aws
git secrets --register-gcp
Before making public a repository
With git-secrets is also possible to scan a repository including all revisions:
git secrets --scan-history
Examples
Let assume we have repo and it has sensitive creds and we are going to commit it. Let's see what will happen when we use git-secrets hook.
C:\github\my_app>git add .
C:\github\my_app>git commit -m "adding plugin"
src/demo.py:9:TENANT_ID ='4cbcc7d8-094d-4006-1049-0d11d61f484d'
src/demo.py:10:CLIENT ='89f62c1d-cabf-4372-b217-7f3dd31f55fb'
src/demo.py:11:SUBSCRIPTION_ID ='99d8e999-a50c-43ab-a03a-e3a8280d0000'
src/demo.py:12:key1= '-----BEGIN RSA PRIVATE KEY-----'
src/demo.py:13:key2= '-----BEGIN EC PRIVATE KEY-----'
src/demo.py:14:key3= '-----BEGIN DSA PRIVATE KEY-----'
src/demo.py:15:key4= '-----BEGIN PGP PRIVATE KEY-----'
src/demo.py:16:server1 = "https://myserver12.cloudapp.net/helloworld"
src/demo.py:17:server2 = 'mys1llsa942342.blob.core.windows.net'
src/demo.py:18:server3 = 'agajsks0-asdask9.queue.core.windows.net'
src/demo.py:20:server5 = 'hasdasd8ja_osow-uuuu.database.windows.net'
src/demo.py:21:server6 = 'asdasdhkak8masda0asdaasdsa.servicebus.windows.net'
src/demo.py:22:server7 = 'hello.timeseries.azure.com'
src/demo.py:23:server8 = 'a234234asdfasd333.accesscontrol.windows.net'
src/demo.py:24:server9 = 'ba29SKA823ww.azurehdinsight.net'
src/demo.py:25:server10 = '23254asgfdgefge.cloudapp.azure.com'
src/demo.py:26:sas_token ='https://myaccount.blob.core.windows.net/sascontainer/sasblob.txt?sv=2015-04-05&st=2015-04-29T22%3A18%3A26Z&se=2015-04-30T02%3A23%3A26Z&sr=b&sp=rw&sip=168.1.5.60-168.1.5.70&spr=https&sig=Z%2FRHIX5Xcg0Mq2rqI3OlWTjEg2tYkboXr1P9ZUXDtkk%3D'
src/demo.py:28:key ='Dl2~.?@#$%^&*_!+=[]{}|\:n()/,`;"'
src/demo.py:29:KEY ='Bf[]tvS1C|-w=k./@A/&h:R/0!@yJLu#'
src/demo.py:30:KEY2='B1234tf4tvS41C3wkb!@&$%^&+()+_*$'
src/demo.py:51: "102a3be2-3a83-423a-a724-12d63eb47288",
src/demo.py:52: "20c843c0-6aac-4f11-9bc2-06220720d699"
[ERROR] Matched one or more prohibited patterns
Possible mitigations:
- Mark false positives as allowed using: git config --add secrets.allowed ...
- Mark false positives as allowed by adding regular expressions to .gitallowed at repository's root directory
- List your configured patterns: git config --get-all secrets.patterns
- List your configured allowed patterns: git config --get-all secrets.allowed
- List your configured allowed patterns in .gitallowed at repository's root directory
- Use --no-verify if this is a one-time false positive
As you see, the plugin scanned the code and found many sensitive creds patterns that is hardcoded like Tenant_ID, SubscriptionID, ClientID, Client key, Private Certs, Azure services endpoint for blob, table, SAS token..etc
Fantasics!!!! Now we have chance to review our code again and make the necessary change to keep our creds safe and secure and our code clean.
Conclusion
Using git-secrets hook is the first line of defense against leaking sensitive creds into the github
If you would like to know more about the tool please visit my repo git-secrets
Microsoft
