Windows Server Summit 2024
Mar 26 2024 08:00 AM - Mar 28 2024 04:30 PM (PDT)
Microsoft Tech Community
LIVE
Tar and Curl Come to Windows!
Published Mar 21 2019 05:14 PM 120K Views
Brass Contributor
First published on TECHNET on Dec 19, 2017
Beginning in Insider Build 17063, we’re introducing two command-line tools to the Windows toolchain: curl and bsdtar. It’s been a long time coming, I know. We'd like to give credit to the folks who’ve created and maintain bsdtar and curl —awesome open-source tools used by millions of humans every day. Let's take a look at two impactful ways these tools will make developing on Windows an even better experience.

1. Developers! Developers! Developers!


Tar and curl are staples in a developer’s toolbox; beginning today, you’ll find these tools are available from the command-line for all SKUs of Windows. And yes, they're the same tools you've come to know and love! If you're unfamiliar with these tools, here's an overview of what they do:

  • Tar: A command line tool that allows a user to extract files and create archives. Outside of PowerShell or the installation of third party software, there was no way to extract a file from cmd.exe. We're correcting this behavior :) The implementation we're shipping in Windows uses libarchive .

  • Curl: Another command line tool that allows for transferring of files to and from servers (so you can, say, now download a file from the internet).


Now not only will you be able to perform file transfers from the command line,  you'll also be able to extract files in formats in addition to .zip (like .tar.gz, for example). PowerShell does already offer similar functionality (it has curl and it's own file extraction utilities), but we recognize that there might be instances where PowerShell is not readily available or the user wants to stay in cmd.


2. The Containers Experience


Now that we’re shipping these tools inbox, you no longer need to worry about using a separate container image as the builder when targeting nanoserver-based containers. Instead, we can invoke the tools like so:


Background


We offer two base images for our containers: windowsservercore and nanoserver . The servercore image is the larger of the two and has support for such things as the full .NET framework. On the opposite end of the spectrum is nanoserver, which is built to be lightweight with as minimal a memory footprint   as possible. It’s capable of running .NET core but, in keeping with the minimalism, we’ve tried to slim down the image size as much as possible. We threw out all components we felt were not mission-critical for the container image.

PowerShell was one of the components that was put on the chopping block for our nanoserver image. PowerShell is a whopping 56 Mb (given that the total size of the nanoserver image is 200 Mb…that’s quite the savings!) But the consequence of removing PowerShell meant there was no way to pull down a package and unzip it from within the container.



If you’re familiar with writing dockerfiles, you’ll know that it’s common practice to pull in all the packages (node, mongo, etc.) you need and install them. Instead, users would have to rely on using a separate image with PowerShell as the “builder” image to accomplish constructing an image. This is clearly not the experience we want our users to have when targeting nanoserver—they’d end up having to download the much larger servercore image.

This is all resolved with the addition of curl and tar. You can call these tools from servercore images as well.




We want your Feedback!


Are there other developer tools you would like to see added to the command-line? Drop a comment below with your thoughts! In the mean time, go grab Insider Build 17063 and get busy curl’ing an tar’ing to your heart’s desire.
18 Comments
Brass Contributor

This is awesome ... but part of me wishes you would pull cmd out rather than powershell out of the nano containers.  I feel like we are stuck between two admin worlds... we keep talking about using PowerShell for CM ... but keep comming back to cmd.  

That said ... curl especially is most welcome.  Certainly going to streamline things.  Even if it does mean cmd lives for another decade.


 

Copper Contributor

New year, same old question. I don't get it MSFT. Why bother including curl if no intent on maintaining it current.

Copper Contributor

Some time ago I've tried to create scripts that can archive files and do http requests:

https://stackoverflow.com/questions/28143160/how-can-i-download-a-file-with-batch-file-without-using...

and 

https://stackoverflow.com/questions/28043589/how-can-i-compress-zip-and-uncompress-unzip-files-and-f...

 

I don't know if a simple , text only script is a third party software but may be in some cases these are worth checking.

Copper Contributor

Good to know that it is coming in built with Win 10. Will it still be there in the upcoming OS of Windows? For one of our projects we are planning to use Curl on client side for transferring files to FTPS server and if that does not come in built in future windows OS, we may need to explore for some other options. 

Copper Contributor

Can anyone tell or update this article - Is this applicable to windows server 2016 and 2019 as well ?

Copper Contributor

No for Server 2016, but it does work on Windows Server 2019, [Version 10.0.17763.1339] at the time of this writing.

Copper Contributor

Where is the libarchive's license?
It doesn't exist in https://aka.ms/notices-win.

Copper Contributor

As of Aug 21, 2020, my Windows 10 v1809 is showing the curl.exe in System32 and SysWOW64 as version 7.55.1 but the current version is 7.72.0 where many of the updates since v7.55.1 have been fixes for security vulnerabilities.

 

What gives Microsoft? As another commenter pointed out, if you are going to include it then WSUS and/or Microsoft Updates _needs_ to keep it current.

 

I'll also point out that you've made it difficult to update or remove the Microsoft provided curl manually since ICALCS shows the owner with FULL permissions to be NT SERVICE\TrustedInstaller with every other defined user as only having Read, Execute.

 

Installing Git for Windows also includes curl but at least those folks are pretty good about keeping curl current. Unfortunately many people will have a path configuration that will use your obsolete, built-in instance.

 

Not cool, Microsoft! Not cool at all.

Copper Contributor

The example uses tar to create a file ending in .zip. So is that actually a ZIP format file, or is it a GZ format file with the wrong filename extension? If the former, strange that the -z option to tar would be co-opted for a different archive format; if the latter, bad practice to give files incorrect filename extensions, as it could cause issues with tools that use extension to determine format.

Edit: It appears that it is the latter, which is bad form.

 

CMD> tar -czvf file.zip zipdir/
CYGWIN> file file.zip
    file.zip: gzip compressed data, last modified: Mon Sep 28 20:26:49 2020, from Unix

Copper Contributor

The SYSTEM32 version of tar is incompatible with our build scripts: it fails with the message "tar: Failed to open '\\.\tape0'"

Copper Contributor

I suggest Microsoft help curl team about maintain curl for Windows

 

By example, there is a Pull request for adding TLS 1.3 for curl windows using schannel, at https://github.com/curl/curl/pull/8419 , and they have question about schannel API.

Copper Contributor

Despite the information provided here, Windows 2016 does NOT have tar. Everything I read basically says "copy the executable from another Windows system". That doesn't help if you don't have other non-2016 Windows systems to copy from. Therefore, I suggest that Microsoft have a secure download site that people in my situation can use.

Copper Contributor

 @keithdtyler 

Yes, the example GIF shows incorrect usage.

To create an actual ZIP file, add the -a switch to automatically select compression format based on filename, e.g.:

 

tar -caf file.zip zipdir

 

 

Copper Contributor

Old thread, but why aren't the curl binaries digitally signed? These files reside in critical file systems spaces, and this is just wrong. It bypasses all the security principles I have ever learned!

Copper Contributor

Hello,

Can anyone let me know how to update curl on windows server 2022, our SOC team is back on me to update curl to latest version. Is there any update from MS for Curl.

Copper Contributor

Hello,

 

When will curl be updated to the latest version (7.88.1)? The current version on my systems is 7.83.1.

Copper Contributor

@dineshk I've opted to take it out of system32/syswow64 and put it back under program files for those systems that truly require it (adding to path if necessary).  This will probably work to update as well, but you have to take ownership of the file first, then icacls then either update or remove.  

 

takeown /F C:\windows\System32\curl.exe
icacls C:\windows\System32\curl.exe /grant administrators:F

del C:\windows\System32\curl.exe

 

and similar for syswow64.  

Copper Contributor

I couldn't find tar's complete documentation anywhere (I found options on forums that are not described in

tar --help

but they actually work). Can someone please give me a link to an online documentation?

Version history
Last update:
‎Mar 21 2019 05:14 PM
Updated by: