Invoke-Webrequest does not return cookie

%3CLINGO-SUB%20id%3D%22lingo-sub-1204733%22%20slang%3D%22en-US%22%3EInvoke-Webrequest%20does%20not%20return%20cookie%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1204733%22%20slang%3D%22en-US%22%3E%3CP%3EHi%2C%20I'm%20trying%20to%20automate%20the%20request%20of%20a%20web%20page%20protected%20by%20a%20password.%20When%20I%20post%20the%20login%20form%20with%20the%20password%2C%20I%20expect%20to%20find%20the%20authentication%20cookie%20set%20by%20the%20website%20to%20be%20included%20in%20the%20session%20variable%2C%20so%20that%20I%20can%20request%20the%20protected%20page%20by%20passing%20the%20cookie.%20However%2C%20in%20the%20response%20to%20the%20POST%2C%20there%20are%20no%20cookies.%20I%20must%20be%20overlooking%20something%20obvious%20%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EWhen%20I%20inspect%20the%20browser%20conversation%2C%20the%20response%20to%20the%20POST%20contains%20the%20following%20header%3A%3C%2FP%3E%3CDIV%20class%3D%22header-value%20source-code%22%3E%26nbsp%3B%3C%2FDIV%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CPRE%20class%3D%22lia-code-sample%20language-markup%22%3E%3CCODE%3ESet-Cookie%3A%20AuthCookie%3De8b931f20709701%20...%20etc%20...%20%3B%20path%3D%2F%3B%20httponly%3B%3C%2FCODE%3E%3C%2FPRE%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CDIV%20class%3D%22header-value%20source-code%22%3E%26nbsp%3B%3C%2FDIV%3E%3CDIV%20class%3D%22header-value%20source-code%22%3EThis%20is%20the%20code%20I%20have%3A%3C%2FDIV%3E%3CDIV%20class%3D%22header-value%20source-code%22%3E%26nbsp%3B%3C%2FDIV%3E%3CDIV%20class%3D%22header-value%20source-code%22%3E%26nbsp%3B%3C%2FDIV%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CPRE%20class%3D%22lia-code-sample%20language-javascript%22%3E%3CCODE%3E%24header1%20%3D%20%40%7B%0A'Host'%3D%22%24(%24ip)%22%0A'User-Agent'%3D%20'Mozilla%2F5.0%20(Windows%20NT%2010.0%3B%20Win64%3B%20x64)%20AppleWebKit%2F537.36%20(KHTML%2C%20like%20Gecko)%20Chrome%2F80.0.3987.122%20Safari%2F537.36%20Edg%2F80.0.361.62'%0A'Accept'%3D%20'text%2Fhtml%2Capplication%2Fxhtml%2Bxml%2Capplication%2Fxml%3Bq%3D0.9%2Cimage%2Fwebp%2Cimage%2Fapng%2C*%2F*%3Bq%3D0.8%2Capplication%2Fsigned-exchange%3Bv%3Db3%3Bq%3D0.9'%0A'Accept-Language'%3D%20'en-GB%2Cen%3Bq%3D0.9%2Cen-US%3Bq%3D0.8%2Cfr%3Bq%3D0.7%2Cnl%3Bq%3D0.6%2Caf%3Bq%3D0.5'%0A'Accept-Encoding'%3D%20'gzip%2C%20deflate'%0A'Content-Type'%3D'application%2Fx-www-form-urlencoded'%0A'Referer'%3D%22%24(%24url1)%22%0A%7D%0A%0A%24login%20%3D%20Invoke-WebRequest%20%24url1%20-SessionVariable%20ws%0A%24login.Forms%5B0%5D.Fields.LogBox%20%3D%20%24pwd%0A%24page%20%3D%20Invoke-WebRequest%20%24url1%20-Body%20%24login.Forms%5B0%5D.Fields%20-Method%20Post%20-Headers%20%24header1%20-WebSession%20%24ws%0A%24page.Headers%0A%24ws.Cookies.GetCookies(%24url1)%3C%2FCODE%3E%3C%2FPRE%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CDIV%20class%3D%22header-value%20source-code%22%3E%26nbsp%3B%3C%2FDIV%3E%3CDIV%20class%3D%22header-value%20source-code%22%3EWhen%20I%20run%20the%20script%2C%20the%20page%20headers%20does%20not%20contain%20any%20cookies%2C%20and%20the%20web%20session%20variable's%20cookies%20are%20also%20empty.%3C%2FDIV%3E%3CDIV%20class%3D%22header-value%20source-code%22%3E%26nbsp%3B%3C%2FDIV%3E%3CDIV%20class%3D%22header-value%20source-code%22%3ECan%20anyone%20point%20me%20in%20the%20right%20direction%20%3F%3C%2FDIV%3E%3CDIV%20class%3D%22header-value%20source-code%22%3E%26nbsp%3B%3C%2FDIV%3E%3CDIV%20class%3D%22header-value%20source-code%22%3EStephane%3C%2FDIV%3E%3CDIV%20class%3D%22header-value%20source-code%22%3E%26nbsp%3B%3C%2FDIV%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1204733%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EWindows%20PowerShell%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1205212%22%20slang%3D%22en-US%22%3ERe%3A%20Invoke-Webrequest%20does%20not%20return%20cookie%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1205212%22%20slang%3D%22en-US%22%3E%3CP%3EHello%2C%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F572423%22%20target%3D%22_blank%22%3E%40StephaneBouillon%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3CP%3Ehave%20you%20tried%20the%20example%206%20from%20the%20Powershell%20help%20for%20Invoke-Webrequest(%26nbsp%3B%3CA%20title%3D%22Invoke-Webrequest%20Help%22%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fpowershell%2Fmodule%2Fmicrosoft.powershell.utility%2Finvoke-webrequest%3Fview%3Dpowershell-7%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fpowershell%2Fmodule%2Fmicrosoft.powershell.utility%2Finvoke-webrequest%3Fview%3Dpowershell-7%3C%2FA%3E%26nbsp%3B%26nbsp%3B)%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EMay%20that%20will%20help%20you%20%3B)%3C%2Fimg%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3Ebr%2C%3C%2FP%3E%3CP%3ENico%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1207193%22%20slang%3D%22en-US%22%3ERe%3A%20Invoke-Webrequest%20does%20not%20return%20cookie%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1207193%22%20slang%3D%22en-US%22%3E%3CP%3EThansk%20for%20your%20time%20and%20effort%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F572641%22%20target%3D%22_blank%22%3E%40Nico_G%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20checked%20that%20example%2C%20and%20it%20talks%20about%20multipart%2Fform-data%20post%20submission%2C%20but%20that%20is%20not%20what%20the%20server%20is%20expecting%2C%20it%20just%20needs%20a%20simple%20application%2Fx-www-form-urlencoded.%20Below%20I%20include%20the%20traffic%20that%20is%20exchanged.%20The%20issue%20I%20have%20is%20that%20when%20I%20do%20it%20programmatically%2C%20the%20cookie%20is%20missing%20from%20the%20post's%20response.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CPRE%20class%3D%22lia-code-sample%20language-markup%22%3E%3CCODE%3EGeneral%0A%20%20Request%20URL%3A%20http%3A%2F%2F10.0.2.49%2Fgeneral%2Fstatus.html%0A%20%20Request%20Method%3A%20POST%0A%20%20Status%20Code%3A%20301%20Moved%20Permanently%0A%20%20Remote%20Address%3A%2010.0.2.49%3A80%0A%20%20Referrer%20Policy%3A%20no-referrer-when-downgrade%0A%0AResponse%20Headers%0A%20%20Cache-Control%3A%20no-cache%0A%20%20Connection%3A%20close%0A%20%20Content-Language%3A%20en-gb%0A%20%20Content-Length%3A%209705%0A%20%20Content-Type%3A%20text%2Fhtml%0A%20%20Location%3A%20%2Fgeneral%2Fstatus.html%0A%20%20Pragma%3A%20no-cache%0A%20%20Server%3A%20debut%2F1.30%0A%20%20Set-Cookie%3A%20AuthCookie%3De8b931f207097012850e8afdecace8a7%3AJWlbdmN8MrhkZCsRxhIPWYu7HH86yGgEAA%253D%253D%3B%20path%3D%2F%3B%20httponly%3B%0A%20%20X-Frame-Options%3A%20DENY%0A%0ARequest%20Headers%0A%20%20Accept%3A%20text%2Fhtml%2Capplication%2Fxhtml%2Bxml%2Capplication%2Fxml%3Bq%3D0.9%2Cimage%2Fwebp%2Cimage%2Fapng%2C*%2F*%3Bq%3D0.8%2Capplication%2Fsigned-exchange%3Bv%3Db3%3Bq%3D0.9%0A%20%20Accept-Encoding%3A%20gzip%2C%20deflate%0A%20%20Accept-Language%3A%20en-GB%2Cen%3Bq%3D0.9%2Cen-US%3Bq%3D0.8%2Cfr%3Bq%3D0.7%2Cnl%3Bq%3D0.6%2Caf%3Bq%3D0.5%0A%20%20Cache-Control%3A%20max-age%3D0%0A%20%20Connection%3A%20keep-alive%0A%20%20Content-Length%3A%20101%0A%20%20Content-Type%3A%20application%2Fx-www-form-urlencoded%0A%20%20Host%3A%2010.0.2.49%0A%20%20Origin%3A%20http%3A%2F%2F10.0.2.49%0A%20%20Referer%3A%20http%3A%2F%2F10.0.2.49%2Fgeneral%2Fstatus.html%0A%20%20Upgrade-Insecure-Requests%3A%201%0A%20%20User-Agent%3A%20Mozilla%2F5.0%20(Windows%20NT%2010.0%3B%20Win64%3B%20x64)%20AppleWebKit%2F537.36%20(KHTML%2C%20like%20Gecko)%20Chrome%2F80.0.3987.122%20Safari%2F537.36%20Edg%2F80.0.361.62%0A%0AForm%20Data%0A%20%20CSRFToken%3A%20cmsr3WdzCu%2Bxd1rW3SW1ZWemTfk6OQAoAA%3D%3D%0A%20%20B133c%3A%20***%20the%20password%20***%0A%20%20loginurl%3A%20%2Fgeneral%2Fstatus.html%3C%2FCODE%3E%3C%2FPRE%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E
New Contributor

Hi, I'm trying to automate the request of a web page protected by a password. When I post the login form with the password, I expect to find the authentication cookie set by the website to be included in the session variable, so that I can request the protected page by passing the cookie. However, in the response to the POST, there are no cookies. I must be overlooking something obvious ?

 

When I inspect the browser conversation, the response to the POST contains the following header:

 

 

 

Set-Cookie: AuthCookie=e8b931f20709701 ... etc ... ; path=/; httponly;

 

 

 
This is the code I have:
 
 

 

 

$header1 = @{
'Host'="$($ip)"
'User-Agent'= 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36 Edg/80.0.361.62'
'Accept'= 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9'
'Accept-Language'= 'en-GB,en;q=0.9,en-US;q=0.8,fr;q=0.7,nl;q=0.6,af;q=0.5'
'Accept-Encoding'= 'gzip, deflate'
'Content-Type'='application/x-www-form-urlencoded'
'Referer'="$($url1)"
}

$login = Invoke-WebRequest $url1 -SessionVariable ws
$login.Forms[0].Fields.LogBox = $pwd
$page = Invoke-WebRequest $url1 -Body $login.Forms[0].Fields -Method Post -Headers $header1 -WebSession $ws
$page.Headers
$ws.Cookies.GetCookies($url1)

 

 

 
When I run the script, the page headers does not contain any cookies, and the web session variable's cookies are also empty.
 
Can anyone point me in the right direction ?
 
Stephane
 
2 Replies

Hello,

 

@StephaneBouillon 

have you tried the example 6 from the Powershell help for Invoke-Webrequest( https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-webrequest?vi...  )?

 

May that will help you ;)

 

br,

Nico

Thansk for your time and effort @Nico_G 

 

I checked that example, and it talks about multipart/form-data post submission, but that is not what the server is expecting, it just needs a simple application/x-www-form-urlencoded. Below I include the traffic that is exchanged. The issue I have is that when I do it programmatically, the cookie is missing from the post's response.

 

General
  Request URL: http://10.0.2.49/general/status.html
  Request Method: POST
  Status Code: 301 Moved Permanently
  Remote Address: 10.0.2.49:80
  Referrer Policy: no-referrer-when-downgrade

Response Headers
  Cache-Control: no-cache
  Connection: close
  Content-Language: en-gb
  Content-Length: 9705
  Content-Type: text/html
  Location: /general/status.html
  Pragma: no-cache
  Server: debut/1.30
  Set-Cookie: AuthCookie=e8b931f207097012850e8afdecace8a7:JWlbdmN8MrhkZCsRxhIPWYu7HH86yGgEAA%3D%3D; path=/; httponly;
  X-Frame-Options: DENY

Request Headers
  Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
  Accept-Encoding: gzip, deflate
  Accept-Language: en-GB,en;q=0.9,en-US;q=0.8,fr;q=0.7,nl;q=0.6,af;q=0.5
  Cache-Control: max-age=0
  Connection: keep-alive
  Content-Length: 101
  Content-Type: application/x-www-form-urlencoded
  Host: 10.0.2.49
  Origin: http://10.0.2.49
  Referer: http://10.0.2.49/general/status.html
  Upgrade-Insecure-Requests: 1
  User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36 Edg/80.0.361.62

Form Data
  CSRFToken: cmsr3WdzCu+xd1rW3SW1ZWemTfk6OQAoAA==
  B133c: *** the password ***
  loginurl: /general/status.html