Invoke-WebRequest and RTFM Moments


Before the weekend, I struggled with a PowerShell utility I had written a couple of months back.

To help matters, I goggled for how to capture the returned HTML Output and write it out a file. Hoping that by doing so, I will more likely find out whether I am being redirected or experiencing security, proxy or other Network errors.



Found a simple code on the Net for capturing the returned HTML response and writing it out in its entirety to a file.


The code simply states that I should use the -OutFile parameter.

$htmlWebResponseObject = Invoke-WebRequest -URI $webSiteURL `
                                  -OutFile $debugFilename 

I added the code to my original file and ended up with something that looks like the following:


#requires -version 3.0


$websiteURL = ''
$FILE_EXT = "html"

$htmlWebResponseObject = $null;

function getTempFilename( [string]$extension)

	#How can I create a temp file with a specific extension with Net

	[string] $fileName = $null;

	#get Temp Folder
	$folderPath = [System.IO.Path]::GetTempPath()

	#get random filename
	$fileName = [System.IO.Path]::GetRandomFileName();

	# change file extension to extension (HTML)
	$fileName = [System.IO.Path]::ChangeExtension($fileName, $extension);

	# add folderpath and filename
	$fileName = [System.IO.Path]::Combine($folderPath, $fileName);	

	return $fileName

	$debugFilename = getTempFilename($FILE_EXT)

	$log = "Debug File :- {0}" -f $debugFilename
	Write-Host $log -ForegroundColor white    

	$htmlWebResponseObject = Invoke-WebRequest -URI $webSiteURL `
								-UseDefaultCredentials `
								-DisableKeepAlive `
								-OutFile $debugFilename
								#-Proxy $INTERNET_PROXY_SERVER `

	# Get Web Request Status Code
	$WROStatusCode = $htmlWebResponseObject.StatusCode 

	# Get Web Request Status Description
	$WROStatusDescription = $htmlWebResponseObject.StatusDescription

	# If we succcesfully connected to Web Site
	if (
		   ( $WROStatusCode -eq $HTTP_SUCCESS)

		$log = "We are good!, let us parse our returned Response Object"
		Write-Host $log -ForegroundColor white     


		$logFormat = "We are not good! Our URL {0} and HTTP Status Code is {1}"
		$log = $logFormat -f $webSiteURL, $WROStatusCode
		Write-Error $log 



	Write-Host "_____________________________________________________________________" -ForegroundColor red

	$strLog = "Failed to connect to URL :- {0}" -f $webSiteURL
	Write-Host $strLog -ForegroundColor red

	$errLine = "Exception Type :- {0}" -f $_.Exception.GetType().FullName
	Write-Host $errLine -ForegroundColor red

	$errLine = "Exception Message :- {0}" -f $_.Exception.Message
	Write-Host $errLine -ForegroundColor red

	$errLine = "Exception Source :- {0}" -f $_.Exception.Source
	Write-Host $errLine -ForegroundColor red

	$lineNumber = $_.InvocationInfo.ScriptLineNumber
	$errLine = "Script Line Number :- {0}" -f $lineNumber
	Write-Host $errLine -ForegroundColor red

	$errLine = "Exception HResult :- {0}" -f $_.Exception.HResult
	Write-Host $errLine -ForegroundColor red

	$errLine = "Exception Inner :- {0}" -f $_.Exception.InnerException
	Write-Host $errLine -ForegroundColor red

	$errLine = "Exception HResult :- {0}" -f $_.Exception.HResult
	Write-Host $errLine -ForegroundColor red  	

	$errLine = "Exception StackTrace :- {0}" -f $_.Exception.StackTrace
	Write-Host $errLine -ForegroundColor red            

	Write-Host "_______________________________________________" -ForegroundColor red


But getting an error stating:

We are not good!



Temp Workaround

Unfortunately, the error was silent. I did not drop down to the exception handler. I simply failed to get a good .STATUSCODE.

To keep things moving, I simply removed the -OutFile parameter, added some work around code, and went on about my way.

RTFM Workaround

As I drove in to work this weekend, I waded into the bad habit of driving and multi-tasking … cleaning out my heavy work-bag full of felled trees.

The documentation clearly states that if I add the Outfile parameter, I need the Passthru parameter to be able get to the pipeline, as well.




Saves the response body in the specified output file. Enter a path and file name. If you omit the path, the default is the current location.

By default, Invoke-WebRequest returns the results to the pipeline. To send the results to a file and to the pipeline, use the Passthru parameter.






      $htmlWebResponseObject = Invoke-WebRequest -URI $webSiteURL `
						-UseDefaultCredentials `
						-DisableKeepAlive `
						-OutFile $debugFilename

Once we added the PassThru argument, we are good; as we get the message emptied below.


We are good!, let us parse our returned Response Object


Another Error

But, as always that is not the complete story.

When one is behind a Corporate Firewall and we have to go through a Proxy Server, we can get the error pasted below:

Failed to connect to URL :-
Exception Type :- System.Net.WebException
Exception Message :- The remote server returned an error: (407) Proxy Authentication Required.
Exception Source :- Microsoft.PowerShell.Commands.Utility
Script Line Number :- 56
Exception HResult :- -2146233079
Exception Inner :-
Exception HResult :- -2146233079
Exception StackTrace :-    at Microsoft.PowerShell.Commands.WebRequestPSCmetResponse(WebRequest request)
   at Microsoft.PowerShell.Commands.WebRequestPSCmdlet.ProcessRecord()

Will deal with this new issue in a later post.




True to the Sabbath, I had a good and restful weekend.

A good family friend, Yemi and Aja, celebrated their 10 year anniversary this weekend.

She changed outfit so many times and had a diva entrances the whole evening.

No, she did not go back to Diana Ross –  I am coming out.

But, out of many, she came out to this one

Collie Buddz – I Feel So Good



Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s