Download XML files with Powershell script

Let’s say we have a scenario where there is a need to download a huge amount of XML files from a central web server for whatever reason. This is an example but I had a situation very similar like this. That’s why I have created a PowerShell script which is able to do the job for me.

The script is created with a function  called Get-WebPageXMLS

function Get-WebPageXMLS($url, $folder)

 

{

Import-Module BitsTransfer

if (-not (Test-Path $folder)) { md $folder }

$ie = New-Object -COMObject InternetExplorer.Application

$ie.Navigate($url)

while ($ie.Busy) { Start-Sleep -Milliseconds 100 }

$xmls = $ie.document.getElementsByTagName(‘a’) | Select-Object -ExpandProperty href |

where { $_ -match “`.xml$” }

$ie.Quit()

$xmls_n = $xmls.Length

if ($xmls_n -gt 0)

 {

$i = 1

$webClient = new-object System.Net.WebClient # .NET class

$proxy = new-object System.Net.WebProxy “Ipaddress:port” # specify your proxy address and port

# replace your credential by your domain, username, and pasword in cause there is authentication

#$proxy.Credentials = New-Object System.Net.NetworkCredential (“Domain\UserName”, “Password”)

$webClient.proxy=$proxy

# specify a header if you want to check info in your logs on your proxy

$webClient.Headers.Add(“user-agent”, “Windows Powershell WebClient”)

#$b=Get-Credential #here you can uncomment this but comment the following

 

 

$webClient.credentials = new-object system.net.networkcredential(domain\username,”password”)

foreach ($lnk in $xmls)

{ 

 

$dst = “$folder\$($lnk.Split(‘/’)[-1])” 

$dn = $lnk.Split(‘/’)[1]

echo “Getting [$i/$xmls_n] $dn” 

  

#$webClient.Credentials = $b #uncomment if you use get-cridential

$webClient.DownloadFile($lnk, $dst)

#Start-BitsTransfer $lnk $dst -Prio High -Display $dn #uncomment if you wish to use bits transfer

$i++ 

}

} else { “Nothing to do – NO xml links found in $url” }

}

cls

Get-WebPageXMLS http://Source                “c:\temp\destination”

 
 

 

 

With $b=Get-Cridential, if the site from where we download the files requires authentication, we can request user input for username and password. This is commented because we can put username and password directly in the script itself.

You can download the sccript here: 

http://techlevel.web.officelive.com/documents.aspx

   

 

 

 

 

 

 

 

  

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: