One question that comes up fairly frequently in our internal mailing list, the newsgroup, and the internet at large is how to call a webservice from PowerShell. In fact, several excellent PowerShellers have written about it: Keith, and Geert. In general, the guidance has been to use wsdl.exe to generate the webservice proxy, compile that proxy into a DLL, then finally load that DLL into memory.

This is a topic that I cover in my upcoming book, and initially wrote a script to automate these proxy generation steps. However, the prerequisite to running a script designed in that matter is fairly huge. Wsdl.exe doesn't come with the .NET Framework, so you need to have the .NET SDK installed. That was something that made me uncomfortable, so I instead opted for a solution that could generate the web service proxy without wsdl.exe.

The .NET Framework supports a few classes that make this possible, although the documentation for them is terrible at best :)

To give a glimpse into the writing process behind my upcoming "Windows PowerShell - The Definitive Guide" (O'Reilly,) I'll occasionally post entries "as the author sees it." This entry discusses calling a webservice from PowerShell.

Program: Connect-WebService

Although "screen scraping" (parsing the HTML of a web page) is the most common way to obtain data from the internet, web services are becoming increasingly common. Web services provide a significant advantage over HTML parsing, as they are much less likely to break when the web designer changes minor features in their design.

The only benefit to web services isn't their more stable interface, however. When working with web services, the .NET Framework allows you to generate proxies that let you interact with the web service as easily as you would work with a regular .NET object. That is because to you, the web service user, these proxies act almost exactly the same as any other .NET object. To call a method on the web service, simply call a method on the proxy.

 The following script allows you to connect to a remote webservice, if you know the location of its service description file (WSDL.) It generates the web service proxy for you, allowing you to interact with it as you would any other .NET object.

[Update: Several readers have requested that this script support web services that reqire credentials, and support web services that return the same type of object. Added parameters to allow this.]

Allow me to be tragically late to the party in pointing out that Bruce Payette’s PowerShell in Action is now available.

As shown by its Amazon page, the reviews are immensely positive.

Unfortunately for me, I’ve sequestered myself from reading any of the PowerShell books until my own is complete. This one, however, calls longingly from co-workers’ desks as I walk down the halls :) Conversations with Bruce are always interesting, and this book is sure to not disappoint.

Over the last while, Scott Hanselman has been blogging a lot about some pretty cool ideas – controlling an IR port through C#, monkeying around with robots, and of course, PowerShell.

This all came together today, with the publishing of the second part of his Coding4Fun series, "Microbric Viper Robot with an Iguanaworks IR Serial Port and PowerShell."

In this final part, he extends the PowerShell Logo example to make a real robot execute your PowerShell script. Check it out – it's mind blowing!

It turns out that the chisel nib on a thick whiteboard marker is actually exceedingly good for whiteboard calligraphy. It gives an x-height of about 20mm, though :)

The only problem is the guide lines – since on a whiteboard, there are none :) I’ve tried penciling them in, but that doesn’t work very well. I think my solution for that will be the true geek way – lasers.

One of the things people commonly ask about is support for rich keyboard macros and hotkeys in PowerShell.  For many, a pointer to the Doskey hotkey reference (which provides the history management functionality in PowerShell’s “cooked mode”) is enough: http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/doskey.mspx?mfr=true.

Once you’ve become used to those features, though, you start to want more – especially if you’ve grown accustomed to some of the power available on traditional Unix shells. Typical questions include:

• How do I use Emacs keystrokes to navigate my history?
• How do I use the keyboard to copy and paste in PowerShell?
• How do I clear the screen with a keystroke?
• How do exit the shell with a keystroke?
• How do I use the keyboard to adjust the font size in PowerShell?

Although PowerShell itself doesn’t directly offer these additional customization options, there is still an outlet – UI Automation and Keystroke macro programs.

Programs that support keystroke macros are great, as they let you automate anything that you can already do with a keyboard (and sometimes the mouse.) For example, you can use the Alt+Space combination in any console window to open up its system menu.  A console system window includes all kinds of tasty treats – copy, paste, find, and even properties.

Using these menus and a program that supports keystroke macros, we can customize most of the things we want to.

I personally use a program called AutoHotkey as my macro program. It has a lot of power, although its scripting language leaves a lot to be desired. I think it would be the coolest thing ever if somebody wrote a macro program that hosted PowerShell – giving you all the benefits of PowerShell scripting for UI automation.

AutoHotkey uses a script to let you write keystroke macros. You tell it what keys to listen for, and what it should do when it sees those keys. Let’s take some simple examples.

One thing you might wonder is how to use Emacs keystrokes to navigate your history. The console cooked mode already supports many of the concepts that you want, but just doesn’t use the keystrokes that you want. You want to push Ctrl-A, but the console uses HOME for that. You want to push Ctrl-E, but the console uses END for that. So in your macro, tell AutoHotkey to send the HOME key when you push Ctrl-A:

;;
;; Beginning of line with ^A if we're in a console window
;;
#IfWinActive, ahk_class ConsoleWindowClass
^a::SendInput {HOME}
#IfWinActive

That sequence tells AutoHotkey: if the class of the current window is ConsoleWindowClass, then run these commands when I press ^a. Those commands are to send the HOME key as input. Similar macros work for the other navigation commands you might want to remap.
These keystrokes can get even more complex, if desired. For example, you want Ctrl-Y (the Emacs keystroke for Paste) to paste into a console window. The console doesn’t directly support that with a keystroke substitution, but the glorious system menu does:

You can automate that with Alt+Space, E, P:

;;
;; Paste to the console window with Ctrl-Y if we're in one
;;
#IfWinActive, ahk_class ConsoleWindowClass
^y::SendInput ! ep
#IfWinActive

Once you move past simple keystroke automation, you can also think of things you’d like to do that automate PowerShell itself – such as clear the screen. For that, you press ESC to clear the current line, “type” CLS, and then press ENTER:

;;
;; Clear screen with ^L if we're in a console window
;;
#IfWinActive, ahk_class ConsoleWindowClass
^l::SendInput {ESC}cls{ENTER}
#IfWinActive

Or, an exceedingly useful command to wrap the current line in parentheses:

;;
;; Wrap current line in brackets
;;
#IfWinActive, ahk_class ConsoleWindowClass
^w::SendInput {HOME}({END})
#IfWinActive

While you’re at it, you might have found that it is extremely annoying to demonstrate a text-based prompt to colleagues, or during a presentation. You fumble with the properties window to pick a font and text size, but these do it all for you:

;;
;; Font larger with ^UP if we're in a console window
;;
#IfWinActive, ahk_class ConsoleWindowClass
^UP::
Sleep 300
SendInput ! p
Sleep 200
SendInput {CTRLDOWN}{TAB}{CTRLUP}
Send !s
Sleep 200
Send {DOWN}
Sleep 200
Send {CTRLDOWN}{SHIFTDOWN}{TAB}{SHIFTUP}{CTRLUP}
Sleep 200
Send {ENTER}
Sleep 200
Send {ENTER}
return
#IfWinActive

;;
;; Font smaller with ^DOWN if we're in a console window
;;
#IfWinActive, ahk_class ConsoleWindowClass
^DOWN::
Sleep 300
SendInput ! p
Sleep 200
SendInput {CTRLDOWN}{TAB}{CTRLUP}
Send !s
Sleep 200
Send {UP}
Sleep 200
Send {CTRLDOWN}{SHIFTDOWN}{TAB}{SHIFTUP}{CTRLUP}
Sleep 200
Send {ENTER}
Sleep 200
Send {ENTER}
return
#IfWinActive

All-in-all, quite a useful way to customize the console environment to your liking.

Martin Zugec recently wrote about a security concern he has with PowerShell: Once you have a credential (i.e.: from Get-Credential,) it is quite easy to get the plain-text password by calling its GetNetworkCredential() method.

The reason we don’t make this more difficult is that you already have the credential. In security, something is either possible or impossible -- "hard" is just another way of saying "possible." We designed the GetNetworkCredential() method to give you the System.Net.NetworkCredential class required by many network classes in the .NET Framework, so we definitely want it to be possible.

You might point out that the password in a credential is a SecureString, so why is it so easy to get access to? To be clear, a SecureString doesn’t protect you from yourself (or anybody using your account,) it protects the credential from people that don’t have access to your user account (but have access to a crash dump, page file, or the like.)

The core point here is that if PowerShell has access to the password in any way, so do you. In fact, getting the plain text of a SecureString is already a one-liner:

[C:\temp]
PS:23 > $secureString = Read-Host -AsSecureString
***********

[C:\temp]
PS:24 > [Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($secureString))
Hello World

Now, let’s say that you are concerned about calling GetNetworkCredential() while somebody watches over your shoulder, hence exposing your password. For this, you can modify the way we display all types of the System.Net.NetworkCredential class with this view definition:

        <View>
            <Name>System.Net.NetworkCredential</Name>
            <ViewSelectedBy>
                <TypeName>System.Net.NetworkCredential</TypeName>
            </ViewSelectedBy>

            <TableControl>
                <TableHeaders>
                    <TableColumnHeader>
                        <Label>UserName</Label>
                        <Width>50</Width>
                    </TableColumnHeader>
                    <TableColumnHeader>
                        <Label>Domain</Label>
                        <Width>50</Width>
                    </TableColumnHeader>
                </TableHeaders>
                <TableRowEntries>
                    <TableRowEntry>
                        <TableColumnItems>
                            <TableColumnItem>
                                <PropertyName>UserName</PropertyName>
                            </TableColumnItem>
                            <TableColumnItem>
                                <PropertyName>Domain</PropertyName>
                            </TableColumnItem>
                        </TableColumnItems>
                    </TableRowEntry>
                 </TableRowEntries>
            </TableControl>
        </View>        

For more information about how to make custom formatting files, these posts are a useful resource:

http://www.leeholmes.com/blog/AddFileDescriptionsToYourDirectoryListingsDESCRIPTIONInMonad.aspx
http://www.leeholmes.com/blog/DESCRIPTIONSupportInMonadPart2.aspx
http://www.leeholmes.com/blog/DESCRIPTIONSupportInMonadPart3.aspx

And, in action:

[C:\temp]
PS:4 > $cred = Get-Credential

cmdlet Get-Credential at command pipeline position 1
Supply values for the following parameters:
Credential

[C:\temp]
PS:6 > $cred.GetNetworkCredential()

UserName                                           Domain
--------                                           ------
leeholm                                            CONTOSO

Ask any driver where they get their gas, and you've opened a deep discussion. There are the few that get it from wherever (including the canteen in the back of their trunk because they always run out,) but most drivers have a preference.

The first question is, "Where do you get your gas?" Do you go for the cheapest gas station you can find, or go to the expensive places because it must be better? For that question, it seems that there really isn't a difference:

A joint study by ABC News and the Maryland State Comptroller's Office examined the difference between name brand and generic gas. Chemists at the Maryland Fuel Testing Laboratory conducted a battery of tests: They verified that the gas was formulated correctly for the season, checked for contaminants like excessive sediments or diesel accidentally mixed with the gas, and they ran the gas through an elaborate engine to ensure that it was all the same 87 octane level.

Here's some good news for consumer: Regular and discount gas are basically similar.

"By and large, it's one and the same," said Bob Crawford of the Maryland Fuel Testing Lab. "You will find results will almost mirror each other. There are going to be slight variations, but gasoline is gasoline."

The primary difference between competing brands is the amount of detergent the distributors add to it. Even still, the difference is generally minor: The EPA requires all gas to have a minimum amount of detergent to keep a car's engine clean.

"You would be paying for brand loyalty, primarily," said Crawford, explaining why brand name gas costs more. "Some people feel more comfortable dealing with a particular brand."

Crawford, who has been studying fuel for 36 years, uses whatever gas is the cheapest. Some cars require premium gas, but generally, regular is fine.

That last quote raises a good point. If the choice of station really doesn't make a difference, what about the octane? Some cars explicitly require Premium gas, but what about the others? Just as with software, the only way to be sure about the value of your gas is to measure it.

For example, I used to drive a 1998 Hyundai Accent. I gave it the cheapest gas, because it didn’t deserve any better :) Then one day, I tried Super Premium Ultra Deluxo gas on a lark, and got 19% better gas mileage.  Since premium fuel is only about 9-10% more expensive than dirt cheap fuel, it is well worth the money. So I turned into one of the people I used to laugh at: dumping super premium gas into a car with 107 HP.

It really comes down to figuring out the gas that gives you the best “miles per dollar,” and stick with it.

(Miles per dollar = Miles per Gallon / Dollars per Gallon)

The exact mileage you get from a given fill-up is easy to determine. Reset your trip odometer to zero just after you fill up. All the way up. The next time you fill your tank, make sure to fill it all the way up again. The pump will tell you how many gallons it gave you. Then, divide the measurement on your trip odometer by the number of gallons you just put in the tank, and you have your car’s mileage from the last fill up. Then, reset your trip odometer and start again.

Different octanes may have an effect on Miles per Gallon for your car, and will most definitely have an effect on Dollars per Gallon. Using the same octane from different suppliers may have an effect on Miles per Gallon, and will probably have an effect on Dollars per Gallon.

So, to figure out the best fuel for you, make a simple chart:

Place / Fuel      Miles per Gallon        Dollars per Gallon            Miles per Dollar
Safeway 89        17                      1.40                          10
Safeway 92        21                      1.65                          12.72
(etc)

For the Dollars per Gallon number, make sure to use the real price. For example, AARCO AM/PM adds a 45-cent surcharge per tank, which increases their advertized dollars per gallon by about 4 cents per gallon.

Experiment for awhile, and settle on the place that gives you the most Miles per Dollar.

An interesting problem recently came up in the PowerShell newsgroup, where somebody was trying to use a tool (jpegtopnm.exe) that let you pipe its binary output into a file. This works on Unix (and from cmd.exe,) but unfortunately generated a corrupt image when done in PowerShell.

When PowerShell launches a native application, one of the benefits we provide is allowing you to use PowerShell commands to work with the output. For example:

[C:\temp]
PS:64 > (IpConfig) -match "Gateway"
   Default Gateway . . . . . . . . . : xxx.yy.zz.a
   Default Gateway . . . . . . . . . :

We support this by splitting the output of the program on its newline characters, and pass each line independently down the pipeline. We support programs that use the Unix newline (\n) as well as the Windows newline (\r\n)

If the program outputs binary data, however, there is a risk of corrupting that data when we redirect it into a file. We will first split the program's output on "\n" or "\r\n", and generate an array of strings that do not have the newline character. When we send them into the file, we use the Windows newline character as a separator -- which effectively converts all 0x0D characters in the program's output to the sequence of 0x0D,0x0A. Hence, the corrupted file.

You can see that in action here (Format-Hex comes from the PowerShell Community Extensions project:)

From a PPM file generated in DOS:

[C:\temp]
PS:74 > cmd
Microsoft Windows [Version 5.2.3790]
(C) Copyright 1985-2003 Microsoft Corp.

C:\temp>jpegtopnm.exe IMAGE_00002.jpg > pic.ppm
jpegto~1.exe: WRITING PPM FILE

C:\temp>exit

[C:\temp]
PS:11 > gi pic.ppm | format-hex -NumBytesPerLine 10

Address:  0  1  2  3  4  5  6  7  8  9 ASCII
-------- ----------------------------- ----------
00000000 50 36 0A 34 38 30 20 36 34 30 P6.480 640
0000000A 0A 32 35 35 0A 20 23 2C 1F 22 .255. #,."
00000014 2B 22 20 2D 24 20 2E 28 21 31 +" -$ .(!1
0000001E 29 20 31 2A 1C 33 28 1A 31 2A ) 1*.3(.1*

From a PPM file generated in PowerShell:

[C:\temp]
PS:65 > .\jpegtopnm.exe IMAGE_00002.jpg | Out-File -Encoding OEM pic2.ppm
jpegto~1.exe: WRITING PPM FILE

[C:\temp]
PS:66 > gi pic2.ppm | Format-Hex -NumBytesPerLine 10

Address:  0  1  2  3  4  5  6  7  8  9 ASCII
-------- ----------------------------- ----------
00000000 50 36 0D 0A 34 38 30 20 36 34 P6..480 64
0000000A 30 0D 0A 32 35 35 0D 0A 20 23 0..255.. #
00000014 2C 1F 22 2B 22 20 2D 24 20 2E ,."+" -$ .
0000001E 28 21 31 29 20 31 2A 1C 33 28 (!1) 1*.3(
00000028 1A 31 2A 1C 33 2A 1C 33 29 1D .1*.3*.3).
00000032 31 28 1F 32 28 1F 30 25 21 30 1(.2(.0%!0

So the solution here is to not have PowerShell interpret the output stream of the program that generates this binary data:

[C:\temp]
PS:67 > Get-ProcessOutputAsBinary C:\temp\jpegtopnm.exe IMAGE_00002.jpg | Out-File -Encoding OEM pic3.ppm
jpegto~1.exe: WRITING PPM FILE

[C:\temp]
PS:68 > gi pic3.ppm | Format-Hex -NumBytesPerLine 10

Address:  0  1  2  3  4  5  6  7  8  9 ASCII
-------- ----------------------------- ----------
00000000 50 36 0A 34 38 30 20 36 34 30 P6.480 640
0000000A 0A 32 35 35 0A 20 23 2C 1F 22 .255. #,."
00000014 2B 22 20 2D 24 20 2E 28 21 31 +" -$ .(!1
0000001E 29 20 31 2A 1C 33 28 1A 31 2A ) 1*.3(.1*
00000028 1C 33 2A 1C 33 29 1D 31 28 1F .3*.3).1(.
00000032 32 28 1F 30 25 21 30 26 22 30 2(.0%!0&"0

The script that does that is here:

## Get-ProcessOutputAsBinary.ps1
## Get the standard output of a process as binary, rather than
## the interpreted array of strings that PowerShell traditionally
## produces.

param([string] $processname, [string] $arguments)

$processStartInfo = New-Object System.Diagnostics.ProcessStartInfo
$processStartInfo.FileName = $processname
$processStartInfo.WorkingDirectory = (Get-Location).Path
if($arguments) { $processStartInfo.Arguments = $arguments }
$processStartInfo.UseShellExecute = $false
$processStartInfo.RedirectStandardOutput = $true

$process = [System.Diagnostics.Process]::Start($processStartInfo)
$process.WaitForExit()
$process.StandardOutput.ReadToEnd()

 

Recently, Jeffrey wrote a post, “Blog your initial PowerShell  experiences.” I thought, “Even better, blog your automation!” So much of what makes people fall in love with PowerShell comes from the little problems it helps you solve. The more of those techniques we can help people learn, the better.

That post turned into “Break your Writer’s Block” – which is good, because there is already a good discussion going about blogging your automation here and here. Make sure to read the comments (if you haven’t already,) as they are an excellent source of reader contribution.

One thing I found myself doing the other day was trying to calculate how much I had contributed to my 401(k) plan. The web interface was annoying, but they did offer an option to download a .QIF file that contained my transaction history.

The.QIF file format uses newlines to separate elements of a record, and the carat (^) to separate records – like this:

!Type:Invst
D01/13/2006
NBuy
Y<Name of Mutual Fund>
I<Something>
Q<Shares>
U<Something>
T<Transaction Amount>
MContribution                 
^
D01/31/2006
NBuy
Y<Name of Mutual Fund>
I<Something>
Q<Shares>
U<Something>
T<Transaction Amount>
MExchange In                  
^
(...)
^
D12/29/2006
NBuy
Y<Name of Mutual Fund>
I<Something>
Q<Shares>
U<Something>
T<Transaction Amount>
MContribution                 

Each field is prefixed by a letter (D, N, Y, I, Q, U, T, M) to signify that field’s purpose. Normally, this task could just be accomplished by selecting all of the “T” lines (Transaction Amount,) and then adding them up. However, not all of the records represent contributions (M = “Contribution”) to the plan. A few of them represent exchanges (M = “Exchange In”, M = “Exchange Out”) between plans, which would result in double-counting those contributions.

So what’s a scripter to do? One line, that’s what:

Get-Content c:\temp\401.txt -Delimiter "^" | Parse-TextObject -Delimiter "`n" | ? { $_.Property9 -like "*MContribution*" } | % { $_.Property8.Replace("T","") } | Measure-Object –Sum

First, we read all of the text from the file as a character stream, using the -Delimiter option to break it into records by splitting on the “^” character. For each of those records, we pass it into the Parse-TextObject script (telling it that fields are delimited by a newline character.) Then, we pass along all records where the 9th property was a contribution, remove the “T” in transaction amount field, and then calculate the sum.

It turns out that the QIF file didn’t contain the granularity I needed (as it rolled my contributions and the company match on contributions together,) but the exercise was helpful in any case.

Some more PowerShell books are becoming available -- this is a very exciting time!

First, there's Don Jones & Jeffrey Hicks' PowerShell: TFM
Second, there's Jerry Lee Ford Jr.'s Microsoft Windows Powershell Programming for the Absolute Beginner

Congrats on a huge milestone, and thank you for your contribution!