Precision Computing

It turns out that the chisel nib on a thick whiteboard marker is actually exceedingly good for whiteboard calligraphy. It gives an x-height of about 20mm, though :)

The only problem is the guide lines – since on a whiteboard, there are none :) I’ve tried penciling them in, but that doesn’t work very well. I think my solution for that will be the true geek way – lasers.

One of the things people commonly ask about is support for rich keyboard macros and hotkeys in PowerShell.  For many, a pointer to the Doskey hotkey reference (which provides the history management functionality in PowerShell’s “cooked mode”) is enough: http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/doskey.mspx?mfr=true.

Once you’ve become used to those features, though, you start to want more – especially if you’ve grown accustomed to some of the power available on traditional Unix shells. Typical questions include:

• How do I use Emacs keystrokes to navigate my history?
• How do I use the keyboard to copy and paste in PowerShell?
• How do I clear the screen with a keystroke?
• How do exit the shell with a keystroke?
• How do I use the keyboard to adjust the font size in PowerShell?

Although PowerShell itself doesn’t directly offer these additional customization options, there is still an outlet – UI Automation and Keystroke macro programs.

Programs that support keystroke macros are great, as they let you automate anything that you can already do with a keyboard (and sometimes the mouse.) For example, you can use the Alt+Space combination in any console window to open up its system menu.  A console system window includes all kinds of tasty treats – copy, paste, find, and even properties.

Using these menus and a program that supports keystroke macros, we can customize most of the things we want to.

I personally use a program called AutoHotkey as my macro program. It has a lot of power, although its scripting language leaves a lot to be desired. I think it would be the coolest thing ever if somebody wrote a macro program that hosted PowerShell – giving you all the benefits of PowerShell scripting for UI automation.

AutoHotkey uses a script to let you write keystroke macros. You tell it what keys to listen for, and what it should do when it sees those keys. Let’s take some simple examples.

One thing you might wonder is how to use Emacs keystrokes to navigate your history. The console cooked mode already supports many of the concepts that you want, but just doesn’t use the keystrokes that you want. You want to push Ctrl-A, but the console uses HOME for that. You want to push Ctrl-E, but the console uses END for that. So in your macro, tell AutoHotkey to send the HOME key when you push Ctrl-A:

;;
;; Beginning of line with ^A if we're in a console window
;;
#IfWinActive, ahk_class ConsoleWindowClass
^a::SendInput {HOME}
#IfWinActive

That sequence tells AutoHotkey: if the class of the current window is ConsoleWindowClass, then run these commands when I press ^a. Those commands are to send the HOME key as input. Similar macros work for the other navigation commands you might want to remap.
These keystrokes can get even more complex, if desired. For example, you want Ctrl-Y (the Emacs keystroke for Paste) to paste into a console window. The console doesn’t directly support that with a keystroke substitution, but the glorious system menu does:

You can automate that with Alt+Space, E, P:

;;
;; Paste to the console window with Ctrl-Y if we're in one
;;
#IfWinActive, ahk_class ConsoleWindowClass
^y::SendInput ! ep
#IfWinActive

Once you move past simple keystroke automation, you can also think of things you’d like to do that automate PowerShell itself – such as clear the screen. For that, you press ESC to clear the current line, “type” CLS, and then press ENTER:

;;
;; Clear screen with ^L if we're in a console window
;;
#IfWinActive, ahk_class ConsoleWindowClass
^l::SendInput {ESC}cls{ENTER}
#IfWinActive

Or, an exceedingly useful command to wrap the current line in parentheses:

;;
;; Wrap current line in brackets
;;
#IfWinActive, ahk_class ConsoleWindowClass
^w::SendInput {HOME}({END})
#IfWinActive

While you’re at it, you might have found that it is extremely annoying to demonstrate a text-based prompt to colleagues, or during a presentation. You fumble with the properties window to pick a font and text size, but these do it all for you:

;;
;; Font larger with ^UP if we're in a console window
;;
#IfWinActive, ahk_class ConsoleWindowClass
^UP::
Sleep 300
SendInput ! p
Sleep 200
SendInput {CTRLDOWN}{TAB}{CTRLUP}
Send !s
Sleep 200
Send {DOWN}
Sleep 200
Send {CTRLDOWN}{SHIFTDOWN}{TAB}{SHIFTUP}{CTRLUP}
Sleep 200
Send {ENTER}
Sleep 200
Send {ENTER}
return
#IfWinActive

;;
;; Font smaller with ^DOWN if we're in a console window
;;
#IfWinActive, ahk_class ConsoleWindowClass
^DOWN::
Sleep 300
SendInput ! p
Sleep 200
SendInput {CTRLDOWN}{TAB}{CTRLUP}
Send !s
Sleep 200
Send {UP}
Sleep 200
Send {CTRLDOWN}{SHIFTDOWN}{TAB}{SHIFTUP}{CTRLUP}
Sleep 200
Send {ENTER}
Sleep 200
Send {ENTER}
return
#IfWinActive

All-in-all, quite a useful way to customize the console environment to your liking.

Martin Zugec recently wrote about a security concern he has with PowerShell: Once you have a credential (i.e.: from Get-Credential,) it is quite easy to get the plain-text password by calling its GetNetworkCredential() method.

The reason we don’t make this more difficult is that you already have the credential. In security, something is either possible or impossible -- "hard" is just another way of saying "possible." We designed the GetNetworkCredential() method to give you the System.Net.NetworkCredential class required by many network classes in the .NET Framework, so we definitely want it to be possible.

You might point out that the password in a credential is a SecureString, so why is it so easy to get access to? To be clear, a SecureString doesn’t protect you from yourself (or anybody using your account,) it protects the credential from people that don’t have access to your user account (but have access to a crash dump, page file, or the like.)

The core point here is that if PowerShell has access to the password in any way, so do you. In fact, getting the plain text of a SecureString is already a one-liner:

[C:\temp]
PS:23 > $secureString = Read-Host -AsSecureString
***********

[C:\temp]
PS:24 > [Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($secureString))
Hello World

Now, let’s say that you are concerned about calling GetNetworkCredential() while somebody watches over your shoulder, hence exposing your password. For this, you can modify the way we display all types of the System.Net.NetworkCredential class with this view definition:

        <View>
            <Name>System.Net.NetworkCredential</Name>
            <ViewSelectedBy>
                <TypeName>System.Net.NetworkCredential</TypeName>
            </ViewSelectedBy>

            <TableControl>
                <TableHeaders>
                    <TableColumnHeader>
                        <Label>UserName</Label>
                        <Width>50</Width>
                    </TableColumnHeader>
                    <TableColumnHeader>
                        <Label>Domain</Label>
                        <Width>50</Width>
                    </TableColumnHeader>
                </TableHeaders>
                <TableRowEntries>
                    <TableRowEntry>
                        <TableColumnItems>
                            <TableColumnItem>
                                <PropertyName>UserName</PropertyName>
                            </TableColumnItem>
                            <TableColumnItem>
                                <PropertyName>Domain</PropertyName>
                            </TableColumnItem>
                        </TableColumnItems>
                    </TableRowEntry>
                 </TableRowEntries>
            </TableControl>
        </View>        

For more information about how to make custom formatting files, these posts are a useful resource:

http://www.leeholmes.com/blog/AddFileDescriptionsToYourDirectoryListingsDESCRIPTIONInMonad.aspx
http://www.leeholmes.com/blog/DESCRIPTIONSupportInMonadPart2.aspx
http://www.leeholmes.com/blog/DESCRIPTIONSupportInMonadPart3.aspx

And, in action:

[C:\temp]
PS:4 > $cred = Get-Credential

cmdlet Get-Credential at command pipeline position 1
Supply values for the following parameters:
Credential

[C:\temp]
PS:6 > $cred.GetNetworkCredential()

UserName                                           Domain
--------                                           ------
leeholm                                            CONTOSO

Ask any driver where they get their gas, and you've opened a deep discussion. There are the few that get it from wherever (including the canteen in the back of their trunk because they always run out,) but most drivers have a preference.

The first question is, "Where do you get your gas?" Do you go for the cheapest gas station you can find, or go to the expensive places because it must be better? For that question, it seems that there really isn't a difference:

A joint study by ABC News and the Maryland State Comptroller's Office examined the difference between name brand and generic gas. Chemists at the Maryland Fuel Testing Laboratory conducted a battery of tests: They verified that the gas was formulated correctly for the season, checked for contaminants like excessive sediments or diesel accidentally mixed with the gas, and they ran the gas through an elaborate engine to ensure that it was all the same 87 octane level.

Here's some good news for consumer: Regular and discount gas are basically similar.

"By and large, it's one and the same," said Bob Crawford of the Maryland Fuel Testing Lab. "You will find results will almost mirror each other. There are going to be slight variations, but gasoline is gasoline."

The primary difference between competing brands is the amount of detergent the distributors add to it. Even still, the difference is generally minor: The EPA requires all gas to have a minimum amount of detergent to keep a car's engine clean.

"You would be paying for brand loyalty, primarily," said Crawford, explaining why brand name gas costs more. "Some people feel more comfortable dealing with a particular brand."

Crawford, who has been studying fuel for 36 years, uses whatever gas is the cheapest. Some cars require premium gas, but generally, regular is fine.

That last quote raises a good point. If the choice of station really doesn't make a difference, what about the octane? Some cars explicitly require Premium gas, but what about the others? Just as with software, the only way to be sure about the value of your gas is to measure it.

For example, I used to drive a 1998 Hyundai Accent. I gave it the cheapest gas, because it didn’t deserve any better :) Then one day, I tried Super Premium Ultra Deluxo gas on a lark, and got 19% better gas mileage.  Since premium fuel is only about 9-10% more expensive than dirt cheap fuel, it is well worth the money. So I turned into one of the people I used to laugh at: dumping super premium gas into a car with 107 HP.

It really comes down to figuring out the gas that gives you the best “miles per dollar,” and stick with it.

(Miles per dollar = Miles per Gallon / Dollars per Gallon)

The exact mileage you get from a given fill-up is easy to determine. Reset your trip odometer to zero just after you fill up. All the way up. The next time you fill your tank, make sure to fill it all the way up again. The pump will tell you how many gallons it gave you. Then, divide the measurement on your trip odometer by the number of gallons you just put in the tank, and you have your car’s mileage from the last fill up. Then, reset your trip odometer and start again.

Different octanes may have an effect on Miles per Gallon for your car, and will most definitely have an effect on Dollars per Gallon. Using the same octane from different suppliers may have an effect on Miles per Gallon, and will probably have an effect on Dollars per Gallon.

So, to figure out the best fuel for you, make a simple chart:

Place / Fuel      Miles per Gallon        Dollars per Gallon            Miles per Dollar
Safeway 89        17                      1.40                          10
Safeway 92        21                      1.65                          12.72
(etc)

For the Dollars per Gallon number, make sure to use the real price. For example, AARCO AM/PM adds a 45-cent surcharge per tank, which increases their advertized dollars per gallon by about 4 cents per gallon.

Experiment for awhile, and settle on the place that gives you the most Miles per Dollar.

An interesting problem recently came up in the PowerShell newsgroup, where somebody was trying to use a tool (jpegtopnm.exe) that let you pipe its binary output into a file. This works on Unix (and from cmd.exe,) but unfortunately generated a corrupt image when done in PowerShell.

When PowerShell launches a native application, one of the benefits we provide is allowing you to use PowerShell commands to work with the output. For example:

[C:\temp]
PS:64 > (IpConfig) -match "Gateway"
   Default Gateway . . . . . . . . . : xxx.yy.zz.a
   Default Gateway . . . . . . . . . :

We support this by splitting the output of the program on its newline characters, and pass each line independently down the pipeline. We support programs that use the Unix newline (\n) as well as the Windows newline (\r\n)

If the program outputs binary data, however, there is a risk of corrupting that data when we redirect it into a file. We will first split the program's output on "\n" or "\r\n", and generate an array of strings that do not have the newline character. When we send them into the file, we use the Windows newline character as a separator -- which effectively converts all 0x0D characters in the program's output to the sequence of 0x0D,0x0A. Hence, the corrupted file.

You can see that in action here (Format-Hex comes from the PowerShell Community Extensions project:)

From a PPM file generated in DOS:

[C:\temp]
PS:74 > cmd
Microsoft Windows [Version 5.2.3790]
(C) Copyright 1985-2003 Microsoft Corp.

C:\temp>jpegtopnm.exe IMAGE_00002.jpg > pic.ppm
jpegto~1.exe: WRITING PPM FILE

C:\temp>exit

[C:\temp]
PS:11 > gi pic.ppm | format-hex -NumBytesPerLine 10

Address:  0  1  2  3  4  5  6  7  8  9 ASCII
-------- ----------------------------- ----------
00000000 50 36 0A 34 38 30 20 36 34 30 P6.480 640
0000000A 0A 32 35 35 0A 20 23 2C 1F 22 .255. #,."
00000014 2B 22 20 2D 24 20 2E 28 21 31 +" -$ .(!1
0000001E 29 20 31 2A 1C 33 28 1A 31 2A ) 1*.3(.1*

From a PPM file generated in PowerShell:

[C:\temp]
PS:65 > .\jpegtopnm.exe IMAGE_00002.jpg | Out-File -Encoding OEM pic2.ppm
jpegto~1.exe: WRITING PPM FILE

[C:\temp]
PS:66 > gi pic2.ppm | Format-Hex -NumBytesPerLine 10

Address:  0  1  2  3  4  5  6  7  8  9 ASCII
-------- ----------------------------- ----------
00000000 50 36 0D 0A 34 38 30 20 36 34 P6..480 64
0000000A 30 0D 0A 32 35 35 0D 0A 20 23 0..255.. #
00000014 2C 1F 22 2B 22 20 2D 24 20 2E ,."+" -$ .
0000001E 28 21 31 29 20 31 2A 1C 33 28 (!1) 1*.3(
00000028 1A 31 2A 1C 33 2A 1C 33 29 1D .1*.3*.3).
00000032 31 28 1F 32 28 1F 30 25 21 30 1(.2(.0%!0

So the solution here is to not have PowerShell interpret the output stream of the program that generates this binary data:

[C:\temp]
PS:67 > Get-ProcessOutputAsBinary C:\temp\jpegtopnm.exe IMAGE_00002.jpg | Out-File -Encoding OEM pic3.ppm
jpegto~1.exe: WRITING PPM FILE

[C:\temp]
PS:68 > gi pic3.ppm | Format-Hex -NumBytesPerLine 10

Address:  0  1  2  3  4  5  6  7  8  9 ASCII
-------- ----------------------------- ----------
00000000 50 36 0A 34 38 30 20 36 34 30 P6.480 640
0000000A 0A 32 35 35 0A 20 23 2C 1F 22 .255. #,."
00000014 2B 22 20 2D 24 20 2E 28 21 31 +" -$ .(!1
0000001E 29 20 31 2A 1C 33 28 1A 31 2A ) 1*.3(.1*
00000028 1C 33 2A 1C 33 29 1D 31 28 1F .3*.3).1(.
00000032 32 28 1F 30 25 21 30 26 22 30 2(.0%!0&"0

The script that does that is here:

## Get-ProcessOutputAsBinary.ps1
## Get the standard output of a process as binary, rather than
## the interpreted array of strings that PowerShell traditionally
## produces.

param([string] $processname, [string] $arguments)

$processStartInfo = New-Object System.Diagnostics.ProcessStartInfo
$processStartInfo.FileName = $processname
$processStartInfo.WorkingDirectory = (Get-Location).Path
if($arguments) { $processStartInfo.Arguments = $arguments }
$processStartInfo.UseShellExecute = $false
$processStartInfo.RedirectStandardOutput = $true

$process = [System.Diagnostics.Process]::Start($processStartInfo)
$process.WaitForExit()
$process.StandardOutput.ReadToEnd()

 

Recently, Jeffrey wrote a post, “Blog your initial PowerShell  experiences.” I thought, “Even better, blog your automation!” So much of what makes people fall in love with PowerShell comes from the little problems it helps you solve. The more of those techniques we can help people learn, the better.

That post turned into “Break your Writer’s Block” – which is good, because there is already a good discussion going about blogging your automation here and here. Make sure to read the comments (if you haven’t already,) as they are an excellent source of reader contribution.

One thing I found myself doing the other day was trying to calculate how much I had contributed to my 401(k) plan. The web interface was annoying, but they did offer an option to download a .QIF file that contained my transaction history.

The.QIF file format uses newlines to separate elements of a record, and the carat (^) to separate records – like this:

!Type:Invst
D01/13/2006
NBuy
Y<Name of Mutual Fund>
I<Something>
Q<Shares>
U<Something>
T<Transaction Amount>
MContribution                 
^
D01/31/2006
NBuy
Y<Name of Mutual Fund>
I<Something>
Q<Shares>
U<Something>
T<Transaction Amount>
MExchange In                  
^
(...)
^
D12/29/2006
NBuy
Y<Name of Mutual Fund>
I<Something>
Q<Shares>
U<Something>
T<Transaction Amount>
MContribution                 

Each field is prefixed by a letter (D, N, Y, I, Q, U, T, M) to signify that field’s purpose. Normally, this task could just be accomplished by selecting all of the “T” lines (Transaction Amount,) and then adding them up. However, not all of the records represent contributions (M = “Contribution”) to the plan. A few of them represent exchanges (M = “Exchange In”, M = “Exchange Out”) between plans, which would result in double-counting those contributions.

So what’s a scripter to do? One line, that’s what:

Get-Content c:\temp\401.txt -Delimiter "^" | Parse-TextObject -Delimiter "`n" | ? { $_.Property9 -like "*MContribution*" } | % { $_.Property8.Replace("T","") } | Measure-Object –Sum

First, we read all of the text from the file as a character stream, using the -Delimiter option to break it into records by splitting on the “^” character. For each of those records, we pass it into the Parse-TextObject script (telling it that fields are delimited by a newline character.) Then, we pass along all records where the 9th property was a contribution, remove the “T” in transaction amount field, and then calculate the sum.

It turns out that the QIF file didn’t contain the granularity I needed (as it rolled my contributions and the company match on contributions together,) but the exercise was helpful in any case.

Some more PowerShell books are becoming available -- this is a very exciting time!

First, there's Don Jones & Jeffrey Hicks' PowerShell: TFM
Second, there's Jerry Lee Ford Jr.'s Microsoft Windows Powershell Programming for the Absolute Beginner

Congrats on a huge milestone, and thank you for your contribution!

On January 1, 2006, many companies began to offer the Roth 401(k): an alternative to the traditional 401(k) that allows you to contribute your money in after-tax dollars, rather than pre-tax dollars. Most (in my opinion incorrectly) summarize the difference as the following choice: "If you think you might be in a higher income tax bracket when you withdraw the funds, invest in the Roth 401(k). Otherwise, invest in a traditional 401(k)." Omar Shahine points to a good summary here: http://www.shahine.com/omar/Roth401k.aspx. (This started as a comment on that article, but quickly grew to a post in of itself.)

Here are some examples, for somebody that is married, filing separately.

Notice the "Difference per $1000" column. That shows how many dollars (per $1000) it costs if you make a mistake. Worrying about making a mistake causes people a lot of heartburn, so let's drill into that a bit more. What's the worst mistake you can make? It's the row that shows you earning $200,000 now, contributing to a Roth 401(k), and then withdrawing $30,000 per year during retirement. Over your entire 30-year retirement, that would cost 30 years * $30,000 per year / 1,000 * 220 wasted per thousand = $198,000. That's about $6600 per year. Not good, but not the end of the world. Given a medium-to-aggressive savings strategy, this scenario is also highly unlikely.

How should you estimate your post-retirement income? It will be based on how much you've managed to save before retirement, which is a dependent on how much you can save per year. Assuming a 6% annual rate of return, this PowerShell calculation shows how to estimate your "nest egg" for retirement, assuming that you contribute $5,000 per year:

PS >$balance = 0
PS >$contribution = 5000
PS >1..30 | % { $balance += $contribution; $balance *= 1.06 }
PS >$balance
419008.386940671

Now, for your annual retirement income, you can play around with your withdraw rate on this $419,008 nest egg until the final balance nears zero:

PS >$balance = 419008; 1..30 | % { $balance -= 28700; $balance *= 1.06 }; $balance
1460.60834058789

So, contributing around $5,000 per year gives you a post-retirement income of about $29,000. That will remain $29,000 if the contributions were through a Roth 401(k), but will turn to about $25,000 if the contributions were through a traditional 401(k).

Here's one thing that may play a part in your 401(k) calculations: how much you can hope to retire with. Both the traditional and Roth 401(k) plans have annual contribution limits of $15,500 (expected to increase by $500 per year.) If you are in a position to save the maximum, your choice of 401 vehicle makes a significant difference on your post-retirement income if you base it on your 401(k) alone.

If you manage to save the maximum contribution per year, that leaves you with:

PS >$balance = 0
PS >$contribution = 15000
PS >1..30 | % { $balance += $contribution; $balance *= 1.06; $contribution += 500 }
PS >$balance
1690372.4723898

Which gives an annual retirement income of about $116,000:

PS >$balance = 1690372; 1..30 | % { $balance -= 115800; $balance *= 1.06 }; $balance
4402.41939379471

If those funds were contributed through a traditional 401(k), that will shrink to about $87,000 after taxes. If those funds were contributed through a Roth 401(k), they will stay ay $116,000. Given the tax advantage of the Traditional 401(k), though, it may make sense in that situation to maximize your contribution to the Traditional 401(k), and apply the money you would have invested in the Roth 401(k) to other investment vehicles.

The interest earnings on those other vehicles will be taxed at 10%, though, so the answer to this is highly dependent on your personal situation. Take a salary that seems to most highly favour the Traditional IRA, but where that person can afford to max out the Roth 401(k). That is at the 35% tax bracket:

## Scenario 1: Max out Traditional

   # A) First, max out Traditional
   $balance = 0
   $contribution = 15000
   1..30 | % { $balance += $contribution; $contribution += 500; $balance *= 1.06 }
   $balance
   1690372

   $balance = 1690372; 1..30 | % { $balance -= 115800; $balance *= 1.06 }; $balance
   = $115,800 per year minus tax

   # B) Then, apply your extra (vs. the Roth) to mutual funds
   $balance = 0
   $contributed = 0
   $marginalTaxRate = 1.35
   $limit = 15000
   $contribution = (($limit * $marginalTaxRate) - $limit) / $marginalTaxRate
   1..30 | % {
      $balance += $contribution; $contributed += $contribution; $balance *= 1.06;
      $limit += 500; $contribution = (($limit * $marginalTaxRate) - $limit) / $marginalTaxRate
   }
   $balance = ($balance - $contributed) * 0.9 + $contributed
   $balance
   411725.799113175

   $balance = 411725; 1..30 | % { $balance -= 36200; $balance *= 1.06 }; $balance
   = 28,200 per year (tax free)

   # C) Then figure out the combination of investments

   Total = 144,000 pre tax (A + B) = 25% effective tax rate
          = 115,050 per year after tax (0.75 * A + B)

## Scenario 2: Max out Roth

   $balance = 0
   $contribution = 15000
   1..30 | % { $balance += $contribution; $balance *= 1.06; $contribution += 500 }
   $balance
   1690372.4723898

   $balance = 1690372; 1..30 | % { $balance -= 115800; $balance *= 1.06 }; $balance
   = 115,800 per year after tax

Lower tax brackets than 35% (ie all of them) favour the Roth even more heavily if that person can afford to max out their contributions. As always, make your own informed decisions regarding investments, but hopefully this exercise can help clarify some things.

[Edit: Updated table -- Brian Kramp reminded me that Roth 401(k) contributions are taxed at your marginal tax rate, not the effective tax rate.]
[Edit2: Added calculations for investing in mutual funds]