Precision Computing

If you're testing your PowerShell scripts (manually or automatically,) one of the first questions you'll end up asking yourself is, "did I test enough?"

This is common problem in all of software development. To the rescue is a simple metric known as "Code Coverage" – a measure of how much code you exercised during the testing of that code.

However, the measurement is just the end result – you need a tool to get you there. When it comes to measuring code coverage in PowerShell scripts, though, there simply aren't any tools yet.

Like performance measurement tools, Code Coverage tools are sometimes driven by instrumentation of the source code, and sometimes driven by sampling the code during runtime. Instrumentation gives the highest accuracy, but we can go a long way by runtime analysis alone. To accomplish that, we'll use our favourite feature to abuse – PowerShell script tracing. The last time we pushed it, we got a sampling profiler out of the deal. Let's do it again for code coverage.

Take, for example, the following source code:

trap { "Error handling!"; continue }                                  
                                                                      
"Got here"                                                            
                                                                      
if($args[0] -eq "Test")                                               
{                                                                     
   "Got TEST as an argument"                                          
}                                                                     
elseif($args[0] -eq "Err0r")                                          
{                                                                     
   throw "Catch Me!"                                                  
}                                                                     
else                                                                  
{                                                                     
   "Didn't get TEST as an argument"                                   
}     

We want to run through three parameters that it takes, and make sure we're exercising everything. Notice how we're even being diligent by testing the "Error" case!

PS C:\temp> $tests = @()                                              
PS C:\temp> $tests += { .\Test-CodeCoverage.ps1 Test }                
PS C:\temp> $tests += { .\Test-CodeCoverage.ps1 SomethingElse }       
PS C:\temp> $tests += { .\Test-CodeCoverage.ps1 Error }               
PS C:\temp>                                                           
PS C:\temp> .\Get-ScriptCoverage.ps1 .\Test-CodeCoverage.ps1 $tests 

What does that give us?

trap { "Error handling!"; continue }                                  
                                                                      
"Got here"                                                            
                                                                      
if($args[0] -eq "Test")                                               
{                                                                     
   "Got TEST as an argument"                                          
}                                                                     
elseif($args[0] -eq "Err0r")                                          
{                                                                     
   throw "Catch Me!"                                                  
}                                                                     
else                                                                  
{                                                                     
   "Didn't get TEST as an argument"                                   
}                                                                     
Coverage Statistics: 66.6666666666667%                                
PS C:\temp>   

Ouch! Why is the error handling code (in red) not being hit? Ah, after further investigation, it turns out that we have a typo in our string comparison. We fix it:

...
elseif($args[0] -eq "Err0r")
...

Becomes

...
elseif($args[0] -eq "Error")
...

And run code coverage again:

trap { "Error handling!"; continue }                                  
                                                                      
"Got here"                                                            
                                                                      
if($args[0] -eq "Test")                                               
{                                                                     
   "Got TEST as an argument"                                          
}                                                                     
elseif($args[0] -eq "Error")                                          
{                                                                     
   throw "Catch Me!"                                                  
}                                                                     
else                                                                  
{                                                                     
   "Didn't get TEST as an argument"                                   
}                                                                     
Coverage Statistics: 100%                                             
PS C:\temp>    

Much better.

Here is the script – under 80 lines of (heavily commented) code:

## Get-ScriptCoverage.ps1                                             
## Test the script named by $testScript for code coverage.            
## The command given by $command must exercise this named             
## script.                                                            
param([string] $testScript, [ScriptBlock[]] $command)                 
                                                                      
# Store the content of the script to be tested                        
$fileContent = gc $testScript -ea Stop                                
                                                                      
## Start a transcript, and log it to a file                           
$tempFile = [IO.Path]::GetTempFilename()                              
Start-Transcript $tempFile                                            
                                                                      
## Turn on line-level tracing, run the command(s),                    
## then turn off line-level tracing again.                            
Set-PsDebug -Trace 1                                                  
$command | Foreach-Object { & $_ }                                    
Set-PsDebug -Trace 0                                                  
                                                                      
## Stop the transcript                                                
Stop-Transcript                                                       
                                                                      
## Get the result of the script coverage run                          
$coverageContent = (gc $tempFile) -match "^DEBUG:"                    
Remove-Item -LiteralPath $tempFile                                    
                                                                      
Clear-Host                                                            
                                                                      
## Clean up interference from other scripts                           
$scriptLines = @()                                                    
$processedLines = @{}                                                 
                                                                      
foreach($originalLine in $coverageContent)                            
{                                                                     
    # Make sure we only process unique lines in the                   
    # transcript                                                      
    if($processedLines[$originalLine]) { continue }                   
    $processedLines[$originalLine] = $true                            
                                                                      
    ## Recover as much as possible from the original script line      
    ## without its debugging information                              
    $originalLine = $originalLine -replace " <<<< ",""                
    $line = $originalLine -replace '\D*\d+\+ (.*)','$1'               
                                                                      
    ## Go through each line in the original script, and see if        
    ## this is actually in the script                                 
    foreach($fileLine in $fileContent)                                
    {                                                                 
        ## If it is, add the debug line to the list of lines          
        ## covered by this scenario                                   
        if($fileLine.Contains($line))                                 
        {                                                             
            $scriptLines += $originalLine                             
        }                                                             
    }                                                                 
}                                                                     
                                                                      
## Find out which line numbers were covered                           
$coveredLines = $scriptLines |                                        
    % { $_ -replace '\D*(\d+)\+ .*','$1' } | Sort -Unique             
                                                                      
$coverageCount = 0                                                    
$possibleCoveredLines = 0                                             
for($counter = 1; $counter -le $fileContent.Count; $counter++)        
{                                                                     
    $color = "Red"                                                    
    $line = $fileContent[$counter - 1]                                
                                                                      
    ## Ignore comments, blank lines, curly                            
    ## braces, and fall-through conditional statements                
    ## in coverage computation (as they are never                     
    ## traced in Set-PsDebug tracing                                  
    if(($line -notmatch '^\s*#') -and                                 
       ($line -notmatch '^\s*{\s*$') -and                             
       ($line -notmatch '^\s*}\s*$') -and                             
       ($line -notmatch '^\s*else') -and                              
       ($line -notmatch '^\s*param\(') -and                           
       ($line.Trim()))                                                
    {                                                                 
        $possibleCoveredLines++                                       
    }                                                                 
    else { $color = "Gray" }                                          
                                                                      
    ## If this line was hit in code coverage, colour it               
    ## green                                                          
    if($coveredLines -contains $counter)                              
    {                                                                 
        $color = "Green"                                              
        $coverageCount++                                              
    }                                                                 
                                                                      
    ## Display the line in the appropriate colour                     
    Write-Host -Fore $color $line                                     
}                                                                     
                                                                      
## Output the coverage statistics                                     
Write-Host ("Coverage Statistics: " +                                 
    "$($coverageCount / $possibleCoveredLines * 100)%")               

For a long time, I've been thinking about learning to fly. One of the random long-distance-drive conversations I like to have is, "What job would you do if you were for some reason prevented from doing what you do now?" For me, flying has always been something I would entertain, but only romantically. The downsides to doing it professionally are huge -- the time away from your family and strict seniority-based promotions being the two largest factors. So instead, I wanted to reward myself with flying lessons the next time I thought it was appropriate. A King of the Hill episode, of all things, changed my mind on that.

In that episode, Hank (the father) and Peggy (the mother) were going through marital problems, and spoke with a counselor. The counselor asked them what they wanted to do when they retired -- something they had been planning toward for a long time. Their dream was to buy some motorcycles and tour the United States together. The counselor's suggestion was simple -- holding off on buying the bikes doesn't really help anything, so just buy the darn things and start enjoying yourselves! The cross-country tours can wait until you have the time to do them, but little local trips can happen right now.

That started the slow chemical reaction in my brain, which ultimately led me to start investigating flying lessons more seriously.

My typical obsessive online research turned up lots of information about learning to fly. Tom Unger’s “Flying Lessons” journal was indispensable, gave a lot of great background information, and what to expect with the process. However, I found very little about picking a flight school around the Seattle area, and sent a mail to an internal mailing list asking for opinions. Four suggestions came up most frequently:

• Pro-Flight Aviation: http://www.flyproflight.com/
• Galvin Flying Service: http://www.galvinflying.com/
• Acu-Wings: http://www.acuwings.com/
• Alternate Air: http://www.alternateair.com/

I ruled out Galvin immediately – their responses to customer complaints on http://www.airnav.com/airport/KBFI/GALVIN were ridiculous and showed complete disregard for their customers.

I ended up looking for operations out of the Renton Municipal Airport (KRNT) as my primary factor. It’s close to my house, and commute time seems to be the most important element in choosing a base. Airport traffic and other factors can be learned at other airports (such as Crest Airpark for its crazy small field, and Boeing Field for its busy airspace,) since most of your practice take-offs and landings come from explicit training exercises (as opposed to being dictated by the base you fly from.)

I first checked out Pro-Flight during a 9:00 AM intro flight, taking up a well-loved Cessna 172. Introductory flights are an amazing deal – even as a leisure activity. They cost under $100 for a scenic tour of the Puget Sound, and you get to do most of the flying yourself.

The biggest challenge (as compared to driving a car) was adapting to a new mental model, and adapting to a very manual control system. When taxiing, having 4 control surfaces at my feet was pretty complex – seeing as I kept on thinking of it like controlling handles on a bike steering column. I would instinctively push right expecting to turn left, but got the hang of it more when I thought about it as differential braking (even when I was using the nose wheel part of the control.) My only point of minor helmet fire came at this point – when we needed to use differential braking to make a turn, but my feet were only on the nose wheel portion of the control (as suggested by the instructor.)

Oh, and ignoring my internal alarm bells is hard enough when starting an automatic transmission car without depressing a clutch – an airplane was much worse.

What surprised me most was how much input you had to give the controls. It felt like there was a good 8 inches of travel in the foot pedals – much more than I was expecting. I was sure I was going to rip the knob off of the dash when I was priming the engine.

After we taxied, takeoff went pretty well. There was no traffic, so we didn’t burn engine hours waiting in line. Actually leaving the pavement was weird, as I again was expecting to need a subtle touch. After pushing the throttle forward slowly like you see them do on big airplanes, the instructor told me to just gun it :) The climb felt surprisingly manual as well, although the instructor did give me a rate of ascent to aim for, which helped a lot. Once I visually leveled off, the instructor pointed out that I was still climbing.  The nose definitely points a lot lower during steady flight than it does during takeoff, but adapting to that came shortly.

We did some minor orientation and directional work, and even got to do some nice steep turns. The ceiling was really indefinite, and we had to avoid some cloud cover. The runway approach went really well as we approached from the North. I guess since there was no traffic, we basically did the 45 degree entry directly into an extended final for the traffic pattern. Lining up felt pretty natural, although I made the approach too low – I was aiming for the numbers instead of the first dash.

So all-in-all, it was a great time up – and they seem like a very capable flight school.

The next day, I took a flight with AcuWings, and I can understand why there is so much support for them. Their office was a little hard to find at first (barely any signs on an already nondescript office building,) but they seemed well-run. Their office was off-base, so we had a very short drive across the road to the actual airplanes. They will be getting a building on the field in the near future, so that will be convenient. I followed the owner to the field who, despite his precision flying and deep knowledge of flight rules, failed to use his signal light even once.

For that intro flight, we flew their Cirrus SR20 (since I wanted to try a completely different airplane.) Different it was – an ’07 model, tons of electronics, tons of power, and no traditional yoke! You control the throttle with your right hand, and control the direction with a joystick-like device on your left. The weather was beautiful: in the mid-60s with unlimited ceiling, and 10 miles of visibility. This made for a much busier airport, which was also good to experience.

My daughter joined us on this ride -- an experience I was thrilled to have been able to share with her.

The aircraft was a lot more responsive than the Cessna, so I initially pitched it up to like 1100 ft/min correcting to the proper 700 ft/min or so during takeoff. We climbed to 3500 feet, and were treated to an awesome view of Mount Rainier. With all of the interactive electronics and gadgets, I had to consciously stop myself from getting too absorbed in the pretty displays, and instead use the cockpit window for what it was intended for :) We did some more level turns, played with the autopilot a bit, and practiced throttle control.

I was amazed to learn that airplanes will “skip” in the atmosphere. If you let them go with wings level and drop the power, they will just settle on a new lower altitude that matches the new power.

After about half an hour in the air, we came back to the Renton base and ran a more typical approach. We flew parallel to the runway (which I was surprised to realize was 30ish degrees off of the North-South axis,) and made a short final from the South due to heavy traffic. The landing went well, probably because it was almost entirely under the control of my instructor :)

After landing, it became very clear to me that this hobby has much more in store for me.

A few years ago, we experimented with a "Live" version of Get-ChildItem that retrieved elegant textual context-based advertisements from the internet as you used PowerShell.

This has been very successful, but unfortunately, a small subset of our trial user population has experienced crashing behaviour due to these new shell modifications. We have been unable to reproduce the issue in our labs, so the users in question have sent us screen casts of the crash. Even still, we've been looking at the code so long that we still can't confirm the problem.

It would really help the PowerShell team if you could take a look and help us identify the issue.

I've posted the screen cast here: http://www.leeholmes.com/projects/powershell_crash/. Make sure you have sound enabled, as that is supposedly an important factor.

[Download: http://www.leeholmes.com/projects/SyntaxHighlight/Start-Colourizer.zip]

In our most recent CTP, we exposed PowerShell's parsing and tokenizing API for the great benefit of ISVs and hobbyists alike. As an initial demonstration of it, Show-ColorizedContent let you display nicely formatted code listing in your console:

PS C:\Temp> .\Show-ColorizedContent.ps1 Show-ColorizedContent.ps1 `
>> -HighlightRanges (4..5+1+30..33)
>>

001 > #requires -version 2.0
002 |
003 | param(
004 >     $filename = $(throw "Please specify a filename."),
005 >     $highlightRanges = @(),
006 |     [System.Management.Automation.SwitchParameter] $excludeLineNumbers)
007 |
008 | # [Enum]::GetValues($host.UI.RawUI.ForegroundColor.GetType()) | % { Write-Host -Fore $_ "$_" }
(...)
026 | $highlightColor = "Green"
027 | $highlightCharacter = ">"
028 |
029 | ## Read the text of the file, and parse it
030 > $file = (Resolve-Path $filename).Path
031 > $content = [IO.File]::ReadAllText($file)
032 > $parsed = [System.Management.Automation.PsParser]::Tokenize($content, [ref] $null) |
033 >     Sort StartLine,StartColumn
034 |
035 | function WriteFormattedLine($formatString, [int] $line)
036 | {
037 |     if($excludeLineNumbers) { return }
038 |
039 |     $hColor = "Gray"
040 |     $separator = "|"

While this is very cool, we can actually push the parsing and tokenizing API even further. How about real time?

The first time I got this working, I actually giggled a little with glee. We're all so used to instant gratification when it comes to syntax highlighting that we sometimes forget about it. Until you drop into an environment that doesn't support it -- such as Notepad.exe or the PowerShell console. But with enough chewing gum and rubber bands, we can at least bring it into the PowerShell console.

As an aside, the V2 CTP also includes an early alpha version of PowerShell's new graphical host, Graphical PowerShell. Graphical PowerShell handles syntax highlighting swimmingly, and then some. If you want even more usability improvements (such as window resizing, tabs, and direct Unicode support,) that's where it'll be.

The primary problem with creating a real time colourizer is that PowerShell doesn't generate any events or notifications when you press a key. This makes cooperative multithreading difficult, so instead we need to take a parasitic approach and dig around in the console's screen buffer.

PowerShell doesn't support script threading either, though, so the step toward real-time syntax highlighting in your console is a slightly modified version of Jim Truher's "Background Jobs and PowerShell" script.

For each job that you create, Jim's script creates a new System.Management.Automation.Runspaces.Runspace instance, and then executes the job in the pipeline. Since the runspace class offers an asynchronous (non-blocking) invoke method, this effectively lets us run PowerShell code in the background of our current process. As written, though, Jim's background job support doesn't offer access to the variables and information of the runspace that launched the job. While normally useful, it does limit the usefulness of jobs for parasitic groveling! So, our main change in the New-Job script is to add a variable to the context of new jobs: $executionContextProxy. Through this variable, child jobs can access the variables and other information of the parent that spawned them. This update is included in the attached zip file.

Once we can launch jobs in the background that have access to our environment, the next step is the actual colourizer. This script runs as the background job. It starts at the bottom of the console buffer (where your cursor is,) and scans upward (and left) until it finds the last line of your prompt. It takes that rectangle, uses the parser API to determine how to colourize it, and then swiftly replaces that section of your console screen buffer with the colourized version.

I say "quickly" here to mean "as quickly as possible." There is a slight chance that the background job will swap in the colourized buffer after you've typed some more. It takes some pretty drastic measures to avoid this, but the occasional visual glitch will happen. This is only visual, however. Once you press ENTER, PowerShell will still execute the command you typed.

Many prompts use the Write-Host to add colourized prompts to the console, and return only a single space in the prompt() function. Since the colourizer can't know exactly how your prompt will appear on the screen, you need to tell it what the last line looks like. In your prompt function, set a global variable (called "promptMarker") to have the same text as the last line of your prompt. For example, if your prompt displays:

PS:15 >

on the screen, then set $GLOBAL:promptMarker = "PS:15 >"

Now, with New-Job.ps1 and Start-Colourizer.ps1 in your path (and a prompt that updates the promptMarker variable,) start the colourizer as a job:

Now, PowerShell updates your console with context-sensitive syntax highlighting as you type.

If you find the default console colours to be ugly, the included Set-ConsoleColourTable.ps1 script tones them down to something much easier on the eyes.

[Download: http://www.leeholmes.com/projects/SyntaxHighlight/Start-Colourizer.zip]

Enjoy!

When O’Reilly put up the overview page for the Windows PowerShell Cookbook, the sample chapter was originally “Looping and Flow Control.” While a useful chapter in its own right, it didn’t really highlight the solution-oriented focus the book takes.

They’ve now updated the sample chapter to one much more representative: Environmental Awareness.

The goal of this chapter is to help you work with the global environment (environment variables and common system paths,) and script’s environment (its location, name, and invocation info.)

It covers:

14. Environmental Awareness
       14.0 Introduction  
       14.1 View and Modify Environment Variables  
       14.2 Access Information About Your Command's Invocation  
       14.3 Program: Investigate the InvocationInfo Variable  
       14.4 Find Your Script's Name  
       14.5 Find Your Script's Location  
       14.6 Find the Location of Common System Paths  
       14.7 Program: Search the Windows Start Menu  
       14.8 Get the Current Location  
       14.9 Safely Build File Paths Out of Their Components  
       14.10 Interact with PowerShell's Global Environment 

You can download it by following the sample chapter link at the book’s overview page.

A question recently came up asking why the -eq operator doesn't work for two different (but equal) SecureStrings.

By default, almost all environments (the .NET Framework included) test if two things are exactly the same – that they are stored in the same place in memory. This is called reference equality.

[D:\documents\WindowsPowerShell]

PS:14 > $test = Read-Host -AsSecureString

****

[D:\documents\WindowsPowerShell]

PS:15 > [Object]::ReferenceEquals($test, $test)

True

[D:\documents\WindowsPowerShell]

PS:16 > $test -eq $test

True

That’s not usually what people want, so each individual type of object is (optionally) responsible for supporting a value-based equality test. They do this by implementing interfaces (software contracts) called IComparable and / or IEquatable:

[D:\documents\WindowsPowerShell]

PS:17 > $string1 = "Test"

[D:\documents\WindowsPowerShell]

PS:18 > $string2 = "Test"

[D:\documents\WindowsPowerShell]

PS:19 > [Object]::ReferenceEquals($string1, $string2)

False

[D:\documents\WindowsPowerShell]

PS:20 > [String].GetInterfaces()

IsPublic IsSerial Name

-------- -------- ----

True     False    IComparable

True     False    ICloneable

True     False    IConvertible

True     False    IComparable`1

True     False    IEnumerable`1

True     False    IEnumerable

True     False    IEquatable`1

[D:\documents\WindowsPowerShell]

PS:21 > $string1 -eq $string2

True

The implementors of the SecureString class have chosen to not implement these contracts, so PowerShell supports only reference equality on these types:

[D:\documents\WindowsPowerShell]

PS:30 > [System.Security.SecureString].GetInterfaces()

IsPublic IsSerial Name

-------- -------- ----

True     False    IDisposable

[D:\documents\WindowsPowerShell]

PS:31 > $test1 = Read-Host -AsSecureString

****

[D:\documents\WindowsPowerShell]

PS:32 > $test2 = Read-Host -AsSecureString

****

[D:\documents\WindowsPowerShell]

PS:33 > $test1 -eq $test2

False

I'm not particularly a drum fan, but this is an amazing feat of skill and technology.

Since I injured my middle left finger playing Guitar Hero 2 way too much when it came out, I've been dying to get back to playing it. ... so I thought "wouldn't it be great to be able to play Guitar Hero on the drums?" So I thought about how that might be accomplished... researched, implemented, borrowed, and here I outline the finished product.

Even just listening to the drum kit during the song sounds great. For more information, Egyokeo has also been kind enough to outline the full build.

The ACoupleOfAdmins blog recently posted a book review of the PowerShell Cookbook (and were kind enough to also write an Amazon Review.) They bring up some excellent points. Mainly,

The Windows PowerShell Cookbook will stay on my shelf as a reference book (for the code samples), but I would look to other resources first (e.g. Windows PowerShell In Action by Bruce Payette), if you need a resource to help learn PowerShell.

I thought long and hard about the depth and breadth of the book. One theme they're picking up on is that the PowerShell Cookbook is not a language focused book, and does not go into gritty detail about each language feature. This is intentional, as I wanted the book to have a very clear and unique value. We already have one PowerShell in Action, so there's really not a need for another.

One thing that's missed by the review is the implicit False Dichotomy -- that you should only have one book on PowerShell. The PowerShell Cookbook is intended to be a reference book (for its code samples and pre-packaged solutions,) while PowerShell in Action is intended to be a guided PowerShell tutorial. There is very little overlap between Bruce's book and the PowerShell Cookbook, and both provide significant value.

Since both Bruce and I wrote books with the intention to benefit the PowerShell community, it would not be in the best interest of anybody to have competing books!

A question came up on the newsgroup recently about why Exchange changes PowerShell’s execution policy from “Restricted” to “RemoteSigned.” Doesn’t that lower PowerShell’s security?

The "Restricted" execution policy isn't intended to be something that PowerShell users live with forever. It's a safe default that protects non PowerShell users from being impacted by PowerShell-based malware.

For example, many home users had never used VBScript, but still got bitten by the flurry of WSH-based viruses that got mailed to them. PowerShell's Restricted execution policy solves this. To an attacker, a computer that has never used PowerShell is the same as a computer that doesn't have PowerShell installed at all. And since attackers care look to affect the largest number of computers possible, PowerShell becomes a less attractive vehicle for their attack. That, in turn, makes PowerShell users safer too. Given all of PowerShell’s security barbs, an attacker is better off using executables or other popular scripting languages as an attack vehicle.

So in light of all this, the Exchange team updates the execution policy to RemoteSigned by our recommendation.

If you are a vendor that ships PowerShell scripts as an integral part of your offering (not just cmdlets,) you should absolutely expect that your users will want to run them. It would be a poor user experience if they had to install your product, and then change their execution policy when they try to use it. There’s also the very real threat of them just trying to make the message go away and picking a needlessly lax execution policy (such as Unrestricted.)

In this situation, we recommend that:

• You sign your scripts with a real Authenticode code signing certificate
• During installation, you change the execution policy to AllSigned if it is currently Restricted. If it is anything but Restricted, leave it alone.
• If your installer supports silent installation, offer an option to not modify the execution policy.

To clarify the “leave it alone” point – a non-default execution policy has been put there for a reason. While you might want to increase security to change the execution policy to AllSigned, it is more likely that you will inadvertently break other installed products, or parts of their IT infrastructure that depends on unsigned scripts.

If you do not have a code signing certificate, we recommend getting one! If you cannot, then your installer should prompt the user to change the execution policy to RemoteSigned if it is currently Restricted. If it is anything but Restricted, leave it alone.

“<This product> includes scripts that help you <manage Active Directory, etc.> Your current PowerShell script execution policy will prevent these scripts from running. Would you like to update the execution policy to allow these scripts to run?” ([Yes – Default] / No / Cancel)

Now what about Exchange? Surely they can afford a code signing certificate!

From their design and user research, they know that a vast majority of their user base will write their own scripts, or run scripts from the community. When they do that, they will ultimately get our execution policy warning, and be forced to make a decision. The most secure decision they can make is the RemoteSigned execution policy, so the Exchange installer makes that decision on their behalf.