PowerShell Cookbook

Search

Categories

 

On this page

Realtime Syntax Highlighting in your PowerShell Console
PowerShell Cookbook Sample Chapter: Environmental Awareness
PowerShell's -EQ Operator: Reference Equality vs Value Equality
Guitar Hero 3 ... for Drums
PowerShell Cookbook vs PowerShell in Action
3rd Parties and PowerShell Execution Policies
Laughing babies are contagious
Syntax Highlighting in PowerShell
Will it Pipe? Brevity and Readability
PowerShell Cookbook Now Available

Archive

Blogroll

Disclaimer
I work for Microsoft.

The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.

RSS 2.0 | Atom 1.0 | CDF

Send mail to the author(s) E-mail

Total Posts: 220
This Year: 20
This Month: 0
This Week: 0
Comments: 533

Sign In

 Tuesday, February 12, 2008
Tuesday, February 12, 2008 8:26:56 AM (Pacific Standard Time, UTC-08:00) ( )

[Download: http://www.leeholmes.com/projects/SyntaxHighlight/Start-Colourizer.zip]

In our most recent CTP, we exposed PowerShell's parsing and tokenizing API for the great benefit of ISVs and hobbyists alike. As an initial demonstration of it, Show-ColorizedContent let you display nicely formatted code listing in your console:


PS C:\Temp> .\Show-ColorizedContent.ps1 Show-ColorizedContent.ps1 ` >> -HighlightRanges (4..5+1+30..33) >> 001 > #requires -version 2.0 002 | 003 | param( 004 > $filename = $(throw "Please specify a filename."), 005 > $highlightRanges = @(), 006 | [System.Management.Automation.SwitchParameter] $excludeLineNumbers) 007 | 008 | # [Enum]::GetValues($host.UI.RawUI.ForegroundColor.GetType()) | % { Write-Host -Fore $_ "$_" } (...)
026 | $highlightColor = "Green" 027 | $highlightCharacter = ">" 028 | 029 | ## Read the text of the file, and parse it 030 > $file = (Resolve-Path $filename).Path 031 > $content = [IO.File]::ReadAllText($file) 032 > $parsed = [System.Management.Automation.PsParser]::Tokenize($content, [ref] $null) | 033 > Sort StartLine,StartColumn 034 | 035 | function WriteFormattedLine($formatString, [int] $line) 036 | { 037 | if($excludeLineNumbers) { return } 038 | 039 | $hColor = "Gray" 040 | $separator = "|"

While this is very cool, we can actually push the parsing and tokenizing API even further. How about real time?

The first time I got this working, I actually giggled a little with glee. We're all so used to instant gratification when it comes to syntax highlighting that we sometimes forget about it. Until you drop into an environment that doesn't support it -- such as Notepad.exe or the PowerShell console. But with enough chewing gum and rubber bands, we can at least bring it into the PowerShell console.

As an aside, the V2 CTP also includes an early alpha version of PowerShell's new graphical host, Graphical PowerShell. Graphical PowerShell handles syntax highlighting swimmingly, and then some. If you want even more usability improvements (such as window resizing, tabs, and direct Unicode support,) that's where it'll be.

The primary problem with creating a real time colourizer is that PowerShell doesn't generate any events or notifications when you press a key. This makes cooperative multithreading difficult, so instead we need to take a parasitic approach and dig around in the console's screen buffer.

PowerShell doesn't support script threading either, though, so the step toward real-time syntax highlighting in your console is a slightly modified version of Jim Truher's "Background Jobs and PowerShell" script.

For each job that you create, Jim's script creates a new System.Management.Automation.Runspaces.Runspace instance, and then executes the job in the pipeline. Since the runspace class offers an asynchronous (non-blocking) invoke method, this effectively lets us run PowerShell code in the background of our current process. As written, though, Jim's background job support doesn't offer access to the variables and information of the runspace that launched the job. While normally useful, it does limit the usefulness of jobs for parasitic groveling! So, our main change in the New-Job script is to add a variable to the context of new jobs: $executionContextProxy. Through this variable, child jobs can access the variables and other information of the parent that spawned them. This update is included in the attached zip file.

Once we can launch jobs in the background that have access to our environment, the next step is the actual colourizer. This script runs as the background job. It starts at the bottom of the console buffer (where your cursor is,) and scans upward (and left) until it finds the last line of your prompt. It takes that rectangle, uses the parser API to determine how to colourize it, and then swiftly replaces that section of your console screen buffer with the colourized version.

I say "quickly" here to mean "as quickly as possible." There is a slight chance that the background job will swap in the colourized buffer after you've typed some more. It takes some pretty drastic measures to avoid this, but the occasional visual glitch will happen. This is only visual, however. Once you press ENTER, PowerShell will still execute the command you typed.

Many prompts use the Write-Host to add colourized prompts to the console, and return only a single space in the prompt() function. Since the colourizer can't know exactly how your prompt will appear on the screen, you need to tell it what the last line looks like. In your prompt function, set a global variable (called "promptMarker") to have the same text as the last line of your prompt. For example, if your prompt displays:

PS:15 >

on the screen, then set $GLOBAL:promptMarker = "PS:15 >"

Now, with New-Job.ps1 and Start-Colourizer.ps1 in your path (and a prompt that updates the promptMarker variable,) start the colourizer as a job:

image

Now, PowerShell updates your console with context-sensitive syntax highlighting as you type.

If you find the default console colours to be ugly, the included Set-ConsoleColourTable.ps1 script tones them down to something much easier on the eyes.

[Download: http://www.leeholmes.com/projects/SyntaxHighlight/Start-Colourizer.zip]

Enjoy!

Comments [11] | | # 
 Monday, December 17, 2007
Monday, December 17, 2007 7:05:55 PM (Pacific Standard Time, UTC-08:00) ( )

When O’Reilly put up the overview page for the Windows PowerShell Cookbook, the sample chapter was originally “Looping and Flow Control.” While a useful chapter in its own right, it didn’t really highlight the solution-oriented focus the book takes.

They’ve now updated the sample chapter to one much more representative: Environmental Awareness.

The goal of this chapter is to help you work with the global environment (environment variables and common system paths,) and script’s environment (its location, name, and invocation info.)

It covers:

14. Environmental Awareness
       14.0 Introduction  
       14.1 View and Modify Environment Variables  
       14.2 Access Information About Your Command's Invocation  
       14.3 Program: Investigate the InvocationInfo Variable  
       14.4 Find Your Script's Name  
       14.5 Find Your Script's Location  
       14.6 Find the Location of Common System Paths  
       14.7 Program: Search the Windows Start Menu  
       14.8 Get the Current Location  
       14.9 Safely Build File Paths Out of Their Components  
       14.10 Interact with PowerShell's Global Environment 

You can download it by following the sample chapter link at the book’s overview page.

Comments [1] | | # 
 Wednesday, December 05, 2007
Wednesday, December 05, 2007 5:43:41 PM (Pacific Standard Time, UTC-08:00) ( )

A question recently came up asking why the -eq operator doesn't work for two different (but equal) SecureStrings.

 

By default, almost all environments (the .NET Framework included) test if two things are exactly the same – that they are stored in the same place in memory. This is called reference equality.

 

[D:\documents\WindowsPowerShell]

PS:14 > $test = Read-Host -AsSecureString

****

 

[D:\documents\WindowsPowerShell]

PS:15 > [Object]::ReferenceEquals($test, $test)

True

 

[D:\documents\WindowsPowerShell]

PS:16 > $test -eq $test

True

 

That’s not usually what people want, so each individual type of object is (optionally) responsible for supporting a value-based equality test. They do this by implementing interfaces (software contracts) called IComparable and / or IEquatable:

 

[D:\documents\WindowsPowerShell]

PS:17 > $string1 = "Test"

 

[D:\documents\WindowsPowerShell]

PS:18 > $string2 = "Test"

 

[D:\documents\WindowsPowerShell]

PS:19 > [Object]::ReferenceEquals($string1, $string2)

False

 

[D:\documents\WindowsPowerShell]

PS:20 > [String].GetInterfaces()

 

IsPublic IsSerial Name

-------- -------- ----

True     False    IComparable

True     False    ICloneable

True     False    IConvertible

True     False    IComparable`1

True     False    IEnumerable`1

True     False    IEnumerable

True     False    IEquatable`1

 

[D:\documents\WindowsPowerShell]

PS:21 > $string1 -eq $string2

True

 

 

The implementors of the SecureString class have chosen to not implement these contracts, so PowerShell supports only reference equality on these types:

 

[D:\documents\WindowsPowerShell]

PS:30 > [System.Security.SecureString].GetInterfaces()

 

IsPublic IsSerial Name

-------- -------- ----

True     False    IDisposable

 

[D:\documents\WindowsPowerShell]

PS:31 > $test1 = Read-Host -AsSecureString

****

 

[D:\documents\WindowsPowerShell]

PS:32 > $test2 = Read-Host -AsSecureString

****

 

[D:\documents\WindowsPowerShell]

PS:33 > $test1 -eq $test2

False

 

Comments [0] | | # 
 Tuesday, November 27, 2007
Wednesday, November 28, 2007 4:35:28 AM (Pacific Standard Time, UTC-08:00) ( )

I'm not particularly a drum fan, but this is an amazing feat of skill and technology.

Since I injured my middle left finger playing Guitar Hero 2 way too much when it came out, I've been dying to get back to playing it. ... so I thought "wouldn't it be great to be able to play Guitar Hero on the drums?" So I thought about how that might be accomplished... researched, implemented, borrowed, and here I outline the finished product.

Even just listening to the drum kit during the song sounds great. For more information, Egyokeo has also been kind enough to outline the full build.

Comments [1] | | # 
 Tuesday, November 20, 2007
Wednesday, November 21, 2007 3:50:47 AM (Pacific Standard Time, UTC-08:00) ( )

The ACoupleOfAdmins blog recently posted a book review of the PowerShell Cookbook (and were kind enough to also write an Amazon Review.) They bring up some excellent points. Mainly,

The Windows PowerShell Cookbook will stay on my shelf as a reference book (for the code samples), but I would look to other resources first (e.g. Windows PowerShell In Action by Bruce Payette), if you need a resource to help learn PowerShell.

I thought long and hard about the depth and breadth of the book. One theme they're picking up on is that the PowerShell Cookbook is not a language focused book, and does not go into gritty detail about each language feature. This is intentional, as I wanted the book to have a very clear and unique value. We already have one PowerShell in Action, so there's really not a need for another.

One thing that's missed by the review is the implicit False Dichotomy -- that you should only have one book on PowerShell. The PowerShell Cookbook is intended to be a reference book (for its code samples and pre-packaged solutions,) while PowerShell in Action is intended to be a guided PowerShell tutorial. There is very little overlap between Bruce's book and the PowerShell Cookbook, and both provide significant value.

Since both Bruce and I wrote books with the intention to benefit the PowerShell community, it would not be in the best interest of anybody to have competing books!

Comments [4] | | # 
 Monday, November 19, 2007
Tuesday, November 20, 2007 1:37:04 AM (Pacific Standard Time, UTC-08:00) ( )

A question came up on the newsgroup recently about why Exchange changes PowerShell’s execution policy from “Restricted” to “RemoteSigned.” Doesn’t that lower PowerShell’s security?

The "Restricted" execution policy isn't intended to be something that PowerShell users live with forever. It's a safe default that protects non PowerShell users from being impacted by PowerShell-based malware.

For example, many home users had never used VBScript, but still got bitten by the flurry of WSH-based viruses that got mailed to them. PowerShell's Restricted execution policy solves this. To an attacker, a computer that has never used PowerShell is the same as a computer that doesn't have PowerShell installed at all. And since attackers care look to affect the largest number of computers possible, PowerShell becomes a less attractive vehicle for their attack. That, in turn, makes PowerShell users safer too. Given all of PowerShell’s security barbs, an attacker is better off using executables or other popular scripting languages as an attack vehicle.

So in light of all this, the Exchange team updates the execution policy to RemoteSigned by our recommendation.

If you are a vendor that ships PowerShell scripts as an integral part of your offering (not just cmdlets,) you should absolutely expect that your users will want to run them. It would be a poor user experience if they had to install your product, and then change their execution policy when they try to use it. There’s also the very real threat of them just trying to make the message go away and picking a needlessly lax execution policy (such as Unrestricted.)

In this situation, we recommend that:

  • You sign your scripts with a real Authenticode code signing certificate
  • During installation, you change the execution policy to AllSigned if it is currently Restricted. If it is anything but Restricted, leave it alone.
  • If your installer supports silent installation, offer an option to not modify the execution policy.

To clarify the “leave it alone” point – a non-default execution policy has been put there for a reason. While you might want to increase security to change the execution policy to AllSigned, it is more likely that you will inadvertently break other installed products, or parts of their IT infrastructure that depends on unsigned scripts.

If you do not have a code signing certificate, we recommend getting one! If you cannot, then your installer should prompt the user to change the execution policy to RemoteSigned if it is currently Restricted. If it is anything but Restricted, leave it alone.

“<This product> includes scripts that help you <manage Active Directory, etc.> Your current PowerShell script execution policy will prevent these scripts from running. Would you like to update the execution policy to allow these scripts to run?” ([Yes – Default] / No / Cancel)

Now what about Exchange? Surely they can afford a code signing certificate!

From their design and user research, they know that a vast majority of their user base will write their own scripts, or run scripts from the community. When they do that, they will ultimately get our execution policy warning, and be forced to make a decision. The most secure decision they can make is the RemoteSigned execution policy, so the Exchange installer makes that decision on their behalf.

 

Comments [0] | | # 
 Thursday, November 08, 2007
Friday, November 09, 2007 12:16:07 AM (Pacific Standard Time, UTC-08:00) ( )

Here's a nice light-hearted video if you need a laugh :)


Video: William laughing

 

Comments [0] | | # 
 Tuesday, November 06, 2007
Wednesday, November 07, 2007 6:22:07 AM (Pacific Standard Time, UTC-08:00) ( )

Since we just released a CTP of PowerShell V2, I thought I'd share a handy little script to demonstrate one of the new APIs we introduced: Show-ColorizedContent.ps1.

This CTP introduces a new tokenizer API that lets you work with PowerShell script content the same way that our parser does -- as a collection of items (tokens) that represent the underlying structure of that script Until now, any tool that works with the data of a PowerShell script needs to parse the script on its own -- usually with fragile regular expressions or other means.

This often works, but usually falls apart on complex scripts:

image

In the first line, "Write-Host" is an argument to the Write-Host cmdlet, but gets parsed as a string. Fair enough, but the second line does not treat the argument the same way. In fact, since it matches a cmdlet name, the argument gets parsed as another cmdlet call. In the here string that follows, the Write-Host cmdlet name gets highlighted again, even though it is really just part of a string.

This is absolutely not a slam on the authors of existing highlighters -- it's just that we've now introduced something that makes life so much easier.

$content = [IO.File]::ReadAllText("c:\temp\ContentTest.ps1")
$errors = [System.Management.Automation.PSParseError[]] @()
[System.Management.Automation.PsParser]::Tokenize($content, [ref] $errors)

This API generates a collection of PSToken objects that give all the information you need to properly dissect a PowerShell script:

PS C:\Temp> [System.Management.Automation.PsParser]::Tokenize($content, [ref] $errors) | ft -auto

Content                           Type Start Length StartLine StartColumn EndLine EndColumn
-------                           ---- ----- ------ --------- ----------- ------- ---------
Write-Host                     Command     0     10         1           1       1        11
Write-Host                      String    11     12         1          12       1        24
...                            NewLine    23      2         1          24       2         1
Write-Host                     Command    25     10         2           1       2        11
Write-Host             CommandArgument    36     10         2          12       2        22
...                            NewLine    46      2         2          22       3         1
...                            NewLine    48      2         3           1       4         1
Write-Host Write-Host           String    50     23         4           1       4        24
...                            NewLine    73      2         4          24       5         1
...                            NewLine    75      2         5           1       6         1
testContent                   Variable    77     12         6           1       6        13
=                             Operator    90      1         6          14       6        15
Write-Host Hello World          String    92     30         6          16       8         3
...                            NewLine   122      2         8           3       9         1

This adds a whole new dimension to the way you can interact with PowerShell. Some natural outcomes are:

  • syntax highlighting
  • preparing a script for production (replacing all aliased commands with their expanded equivalent, etc)
  • script refactoring
  • FxCop / Style guideline checks
  • PowerTab :)

As a starter example, I've attached Show-ColorizedContent.ps1 -- a script to colorize PowerShell scripts in a console window. Its primary goal is to support demonstrations of PowerShell snippets. For that, it adds line numbers to let you easily refer to portions of your script. It also includes a -HighlightRanges parameter to let you highlight specific ranges of the script. The -HighlightRanges parameter is an array of line numbers, which you can easily create using PowerShell's standard array range syntax:


PS C:\Temp> .\Show-ColorizedContent.ps1 Show-ColorizedContent.ps1 ` >> -HighlightRanges (4..5+1+30..33) >> 001 > #requires -version 2.0 002 | 003 | param( 004 > $filename = $(throw "Please specify a filename."), 005 > $highlightRanges = @(), 006 | [System.Management.Automation.SwitchParameter] $excludeLineNumbers) 007 | 008 | # [Enum]::GetValues($host.UI.RawUI.ForegroundColor.GetType()) | % { Write-Host -Fore $_ "$_" } (...)
026 | $highlightColor = "Green" 027 | $highlightCharacter = ">" 028 | 029 | ## Read the text of the file, and parse it 030 > $file = (Resolve-Path $filename).Path 031 > $content = [IO.File]::ReadAllText($file) 032 > $parsed = [System.Management.Automation.PsParser]::Tokenize($content, [ref] $null) | 033 > Sort StartLine,StartColumn 034 | 035 | function WriteFormattedLine($formatString, [int] $line) 036 | { 037 | if($excludeLineNumbers) { return } 038 | 039 | $hColor = "Gray" 040 | $separator = "|"

Enjoy -- you can download it here.

Comments [5] | | # 
 Thursday, November 01, 2007
Friday, November 02, 2007 5:26:52 AM (Pacific Daylight Time, UTC-07:00) ( )

Scott Hanselman and I recently chatted about using PowerShell for a bit of log analysis. The majority of his solution (in green) ended up being quite elegant, flowing into a pipeline nearly as easily as you might speak it:

PS C:\> $re =[regex]"\d{2}(?=[_.])"; import-csv file.csv |
select File, Hits, @{Name="Show";Expression={$re.matches($_.File)[0] } } | sort Show -desc | group Show |
select Name,
{($_.Group | Measure-Object -Sum Hits).Sum }

(Of course, one should never pronounce a regex aloud in polite company.)

The bit in red took a little hammering on, but eventually produced the desired results. At this point, I mentioned that I didn't think the final solution was a good demonstration of PowerShell's pipeline power. 80% of it, absolutely. But since the last bit took some fussing, the solution stopped being an example of how easy the pipeline makes everything, and instead became an example of how you could write a pipeline to do anything.

If I was to blog it with the intent of education, I probably would have written a more scripty function:

function Get-ShowHits
{
    $regex = '/hanselminutes_(\d+).*'
    $shows = Import-CSv File.csv | Select File,Hits | Group { $_.File -replace $regex,'$1' }

    foreach($show in $shows)
    {
        $showOutput = New-Object System.Management.Automation.PsObject
        $showOutput | Add-Member NoteProperty Name $show.Name
        $showOutput | Add-Member NoteProperty Hits ($show.Group | Measure-Object -Sum Hits).Sum
        $showOutput
    }
}
Get-ShowHits | Sort -Desc Hits

This example illustrates a couple great points about PowerShell:

  • You can write functions and scripts to wrap complex functionality into a more usable form
  • You can write pipelines to easily express powerfull object flows (in the $shows line)
  • You can create your own objects with their own properties -- and manipulate them just as easily

But the most important point about these two examples is how easy they are to modify and extend.

Jon Udell keyed in on Scott's post, and in the comments of the two blogs, language comparisons quickly blossomed. Hey, we've been here before!

Specifically,

These types of problems are like mosquito bites for me - I can’t stop itching at them. A better ruby one-liner. Doesn’t print a header but not a big deal…

CSV.read("test.csv").inject(Hash.new(0)) {|h,row| h[row[0][/\d{4}/].to_i] += row[1].to_i;h}.sort.each {|i| puts("#{i[0]}\t#{i[1]}")}

Once you have a solution that works, a natural scripter's passion is to tinker it down to one line. It's no longer educational, intelligible, or extendable, but it's fun. You can do that in PowerShell, too:

$foo = @{}; ipcsv test.csv | % { $foo[0+($_.File -replace '.*?(\d+).*','$1')] += (0+$_.Hits) }; $foo.GetEnumerator() | Sort Value

Mmmm. Pipeline smoke.

Comments [0] | | # 
 Monday, October 29, 2007
Tuesday, October 30, 2007 12:02:20 AM (Pacific Daylight Time, UTC-07:00) ( )

As just announced by O’Reilly, the PowerShell Cookbook is now available!

As your experience grows in any technology, you learn and benefit from the combination of two distinct types of knowledge:

  1. What can I do with the technology?
  2. How do I accomplish a specific task in that technology?

Question #1 focuses on the technology. You learn the answers to #1 from training, exploration, podcasts, and technology-specific books. This is where you immerse yourself in the looping constructs, variable syntax, etc. In PowerShell’s case, that’s (not surprisingly) turned out to be PowerShell in Action, along with the many other excellent PowerShell learning resources out there.

Question #2 focuses on the task. You know what you want to do, and the only think in your way is technology. You learn the answers to #2 from experience, and lots of it. Experience is in short supply when you first start working with a technology, so blog posts and internet searches often tentatively fill that void. Until the O’Reilly cookbook arrives, that is, at which point you have pre-canned solutions to many of your most vexing questions. In PowerShell’s case, we finally have a resource to fill that void – the PowerShell Cookbook.

The PowerShell Cookbook focuses squarely on showing you how to use PowerShell to get your job done. It builds on a huge base of distilled knowledge, and includes:

  • Solutions to the most popular and searched-for TechNet / Script Center topics
  • Scripts that address the most common community, newsgroup, and new user questions
  • Scripts that wrap around and hide the complexity of advanced (but very useful) PowerShell scripting techniques
  • Task-based introduction to all of PowerShell’s major features

In addition to all of this pre-distilled knowledge, the cookbook’s appendix provides a very thorough and complete PowerShell reference. Thorough enough to be a book all of its own, but now expanded and finally in a printed format. It includes:

  • A complete reference to the PowerShell language and environment
  • An exhaustive regular expression reference, along with PowerShell examples
  • A complete listing of all PowerShell automatic variables
  • A guide to PowerShell’s standard verbs, which helps you write scripts that match the PowerShell naming guidelines
  • Hand-picked lists of the .NET classes, WMI classes, and COM objects most useful to system administrators
  • An exhaustive String and DateTime formatting reference, along with PowerShell examples

Since the book focuses so strongly on the same crowd that frequents Microsoft ScriptCenter, I tapped Dean Tsaltas (one of the original Scripting Guys) for feedback and a forward. Perhaps my greatest disappointment about the book is that we didn’t have room for his fantastic cover quote. I would be remiss to relegate his unbiased opinion to the Cavern-Of-Thoughts-Unpublished, so I include it here:

"Must-read of the season!  A stunning achievement! Tantalizing, witty, and entirely satisfying. Oh, wait -- that's the foreword."

Dean Tsaltas, Scripting Guy emeritus, author of the Foreword.

So, enjoy the book  — or at least the foreword :)

Comments [4] | | #