As one of the guest commentators, I was asked to write a script to monitor uptime on servers in the environment.

To make this as efficient as possible, the solution leveraged WMI jobs to process 32 computers at a time. What’s most amazing about Windows PowerShell jobs running 32 tasks in parallel is not that it makes things 32 times faster. It’s that it makes them even faster than that! The reason is a branch of computer science called “queueing theory.”

For the surprising and cool reasons why, see the blog post here: http://blogs.technet.com/b/heyscriptingguy/archive/2012/04/23/expert-commentary-2012-scripting-games-advanced-event-6.aspx.

Unfortunately, trying to get a full colour palette (13 colours in stock) means having so much plastic you’ll never get through it all. Makerbot tried offering rainbow colour packs before, but they are now no longer for sale. I suggested it again via email, but it sounds like they’re pretty slammed with making Replicators :) 

Here’s the idea:

Standard Variety Pack – $100

Silver, Dark Brown, Yellow, Green, Red, Magenta, Orange, Blue

You get 1/8th of a spool (~ 150m) of each – for 1kg in total.

Electric Pack – $100

Light Brown, Fluorescent Yellow, Nuclear Green, Fluorescent Red

You get /14 of a spool (~ 300m) of each – for 1kg in total.

In searching elsewhere online, the only alternative I’ve found is Faberdashery – where the same deal would cost about $250 USD. I’ve done some back-of-the-envelope calculating, and I think a group buy would be an effective way to get a rainbow colour pack at cost-effective prices.

If you’re interested, mail me: makerbot @ this domain.

I’ve read a bunch of great posts on Nate Lawson’s awesome security blog, and decided that I wanted to read it beginning to end.

If you are the owner of said WordPress blog, the solution is easy – use WordPress’ built-in Export feature. There are even handy services that will turn this into a PDF, eBook, or even printed book.

If you’re not the owner, things aren’t so easy:

1. Sit in front of the computer
2. Go to the oldest month in the archives menu that I hadn’t yet visited
3. Read that page
4. Click “Next Page” until those links stop appearing
5. Go back to the home page
6. (Repeat steps 2-5 until you’re done with the blog)

Oh, and if you intend to take a break at any point in time, add in a few “try to remember where you were, and find that blog post again” entries.

What I was really hoping for was:

1. Open a PDF on my Kindle, and read the entire thing in chronological order, letting the Kindle software keep track of where I am.

It turns out that the difference between reality and desire is about twelve lines of PowerShell!

PowerShell’s recent technology previews (and the Windows 8 consumer and developer preview) include the Invoke-WebRequest cmdlet.Think wget / curl, but with PowerShell’s traditional object-based awesome-sauce. For example:

PS C:\temp> Invoke-WebRequest http://www.leeholmes.com/blog |
>>     Foreach-Object Links |
>>     Where-Object InnerText -match "August" |
>>     Foreach-Object Href       

http://www.leeholmes.com/blog/2011/08/

http://www.leeholmes.com/blog/2010/08/

http://www.leeholmes.com/blog/2008/08/

http://www.leeholmes.com/blog/2007/08/

http://www.leeholmes.com/blog/2006/08/

http://www.leeholmes.com/blog/2005/08/

When you look at links to the monthly archives, they all follow the pattern:

//">//">http://www.example.com/url/<number><number><number><number>/<number><number>/

When you visit any of these pages, they have another link. The exact text depends on the blog itself – but it may be “Earlier Entries”, “Next Page”, or similar:

PS C:\temp> $page = Invoke-WebRequest http://www.leeholmes.com/blog/2005/06/
PS C:\temp> $page.Links | Where-Object InnerText -match "Earlier Entries" |
>>     Select-Object -First 1
>>                                                                                                                      

innerHTML : Earlier Entries ?
innerText : Earlier Entries ?
outerHTML : <A href="http://www.leeholmes.com/blog/2005/06/page/2/">Earlier Entries ?</A>
outerText : Earlier Entries ?
tagName   : A
href      : http://www.leeholmes.com/blog/2005/06/page/2/                                                               

Given that knowledge, we can automate the download of the entire blog, dumping it into an HTML file as we go. As a final step, we print this HTML to PDF, and upload it to our Kindle or other reading device.

Note to purists: this HTML file is brutally malformed. It is a collection of HTML pages packed into the same file, rather than one HTML page with all the important content. It is of course possible to make this a valid HTML file by manipulating the content before writing it – there’s just no need to do it if the destination is a PDF anyhow.

And how about time effort? In the end, I had a PDF of the entire blog on my Kindle 20 minutes after first having thought of it.

Here’s the PowerShell script that automates this all – cleaned up for your consumption, of course :)

## Things you might want to change
$blogUrl = "http:/www.leeholmes.com/blog"
$archiveLinkPattern = '/\d\d\d\d/\d\d/$'
$nextPageText = "Earlier Entries"            

## Get the page
$r = Invoke-WebRequest $blogUrl            

## Extract the archives links
$links = $r.Links | Where-Object href -match $archiveLinkPattern |
    Foreach-Object href            

## Sort the archives in reverse order
$links = $links[$links.Count..0]            

## Go through each archive page
foreach($link in $links)
{
    ## Create a variable to hold the HTML content for this month
    $monthExport = ""            

    do
    {
        ## Get the archives for that month
        $month = Invoke-WebRequest $link            

        ## Get the page content, and put it at the beginning of the
        ## monthExport variable. That's because "Earlier Entries"
        ## should be placed before the content we just got.
        $monthExport = $month.Content + "`r`n" + $monthExport            

        ## Find the link to "Earlier Entires"
        $link = $month.Links | ? innertext -match $nextPageText |
            Foreach-Object href | Select-Object -First 1            

    ## Keep on doing this while we found an "Earlier Entries" link
    } while($link)            

    ## Now that we're done with the month, put it at the end of the
    ## HTML file (since we're processing months in order)
    $monthExport >> leeholmes.html
}

clear;$00=(0..1250|%{9608}),(0..7645|%{9617})|%{$_};(-10..29)|
%{$OO='';$O0=$_;-10..64|%{$0O=$_;$OO+=[char]($00[$0O*$0O-48*$0O+
1720+4*$O0*$O0-96*$O0],@($00[$0O*$0O-52*$0O+1644+4*$O0*$O0-88*
$O0],9617,9617)[(0,1)[($0O-lt28)]+($O0-gt12)])[(0,1)[$0O-gt24]*
($O0-lt14)]};$OO};0..573892165|%{$OO=@((($OO+0)*4*$_*$_/(4*$_*
$_-1)),1d)[$_-lt1];write-progress ":-)"($OO*2)}

I suppose that’s not really fair. Here it is in all of its syntax-highlighted glory:

clear;$00=(0..1250|%{9608}),(0..7645|%{9617})|%{$_};(-10..29)|
%{$OO='';$O0=$_;-10..64|%{$0O=$_;$OO+=[char]($00[$0O*$0O-48*$0O+
1720+4*$O0*$O0-96*$O0],@($00[$0O*$0O-52*$0O+1644+4*$O0*$O0-88*
$O0],9617,9617)[(0,1)[($0O-lt28)]+($O0-gt12)])[(0,1)[$0O-gt24]*
($O0-lt14)]};$OO};0..573892165|%{$OO=@((($OO+0)*4*$_*$_/(4*$_*
$_-1)),1d)[$_-lt1];write-progress ":-)"($OO*2)}

My favourite is Richard Siddaway’s summary, freshly updated today: http://richardspowershellblog.wordpress.com/2012/03/11/powershell-booksmarch-2012/. The ecosystem of PowerShell books has really blossomed – I love that there are two books on managing VMWare on this list!

Jonathan Medd also has a good list here: http://www.jonathanmedd.net/2011/01/recommended-powershell-books.html.

A resource I am hopeful about is Don Jones’ recently launched http://www.powershellbooks.com. Its goal is to “help you select the Windows PowerShell book or books that best fit your current learning and reference needs.” Several of Don’s books are in every “PowerShell must have” list. His “Learn PowerShell in a Month of Lunches” book for beginners is tearing up the charts. Right now, the review site only covers books by Manning Press (plus an additional book that Don wrote), so it’s not an objective survey. I hope he expands his scope to make it one. It also doesn’t actually review the books (it links to product pages), but I’m still crossing my fingers because Don’s touch has a way of turning things into gold :)

The books that tend to get left off these lists are the domain-specific ones: PowerShell + SQL, PowerShell + Exchange, PowerShell + SharePoint. Those ecosystems are becoming robust enough to support a survey of “PowerShell SQL” books, for example, although I’m not aware of any that have done so.

If you know of any good book comparison lists, please let me know!

After thinking about what to do with them, I hereby announce the PowerShell Pay-it-Forward Award.

If you would like a free signed copy of any of these books, I have a simple request. Go forth on the internet: Stack Overflow, blogs, Twitter, whatever. Find a PowerShell newbie with a gnarly problem. Not just a syntax problem, but some hairy business problem. Work with them to solve it.

Post your results back here in the comments (with a link to the interaction,) and I’ll give books to those that have helped a user (or users) the most.

I’ll announce the winners on December 1st.

Good luck!

There are many different ways to calculate unique identifiers for items. These different ways are called Hash Algorithms. Each serves a different purpose, but some popular ones right now are are MD5, SHA1, and SHA256.

Why so many?

The reason is security. Our last post finished with the question:

Using hash codes for simple content verification is one thing. But what if an attacker is trying to trick you with hash code collisions? For example, can you use that Visual Studio download if it’s from your worst enemy – but the hash code still matches? It depends on the algorithm!

The security industry first started touching on the importance of “secure” hash algorithms in the 1970s. There were a flurry of proposals in the 80s that were immediately dismissed with security problems, but one called “MD5” was introduced in the early 90s that seemed like it might last the test of time.

In 2004, however, advances in security research found issues in MD5 – thus “breaking MD5”. While you can still use MD5 just fine when security is not a concern, you cannot trust a version of Visual Studio from your enemy just because the MD5 hash value matches.

Here are the four features of an ideal cryptographic hash algorithm:

• it is easy (but not necessarily quick) to compute the hash value for any given message
• it is infeasible to generate a message that has a given hash
• it is infeasible to modify a message without changing the hash
• it is infeasible to find two different messages with the same hash

If you get #2, then for the most part you get #3 and #4 automatically.

How does one “break” a hash algorithm? Let’s take a look a a few from the .NET Framework for an interesting exercise in reverse engineering.

As a background, every object in the .NET Framework supports a method called “GetHashCode()”. It does a good job of returning a small random(ish) identifier for an object, so it is used primarily for putting objects into hash tables. This method returns a 32-bit integer, and is not designed for security purposes:

publicvirtualint GetHashCode()

The HashAlgorithm class is designed for security purposes, however, and should be used when security is a concern. We won’t use that class as an example, because if I can break that class in this blog, then it wouldn’t be in the .NET Framework!

Breaking Int32

Let’s see how this works on integers, and see how it would fare as a cryptographic hash algorithm:

PS > (1).GetHashCode()
1
PS > (2).GetHashCode()
2
PS > (1337).GetHashCode()
1337
PS > (-1337).GetHashCode()
-1337

I think we see a pattern :) It looks like .NET is returning our input right back at us. It’s a sure way to prevent collisions, that’s for sure! If we open Int32.GetHashCode() in .NET Reflector (or ILSpy), we can see why:

public override int GetHashCode()

{

    return this;

}

The score sheet?

☑	it is easy to compute the hash value for any given message
☒	it is infeasible to generate a message that has a given hash
☑	it is infeasible to modify a message without changing the hash
☑	it is infeasible to find two different messages with the same hash

Not bad, but you can’t do much with just integers.

Breaking Int64

That wasn’t too challenging to break. What about Int64? That doesn’t fit in 32 bits by definition, so let’s see if there is any pattern:

PS > ([Int64] 1).GetHashCode()
1
PS > ([Int64] 1337).GetHashCode()
1337
PS > ([Int64] 12345678987654321).GetHashCode()
1656367333

For “low” numbers (below 32 bits?) it seems to be the same as the input number. After that, not quite sure. We can’t break this one by intuition. Let’s see the source:

public override int GetHashCode()

{

    return (int)this ^ (int)(this >> 32);

}

What the .NET Framework is doing here is pretty simple:

(int)this	Takes the lower 32-bits of the 64-bit number.
(int)(this >>32)	Shifts the number to the right by 32 places, and then takes the lower 32 bits of the number. That is the same thing as taking the upper 32 bits of the number.
(int)this ^(int)(this >>32)	Uses the binary XOR operator to combine the upper 32 bits and the lower 32 bits.

The first row explains why all of our “low” numbers seemed to just return themselves. The low numbers were the lower 32 bits, and there were no upper 32 bits.

Let’s take a look at:

PS > ([Int64] 12345678987654321).GetHashCode()
1656367333

If we convert this to the upper and lower 32 bits, we get:

PS > [Convert]::ToString(12345678987654321, 2)
101011110111000101010001100010100100011111010010110001
PS > [Convert]::ToString(12345678987654321, 2).PadLeft(64, "0")
0000000000101011110111000101010001100010100100011111010010110001

Now, split them into the top and bottom 32 bits take the binary exclusive OR (either bit is 1, but not both):

Upper 32	00000000001010111101110001010100
Lower 32	01100010100100011111010010110001
XOR	01100010101110100010100011100101

Taking a look at the final result, it looks like we have the same as what .NET gave us:

PS > [Convert]::ToInt32("01100010101110100010100011100101", 2)
1656367333

PS > ([Int64] 12345678987654321).GetHashCode()
1656367333

Now that we understand the algorithm, let’s see how we might break the strongest guarantee of a hash algorithm – namely, “it is infeasible to generate a message that has a given hash.”

Let’s pretend we want to generate a 64-bit number with the hash value of 1337. That has the following binary value:

PS > [Convert]::ToString(1337, 2).PadLeft(32, "0")
00000000000000000000010100111001

Since the 64-bit hash is generated by combining the upper and lower halves of the number, the simplest way is to simply ignore the upper 32 bits:

Upper 32	00000000000000000000000000000000
Lower 32	00000000000000000000010100111001
XOR	00000000000000000000010100111001

That was simple enough. What about breaking the next strongest feature: “it is infeasible to find two different messages with the same hash.”

Given that we know how to find an Int64 for any given hash value, how can we find multiple Int64s that have the same hash value?

To figure this out, we’ll use a trick from the XOR operation frequently used in XOR cyphers and XOR encryption:

If A XOR B = C

Then A = B XOR C

And B = A XOR C

If we pretend that A is the upper 32 bits of a number, B is the lower 32 bits of a number, and C is the hash code, then we have the following options:

Upper 32 bits = (Lower 32 bits) XOR (Hash code)

or

Lower 32 bits = (Upper 32 bits) XOR (Hash code)

Given that, here’s all we have to do to find a number with a specific hash code:

• Pick an arbitrary number for the lower (or upper) 32 bits
• XOR it with the hash code
• The result will be the other 32 bits you need

Let’s look at an example, using 10001011001010110101101011010111 as a randomly-chosen number:

Upper 32	10001011001010110101101011010111
Hash Code	00000000000000000000010100111001
XOR (= Lower 32)	?

PowerShell has the “-bxor” operator, so let’s make this easier on ourselves:

PS > $upper32 = [Convert]::ToInt32("10001011001010110101101011010111", 2)
PS > $hash = [Convert]::ToInt32("00000000000000000000010100111001", 2)
PS > $lower32 = $upper32 -bxor $hash
PS > [Convert]::ToString($lower32, 2)
10001011001010110101111111101110

So now we have a hash collision – let’s be sure:

PS > $number = [Convert]::ToInt64(
>>     "10001011001010110101101011010111" +
>>     "10001011001010110101111111101110", 2)
>>
PS > $number
-8418535196639666194

Let’s see:

PS > ([Int64] -8418535196639666194).GetHashCode()
1337

Bingo. We can now generate hash collisions for arbitrary hash codes. What does that mean?

☑	it is easy to compute the hash value for any given message
☒	it is infeasible to generate a message that has a given hash
☒	it is infeasible to modify a message without changing the hash
☒	it is infeasible to find two different messages with the same hash

Bonus question: how many hash collisions can you generate for a given hash code? And how might you do so systematically?

Chew on that a bit – in our next post, we’ll take a crack at reverse-engineering String.GetHashCode().

You play a real guitar.

You learn the real notes.

Your fingers hurt just like every great guitarist before you.

For some context, guitar has been part of my life for a long time – although my proficiency with it has waxed and waned (currently waning). I made the most progress with classical guitar, but ran into a cliff of frustration when songs started calling for notes out of the basic first few positions. A couple of editions of the PowerShell Cookbook will put a dent in your free time, so that may have something to do with it, too :)

This kind of “Guitar Hero but with real guitar” game has been a pipe dream for a long time. A few years ago, the Music Wizard folks started shopping around a new game called “Guitar Wizard”. After hearing about it, I promptly signed up for their mailing list – but they’ve now removed all of the product pages from their site and I suspect have abandoned the project entirely.

The primary reason behind this lack of advancement in the field has long been technical. Most software that tries to convert raw guitar sounds into something that a computer can understand ends up failing pretty fast. The note recognition ends up being glitchy, inaccurate, and frustrating. Things have heated up recently, with applications like JamGuru (PC) and Rock Prodigy (iPad) that help you learn the tablature versions of songs through sophisticated note recognition.

Rock Band 3 introduced “Pro Mode” for guitar (which has been getting good reviews), but there have been mixed feelings about the way the guitar interface is presented. Also, they took another approach to resolving the technical issues by making you use their specific guitar. That second point really hurts if you have a well-loved (or unloved!) guitar already.

While computer scientists are out flogging and publishing minor advances in audio recognition, Rocksmith and this new breed of apps have somehow just quietly and magically figured it out, and shipped it. The note recognition truly does feel magical, and never crosses your mind after that.

After getting a brief intro on how to use the system, Rocksmith starts you off with a simple lead part to “(I Can’t Get No) Satisfaction” by the Rolling Stones.

It’s really you

At even its simplest levels, you truly feel like you’re playing the song. This is where Rocksmith stands out as a music game. In rhythm games like Rock Band, you use the guitar controller (or real guitar) to hit the right notes at the right time. As long as you do that correctly, the game continues to replace you with the pre-recorded guitar track of the song. If you make a mistake, it replaces you with a squelchy sound of a guitar mistake. What kind of mistake, you might ask? Were you off by a fret? Too late? Too early? Who knows!

Rocksmith, on the other hand, plays the actual sounds from your guitar. They send it through a virtual amplifier and effects chain so that you have the same tone as the guitar from the original song, but you’ll hear it immediately if you’re off by a fret. The note will sound too high or low (just as it does in real guitar), so you can easily correct by ear alone.

Don’t be afraid to experiment

If you’ve played other rhythm games like Rock Band, you’ve probably looked down the “Note Highway” with dread as 2 minutes of the same boring pattern approach.

Rocksmith solves this problem wonderfully. Want to noodle around for the beginning of the song? Add some trills, double-plucks or experimental harmonies? Go ahead! Rocksmith doesn’t penalize you for these – as long as you still at least hit the notes that it’s displayed.

Speaking of noodling around, Rocksmith has the best load screen I’ve ever seen. It activates the virtual amp whenever it’s “Loading…”, so you can take that time to warm up, play around, or experiment with new ideas.

As I mentioned earlier, Rocksmith creates your in-game sound by sending your guitar through a virtual amplifier and effects chain. Once you start completing songs, Rocksmith makes these components available to you in a free-form playground called “Amp Mode”. You can pick pre-set sounds from any of the songs you earned them from, or mix and match components (pedals, amps, etc.) to make something completely different. Spend a minute, spend an hour, either way you’ll have fun.

You learn actual guitar

After playing the songs for awhile, you’ll amaze yourself by picking up the guitar when it’s not even plugged in. You’ll play the same thing you’ve been playing all along, but then realize the sounds of the song are coming from your guitar, not your video game. They don’t include Stairway to Heaven in the song list, so you’ll be safe playing these at the local guitar shop, too!

After you’ve been playing songs for a while, Rocksmith opens up a handful of guitar trainers / challenge games. These games let you practice specific techniques (chords, scales, neck position, bends) that improve your actual guitar skill, as well as in-game performance.

So, far I’ve only unlocked two challenge games: Ducks (which teaches you how to find fret positions quickly) and Dawn of the Chordead (which teaches you to play and recall chords). Dawn of the Chordead unquestionably teaches you transferrable guitar skills. Being able to quickly find and play a chord (and then smoothly transition to the next one) is crucial for playing many songs. Ducks makes you a much better game player, but offers limited benefit for real guitar (see My One Big Complaint).

What’s not to like?

While I’m a big fan of the game, there are some rough edges. First off, you need to use your regular game controller to navigate through the game. When you’ve got a guitar strapped around your neck and the wireless controllers are going to sleep, this is just annoying. Why you can’t just use the guitar to control the entire game is beyond me – Rocksmith has already proven that it can accurately recognize input from a real guitar!

Second is practice mode. When a song starts to prove challenging, they offer a neat set of trainers that let you just practice through a difficult section. These trainers let you gradually increase difficulty or speed for a specific section. It would be nice if there was a more holistic practice mode that let you master a song section-by-section by increasing difficulty and then speed for each section. What’s most annoying about the practice mode is that you’ve got five “lives”. If you can’t play it fast enough or accurately enough in five attempts, then it kicks you back to the practice screen. From there, you have to click through all the menus, loading screen, tuning process, etc. It’s just a lot of process for something that should be the most forgiving in the entire game.

My One Big Complaint

My One Big Complaint stems from wanting the skills from the game to have more benefit to my real guitar playing.

As I mentioned before, guitar mastery is heavily influenced by how well you know the fretboard. To show you where to put your fingers when you play, Rocksmith sends fret numbers down the Note Highway:

This is very similar to Guitar tablature, and is a great way to learn a song. The alternative is the traditional music staff, which even professional musicians have difficulty sight-reading and would be a deal breaker for the game.

However, becoming a lightning-fast tablature reader hits a dead-end at some point. It’s like becoming a god at watercolour paint-by-numbers. Just as you need to learn to draw and choose pigment at some point while becoming an artist, a guitarist will eventually need to know what notes they are playing (C,D,E, etc.) On this point, Rocksmith misses a great opportunity. If they simply offered an option to replace the coloured blocks with coloured letters (i.e.: yellow on the 5th fret = D), they’d unlock a vast world of true musicianship to millions of fledgling guitarists. Rock Prodigy offers this in their interface, and it help immensely:

I bought it, and would buy it again

When you add up all the pros and cons, there isn’t really a question in my mind. If my cat ate the game tonight, I would buy it again tomorrow.

[Updated 10/04/2011 – V2 includes support for audio input]

If you’re interested in Morse Code as a hobby or just as a neat lo-fi communication mechanism, one thing that might set you back is the lack of practical opportunities to use it.

You can find lots of Morse Code apps that let you practice into a little text box, but nothing that you can tweet with, blog with chat with, etc.

Morser is a simple little program that solves this need.

After launching it, press Control-Alt-M to enable or disable it. It works no matter what application you have running, so you can minimize it and switch to an application that you want to use it with. When enabled, Morser acts as a straight keyer. It listens for presses of your keyboard’s down arrow and automatically types the letter that corresponds to your Morse code input.

If you hook up any audio device (i.e.: a real actual keyer) to an input jack, Morser converts the audio input to text as well.

If you have funny characters to enter, just type them on your keyboard as you normally would.

The Control-Alt-M hotkey also works in any application, so you don’t need to switch back to Morser to enable or disable it.

Here’s an example of it interpreting somebody’s YouTube video. I hooked my audio output to my Mic Input.

Enjoy!

His first wallpaper was a screaming crowd. In my head, they are people chanting for the power of PowerShell, or the IT rock star that saved the deployment. It’s been my background since he posted it, and has graced many a conference presentation.

He’s just created another two with a Superman theme. It makes me think of a bunch of out-of-date
systems out there needing automation – so the PowerShell IT admin superhero kicks into action :)

Check out his post on the PowerShell Magazine site for the full resolution downloads!