Precision Computing

A question came up recently in the newsgroup, asking why adding seemingly different items to an ArrayList resulted in the ArrayList being full of the same item:

PS >$table = new-object Collections.ArrayList
PS >$row = 1 | select Col1,Col2,Col3
PS >$row.Col1 = "Column 1"
PS >$row.Col2 = "Column 2"
PS >[void] $table.Add($row)
PS >$row.Col1 = "Column 1, again"
PS >$row.Col2 = "Column 2, again"
PS >[void] $table.Add($row)
PS >$table

Col1                       Col2                       Col3
----                       ----                       ----
Column 1, again            Column 2, again
Column 1, again            Column 2, again

The source of the difference comes from the type of data that you assign to a variable.  Data falls into two categories: value types, and reference types.  Some data falls in the "value type" category, where its value is set in stone.  Changing the value actually creates a new one, as does passing it around:

PS > $a = "Foo"
PS > $b = $a
PS > $a += " Test"
PS > $a
Foo Test
PS > $b
Foo

Think of this like a promise you give to somebody.  If that person gives the same promise to somebody else, it's a new promise.  If you change your promise, it's a new promise yet again. 
 
Value types are the .Net primitives: string, int, bool, double, float, etc.
 
The other type category is called a "reference type."  Think of this like a bag you give to somebody.  If that person gives the bag to somebody else, it's still the same bag.  If you put stuff in the bag, take stuff out of the bag, or modify the bag -- it's still the same bag.
 
Most types are reference types.  Examples include ArrayList, StringBuilder, and some of the PowerShell primitives (ie: the Hashtable) -- since they are not .Net primitives.
 
The expression, "$row = 1 | select Col1, Col2, Col3" gives $row a PsObject -- which is a reference type.  When you change values in $row, you change it for everything that is talking about $row – including the copies inside the ArrayList.

In this case, the solution is to create new rows each time:

PS >$table = new-object Collections.ArrayList
PS >$row = 1 | select Col1,Col2,Col3
PS >$row.Col1 = "Column 1"
PS >$row.Col2 = "Column 2"
PS >[void] $table.Add($row)
PS >$row = 1 | select Col1,Col2,Col3
PS >$row.Col1 = "Column 1, again"
PS >$row.Col2 = "Column 2, again"
PS >[void] $table.Add($row)
PS >$table

Col1                       Col2                       Col3
----                       ----                       ----
Column 1                   Column 2
Column 1, again            Column 2, again

One of my favourite internal blogs has long been Terry Zink’s blog on spam analysis and fighting – from the perspective of somebody working on the Microsoft Exchange Hosted Services team.

After some prodding by me (and probably others,) his blog is now public!

And to make things even better, he’s Canadian :)

Something extremely helpful in learning a technology such as PowerShell is a pocket guide and quick reference.  It doesn’t necessarily replace a structured tutorial as a learning resource, but fills an important role as a daily desktop reference.

Over the last while, I’ve been working with O’Reilly to create such a guide – as the most ambitious addition to their “Short Cuts” program to date.

The reference will be available in the very near future, so stay tuned!

Jim Truher just posted a script that he's been polishing for the last few months -- Background "jobs" and PowerShell.  It's awesome, and one of the first things that people ask for once they start settling in to PowerShell.

What I love about being surrounded by a bunch of other PowerShell geeks is watching scripts like this come together.  If I remember correctly, this script started off as a response to a question on our internal PowerShell distribution list.  Marcel showed some examples of how to work with Runspaces through the scripting language, Jim added a bunch to it, and it quickly snowballed from there.

Even cooler, though, is watching the same thing happen outside of Redmond.

Probably the most useful network tool on any operating system is Telnet.  Not for connecting to Telnet servers, of course, as the Telnet protocol is about as insecure as they come.  Instead, it’s useful for debugging connection problems with arbitrary ports and arbitrary protocols.

Debugging an HTTP problem?  You can Telnet into port 80 to help you resolve it.
Debugging a mail retrieval issue?  You can Telnet into port 110 to help you resolve it.
Debugging a mail sending issue?  You can Telnet into port 25 to help you resolve it.
<etc>

Unfortunately, this workhorse was removed from Vista’s default installation.  Here’s a simple PowerShell replacement script.  It’s great for debugging, but useless (of course) for terminal emulation:

## Connect-Computer.ps1
## Interact with a service on a remote TCP port
param(
    [string] $remoteHost = "localhost",
    [int] $port = 80
     )

## Open the socket, and connect to the computer on the specified port
write-host "Connecting to $remoteHost on port $port"
$socket = new-object System.Net.Sockets.TcpClient($remoteHost, $port)
if($socket -eq $null) { return; }

write-host "Connected.  Press ^D followed by [ENTER] to exit.`n"

$stream = $socket.GetStream()
$writer = new-object System.IO.StreamWriter($stream)

$buffer = new-object System.Byte[] 1024
$encoding = new-object System.Text.AsciiEncoding

while($true)
{
   ## Allow data to buffer for a bit
   start-sleep -m 500

   ## Read all the data available from the stream, writing it to the
   ## screen when done.
   while($stream.DataAvailable) 
   { 
      $read = $stream.Read($buffer, 0, 1024)   
      write-host -n ($encoding.GetString($buffer, 0, $read)) 
   }

   ## Read the user's command, quitting if they hit ^D
   $command = read-host
   if($command -eq ([char] 4)) { break; }
 
   ## Write their command to the remote host     
   $writer.WriteLine($command)
   $writer.Flush()
}

## Close the streams
$writer.Close()
$stream.Close()

I’ve always loved Scott’s ultimate list of tools.  To make things even better, PowerShell now plays a big part in this year’s list :)

To add to the list, here are the first tools (not covered by Scott) I add to my systems:

Start | Control Panel | Regional Settings | … | Dvorak

Setting my system to the Dvorak keyboard layout is one of the first things I do on my non-work computers.  Typing feels much more efficient and effortless as compared to the QWERTY keyboard layout.

MakeMeAdmin

Aaron Margosis’ essential tool that lets you easily work with your system as a limited user.  Much more efficient than XP’s Fast User Switching.

Vim / Emacs

The venerable editors.  I seem to switch between them every few years – I’m currently using Vim because Emacs gives me grief over Unicode files.  Of course, syntax highlighting for PowerShell scripts is a must:
 VIM: http://www.vim.org/scripts/script.php?script_id=1327
 Emacs: http://www.viveksharma.com/techlog/2006/08/05/updated-powershell-script-editing-mode-for-emacs-powershell-modeel/

KeePass

Strong “Password Safe” program that also has a Pocket PC port.  Allows me to use strong passwords on all of my accounts, without having to remember them all.

Clipomatic

A helpful utility that caches the last ten entries in your clipboard.  I’m not gaga about this one, but it seems to do the best job of the free ones I’ve found.  Unfortunately, this app suffers (as most clipboard monitor software does) from Windows’ Remote Desktop messing around with the clipboard.

Tortoise Subversion

Checking all of your important documents into source control is a backup dream come true.  Especially when your source control server doubles as a cat feeder :)

FileZilla

The best free FTP program I’ve found.  SmartFTP used to fit that bill until it started nagging.

RealAlternative / QuicktimeAlternative  / K-Lite Codec Pack

A triumvirate that makes your media viewing life so much easier.  Provides alternatives to the evil RealPlayer and Quicktime bloatware, and also gives you the codecs required to play most popular file formats.

Windiff / ExamDiff

For file comparison tools, I use Windiff the most because people at work are used to it.  However, ExamDiff really is a slick application.

VncViewer

Nice little app to let you connect to a remote VNC session.

Putty

Although I prefer TerraTerm as an SSH client in general, Putty won its place in my tools directory because it requires only a single file.

AutoIt

A cool Windows UI automation program.  I especially like its COM interface, which can be easily scripted via PowerShell.

PerfCompare

A little benchmarking application to help you compare the efficiency of C# code snippets.

Packetyzer

A great network sniffer and protocol analyzer.

Items already in Scott’s list that deserve more attention:

Fiddler

A great HTTP inspector that just continues to get better.  I lived in this program when I worked for MSN Encarta, and heavily lean on it when playing with online hacking games

AutoHotKey

This program is much more than “AutoCorrect for Windows.”  It has powerful scripting abilities, and lets you automate tons of tasks in Windows.

Paint.Net

An awesome image editor that definitely replaces the open source “GIMP”

Video: DIY Automatic Cat Feeder [2.3 mb]

I have two wonderful cats, and many wonderful neighbours.  Naturally, for my vacation, I combined the best of both worlds and had one of the neighbours watching my cats. 

Despite this, I didn’t want to force my neighbours to come over twice a day to feed the cats.  So I sought out to make an automated cat feeder and water dispenser out of parts lying around the house.  As for the cat litter?  That’s fine – that can be left to stupid humans  :)

The Automatic Cat Feeder

Naturally, I looked around the internet first for ideas and plans.  I knew that the ‘Maker’ attitude could do better than the expensive pre-made contraptions you can find in stores.  I stumbled on the nifty hack in Make Magazine that repurposed an old VCR, but it portions out soft cat food – and requires some pretty complex mechanical pieces.

At first, I tried playing around with some little stepper motors and gears I had laying around in my electronics toolbox.  After applying some electricity, I realized that the motors had nowhere near enough power to move anything significant.  So I went down to my computer junk box in the garage in search of inspiration.

As I dug around this box, I found an old CD Rom drive and power supply.  The thought struck me that I could use the ejecting tray of the CD Rom as a solenoid to push the trigger mechanism of some sort of physical contraption.  But then I had a bootstrapping problem – what can I use to push the eject button of the CD Rom on schedule?

After some more thought, I realized that I could just use my spare (working) computer as the basis of the cat feeder.  It’s also my home’s Subversion source control server – a rare mix of server workloads indeed!  It has a CD Rom drive, so I could just use software to open and close it. 

The computer runs Ubuntu Linux, so a crontab entry controls the scheduling.  The script calls eject /mnt/cdrom to open the CD Rom, delays, and eject –t /mnt/cdrom to close it again.  It actually does this twice, as I found it made the portions more consistent.

Since it’s a full fledged Ubuntu system on my wireless network, it allowed for an obscene show of technology.  I was able to use my JasJar PDA phone to SSH into the box, and feed the cats on demand just by running the script.

With that as the basis, the only thing left was to design the physical contraption to actually feed the cats.  My first thought was some sort of rotating mechanism at the bottom of a container to dispense portions from pie-shaped wedges.  However, my primary concern (aside from killing the cats if the idea fails) was that the oddly shaped food pieces might jam the mechanism. 

From there, my second idea was to fill a box with cat food, and have a little trap door at the bottom.  And that’s the plan I went with.  An elastic holds the trap door closed, and the CD Rom tray pushes the door open.  When the CD Rom closes, the elastic shuts the trap door again.  Binding is no problem, as the food just binds in the trap door.

Finally, a cardboard chute directs all of the food into the bowl, and a big cardboard box encloses everything to prevent the cats from getting curious.

The Automatic Water Dispenser

This idea is extremely simple, and is based on the same physical concept that governs most other automatic water dispensers.  Punch a pencil-sized hole about 1 inch from the bottom of a large jug.  Fill with water, seal, then place in a dish greater than one inch high.

Water flows out of the jug as long as the water level is below the hole at the bottom.  When water flows out, the air pressure in jug decreases until it sucks in some air to equalize.  When the water level covers the hole, though, the air pressure can no longer equalize, so the water flow stops.

When the cats drink the water level down a bit, the jug can once again equalize its air pressure, and lets more water out.

Both contraptions worked extremely well for over a month (including testing time and vacation time.)  I’ve now taken apart the cat feeder, though, as it’s not the most aesthetic addition to my kitchen.

[Edit: Some kind soul put the video on YouTube for me.]
[Edit2: More detailed build instructions and meta-blogging here.]

Although the New-Object cmdlet is powerful, it doesn’t yet handle creating generic types very elegantly.  For a simple parameterized type, you can use the syntax that the .Net framework uses under the hood:

New-Object "System.Collections.ObjectModel.Collection``1[System.Int32]"

However, that begins to fall apart if you want to use types defined outside of the mscorlib DLL, or want to create complex generic types (for example, ones that refer to other generic types.)

The following script, New-GenericObject, creates objects of generic types.

## New-GenericObject.ps1
## Creates an object of a generic type:
##
## Usage:
##
##   # Simple generic collection
##   $list = New-GenericObject System.Collections.ObjectModel.Collection System.Int32
##
##   # Generic dictionary with two types
##   New-GenericObject System.Collections.Generic.Dictionary System.String,System.Int32
##
##   # Generic list as the second type to a generic dictionary
##   $secondType = New-GenericObject System.Collections.Generic.List Int32
##   New-GenericObject System.Collections.Generic.Dictionary System.String,$secondType.GetType()
##
##   # Generic type with a non-default constructor
##   New-GenericObject System.Collections.Generic.LinkedListNode System.Int32 10
##

param(
    [string] $typeName = $(throw "Please specify a generic type name"),
    [string[]] $typeParameters = $(throw "Please specify the type parameters"),
    [object[]] $constructorParameters
    )

## Create the generic type name
$genericTypeName = $typeName + '`' + $typeParameters.Count
$genericType = [Type] $genericTypeName

if(-not $genericType)
{
    throw "Could not find generic type $genericTypeName"
}

## Bind the type arguments to it
[type[]] $typedParameters = $typeParameters
$closedType = $genericType.MakeGenericType($typedParameters)
if(-not $closedType)
{
    throw "Could not make closed type $genericType"
}

## Create the closed version of the generic type
,[Activator]::CreateInstance($closedType, $constructorParameters)

[Edit: Bruce Payette pointed out that casting a string to a [Type] does all the hard work of the previous GetType function I wrote.  Also fixes a problem when you create generic types that PowerShell attempts to enumerate over when you return them.]

This has been coming up a bunch in the last little while, and I realized that I haven’t had a very good resource to point people to when they ask how to make a script or a function deal with pipeline input.

Scripts, functions, and script blocks all have access to the $input variable, which provides an enumerator over the elements in the incoming pipeline.  When pipelining is a core scenario, though,  these constructs also support the cmdlet-style statement blocks of begin, process, and end.  In those blocks, the $_ variable represents the current input object.  Along this line, a Filter (get-help about_Filter) is just a shorthand representation of a function whose body is composed entirely of a process block.

The following script gives an example of using the cmdlet-style keywords in a script.  It is an update to a script I wrote nearly a year ago: a PowerShell Hex Formatter.

## format-hex.ps1
## Convert a byte array into a hexidecimal dump
##
## Example usage:
## get-content 'c:\windows\Coffee Bean.bmp' -encoding byte | format-hex | more

## Convert the input to an array of bytes.  This is a strongly-typed variable,
## so that we're not trying to iterate over strings, directory entries, etc.
## [byte[]] $bytes = $(foreach($byte in $input) { $byte })

begin 
{
    ## Store our header, and formatting information
    $counter = 0
    $header = "            0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F"
    $nextLine = "{0}   " -f  [Convert]::ToString($counter, 16).ToUpper().PadLeft(8, '0')
    $asciiEnd = ""

    ## Output the header
    "`r`n$header`r`n"
}

process
{
    ## Display each byte, in 2-digit hexidecimal, and add that to the left-hand
    ## side.
    $nextLine += "{0:X2} " -f $_

    ## If the character is printable, add its ascii representation to
    ## the right-hand side.  Otherwise, add a dot to the right hand side.
    if(($_ -ge 0x20) -and ($_ -le 0xFE))
    {
       $asciiEnd += [char] $_
    }
    else
    {
       $asciiEnd += "."
    }

    $counter++;

    ## If we've hit the end of a line, combine the right half with the left half,
    ## and start a new line.
    if(($counter % 16) -eq 0)
    {
       "$nextLine $asciiEnd"
       $nextLine = "{0}   " -f [Convert]::ToString($counter, 16).ToUpper().PadLeft(8, '0')
       $asciiEnd = "";
    }
}

end
{
    ## At the end of the file, we might not have had the chance to output the end
    ## of the line yet.  Only do this if we didn't exit on the 16-byte boundary,
    ## though.
    if(($counter % 16) -ne 0)
    {
       while(($counter % 16) -ne 0)
       {
          $nextLine += "   "
          $asciiEnd += " "
          $counter++;
       }
       "$nextLine $asciiEnd"
    }

    ""
}

Keith Hill gives a good example of this technique as well in his Format-Xml function.

After visiting 5 countries (Italy, Croatia, Greece, France, Spain,) 2 micro-states (Monaco, Vatican City,) and returning with a connection in Sweden, I’m back – and now 367% as worldly!

The vacation was truly eye-opening.  My only previous international experience (aside from growing up in Canada, and moving here to the United States) was a low-budget resort trip in the Dominican Republic.  That trip gave me a filtered taste of a different culture, but had nowhere near the impact of a Mediterranean cruise around Europe.

By far, the biggest impact came from experiencing the rich cultural fabric formed by thousands of years of history.  The cruise was very port-heavy (12 ports in 14 days,) so I finally had the chance to experience:

• the canals of Venice
• the immense fortifications of Dubrovnik
• the hospitality of Corfu
• the grottos of Capri and Amalfi
• the ruins of Pompeii
• the opulence of the Vatican
• the ancient architectural masterpieces of Rome
• the beaches of Corsica
• the duomo and leaning tower of Pisa
• the duomo of Florence
• the exclusivity (*cough* expensive designer shops *cough*) of Portofino
• the wealth of Monte Carlo
• the beautiful churches of Marseille
• the amazing architecture of Barcelona

To put North America’s 500-year history in perspective, I gazed on the steps of Barcelona’s Palau Reial Mayor – the same steps Christopher Columbus walked up to announce his discovery of America.  I stood in the Coliseum of Rome – where an exceptionally old participant in the project would have also have experienced the death of Christ.  I walked the streets of ancient Pompeii, where ruins built upon ruins vastly predate even that.

An aspect I didn’t anticipate was the awkwardness of trying to stumble around language barriers caused by me – and the rudeness of speaking with my travel companions in a language foreign to most others within earshot.

All in all, it was a great experience and I look forward to experiencing the next!