Precision Computing

Jim Truher just posted a script that he's been polishing for the last few months -- Background "jobs" and PowerShell.  It's awesome, and one of the first things that people ask for once they start settling in to PowerShell.

What I love about being surrounded by a bunch of other PowerShell geeks is watching scripts like this come together.  If I remember correctly, this script started off as a response to a question on our internal PowerShell distribution list.  Marcel showed some examples of how to work with Runspaces through the scripting language, Jim added a bunch to it, and it quickly snowballed from there.

Even cooler, though, is watching the same thing happen outside of Redmond.

Probably the most useful network tool on any operating system is Telnet.  Not for connecting to Telnet servers, of course, as the Telnet protocol is about as insecure as they come.  Instead, it’s useful for debugging connection problems with arbitrary ports and arbitrary protocols.

Debugging an HTTP problem?  You can Telnet into port 80 to help you resolve it.
Debugging a mail retrieval issue?  You can Telnet into port 110 to help you resolve it.
Debugging a mail sending issue?  You can Telnet into port 25 to help you resolve it.
<etc>

Unfortunately, this workhorse was removed from Vista’s default installation.  Here’s a simple PowerShell replacement script.  It’s great for debugging, but useless (of course) for terminal emulation:

## Connect-Computer.ps1
## Interact with a service on a remote TCP port
param(
    [string] $remoteHost = "localhost",
    [int] $port = 80
     )

## Open the socket, and connect to the computer on the specified port
write-host "Connecting to $remoteHost on port $port"
$socket = new-object System.Net.Sockets.TcpClient($remoteHost, $port)
if($socket -eq $null) { return; }

write-host "Connected.  Press ^D followed by [ENTER] to exit.`n"

$stream = $socket.GetStream()
$writer = new-object System.IO.StreamWriter($stream)

$buffer = new-object System.Byte[] 1024
$encoding = new-object System.Text.AsciiEncoding

while($true)
{
   ## Allow data to buffer for a bit
   start-sleep -m 500

   ## Read all the data available from the stream, writing it to the
   ## screen when done.
   while($stream.DataAvailable) 
   { 
      $read = $stream.Read($buffer, 0, 1024)   
      write-host -n ($encoding.GetString($buffer, 0, $read)) 
   }

   ## Read the user's command, quitting if they hit ^D
   $command = read-host
   if($command -eq ([char] 4)) { break; }
 
   ## Write their command to the remote host     
   $writer.WriteLine($command)
   $writer.Flush()
}

## Close the streams
$writer.Close()
$stream.Close()

I’ve always loved Scott’s ultimate list of tools.  To make things even better, PowerShell now plays a big part in this year’s list :)

To add to the list, here are the first tools (not covered by Scott) I add to my systems:

Start | Control Panel | Regional Settings | … | Dvorak

Setting my system to the Dvorak keyboard layout is one of the first things I do on my non-work computers.  Typing feels much more efficient and effortless as compared to the QWERTY keyboard layout.

MakeMeAdmin

Aaron Margosis’ essential tool that lets you easily work with your system as a limited user.  Much more efficient than XP’s Fast User Switching.

Vim / Emacs

The venerable editors.  I seem to switch between them every few years – I’m currently using Vim because Emacs gives me grief over Unicode files.  Of course, syntax highlighting for PowerShell scripts is a must:
 VIM: http://www.vim.org/scripts/script.php?script_id=1327
 Emacs: http://www.viveksharma.com/techlog/2006/08/05/updated-powershell-script-editing-mode-for-emacs-powershell-modeel/

KeePass

Strong “Password Safe” program that also has a Pocket PC port.  Allows me to use strong passwords on all of my accounts, without having to remember them all.

Clipomatic

A helpful utility that caches the last ten entries in your clipboard.  I’m not gaga about this one, but it seems to do the best job of the free ones I’ve found.  Unfortunately, this app suffers (as most clipboard monitor software does) from Windows’ Remote Desktop messing around with the clipboard.

Tortoise Subversion

Checking all of your important documents into source control is a backup dream come true.  Especially when your source control server doubles as a cat feeder :)

FileZilla

The best free FTP program I’ve found.  SmartFTP used to fit that bill until it started nagging.

RealAlternative / QuicktimeAlternative  / K-Lite Codec Pack

A triumvirate that makes your media viewing life so much easier.  Provides alternatives to the evil RealPlayer and Quicktime bloatware, and also gives you the codecs required to play most popular file formats.

Windiff / ExamDiff

For file comparison tools, I use Windiff the most because people at work are used to it.  However, ExamDiff really is a slick application.

VncViewer

Nice little app to let you connect to a remote VNC session.

Putty

Although I prefer TerraTerm as an SSH client in general, Putty won its place in my tools directory because it requires only a single file.

AutoIt

A cool Windows UI automation program.  I especially like its COM interface, which can be easily scripted via PowerShell.

PerfCompare

A little benchmarking application to help you compare the efficiency of C# code snippets.

Packetyzer

A great network sniffer and protocol analyzer.

Items already in Scott’s list that deserve more attention:

Fiddler

A great HTTP inspector that just continues to get better.  I lived in this program when I worked for MSN Encarta, and heavily lean on it when playing with online hacking games

AutoHotKey

This program is much more than “AutoCorrect for Windows.”  It has powerful scripting abilities, and lets you automate tons of tasks in Windows.

Paint.Net

An awesome image editor that definitely replaces the open source “GIMP”

Video: DIY Automatic Cat Feeder [2.3 mb]

I have two wonderful cats, and many wonderful neighbours.  Naturally, for my vacation, I combined the best of both worlds and had one of the neighbours watching my cats. 

Despite this, I didn’t want to force my neighbours to come over twice a day to feed the cats.  So I sought out to make an automated cat feeder and water dispenser out of parts lying around the house.  As for the cat litter?  That’s fine – that can be left to stupid humans  :)

The Automatic Cat Feeder

Naturally, I looked around the internet first for ideas and plans.  I knew that the ‘Maker’ attitude could do better than the expensive pre-made contraptions you can find in stores.  I stumbled on the nifty hack in Make Magazine that repurposed an old VCR, but it portions out soft cat food – and requires some pretty complex mechanical pieces.

At first, I tried playing around with some little stepper motors and gears I had laying around in my electronics toolbox.  After applying some electricity, I realized that the motors had nowhere near enough power to move anything significant.  So I went down to my computer junk box in the garage in search of inspiration.

As I dug around this box, I found an old CD Rom drive and power supply.  The thought struck me that I could use the ejecting tray of the CD Rom as a solenoid to push the trigger mechanism of some sort of physical contraption.  But then I had a bootstrapping problem – what can I use to push the eject button of the CD Rom on schedule?

After some more thought, I realized that I could just use my spare (working) computer as the basis of the cat feeder.  It’s also my home’s Subversion source control server – a rare mix of server workloads indeed!  It has a CD Rom drive, so I could just use software to open and close it. 

The computer runs Ubuntu Linux, so a crontab entry controls the scheduling.  The script calls eject /mnt/cdrom to open the CD Rom, delays, and eject –t /mnt/cdrom to close it again.  It actually does this twice, as I found it made the portions more consistent.

Since it’s a full fledged Ubuntu system on my wireless network, it allowed for an obscene show of technology.  I was able to use my JasJar PDA phone to SSH into the box, and feed the cats on demand just by running the script.

With that as the basis, the only thing left was to design the physical contraption to actually feed the cats.  My first thought was some sort of rotating mechanism at the bottom of a container to dispense portions from pie-shaped wedges.  However, my primary concern (aside from killing the cats if the idea fails) was that the oddly shaped food pieces might jam the mechanism. 

From there, my second idea was to fill a box with cat food, and have a little trap door at the bottom.  And that’s the plan I went with.  An elastic holds the trap door closed, and the CD Rom tray pushes the door open.  When the CD Rom closes, the elastic shuts the trap door again.  Binding is no problem, as the food just binds in the trap door.

Finally, a cardboard chute directs all of the food into the bowl, and a big cardboard box encloses everything to prevent the cats from getting curious.

The Automatic Water Dispenser

This idea is extremely simple, and is based on the same physical concept that governs most other automatic water dispensers.  Punch a pencil-sized hole about 1 inch from the bottom of a large jug.  Fill with water, seal, then place in a dish greater than one inch high.

Water flows out of the jug as long as the water level is below the hole at the bottom.  When water flows out, the air pressure in jug decreases until it sucks in some air to equalize.  When the water level covers the hole, though, the air pressure can no longer equalize, so the water flow stops.

When the cats drink the water level down a bit, the jug can once again equalize its air pressure, and lets more water out.

Both contraptions worked extremely well for over a month (including testing time and vacation time.)  I’ve now taken apart the cat feeder, though, as it’s not the most aesthetic addition to my kitchen.

[Edit: Some kind soul put the video on YouTube for me.]
[Edit2: More detailed build instructions and meta-blogging here.]

Although the New-Object cmdlet is powerful, it doesn’t yet handle creating generic types very elegantly.  For a simple parameterized type, you can use the syntax that the .Net framework uses under the hood:

New-Object "System.Collections.ObjectModel.Collection``1[System.Int32]"

However, that begins to fall apart if you want to use types defined outside of the mscorlib DLL, or want to create complex generic types (for example, ones that refer to other generic types.)

The following script, New-GenericObject, creates objects of generic types.

## New-GenericObject.ps1
## Creates an object of a generic type:
##
## Usage:
##
##   # Simple generic collection
##   $list = New-GenericObject System.Collections.ObjectModel.Collection System.Int32
##
##   # Generic dictionary with two types
##   New-GenericObject System.Collections.Generic.Dictionary System.String,System.Int32
##
##   # Generic list as the second type to a generic dictionary
##   $secondType = New-GenericObject System.Collections.Generic.List Int32
##   New-GenericObject System.Collections.Generic.Dictionary System.String,$secondType.GetType()
##
##   # Generic type with a non-default constructor
##   New-GenericObject System.Collections.Generic.LinkedListNode System.Int32 10
##

param(
    [string] $typeName = $(throw "Please specify a generic type name"),
    [string[]] $typeParameters = $(throw "Please specify the type parameters"),
    [object[]] $constructorParameters
    )

## Create the generic type name
$genericTypeName = $typeName + '`' + $typeParameters.Count
$genericType = [Type] $genericTypeName

if(-not $genericType)
{
    throw "Could not find generic type $genericTypeName"
}

## Bind the type arguments to it
[type[]] $typedParameters = $typeParameters
$closedType = $genericType.MakeGenericType($typedParameters)
if(-not $closedType)
{
    throw "Could not make closed type $genericType"
}

## Create the closed version of the generic type
,[Activator]::CreateInstance($closedType, $constructorParameters)

[Edit: Bruce Payette pointed out that casting a string to a [Type] does all the hard work of the previous GetType function I wrote.  Also fixes a problem when you create generic types that PowerShell attempts to enumerate over when you return them.]

This has been coming up a bunch in the last little while, and I realized that I haven’t had a very good resource to point people to when they ask how to make a script or a function deal with pipeline input.

Scripts, functions, and script blocks all have access to the $input variable, which provides an enumerator over the elements in the incoming pipeline.  When pipelining is a core scenario, though,  these constructs also support the cmdlet-style statement blocks of begin, process, and end.  In those blocks, the $_ variable represents the current input object.  Along this line, a Filter (get-help about_Filter) is just a shorthand representation of a function whose body is composed entirely of a process block.

The following script gives an example of using the cmdlet-style keywords in a script.  It is an update to a script I wrote nearly a year ago: a PowerShell Hex Formatter.

## format-hex.ps1
## Convert a byte array into a hexidecimal dump
##
## Example usage:
## get-content 'c:\windows\Coffee Bean.bmp' -encoding byte | format-hex | more

## Convert the input to an array of bytes.  This is a strongly-typed variable,
## so that we're not trying to iterate over strings, directory entries, etc.
## [byte[]] $bytes = $(foreach($byte in $input) { $byte })

begin 
{
    ## Store our header, and formatting information
    $counter = 0
    $header = "            0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F"
    $nextLine = "{0}   " -f  [Convert]::ToString($counter, 16).ToUpper().PadLeft(8, '0')
    $asciiEnd = ""

    ## Output the header
    "`r`n$header`r`n"
}

process
{
    ## Display each byte, in 2-digit hexidecimal, and add that to the left-hand
    ## side.
    $nextLine += "{0:X2} " -f $_

    ## If the character is printable, add its ascii representation to
    ## the right-hand side.  Otherwise, add a dot to the right hand side.
    if(($_ -ge 0x20) -and ($_ -le 0xFE))
    {
       $asciiEnd += [char] $_
    }
    else
    {
       $asciiEnd += "."
    }

    $counter++;

    ## If we've hit the end of a line, combine the right half with the left half,
    ## and start a new line.
    if(($counter % 16) -eq 0)
    {
       "$nextLine $asciiEnd"
       $nextLine = "{0}   " -f [Convert]::ToString($counter, 16).ToUpper().PadLeft(8, '0')
       $asciiEnd = "";
    }
}

end
{
    ## At the end of the file, we might not have had the chance to output the end
    ## of the line yet.  Only do this if we didn't exit on the 16-byte boundary,
    ## though.
    if(($counter % 16) -ne 0)
    {
       while(($counter % 16) -ne 0)
       {
          $nextLine += "   "
          $asciiEnd += " "
          $counter++;
       }
       "$nextLine $asciiEnd"
    }

    ""
}

Keith Hill gives a good example of this technique as well in his Format-Xml function.

After visiting 5 countries (Italy, Croatia, Greece, France, Spain,) 2 micro-states (Monaco, Vatican City,) and returning with a connection in Sweden, I’m back – and now 367% as worldly!

The vacation was truly eye-opening.  My only previous international experience (aside from growing up in Canada, and moving here to the United States) was a low-budget resort trip in the Dominican Republic.  That trip gave me a filtered taste of a different culture, but had nowhere near the impact of a Mediterranean cruise around Europe.

By far, the biggest impact came from experiencing the rich cultural fabric formed by thousands of years of history.  The cruise was very port-heavy (12 ports in 14 days,) so I finally had the chance to experience:

• the canals of Venice
• the immense fortifications of Dubrovnik
• the hospitality of Corfu
• the grottos of Capri and Amalfi
• the ruins of Pompeii
• the opulence of the Vatican
• the ancient architectural masterpieces of Rome
• the beaches of Corsica
• the duomo and leaning tower of Pisa
• the duomo of Florence
• the exclusivity (*cough* expensive designer shops *cough*) of Portofino
• the wealth of Monte Carlo
• the beautiful churches of Marseille
• the amazing architecture of Barcelona

To put North America’s 500-year history in perspective, I gazed on the steps of Barcelona’s Palau Reial Mayor – the same steps Christopher Columbus walked up to announce his discovery of America.  I stood in the Coliseum of Rome – where an exceptionally old participant in the project would have also have experienced the death of Christ.  I walked the streets of ancient Pompeii, where ruins built upon ruins vastly predate even that.

An aspect I didn’t anticipate was the awkwardness of trying to stumble around language barriers caused by me – and the rudeness of speaking with my travel companions in a language foreign to most others within earshot.

All in all, it was a great experience and I look forward to experiencing the next!

In my last post, I gave an example of how to use P/Invoke through PowerShell  to implement functionality not immediately available via the .Net framework.  My example was to get the owner of a process, and Abhishek wisely pointed out that  this is easier through WMI.

So here’s another example.  It’s actually the one I had written originally, but it didn’t give me an opportunity to illustrate [Ref] parameters, or [Out] parameters via P/Invoke.

param([int] $opacity)

$GWL_EXSTYLE = -20
$WS_EX_LAYERED = 0x80000
$LWA_ALPHA = 0x00000002

## Invoke a Win32 P/Invoke call.
function Invoke-Win32([string] $dllName, [Type] $returnType, 
   [string] $methodName, [Type[]] $parameterTypes, [Object[]] $parameters)
{
   (... from last post ...)
   # http://www.leeholmes.com/blog/GetTheOwnerOfAProcessInPowerShellPInvokeAndRefOutParameters.aspx
}

function GetWindowLong($hWnd, $style)
{
   $parameterTypes = [IntPtr], [int]
   $parameters = [IntPtr] $hWnd, [int] $style

   Invoke-Win32 "user32" ([Int]) "GetWindowLong" $parameterTypes $parameters
}

function SetWindowLong($hWnd, $style, $value)
{
   $parameterTypes = [IntPtr], [int], [int]
   $parameters = [IntPtr] $hWnd, [int] $style, [int] $value

   Invoke-Win32 "user32" ([Int]) "SetWindowLong" $parameterTypes $parameters
}


function GetForegroundWindow
{
   Invoke-Win32 "user32" ([Int]) "GetForegroundWindow"
}

function SetLayeredWindowAttributes([byte] $opacity)
{
   $window = GetForegroundWindow

   $oldStyle = GetWindowLong $window $GWL_EXSTYLE
   $result = SetWindowLong $window $GWL_EXSTYLE ($oldStyle -bor $WS_EX_LAYERED)
   if($result -gt 0)
   {
      $parameterTypes = [IntPtr], [int], [byte], [int]
      $parameters = [IntPtr] $window, [int] 0x00FFFFFF, [byte] $opacity, [int] $LWA_ALPHA

      [void] (Invoke-Win32 "user32" ([Int]) "SetLayeredWindowAttributes" $parameterTypes $parameters)
   }
}

if($opacity -gt 100)
{
   throw "Opacity must be between 0 and 100"
}

"Switch to the window that you want to adjust.  The change will happen in 2 seconds."
"(Note that console windows are not supported)"

sleep 2
SetLayeredWindowAttributes ($opacity * 255 / 100)

P.S.  I’m off to the Mediterranean until the middle of August.  Can you feed the cats while I’m gone?  :)

An often asked question in the Managed Code world is “How do I get the owner of a process?”  The same question is starting to come up around PowerShell, and the answer to both is:

The .Net Framework does not yet support this API, so the solution is to write the P/Invoke calls into the Win32 API yourself.

This gets a little more complicated with PowerShell, though, since we abstract the .Net framework one level even further. 

A good (and most flexible) solution is to write a cmdlet.  That’s the approach that Tony took in his Add-ProcessOwner cmdlet.  However, for the large subset of P/Invoke calls that have relatively simple signatures, we can use the PowerShell scripting language to do what we want.  The solution is Invoke-Win32 – a function which uses code generation to dynamically create a.Net P/Invoke definition, and then calls that P/Invoke method.  The very bright Juoku Kynsijarvi gave an example of this in the PowerShell newsgroup a long time ago.

This script also gives an example of the [Ref] type we added in our most recent drop.  The [Ref] type is a reference to a variable.  When you pass a [Ref] type to a native method with an OUT parameter, we allow that method to change the value of your variable.

If you want to interact with the value of a variable in a [Ref] type in PowerShell, you explicitly get and set its .Value property.

##############################################################################
##
## Get-ProcessOwner.ps1
## Get the owner of a process.
##
##############################################################################

param([System.Diagnostics.Process] $process)

$TOKEN_QUERY = 0x0008

function Main 
{
   $token = [IntPtr]::Zero
   $processHandle = $process.Handle
   
   if($processHandle -ne $null)
   {
      if((OpenProcessToken $process.Handle $TOKEN_QUERY ([Ref] $token)) -eq 0)
      {
         throw "Could not retrieve token for $($process.ProcessName)"
      }
   }
   
   if($token -ne 0)
   {
      new-object System.Security.Principal.WindowsIdentity $token
      CloseHandle $token > $null
   }
}

## Invoke a Win32 P/Invoke call.
function Invoke-Win32([string] $dllName, [Type] $returnType, 
   [string] $methodName, [Type[]] $parameterTypes, [Object[]] $parameters)
{
   ## Begin to build the dynamic assembly
   $domain = [AppDomain]::CurrentDomain
   $name = New-Object Reflection.AssemblyName 'PInvokeAssembly'
   $assembly = $domain.DefineDynamicAssembly($name, 'Run')
   $module = $assembly.DefineDynamicModule('PInvokeModule')
   $type = $module.DefineType('PInvokeType', "Public,BeforeFieldInit")

   ## Go through all of the parameters passed to us.  As we do this,
   ## we clone the user's inputs into another array that we will use for
   ## the P/Invoke call.  
   $inputParameters = @()
   $refParameters = @()
  
   for($counter = 1; $counter -le $parameterTypes.Length; $counter++)
   {
      ## If an item is a PSReference, then the user 
      ## wants an [out] parameter.
      if($parameterTypes[$counter - 1] -eq [Ref])
      {
         ## Remember which parameters are used for [Out] parameters
         $refParameters += $counter

         ## On the cloned array, we replace the PSReference type with the 
         ## .Net reference type that represents the value of the PSReference, 
         ## and the value with the value held by the PSReference.
         $parameterTypes[$counter - 1] = 
            $parameters[$counter - 1].Value.GetType().MakeByRefType()
         $inputParameters += $parameters[$counter - 1].Value
      }
      else
      {
         ## Otherwise, just add their actual parameter to the
         ## input array.
         $inputParameters += $parameters[$counter - 1]
      }
   }

   ## Define the actual P/Invoke method, adding the [Out]
   ## attribute for any parameters that were originally [Ref] 
   ## parameters.
   $method = $type.DefineMethod($methodName, 'Public,HideBySig,Static,PinvokeImpl', 
      $returnType, $parameterTypes)
   foreach($refParameter in $refParameters)
   {
      $method.DefineParameter($refParameter, "Out", $null)
   }

   ## Apply the P/Invoke constructor
   $ctor = [Runtime.InteropServices.DllImportAttribute].GetConstructor([string])
   $attr = New-Object Reflection.Emit.CustomAttributeBuilder $ctor, $dllName
   $method.SetCustomAttribute($attr)

   ## Create the temporary type, and invoke the method.
   $realType = $type.CreateType()
   $realType.InvokeMember($methodName, 'Public,Static,InvokeMethod', $null, $null, 
      $inputParameters)

   ## Finally, go through all of the reference parameters, and update the
   ## values of the PSReference objects that the user passed in.
   foreach($refParameter in $refParameters)
   {
      $parameters[$refParameter - 1].Value = $inputParameters[$refParameter - 1]
   }
}

function OpenProcessToken([IntPtr] $handle, [UInt32] $tokenAccess, [Ref] $token)
{
   $parameterTypes = [IntPtr], [UInt32], [Ref] 
   $parameters = $handle, $tokenAccess, $token

   Invoke-Win32 "advapi32.dll" ([Int]) "OpenProcessToken" $parameterTypes $parameters
}

function CloseHandle([IntPtr] $handle)
{
   $parameterTypes = [IntPtr]
   $parameters = [IntPtr] $handle

   Invoke-Win32 "kernel32.dll" ([Bool]) "CloseHandle" $parameterTypes $parameters
}

. Main

 

One question that comes up frequently is, “Why does PowerShell not change its [System.Environment]::CurrentDirectory as I navigate around the shell?”

One of the difficult aspects of this comes from the fact that PowerShell supports multiple pipelines of execution.  Although it’s not directly exposed yet, users will soon be able to suspend jobs to the background, and other concurrent tasks.

The current directory affects the entire process, so if we change the directory as you navigate around the shell, you risk corrupting the environment of jobs you have running in the background.

When you use filenames in .Net methods, the best practice is to use fully-qualified path names.  The Resolve-Path cmdlet makes this easy:

[C:\temp]
PS:37 > $reader = new-object System.Xml.XmlTextReader (Resolve-Path baseline.xml)
Suggestion: An alias for Resolve-Path is rvpa

[C:\temp]
PS:38 > $reader.BaseURI
file:///C:/temp/baseline.xml

If you wish to access a path that doesn’t already exist, use the Join-Path  cmdlet:

(Join-Path (Get-Location) newfile.txt)