Precision Computing

As people begin to explore PowerShell, the question often comes up – "Will PowerShell support the cmdlet or language feature <foo>?"

For example (from the newsgroup,) "Is there a way to get the unmodified contents of a file – since Get-Content splits it into lines?"  Or from Occasionally Rabid Programmer, "… Is there a zipping PowerShell class yet?"  For many of these questions, the answer is, "Yes. Since PowerShell gives you access to all of the .Net Framework, you could also ask 'Does the .Net Framework support <Foo> yet?'"

Clearly, PowerShell is evolving, and we have no plans to make you write everything yourself.

For many scenarios, though, people will often even solve their problem using classes from the .NET Framework, but worry that it isn't "PowerShell-y" enough.

There is often a distinction, but usually there is not.  We designed PowerShell to allow you to smoothly glide between typical scripting tasks and more complex programming tasks -- our support for .Net being a key to this spectacular amount of breadth.

For example, PowerShell will likely not ever support random I/O on a file stream through a cmdlet or language primitive.  That's a task that the .NET Framework handles well, using a model that users are familiar with.  It doesn't make much sense for PowerShell to re-implement all of that, just as there will always be sections of the Win32 API that .Net users need to call through P/Invoke techniques.

As you move away from goals obviously covered by language primitives and stock cmdlets, there comes a point where it just makes sense to leverage the powerful features of .NET interop.  Learning the breadth of the.NET Framework Library takes some patience, but is well worth it.  You can start by browsing and searching on http://msdn.microsoft.com/library/, and it is also a topic that my upcoming PowerShell Quick Reference deals with in some depth.

A question came up today on the newsgroup asking why the following didn't work:

PS >$secureString = ConvertTo-SecureString "Hello"
ConvertTo-SecureString : Cannot process argument because the value of argume
nt "input" is invalid. Change the value of the "input" argument and run the
operation again.
At line:1 char:39
+ $secureString = ConvertTo-SecureString  <<<< "Hello"
PS >

For some background -- a SecureString is a type of string that PowerShell (and .Net) keeps encrypted in memory.  Even if an attacker can explore the memory on your computer (like the contents of a swap file, for example,) they cannot gain access to the secret protected by the SecureString.

Although you can pass around SecureStrings with impunity, applications must be extremely careful at the boundaries -- when creating SecureStrings and retrieving the encrypted data from them.  This means doing things like reading your password input character by character, then removing each character from memory as soon as possible.  If the data is ever stored as a regular string, it stays in memory until the process exits.

By typing a regular string onto the command line (like in the example,) the string can no longer be made secure.  That specific string stays in memory until PowerShell exits.  This is why ConvertTo-SecureString only accepts the encrypted output of ConvertFrom-SecureString.  Only in that way can we retain the security guarantee of SecureStrings.

That said, most people aren't that concerned about an attacker spying on their machine's memory, or digging through their Windows pagefile.  In many situations, the benefit of being able to automate these situations vastly outweights the potential security risk.

For the upcoming release candidate, we've added some new functionality to allow this:

$secureString = ConvertTo-SecureString "Hello" -AsPlainText -Force

(The force flag lets you bypass the warning I just gave you :)   )

Until then, you can create SecureStrings from plain text this way:

$text = "Hello World"
$secureString = new-object Security.SecureString
$text.ToCharArray() | % { $secureString.AppendChar($_) }

[Edit: Ouch... my first dupe!]

The PowerShell range operator allows you to generate lists of numbers out of more compact expressions:

PS >1..5
1
2
3
4
5

PS >6..4
6
5
4

Something that came up on the newsgroup was the desire for range operators on data types other than the numeric ones.  It’s a feature we wanted to add, but weren’t able to.  In the meantime, this script / function accomplishes the goal quite well:

PS >range Friday Monday DayOfWeek
Friday
Thursday
Wednesday
Tuesday
Monday

PS >range y v char
y
x
w
v

Over the past few days, the Exchange team has been demo-blogging features of their new administrative interface.  The power is obvious, and the benefit it brings to administrators is palpable.

They’ve built their entire administration surface on PowerShell cmdlets, so today’s post really ties the series together nicely. 

They go through 14 of their most common scenarios with screen shots to illustrate all of the GUI actions.  Then, for each, they show some incredibly intuitive PowerShell one-liners that accomplish the same task (and more!)

The Exchange team deserves a lot of credit – this is truly a phenomenal advance in the administration world.  Even if you’ve seen it discussed before, you should definitely take a look:

"Management Console Overview": http://msexchangeteam.com/archive/2006/08/21/428728.aspx
"Recipient Management Overview": http://msexchangeteam.com/archive/2006/08/22/428740.aspx
"Recipient Management One-Liners": http://msexchangeteam.com/archive/2006/09/05/428833.aspx

A question came up recently in the newsgroup, asking why adding seemingly different items to an ArrayList resulted in the ArrayList being full of the same item:

PS >$table = new-object Collections.ArrayList
PS >$row = 1 | select Col1,Col2,Col3
PS >$row.Col1 = "Column 1"
PS >$row.Col2 = "Column 2"
PS >[void] $table.Add($row)
PS >$row.Col1 = "Column 1, again"
PS >$row.Col2 = "Column 2, again"
PS >[void] $table.Add($row)
PS >$table

Col1                       Col2                       Col3
----                       ----                       ----
Column 1, again            Column 2, again
Column 1, again            Column 2, again

The source of the difference comes from the type of data that you assign to a variable.  Data falls into two categories: value types, and reference types.  Some data falls in the "value type" category, where its value is set in stone.  Changing the value actually creates a new one, as does passing it around:

PS > $a = "Foo"
PS > $b = $a
PS > $a += " Test"
PS > $a
Foo Test
PS > $b
Foo

Think of this like a promise you give to somebody.  If that person gives the same promise to somebody else, it's a new promise.  If you change your promise, it's a new promise yet again. 
 
Value types are the .Net primitives: string, int, bool, double, float, etc.
 
The other type category is called a "reference type."  Think of this like a bag you give to somebody.  If that person gives the bag to somebody else, it's still the same bag.  If you put stuff in the bag, take stuff out of the bag, or modify the bag -- it's still the same bag.
 
Most types are reference types.  Examples include ArrayList, StringBuilder, and some of the PowerShell primitives (ie: the Hashtable) -- since they are not .Net primitives.
 
The expression, "$row = 1 | select Col1, Col2, Col3" gives $row a PsObject -- which is a reference type.  When you change values in $row, you change it for everything that is talking about $row – including the copies inside the ArrayList.

In this case, the solution is to create new rows each time:

PS >$table = new-object Collections.ArrayList
PS >$row = 1 | select Col1,Col2,Col3
PS >$row.Col1 = "Column 1"
PS >$row.Col2 = "Column 2"
PS >[void] $table.Add($row)
PS >$row = 1 | select Col1,Col2,Col3
PS >$row.Col1 = "Column 1, again"
PS >$row.Col2 = "Column 2, again"
PS >[void] $table.Add($row)
PS >$table

Col1                       Col2                       Col3
----                       ----                       ----
Column 1                   Column 2
Column 1, again            Column 2, again

One of my favourite internal blogs has long been Terry Zink’s blog on spam analysis and fighting – from the perspective of somebody working on the Microsoft Exchange Hosted Services team.

After some prodding by me (and probably others,) his blog is now public!

And to make things even better, he’s Canadian :)

Something extremely helpful in learning a technology such as PowerShell is a pocket guide and quick reference.  It doesn’t necessarily replace a structured tutorial as a learning resource, but fills an important role as a daily desktop reference.

Over the last while, I’ve been working with O’Reilly to create such a guide – as the most ambitious addition to their “Short Cuts” program to date.

The reference will be available in the very near future, so stay tuned!

Jim Truher just posted a script that he's been polishing for the last few months -- Background "jobs" and PowerShell.  It's awesome, and one of the first things that people ask for once they start settling in to PowerShell.

What I love about being surrounded by a bunch of other PowerShell geeks is watching scripts like this come together.  If I remember correctly, this script started off as a response to a question on our internal PowerShell distribution list.  Marcel showed some examples of how to work with Runspaces through the scripting language, Jim added a bunch to it, and it quickly snowballed from there.

Even cooler, though, is watching the same thing happen outside of Redmond.

Probably the most useful network tool on any operating system is Telnet.  Not for connecting to Telnet servers, of course, as the Telnet protocol is about as insecure as they come.  Instead, it’s useful for debugging connection problems with arbitrary ports and arbitrary protocols.

Debugging an HTTP problem?  You can Telnet into port 80 to help you resolve it.
Debugging a mail retrieval issue?  You can Telnet into port 110 to help you resolve it.
Debugging a mail sending issue?  You can Telnet into port 25 to help you resolve it.
<etc>

Unfortunately, this workhorse was removed from Vista’s default installation.  Here’s a simple PowerShell replacement script.  It’s great for debugging, but useless (of course) for terminal emulation:

## Connect-Computer.ps1
## Interact with a service on a remote TCP port
param(
    [string] $remoteHost = "localhost",
    [int] $port = 80
     )

## Open the socket, and connect to the computer on the specified port
write-host "Connecting to $remoteHost on port $port"
$socket = new-object System.Net.Sockets.TcpClient($remoteHost, $port)
if($socket -eq $null) { return; }

write-host "Connected.  Press ^D followed by [ENTER] to exit.`n"

$stream = $socket.GetStream()
$writer = new-object System.IO.StreamWriter($stream)

$buffer = new-object System.Byte[] 1024
$encoding = new-object System.Text.AsciiEncoding

while($true)
{
   ## Allow data to buffer for a bit
   start-sleep -m 500

   ## Read all the data available from the stream, writing it to the
   ## screen when done.
   while($stream.DataAvailable) 
   { 
      $read = $stream.Read($buffer, 0, 1024)   
      write-host -n ($encoding.GetString($buffer, 0, $read)) 
   }

   ## Read the user's command, quitting if they hit ^D
   $command = read-host
   if($command -eq ([char] 4)) { break; }
 
   ## Write their command to the remote host     
   $writer.WriteLine($command)
   $writer.Flush()
}

## Close the streams
$writer.Close()
$stream.Close()

I’ve always loved Scott’s ultimate list of tools.  To make things even better, PowerShell now plays a big part in this year’s list :)

To add to the list, here are the first tools (not covered by Scott) I add to my systems:

Start | Control Panel | Regional Settings | … | Dvorak

Setting my system to the Dvorak keyboard layout is one of the first things I do on my non-work computers.  Typing feels much more efficient and effortless as compared to the QWERTY keyboard layout.

MakeMeAdmin

Aaron Margosis’ essential tool that lets you easily work with your system as a limited user.  Much more efficient than XP’s Fast User Switching.

Vim / Emacs

The venerable editors.  I seem to switch between them every few years – I’m currently using Vim because Emacs gives me grief over Unicode files.  Of course, syntax highlighting for PowerShell scripts is a must:
 VIM: http://www.vim.org/scripts/script.php?script_id=1327
 Emacs: http://www.viveksharma.com/techlog/2006/08/05/updated-powershell-script-editing-mode-for-emacs-powershell-modeel/

KeePass

Strong “Password Safe” program that also has a Pocket PC port.  Allows me to use strong passwords on all of my accounts, without having to remember them all.

Clipomatic

A helpful utility that caches the last ten entries in your clipboard.  I’m not gaga about this one, but it seems to do the best job of the free ones I’ve found.  Unfortunately, this app suffers (as most clipboard monitor software does) from Windows’ Remote Desktop messing around with the clipboard.

Tortoise Subversion

Checking all of your important documents into source control is a backup dream come true.  Especially when your source control server doubles as a cat feeder :)

FileZilla

The best free FTP program I’ve found.  SmartFTP used to fit that bill until it started nagging.

RealAlternative / QuicktimeAlternative  / K-Lite Codec Pack

A triumvirate that makes your media viewing life so much easier.  Provides alternatives to the evil RealPlayer and Quicktime bloatware, and also gives you the codecs required to play most popular file formats.

Windiff / ExamDiff

For file comparison tools, I use Windiff the most because people at work are used to it.  However, ExamDiff really is a slick application.

VncViewer

Nice little app to let you connect to a remote VNC session.

Putty

Although I prefer TerraTerm as an SSH client in general, Putty won its place in my tools directory because it requires only a single file.

AutoIt

A cool Windows UI automation program.  I especially like its COM interface, which can be easily scripted via PowerShell.

PerfCompare

A little benchmarking application to help you compare the efficiency of C# code snippets.

Packetyzer

A great network sniffer and protocol analyzer.

Items already in Scott’s list that deserve more attention:

Fiddler

A great HTTP inspector that just continues to get better.  I lived in this program when I worked for MSN Encarta, and heavily lean on it when playing with online hacking games

AutoHotKey

This program is much more than “AutoCorrect for Windows.”  It has powerful scripting abilities, and lets you automate tons of tasks in Windows.

Paint.Net

An awesome image editor that definitely replaces the open source “GIMP”