Archives for the Month of April, 2006

Score One for Corporate Transparency

Knowledge is power – and power is a limited resource.  At least, that’s what some folks believe as they obsessively guard information from others.

That’s a load of tripe.  It creates a work environment so fundamentally backwards: employees direct their efforts toward breaking into the cliques, cabals, and secret societies just to get their jobs done – leaving no time for more important things (like customers.)

There are plenty of examples of this inside of Microsoft.  Rather than give a bad example of information hoarding, let me instead give a powerful example of transparency – the folks behind the Windows Live Mail team.

I had a friend (also a Windows Live Mail customer) mail me yesterday in desperation.  He’d been struggling with an issue in his mail account, and contact with official support channels proved fruitless.  They told him it was a known bug, and would be fixed in the future.  He asked me if I could provide any additional information.

Traditionally, this would result in me contacting a blogger from the product, or sending a mail to their internal discussion alias (if they have one.)  That’s good, but you’re still taxing the resources of the product team.  Self-serve is better.

My first stop was the internal site that lets us search bugs.  Most products publish their bug databases to this site, although many refrain.  So, first tally for transparency – publishing your bugs.  I found a few related to the issue – one with the comment, “this has been turned into a feature, and will be tracked for <milestone x>.”  It included a (broken :)) link to the internal Windows Live Mail SharePoint feature tracking site.

Normally, this would be pretty much the end of the road.  I could contact the last person that made comments in the bug and ask them the question.  That’s good, but you’re still taxing the resources of the product team.  Self-serve is better.

Instead, I was able to go to the feature tracking site and look at what they listed for <milestone x.>  Second tally for transparency: publishing your work items.  I found the work item, and was able to open its full details.  They included the development and PM contacts, and also allowed me to verify that this was the same issue I was looking for.  However, I had no idea when <milestone x> would be live.

Next, I looked at their published master calendar on the same Sharepoint site.  Third tally for transparency: publishing your schedule.  It lists all of the milestone dates, code complete dates, integration dates, release to operations, release to web (live site,) and more.  I simply searched for <milestone x,> and found its release date (which happens to be soon).

So, my final email to the contacts listed in the feature details?

Hi <PM> and <Dev>;

I’ve got a friend who is ... eagerly looking forward to this feature – “<Feature>".  From <feature tracking site>, it looks like the feature is scheduled to ship with <milestone x>. 

From <ship calendar site>, it looks like <milestone x> code goes live on <date>.  Did the feature make it into <milestone x>?

The answer was “yes.” I was able to do nearly all of the groundwork required to become an ambassador for an entirely different team.  Rather than jealously guard the information, the team created value from thin air.  Kudos to the Windows Live Mail team for being so progressive.

The Story Behind the Naming and Location of PowerShell Profiles

“<My Documents>\WindowsPowerShell\Microsoft.PowerShell_profile.ps1.”
and
“<Installation Directory>\Microsoft.PowerShell_profile.ps1.”

It’s probably the 63 characters in PowerShell that have the most thought behind them.  You might ask, “Could it be any longer?  More random?”

Sure!  Other considerations were:

[C:\temp]
PS:55 > $random = new-object Random

[C:\temp]
PS:56 > $output = $null1..100 | % { $output += [char]$random.Next(97122) }; "$output.ps1"
wvrybyxwmtcftvhnsvbednocjrdkcyysjtwnhbkkfrgtamxxdbeckjjjgopivnumtjuaxsgrlpylhucvtegauwhajnhrlbnqwsyp.ps1

[C:\temp]
PS:57 > $output = $null1..100 | % { $output += [char]$random.Next(97122) }; "$output.ps1"
rwyjumqtjxbklsorgtrwbuqklnfjbhonulevewfhfnpllvgslvkcacowgowgggrbinpynsminjqneeypglwewlmswhopaxbxuocx.ps1

[C:\temp]
PS:58 > $output = $null1..100 | % { $output += [char]$random.Next(97122) }; "$output.ps1"
ggfsjbnqlbrbogytjmdvavcfevioirsocbjageuvwvhuomasxgohkfwjeynnryhsidynwynerhxdogpqupqcclevdavgpgexonml.ps1

[C:\temp]
PS:59 > $output = $null1..100 | % { $output += [char]$random.Next(97122) }; "$output.ps1"
mmuyrkodgttwjwschfqvxckwdgnlcynbtdditrjrvboifywsnbqjwpnkbxulvcahfscfskhpuxawnbqeiuxbieddpsgvvxutahdm.ps1

 

🙂

Actually, the reason we introduced the $profile variable is to make this as short as possible to you, while still serving our extensibility needs.

So, first off: “My Documents.”

A disconcerting trend is for applications to spew their files all over this directory.  “My Widgets,” “My eBooks,” etc.  They do this even if you’ve never launched the application, or used the given file.  Knowing that, we still ask users to place their profile in their “My Documents.”  This is for two reasons. 

First, we don't generate the files in this directory - they are user-authored documents, not application configuration files.  The folder will also contain other user-authored documents, such as types and formatting customizations.  Even more, the profile is one of a shell user's most treasured documents.  Users quickly consider this to be a vitally personal document - along with their .emacs / .vimrc.  The internet is full of these "dotfiles" that people are proud enough to share.  It is already one of the first and most common things that PowerShell users share. In Word, you generate Word documents.  In Excel, you generate spreadsheets. In PowerShell, you generate profiles, scripts, and ps1xml extension files.  If a user can't point their finger at a file in "My Documents" and remember putting it there, then it's in the wrong spot.

This used to be in a directory called "PsConfiguration" (to be consistent with the installation directory of the All Users' version.)  Since that is no longer where we store the machine-wide profile, this has now changed to a much more understandable "WindowsPowerShell."

Second, the “Application Data” is hidden by default.  That breaks tab completion, discoverability, and so much more for this commonly-used document.

Next: “<Installation Directory>” (for the all-user profiles)

We chose this location because it is the only reasonable and secure location to place a startup script that runs for every user.  If a malicious user can edit this file, they can add commands to it that will elevate that user to Administrator the next time the Administrator logs in (and automatically runs that profile script.) 

We used to store these files in a protected directory under "All Users Documents," but that could still cause security problems.  If the administrator deletes the folder from the “All Users” directory, the Windows ACL lets any user recreate it.  A malicious user could then add an auto-running profile there.

It should really go into an /etc directory, but that oddly seems to be missing in the version of Unix called Windows that I'm currently running 🙂

Finally, “Microsoft.PowerShell_profile.ps1”

The PowerShell model supports the concept of multiple shells (“hosts”).  For example, the PowerShell.exe console application is the host that most users will know and love.  It has a certain Shell ID.  Karl Prosser's PowerShell Analyzer is another shell.  It has its own Shell ID. In the future, we will have a GUI host that also has its own Shell ID.  You may want code that runs in one of those shells, but not the others. Profile code that accesses $host.UI.RawUI is something that you would probably want to be specific to a shell, for example.

In those situations, you would use the $profile variable. That variable refers to the user-specific profile for the current shell.  Its name derives primarily from the Shell ID of the current shell.  In contrast, the plain "profile.ps1" file is loaded for every shell.

Technically, the PowerShell engine loads profiles in the following order:

  1. "All users" profile is loaded from "<Installation Directory>\profile.ps1"
  2. "All users," host-specific profile is loaded from "<Installation Directory>\Microsoft.PowerShell_profile.ps1"
  3. Current user profile is loaded from "<My Documents>\WindowsPowerShell\profile.ps1"
  4. Current User, host-specific profile is loaded from "<My Documents>\WindowsPowerShell\Microsoft.PowerShell_profile.ps1"

With only one shell (PowerShell.exe,) it doesn't really matter which you use if you use only PowerShell.exe.  I personally use the zenfully correct $profile to hold my modifications, though.

[Edit: Fixed typo in "All Users" document load location.  Thanks, Tony!]
[Edit 10/04/06: Updated to describe the more recent changes in RC2]

Monad Evolves to Windows PowerShell

Bob Muglia’s Microsoft Management Summit (MMS) keynote this morning unveiled a series of exciting announcements surrounding Monad – now known as Windows PowerShell:

This marks a significant milestone in the Windows PowerShell product lifecycle, and brings us an important step closer to getting a finished product in your hands.

The MMS announcement (and updated beta) will also bring many more users into our community.  If you’re one of them, welcome!  Here are some helpful “getting started” resources (thanks to Bob Wells for collecting these!)

Microsoft TechNet Script Center
===============================
  
http://www.microsoft.com/technet/scriptcenter/default.mspx

   Scripting with the Microsoft Shell
  
http://www.microsoft.com/technet/scriptcenter/hubs/msh.mspx

Webcasts
========
   Next Generation Command Line Scripting with Monad (Part 1 of 2)
  
http://msevents.microsoft.com/cui/eventdetail.aspx?eventID=1032277850&Culture=en-US

   Next Generation Command Line Scripting with Monad (Part 2 of 2)
  
http://msevents.microsoft.com/cui/eventdetail.aspx?eventID=1032277852&Culture=en-US

Miscellaneous
=============
   Articles:
      "Monad: The Future of Windows Scripting", by Thomas Lee
      TechNet Magazine, November o December 2005, Scripting Column
     
http://www.microsoft.com/technet/technetmag/issues/2005/11/Scripting/default.aspx

      "A guided tour of the Microsoft Command Shell", by Ryan Paul
     
http://arstechnica.com/guides/other/msh.ars

   Blogs:
      PowerShell Team Blog
     
http://blogs.msdn.com/PowerShell/

      Arul Kumaravel's Blog
      http://blogs.msdn.com/ArulK

      Precision Computing: Software Design and Development (Lee Holmes' blog)
     
http://www.leeholmes.com/blog/default.aspx

     
   Books:
      Monad
      Introducing the MSH Command Shell and Language
      by Andy Oakley
      First Edition December 2005
      ISBN: 0-596-10009-4
      206 pages, $34.95 US, $48.95 CA, £24.95 UK
     
http://www.oreilly.com/catalog/msh/

   Newsgroups:
      Microsoft Windows Server Scripting
     
nntp://microsoft.public.windows.server.scripting

   Sample Scripts:
      The Monad Script Centre
     
http://www.reskit.net/monad/samplescripts.htm

   Videos:
      Channel 9 Forums >> The Videos >> Jeffrey Snover >> Monad explained
     
http://channel9.msdn.com/ShowPost.aspx?PostID=25506

      Channel 9 Forums >> The Videos >> Jeffrey Snover >> Monad demonstrated
     
http://channel9.msdn.com/ShowPost.aspx?PostID=25915

      Channel 9 Forums >> The Videos >> Jeffrey Snover >> More talking about Monad
     
http://channel9.msdn.com/Showpost.aspx?postid=127819

   Wikis:
      Wikipedia
     
http://en.wikipedia.org/wiki/MSH_(shell)

      Channel 9 MSH Wiki
     
http://channel9.msdn.com/wiki/default.aspx/Channel9.MSHWiki

   Additional Resource Listings (like this list, only different):
      Monad-MSH
     
http://www.reskit.net/monad

 

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Using msh.exe interactively from within other programs

In a recent post, Andy describes his quest to use msh.exe as the Emacs interactive shell.  He got single commands to work, but the interactive experience does not output any prompts.  This will be the experience from any application that redirects Monad’s standard input stream.

First, the syntax to make Monad read commands from a redirected input stream is “-command –“.  This places Monad into batch input mode.  In the batch input mode, Monad treats standard input as though it is content in a script, allowing a scenario like:

type test.msh | msh -command - > output.txt

Batch input mode suppresses prompting so that the prompts do not interfere with the script’s output.  You will see a similar problem if you try to use Monad directly as a Telnet / SSH endpoint

We are aware of this limitation, and hope to fix the issue in V2.  As always, we invite you to provide your feedback on this decision at the Monad Microsoft Connect site.

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Breaking Open the dir-LIVE Script

I mentioned earlier that I was going to dissect the dir-LIVE script – it’s taken a bit longer to get to it than I had planned, but I tried to make even this joke a learning opportunity.

When you open the script, the first thing you’ll notice is that it’s mostly composed of junk characters.  “Wow,” you might say – “how does that turn into a directory listing?”

 It’s really just a slight-of-hand – not a super cool obfuscation technique.

This encoding is entirely enabled by Monad’s ability to execute the content of a string as though it were script.  For example:

MSH C:\temp> invoke-command "2+2"
4

The $contents variable isn’t just a script in a string, though.  It’s encoded.  The .Net framework natively supports a method of encoding called “Base64 encoding,” one of the most common ways to transport binary data over connections that support only plain ASCII text.  Email is one example of a technology that often uses Base64 encoding.  However, I just did it to make the script more difficult to read 🙂 

So, I took my original script, Base64-encoded it, and put the results in a string.  The joke script then decodes the string, and invokes it.  This was not a security measure though (and should never be considered one,) as it is trivial to modify the script to print out the content rather than execute it.  Admit it – you had some fun hacking my script to read it 🙂

Executing the content of a string is normally a one-step operation – you use the “invoke-command” cmdlet.  However, we’ve renamed the cmdlet in our internal builds.  This missing cmdlet would have broken the script for any of the many internal Microsoft folks running our next drop of code.

To get around this, the script checks to see if the “invoke-command” cmdlet exists.  If it does, it remembers to use that command.  Otherwise, it knows to use the new name, “invoke-expression.”  The script then uses Monad’s “&” syntax to execute the appropriate “invoke-*” command, with the decoded string as its argument.

Here is the script I wrote to do this obfuscation automatically:

## Obfuscates a script

param([string] $inFile)

$content = [String]::Join("`n", @(get-content $inFile))
$bytes = (new-object System.Text.UnicodeEncoding).GetBytes($content)
$encoded = [Convert]::ToBase64String($bytes)
$destinationContent = @'
$contents = @"

'@

$destinationContent += $encoded

$destinationContent += @'

"@

$contentBytes = [Convert]::FromBase64String(($contents -replace "``n",""))
$contentString = (new-object System.Text.UnicodeEncoding).GetString($contentBytes)

$invoker = get-command -ea SilentlyContinue invoke-command
if(-not $invoker) { $invoker = get-command invoke-expression }

& $invoker $contentString
'@

$destinationContent

So, here is the script in its original form:

## dir-LIVE.msh
## Participate in the Web 2.0 revolution

$generator = new-object Random
$oldForeground = $host.UI.RawUI.ForegroundColor

$sponsored = @(
   ,("System""$($env:WINDIR)""Meet fun young DLLs in your area")
   ,("Temp""$($env:TEMP)""Stash your stuff.  No credit card required.")
   ,("Home""$([Environment]::GetFolderPath(`"Personal`"))""Secure a lower mortgage today.  0% refinancing available!")
   ,("Autorun""HKLM:\Software\Microsoft\Windows\CurrentVersion\Run""90% of all PCs have spyware.  DO YOU?")
   ,("Certificates""Cert:\CurrentUser\My""Locate $($env:USERNAME).  Perform a background check on ANYBODY!")
   ,("Aliases""Alias:\ ""New season, new time.  Only on ABC.")
   ,("Recycle Bin""C:\RECYCLER\S-1-5-21-823518204-813497703-1708537768""Looking for junk?  Find exactly what you want - only on EBAY.")
   ,("Current Location""$(get-location)""You are broadcasting your CWD!  Protect your system from hackers.")
)

$newCommand = $myInvocation.Line.Replace($myInvocation.InvocationName, "get-childitem")
foreach($alias in (get-alias | where { $_.Definition -eq $myInvocation.InvocationName }))
{
   $newCommand = $newCommand -replace $alias.Name,"get-childitem"
}

$invoker = get-command -ea SilentlyContinue invoke-command
if(-not $invoker) { $invoker = get-command invoke-expression }

$directoryContent = ((& $invoker $newCommand) | out-string).Split("`n")

function Main
{

   $lineCounter = 0
   foreach($line in $directoryContent)
   {
      $output = $line.TrimEnd()

      if(($lineCounter -eq 7) -or 
         (($lineCounter -gt 7) -and (($lineCounter % 21) -eq 0)))
      {
         $index = $generator.Next(0, $sponsored.Count)
         $host.UI.RawUI.ForegroundColor = "Yellow"
         GetSponsorLink $index
         $host.UI.RawUI.ForegroundColor = $oldForeground
      }

      $output
      $lineCounter++
   }
}

function GetSponsorLink
{
   param([int] $index)

   ""
   "+---------------------------------------------------------------------+"
   "|                                                                     |"
   "| SPONSORED CHILDITEMS:                                               |"
   "|                                                                     |"

   $currentAd = $sponsored[$index]
   $trimmedDescription = $($currentAd[1]).SubString(0,[Math]::Min($currentAd[1].Length,48))
   $adText = "| $($currentAd[0]) - $trimmedDescription".PadRight(70) + "|`n"
   $adText += "| $($currentAd[2])".PadRight(70) + "|`n"
   $adText += "|".PadRight(70) + "|"
   $adText

   "+---------------------------------------------------------------------+"
   ""
}

. Main

 

The script is fairly straight-forward, except for one part.  At a high-level, it does the following:

  1. Defines the 8 random advertisements that could be placed in a listing
  2. <gloss over> Complex magic to get a directory listing </gloss over>
  3. Injects advertisements approximately every 21 child items

#2 is the part that is not straight forward.  Get-ChildItem is a complex cmdlet – it supports filters, targets, recursive descents, and even dynamic parameters like “-codesign” on the certificate provider.  I had no intention of making this a production-ready replacement for Get-ChildItem, but I wanted the –LIVE version to proxy the parameters as seamlessly as possible.

To do this, we first get the command as typed by the user:

$newCommand = $myInvocation.Line(...

And replace the current script’s name with “get-childitem”:

(…).Replace($myInvocation.InvocationName, "get-childitem")

So, for example:

When we run the script like this:

C:\temp\dir-LIVE.msh . *.cs -rec -codesign

That becomes $myInvocation.Line.  $myInvocation.InvocationName becomes “c:\temp\dir-LIVE.msh”.  So we get:

Get-childitem . *.cs –rec –codesign

If you found buggy behaviour (ie: “dir-LIVE dir-LIVE.msh”), this string replacement is almost certainly at fault – as Monad does not yet support transparent proxying to other commands.

I also over-engineered this one a bit in the next stage.  If you aliased anything to “dir-LIVE.msh,” then the script does the same replacement technique for all aliases that are defined to replace dir-LIVE.msh.

And there you have it … obfuscation, a way to cope with breaking changes, and a fragile way to proxy other cmdlets.

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Monad Team Foundation Source Control Provider Now Available

After people learn about Monad providers, one of the most common things I hear is -- "I wish there was one for Team Foundation's Source Control."

James Manning (from the TFS team) agrees!  He wrote one in November of last year, with hopes to push it out as a PowerToy.  Instead, he did us one better by putting it up on the MSH Community Extensions workspace!

MSH jmanning-test:\> dir -name -r
60406-testing
60406-testing\ConsoleApplication77
60406-testing\ConsoleApplication77\ConsoleApplication77
60406-testing\ConsoleApplication77\ConsoleApplication77.sln
60406-testing\ConsoleApplication77\ConsoleApplication77.vssscc
60406-testing\ConsoleApplication77\ConsoleApplication77\ConsoleApplication77.csproj
60406-testing\ConsoleApplication77\ConsoleApplication77\ConsoleApplication77.csproj.vspscc
60406-testing\ConsoleApplication77\ConsoleApplication77\Program.cs
60406-testing\ConsoleApplication77\ConsoleApplication77\Properties
60406-testing\ConsoleApplication77\ConsoleApplication77\Properties\AssemblyInfo.cs
MSH jmanning-test:\> dir -r | ft -a changesetid,serveritem

ChangesetId ServerItem
----------- ----------
          2 $/60406-testing
          3 $/60406-testing/ConsoleApplication77
          3 $/60406-testing/ConsoleApplication77/ConsoleApplication77
          3 $/60406-testing/ConsoleApplication77/ConsoleApplication77.sln
          3 $/60406-testing/ConsoleApplication77/ConsoleApplication77.vssscc
          3 $/60406-testing/ConsoleApplication77/ConsoleApplication77/ConsoleApplication77.csproj
          3 $/60406-testing/ConsoleApplication77/ConsoleApplication77/ConsoleApplication77.csproj.vspscc
          3 $/60406-testing/ConsoleApplication77/ConsoleApplication77/Program.cs
          3 $/60406-testing/ConsoleApplication77/ConsoleApplication77/Properties
          3 $/60406-testing/ConsoleApplication77/ConsoleApplication77/Properties/AssemblyInfo.cs

 

Enjoy.

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

HOWTO: Win any contest by being the only one competing

As pointed out by Scott Hanselman, somehow -- my get-answer script is being used as cudgel in a
Monad vs Ruby debate.  Ted Neward mentions how cool the script is (with a title bound to
stir Ruby natives.) Glenn Vanderburg, in kind, jumps to Ruby's defense.

As far as the language goes, Glenn's main point is that Ruby is much more concise than Monad.

This is where the title of my post comes from -- it's easy to win any contest if you're the
only one competing in it 🙂  I write my aricles with a heavy prejudice towards teaching,
illustration, and explanation.  Without that style, the Monad script can be written in a form
that is nearly line-for-line equivalent to the Ruby example:

param([string] $question = $(throw "Please ask a question.")) 

[void] [Reflection.Assembly]::LoadWithPartialName("System.Web")
$webClient = new-object System.Net.WebClient

$text =  $webClient.DownloadString("http://search.msn.com/encarta/results.aspx?q=$([Web.HttpUtility]::UrlEncode($question))")

if($text -match "<div id=`"results`">(.+?)</div></div><h2>Results</h2>"

   $partialText = $matches[1]
   $partialText = $partialText -replace "<\s*a.*?>.+?</a>"""
   $partialText = $partialText -replace "</(div|span)>""`n" 
   $partialText = $partialText -replace "<[^>]*>"""
   $partialText = ($partialText -replace "`n`n","`n").TrimEnd()
   $partialText

else 

  "No answer found." 
}

 

In any case, computer languages are not a zero-sum game.  You don't have to pick between Monad
and Ruby -- I don't.

 

[Edit: Bruce Payette offers this alternate "one-liner" 🙂 ]

Just for grins, here's the same thing as a 1-liner (i.e. 1 single expression.)

.{[Reflection.Assembly]::LoadWithPartialName("System.Web") -and "$args" -or $("Please ask a question.";return) -and (new-object Net.WebClient).DownloadString("http://search.msn.com/encarta/results.aspx?q=$([Web.HttpUtility]::UrlEncode(`"$args`"))") -match "<div id=`"results`">(.+?)</div></div><h2>Results</h2>" -and $(return ($matches[1] -replace "<\s*a.*?>.+?</a>", "" -replace "</(div|span)>", "`n"  -replace "<[^>]*>", "" -replace "`n`n","`n").TrimEnd()) -or $("No answer found.";return)}

Paste this into an Msh window followed by a question and you'll get your answer. (We need to run an obfuscated msh contest sometime soon...)

In practice, this isn't a great example for doing a language comparison - it's more about libraries and regular expressions than it is about language features.

 

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Monad Breaking Change – Alignment with Live.com strategy

Exchange has been taking significant heat from beta customers that neither it (nor Monad) align strongly with the Windows Live corporate strategy.  This is blocking adoption by many customers, especially those accustomed to the new Windows Live Mail (Kahuna) interface.

This change brings some of that branding synergy into the get-childitem cmdlet.

You can get a prototype of this new functionality here: dir-live.msh.txt (7.23 KB).  Rename to "dir-LIVE.msh" after downloading.  This is a breaking change, so please provide us with your feedback.

 

[Edit: Alas, this was not meant to be.  Perhaps in version 2.  Happy April Fools' :)]

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]