Archives for the Month of October, 2005

Selecting an execution policy — get-help about_signing.

As Adam points out in his most recent entry, our last release changed the default execution policy to a mode called "Restricted."  The first time you run a script in the new shell, you’ll see the following error message:

The file C:\my_script.msh cannot be loaded. The execution of scripts
is disabled on this system. Please see "get-help about_signing" for
more details.

This is probably not the mode you want to stay with, as it doesn’t run those awesome scripts you write.  Yeah, that’s right, YOUR awesome scripts.  Remember how nice I’m being to you, as I’m about to ask a favour 🙂

This has probably been one of the most difficult features we’ve implemented, but the reasons are not technical.  It’s difficult, because we know that it makes the shell nearly unusable out of the box.  It’s difficult, because we know we’ve essentially added a manual step to the automatic installation of Monad.

Why?  It’s for your sake, and for the safety of the ecosystem. 

In today’s malicious environment, certain software categories need to be held to a higher standard of security.  For these components, secure by default usually means unusable by default.  Or in other words, “some assembly required.” 

By default, firewalls block nearly all inbound traffic.  IIS serves only static HTML.  Internet Explorer’s “enhanced security configuration” visits only trusted sites.  Outlook disables all dynamic content, images, active links, and optionally even all HTML.  Taken to the extreme, consider the story of OpenBSD:

The open software development model has allowed the organisation to upgrade with an uncompromising view to security enhancement - anything that stands in the way of a more secure environment has been savagely pruned, the result being a clean, slippery system with a surface too tight for conventional breach.

How does this help you?  For one thing, we hope it makes it less likely that your computer administrator at work reflexively restricts you from using MSH altogether.

Getting back to the title of the post, how do you get out of this restricted mode, and which execution policy should you pick?  This is answered by a document included in the distribution zip file, “about_signing.help.txt.”  Although this is a help file for the product, we weren’t able to include it in the actual installer in time for release.  To make it work like the rest of the about_* help files, copy it to your MSH installation directory.

I personally set my execution policy to AllSigned, although most will probably want to run under the RemoteSigned policy that earlier drops were configured to use.  However, security decisions always involve tradeoffs.  Please read the about_signing documentation to understand the implications that your execution policy has.

As you install this new version of Monad, almost all of you will change your execution policy.  Some of you will also blog your installation experience.  A subset of you “installing bloggers” will change it to “Unrestricted,” based on your personal evaluation of its tradeoffs.  I honestly think you should run in RemoteSigned mode instead, but it’s not my job to convince you.  However, most of you ran under the RemoteSigned execution policy for the past several months without even realizing it existed.

Now, it’s time for me to ask that favour: please let your readership make their own decision about an execution policy, rather than suggest they run in Unrestricted mode.  They might not want to make the same security decision you have, but might not realize there is any other alternative.  The documentation provides the background they need to make an informed choice.

Also, I’d like to express my thanks to our Monad MVPs that provided feedback on an earlier piece I had written to summarize the Monad execution policies.  Since our documentation team didn’t have the time to write about_signing, it was written by yours truly.  Your feedback was extremely valuable in helping me craft the public documentation on the subject.

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

PDC videos now available

Many thanks to Cyrus for pointing this out.  Videos from the PDC are now available for streaming via http://microsoft.sitestream.com/PDC05/.  The site is running rather slowly right now (it's not being Slashdotted, that's for sure,) so bring a bag of patience with you.

I, of course, recommend you start with Jeffrey and Jim's Monad presentation (during which I was Proctoring at the hands-on-labs.)  As Jeffrey quotes, "This is not your Dad's filesystem".  Then, if you haven't already, check out this year's belle of the ball, Linq 🙂

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Get ready to learn Regular Expressions!

I'm sure you've noticed by now.  One of the things you end up doing a lot in shells and scripting languages is parsing text.  We've done it many times in articles on this blog.  We've seen it in many articles on other blogs.  We've all sat in quiet reverence of the wondrous power that is Regular Expressions.

And you want to be part of the Regex Illuminati.

Eric has just started a series of exercises on Learning Regular Expresions. In his words, this is

"The first in a series of exercises designed to teach you more about regular expressions, written by a guy who got partway through writing a regex book."

I find Eric's "Regular Expression Workbench" to be invaluable in working with regular expressions.  I think you'll enjoy (or at least benefit from!) this series of articles.

Pay close attention -- I'll be blogging a script in the near future that will turn your regular expression skills into Real Ultimate Power.  Regular expressions are cool; and by cool, I mean totally sweet.

Jeffrey Rocks Channel 9 Again

Since the first Channel 9 interviews were so popular, Jeffrey recently gave another interview on Channel 9.  With the intentions of talking about the PDC and the CLR, he ended up talking about nearly everything 🙂

Check it out! http://channel9.msdn.com/Showpost.aspx?postid=127819

Hex Dumper in Monad

Edit: See an updated version: Here

DESCRIPT.ION support in Monad — Part 3

Last time, we wrote code to refresh our description data while we navigate the file system.  We hadn't yet integrated it to our directory listings, so let's do that now.

Step 3: Customize the format.mshxml for directory listings

Our final step is to customize the formatting information that Monad uses to output a directory listing.  This is a fairly small change that mimics the way we output the “LastWriteTime” column.

First, open “FileSystem.format.mshxml” from your Monad installation directory.  This file tells Monad how convert the raw .Net File System objects into a form more suitable for humans.  The tweaking we are about to do illustrates one of the fundamental powers of Monad.  Data in the pipeline retains its full object-oriented fidelity until it reaches your screen.  This final stage of the pipeline is called “Formatting and Output.”

Now, before we customize our directory listing, let’s go over the different parts of the formatting file to see what it does.

The formatting files are XML.  The root <Configuration> node contains the following high-level configuration nodes: SelectionSets, Controls, and ViewDefinitions.

<Configuration>
    <SelectionSets>
        <SelectionSet>
           ...
        </SelectionSet>
    </SelectionSets>

    <!-- # GLOBAL CONTROL DEFINITIONS # -->
    <Controls>
        <Control>
           ...
        </Control>
    </Controls>
   
    <!-- # VIEW DEFINITIONS # -->
    <ViewDefinitions>
       <View>
          ...
       </View>
       <View>
          ...
       </View>
    </ViewDefinitions>
</Configuration>

The SelectionSets node lists a set of .Net types.  Later, the custom views from this file use this SelectionSet to help Monad determine the object types to which Monad should apply this formatting information.

The Controls node defines a custom header.  Later, custom views from this file use this header on groups of file system objects.

The Views form the real core of this file:

<ViewDefinitions>
   <View>
      <Name>children</Name>
      <ViewSelectedBy>
         ...
      </ViewSelectedBy>
      <GroupBy>
         ...
      </GroupBy>
      <TableControl>
         <TableHeaders>
            ...
         </TableHeaders>
         <TableRowEntries>
            ...
         </TableRowEntries>
      </TableControl>
   </View>
   <View>
      ...
      <ListControl>
         ...
      </ListControl>
   </View>
   <View>
      ...
      <WideControl>
         ...
      </WideControl>
   </View>
</ViewDefinitions>

We start by naming the view, and then use the ViewSelectedBy node to reference the list of .Net objects to which this view applies.  This reference is the name of one of the SelectionSets nodes earlier in the file.  Next, the GroupBy node tells Monad how to group the objects.  For example, a directory listing is grouped by its parent.  Each grouping gets a header, as defined by one of the Controls earlier in the file.  This is most evident when you do a recursive directory listing, as in “dir –rec.”

Then, we define three views: a TableControl, a ListControl, and a WideControl.  These correspond to the following statements, respectively:

 dir | format-table
 dir | format-list
 dir | format-wide

We’ll focus on (and customize) only the TableControl.

A TableControl has a header row, followed by data rows.  We define the header format with a sequence of TableColumnHeader nodes, then the row format with a sequence of TableRowEntry nodes.

Headers have a label, width, and alignment.  In this file, row entries reference either a direct property of the underlying object, or a ScriptBlock.  As illustrated by the LastWriteTime entry, ScriptBlocks reference the current pipeline object using the “$_” automatic variable.

When Monad loads its formatting data, the first file to define a view wins.  Ours will define the TableControl for DirectoryInfo objects first, so it will override the view that the default FileSystem.format.mshxml defines.

To make Monad look at our file before the default, add the following lines to your profile.msh:

## Update the formatting XML, overriding one of the filesystem
## views with the one we define in FileSystem.Description.Format.mshxml
$formatFile = "$(parse-path $profile)\FileSystem.Description.Format.mshxml"
update-FormatData -prependpath $formatFile

Now, copy the FileSystem.format.mshxml into the same directory as your profile, but call it FileSystem.Description.format.mshxml.  After you copy, change its read-only bit so that you can edit it.  Open the file, now at “My Documents”\msh\FileSystem.Description.format.mshxml.  We’ll place our customizations in this file, rather than hacking on the one that ships with Monad. 

For a test, go to a temporary directory and create a file with a very long name:

MSH:7 C:\Temp >"hi" > ("a"*200)
MSH:8 C:\Temp >dir

    Directory: FileSystem::C:\Temp

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---         10/9/2005   3:16 PM          4 aaaaaaaaaaaaaa
                                             aaaaaaaaaaaaaa
                                             aaaaaaaaaaaaaa
                                             aaaaaaaaaaaaaa
                                             aaaaaaaaaaaaaa
                                             aaaaaaaaaaaaaa
                                             aaaaaaaaaaaaaa
                                             aaaaaaaaaaaaaa
                                             aaaaaaaaaaaaaa
                                             aaaaaaaaaaaaaa
                                             aaaaaaaaaaaaaa
                                             aaaaaaaaaaaaaa
                                             aaaaaaaaaaaaaa
                                             aaaaaaaaaaaaaa
                                             aaaa

In your custom formatting file, delete the <wrap /> node, reload MSH, and then do another listing in that directory:

MSH:2 C:\Temp >dir

    Directory: FileSystem::C:\Temp

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---         10/9/2005   3:16 PM          4 aaaaaaaaaaa...

You can delete the long file now, as we’re done with it.

Let’s finally get our description column into directory listings.  In your custom formatting file, do the following:

  1. Delete the SelectionSets node, and all of its children.  The defaults will serve us just fine.
  2. Delete the Controls node, and all of its children.  The defaults will serve us just fine.
  3. Delete the View node that contains the ListControl.
  4. Delete the View node that contains the WideControl.
  5. Change the width of the LastWriteTime header to 20.
  6. After the last Table Column Header (labeled “<TableColumnHeader/>”,) create a new TableColumnHeader node:

    <TableColumnHeader>
       <Label>Description</Label>
       <Alignment>left</Alignment>
       <Width>30</Width>
    </TableColumnHeader>

  7. After the last Table Column Item (the one for “Name,”) create a new TableColumnItem node:

    <TableColumnItem>
       <ScriptBlock>
          ${lee.holmes.descriptions}[$_.Name]
       </ScriptBlock>
    </TableColumnItem>

  8. Your file should now look like this [FileSystem.Description.Format.mshxml]
  9. Create a file called “DESCRIPT.ION” in your profile directory.  Place the following in it:

    DESCRIPT.ION Descriptions file
    FileSystem.Description.Format.mshxml Formatting customizations
    profile.msh MSH custom profile

  10. Restart your MSH shell, navigate to your profile directory, and get a directory listing:

And there you have it.  DESCRIPT.ION support in Monad.

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

DESCRIPT.ION Support in Monad – Part 2

Last time, we wrote code to parse DESCRIPT.ION files -- and entertained the whole family while we were at it.  This was a manual step, though, so we need to figure out how to make it happen as we use the shell.

Step 2: Automatically refresh the hashtable

In order for this feature to be of any use, we need to ensure that the descriptions hashtable is updated before the directory output hits the screen.  Our primary choices are:

  • When we format the directory listing
  • When we list the directory items
  • When we change directories

The formatting files should primarily be used for formatting, not heavy procedural code, so that rules out the formatting option.  Directory listing (get-childitem) is really the best injection point, but has many commonly used parameters.  Because of that, shimming ourselves into that process is likely to be buggy and complex.  That rules out that option.  That leaves “when we change directories” as the best place to insert our code.

I personally use the ‘cd’ alias to change directories on my system.  Since we want to inject some code into this process, we’ll have to make a new function called ‘cd.’  This function changes to the new directory, and then refreshes the “lee.holmes.descriptions” hashtable with filenames and descriptions from the current directory.  One thing to note is that aliases take precedence over functions.  ‘cd’ is a built-in alias, so we’ll have to also delete the current ‘cd’ alias if it exists.  As a note, this will have a bug / missing feature.  By overriding only the cd functionality, we’ll only be able to get descriptions for items in the current directory.  Something like “dir c:\” won’t show us descriptions for files in other directories.

These are the last handful of lines in my profile-custom.msh:

## Remove the CD alias.  It will override our function if we don't.
if(test-path Alias:\cd) { remove-item Alias:\cd }

## Initialize our default 'descriptions' hashtable to hold
## a mapping of filename to description.
${GLOBAL:lee.holmes.descriptions} = @{}

## Change to a new location.  Once there, refresh the 'descriptions'
## hashtable with the filenames and descriptions of items in the
## current location.
function cd($location)
{
   ## We could use set-location here, but this will also let
   ## us use pop-location to go backwards in our directory navigation
   ## history.  
   push-location $location

   ## Clear the descriptions hashtable, and define the regular expression
   ## that parses a DESCRIPT.ION file
   ${GLOBAL:lee.holmes.descriptions} = @{}
   $descriptionRegex = "`"(?<file>[^`"]*)`" (?<text>.*)|(?<file>[^ ]*) (?<text>.*)"

   ## Go through each line in DESCRIPT.ION, if it exists, and parse out the
   ## filename / description.
   if(test-path descript.ion)
   {
      foreach($line in get-content descript.ion)
      {
         if($line -match $descriptionRegex)
         {
            ${lee.holmes.descriptions}[$matches.file] = $matches.text
         }
      }
   }
}

Stay tuned -- we'll wrap this up next time.

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Add File Descriptions to your Directory Listings – DESCRIPT.ION in Monad

In the BBS (Bulletin Board Service) days, one of the common past times was to connect to a board and browse long file listings in search of neat things to download.  When the names of files were restricted to 11 characters (8.3,) you didn’t have much to go by.  BBS software soon came up with a meta-file, called “descript.ion,” which mapped file names to longer, more verbose descriptions.  During a file listing, the BBS software read this file and displayed these detailed descriptions inline.  Fastidious SysOps (Sytem Operators, of which I was proudly one for many years) spent long hours painstakingly describing files for the benefit of both themselves, and their users.

It was enormously helpful, but is rarely supported by software in today’s new-fangled “long filename” era.  Let’s change that.

To add DESCRIPT.ION support to Monad, we’ll do the following:

  1. Write some code that parses the file, and stores the filename-to-description mapping into a form more suitable for later consumption.  We’ll store this into a global hashtable, called “lee.holmes.descriptions.”
  2. Automatically refresh the hashtable while we use the shell.  To do this, we’ll augment the “cd” function.
  3. Create a new FileSystem.format.mshxml snippet to add a “Description” column to our directory listings.

Step 1: Parse the file

The format of descript.ion files is very simple.  It is a text file, with lines in the format of:
<filename><single space><arbitrary description>

For example:

DESCRIPT.ION Mapping of file names to descriptions

It is now possible for filenames to have spaces in them, so the format has evolved a little to additionally include:

<“quoted filename”><single space><arbitrary description>

To save you the work, a regular expression that parses a line of this file format is:
$descriptionRegex = "`"(?<file>[^`"]*)`" (?<text>.*)|(?<file>[^ ]*) (?<text>.*)"

In English:
 Find a quote.  Put anything before the next quote into the “file” capture.  Find a space.  Put anything after the space into the “text” capture.
 or
 Put anything before the first space into the “file” capture.  Find a space.  Put anything after the space into the “text” capture. 

This regular expression uses named captures to make our further work a little easier:

MSH:11 D:\Lee\MSH >$descriptionRegex = "`"(?<file>[^`"]*)`" (?<text>.*)|(?<file>[^ ]*) (?<text>.*)"
MSH:12 D:\Lee\MSH >"DESCRIPT.ION Mapping of file names to descriptions" -match $descriptionRegex
True
MSH:13 D:\Lee\MSH >$matches

Key                            Value
---                            -----
text                           Mapping of file names to descriptions
file                           DESCRIPT.ION
0                              DESCRIPT.ION Mapping of file names to descriptions

MSH:14 D:\Lee\MSH >$matches.file
DESCRIPT.ION
MSH:15 D:\Lee\MSH >$matches.text
Mapping of file names to descriptions

So, the next step is to write some code that iterates over every line is DESCRIPT.ION (if it exists,) and store the file / text mappings to a hashtable:

   ${GLOBAL:lee.holmes.descriptions} = @{}
   $descriptionRegex = "`"(?<file>[^`"]*)`" (?<text>.*)|(?<file>[^ ]*) (?<text>.*)"

   if(test-path descript.ion)
   {
      foreach($line in get-content descript.ion)
      {
         if($line -match $descriptionRegex)
         {
            ${lee.holmes.descriptions}[$matches.file] = $matches.text
         }
      }
   }

I’m doing two weird things with my variable here.  First, I’m naming it “lee.holmes.descriptions.”  I do this because it’s a shell-wide variable, and I don’t want to 

  • be interfered with
  • interfere with others

Second, I put the variable name in curly braces.  Why?  Monad has very permissive rules for variable names, and curly braces save you from having to escape every special character – such as the period.

Try this, and become an instant life-of-the-party:

  • Type a dollar sign
  • Type an open curly-brace
  • Sit on your keyboard, have your cat walk on it, and try to type your name with your forehead
  • Type a closed curly brace, followed by “ = $true”
  • Dir variable:

Anyways, back to business.  We'll continue with step 2 next time.

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Monad documentation pack now available

I mentioned to many of you during the PDC that we’d be working hard to make our printed hands-on labs available for download.  As Adam points out, get ‘em while they’re still hot!

 

We’ve even gone a bit further, and made the “Getting Started” guide and a few quick references available in the documentation pack, available via the Microsoft download center.

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Helpful overview of Monad

Although all of you are probably Monad junkies already, I thought I'd point out Thomas Lee's very helpful Introduction to Monad article.  Great stuff!

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]