PowerShell Cookbook

Search

Categories

 

On this page

Scripting with the Microsoft Shell -- Now on Script Center
Accessing Performance Counters in PowerShell
burn-console Part III: The Most Efficient MSH Script
burn-console Digression: An MSH Script Profiler Part 2
A Legal Take on the JC Penny Price Gouging
burn-console digression: an MSH script profiler
burn-console.msh Part II - A working implementation
Cracking Safes With Thermal Imaging
Burn-Console: A Fire-Effect Demo in MSH
First Monad Book Soon on Sale

Archive

Blogroll

Disclaimer
I work for Microsoft.

The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.

RSS 2.0 | Atom 1.0 | CDF

Send mail to the author(s) E-mail

Total Posts: 218
This Year: 18
This Month: 0
This Week: 0
Comments: 529

Sign In

 Tuesday, December 13, 2005
Tuesday, December 13, 2005 4:35:12 PM (Pacific Standard Time, UTC-08:00) ( )

The scripting guys continue to impart their magic scripting touch, now with a great big dose of MSH.

Their "Scripting with the Microsoft Shell" Script Center went live recently, and it's already chock-full of sample scripts -- most applying the power of WMI interop.

Their first article covers Accessing WMI From the Microsoft Shell -- check it out!

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Comments [0] | | # 
 Friday, December 09, 2005
Friday, December 09, 2005 7:30:45 PM (Pacific Standard Time, UTC-08:00) ( )

The question came up on the newsgroup on how to monitor CPU usage in PowerShell.  I wrote a script to demonstrate this some time ago with the intent to write about it – so now is probably an ideal time.

The following poll-process script retrieves the process name, main window title, processor usage, disk activity, and working set.  It continually refreshes the display to give you a task manager-like experience.

One advantage it offers over Task Manager is the disk activity column.  Since total disk activity is actually measured by several separate perf counters, it uses a heuristic to combine them into a single number.

Now, when you hear something drilling holes in your hard drive, you can find out what it is.

## poll-process.ps1
## Continuously display a process list, sorted
## by the desired criteria.
##
## usage: poll-process [sortCriteria] [pollInterval]
##
## sortCriteria must be one of "Id", "ProcessName", "MainWindowTitle", 
##                              "Processor", "Disk", or "WorkingSet"
## pollInterval is specified in milliseconds
##
param(
    [string] $sortCriteria = "Processor",
    [int] $pollInterval = 750
    )

function main
{
    ## Store the performance counters we need
    ## for the CPU, and Disk I/O numbers
    $cpuPerfCounters = @{}
    $ioOtherOpsPerfCounters = @{}
    $ioOtherBytesPerfCounters = @{}
    $ioDataOpsPerfCounters = @{}
    $ioDataBytesPerfCounters = @{}
    $processes = $null
    $lastPoll = get-date
    
    $lastSnapshotCount = 0
    $lastWindowHeight = 0
    
    ## The coordinates to which we position the output
    $coords = new-object System.Management.Automation.Host.Coordinates 0,0
    clear-host

    while(-not $host.UI.RawUI.KeyAvailable)
    {
        ## Set the cursor position, get the processes, and store
        ## the time of the snapshot
        $host.UI.RawUI.CursorPosition = $coords
        $processes = get-process | sort Id

        ## Go through all of the processes we captured
        foreach($process in $processes)
        {
            ## Get the disk activity, based on I/O Perf Counters,
            ## for the process in question.  Then, add it as a note.
            $activity = get-diskActivity $process
            $process | add-member NoteProperty Disk $activity

            $cpuPercent = get-cpuPercent $process
            $process | add-member NoteProperty Processor $cpuPercent
         }

        $windowHeight = $host.Ui.RawUi.WindowSize.Height
        ## Since clear-host makes the screen flash, we only do so when
        ## we have fewer processes than we did the last time
        if(($processes.Count -lt $lastSnapshotCount) -or `
            (-not ($lastWindowHeight -eq $windowHeight)))
        { 
            clear-host 
        }
        
        ## Tailor the length of the list to the height of the 
        ## window.  If the window is to short, show no output.
        if($windowHeight -le 7
        { 
            $output = $null 
        }
        else 
        {
            $output = $processes | sort -desc $sortCriteria | `
                select-object -First ($windowHeight - 7)
        }
        
        ## Display the results
        $output | format-table Id,ProcessName,MainWindowTitle,Processor,Disk,WorkingSet
        
        if($processes.Count -gt ($windowHeight - 7))
        {
            $notDisplayed = ($processes.Count - $output.Count)
            "[ $notDisplayed process(es) not shown ]"
        }
        
        $lastSnapshotCount = $processes.Count
        $lastWindowHeight = $windowHeight
        
        ## Sleep for their desired amount of elapsed time,
        ## adjusted for how much time we've actually spent working.
        $elapsed = [int] (get-date).Subtract($lastPoll).TotalMilliseconds
        $lastPoll = get-date
        if($pollInterval -gt $elapsed)
        { 
            start-sleep -m ($pollInterval - $elapsed)
        }
    }
}

## As a heuristic, gets the total IO and Data operations per second, and
## returns their sum.
function get-diskActivity (
    $process = $(throw "Please specify a process for which to get disk usage.")
    )
{
    $processName = get-processName $process
    
    ## We store the performance counter objects in a hashtable.  If we don't,
    ## then they fail to return any information for a few seconds.
    if(-not $ioOtherOpsPerfCounters[$processName])
    {
        $ioOtherOpsPerfCounters[$processName] = `
            new-object System.Diagnostics.PerformanceCounter `
                "Process","IO Other Operations/sec",$processName
    }
    if(-not $ioOtherBytesPerfCounters[$processName])
    {
        $ioOtherBytesPerfCounters[$processName] = `
            new-object System.Diagnostics.PerformanceCounter `
                "Process","IO Other Bytes/sec",$processName
    }
    if(-not $ioDataOpsPerfCounters[$processName])
    {
        $ioDataOpsPerfCounters[$processName] = `
            new-object System.Diagnostics.PerformanceCounter `
                "Process","IO Data Operations/sec",$processName
    }
    if(-not $ioDataBytesPerfCounters[$processName])
    {
        $ioDataBytesPerfCounters[$processName] = `
            new-object System.Diagnostics.PerformanceCounter `
                "Process","IO Data Bytes/sec",$processName
    }


    ## If a process exits between the time we capture the processes and now,
    ## then we will be unable to get its NextValue().  This trap simply
    ## catches the error and continues.
    trap { continue; }

    ## Get the performance counter values
    $ioOther = (100 * $ioOtherOpsPerfCounters[$processName].NextValue()) + `
        ($ioOtherBytesPerfCounters[$processName].NextValue())
    $ioData = (100 * $ioDataOpsPerfCounters[$processName].NextValue()) + `
        ($ioDataBytesPerfCounters[$processName].NextValue())
    
    return [int] ($ioOther + $ioData)    
}

## Get the percentage of time spent by a process.
## Note: this is multiproc "unaware."  We need to divide the
## result by the number of processors.
function get-cpuPercent (
    $process = $(throw "Please specify a process for which to get CPU usage.")
    )
{
    $processName = get-processName $process
    
    ## We store the performance counter objects in a hashtable.  If we don't,
    ## then they fail to return any information for a few seconds.
    if(-not $cpuPerfCounters[$processName])
    {
        $cpuPerfCounters[$processName] = `
            new-object System.Diagnostics.PerformanceCounter `
                "Process","% Processor Time",$processName
    }

    ## If a process exits between the time we capture the processes and now,
    ## then we will be unable to get its NextValue().  This trap simply
    ## catches the error and continues.
    trap { continue; }

    ## Get the performance counter values
    $cpuTime = ($cpuPerfCounters[$processName].NextValue() / $env:NUMBER_OF_PROCESSORS)
    return [int] $cpuTime
}

## Performance counters are keyed by process name.  However,
## processes may share the same name, so duplicates are named
## <process>#1, <process>#2, etc.
function get-processName (
    $process = $(throw "Please specify a process for which to get the name.")
    )
{
    ## If a process exits between the time we capture the processes and now,
    ## then we will be unable to get its information.  This simply
    ## ignores the error.
    $errorActionPreference = "SilentlyContinue"

    $processName = $process.ProcessName
    $localProcesses = get-process -ProcessName $processName | sort Id
    
    if(@($localProcesses).Count -gt 1)
    {
        ## Determine where this one sits in the list
        $processNumber = -1
        for($counter = 0; $counter -lt $localProcesses.Count; $counter++)
        {
            if($localProcesses[$counter].Id -eq $process.Id) { break }
        }
        
        ## Append its unique identifier, if required
        $processName += "#$counter"
    }
    
    return $processName
}

. main

[Update: MOW wrote a piece about Perf Counters as well, based on the same newsgroup thread: Getting Performance Monitor Info from Monad]

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

[Edit: Script updated for PowerShell RC2]

Comments [5] | | # 
 Tuesday, December 06, 2005
Wednesday, December 07, 2005 6:49:19 AM (Pacific Standard Time, UTC-08:00) ( )

Now that we have our script profiler up and running, we instrument our script a little to mark regions we are concerned about.  You can download that starting script here: [burn-console-1.profiler.msh]   After running the script profiler, we get the performance breakdown:

Breakdown by line:
----------------------------
 15%: Line  123 -             if($colour -lt 20) { $colour -= 1 }
 14%: Line  122 -             if($colour -le 70) { $colour -= 3 }
 11%: Line  124 -             if($colour -lt 0) { $colour = 0 }
 10%: Line  128 -             $tempWorkingBuffer[$baseOffset -
  8%: Line  117 -             $colour /= 4.0
  7%: Line  121 -             if($colour -gt 70) { $colour -= 1 }
  6%: Line  154 -             $nextScreen[$row, $column] = `
  6%: Line  113 -             $colour = $screenBuffer[$baseOffset]
  6%: Line  115 -             $colour += $screenBuffer[$baseOffset + 1]
  6%: Line  116 -             $colour += $screenBuffer[$baseOffset +
  5%: Line  114 -             $colour += $screenBuffer[$baseOffset - 1]
  5%: Line  109 -             $baseOffset = ($windowWidth * $row) +
  0%: Line   90 -         if($random.NextDouble() -ge 0.20)
  0%: Line  152 -         for($column = 0; $column -lt $windowWidth;
  0%: Line   94 -             $screenBuffer[($windowHeight - 2) *

Breakdown by marked regions:
----------------------------
  6%: updateScreen
  0%: startFireLastRow
 93%: propigate
  0%: Unmarked
  0%: main


It looks like we’re spending 93% of our time propagating the fire.  When we investigate how we’re spending the time, none of the hot spots are individually egregious.  Since we’re in such a tight loop, we’re spending most of the time comparing colours.  We can be a little smarter with our ‘if’ statements, attempting to minimize the number of comparisons and variable assignments.  Before optimization, we have 4 checks per loop, and (a weighted average of) just over 1 assignment per loop.  If we write it as below, we reduce that to (a weighted average of) 1 check per loop, and (a weighted average even closer to) 1 assignment per loop.

if($colour -gt 70)

    $colour -= 1 
}
else
{
    $colour -= 3
    
    if($colour -lt 1)
    { 
        $colour = 0 
    }
    elseif($colour -lt 20)
    { 
        $colour -= 1 
    }
}

This brings us up to 0.57 frames per second.  A great improvement, but we’re obviously not done yet.  We run the profiler again, and see that the hotspots have moved:

Breakdown by line:
----------------------------
 13%: Line  141 -             $tempWorkingBuffer[$baseOffset -
  9%: Line  113 -             $colour = $screenBuffer[$baseOffset]
  9%: Line  117 -             $colour /= 4.0
  9%: Line  115 -             $colour += $screenBuffer[$baseOffset + 1]
  9%: Line  131 -                     $colour = 0
  8%: Line  129 -                 if($colour -lt 1)
  8%: Line  127 -                 $colour -= 3
  7%: Line  121 -             if($colour -gt 70)
  7%: Line  116 -             $colour += $screenBuffer[$baseOffset +
  7%: Line  114 -             $colour += $screenBuffer[$baseOffset - 1]
  5%: Line  109 -             $baseOffset = ($windowWidth * $row) +
  5%: Line  167 -             $nextScreen[$row, $column] = `
  3%: Line  244 -             $bufferCell = `
  1%: Line  240 -             $character =
  0%: Line  252 -             $paletteIndex++
  0%: Line  241 -             $fgColour =
  0%: Line  242 -             $bgColour =
  0%: Line  165 -         for($column = 0; $column -lt $windowWidth;

Breakdown by marked region:
----------------------------
  5%: updateScreen
  0%: startFireLastRow
 90%: propigate
  0%: Unmarked
  5%: main

There’s not too much we can do, but there are a few lines representing access and manipulation of the $colour variable as we compute the average.  So we’ll put those into one line:

$colour = ($screenBuffer[$baseOffset] +
    $screenBuffer[$baseOffset - 1] +
    $screenBuffer[$baseOffset + 1] +
    $screenBuffer[$baseOffset + $windowWidth]) / 4.0

The frame rate barely changes, so let’s see what the new hot spots are:

Breakdown by line:
----------------------------
 19%: Line  140 -             $tempWorkingBuffer[$baseOffset -
 13%: Line  113 -             $colour = ($screenBuffer[$baseOffset] +
 12%: Line  109 -             $baseOffset = ($windowWidth * $row) +
 11%: Line  126 -                 $colour -= 3
 11%: Line  128 -                 if($colour -lt 1)
 11%: Line  120 -             if($colour -gt 70)
 10%: Line  130 -                     $colour = 0
  9%: Line  166 -             $nextScreen[$row, $column] = `
  1%: Line  240 -             $fgColour =
  1%: Line  243 -             $bufferCell = `
  0%: Line  239 -             $character =
  0%: Line  122 -                 $colour -= 1
  0%: Line  251 -             $paletteIndex++
  0%: Line  106 -         for($column = 1; $column -lt ($windowWidth - 1);
  0%: Line  219 -                 if($apparentBrightness -gt

Breakdown by marked region:
----------------------------
  9%: updateScreen
  0%: startFireLastRow
 87%: propigate
  0%: Unmarked
  4%: main

There’s really not much left for us to optimize.  We seem to be spending a lot of time computing the $baseOffset variable, though.  Although we can’t improve the efficiency of the statement, we can execute it less frequently.   Rather than compute it from scratch each time via multiplication, we’ll simply increment it once per column.

We can continue with this iterative refinement, giving an end result of about 0.68 frames per second – 70% faster than when we started.  You can download the optimized version here: [burn-console-2.working.msh].  The problems have now shifted to things we have no control over:

Breakdown by line:
----------------------------
 20%: Line  171 -             $nextScreen[$row, $column] = `
 20%: Line  111 -             $colour = ($screenBuffer[$baseOffset] +
 18%: Line  141 -             $tempWorkingBuffer[$baseOffset -
 17%: Line  144 -             $baseOffset++
 17%: Line  118 -             if($colour -gt 0)
  1%: Line  120 -                 if($colour -gt 70)
  1%: Line  248 -             $bufferCell = `
  1%: Line  126 -                     $colour -= 3
  1%: Line  122 -                     $colour -= 1
  0%: Line  134 -                         $colour -= 1
  0%: Line  128 -                     if($colour -lt 1)
  0%: Line  246 -             $bgColour =
  0%: Line   94 -             $screenBuffer[($windowHeight - 2) *
  0%: Line  130 -                         $colour = 0
  0%: Line  162 -     $nextScreen =

Breakdown by marked region:
----------------------------
 20%: updateScreen
  1%: startFireLastRow
 77%: propigate
  0%: Unmarked
  3%: main

To solve this problem, we’ll delve into a trick from the Demo Scene.  Did you ever use inline assembler in C?  We’ll do something similar, but with inline C# in MSH.

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Comments [0] | | # 
 Sunday, December 04, 2005
Monday, December 05, 2005 7:07:42 AM (Pacific Standard Time, UTC-08:00) ( )

In the last article, we talked about the theory behind a sampling profiler.  Let’s go over the code to implement one.  We’ll use this to guide our future optimizations.  You can also download the script here: profile-transcript.msh.

################################################################################
##
## profile-transcript.msh
##
## Computes the performance characteristics of a script, based on the transcript
## of it running at trace level 1.
##
## To profile a script:
##    1) Turn on MSH script tracing in the window that will run the script: 
##       set-mshdebug –trace 1
##    2) Turn on the transcript for the window that will run the script:
##       start-transcript
##       (Note the filename that Monad provides as the logging destination.)
##    3) Type in the script name, but don't actually start it.
##    4) Open another MSH window, and navigate to the directory holding 
##       this script.  Type in '.\profile-transcript.msh <path-to-transcript>',
##       replacing <path-to-transcript> with the path given in step 2.  Don't
##       press <Enter> yet.
##    5) Switch to the profiled script window, and start the script.
##       Switch to the window containing profile-transcript, and press <Enter>
##    6) Wait until your profiled script exits, or has run long enough to be
##       representative of its work.  To be statistically accurate, your script
##       should run for at least ten seconds.
##    7) Switch to the window running profile-transcript.msh, and press a key.
##    8) Switch to the window holding your profiled script, and type:
##       stop-transcript
##    9) Delete the transcript.
##
## Note: You can profile regions of code (ie: functions) rather than just lines
## by placing the following call at the start of the region:
##       write-debug "ENTER <region_name>"
## and the following call and the end of the region:
##       write-debug "EXIT"
## This is implemented to account exclusively for the time spent in that 
## region, and does not include time spent in regions contained within the
## region.  For example, if FunctionA calls FunctionB, and you've surrounded
## each by region markers, the statistics for FunctionA will not include the
## statistics for FunctionB.
##
################################################################################

param($logFilePath = $(throw "Please specify a path to the transcript log file."))

function Main
{
    ## Run the actual profiling of the script.  $uniqueLines gets
    ## the mapping of line number to actual script content.
    ## $samples gets a hashtable mapping line number to the number of times
    ## we observed the script running that line.
    $uniqueLines = @{}
    $samples = GetSamples $uniqueLines
    
    "Breakdown by line:"
    "----------------------------"

    ## Create a new hash table that flips the $samples hashtable -- 
    ## one that maps the number of times sampled to the line sampled.
    ## Also, figure out how many samples we got altogether.
    $counts = @{}
    $totalSamples = 0;
    foreach($item in $samples.Keys) 
    { 
       $counts[$samples[$item]] = $item 
       $totalSamples += $samples[$item]
    }

    ## Go through the flipped hashtable, in descending order of number of 
    ## samples.  As we do so, output the number of samples as a percentage of
    ## the total samples.  This gives us the percentage of the time our script
    ## spent executing that line.
    foreach($count in ($counts.Keys | sort -desc))
    {
       $line = $counts[$count]
       $percentage = "{0:#0}" -f ($count * 100 / $totalSamples)
       "{0,3}%: Line {1,4} -{2}" -f $percentage,$line,
          $uniqueLines[$line]
    }

    ## Go through the transcript log to figure out which lines are part of any
    ## marked regions.  This returns a hastable that maps region names to
    ## the lines they contain.
    ""
    "Breakdown by marked regions:"
    "----------------------------"
    $functionMembers = GenerateFunctionMembers
    
    ## For each region name, cycle through the lines in the region.  As we
    ## cycle through the lines, sum up the time spent on those lines and output
    ## the total.
    foreach($key in $functionMembers.Keys)
    {
        $totalTime = 0
        foreach($line in $functionMembers[$key])
        {
            $totalTime += ($samples[$line] * 100 / $totalSamples)
        }
        
        $percentage = "{0:#0}" -f $totalTime
        "{0,3}%: {1}" -f $percentage,$key
    }
}

## Run the actual profiling of the script.  $uniqueLines gets
## the mapping of line number to actual script content.
## Return a hashtable mapping line number to the number of times
## we observed the script running that line.
function GetSamples($uniqueLines)
{
    ## Open the log file.  We use the .Net file I/O, so that we keep monitoring
    ## just the end of the file.  Otherwise, we would make our timing innacurate
    ## as we scan the entire length of the file every time.
    $logStream = [System.IO.File]::Open($logFilePath, "Open""Read""ReadWrite")
    $logReader = new-object System.IO.StreamReader $logStream

    $random = new-object Random
    $samples = @{}

    $lastCounted = $null
    
    ## Gather statistics until the user presses a key.
    while(-not $host.UI.RawUI.KeyAvailable)
    {
       ## We sleep a slightly random amount of time.  If we sleep a constant
       ## amount of time, we run the very real risk of improperly sampling 
       ## scripts that exhibit periodic behaviour.
       $sleepTime = [int] ($random.NextDouble() * 100.0)
       start-sleep -m $sleepTime

       ## Get any content produced by the transcript since our last poll.
       ## From that poll, extract the last DEBUG statement (which is the last
       ## line executed.)
       $rest = $logReader.ReadToEnd()
       $lastEntryIndex = $rest.LastIndexOf("DEBUG: ")

       ## If we didn't get a new line, then the script is still working on the
       ## last line that we captured.
       if($lastEntryIndex -lt 0
       { 
          if($lastCounted) { $samples[$lastCounted] ++ }
          continue
       }
       
       ## Extract the debug line.
       $lastEntryFinish = $rest.IndexOf("\n", $lastEntryIndex)
       if($lastEntryFinish -eq -1) { $lastEntryFinish = $rest.length }

       $scriptLine = $rest.Substring($lastEntryIndex, ($lastEntryFinish - $lastEntryIndex)).Trim()
       if($scriptLine -match 'DEBUG:[ \t]*([0-9]*)\+(.*)')
       {
           ## Pull out the line number from the line
           $last = $matches[1]
           
           $lastCounted = $last
           $samples[$last] ++
           
           ## Pull out the actual script line that matches the line number
           $uniqueLines[$last] = $matches[2]
       }

       ## Discard anything that's buffered during this poll, and start waiting
       ## again
       $logReader.DiscardBufferedData()
    }

    ## Clean up
    $logStream.Close()
    $logReader.Close()
    
    return $samples
}

## Go through the transcript log to figure out which lines are part of any
## marked regions.  This returns a hastable that maps region names to
## the lines they contain.
function GenerateFunctionMembers
{
    ## Create a stack that represents the callstack.  That way, if a marked
    ## region contains another marked region, we attribute the statistics
    ## appropriately.
    $callstack = new-object System.Collections.Stack
    $currentFunction = "Unmarked"
    $callstack.Push($currentFunction)

    $functionMembers = @{}

    ## Go through each line in the transcript file, from the beginning
    foreach($line in (get-content $logFilePath))
    {
        ## Check if we're entering a monitor block
        ## If so, store that we're in that function, and push it onto
        ## the callstack.
        if($line -match 'write-debug "ENTER (.*)"')
        {
            $currentFunction = $matches[1]
            $callstack.Push($currentFunction)
        }
        ## Check if we're exiting a monitor block
        ## If so, clear the "current function" from the callstack,
        ## and store the new "current function" onto the callstack.
        elseif($line -match 'write-debug "EXIT"')
        {
            [void] $callstack.Pop()
            $currentFunction = $callstack.Peek()
        }
        ## Otherwise, this is just a line with some code.
        ## Add the line number as a member of the "current function"
        else
        {
            if($line -match 'DEBUG:[ \t]*([0-9]*)\+')
            {
                ## Create the arraylist if it's not initialized
                if(-not $functionMembers[$currentFunction])
                {
                    $functionMembers[$currentFunction] = new-object `
                        System.Collections.ArrayList
                }
                
                ## Add the current line to the ArrayList
                if(-not $functionMembers[$currentFunction].Contains($matches[1]))
                {
                    [void] $functionMembers[$currentFunction].Add($matches[1])
                }
            }
        }
    }
    
    return $functionMembers
}

. Main

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Comments [0] | | # 
 Thursday, December 01, 2005
Friday, December 02, 2005 5:36:46 AM (Pacific Standard Time, UTC-08:00) ( )

AKA: Welcome Don McGowan to the Party :)  Don works in the Law and Corporate Affairs organization at Microsoft, and has always been a bright and illuminating voice in our internal discussions.

Case in point -- a bunch of people internally are pretty upset that JC Penny is gouging customers by charging $800 for a vanilla Xbox  Premium system.  Where most people speculate, Don lays down this awesome summary of the situation:

There's plenty of people out there asking why Microsoft doesn't "do something about this".  Well, one reason is that we can't.

It’s great that he’s started blogging.  GrokLaw used to be a good source of information for non-lawyers, but they really rose to fame with their coverage of the SCO case.  Because of that, they’ve really started pandering pretty heavily to the OSDN audience.

[Edit: Fixed a typo]

Comments [2] | | # 
 Wednesday, November 30, 2005
Thursday, December 01, 2005 7:01:35 AM (Pacific Standard Time, UTC-08:00) ( )

We now have implemented a working fire effect in a form that’s extremely easy to read.  The intention of the code is clear, our functions are nice and modular, and the algorithms are clear.  However, we need to deal with some performance issues.  For that, we'll take a small digression.

At this point, if performance is of utmost importance, we might consider simply porting our script to a compiled C# cmdlet.  However, we’ll continue this in MSH, applying highly-targeted optimizations to bring the performance up to an acceptable level.

When tackling performance problems, the first rule is to measure the problem.  I can’t stress this highly enough.  It stuns (and saddens) me when I see people parade around on an optimization binge without ever enlisting the help of a profiler, or other measurement tool.  Unless you can guide your optimization efforts with hard performance data, you are almost certainly directing them to the wrong spots.  Random cute performance hacks will quickly turn your code to garbage, often with no appreciable performance gain.  I’ve done my fair share of micro-optimization, but it’s always been under the guidance of hard data to support it. 

Now, let’s investigate sources of our performance problems by measuring them.

The easiest way is to probably comment out function calls to see their effect on our refresh rate.  If we comment out the calls to both updateBuffer and updateScreen in our Main function, then we get about 8,000 frames per second.  If we comment out only the call to updateBuffer, we get about 3.7 frames per second.  If we comment out only the call to updateScreen, we get about 0.54 frames per second.  We can continue to comment out chunks of code, slowly determining what causes the most drastic effect on our script’s performance.

However, that’s not very elegant.  It’s slow, and inefficient.  We should really rely on a profiler to provide us the information we need.  But MSH doesn’t have a script profiler.

Is that going to stop us?  Not in the least.  In fact, it will barely slow us down.  With a surprisingly small amount of bubble gum and handy twine, we’ll write our own – in MSH, of course.

There are two approaches one can take when writing a profiler.  The first is an instrumenting profiler, and the second is a sampling profiler.

Instrumenting profilers inject code at the beginning and end of every call to measure the time spent in that call.  The idea is fairly straight-forward, but the technique is fairly invasive.  The instrumentation process itself can skew the performance profile of a program, so the profiler needs to calibrate itself to account for that.  In addition, robust code instrumentation is extremely difficult.  Binary instrumentation is much easier, so nearly all instrumenting profilers take this approach.

Sampling profilers semi-randomly peek into a program to see what statement it is currently executing.  If we see the program executing a certain statement half of the times we check, we can be fairly confident that our program is spending about 50% of its time executing that statement.  Likewise, if we see the program executing a certain statement 8 out of the 50 times we check, we can be fairly confident that our program is spending about 16% of its time executing that statement.  When we apply this idea to a much larger sample set, we can fairly accurately map the execution time of each individual statement.

We’re going to primarily implement a sampling profiler for several reasons:

  • There is no concept of an MSH Script Binary.  If we implement an instrumenting profiler, we have only the very perilous path of mangling the script itself.
  • Monad has several bits of infrastructure available that make a sampling profiler surprisingly easy to implement.

So how do we peek in on the MSH engine to see which line of script it’s currently executing?  Very indirectly. 

  1. Turn on MSH script tracing (at level 1,) as outlined as part of Jon Newman’s excellent series of posts.
    MSH:116 C:\Temp >set-mshdebug –trace 1
  2. Turn on file logging for our session:
    MSH:117 C:\Temp >start-transcript
  3. Start our script

Once the script starts, we get entries similar to this in the transcript log for our session:

(…)
DEBUG:  113+             $colour = $screenBuffer[$baseOffset]
DEBUG:  114+             $colour += $screenBuffer[$baseOffset - 1]
DEBUG:  115+             $colour += $screenBuffer[$baseOffset + 1]
DEBUG:  116+             $colour += $screenBuffer[$baseOffset +
$windowWidth]
(…)

Now, we can see what line of script the engine is currently executing by looking at the last bit of output that made it to the log.

Here is an example of the output – we’ll go over the code next time:

Breakdown by line:
----------------------------
 15%: Line  123 -             if($colour -lt 20) { $colour -= 1 }
 14%: Line  122 -             if($colour -le 70) { $colour -= 3 }
 11%: Line  124 -             if($colour -lt 0) { $colour = 0 }
 10%: Line  128 -             $tempWorkingBuffer[$baseOffset -
  8%: Line  117 -             $colour /= 4.0
  7%: Line  121 -             if($colour -gt 70) { $colour -= 1 }
  6%: Line  154 -             $nextScreen[$row, $column] = `
  6%: Line  113 -             $colour = $screenBuffer[$baseOffset]
  6%: Line  115 -             $colour += $screenBuffer[$baseOffset + 1]
  6%: Line  116 -             $colour += $screenBuffer[$baseOffset +
  5%: Line  114 -             $colour += $screenBuffer[$baseOffset - 1]
  5%: Line  109 -             $baseOffset = ($windowWidth * $row) +
  0%: Line   90 -         if($random.NextDouble() -ge 0.20)
  0%: Line  152 -         for($column = 0; $column -lt $windowWidth;
  0%: Line   94 -             $screenBuffer[($windowHeight - 2) *

Breakdown by marked regions:
----------------------------
  6%: updateScreen
  0%: startFireLastRow
 93%: propigate
  0%: Unmarked
  0%: main

[Edit: Monad has now been renamed to Windows PowerShell. This script or discussion may require slight adjustments before it applies directly to newer builds.]

Comments [1] | | # 
 Friday, November 25, 2005
Saturday, November 26, 2005 2:04:00 AM (Pacific Standard Time, UTC-08:00) ( )

Now that we’ve generated the palette, the most complex part of the algorithm is actually behind us.  The remaining code implements the fire algorithm, and is fairly simple:

  1. Generate fire on the bottom row of the screen.
  2. Move the already existing fire upwards on the screen.  Rather than just move every cell of fire upwards, though, we take into account the fact that heat is always affected by nearby heat.  So the fire that we move upwards will actually be the average heat from its four neighboring cells.

Another technique that we implement is called “double buffering.”  If we were to update each cell on the screen as we compute it, the graceful effect of frame-by-frame animation would be destroyed.  It looks like the progressive text of a teleprompter, as compared to the subtle scrolling of credits at the end of a movie.  To combat this, we draw each new frame onto a pretend screen (which is one buffer.)  As soon as we’re finished our calculations, we bulk-update the real screen (which is the second buffer.)

The updated implementation is below, and also here: burn-console-1.working.msh.  The two new functions are updateBuffer and updateScreen, along with a little bit of new code in the main function.  The performance is atrocious, though.  I get about 0.4 frames per second on my computer, so we’ll work on improving that over the next two posts.  In fact, we’ll be in the mid 30s for frames per second by the time we’re done.

BTW – for a bit more visual realism, change your console font to either the 4x6 raster font, or the 5pt Lucida Console font.

###############################################################################
## burn-console.msh
##
## Create a fire effect in MSH, using the Console text buffer as the rendering
## surface.
##
## Great overview of the fire effect algorithm here: 
## http://freespace.virgin.net/hugo.elias/models/m_fire.htm
##
###############################################################################

function main
{
    ## Rather than a simple red fire, we'll introduce oranges and yellows
    ## by including Yellow as one of the base colours
    $colours = "Yellow","Red","DarkRed","Black"
    
    ## The four characters that we use to dither with, along with the 
    ## percentage of the foreground colour that they show
    $dithering = "█","▓","▒","░"
    $ditherFactor = 1,0.75,0.5,0.25
    
    ## Hold the palette.  We actually store each entry as a BufferCell,
    ## since we need to retain a foreground colour, background colour,
    ## and dithering character.
    $palette = new-object System.Management.Automation.Host.BufferCell[] 256
    
    ## Resize the console to 70, 61 so we have a consistent buffer
    ## size for performance comparison.
    $bufferSize = new-object System.Management.Automation.Host.Size 70,61
    $host.UI.RawUI.WindowSize = $bufferSize

    ## Retrieve some commonly used dimensions
    $windowWidth = $host.UI.RawUI.WindowSize.Width
    $windowHeight = $host.UI.RawUI.WindowSize.Height
    $origin = `
        new-object System.Management.Automation.Host.Coordinates 0,0
    $dimensions = `
        new-object System.Management.Automation.Host.Rectangle `
            0,0,$windowWidth,$windowHeight
    
    ## Create our random number generator
    $random = new-object Random
    $workingBuffer = new-object System.Int32[] ($windowHeight * $windowWidth)
    $screenBuffer = new-object System.Int32[] ($windowHeight * $windowWidth)
    
    clear-host
    
    ## Generate the palette
    generatePalette
    # displayPalette
    # return;

    ## Update the buffer, then update the screen until the user presses a key.  
    ## Keep track of the total time and frames generated to let us display
    ## performance statistics.
    $frameCount = 0
    $totalTime = time-expression {
        while(! $host.UI.RawUI.KeyAvailable)
        {
            updateBuffer
            updateScreen
            $frameCount++
        }
    }
    
    ## Clean up and exit
    $host.UI.RawUI.ForegroundColor = "Gray"
    $host.UI.RawUI.BackgroundColor = "Black"
    
    write-host
    write-host "$($frameCount / $totalTime.TotalSeconds) frames per second."
}

## Update a back-buffer to hold all of the information we want to display on
## the screen.  To do this, we first re-generate the fire pixels on the bottom 
## row.  With that done, we visit every pixel in the screen buffer, and figure
## out the average heat of its neighbors.  Once we have that average, we move
## that average heat one pixel up.
function updateBuffer
{
    ## Start fire on the last row of the screen buffer
    for($column = 0; $column -lt $windowWidth; $column++)
    {
        ## There is an 80% chance that a pixel on the bottom row will
        ## start new fire.
        if($random.NextDouble() -ge 0.20)
        {
            ## The chosen pixel gets a random amount of heat.  This gives
            ## us a lot of nice colour variation.
            $screenBuffer[($windowHeight - 2) * ($windowWidth) + $column] = `
                [int] ($random.NextDouble() * 255)
        }
    }
    
    $tempWorkingBuffer = $screenBuffer.Clone()
    
    ## Propigate the fire
    for($row = 1; $row -lt ($windowHeight - 1); $row++)
    {
        for($column = 1; $column -lt ($windowWidth - 1); $column++)
        {
            ## BaseOffset is the location of the current pixel
            $baseOffset = ($windowWidth * $row) + $column
    
            ## Get the average colour from the four pixels surrounding
            ## the current pixel
            $colour = $screenBuffer[$baseOffset]
            $colour += $screenBuffer[$baseOffset - 1]
            $colour += $screenBuffer[$baseOffset + 1]
            $colour += $screenBuffer[$baseOffset + $windowWidth]
            $colour /= 4.0

            ## Cool it off a little.  We apply uneven cooling, otherwise
            ## the cool dark red tends to stretch up for too long.
            if($colour -gt 70) { $colour -= 1 }
            if($colour -le 70) { $colour -= 3 }
            if($colour -lt 20) { $colour -= 1 }
            if($colour -lt 0) { $colour = 0 }

            ## Store the result into the previous row -- that is, one buffer 
            ## cell up.
            $tempWorkingBuffer[$baseOffset - $windowWidth] = `
                [int] [Math]::Floor($colour)
        }
    }
    
    $SCRIPT:workingBuffer = $tempWorkingBuffer
}

## Take the contents of our working buffer and blit it to the screen
## We do this in one highly-efficent step (the SetBufferContents) so that
## users don't see each individial pixel get updated.
function updateScreen
{
    ## Create a working buffer to hold the next screen that we want to
    ## create.
    $nextScreen = $host.UI.RawUI.GetBufferContents($dimensions)
    
    ## Go through our working buffer (that holds our next animation frame)
    ## and place its contents into the buffer that we will soon blast into
    ## the real RawUI
    for($row = 0; $row -lt $windowHeight; $row++)
    {
        for($column = 0; $column -lt $windowWidth; $column++)
        {
            $nextScreen[$row, $column] = `
                $palette[$workingBuffer[($row * $windowWidth) + $column]]
        }
    }
    
    ## Bulk update the RawUI's buffer with the contents of our next screen
    $host.UI.RawUI.SetBufferContents($origin, $nextScreen)
    
    ## And finally update our representation of the screen buffer to hold
    ## what actually is on the screen
    $SCRIPT:screenBuffer = $workingBuffer.Clone()
}

## Generates a palette of 256 colours.  We create every combination of 
## foreground colour, background colour, and dithering character, and then
## order them by their visual intensity.
##
## The visual intensity of a colour can be expressed by the NTSC luminance 
## formula.  That formula depicts the apparent brightness of a colour based on 
## our eyes' sensitivity to different wavelengths that compose that colour.
## http://en.wikipedia.org/wiki/Luminance_%28video%29
function generatePalette
{
    ## The apparent intensities of our four primary colours.
    ## However, the formula under-represents the intensity of our straight
    ## red colour, so we artificially inflate it.
    $luminances = 225.93,106.245,38.272,0
    $apparentBrightnesses = @{}

    ## Cycle through each foreground, background, and dither character
    ## combination.  For each combination, find the apparent intensity of the 
    ## foreground, and the apparent intensity of the background.  Finally,
    ## weight the contribution of each based on how much of each colour the
    ## dithering character shows.
    ## This provides an intensity range between zero and some maximum.
    ## For each apparent intensity, we store the colours and characters
    ## that create that intensity.
    $maxBrightness = 0
    for($fgColour = 0; $fgColour -lt $colours.Count; $fgColour++)