Archives for the Month of January, 2009

Scripting WinDbg with PowerShell

A while back, Roberto Farah published a script library to help control WinDbg through PowerShell. I’ve been using WinDbg for more debugging lately, and decided (after following one to many object references by hand) that I needed to script my investigations.

PowerDbg is definitely helpful – Roberto has tons of great scripts published that help analyze all kinds of interesting data. It also made me think of an alternative approach that works around some of the problem areas – PowerDbg uses SendKeys, window focusing, and used WinDbg logging as the communication mechanism. After some investigation, I thought that automation of the command-line version (cdb.exe) through its input and output streams might be easier and more efficient – which indeed it was.

You set up a remote from the Windbg instance you want to control, and then the WinDbg module (below) connects to the remote session (by manipulating standard in / standard out of cdb.exe) to manage its output.

Windows PowerShell V2
Copyright (C) 2008 Microsoft Corporation. All rights reserved.
PS C:\Users\leeholm> Import-Module Windbg
PS C:\Users\leeholm> Connect-Windbg "tcp:Port=10456,Server=SERVER"
PS C:\Users\leeholm> Invoke-WindbgCommand .symfix
PS C:\Users\leeholm> Invoke-WindbgCommand .reload
0:009> Reloading current modules
................................................................
...........
PS C:\Users\leeholm> Invoke-WindbgCommand k
0:009> ChildEBP RetAddr
0460fbe8 76ebd0d0 ntdll!DbgBreakPoint [d:\foo.c @ 206]
0460fc18 76fc4911 ntdll!DbgUiRemoteBreakin+0x3c [d:\bar.c @ 298]
0460fc24 76e6e4b6 kernel32!BaseThreadInitThunk+0xe [d:\baz.c @ 66]
0460fc64 76e6e489 ntdll!__RtlUserThreadStart+0x23 [d:\bing.c @ 2740]
0460fc7c 00000000 ntdll!_RtlUserThreadStart+0x1b [d:\bing.c @ 2672]
PS C:\Users\leeholm>

Here’s an example function to demonstrate its use:

001
002
003
004
005
006
007
008
009
010
011
012
013
014
015
016
017
018
019
020
021
022
023
024
025
026
027
028
029
030
031
032
033
034
035
036
037
038
039
040
041
042
043
044
045
046
047
048
049
050
051
052
053
054
################################################################################
##
## Resolve-PsInvocationInfo.ps1
## Resolves the command and script information from an active PowerShell
## process
##
################################################################################
Import-Module windbg
Connect-Windbg "tcp:Port=10456,Server=SERVER"

function Resolve-Member($address, $member)
{
    $output = Invoke-WindbgCommand !dumpobj $address
   
    if(-not $member)
    {
        $output
    }
    else
    {
        $currentMember = ($member -split "\.")[0]
        $remaining = $member.Remove(0, $currentMember.Length).TrimStart('.')
       
        Write-Verbose "Looking for $currentMember on $address"
       
        $output = $output | 
            Select-String ("(instance|shared).*$currentMember" + '$') |
            Select-String -NotMatch 00000000

        if($output)
        {
            $columns = -split $output
            $address = $columns[6]
           
            Resolve-Member $address $remaining
        }
        else
        {
            Write-Error "Could not resolve $currentMember on $address"
        }
    }
}

$null = Invoke-WindbgCommand .loadby sos mscorwks
$output = Invoke-WindbgCommand ("!dumpheap -type " +
    "System.Management.Automation.InvocationInfo -short")
foreach($line in $output)
{
    Resolve-Member $line commandInfo.name
    Resolve-Member $line scriptToken._script
    Resolve-Member $line scriptToken._line
}

Disconnect-Windbg

And the module itself (WinDbg.psm1, place in a “WinDbg” folder in your modules folder):

(Edit: 05/13/09 - Updated to support local kernel debugging)

001
002
003
004
005
006
007
008
009
010
011
012
013
014
015
016
017
018
019
020
021
022
023
024
025
026
027
028
029
030
031
032
033
034
035
036
037
038
039
040
041
042
043
044
045
046
047
048
049
050
051
052
053
054
055
056
057
058
059
060
061
062
063
064
065
066
067
068
069
070
071
072
073
074
075
076
077
078
079
080
081
082
083
084
085
086
087
088
089
090
091
092
093
094
095
096
097
098
099
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
################################################################################
##
## WinDbg.psm1
## Automate WinDbg with PowerShell scripting
##
## To use:
## In WinDbg, set up a server: .server tcp:Port=10456
##
## In PowerShell
##
## Import-Module windbg
## Connect-Windbg -remote "tcp:Port=10456,Server=<server>"
## Other remote connection strings / protocols work, too.
##
## For local kernel debugging:
##
## Import-Module Windbg -ArgumentList 'C:\<path>\kd.exe'
## Connect-Windbg -ArgumentList "-kl"
## Invoke-WindbgCommand "!process"
##
## Cleaning up:
## Disconnect-Windbg
##
################################################################################

param($cdbPath = "C:\Program Files\Debugging Tools for Windows\cdb.exe")

$SCRIPT:windbgProcess = $null
$SCRIPT:currentConnection = $null

## Connect to a windbg remote session
function Connect-Windbg
{
    param(
        [Parameter(ParameterSetName = "Remote", Mandatory = $true)]
        $remote,

        [Parameter(ParameterSetName = "ArgumentList", Mandatory = $true)]
        $argumentList
        
        )
        
    if($SCRIPT:currentConnection -and ($SCRIPT:currentConnection -ne $remote))
    {
        throw "Already connected to $remote. Use Disconnect-Windbg, " + 
            "then connect to another instance."
    }
    
    ## Launch cdb.exe, the command-line version of WinDbg.
    ## Take control of its input and output streams, which we'll use
    ## to capture commands and their output.
    if(-not $SCRIPT:currentConnection)
    {
        $processStartInfo = New-Object System.Diagnostics.ProcessStartInfo
        $processStartInfo.FileName = $cdbPath
        $processStartInfo.WorkingDirectory = (Get-Location).Path

        if($remote)
        {
                $processStartInfo.Arguments = "-remote $remote"
        }
        else
        {
                $processStartInfo.Arguments = $argumentList
        }
        
        $processStartInfo.UseShellExecute = $false
        $processStartInfo.RedirectStandardInput = $true
        $processStartInfo.RedirectStandardOutput = $true 

        $SCRIPT:windbgProcess = 
            [System.Diagnostics.Process]::Start($processStartInfo)
        $SCRIPT:currentConnection = $remote

        if(-not $SCRIPT:windbgProcess)
        {
            return
        }
        
        ## Ignore the stuff that was in the session before we
        ## connected
        $null = Receive-WindbgOutput
    }
}

## Disconnect from the session
function Disconnect-Windbg
{
    if($SCRIPT:windbgProcess -and (-not $SCRIPT:windbgProcess.HasExited))
    {
        $SCRIPT:windbgProcess.StandardOutput.Close()
        $SCRIPT:windbgProcess.StandardInput.Close()

        
$SCRIPT:windbgProcess.Kill()
    }
    
    $SCRIPT:currentConnection = $null
    $SCRIPT:windbgProcess = $null
}

## Invoke a command in the connected WinDbg session, and return
## its output
function Invoke-WindbgCommand
{
    if(-not $SCRIPT:windbgProcess)
    {
        throw "Not connected. Use Connect-Windbg to connect to an " +
            "instance of WinDbg."
    }
    
    $SCRIPT:windbgProcess.StandardInput.WriteLine("$args")
    Receive-WindbgOutput
}

## Retrieve pending output from the connected WinDbg session
function Receive-WindbgOutput
{
    ## Add a special tag so that we know the end of the command
    ## response
    $sent = "PSWINDBG_COMPLETE_{0:o}" -f [DateTime]::Now        
    $SCRIPT:windbgProcess.StandardInput.WriteLine(".echo $sent")
    
    $received = New-Object System.Text.StringBuilder

    ## Wait for the response to end
    while($received.ToString().IndexOf($sent) -lt 0)
    {
        $null = $SCRIPT:windbgProcess.StandardOutput.EndOfStream
        while($SCRIPT:windbgProcess.StandardOutput.Peek() -ge 0)
        {
            $null = $received.Append(
                $SCRIPT:windbgProcess.StandardOutput.Read(), 1)
        }
    }
    
    ## Split it into lines, and return everything but the new
    ## prompt, and the "end response" tag
    $output = $received.ToString().Split("`r`n")
    if($output.Length -gt 2)
    {
        $output[0..($output.Length - 3)]
    }
}

Export-ModuleMember -Function Connect-Windbg
Export-ModuleMember -Function Disconnect-Windbg
Export-ModuleMember -Function Invoke-WindbgCommand

$MyInvocation.MyCommand.ScriptBlock.Module.OnRemove = { Disconnect-Windbg }

PowerShell P/Invoke Walkthrough

In version 1 of PowerShell, it was possible to access Win32 APIs in one of two ways: by generating a dynamic assembly on the fly (you wouldn’t really do this for one-off calls, but would probably do it in a script that makes it easier to invoke Win32 APIs,) or by looking up the P/Invoke definition for that API call, and compiling in-line C# to access it.

These are both good approaches, but we wanted to use the Add-Type cmdlet to really nail the language interop scenarios for PowerShell V2. Add-Type offers four basic modes of operation:

PS:13 > Get-Command Add-Type | Select -Expand ParameterSets | Select Name

Name
----
FromSource
FromMember
FromPath
FromAssemblyName

These are:

  • FromSource: Compile some C# (or other language) code that completely defines a type. This is useful when you want to define an entire class, its methods, namespace, etc. You supply the actual code as the value to the –TypeDefinition parameter, usually through a variable.
  • FromPath: Compile from a file on disk, or load the types from an assembly at that location.
  • FromAssemblyName: Load an assembly from the GAC by its shorter name. This is is not the same as [Reflection.Assembly]::LoadWithPartialName, since that introduces your script to many subtle breaking changes. Instead, PowerShell maintains a large mapping table that converts the shorter name you type a strongly-named assembly reference.
  • FromMember: Generates a type out of a member definition (or set of them.) For example, if you specify only a method definition, PowerShell automatically generates the wrapper class for you. This parameter set is explicitly designed to easily support P/Invoke calls.

Now, how do you use the FromMember parameter set to call a Win32 API? Let’s see. First, imagine that you want to access sections of an INI file. This previous blog post gives an example for V1.

PowerShell doesn’t have a native way to manage INI files, and neither does the .NET Framework. However, the Win32 API does, through a call to GetPrivateProfileString. The .NET framework lets you access Win32 functions through a technique called P/Invoke (Platform Invocation Services.) Most calls boil down to a simple “P/Invoke definition,” which usually takes a lot of trial and error. However, a great community has grown around these definitions, resulting in an enormous resource called P/Invoke .NET. The P/Invoke Interop Assistant is an awesome tool from the CLR team that also generates these definitions.

First, we’ll create a script, Get-PrivateProfileString.ps1. It’s a template for now:

## Get-PrivateProfileString.ps1
param(
    $file,
    $category,
    $key)

$null

So first, we visit P/Invoke .NET and search for GetPrivateProfileString:

image

Click into the definition, and we see the C# signature:

image

Copy that signature as a here-string into our script. Notice that we’ve added public to the declaration. The signatures on PInvoke.NET assume that you’ll call the method from within the C# class that defines it. We’ll be calling it from scripts, so we need to change its visibility.

## Get-PrivateProfileString.ps1
param(
    $file,
    $category,
    $key)

$signature = @'
[DllImport("kernel32.dll")]
public static extern uint GetPrivateProfileString(
    string lpAppName,
    string lpKeyName,
    string lpDefault,
    StringBuilder lpReturnedString,
    uint nSize,
    string lpFileName);
'@

$null

Now, we add the call to Add-Type. This signature becomes the building block for a new class, so we only need to give it a name. To prevent its name from colliding with other classes with the same name, we also put it in a namespace. The name of our script is a good choice:

## Get-PrivateProfileString.ps1
param(

    $file,
    $category,
    $key)

$signature = @'
[DllImport("kernel32.dll")]
public static extern uint GetPrivateProfileString(
    string lpAppName,
    string lpKeyName,
    string lpDefault,
    StringBuilder lpReturnedString,
    uint nSize,
    string lpFileName);
'@

$type = Add-Type -MemberDefinition $signature `
    -Name Win32Utils -Namespace GetPrivateProfileString `
    -PassThru

$null

When we try to run this script, though, we get an error:

The type or namespace name 'StringBuilder' could not be found (are you missing a
using directive or an assembly reference?)
c:\Temp\obozeqo1.0.cs(12) :    string lpDefault,
c:\Temp\obozeqo1.0.cs(13) : >>>    StringBuilder lpReturnedString,
c:\Temp\obozeqo1.0.cs(14) :    uint nSize,

Indeed we are. StringBuilder is defined in the System.Text namespace, but the using directive goes at the top of the program by the class definition. Since we’re letting PowerShell define the type for us, we can either rename it to System.Text.StringBuilder, or add a –UsingNamespace parameter. (Aside: PowerShell adds references to the System and System.Runtime.InteropServices namespaces by default.) Let’s do the latter:

## Get-PrivateProfileString.ps1
param(
    $file,
    $category,
    $key)

$signature = @'
[DllImport("kernel32.dll")]
public static extern uint GetPrivateProfileString(
    string lpAppName,
    string lpKeyName,
    string lpDefault,
    StringBuilder lpReturnedString,
    uint nSize,
    string lpFileName);
'@

$type = Add-Type -MemberDefinition $signature `
    -Name Win32Utils -Namespace GetPrivateProfileString `
    -Using System.Text -PassThru

$null

Now, we can plug in all of the necessary parameters. The GetPrivateProfileString puts its output in a StringBuilder, so we’ll have to feed it one, and return its contents:

## Get-PrivateProfileString.ps1
param(
    $file,
    $category,
    $key)

$signature = @'
[DllImport("kernel32.dll")]
public static extern uint GetPrivateProfileString(
    string lpAppName,
    string lpKeyName,
    string lpDefault,
    StringBuilder lpReturnedString,
    uint nSize,
    string lpFileName);
'@

$type = Add-Type -MemberDefinition $signature `
    -Name Win32Utils -Namespace GetPrivateProfileString `
    -Using System.Text -PassThru
   
$builder = New-Object System.Text.StringBuilder 1024
$type::GetPrivateProfileString($category,
    $key, "", $builder, $builder.Capacity, $file)
   
$builder.ToString()

So now we have it. With just a few lines of code, we’ve defined and invoked a Win32 API call.

[C:\Users\leeholm]
PS:1 > Get-PrivateProfileString c:\windows\system32\tcpmon.ini "<Generic Network Card>" Name
Generic Network Card

DateTime Casts are Language Primitives

One thing that sometimes comes as a surprise (most recently as a comment on the PowerShell blog) is that DateTime casts are always parsed in the en-US DateTime format:

Casting to a DateTime object doesn't respect the current culture of the thread. For example:

> [Threading.Thread]::CurrentThread.CurrentCulture

LCID  Name

----  ----

2057  en-GB

> [DateTime] "01/02/03"

02 January 2003

> # Note US date interpretation.

This is an explicit design goal.

To prevent subtle internationalization issues from popping into your scripts, PowerShell treats "[DateTime] '11/26/2007'" (a date constant) like a language feature – just as it does [Double] 10.5 (a numeric constant.) Not all cultures use the decimal point as the fractions separator, but programming languages standardize on it. Not all cultures use the en-US DateTime format, resulting in millions of internationalization bugs when people don’t consider the impact of having their software run in those cultures.

Imagine a random script that you find on the internet. It was probably written on a system that uses a different culture than you or I. For the script to run correctly our machines, the author needs to write in a culturally-sensitive way:

1) The approach taken by traditional software development for internationalization:

$date = New-Object System.DateTime 2007,11,27,17,56,00

or (an imagined PowerShell syntax)

$date = [DateTime->pt-PT] "terça-feira, 27 de Novembro de 2007 9:34:50"

2) An approach that assumes a specific culture, which is the approach PowerShell takes:

$date = [DateTime] "11/27/2007 5:56:00 pm"

Approach #2 is much more user-friendly, as it minimizes what you need to remember. In approach #1, you always need to remember to specify your locale when you create the DateTime. In approach #2, you always need to remember to use the en-US format (as you already do when writing floating-point constants.)

The additional benefit of assuming a specific culture is that you are protected from people that do NOT write the script in a culturally-sensitive way. While some advanced scripters might be able to understand approach #1, most scripters (and even professional software developers) do not. History has shown us that software not explicitly TESTED for internationalization fails spectacularly. I don't mean "written for international markets," I mean TESTED for them.

When an admin bashes together a script because their hair is on fire, and then wants to share it later -- testing for other regions is not high on the list of things to do.

Should I Refinance? PowerShell the Financial Advisor

When it comes to personal finance, research and reading sometimes can’t answer all of your questions. Especially when it comes to projecting the future, many subtle factors are simply too difficult to easily describe in rules of guidance. Because of this, they tend to be ignored.

With interest rates sinking, now’s a good time to evaluate a mortgage refinance. The online calculators help you figure out if it’s right for you, but they all miss important facts.

Let’s pretend that you refinance 5 years into your current 30-year mortgage (that started at 108k), plan to move in 5 years, and your house value doesn’t change. You’ve now got a 100k mortgage at 6%, and want to re-finance at 5%.

The immediate benefit pointed out is that your monthly payment will drop, and that the new interest-heavy payment will be even more tax deductible.

At 6%, you’re paying $647 per month.
At 5%, you’re paying $585 per month.

However,

  • All of the re-finance quotes and calculators put your remaining balance into a new 30-year mortgage. That pushes the quoted monthly figures even lower than $585 – to be expected if you spread 25 years of payments over 30 years! If you want to take the approach of not building principal to reduce monthly payments, you might as well go for an interest-only loan, or rent.
  • Even if you put yourself into a 25-year mortgage instead, loan amortization over the first 5 years of that new loan may be much more interest-heavy than the mid years of your current loan. That means a reduction in your principal payments, which is a direct loss of money when you sell your house in 5 years. In this specific scenario (you are only 5 years into a mortgage,) you’re actually building principal faster in the new loan.
    • Your current loan will build $10,117 in principal in this time
    • A new loan will build $11,252 in principal in this time

As we can see from these numbers, this is not a negative impact of the scenario I laid out above. However, it will impact more mature loans.

  • Re-financing means paying the closing costs all-over again. If you assume 10k out-of-pocket, you can instead apply that as a balloon payment and drop your monthly payment to $579 per month – lower than the rate you would get from a re-finance.
  • If you don’t want to pay the closing costs out of pocket and instead roll them into your mortgage, that gives you a $110k loan @ $643 per month.

These calculations are obviously sensitive to real numbers, but they describe core decision points that are not covered in most literature.

Since we have a powerful scripting language at our disposal, however, we are in a position to let a computer model help guide our decisions.

We’ve done this before (Traditional or Roth 401(k)?,) so let’s do it again.

## Assumptions:
##     You will invest the closing fee if you don't need to spend it
##     You will bank any improvement in monthly fees

$currentLoan = 100000
$currentRate = 6.125 / 100
$newRate  = 4.25 / 100
$origination = 1.00 / 100
$closing = 5000
$monthsTotal = 25 * 12

## Scenarios:
$scenarios = @(
    ## 0: Current loan. Invest closing fee instead @ 3%
    @{ LoanAmount = $currentLoan; Interest = $currentRate;
       Saved = $closing * [Math]::Pow(1.03, 10) },
    
    ## 1: Put closing fee against principal on current loan
    @{ LoanAmount = $currentLoan - $closing; Interest = $currentRate;
       Saved = 0 },
    
    ## 2: Refi: Add origination fee onto mortage, and spend

    ## closing fee to refinance
    @{ LoanAmount = $currentLoan + ($currentLoan * $origination);
       Interest = $newRate; Saved = 0 },
    
    ## 3: Refi: Add origination and closing fees onto mortage,
    ## and invest closing fee
    @{ LoanAmount = $currentLoan +
           ($currentLoan * $origination) + $closing;
       Interest = $newRate; Saved = $closing * [Math]::Pow(1.03, 10) }
)

## Calculate the monthly payments
$maxPayment = 0
$counter = 0
foreach($scenario in $scenarios)
{
    $scenario.InterestPerPeriod = $scenario.Interest / 12

    $scenario.MonthlyPayment = $scenario.LoanAmount * 
        $scenario.InterestPerPeriod *
        [Math]::Pow(1 + $scenario.InterestPerPeriod, $monthsTotal) /
        ([Math]::Pow(1 + $scenario.InterestPerPeriod, $monthsTotal) - 1)

    "Monthly payment $counter is: " + $scenario.MonthlyPayment
    if($scenario.MonthlyPayment -gt $maxPayment)
    {
        $maxPayment = $scenario.MonthlyPayment
    }
    
    $scenario.Balance = $scenario.LoanAmount
    $counter++
}

## Apply the monthly payments. This applies standard
## amortization, which pays off the interest that built up
## that month before applying payments to principal.
$results = @()
for($month = 1; $month -le $monthsTotal; $month++)
{
    $outputObject = New-Object PsObject
    Add-Member -in $outputObject NoteProperty Month $month

    ## Calculate the principal, interest, and total savings
    ## for each scenario
    $counter = 0
    foreach($scenario in $scenarios)
    {
        $scenario.Balance *= (1 + $scenario.InterestPerPeriod)
        $interestPayment = $scenario.InterestPerPeriod * $scenario.Balance
        $principal = $scenario.MonthlyPayment - $interestPayment
        
        # Put the following into the bank / savings:
        # the principal payment, the difference between this payment
        # and the most expensive, and the tax-writeoff aspect of the
        # interest
        $scenario.Saved += $principal
        $scenario.Saved += $maxPayment - $scenario.MonthlyPayment
        $scenario.Saved += $interestPayment * 0.3
        
        Add-Member -in $outputObject `

            NoteProperty Interest_$counter ("{0:C}" -f $interestPayment)
        Add-Member -in $outputObject `
            NoteProperty Principal_$counter ("{0:C}" -f $principal)
        Add-Member -in $outputObject `
            NoteProperty Balance_$counter ("{0:C}" -f $scenario.Balance)
        Add-Member -in $outputObject `
            NoteProperty Saved_$counter ("{0:C}" -f $scenario.Saved)
        
        
        $scenario.Balance -= $scenario.MonthlyPayment
        $counter++
    }
    
    $results += $outputObject
}

$results| Format-Table Month,S*,B*
#$results| Format-Table Month,S*,I*,P*,B*

This gives you a table of information that lets you easily determine which scenario is best for you, and how long the refinance will take to pay for itself:

________________________________________________________________________________________________________________________
PS E:\Lee> E:\Lee\RefinanceCalculator_Example.ps1
Monthly payment 0 is: 651.963964563932
Monthly payment 1 is: 619.365766335736
Monthly payment 2 is: 547.155481990353
Monthly payment 3 is: 574.24238703938

        Month Saved_0       Saved_1       Saved_2      Saved_3      Balance_0    Balance_1    Balance_2    Balance_3  
        ----- -------       -------       -------      -------      ---------    ---------    ---------    ---------  
            1 $7,012.43     $310.80       $400.68      $7,107.82    $100,510.42  $95,484.90   $101,357.71  $106,375.42
            2 $7,305.79     $622.09       $801.83      $7,496.56    $100,368.15  $95,349.74   $101,167.59  $106,175.89
            3 $7,599.66     $933.86       $1,203.46    $7,885.79    $100,225.15  $95,213.89   $100,976.80  $105,975.65
            4 $7,894.04     $1,246.12     $1,605.56    $8,275.52    $100,081.43  $95,077.35   $100,785.33  $105,774.70

If it doesn’t cover a scenario you’re wondering about, well that’s OK too! That’s the beauty of scripts – you can change them.