Keep the error location when the error comes from inside Pester, and name the assertion

[nohwnd]

Fix #2980
Fix #2977

Two changes, they belong together. 1, 2, 3 | Should-Be 1, 2, 3 said "this assertion" and printed no location at all. The location part is the wider one, any error thrown from inside a Pester function lost its whole stack trace, Mock for a command that does not exist had the same problem.

demo.tests.ps1:

Describe 'Stack trace' {
    It 'collection on -Expected' {
        1, 2, 3 | Should-Be 1, 2, 3
    }
    It 'collection on -Expected, assertion that calls the guard directly' {
        1 | Should-BeSame 1, 2
    }
    It 'mock of a command that does not exist' {
        Mock Get-NoSuchCommand { }
    }
    It 'assertion fails inside a helper function' {
        function Test-Thing ($Value) {
            $Value | Should-Be 2
        }
        Test-Thing -Value 1
    }
}

Before, on 6.1.0 with the default StackTraceVerbosity:

  [-] collection on -Expected 27ms
   ArgumentException: You provided a collection to the -Expected parameter. Using a collection on the -Expected side is not allowed by this assertion, because it leads to unexpected behavior. To compare collections use Should-BeCollection, or a more specialized collection assertion such as Should-Any or Should-All.
  [-] collection on -Expected, assertion that calls the guard directly 2ms
   ArgumentException: You provided a collection to the -Expected parameter. Using a collection on the -Expected side is not allowed by this assertion, because it leads to unexpected behavior. To compare collections use Should-BeCollection, or a more specialized collection assertion such as Should-Any or Should-All.
  [-] mock of a command that does not exist 13ms
   CommandNotFoundException: Could not find Command Get-NoSuchCommand
  [-] assertion fails inside a helper function 32ms
   Expected [int] 2, but got [int] 1.
   at $Value | Should-Be 2, demo.tests.ps1:13

Three of the four have no location. The fourth points at the assertion on line 13, but not at line 15 that called the function, so in a bigger suite you do not know which of the calls to Test-Thing failed.

After:

  [-] collection on -Expected 36ms
   ArgumentException: You provided a collection to the -Expected parameter. Using a collection on the -Expected side is not allowed by Should-Be, because it leads to unexpected behavior. To compare collections use Should-BeCollection, or a more specialized collection assertion such as Should-Any or Should-All.
   at <ScriptBlock>, demo.tests.ps1:3
  [-] collection on -Expected, assertion that calls the guard directly 3ms
   ArgumentException: You provided a collection to the -Expected parameter. Using a collection on the -Expected side is not allowed by Should-BeSame, because it leads to unexpected behavior. To compare collections use Should-BeCollection, or a more specialized collection assertion such as Should-Any or Should-All.
   at <ScriptBlock>, demo.tests.ps1:6
  [-] mock of a command that does not exist 19ms
   CommandNotFoundException: Could not find Command Get-NoSuchCommand
   at <ScriptBlock>, demo.tests.ps1:9
  [-] assertion fails inside a helper function 43ms
   Expected [int] 2, but got [int] 1.
   at $Value | Should-Be 2, demo.tests.ps1:13
   at <ScriptBlock>, demo.tests.ps1:15

Every line number points at the line in the file above that caused the failure.

Why?

ConvertTo-FailureLines walked the trace and broke on the first frame that was inside Pester, with Should<End>|Invoke-Assertion as the only exemption:

foreach ($line in $traceLines) {
    if ($line -match $isPesterFunction -and $line -notmatch $isShould) {
        break
    }
    ...
}

That exemption is Pester 4 and 5. This is the trace of the first test, StackTraceVerbosity = Full:

at Ensure-ExpectedIsNotCollection, .../Pester.psm1:6816
at EnsureScalar, .../Pester.psm1:7370
at Should-Be, .../Pester.psm1:8654
at <ScriptBlock>, demo.tests.ps1:3            <- the line the user needs
at <ScriptBlock>, .../Pester.psm1:2068
at Invoke-ScriptBlock, .../Pester.psm1:2215
at Invoke-TestItem, .../Pester.psm1:1283
... 21 more frames of the runtime

The first frame is Pester and it is not Should<End>, so we broke on it and the trace came out empty, one line above the one we wanted. StackTraceVerbosity = Full was the only way to see it, and then you get all 28 frames.

Assertion failures kept their line only because of the special case above the loop, which builds it from TargetObject when FullyQualifiedErrorId is PesterAssertionFailed. Everything else lost the location.

Adding Ensure-ExpectedIsNotCollection to the exemption list fixes Should-Be and breaks again on the next internal helper we write. The rule is positional now instead. Skip Pester frames until the first frame that comes from user code, stop on the first Pester frame after that. Applied to the trace above it skips the first three, keeps demo.tests.ps1:3, and stops on Pester.psm1:2068.

Since the trace does not show Should-Be anymore either, the message has to name the assertion. Ensure-ExpectedIsNotCollection takes a name, EnsureScalar passes $this.Caller, which is the $PSCmdlet of the assertion, so every assertion that guards a scalar names itself with nothing to do per assertion. Should-BeSame and Should-NotBeSame pass their own name because they call the guard directly and not through $assert.

Behavior change

The fourth test is the one to look at. An assertion that fails inside a helper function now prints the call as a second line, which is what the comment in the loop already asked for:

we want to be able to see that we invoked the assertion inside of function a

When the assertion sits directly in the It nothing changes. The frame from the trace is then the same file and line as the line built from TargetObject, and it is skipped so the location is not printed twice.

Tests

Output.Tests.ps1 gets four: an error thrown from inside Pester keeps the line from the test file, the frames inside Pester are dropped, an assertion that fails in a function produces both the assertion line and the call, and the same file and line is not printed twice when the assertion is directly in the file. Ensure-ExpectedIsNotCollection.Tests.ps1 gets one for the name in the message.

The Format-ErrorMessage tests had the line number of the 1 / 0 written into them as 385, adding tests above them moved it. They read it from the error record now.

Full suite passes locally, 2896 tests.
🤖