Skip to content

Specify how consumers should handle parse errors #4

@jonathanKingston

Description

@jonathanKingston

With a new version of TAP brings the ability to make stricter the behaviour around parsing and consuming TAP.

I think that (quick notes please feed back):

  • Implementations should fail hard and fast

    • We may likely want to specify a fatal error type behaviour which consumers can choose to do
    • Failing hard should be the default behaviour
    • Implementations may however choose to ignore fatal errors but they still should be reported with the same importance as other test cases
    • Fatal errors should have set meanings codes which are human readable
  • Implementations should only support the version they were designed for, when encountering a new version of TAP, the test should fail

  • Reporters should also have a set of fatal errors that can be appended to their output which means something went fatally wrong with the tests but perhaps not within the tests themselves (Lets say a network error on the socket)

    • This could be something as simple as:
     TAP version 14
     1..10
     ok 1 - reason
     not ok fatal
    

    and/or interpretations might be expected to provide an out message at the end to confirm all was ok

     TAP version 14
     1..1
     ok 1 - reason
     ok complete
    

    These could be treated as less fatal than a bail out - however we could specify more around that instead.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions