-
Notifications
You must be signed in to change notification settings - Fork 8
Description
With a new version of TAP brings the ability to make stricter the behaviour around parsing and consuming TAP.
I think that (quick notes please feed back):
-
Implementations should fail hard and fast
- We may likely want to specify a fatal error type behaviour which consumers can choose to do
- Failing hard should be the default behaviour
- Implementations may however choose to ignore fatal errors but they still should be reported with the same importance as other test cases
- Fatal errors should have set meanings codes which are human readable
-
Implementations should only support the version they were designed for, when encountering a new version of TAP, the test should fail
-
Reporters should also have a set of fatal errors that can be appended to their output which means something went fatally wrong with the tests but perhaps not within the tests themselves (Lets say a network error on the socket)
- This could be something as simple as:
TAP version 14 1..10 ok 1 - reason not ok fataland/or interpretations might be expected to provide an out message at the end to confirm all was ok
TAP version 14 1..1 ok 1 - reason ok completeThese could be treated as less fatal than a bail out - however we could specify more around that instead.