Add test metadata to each request

[filip26]

Hi,
because there is no way to get a reason nor an origin request from generated reports, it would be helpful to add test metadata to options or as a separate object.

e.g.

options: {  checks: [..],  testNumber:  ... ,  testType: "Interop", test params}

The syntax does not matter, Just anything what would allow a developer to target log search query more finely than going manually through all requests trying to figuring out if the 400 is correct or not.

[BigBlueHat]

Are you wanting these sent in with each request? Not sure adding to the request payload would be a good idea...

[filip26]

Yes, the goal is to help an implementer to find a request associated with a test case execution. You already have an option object in the payload, you can add another one, e.g. meta, but the name nor a location is not important as long as it's part of payload (a search in payload is usually supported, search in HTTP headers is not in my case) with no issue keeping it backward compatible.

[filip26]

A side note, helping an implementer to pass a suite is critical, and this is the minimum you can do, considering the reports give nothing but status. I see tendention to prioritize the suite operator perspective over an implementers` needs, which is destructive at the end of the day.

An implementer has no motivation to spend hours trying to figure out why a test has failed in a report, especially in a case of interop tests where only the test operator has the data.

[aljones15]

Just so you know suite.log is published along side each test run and the newer https://github.com/w3c/vc-di-ecdsa-test-suite/blob/gh-pages/index.json index.json give out the report in json form where you can in turn see the actual error object, so I think what we should do here is record more metadata, and figure out a way of linking to the index.json file so you can see the raw json from the test run.

[filip26]

thanks, but the log contains no useful data allowing to reproduce the test, a full request body is needed - generally I'm constantly asking (for more than two years) for a way how to get the origin request, easily if possible and you keep refusing any constructive solution I bring up, even it's almost no job on your side but life safer on impleterer's side.

Hm...this looks like a hostile approach towards implementers which is not helpful nor constructive.