Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch most of the CTS to ask for all features #4178

Open
greggman opened this issue Feb 6, 2025 · 0 comments
Open

Switch most of the CTS to ask for all features #4178

greggman opened this issue Feb 6, 2025 · 0 comments

Comments

@greggman
Copy link
Contributor

greggman commented Feb 6, 2025

Similar to #3363 , I'm wondering if most of the CTS should just run with all features requested and only a few specific tests check that if something isn't requested is not usable.

Some random thoughts

  1. The CTS would request less devices - not sure how much of a win that is but the code to cache devices exists for some reason and if the majority of tests requested everything they'd all get the same device.

  2. Some experimental features have broken valid common WebGPU code. These issues aren't caught by the current CTS because the features are not requested but these issues will break sites for devs running with experimental features enabled.

    I'd expect most devs would want to be able to use the internet at large and their own projects even when they've enabled experimental features.

    If that's not clear, imagine a feature 'experimental-ray-tracing', if enabled, it breaks renderPasEncoder.draw(...). Any site that enables all features with requestDevice({ requiredFeatures: adapter.features }) starts failing. Ok, fine, that only affects people with these experimental features enabled. But, that could be everyone on a Canary/Nightly/Technology-Preview version of a browser as well as playtesters/beta testers and others.

    Further, the dev themselves might be testing out 'experimental-bindless'. They arguably shouldn't have to workaround the previous issue if they enable all features. Yet if they don't they can't test and have to go refactor their code around browser bugs.

    If the CTS surfaced this issue it would be unlikely the 'experimental-ray-tracing' feature would have shipped until the issue was resolved. Just enabling all the features in 1 test doesn't cover this because, as the example above shows, the breakage only happens on certain usage. Turning on all the features has a higher chance of finding this broken usage.

  3. Some checks become simpler?

    I'm not sure this is true but, right now most tests have 2 ways to deal with features and limits

    1. request them on the device via t.selectDeviceOrSkipTest('name-of-feature')

      this has to happen before the test function in beforeAllSubcases

    2. check something on the device t.skipIf(t.device.limits.maxColorAttachments < 5)

      this has to happen in the test function, after the device is created

    Enabling all the features would switch to just method 2.

    t.skipIf(t.device.limits.maxColorAttachments < 5);
    t.skipIf(!t.device.features.has('name-of-feature');

    I don't know if that's simpler or not but it would move most of the checks to one place
    instead of spread out as they often are now.

    // current
    .beforeAllSubcases(t => {
      t.skipIfTextureFormatNotSupported(t.params.format);
      t.skipIfColorRenderableNotSupportedForFormat(t.params.format);
    })
    .fn(t => {
      const { format, attachmentCount, isAsync } = t.params;
      const info = kTextureFormatInfo[format];
    
      t.skipIf(attachmentCount > t.device.limits.maxColorAttachments);
      ...

    becomes

    // new
    .fn(t => {
      const { format, attachmentCount, isAsync } = t.params;
      const info = kTextureFormatInfo[format];
    
      t.skipIfTextureFormatNotSupported(t.params.format);
      t.skipIfColorRenderableNotSupportedForFormat(t.params.format);
      t.skipIf(attachmentCount > t.device.limits.maxColorAttachments);
      ...
@greggman greggman changed the title Switch most of the the CTS to ask for all features Switch most of the CTS to ask for all features Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant