Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

First pass at defining tracked sources #1361

Merged
merged 5 commits into from
Oct 21, 2024
Merged

First pass at defining tracked sources #1361

Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1cc1abe
First pass at defining tracked sources
cabanier Mar 2, 2024
dc603b5
Some refining on tracked sources
cabanier Mar 3, 2024
b1f3e57
rename auxiliary to tracked
cabanier Mar 6, 2024
96dc99a
Address @toji's review comments
cabanier Apr 11, 2024
daa8a66
address review comments
cabanier Oct 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 98 additions & 24 deletions index.bs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -630,6 +630,7 @@ enum XRVisibilityState {
readonly attribute Float32Array? supportedFrameRates;
[SameObject] readonly attribute XRRenderState renderState;
[SameObject] readonly attribute XRInputSourceArray inputSources;
[SameObject] readonly attribute XRInputSourceArray trackedSources;
readonly attribute FrozenArray<DOMString> enabledFeatures;
readonly attribute boolean isSystemKeyboardSupported;

Expand Down Expand Up @@ -831,12 +832,14 @@ When this method is invoked, the user agent MUST run the following steps:

</div>

Each {{XRSession}} has a <dfn>list of active XR input sources</dfn> (a [=/list=] of {{XRInputSource}}) which MUST be initially an empty [=/list=].
Each {{XRSession}} has a <dfn>list of active XR input sources</dfn> (a [=/list=] of {{XRInputSource}}) and a <dfn>list of active XR tracked sources</dfn> (a [=/list=] of {{XRInputSource}}) which MUST be initially an empty [=/list=].

Each {{XRSession}} has an <dfn for="XRSession">XR device</dfn>, which is an [=/XR device=] set at initialization.

The <dfn attribute for="XRSession">inputSources</dfn> attribute returns the {{XRSession}}'s [=list of active XR input sources=].

The <dfn attribute for="XRSession">trackedSources</dfn> attribute returns the {{XRSession}}'s [=list of active XR tracked sources=].

The user agent MUST monitor any [=XR input source=]s associated with the [=XRSession/XR device=], including detecting when [=XR input source=]s are added, removed, or changed.

Each {{XRSession}} has a <dfn for=XRSession>promise resolved</dfn> flag, initially `false`.
Expand All @@ -848,13 +851,23 @@ NOTE: The purpose of this flag is to ensure that the [=XRSession/add input sourc
When <dfn for="XRSession" lt="add input source">new [=XR input source=]s become available</dfn> for {{XRSession}} |session|, the user agent MUST run the following steps:

1. If |session|'s [=XRSession/promise resolved=] flag is not set, abort these steps.
1. Let |added| be a new [=/list=].
1. Let |added primary sources| be a new [=/list=].
1. Let |added tracked sources| be a new [=/list=].
1. For each new [=XR input source=]:
1. Let |inputSource| be a [=new=] {{XRInputSource}} in the [=relevant realm=] of this {{XRSession}}.
1. Add |inputSource| to |added|.
1. Let |inputSource| be a [=new=] {{XRInputSource}} in the [=relevant realm=] of this {{XRSession}}, then perform the following step:
<dl class="switch">
: If |inputSource| is a [=primary input source=]:
::
Add |inputSource| to |added primary sources|.
: Otherwise:
::
Add |inputSource| to |added tracked sources|.
</dl>
1. [=Queue a task=] to perform the following steps:
1. [=list/Extend=] |session|'s [=list of active XR input sources=] with |added|.
1. Fire an {{XRInputSourcesChangeEvent}} named {{inputsourceschange!!event}} on |session| with {{XRInputSourcesChangeEvent/added}} set to |added|.
1. [=list/Extend=] |session|'s [=list of active XR input sources=] with |added primary sources|.
1. If |added primary sources| is not empty, fire an {{XRInputSourcesChangeEvent}} named {{inputsourceschange!!event}} on |session| with {{XRInputSourcesChangeEvent/added}} set to |added primary sources|.
1. [=list/Extend=] |session|'s [=list of active XR tracked sources=] with |added tracked sources|.
1. If |added tracked sources| is not empty, fire an {{XRTrackedSourcesChangeEvent}} named {{trackedsourceschange!!event}} on |session| with {{XRTrackedSourcesChangeEvent/added}} set to |added tracked sources|.

</div>

Expand All @@ -863,34 +876,63 @@ When <dfn for="XRSession" lt="add input source">new [=XR input source=]s become
When any previously added <dfn for="XRSession" lt="remove input source">[=XR input source=]s are no longer available</dfn> for {{XRSession}} |session|, the user agent MUST run the following steps:

1. If |session|'s [=XRSession/promise resolved=] flag is not set, abort these steps.
1. Let |removed| be a new [=/list=].
1. Let |removed primary sources| be a new [=/list=].
1. Let |removed tracked sources| be a new [=/list=].
1. For each [=XR input source=] that is no longer available:
1. Let |inputSource| be the {{XRInputSource}} in |session|'s [=list of active XR input sources=] associated with the [=XR input source=].
1. Add |inputSource| to |removed|.
1. Let |inputSource| be the {{XRInputSource}} in |session|'s [=list of active XR input sources=] associated with the [=XR input source=], then perform the following step:
<dl class="switch">
: If |inputSource| is a [=primary input source=]:
::
Add |inputSource| to |removed primary sources|.
: Otherwise:
::
Add |inputSource| to |removed tracked sources|.
</dl>
1. [=Queue a task=] to perform the following steps:
1. [=list/Remove=] each {{XRInputSource}} in |removed| from |session|'s [=list of active XR input sources=].
1. Fire an {{XRInputSourcesChangeEvent}} named {{inputsourceschange!!event}} on |session| with {{XRInputSourcesChangeEvent/removed}} set to |removed|.
1. [=list/Remove=] each {{XRInputSource}} in |removed primary sources| from |session|'s [=list of active XR input sources=].
1. ire an {{XRInputSourcesChangeEvent}} named {{inputsourceschange!!event}} on |session| with {{XRInputSourcesChangeEvent/removed}} set to |removed primary sources|.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. ire an {{XRInputSourcesChangeEvent}} named {{inputsourceschange!!event}} on |session| with {{XRInputSourcesChangeEvent/removed}} set to |removed primary sources|.
1. If |removed primary sources| is not empty, fire an {{XRInputSourcesChangeEvent}} named {{inputsourceschange!!event}} on |session| with {{XRInputSourcesChangeEvent/removed}} set to |removed primary sources|.

1. [=list/Remove=] each {{XRInputSource}} in |removed tracked sources| from |session|'s [=list of active XR tracked sources=].
1. If |removed tracked sources| is not empty, fire an {{XRTrackedSourcesChangeEvent}} named {{trackedsourceschange!!event}} on |session| with {{XRTrackedSourcesChangeEvent/removed}} set to |removed tracked sources|.

Note: The user agent MAY fire this event when an input source temporarily loses both position and orientation tracking. It is recommended that this only be done for physical handheld controller input sources. It is not recommended that this event be fired when this happens for tracked hand input sources, because this will happen often, nor is it recommended when this happens for tracker object input sources, since this makes it harder for the application to maintain a notion of identity.

</div>

<div class="algorithm" data-algorithm="on-input-source-change">

When the <dfn for="XRSession" export lt="change input source">{{XRInputSource/handedness}}, {{XRInputSource/targetRayMode}}, {{XRInputSource/profiles}}, or presence of a {{XRInputSource/gripSpace}} for any [=XR input source=]s change</dfn> for {{XRSession}} |session|, the user agent MUST run the following steps:
When the <dfn for="XRSession" export lt="change input source">{{XRInputSource/handedness}}, {{XRInputSource/targetRayMode}}, {{XRInputSource/profiles}}, presence of a {{XRInputSource/gripSpace}} or the state of the [=primary input source=] or [=tracked input source=] for any [=XR input source=]s change</dfn> for {{XRSession}} |session|, the user agent MUST run the following steps:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reads somewhat confusingly to me. I believe you are trying to say that the algorithm should fire if an input source switched from primary to tracked or visa-versa, but "the state of the primary input source or tracked input source" makes it sound like any state change (like a button press) could trigger this.

Not 100% confident in what a better phrasing would be, but perhaps something like this?

Suggested change
When the <dfn for="XRSession" export lt="change input source">{{XRInputSource/handedness}}, {{XRInputSource/targetRayMode}}, {{XRInputSource/profiles}}, presence of a {{XRInputSource/gripSpace}} or the state of the [=primary input source=] or [=tracked input source=] for any [=XR input source=]s change</dfn> for {{XRSession}} |session|, the user agent MUST run the following steps:
When the <dfn for="XRSession" export lt="change input source">{{XRInputSource/handedness}}, {{XRInputSource/targetRayMode}}, {{XRInputSource/profiles}}, presence of a {{XRInputSource/gripSpace}} or the status as a [=primary input source=] or [=tracked input source=] for any [=XR input source=]s change</dfn> for {{XRSession}} |session|, the user agent MUST run the following steps:


1. If |session|'s [=XRSession/promise resolved=] flag is not set, abort these steps.
1. Let |added| be a new [=/list=].
1. Let |removed| be a new [=/list=].
1. Let |added primary sources| be a new [=/list=].
1. Let |removed primary sources| be a new [=/list=].
1. Let |added tracked sources| be a new [=/list=].
1. Let |removed tracked sources| be a new [=/list=].
1. For each changed [=XR input source=]:
1. Let |oldInputSource| be the {{XRInputSource}} in |session|'s [=list of active XR input sources=] previously associated with the [=XR input source=].
1. Let |newInputSource| be a [=new=] {{XRInputSource}} in the [=relevant realm=] of |session|.
1. Add |oldInputSource| to |removed|.
1. Add |newInputSource| to |added|.
1. Let |oldInputSource| be the {{XRInputSource}} in |session|'s [=list of active XR input sources=] previously associated with the [=XR input source=], then perform the following step:
<dl class="switch">
: If |oldInputSource| is a [=primary input source=] or its state changed from a [=primary input source=] to [=tracked input source=] a:
::
Add |oldInputSource| to |removed primary sources|.
: Otherwise:
::
Add |oldInputSource| to |removed tracked sources|.
</dl>
1. Let |newInputSource| be a [=new=] {{XRInputSource}} in the [=relevant realm=] of |session|, then perform the following step:
<dl class="switch">
: If |newInputSource| is a [=primary input source=] or its state changed from a [=tracked input source=] to [=primary input source=] :
::
Add |newInputSource| to |added primary sources|.
: Otherwise:
::
Add |newInputSource| to |added tracked sources|.
</dl>
1. [=Queue a task=] to perform the following steps:
1. [=list/Remove=] each {{XRInputSource}} in |removed| from |session|'s [=list of active XR input sources=].
1. [=list/Extend=] |session|'s [=list of active XR input sources=] with |added|.
1. Fire an {{XRInputSourcesChangeEvent}} named {{inputsourceschange!!event}} on |session| with{{XRInputSourcesChangeEvent/added}} set to |added| and {{XRInputSourcesChangeEvent/removed}} set to |removed|.
1. [=list/Remove=] each {{XRInputSource}} in |removed primary sources| from |session|'s [=list of active XR input sources=].
1. [=list/Extend=] |session|'s [=list of active XR input sources=] with |added primary sources|.
1. If |added primary sources| or |removed primary sources| are not empty, fire an {{XRInputSourcesChangeEvent}} named {{inputsourceschange!!event}} on |session| with{{XRInputSourcesChangeEvent/added}} set to |added primary sources| and {{XRInputSourcesChangeEvent/removed}} set to |removed primary sources|.
1. [=list/Remove=] each {{XRInputSource}} in |removed tracked sources| from |session|'s [=list of active XR tracked sources=].
1. [=list/Extend=] |session|'s [=list of active XR input sources=] with |added tracked sources|.
1. If |added tracked sources| or |removed tracked sources| are not empty, fire an {{XRTrackedSourcesChangeEvent}} named {{trackedsourceschange!!event}} on |session| with {{XRTrackedSourcesChangeEvent/added}} set to |added tracked sources| and {{XRTrackedSourcesChangeEvent/removed}} set to |removed tracked sources|.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See note below about the event type.


</div>

Expand Down Expand Up @@ -935,6 +977,8 @@ The <dfn attribute for="XRSession">onend</dfn> attribute is an [=Event handler I

The <dfn attribute for="XRSession">oninputsourceschange</dfn> attribute is an [=Event handler IDL attribute=] for the {{inputsourceschange}} event type.

The <dfn attribute for="XRSession">ontrackedsourceschange</dfn> attribute is an [=Event handler IDL attribute=] for the {{trackedsourceschange}} event type.

The <dfn attribute for="XRSession">onselectstart</dfn> attribute is an [=Event handler IDL attribute=] for the {{XRSession/selectstart}} event type.

The <dfn attribute for="XRSession">onselectend</dfn> attribute is an [=Event handler IDL attribute=] for the {{selectend}} event type.
Expand Down Expand Up @@ -1890,7 +1934,7 @@ Note: {{XRInputSource}}s in an {{XRSession}}'s {{XRSession/inputSources}} array

An [=XR input source=] is a <dfn>primary input source</dfn> if it supports a <dfn>primary action</dfn>. The [=primary action=] is a platform-specific action that, when engaged, produces {{XRSession/selectstart}}, {{XRSession/selectend}}, and {{XRSession/select}} events. Examples of possible [=primary action=]s are pressing a trigger, touchpad, or button, speaking a command, or making a hand gesture. If the platform guidelines define a recommended primary input then it should be used as the [=primary action=], otherwise the user agent is free to select one. The device MUST support at least one [=primary input source=].

An [=XR input source=] is an <dfn>auxiliary input source</dfn> if it does not support a [=primary action=], for example [=transient input sources=] associated with secondary screen touches on a multitouch device.
An [=XR input source=] is an <dfn>tracked input source</dfn> if it does not support a [=primary action=], for example [=transient input sources=] associated with secondary screen touches on a multitouch device.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the example used here is probably worth updating? The current one is a bit difficult to reason about. Also, the example should probably be non-normative. Maybe something like:

Suggested change
An [=XR input source=] is an <dfn>tracked input source</dfn> if it does not support a [=primary action=], for example [=transient input sources=] associated with secondary screen touches on a multitouch device.
An [=XR input source=] is an <dfn>tracked input source</dfn> if it does not support a [=primary action=]. These inputs are primarily intended to provide pose data.
Note: An example of a [=tracked input source=] would be tracking attachments for a users legs or a prop. Tracked hands may also be considered a [=tracked input source=] if there is no gesture recognition being performed to detect [=primary action=]s.

This probably also warrants a clarification in the WebXR Gamepads module that tracked input source MAY still expose gamepad data.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should also have a discussion about what the various spaces on the tracked sources mean? For example, targetRaySpace is somewhat nonsensical for a tracking puck, but still makes some sense for tracked hands or controllers that aren't held. But for a device that isn't firing any select events a "targetRay" is kind of a funny minsomer.

I'm comfortable with some cognitive dissonance in this area, since it's a relatively late addition, but I wouldn't mind it if we could provide some guidance for how future implementers should think about these spaces if we can.


<div class="algorithm" data-algorithm="on-input-start">

Expand Down Expand Up @@ -1957,7 +2001,7 @@ When an [=XR input source=] |source| for {{XRSession}} |session| has its [=prima
Transient input {#transient-input}
---------------

Some [=/XR device=]s may support <dfn>transient input sources</dfn>, where the [=XR input source=] is only meaningful while performing a <dfn>transient action</dfn>, either the [=primary action=] for a [=primary input source=], or a device-specific <dfn>auxiliary action</dfn> for an [=auxiliary input source=].
Some [=/XR device=]s may support <dfn>transient input sources</dfn>, where the [=XR input source=] is only meaningful while performing a <dfn>transient action</dfn>, either the [=primary action=] for a [=primary input source=], or a device-specific <dfn>auxiliary action</dfn> for an [=tracked input source=].

One example would be mouse, touch, or stylus input against an {{XRSessionMode/"inline"}} {{XRSession}}, which MUST produce a transient {{XRInputSource}} with a {{targetRayMode}} set to {{screen}}, treated as a [=primary action=] for the [=primary pointer=], and as a non-primary [=auxiliary action=] for a non-primary pointer.

Expand Down Expand Up @@ -2457,7 +2501,7 @@ When the user agent has to <dfn>fire an input source event</dfn> with name |name
XRInputSourcesChangeEvent {#xrinputsourceschangeevent-interface}
-------------------------

{{XRInputSourcesChangeEvent}}s are fired to indicate changes to the {{XRInputSource}}s that are available to an {{XRSession}}.
{{XRInputSourcesChangeEvent}}s are fired to indicate changes to the [=list of active XR input sources=] that are available to an {{XRSession}}.

<pre class="idl">
[SecureContext, Exposed=Window]
Expand All @@ -2482,6 +2526,34 @@ The <dfn attribute for="XRInputSourcesChangeEvent">added</dfn> attribute is a [=

The <dfn attribute for="XRInputSourcesChangeEvent">removed</dfn> attribute is a [=/list=] of {{XRInputSource}}s that were removed from the {{XRSession}} at the time of the event.

XRTrackedSourcesChangeEvent {#xrtrackedsourceschangeevent-interface}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't need to be a different interface from XRInputSourcesChangeEvent. They're both identical in structure, and if devs want to introspect them to figure out it it's a primary or tracked event they can look at the event name.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't remember if this was discussed at the face to face?
I think we need a different event otherwise existing experiences will start getting additional events that they won't know how to handle.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify: We can how a different event but still have it use the same interface. The same way that we have a single MouseEvent interface that is provided for mousedown, mouseup, and click events. The proposal here is not to push everything into a single event, it's to re-use the data structure returned with the separate events for input vs. tracked sources.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This issue is still outstanding. If you removed the XRTrackedSourcesChangeEvent and XRTrackedSourcesChangeEventInit and changed the trackedsourceschange event to use XRInputSourcesChangeEvent then I think this is mergable.

-------------------------

{{XRTrackedSourcesChangeEvent}}s are fired to indicate changes to the [=list of active XR tracked sources=] that are available to an {{XRSession}}.

<pre class="idl">
[SecureContext, Exposed=Window]
interface XRTrackedSourcesChangeEvent : Event {
constructor(DOMString type, XRTrackedSourcesChangeEventInit eventInitDict);
[SameObject] readonly attribute XRSession session;
[SameObject] readonly attribute FrozenArray&lt;XRInputSource&gt; added;
[SameObject] readonly attribute FrozenArray&lt;XRInputSource&gt; removed;
};

dictionary XRTrackedSourcesChangeEventInit : EventInit {
required XRSession session;
required FrozenArray&lt;XRInputSource&gt; added;
required FrozenArray&lt;XRInputSource&gt; removed;

};
</pre>

The <dfn attribute for="XRTrackedSourcesChangeEvent">session</dfn> attribute indicates the {{XRSession}} that generated the event.

The <dfn attribute for="XRTrackedSourcesChangeEvent">added</dfn> attribute is a [=/list=] of {{XRInputSource}}s that were added to the {{XRSession}} at the time of the event.

The <dfn attribute for="XRTrackedSourcesChangeEvent">removed</dfn> attribute is a [=/list=] of {{XRInputSource}}s that were removed from the {{XRSession}} at the time of the event.


XRReferenceSpaceEvent {#xrreferencespaceevent-interface}
---------------------
Expand Down Expand Up @@ -2519,6 +2591,8 @@ A user agent MUST [=fire an event=] named <dfn event for="XRSession">end</dfn> u

A user agent MUST [=fire an event=] named <dfn event for="XRSession">inputsourceschange</dfn> using {{XRInputSourcesChangeEvent}} on an {{XRSession}} when the session's [=list of active XR input sources=] has changed.

A user agent MUST [=fire an event=] named <dfn event for="XRSession">trackedsourceschange</dfn> using {{XRTrackedSourcesChangeEvent}} on an {{XRSession}} when the session's [=list of active XR tracked sources=] has changed.

A user agent MUST [=fire an event=] named <dfn event for="XRSession">selectstart</dfn> using {{XRInputSourceEvent}} on an {{XRSession}} when one of its {{XRInputSource}}s begins its [=primary action=]. The event MUST be of type .

A user agent MUST [=fire an event=] named <dfn event for="XRSession">selectend</dfn> using {{XRInputSourceEvent}} on an {{XRSession}} when one of its {{XRInputSource}}s ends its [=primary action=] or when an {{XRInputSource}} that has begun a [=primary action=] is disconnected.
Expand Down
Loading