[Public WebGL] WEBGL_texture_from_depth_video extension proposal

Kenneth Russell [email protected]
Wed Nov 12 14:07:25 PST 2014

Rob, Ningxin (in particular), all,

After more thought and offline chat with Florian, it does seem
unfortunate to introduce a WebGL extension that will have to be
supported forever, just to handle a potential brief time period when
the Media Capture Depth Stream Extensions are supported, but the WebGL
texture upload path isn't. (Hopefully, both pieces of support will
land at the same time).

Per Florian's suggestion, what if we simply add a new section 5.7
"WebGLRenderingContext Interface" to
http://w3c.github.io/mediacapture-depth/ stating:

 - An HTMLVideoElement containing a depth stream may be uploaded to a
WebGL texture with the same bit depth as the depth stream.
 -- In WebGL 1.0 [1], this includes RGB/UNSIGNED_SHORT_5_6_5 and
RGBA/UNSIGNED_SHORT_4_4_4_4 textures.
 -- In WebGL 2.0 [2], this includes RED_INTEGER/SHORT/R16I and

Then include the code snippet from WEBGL_texture_from_depth_video,
stating clearly that this is the code path for WebGL 1.0, and that
WebGL 2.0 will avoid the unpacking of the red, green and blue

[1] https://www.khronos.org/registry/webgl/specs/1.0/ (or
https://www.khronos.org/registry/webgl/specs/latest/1.0/ , your
[2] https://www.khronos.org/registry/webgl/specs/latest/2.0/

Then the WEBGL_texture_from_depth_video extension proposal isn't
needed. If the Media Capture Depth Stream extensions are supported at
all by the browser, then uploading depth streams to WebGL will be
implicitly supported.

What do you think?



On Wed, Nov 12, 2014 at 3:24 AM, Florian Bösch <[email protected]> wrote:
> I'd really rather not have an extension, and lengthy discussion on the
> nonsensical nature of it for something that's essentially a few lines of
> code ala pseudocode:
> if obj.isVideo
>   if internalFormat.bitDepth == video.bitDepth
>     texImage2D obj.bytes
>   else
>    throw "bit depths do not match"
> Not supporting the video formats, that the browser implements isn't a
> "capability". It's a bug.
> On Wed, Nov 12, 2014 at 11:55 AM, Florian Bösch <[email protected]> wrote:
>> In the right thread again, a bit more structured. I don't like this being
>> an extension, I don't think it's an extension to begin with. It's basically
>> just saying "a 16-bpp texture can be filled from an array of bytes with the
>> matching size", and then tacks on a tutorial on how to interprete those
>> bytes in a shader. It's an "extension" that regulates user behavior and
>> external image formats, that's got nothing todo with WebGL to begin with.
>> I'd propse adding a very simple change to the WebGL section of
>> http://w3c.github.io/mediacapture-depth/
>>> The per-pixel bit depth of a depth video stream is 16-bit. It can be
>>> uploaded to any WebGL texture of matching per pixel bit depth. The depth is
>>> an unsigned short value and it can be interpreted in a shader as follows...
>> On Wed, Nov 12, 2014 at 1:26 AM, Jeff Gilbert <[email protected]>
>> wrote:
>>> It seems like it would already require a bunch of code, to me. I'm not
>>> sure how using packed 565 instead of something else requires less code.
>>> The current proposal is an ugly hack, and I'd like to be sure there's no
>>> better way before considering implementing it.
>>> "We already have this working in prototype" is not a convincing argument
>>> for the quality of a spec.
>>> If all of the target devices support depth textures, but not GLES3, the
>>> spec should use depth textures.
>>> In GLES3, we get R16UI, though it isn't normalized. It has the same
>>> capabilities as DEPTH_COMPONENT16, anyways. (sampleable and renderable, but
>>> not filterable) It seems like you'd want normalization for something like
>>> this, but maybe not? If renderability isn't important, R32F could give
>>> normalization with sufficient precision.
>>> Regardless of all of this, like GL extensions do, it would be better if
>>> this extension proposal had a brief discussion of reasoning for why it makes
>>> the tradeoffs it does.
>>> -Jeff
>>> ----- Original Message -----
>>> From: "Rob Manson" <[email protected]>
>>> To: "Kenneth Russell" <[email protected]>
>>> Cc: "Daniel Koch" <[email protected]>, "Jeff Gilbert"
>>> <[email protected]>, "Florian Bösch" <[email protected]>, "public webgl"
>>> <[email protected]>, "Ningxin Hu" <[email protected]>
>>> Sent: Tuesday, November 11, 2014 4:11:06 PM
>>> Subject: Re: [Public WebGL] WEBGL_texture_from_depth_video extension
>>> proposal
>>> +1 for this and to us not creating more work for implementers.
>>> On 12/11/14 10:46 AM, Kenneth Russell wrote:
>>> > I still think it's reasonable to limit depth camera texture uploads to
>>> > just 5_6_5 textures in WebGL 1.0. WebGL 2.0 implementations are
>>> > actively being developed and spending excessive time on the 1.0 code
>>> > paths will just delay their release. Intel's already implemented the
>>> > depth stream upload path to 5_6_5 textures and shown some demos
>>> > publicly, so integrating this support will take very little time.

You are currently subscribed to [email protected]
To unsubscribe, send an email to [email protected] with
the following command in the body of your email:
unsubscribe public_webgl

More information about the public_webgl mailing list