[Public WebGL] WEBGL_texture_from_depth_video extension proposal

Rob Manson [email protected]
Thu Nov 13 15:01:05 PST 2014

Hi Ken/Florian,

yep I think you make a really good point here.

There's some potential issues under the hood that we're looking into and 
we're also exploring a different option that's come out of our 
discussions with another WG.

So lets put this on hold and we'll get back to you as soon as we have a 
consolidated response.

Thanks again for all your time and constructive feedback.


On 13/11/14 9:07 AM, Kenneth Russell wrote:
> Rob, Ningxin (in particular), all,
> After more thought and offline chat with Florian, it does seem
> unfortunate to introduce a WebGL extension that will have to be
> supported forever, just to handle a potential brief time period when
> the Media Capture Depth Stream Extensions are supported, but the WebGL
> texture upload path isn't. (Hopefully, both pieces of support will
> land at the same time).
> Per Florian's suggestion, what if we simply add a new section 5.7
> "WebGLRenderingContext Interface" to
> http://w3c.github.io/mediacapture-depth/ stating:
>   - An HTMLVideoElement containing a depth stream may be uploaded to a
> WebGL texture with the same bit depth as the depth stream.
>   -- In WebGL 1.0 [1], this includes RGB/UNSIGNED_SHORT_5_6_5 and
> RGBA/UNSIGNED_SHORT_4_4_4_4 textures.
>   -- In WebGL 2.0 [2], this includes RED_INTEGER/SHORT/R16I and
> Then include the code snippet from WEBGL_texture_from_depth_video,
> stating clearly that this is the code path for WebGL 1.0, and that
> WebGL 2.0 will avoid the unpacking of the red, green and blue
> channels.
> [1] https://www.khronos.org/registry/webgl/specs/1.0/ (or
> https://www.khronos.org/registry/webgl/specs/latest/1.0/ , your
> choice)
> [2] https://www.khronos.org/registry/webgl/specs/latest/2.0/
> Then the WEBGL_texture_from_depth_video extension proposal isn't
> needed. If the Media Capture Depth Stream extensions are supported at
> all by the browser, then uploading depth streams to WebGL will be
> implicitly supported.
> What do you think?
> Thanks,
> -Ken
> On Wed, Nov 12, 2014 at 3:24 AM, Florian Bösch <[email protected]> wrote:
>> I'd really rather not have an extension, and lengthy discussion on the
>> nonsensical nature of it for something that's essentially a few lines of
>> code ala pseudocode:
>> if obj.isVideo
>>    if internalFormat.bitDepth == video.bitDepth
>>      texImage2D obj.bytes
>>    else
>>     throw "bit depths do not match"
>> Not supporting the video formats, that the browser implements isn't a
>> "capability". It's a bug.
>> On Wed, Nov 12, 2014 at 11:55 AM, Florian Bösch <[email protected]> wrote:
>>> In the right thread again, a bit more structured. I don't like this being
>>> an extension, I don't think it's an extension to begin with. It's basically
>>> just saying "a 16-bpp texture can be filled from an array of bytes with the
>>> matching size", and then tacks on a tutorial on how to interprete those
>>> bytes in a shader. It's an "extension" that regulates user behavior and
>>> external image formats, that's got nothing todo with WebGL to begin with.
>>> I'd propse adding a very simple change to the WebGL section of
>>> http://w3c.github.io/mediacapture-depth/
>>>> The per-pixel bit depth of a depth video stream is 16-bit. It can be
>>>> uploaded to any WebGL texture of matching per pixel bit depth. The depth is
>>>> an unsigned short value and it can be interpreted in a shader as follows...
>>> On Wed, Nov 12, 2014 at 1:26 AM, Jeff Gilbert <[email protected]>
>>> wrote:
>>>> It seems like it would already require a bunch of code, to me. I'm not
>>>> sure how using packed 565 instead of something else requires less code.
>>>> The current proposal is an ugly hack, and I'd like to be sure there's no
>>>> better way before considering implementing it.
>>>> "We already have this working in prototype" is not a convincing argument
>>>> for the quality of a spec.
>>>> If all of the target devices support depth textures, but not GLES3, the
>>>> spec should use depth textures.
>>>> In GLES3, we get R16UI, though it isn't normalized. It has the same
>>>> capabilities as DEPTH_COMPONENT16, anyways. (sampleable and renderable, but
>>>> not filterable) It seems like you'd want normalization for something like
>>>> this, but maybe not? If renderability isn't important, R32F could give
>>>> normalization with sufficient precision.
>>>> Regardless of all of this, like GL extensions do, it would be better if
>>>> this extension proposal had a brief discussion of reasoning for why it makes
>>>> the tradeoffs it does.
>>>> -Jeff
>>>> ----- Original Message -----
>>>> From: "Rob Manson" <[email protected]>
>>>> To: "Kenneth Russell" <[email protected]>
>>>> Cc: "Daniel Koch" <[email protected]>, "Jeff Gilbert"
>>>> <[email protected]>, "Florian Bösch" <[email protected]>, "public webgl"
>>>> <[email protected]>, "Ningxin Hu" <[email protected]>
>>>> Sent: Tuesday, November 11, 2014 4:11:06 PM
>>>> Subject: Re: [Public WebGL] WEBGL_texture_from_depth_video extension
>>>> proposal
>>>> +1 for this and to us not creating more work for implementers.
>>>> On 12/11/14 10:46 AM, Kenneth Russell wrote:
>>>>> I still think it's reasonable to limit depth camera texture uploads to
>>>>> just 5_6_5 textures in WebGL 1.0. WebGL 2.0 implementations are
>>>>> actively being developed and spending excessive time on the 1.0 code
>>>>> paths will just delay their release. Intel's already implemented the
>>>>> depth stream upload path to 5_6_5 textures and shown some demos
>>>>> publicly, so integrating this support will take very little time.
> -----------------------------------------------------------
> You are currently subscribed to [email protected]
> To unsubscribe, send an email to [email protected] with
> the following command in the body of your email:
> unsubscribe public_webgl
> -----------------------------------------------------------

You are currently subscribed to [email protected]
To unsubscribe, send an email to [email protected] with
the following command in the body of your email:
unsubscribe public_webgl

More information about the public_webgl mailing list