[Public WebGL] WEBGL_texture_from_depth_video extension proposal

Florian Bösch [email protected]
Thu Nov 6 02:29:09 PST 2014

On Thu, Nov 6, 2014 at 9:08 AM, Kenneth Russell <[email protected]> wrote:

> The reason to expose an extension from my point of view is (a) to allow
> detection of support for uploading depth videos to WebGL and (b) to provide
> a single, efficient upload path that users can rely on.
a) Is support depending on anything else than the browser implementing it?

b) Is that goal best served with an extension?

Since depth texture streaming interacts with various other specifications
(canvas, typed arrays and webgl 1, 2, 2.1, 3, depth textures, floating
point textures and <video>), it would seem to me the most consistent and
easiest to use entry point for any discovering support and consistently
supporting a feature would be the media capture depth specification, rather
than modify half a dozen other specifications and add pieces to it.

> 3) The style of how to support the upload isn't very friendly to users
>> because it requires manual conversion in the fragment shader. I'd much
>> rather see float/half-float texture formats being satisfied, where the
>> browser knows about the surface format of the depth texture, and converts
>> it (on GPU, without blocking operation) to the specified floating point
>> format for upload. This'd have the advantage of offering users a
>> transparent support for depth images, regardless of their underlying format.
> Yes, with WebGL 1.0 the conversion is a little painful. This will become
> trivial with WebGL 2.0 and the availability of R16UI / R16I textures. I'd
> like to focus on getting WebGL 2.0 out the door, but we'd appreciate any
> concrete suggestions you have for the Media Capture folks about better data
> upload formats to use with WebGL 1.0.
> -Ken

There's some concern for me for backwards/forwards compatibility and
extension proliferation. The current extension specification defines as
only supported format upload format UNSIGNED_SHORT_5_6_5.

What if...

   - A new device appears that delivers depth fields as UNSIGNED_BYTE -->
   - A new device appears that delivers depth fields as UNSIGNED_INT ->
   - A new device appears that delivers depth fields as Float ->
   - A new texture format should be supported (float) ->
   - A new texture format should be supported (depth_component) ->
   - WebGL 2.0's texture formats should be supported ->
   - WebGL 2.1's texture formats should be supported ->
   - etc.

I'd like depth upload to work for the following WebGL 1 formats:

   - gl.RGB, gl.UNSIGNED_BYTE -> greyscale 0 - 255
   - gl.RGBA, gl.UNSIGNED_BYTE -> greyscale 0 - 255, alpha 1
   - gl.RGBA, gl.FLOAT -> greyscale 0 - 1 (32-bit float), alpha 1
   - gl.RGB, gl.FLOAT -> greyscale 0 - 1 (32-bit float)
   - gl.LUMINANCE, gl.UNSIGNED_BYTE -> 0 - 255
   - gl.LUMINANCE, gl.FLOAT -> 0-1
   - gl.RGBA, ext.HALF_FLOAT_OES -> 0-1 (16-bit float), alpha 1
   - gl.RGB, ext.HALF_FLOAT_OES -> 0-1 (16-bit float)
   - ext.DEPTH_COMPONENT, gl.UNSIGNED_SHORT -> 0-2^16-1
   - ext.DEPTH_COMPONENT, gl.UNSIGNED_INT -> 0-2^32-1
   - ext.DEPTH_COMPONENT, ext.UNSIGNED_INT_24_8_WEBGL -> 0-2^24-1, stencil 0

I'd also like to see depth uploads work for the about 30 internal formats
that is has, including things that where in extensions in WebGL 1 and are
now core in WebGL 2.

I'd also like to see depth uploads work with any of the changed texture
formats coming to WebGL 2.1 and WebGL 3.0.

And I'd like this to work without having to introduce 3 dozen new
extensions to make it happen in the years to come...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://khronos.org/pipermail/public_webgl_khronos.org/attachments/20141106/039c8654/attachment.html>

More information about the public_webgl mailing list