[Public WebGL] WEBGL_texture_from_depth_video extension proposal
Mon Nov 10 19:24:21 PST 2014
> It'll only be efficient to upload depth videos to WebGL textures using
> the internal format which avoids converting the depth values during
> the upload process. That's why UNSIGNED_SHORT_5_6_5 was chosen as the
> single supported format for uploading this content to WebGL 1.0. It's
> not desirable for either the browser implementer or the web developer
> to support uploading depth videos to lots of random texture formats if
> they won't be efficient. The Media Capture group should comment on
> what formats depth cameras tend to output, and are likely to output in
> the future.
At the moment we have only been able to find devices that are shipping
or are planning to ship with 16 bit depth support. Older Kinect devices
etc. used to support 8 bit or 11 bit depth support, but this has now
been upgraded to 16 bits.
This makes sense if you look at what 16 bits really means and how that
relates to the technology. If the units that these 16 bits represent are
mm (which is the defacto standard) then the maximum distance that can be
represented is 655.36m.
For any "Time of Flight" or "Structured Light" depth sensor this is an
extremely long way. In fact many of these sensors don't work well in
sunlight, so unless you have a room that is over half a kilometer wide
or tall then needing more than 16 bits really isn't an issue 8) Not to
mention that the sensor simply isn't that sensitive.
Even with lidar it's very unlikely that consumer depth sensors are going
to be scanning areas that have a radius of over half a kilometer (e.g.
1.2km across). The amount of noise and error in this scale is likely to
be significant and resolving this will likely keep the price of this
type of hardware out of the consumer reach for quite some time.
Of course, any technology prediction is likely to be wrong and we're
open to other opinions and suggestions. But I think we've made a
reasonable and evidence based assumption to build upon.
>> Since depth texture streaming interacts with various other specifications
>> >(canvas, typed arrays and webgl 1, 2, 2.1, 3, depth textures, floating point
>> >textures and <video>), it would seem to me the most consistent and easiest
>> >to use entry point for any discovering support and consistently supporting a
>> >feature would be the media capture depth specification, rather than modify
>> >half a dozen other specifications and add pieces to it.
> I agree in general. If it's possible to incorporate this sort of
> feature detection into the Media Capture spec that would be fine.
> Perhaps Rob, Ningxin or someone else from that group can comment.
We have been discussing feature detection and it seems likely that devs
would be able to detect the pipeline they're interested in.
So if they want to use WebGL/shaders to process the depth stream they
would detect if the WEBGL_texture_from_depth_video extension is supported.
If they want to use Canvas2D (or where we land on that) then they would
test to see if .getDepthData() is supported.
However, we're still discussing the core capability detection for depth
streams and are open to suggestions.
BTW: Florian, we really aren't modifying "half a dozen other
specifications". With WebGL it's just an extension not a modification.
With Canvas it is a modification and another alternative is under
discussion. With MediaStreams this is a modification - but this is the
core of our work and fits squarely in our TF's scope. The rest you
mention are untouched by us - typed arrays, <video>, etc.
You are currently subscribed to [email protected]
To unsubscribe, send an email to [email protected] with
the following command in the body of your email:
More information about the public_webgl