[Public WebGL] OES_EGL_image_external Extension proposal

Byungseon Shin [email protected]
Fri Jan 6 12:10:35 PST 2017


First of all, I missed to reply-all, so I am adding public-webgl alias
again.

Second, I agree with your point. I just was trying to explain that
application can control the rendering logic.

We can propose alternative approach by making explicit calling of
EGLImageTargetTexture2DOES.
Then, application can decide if new frame will be used or not.

var update = function(){
  updateTexture(); // ext.EGLImageTargetTexture2DOES
  render();
  requestAnimationFrame(update);
}
requestAnimationFrame(update);


On Sat, Jan 7, 2017 at 4:46 AM Florian Bösch <[email protected]> wrote:

> That's not how WebGL works, this is:
>
> var update = function(){
>   render();
>   requestAnimationFrame(update);
> }
> requestAnimationFrame(update);
>
> If you block in the JS thread on purpose you achieve nothing because no
> matter how long you block, the frame is only going to be presented at the
> next animationFrame composit, which is synced to vsync (or g-sync, or
> direct-present sync, etc.)
>
> You do run a risk though of missing the sync, in which case you drop a
> frame, which is bad (jitter/glitching/nausea (in HMD users) etc.).
>
> On Fri, Jan 6, 2017 at 8:43 PM, Byungseon Shin <[email protected]>
> wrote:
>
> As in the proposal, application can skip calling of render function which
> will call swapBuffers as following:
> ----------------------------------------------------
> update();
> while (true) {
>    ...
>     // check the gap between frame time of WebGL and Video
>     if ( gap < allowable_delay)
>         render()
> }
>
> On Sat, Jan 7, 2017 at 4:36 AM Florian Bösch <[email protected]> wrote:
>
> Sure, but WebGL applications have no control over and can't influence
> (beyond not being late) when they're presented on screen. They do not
> control the swapBuffers command.
>
> On Fri, Jan 6, 2017 at 8:32 PM, Byungseon Shin <[email protected]>
> wrote:
>
> Hi Florian,
>
> To control the sync, I think, application can use the gap between Video
> frame time and WebGL frame time.
>
> Kind regards,
> Byungseon Shin
>
> On Sat, Jan 7, 2017 at 4:14 AM Florian Bösch <[email protected]> wrote:
>
> Unless I'm mistaken, knowing the time alone isn't going to help that much.
>
> The dynamic_texture extension attempted to solve this problem with setting
> a presentation time for a frame. I'm not sure how realistic it is to try to
> time presentation time, so that might have been overly ambitious.
>
> On Fri, Jan 6, 2017 at 8:01 PM, Byungseon Shin <[email protected]>
> wrote:
>
> Hi WebGL working group members,
>
> We have updated proposal by adding following function :
> - Provides time of frame, texture width and height of HTMLVideoElement's
> EGLImage.
>
> Please take a look at the pull request:
> <https://github.com/KhronosGroup/WebGL/pull/2243>
>
> Kind regards,
> Byungseon Shin
>
>
>
> On Tue, Jan 3, 2017 at 8:48 AM Byungseon Shin <[email protected]> wrote:
>
> Hi Florian,
>
> We are working on update which will cover dynamic_texture concerns.
> I plan to upload new version on github for reviewing by this week.
>
> Happy 2017 year!
> Byungseon Shin
>
> On Tue, Jan 3, 2017 at 1:58 AM Florian Bösch <[email protected]> wrote:
>
> What is the status of the
> https://www.khronos.org/registry/webgl/extensions/proposals/OES_EGL_image_external/
> extension? Is work on the spec still going on or is this the final version
> that takes into account dynamic_texture concerns?
>
> On Wed, Oct 12, 2016 at 3:59 AM, Byungseon Shin <[email protected]>
> wrote:
>
> Hi Ken,
>
> Thanks for the feedback.
> Of course, we are willing to devote time and eager to enhance WebGL video
> rendering performance.
> We prefer to use our proposal as a base but we could revise
> dynamic_texture as well.
>
> Kind regards,
> Byungseon Shin
>
>
> On Wed, Oct 12, 2016 at 8:48 AM Kenneth Russell <[email protected]> wrote:
>
> Hi Byungseon,
>
> Thanks for putting together this extension proposal.
>
> In conversations with groups within Google that are trying to do 360
> degree video rendering in WebGL, they need more information than your
> extension would provide; specifically, they would need the exact timestamp
> of the frame currently being rendered, as well as other per-frame metadata
> like the current width and height of the texture (for variable bitrate
> video streams).
>
> Mark Callow's WEBGL_dynamic_texture extension proposal
> https://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture/
> provides the controls needed. However, per working group discussions, the
> current extension's a little too complicated. I think if that extension
> were simplified a little bit that it would provide all of the performance
> benefits yours offers, and the controls that are known to be necessary.
>
> Do you think you'd be willing to devote some time to either extending your
> proposal (I can provide specific feedback) or editing down Mark's proposal
> to handle these use cases? I'd prefer to edit down WEBGL_dynamic_texture,
> if you're willing.
>
> Thanks,
>
> -Ken
>
>
>
> On Tue, Oct 11, 2016 at 8:27 AM, Byungseon Shin <[email protected]>
> wrote:
>
> Hi Corentin,
>
> On Tue, Oct 11, 2016 at 11:16 PM Corentin Wallez <[email protected]>
> wrote:
>
> The problem with using an EGL_image type of extension is that the texture
> data store becomes shared between the new texture and the object it was
> created from. This is great when the application controls both sides and
> can ensure read and writes are probably synchronized, but in the case of
> video here, one side would write without the application being able to
> control synchronization.
>
> The standard way to do this type of video / texture binding in EGL is to
> use EGL_KHR_stream
> <https://www.khronos.org/registry/egl/extensions/KHR/EGL_KHR_stream.txt> which
> is a much more complex and might not be an improvement over what developers
> can do with the current APIs.
>
>
> As in proposal revision#3 and as Florian already mentioned,
> Current proposal support "bind once and update implicitly" concept :  "ext.EGLImageTargetTexture2DOES(ext.TEXTURE_EXTERNAL_OES,
> videoElement); that the texture is "bound" to the video and no further
> calls need to be emitted (as would presently be the case)"
>
> In a that sense, we could abstract the details implementation into
> MediaPlayer Side like synchronizing issues.
>
> Even EGL_KHR_stream provides more details synchronization issues but still
> lot's of GPU driver and Video Decoder need to support the latest
> specification.
> But OES_EGL_image_external extension is already supported by most of the
> GPU drivers and Video decoders.
>
> And one of the performance bottleneck is to converting EGLImage input to
> TEXTURE_2D to handled by WebGL application.
>
>
> On Tue, Oct 11, 2016 at 6:54 AM, Byungseon Shin <[email protected]>
> wrote:
>
> Hi Florian, Maksims,
>
> Thanks for the feedback.
>
> I have updated proposal to explain how application works.
> Please see the attached updated proposal.
>
> To render an update frame, we need to call ext.EGLImageTargetTexture2DOES
> to bind newly generated texture buffer from video decoder.
>
> By supporting OES_EGL_image_external extension, application can use direct
> texture when video driver provides EGLImage like OpenGL ES natvie
> applications.
>
> Our proposal is just focusing on extending format of texture compatible
> with OpenGL ES extension and does not conflict with previous proposals.
>
> So, we provides a simplest way to adopt OES_EGL_image_external extension.
>
> By using references implementations of browser, WebGL renders video more
> than 10 times faster than TEXTURE_2D format with Full HD resolution output.
>
> Kind regards,
> Byungseon Shin
>
> On Tue, Oct 11, 2016 at 4:51 PM Maksims Mihejevs <[email protected]>
> wrote:
>
> Worth mentioning that in web there are multiple sources that have
> independent redraw mechanics, that includes video as well as canvas
> elements. It might make sense to have single extension for providing direct
> access to those sources without need to re-upload it.
>
> I have assumptions that internally in browsers, both video and canvas have
> their buffers, so it might lead to very similar implementation as from
> internal side as well as from webgl api side.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://khronos.org/pipermail/public_webgl_khronos.org/attachments/20170106/68a07288/attachment.html>


More information about the public_webgl mailing list