[Public WebGL] OES_EGL_image_external Extension proposal

Byungseon Shin [email protected]
Fri Jan 6 11:01:01 PST 2017


Hi WebGL working group members,

We have updated proposal by adding following function :
- Provides time of frame, texture width and height of HTMLVideoElement's
EGLImage.

Please take a look at the pull request:
<https://github.com/KhronosGroup/WebGL/pull/2243>

Kind regards,
Byungseon Shin



On Tue, Jan 3, 2017 at 8:48 AM Byungseon Shin <[email protected]> wrote:

> Hi Florian,
>
> We are working on update which will cover dynamic_texture concerns.
> I plan to upload new version on github for reviewing by this week.
>
> Happy 2017 year!
> Byungseon Shin
>
> On Tue, Jan 3, 2017 at 1:58 AM Florian Bösch <[email protected]> wrote:
>
> What is the status of the
> https://www.khronos.org/registry/webgl/extensions/proposals/OES_EGL_image_external/
> extension? Is work on the spec still going on or is this the final version
> that takes into account dynamic_texture concerns?
>
> On Wed, Oct 12, 2016 at 3:59 AM, Byungseon Shin <[email protected]>
> wrote:
>
> Hi Ken,
>
> Thanks for the feedback.
> Of course, we are willing to devote time and eager to enhance WebGL video
> rendering performance.
> We prefer to use our proposal as a base but we could revise
> dynamic_texture as well.
>
> Kind regards,
> Byungseon Shin
>
>
> On Wed, Oct 12, 2016 at 8:48 AM Kenneth Russell <[email protected]> wrote:
>
> Hi Byungseon,
>
> Thanks for putting together this extension proposal.
>
> In conversations with groups within Google that are trying to do 360
> degree video rendering in WebGL, they need more information than your
> extension would provide; specifically, they would need the exact timestamp
> of the frame currently being rendered, as well as other per-frame metadata
> like the current width and height of the texture (for variable bitrate
> video streams).
>
> Mark Callow's WEBGL_dynamic_texture extension proposal
> https://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture/
> provides the controls needed. However, per working group discussions, the
> current extension's a little too complicated. I think if that extension
> were simplified a little bit that it would provide all of the performance
> benefits yours offers, and the controls that are known to be necessary.
>
> Do you think you'd be willing to devote some time to either extending your
> proposal (I can provide specific feedback) or editing down Mark's proposal
> to handle these use cases? I'd prefer to edit down WEBGL_dynamic_texture,
> if you're willing.
>
> Thanks,
>
> -Ken
>
>
>
> On Tue, Oct 11, 2016 at 8:27 AM, Byungseon Shin <[email protected]>
> wrote:
>
> Hi Corentin,
>
> On Tue, Oct 11, 2016 at 11:16 PM Corentin Wallez <[email protected]>
> wrote:
>
> The problem with using an EGL_image type of extension is that the texture
> data store becomes shared between the new texture and the object it was
> created from. This is great when the application controls both sides and
> can ensure read and writes are probably synchronized, but in the case of
> video here, one side would write without the application being able to
> control synchronization.
>
> The standard way to do this type of video / texture binding in EGL is to
> use EGL_KHR_stream
> <https://www.khronos.org/registry/egl/extensions/KHR/EGL_KHR_stream.txt> which
> is a much more complex and might not be an improvement over what developers
> can do with the current APIs.
>
>
> As in proposal revision#3 and as Florian already mentioned,
> Current proposal support "bind once and update implicitly" concept :  "ext.EGLImageTargetTexture2DOES(ext.TEXTURE_EXTERNAL_OES,
> videoElement); that the texture is "bound" to the video and no further
> calls need to be emitted (as would presently be the case)"
>
> In a that sense, we could abstract the details implementation into
> MediaPlayer Side like synchronizing issues.
>
> Even EGL_KHR_stream provides more details synchronization issues but still
> lot's of GPU driver and Video Decoder need to support the latest
> specification.
> But OES_EGL_image_external extension is already supported by most of the
> GPU drivers and Video decoders.
>
> And one of the performance bottleneck is to converting EGLImage input to
> TEXTURE_2D to handled by WebGL application.
>
>
> On Tue, Oct 11, 2016 at 6:54 AM, Byungseon Shin <[email protected]>
> wrote:
>
> Hi Florian, Maksims,
>
> Thanks for the feedback.
>
> I have updated proposal to explain how application works.
> Please see the attached updated proposal.
>
> To render an update frame, we need to call ext.EGLImageTargetTexture2DOES
> to bind newly generated texture buffer from video decoder.
>
> By supporting OES_EGL_image_external extension, application can use direct
> texture when video driver provides EGLImage like OpenGL ES natvie
> applications.
>
> Our proposal is just focusing on extending format of texture compatible
> with OpenGL ES extension and does not conflict with previous proposals.
>
> So, we provides a simplest way to adopt OES_EGL_image_external extension.
>
> By using references implementations of browser, WebGL renders video more
> than 10 times faster than TEXTURE_2D format with Full HD resolution output.
>
> Kind regards,
> Byungseon Shin
>
> On Tue, Oct 11, 2016 at 4:51 PM Maksims Mihejevs <[email protected]>
> wrote:
>
> Worth mentioning that in web there are multiple sources that have
> independent redraw mechanics, that includes video as well as canvas
> elements. It might make sense to have single extension for providing direct
> access to those sources without need to re-upload it.
>
> I have assumptions that internally in browsers, both video and canvas have
> their buffers, so it might lead to very similar implementation as from
> internal side as well as from webgl api side.
>
>
> On 11 Oct 2016 12:12 p.m., "Florian Bösch" <[email protected]> wrote:
>
> Am I correct in assuming that once you've called ext.
> EGLImageTargetTexture2DOES(ext.TEXTURE_EXTERNAL_OES, videoElement); that
> the texture is "bound" to the video and no further calls need to be emitted
> (as would presently be the case) for it to stay current?
>
> If that's the case, I think it's a good idea, it takes some work off web
> application programmers to make sure the video is updated, and it should
> also in all cases avoid a texture download/reupload.
>
> There are some other extensions that have attempted to deal with this
> problem.
>
>    -
>    https://www.khronos.org/registry/webgl/extensions/rejected/WEBGL_texture_from_depth_video/
>    : this extension is rejected because it introduced a complex behavioral
>    changes/semantics whose benefits where unclear and the champions of this
>    extension stopped responding to questions and didn't offer any improvements
>    on the extension specification.
>    -
>    https://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture/
>    : this extension is currently in a proposal state and it covers similar
>    functionality as EGL_image_external. Where it fundamentally differs is that
>    it also deals with format conversions (YUV 442 to rgb etc.) and accurate
>    timing for presentation of a video frame.
>
>  Perhaps you could elaborate a little why OES_EGL_image_external would be
> preferrable to WEBGL_dynamic_texture, and how the issues that
> EGL_image_external does not tackle (timing, format conversion, other
> things) would be handled by the web application programmer?
>
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://khronos.org/pipermail/public_webgl_khronos.org/attachments/20170106/82e5f17a/attachment.html>


More information about the public_webgl mailing list