[Public WebGL] OES_EGL_image_external Extension proposal

Florian Bösch [email protected]
Fri Jan 6 11:14:48 PST 2017


Unless I'm mistaken, knowing the time alone isn't going to help that much.

The dynamic_texture extension attempted to solve this problem with setting
a presentation time for a frame. I'm not sure how realistic it is to try to
time presentation time, so that might have been overly ambitious.

On Fri, Jan 6, 2017 at 8:01 PM, Byungseon Shin <[email protected]> wrote:

> Hi WebGL working group members,
>
> We have updated proposal by adding following function :
> - Provides time of frame, texture width and height of HTMLVideoElement's
> EGLImage.
>
> Please take a look at the pull request:
> <https://github.com/KhronosGroup/WebGL/pull/2243>
>
> Kind regards,
> Byungseon Shin
>
>
>
> On Tue, Jan 3, 2017 at 8:48 AM Byungseon Shin <[email protected]> wrote:
>
>> Hi Florian,
>>
>> We are working on update which will cover dynamic_texture concerns.
>> I plan to upload new version on github for reviewing by this week.
>>
>> Happy 2017 year!
>> Byungseon Shin
>>
>> On Tue, Jan 3, 2017 at 1:58 AM Florian Bösch <[email protected]> wrote:
>>
>> What is the status of the https://www.khronos.org/
>> registry/webgl/extensions/proposals/OES_EGL_image_external/ extension?
>> Is work on the spec still going on or is this the final version that takes
>> into account dynamic_texture concerns?
>>
>> On Wed, Oct 12, 2016 at 3:59 AM, Byungseon Shin <[email protected]>
>> wrote:
>>
>> Hi Ken,
>>
>> Thanks for the feedback.
>> Of course, we are willing to devote time and eager to enhance WebGL video
>> rendering performance.
>> We prefer to use our proposal as a base but we could revise
>> dynamic_texture as well.
>>
>> Kind regards,
>> Byungseon Shin
>>
>>
>> On Wed, Oct 12, 2016 at 8:48 AM Kenneth Russell <[email protected]> wrote:
>>
>> Hi Byungseon,
>>
>> Thanks for putting together this extension proposal.
>>
>> In conversations with groups within Google that are trying to do 360
>> degree video rendering in WebGL, they need more information than your
>> extension would provide; specifically, they would need the exact timestamp
>> of the frame currently being rendered, as well as other per-frame metadata
>> like the current width and height of the texture (for variable bitrate
>> video streams).
>>
>> Mark Callow's WEBGL_dynamic_texture extension proposal
>> https://www.khronos.org/registry/webgl/extensions/
>> proposals/WEBGL_dynamic_texture/ provides the controls needed. However,
>> per working group discussions, the current extension's a little too
>> complicated. I think if that extension were simplified a little bit that it
>> would provide all of the performance benefits yours offers, and the
>> controls that are known to be necessary.
>>
>> Do you think you'd be willing to devote some time to either extending
>> your proposal (I can provide specific feedback) or editing down Mark's
>> proposal to handle these use cases? I'd prefer to edit down
>> WEBGL_dynamic_texture, if you're willing.
>>
>> Thanks,
>>
>> -Ken
>>
>>
>>
>> On Tue, Oct 11, 2016 at 8:27 AM, Byungseon Shin <[email protected]>
>> wrote:
>>
>> Hi Corentin,
>>
>> On Tue, Oct 11, 2016 at 11:16 PM Corentin Wallez <[email protected]>
>> wrote:
>>
>> The problem with using an EGL_image type of extension is that the texture
>> data store becomes shared between the new texture and the object it was
>> created from. This is great when the application controls both sides and
>> can ensure read and writes are probably synchronized, but in the case of
>> video here, one side would write without the application being able to
>> control synchronization.
>>
>> The standard way to do this type of video / texture binding in EGL is to
>> use EGL_KHR_stream
>> <https://www.khronos.org/registry/egl/extensions/KHR/EGL_KHR_stream.txt> which
>> is a much more complex and might not be an improvement over what developers
>> can do with the current APIs.
>>
>>
>> As in proposal revision#3 and as Florian already mentioned,
>> Current proposal support "bind once and update implicitly" concept :  "
>> ext.EGLImageTargetTexture2DOES(ext.TEXTURE_EXTERNAL_OES, videoElement);
>> that the texture is "bound" to the video and no further calls need to be
>> emitted (as would presently be the case)"
>>
>> In a that sense, we could abstract the details implementation into
>> MediaPlayer Side like synchronizing issues.
>>
>> Even EGL_KHR_stream provides more details synchronization issues but
>> still lot's of GPU driver and Video Decoder need to support the latest
>> specification.
>> But OES_EGL_image_external extension is already supported by most of the
>> GPU drivers and Video decoders.
>>
>> And one of the performance bottleneck is to converting EGLImage input to
>> TEXTURE_2D to handled by WebGL application.
>>
>>
>> On Tue, Oct 11, 2016 at 6:54 AM, Byungseon Shin <[email protected]>
>> wrote:
>>
>> Hi Florian, Maksims,
>>
>> Thanks for the feedback.
>>
>> I have updated proposal to explain how application works.
>> Please see the attached updated proposal.
>>
>> To render an update frame, we need to call ext.EGLImageTargetTexture2DOES
>> to bind newly generated texture buffer from video decoder.
>>
>> By supporting OES_EGL_image_external extension, application can use
>> direct texture when video driver provides EGLImage like OpenGL ES natvie
>> applications.
>>
>> Our proposal is just focusing on extending format of texture compatible
>> with OpenGL ES extension and does not conflict with previous proposals.
>>
>> So, we provides a simplest way to adopt OES_EGL_image_external extension.
>>
>> By using references implementations of browser, WebGL renders video more
>> than 10 times faster than TEXTURE_2D format with Full HD resolution output.
>>
>> Kind regards,
>> Byungseon Shin
>>
>> On Tue, Oct 11, 2016 at 4:51 PM Maksims Mihejevs <[email protected]>
>> wrote:
>>
>> Worth mentioning that in web there are multiple sources that have
>> independent redraw mechanics, that includes video as well as canvas
>> elements. It might make sense to have single extension for providing direct
>> access to those sources without need to re-upload it.
>>
>> I have assumptions that internally in browsers, both video and canvas
>> have their buffers, so it might lead to very similar implementation as from
>> internal side as well as from webgl api side.
>>
>>
>> On 11 Oct 2016 12:12 p.m., "Florian Bösch" <[email protected]> wrote:
>>
>> Am I correct in assuming that once you've called ext.EGLImageTargetTextu
>> re2DOES(ext.TEXTURE_EXTERNAL_OES, videoElement); that the texture is
>> "bound" to the video and no further calls need to be emitted (as would
>> presently be the case) for it to stay current?
>>
>> If that's the case, I think it's a good idea, it takes some work off web
>> application programmers to make sure the video is updated, and it should
>> also in all cases avoid a texture download/reupload.
>>
>> There are some other extensions that have attempted to deal with this
>> problem.
>>
>>    - https://www.khronos.org/registry/webgl/extensions/
>>    rejected/WEBGL_texture_from_depth_video/
>>    <https://www.khronos.org/registry/webgl/extensions/rejected/WEBGL_texture_from_depth_video/>
>>    : this extension is rejected because it introduced a complex behavioral
>>    changes/semantics whose benefits where unclear and the champions of this
>>    extension stopped responding to questions and didn't offer any improvements
>>    on the extension specification.
>>    - https://www.khronos.org/registry/webgl/extensions/
>>    proposals/WEBGL_dynamic_texture/
>>    <https://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture/>
>>    : this extension is currently in a proposal state and it covers similar
>>    functionality as EGL_image_external. Where it fundamentally differs is that
>>    it also deals with format conversions (YUV 442 to rgb etc.) and accurate
>>    timing for presentation of a video frame.
>>
>>  Perhaps you could elaborate a little why OES_EGL_image_external would be
>> preferrable to WEBGL_dynamic_texture, and how the issues that
>> EGL_image_external does not tackle (timing, format conversion, other
>> things) would be handled by the web application programmer?
>>
>>
>>
>>
>>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://khronos.org/pipermail/public_webgl_khronos.org/attachments/20170106/61216ddd/attachment.html>


More information about the public_webgl mailing list