[Public WebGL] OES_EGL_image_external Extension proposal

Kenneth Russell [email protected]
Tue Oct 11 16:48:10 PDT 2016


Hi Byungseon,

Thanks for putting together this extension proposal.

In conversations with groups within Google that are trying to do 360 degree
video rendering in WebGL, they need more information than your extension
would provide; specifically, they would need the exact timestamp of the
frame currently being rendered, as well as other per-frame metadata like
the current width and height of the texture (for variable bitrate video
streams).

Mark Callow's WEBGL_dynamic_texture extension proposal
https://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture/
provides the controls needed. However, per working group discussions, the
current extension's a little too complicated. I think if that extension
were simplified a little bit that it would provide all of the performance
benefits yours offers, and the controls that are known to be necessary.

Do you think you'd be willing to devote some time to either extending your
proposal (I can provide specific feedback) or editing down Mark's proposal
to handle these use cases? I'd prefer to edit down WEBGL_dynamic_texture,
if you're willing.

Thanks,

-Ken



On Tue, Oct 11, 2016 at 8:27 AM, Byungseon Shin <[email protected]> wrote:

> Hi Corentin,
>
> On Tue, Oct 11, 2016 at 11:16 PM Corentin Wallez <[email protected]>
> wrote:
>
>> The problem with using an EGL_image type of extension is that the texture
>> data store becomes shared between the new texture and the object it was
>> created from. This is great when the application controls both sides and
>> can ensure read and writes are probably synchronized, but in the case of
>> video here, one side would write without the application being able to
>> control synchronization.
>>
>> The standard way to do this type of video / texture binding in EGL is to
>> use EGL_KHR_stream
>> <https://www.khronos.org/registry/egl/extensions/KHR/EGL_KHR_stream.txt> which
>> is a much more complex and might not be an improvement over what developers
>> can do with the current APIs.
>>
>>
> As in proposal revision#3 and as Florian already mentioned,
> Current proposal support "bind once and update implicitly" concept :  "
> ext.EGLImageTargetTexture2DOES(ext.TEXTURE_EXTERNAL_OES, videoElement);
> that the texture is "bound" to the video and no further calls need to be
> emitted (as would presently be the case)"
>
> In a that sense, we could abstract the details implementation into
> MediaPlayer Side like synchronizing issues.
>
> Even EGL_KHR_stream provides more details synchronization issues but still
> lot's of GPU driver and Video Decoder need to support the latest
> specification.
> But OES_EGL_image_external extension is already supported by most of the
> GPU drivers and Video decoders.
>
> And one of the performance bottleneck is to converting EGLImage input to
> TEXTURE_2D to handled by WebGL application.
>
>
>> On Tue, Oct 11, 2016 at 6:54 AM, Byungseon Shin <[email protected]>
>> wrote:
>>
>> Hi Florian, Maksims,
>>
>> Thanks for the feedback.
>>
>> I have updated proposal to explain how application works.
>> Please see the attached updated proposal.
>>
>> To render an update frame, we need to call ext.EGLImageTargetTexture2DOES
>> to bind newly generated texture buffer from video decoder.
>>
>> By supporting OES_EGL_image_external extension, application can use
>> direct texture when video driver provides EGLImage like OpenGL ES natvie
>> applications.
>>
>> Our proposal is just focusing on extending format of texture compatible
>> with OpenGL ES extension and does not conflict with previous proposals.
>>
>> So, we provides a simplest way to adopt OES_EGL_image_external extension.
>>
>> By using references implementations of browser, WebGL renders video more
>> than 10 times faster than TEXTURE_2D format with Full HD resolution output.
>>
>> Kind regards,
>> Byungseon Shin
>>
>> On Tue, Oct 11, 2016 at 4:51 PM Maksims Mihejevs <[email protected]>
>> wrote:
>>
>> Worth mentioning that in web there are multiple sources that have
>> independent redraw mechanics, that includes video as well as canvas
>> elements. It might make sense to have single extension for providing direct
>> access to those sources without need to re-upload it.
>>
>> I have assumptions that internally in browsers, both video and canvas
>> have their buffers, so it might lead to very similar implementation as from
>> internal side as well as from webgl api side.
>>
>>
>> On 11 Oct 2016 12:12 p.m., "Florian Bösch" <[email protected]> wrote:
>>
>> Am I correct in assuming that once you've called ext.EGLImageTargetTextu
>> re2DOES(ext.TEXTURE_EXTERNAL_OES, videoElement); that the texture is
>> "bound" to the video and no further calls need to be emitted (as would
>> presently be the case) for it to stay current?
>>
>> If that's the case, I think it's a good idea, it takes some work off web
>> application programmers to make sure the video is updated, and it should
>> also in all cases avoid a texture download/reupload.
>>
>> There are some other extensions that have attempted to deal with this
>> problem.
>>
>>    - https://www.khronos.org/registry/webgl/extensions/
>>    rejected/WEBGL_texture_from_depth_video/
>>    <https://www.khronos.org/registry/webgl/extensions/rejected/WEBGL_texture_from_depth_video/>
>>    : this extension is rejected because it introduced a complex behavioral
>>    changes/semantics whose benefits where unclear and the champions of this
>>    extension stopped responding to questions and didn't offer any improvements
>>    on the extension specification.
>>    - https://www.khronos.org/registry/webgl/extensions/
>>    proposals/WEBGL_dynamic_texture/
>>    <https://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture/>
>>    : this extension is currently in a proposal state and it covers similar
>>    functionality as EGL_image_external. Where it fundamentally differs is that
>>    it also deals with format conversions (YUV 442 to rgb etc.) and accurate
>>    timing for presentation of a video frame.
>>
>>  Perhaps you could elaborate a little why OES_EGL_image_external would be
>> preferrable to WEBGL_dynamic_texture, and how the issues that
>> EGL_image_external does not tackle (timing, format conversion, other
>> things) would be handled by the web application programmer?
>>
>>
>>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://khronos.org/pipermail/public_webgl_khronos.org/attachments/20161011/2a61340a/attachment.html>


More information about the public_webgl mailing list