From pya...@ Sat Oct 1 02:00:21 2016 From: pya...@ (=?UTF-8?Q?Florian_B=C3=B6sch?=) Date: Sat, 1 Oct 2016 11:00:21 +0200 Subject: [Public WebGL] Interaction of WebGL 2 query objects and WebGL timer queries extension EXT_disjoint_timer_query In-Reply-To: References: Message-ID: That'll work. I personally find it a bit clumsy to have to have two extensions to do it. On the other hand, it wouldn't be good if there was an extension that had different stuff on it depending if you got it from a webgl1 or webgl2 context. Are there other extensions that presently exist in WebGL1 which have a partial coverage in WebGL2 which would need a _webgl2 version? On Sat, Oct 1, 2016 at 5:21 AM, Kenneth Russell wrote: > Thanks for your feedback. The working group discussed this and proposes to > expose a slimmed-down IDL for WebGL 2.0 under the same extension name that > reuses all of the core context's query functionality. Please review and > comment: > > https://github.com/KhronosGroup/WebGL/pull/2076 > > Thanks, > > -Ken > > > On Thu, Sep 29, 2016 at 3:39 AM, Florian B?sch wrote: > >> Yeah that's the thing I mean. It needs to get resolved and unambiguously >> (and preferably following GL semantics) or tool chains spring into >> existence doing the wrong things. >> >> On Thu, Sep 29, 2016 at 11:18 AM, Jukka Jyl?nki wrote: >> >>> For reference, here is the feature added to Emscripten: >>> https://github.com/kripken/emscripten/pull/4575 >>> >>> 2016-09-29 2:04 GMT+03:00 Kenneth Russell : >>> >>>> On Mon, Sep 26, 2016 at 4:03 PM, Florian B?sch >>>> wrote: >>>> >>>>> On Tue, Sep 27, 2016 at 12:41 AM, Kenneth Russell >>>>> wrote: >>>>> >>>>>> On Sun, Sep 25, 2016 at 1:34 AM, Florian B?sch >>>>>> wrote: >>>>>> >>>>>>> On Sun, Sep 25, 2016 at 5:37 AM, Kenneth Russell >>>>>>> wrote: >>>>>>>> >>>>>>>> Good question. Yes, they should be considered separate. >>>>>>>> >>>>>>> >>>>>>> I don't agree with that interpretation. >>>>>>> >>>>>> >>>>>> The IDL is clear. As I mentioned, it could be modified, but right now >>>>>> WebGLTimerQueryEXT and WebGLQuery are distinct types. >>>>>> >>>>> >>>>> I mean the interpretation that they should be considered separate, not >>>>> what the IDL says they should be. >>>>> >>>>> >>>>>> >>>>>>> On Fri, Sep 23, 2016 at 7:20 PM, Geoff Lang >>>>>>> wrote: >>>>>>> >>>>>>>> For OpenGL in general, glGenQueries and glGenQueriesEXT are >>>>>>>> allowed to be the same function pointer >>>>>>>> >>>>>>> >>>>>>> glBeginQuery, glBeginQueryEXT (disjoint timer) and glBeginQueryEXT >>>>>>> (occlusion) are clearly returning the same kind of object, and they're >>>>>>> identical. >>>>>>> >>>>>>> It would be quite unfortunate if you had to track which extension >>>>>>> object you operated with in order to know which interface to use with that >>>>>>> particular extension object. If we where to interprete it that way, then >>>>>>> the API interface to these functions should not reside on the extension >>>>>>> object, but be on the query object. As in: >>>>>>> >>>>>>> query = disjoint_ext.create(); >>>>>>> query.begin(); >>>>>>> query.end(); >>>>>>> >>>>>> >>>>>> With occlusion queries being folded into the core WebGL 2.0 spec, the >>>>>> only outlier is EXT_disjoint_timer_query. >>>>>> >>>>>> We can revisit this after the first iteration of WebGL 2.0 ships. >>>>>> >>>>> >>>>> I think that's too late. It's my opinion that following the OpenGL >>>>> semantic on symbols that these two objects should be considered the same. >>>>> It wouldn't be good if code emerged that treated them as if they wheren't, >>>>> if we intend to make it so they aren't, and vice versa. >>>>> >>>> >>>> This is a good point. The working group will discuss this on tomorrow's >>>> conference call and I'll file an issue on the KhronosGroup/WebGL issue >>>> tracker assuming we decide to make a change to the EXT_disjoint_timer_query >>>> spec, IDL, and conformance test. >>>> >>>> -Ken >>>> >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kbr...@ Mon Oct 3 13:44:19 2016 From: kbr...@ (Kenneth Russell) Date: Mon, 3 Oct 2016 13:44:19 -0700 Subject: [Public WebGL] Interaction of WebGL 2 query objects and WebGL timer queries extension EXT_disjoint_timer_query In-Reply-To: References: Message-ID: On Sat, Oct 1, 2016 at 2:00 AM, Florian B?sch wrote: > That'll work. I personally find it a bit clumsy to have to have two > extensions to do it. On the other hand, it wouldn't be good if there was an > extension that had different stuff on it depending if you got it from a > webgl1 or webgl2 context. > Actually, the current intention was to fetch it via the same extension name. Note the text in EXT_disjoint_timer_query_webgl2/extension.xml in https://github.com/KhronosGroup/WebGL/pull/2076 . If this is undesirable, and it would be better to actually fetch it via 'EXT_disjoint_timer_query_webgl2', please let me know -- or comment on the pull request. Are there other extensions that presently exist in WebGL1 which have a > partial coverage in WebGL2 which would need a _webgl2 version? > This is the only problematic one. Most of the WebGL 1 extensions are folded into the core and aren't exposed on the WebGL 2 context. Here's the full list of extensions exposed in WebGL 2 in Chrome Canary on macOS: EXT_color_buffer_float EXT_disjoint_timer_query EXT_texture_filter_anisotropic OES_texture_float_linear WEBGL_compressed_texture_s3tc WEBGL_debug_renderer_info WEBGL_debug_shaders WEBGL_lose_context The others are pretty trivial. -Ken > > On Sat, Oct 1, 2016 at 5:21 AM, Kenneth Russell wrote: > >> Thanks for your feedback. The working group discussed this and proposes >> to expose a slimmed-down IDL for WebGL 2.0 under the same extension name >> that reuses all of the core context's query functionality. Please review >> and comment: >> >> https://github.com/KhronosGroup/WebGL/pull/2076 >> >> Thanks, >> >> -Ken >> >> >> On Thu, Sep 29, 2016 at 3:39 AM, Florian B?sch wrote: >> >>> Yeah that's the thing I mean. It needs to get resolved and unambiguously >>> (and preferably following GL semantics) or tool chains spring into >>> existence doing the wrong things. >>> >>> On Thu, Sep 29, 2016 at 11:18 AM, Jukka Jyl?nki >>> wrote: >>> >>>> For reference, here is the feature added to Emscripten: >>>> https://github.com/kripken/emscripten/pull/4575 >>>> >>>> 2016-09-29 2:04 GMT+03:00 Kenneth Russell : >>>> >>>>> On Mon, Sep 26, 2016 at 4:03 PM, Florian B?sch >>>>> wrote: >>>>> >>>>>> On Tue, Sep 27, 2016 at 12:41 AM, Kenneth Russell >>>>>> wrote: >>>>>> >>>>>>> On Sun, Sep 25, 2016 at 1:34 AM, Florian B?sch >>>>>>> wrote: >>>>>>> >>>>>>>> On Sun, Sep 25, 2016 at 5:37 AM, Kenneth Russell >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Good question. Yes, they should be considered separate. >>>>>>>>> >>>>>>>> >>>>>>>> I don't agree with that interpretation. >>>>>>>> >>>>>>> >>>>>>> The IDL is clear. As I mentioned, it could be modified, but right >>>>>>> now WebGLTimerQueryEXT and WebGLQuery are distinct types. >>>>>>> >>>>>> >>>>>> I mean the interpretation that they should be considered separate, >>>>>> not what the IDL says they should be. >>>>>> >>>>>> >>>>>>> >>>>>>>> On Fri, Sep 23, 2016 at 7:20 PM, Geoff Lang >>>>>>>> wrote: >>>>>>>> >>>>>>>>> For OpenGL in general, glGenQueries and glGenQueriesEXT are >>>>>>>>> allowed to be the same function pointer >>>>>>>>> >>>>>>>> >>>>>>>> glBeginQuery, glBeginQueryEXT (disjoint timer) and glBeginQueryEXT >>>>>>>> (occlusion) are clearly returning the same kind of object, and they're >>>>>>>> identical. >>>>>>>> >>>>>>>> It would be quite unfortunate if you had to track which extension >>>>>>>> object you operated with in order to know which interface to use with that >>>>>>>> particular extension object. If we where to interprete it that way, then >>>>>>>> the API interface to these functions should not reside on the extension >>>>>>>> object, but be on the query object. As in: >>>>>>>> >>>>>>>> query = disjoint_ext.create(); >>>>>>>> query.begin(); >>>>>>>> query.end(); >>>>>>>> >>>>>>> >>>>>>> With occlusion queries being folded into the core WebGL 2.0 spec, >>>>>>> the only outlier is EXT_disjoint_timer_query. >>>>>>> >>>>>>> We can revisit this after the first iteration of WebGL 2.0 ships. >>>>>>> >>>>>> >>>>>> I think that's too late. It's my opinion that following the OpenGL >>>>>> semantic on symbols that these two objects should be considered the same. >>>>>> It wouldn't be good if code emerged that treated them as if they wheren't, >>>>>> if we intend to make it so they aren't, and vice versa. >>>>>> >>>>> >>>>> This is a good point. The working group will discuss this on >>>>> tomorrow's conference call and I'll file an issue on the KhronosGroup/WebGL >>>>> issue tracker assuming we decide to make a change to the >>>>> EXT_disjoint_timer_query spec, IDL, and conformance test. >>>>> >>>>> -Ken >>>>> >>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sun...@ Mon Oct 10 23:19:03 2016 From: sun...@ (Byungseon Shin) Date: Tue, 11 Oct 2016 06:19:03 +0000 Subject: [Public WebGL] OES_EGL_image_external Extension proposal Message-ID: To WebGL working group members, LGE proposes to add OES_EGL_image_external Extension at WebGL Specification. Please see the attached proposal. Kind regards, Byungseon Shin ----------------------------------------------------- Software Architect TV Web Service Team LG Electronics Inc. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: From pya...@ Mon Oct 10 23:40:39 2016 From: pya...@ (=?UTF-8?Q?Florian_B=C3=B6sch?=) Date: Tue, 11 Oct 2016 08:40:39 +0200 Subject: [Public WebGL] OES_EGL_image_external Extension proposal In-Reply-To: References: Message-ID: Am I correct in assuming that once you've called ext. EGLImageTargetTexture2DOES(ext.TEXTURE_EXTERNAL_OES, videoElement); that the texture is "bound" to the video and no further calls need to be emitted (as would presently be the case) for it to stay current? If that's the case, I think it's a good idea, it takes some work off web application programmers to make sure the video is updated, and it should also in all cases avoid a texture download/reupload. There are some other extensions that have attempted to deal with this problem. - https://www.khronos.org/registry/webgl/extensions/rejected/WEBGL_texture_from_depth_video/ : this extension is rejected because it introduced a complex behavioral changes/semantics whose benefits where unclear and the champions of this extension stopped responding to questions and didn't offer any improvements on the extension specification. - https://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture/ : this extension is currently in a proposal state and it covers similar functionality as EGL_image_external. Where it fundamentally differs is that it also deals with format conversions (YUV 442 to rgb etc.) and accurate timing for presentation of a video frame. Perhaps you could elaborate a little why OES_EGL_image_external would be preferrable to WEBGL_dynamic_texture, and how the issues that EGL_image_external does not tackle (timing, format conversion, other things) would be handled by the web application programmer? -------------- next part -------------- An HTML attachment was scrubbed... URL: From max...@ Tue Oct 11 00:51:26 2016 From: max...@ (Maksims Mihejevs) Date: Tue, 11 Oct 2016 08:51:26 +0100 Subject: [Public WebGL] OES_EGL_image_external Extension proposal In-Reply-To: References: Message-ID: Worth mentioning that in web there are multiple sources that have independent redraw mechanics, that includes video as well as canvas elements. It might make sense to have single extension for providing direct access to those sources without need to re-upload it. I have assumptions that internally in browsers, both video and canvas have their buffers, so it might lead to very similar implementation as from internal side as well as from webgl api side. On 11 Oct 2016 12:12 p.m., "Florian B?sch" wrote: > Am I correct in assuming that once you've called ext.EGLImageTargetTextu > re2DOES(ext.TEXTURE_EXTERNAL_OES, videoElement); that the texture is > "bound" to the video and no further calls need to be emitted (as would > presently be the case) for it to stay current? > > If that's the case, I think it's a good idea, it takes some work off web > application programmers to make sure the video is updated, and it should > also in all cases avoid a texture download/reupload. > > There are some other extensions that have attempted to deal with this > problem. > > - https://www.khronos.org/registry/webgl/extensions/ > rejected/WEBGL_texture_from_depth_video/ > > : this extension is rejected because it introduced a complex behavioral > changes/semantics whose benefits where unclear and the champions of this > extension stopped responding to questions and didn't offer any improvements > on the extension specification. > - https://www.khronos.org/registry/webgl/extensions/ > proposals/WEBGL_dynamic_texture/ > > : this extension is currently in a proposal state and it covers similar > functionality as EGL_image_external. Where it fundamentally differs is that > it also deals with format conversions (YUV 442 to rgb etc.) and accurate > timing for presentation of a video frame. > > Perhaps you could elaborate a little why OES_EGL_image_external would be > preferrable to WEBGL_dynamic_texture, and how the issues that > EGL_image_external does not tackle (timing, format conversion, other > things) would be handled by the web application programmer? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sun...@ Tue Oct 11 03:54:34 2016 From: sun...@ (Byungseon Shin) Date: Tue, 11 Oct 2016 10:54:34 +0000 Subject: [Public WebGL] OES_EGL_image_external Extension proposal In-Reply-To: References: Message-ID: Hi Florian, Maksims, Thanks for the feedback. I have updated proposal to explain how application works. Please see the attached updated proposal. To render an update frame, we need to call ext.EGLImageTargetTexture2DOES to bind newly generated texture buffer from video decoder. By supporting OES_EGL_image_external extension, application can use direct texture when video driver provides EGLImage like OpenGL ES natvie applications. Our proposal is just focusing on extending format of texture compatible with OpenGL ES extension and does not conflict with previous proposals. So, we provides a simplest way to adopt OES_EGL_image_external extension. By using references implementations of browser, WebGL renders video more than 10 times faster than TEXTURE_2D format with Full HD resolution output. Kind regards, Byungseon Shin On Tue, Oct 11, 2016 at 4:51 PM Maksims Mihejevs wrote: > Worth mentioning that in web there are multiple sources that have > independent redraw mechanics, that includes video as well as canvas > elements. It might make sense to have single extension for providing direct > access to those sources without need to re-upload it. > > I have assumptions that internally in browsers, both video and canvas have > their buffers, so it might lead to very similar implementation as from > internal side as well as from webgl api side. > > > On 11 Oct 2016 12:12 p.m., "Florian B?sch" wrote: > > Am I correct in assuming that once you've called ext. > EGLImageTargetTexture2DOES(ext.TEXTURE_EXTERNAL_OES, videoElement); that > the texture is "bound" to the video and no further calls need to be emitted > (as would presently be the case) for it to stay current? > > If that's the case, I think it's a good idea, it takes some work off web > application programmers to make sure the video is updated, and it should > also in all cases avoid a texture download/reupload. > > There are some other extensions that have attempted to deal with this > problem. > > - > https://www.khronos.org/registry/webgl/extensions/rejected/WEBGL_texture_from_depth_video/ > : this extension is rejected because it introduced a complex behavioral > changes/semantics whose benefits where unclear and the champions of this > extension stopped responding to questions and didn't offer any improvements > on the extension specification. > - > https://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture/ > : this extension is currently in a proposal state and it covers similar > functionality as EGL_image_external. Where it fundamentally differs is that > it also deals with format conversions (YUV 442 to rgb etc.) and accurate > timing for presentation of a video frame. > > Perhaps you could elaborate a little why OES_EGL_image_external would be > preferrable to WEBGL_dynamic_texture, and how the issues that > EGL_image_external does not tackle (timing, format conversion, other > things) would be handled by the web application programmer? > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: From cwa...@ Tue Oct 11 07:16:23 2016 From: cwa...@ (Corentin Wallez) Date: Tue, 11 Oct 2016 10:16:23 -0400 Subject: [Public WebGL] OES_EGL_image_external Extension proposal In-Reply-To: References: Message-ID: The problem with using an EGL_image type of extension is that the texture data store becomes shared between the new texture and the object it was created from. This is great when the application controls both sides and can ensure read and writes are probably synchronized, but in the case of video here, one side would write without the application being able to control synchronization. The standard way to do this type of video / texture binding in EGL is to use EGL_KHR_stream which is a much more complex and might not be an improvement over what developers can do with the current APIs. On Tue, Oct 11, 2016 at 6:54 AM, Byungseon Shin wrote: > Hi Florian, Maksims, > > Thanks for the feedback. > > I have updated proposal to explain how application works. > Please see the attached updated proposal. > > To render an update frame, we need to call ext.EGLImageTargetTexture2DOES > to bind newly generated texture buffer from video decoder. > > By supporting OES_EGL_image_external extension, application can use direct > texture when video driver provides EGLImage like OpenGL ES natvie > applications. > > Our proposal is just focusing on extending format of texture compatible > with OpenGL ES extension and does not conflict with previous proposals. > > So, we provides a simplest way to adopt OES_EGL_image_external extension. > > By using references implementations of browser, WebGL renders video more > than 10 times faster than TEXTURE_2D format with Full HD resolution output. > > Kind regards, > Byungseon Shin > > On Tue, Oct 11, 2016 at 4:51 PM Maksims Mihejevs > wrote: > >> Worth mentioning that in web there are multiple sources that have >> independent redraw mechanics, that includes video as well as canvas >> elements. It might make sense to have single extension for providing direct >> access to those sources without need to re-upload it. >> >> I have assumptions that internally in browsers, both video and canvas >> have their buffers, so it might lead to very similar implementation as from >> internal side as well as from webgl api side. >> >> >> On 11 Oct 2016 12:12 p.m., "Florian B?sch" wrote: >> >> Am I correct in assuming that once you've called ext.EGLImageTargetTextu >> re2DOES(ext.TEXTURE_EXTERNAL_OES, videoElement); that the texture is >> "bound" to the video and no further calls need to be emitted (as would >> presently be the case) for it to stay current? >> >> If that's the case, I think it's a good idea, it takes some work off web >> application programmers to make sure the video is updated, and it should >> also in all cases avoid a texture download/reupload. >> >> There are some other extensions that have attempted to deal with this >> problem. >> >> - https://www.khronos.org/registry/webgl/extensions/ >> rejected/WEBGL_texture_from_depth_video/ >> >> : this extension is rejected because it introduced a complex behavioral >> changes/semantics whose benefits where unclear and the champions of this >> extension stopped responding to questions and didn't offer any improvements >> on the extension specification. >> - https://www.khronos.org/registry/webgl/extensions/ >> proposals/WEBGL_dynamic_texture/ >> >> : this extension is currently in a proposal state and it covers similar >> functionality as EGL_image_external. Where it fundamentally differs is that >> it also deals with format conversions (YUV 442 to rgb etc.) and accurate >> timing for presentation of a video frame. >> >> Perhaps you could elaborate a little why OES_EGL_image_external would be >> preferrable to WEBGL_dynamic_texture, and how the issues that >> EGL_image_external does not tackle (timing, format conversion, other >> things) would be handled by the web application programmer? >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From pya...@ Tue Oct 11 07:48:40 2016 From: pya...@ (=?UTF-8?Q?Florian_B=C3=B6sch?=) Date: Tue, 11 Oct 2016 16:48:40 +0200 Subject: [Public WebGL] OES_EGL_image_external Extension proposal In-Reply-To: References: Message-ID: The proposed specification mentions: Written against the WebGL API 1.0 > specification. > Promoted to core and no longer available as an extension in WebGL API 2.0 > specification. The present WebGL 2.0 specification draft does not contain these EGL interfaces: https://www.khronos.org/registry/webgl/specs/latest/2.0/ and neither does the OpenGL ES 3.0 specification: https://www.khronos.org/files/opengles3-quick-reference-card.pdf Could you clarify? Another question I have is, how do you handle YUV and such as are commonly provided by video decoders? By using references implementations of browser, WebGL renders video more > than 10 times faster than TEXTURE_2D format with Full HD resolution output. I'm afraid I don't quite follow why this would only be possible with this extension. Would it not be possible for a WebGL implementation to use EGL texture external and reinterprete a texImage2D call to call that function? On Tue, Oct 11, 2016 at 12:54 PM, Byungseon Shin wrote: > Hi Florian, Maksims, > > Thanks for the feedback. > > I have updated proposal to explain how application works. > Please see the attached updated proposal. > > To render an update frame, we need to call ext.EGLImageTargetTexture2DOES > to bind newly generated texture buffer from video decoder. > > By supporting OES_EGL_image_external extension, application can use direct > texture when video driver provides EGLImage like OpenGL ES natvie > applications. > > Our proposal is just focusing on extending format of texture compatible > with OpenGL ES extension and does not conflict with previous proposals. > > So, we provides a simplest way to adopt OES_EGL_image_external extension. > > By using references implementations of browser, WebGL renders video more > than 10 times faster than TEXTURE_2D format with Full HD resolution output. > > Kind regards, > Byungseon Shin > > On Tue, Oct 11, 2016 at 4:51 PM Maksims Mihejevs > wrote: > >> Worth mentioning that in web there are multiple sources that have >> independent redraw mechanics, that includes video as well as canvas >> elements. It might make sense to have single extension for providing direct >> access to those sources without need to re-upload it. >> >> I have assumptions that internally in browsers, both video and canvas >> have their buffers, so it might lead to very similar implementation as from >> internal side as well as from webgl api side. >> >> >> On 11 Oct 2016 12:12 p.m., "Florian B?sch" wrote: >> >> Am I correct in assuming that once you've called ext.EGLImageTargetTextu >> re2DOES(ext.TEXTURE_EXTERNAL_OES, videoElement); that the texture is >> "bound" to the video and no further calls need to be emitted (as would >> presently be the case) for it to stay current? >> >> If that's the case, I think it's a good idea, it takes some work off web >> application programmers to make sure the video is updated, and it should >> also in all cases avoid a texture download/reupload. >> >> There are some other extensions that have attempted to deal with this >> problem. >> >> - https://www.khronos.org/registry/webgl/extensions/ >> rejected/WEBGL_texture_from_depth_video/ >> >> : this extension is rejected because it introduced a complex behavioral >> changes/semantics whose benefits where unclear and the champions of this >> extension stopped responding to questions and didn't offer any improvements >> on the extension specification. >> - https://www.khronos.org/registry/webgl/extensions/ >> proposals/WEBGL_dynamic_texture/ >> >> : this extension is currently in a proposal state and it covers similar >> functionality as EGL_image_external. Where it fundamentally differs is that >> it also deals with format conversions (YUV 442 to rgb etc.) and accurate >> timing for presentation of a video frame. >> >> Perhaps you could elaborate a little why OES_EGL_image_external would be >> preferrable to WEBGL_dynamic_texture, and how the issues that >> EGL_image_external does not tackle (timing, format conversion, other >> things) would be handled by the web application programmer? >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From sun...@ Tue Oct 11 08:10:54 2016 From: sun...@ (Byungseon Shin) Date: Tue, 11 Oct 2016 15:10:54 +0000 Subject: [Public WebGL] OES_EGL_image_external Extension proposal In-Reply-To: References: Message-ID: Hi Florian, On Tue, Oct 11, 2016 at 11:48 PM Florian B?sch wrote: > The proposed specification mentions: > > Written against the WebGL API 1.0 > specification. > Promoted to core and no longer available as an extension in WebGL API 2.0 > specification. > > > The present WebGL 2.0 specification draft does not contain these EGL > interfaces: https://www.khronos.org/registry/webgl/specs/latest/2.0/ and > neither does the OpenGL ES 3.0 specification: > https://www.khronos.org/files/opengles3-quick-reference-card.pdf > > Could you clarify? > > I mistakenly left the string in the template and fixed at attached proposal revision #3. > Another question I have is, how do you handle YUV and such as are commonly > provided by video decoders? > > Video decoder output shared as a form of EGLImage as a YUV format. > By using references implementations of browser, WebGL renders video more > than 10 times faster than TEXTURE_2D format with Full HD resolution output. > > > I'm afraid I don't quite follow why this would only be possible with this > extension. Would it not be possible for a WebGL implementation to use EGL > texture external and reinterprete a texImage2D call to call that function? > > > Yes, it could be possibly implemented, but we want to support both OES_EGL_external texture and TEXTURE_2D. And eventually share the sample code to our partner. > > > > On Tue, Oct 11, 2016 at 12:54 PM, Byungseon Shin > wrote: > > Hi Florian, Maksims, > > Thanks for the feedback. > > I have updated proposal to explain how application works. > Please see the attached updated proposal. > > To render an update frame, we need to call ext.EGLImageTargetTexture2DOES > to bind newly generated texture buffer from video decoder. > > By supporting OES_EGL_image_external extension, application can use direct > texture when video driver provides EGLImage like OpenGL ES natvie > applications. > > Our proposal is just focusing on extending format of texture compatible > with OpenGL ES extension and does not conflict with previous proposals. > > So, we provides a simplest way to adopt OES_EGL_image_external extension. > > By using references implementations of browser, WebGL renders video more > than 10 times faster than TEXTURE_2D format with Full HD resolution output. > > Kind regards, > Byungseon Shin > > On Tue, Oct 11, 2016 at 4:51 PM Maksims Mihejevs > wrote: > > Worth mentioning that in web there are multiple sources that have > independent redraw mechanics, that includes video as well as canvas > elements. It might make sense to have single extension for providing direct > access to those sources without need to re-upload it. > > I have assumptions that internally in browsers, both video and canvas have > their buffers, so it might lead to very similar implementation as from > internal side as well as from webgl api side. > > > On 11 Oct 2016 12:12 p.m., "Florian B?sch" wrote: > > Am I correct in assuming that once you've called ext. > EGLImageTargetTexture2DOES(ext.TEXTURE_EXTERNAL_OES, videoElement); that > the texture is "bound" to the video and no further calls need to be emitted > (as would presently be the case) for it to stay current? > > If that's the case, I think it's a good idea, it takes some work off web > application programmers to make sure the video is updated, and it should > also in all cases avoid a texture download/reupload. > > There are some other extensions that have attempted to deal with this > problem. > > - > https://www.khronos.org/registry/webgl/extensions/rejected/WEBGL_texture_from_depth_video/ > : this extension is rejected because it introduced a complex behavioral > changes/semantics whose benefits where unclear and the champions of this > extension stopped responding to questions and didn't offer any improvements > on the extension specification. > - > https://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture/ > : this extension is currently in a proposal state and it covers similar > functionality as EGL_image_external. Where it fundamentally differs is that > it also deals with format conversions (YUV 442 to rgb etc.) and accurate > timing for presentation of a video frame. > > Perhaps you could elaborate a little why OES_EGL_image_external would be > preferrable to WEBGL_dynamic_texture, and how the issues that > EGL_image_external does not tackle (timing, format conversion, other > things) would be handled by the web application programmer? > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: From sun...@ Tue Oct 11 08:27:25 2016 From: sun...@ (Byungseon Shin) Date: Tue, 11 Oct 2016 15:27:25 +0000 Subject: [Public WebGL] OES_EGL_image_external Extension proposal In-Reply-To: References: Message-ID: Hi Corentin, On Tue, Oct 11, 2016 at 11:16 PM Corentin Wallez wrote: > The problem with using an EGL_image type of extension is that the texture > data store becomes shared between the new texture and the object it was > created from. This is great when the application controls both sides and > can ensure read and writes are probably synchronized, but in the case of > video here, one side would write without the application being able to > control synchronization. > > The standard way to do this type of video / texture binding in EGL is to > use EGL_KHR_stream > which > is a much more complex and might not be an improvement over what developers > can do with the current APIs. > > As in proposal revision#3 and as Florian already mentioned, Current proposal support "bind once and update implicitly" concept : "ext.EGLImageTargetTexture2DOES(ext.TEXTURE_EXTERNAL_OES, videoElement); that the texture is "bound" to the video and no further calls need to be emitted (as would presently be the case)" In a that sense, we could abstract the details implementation into MediaPlayer Side like synchronizing issues. Even EGL_KHR_stream provides more details synchronization issues but still lot's of GPU driver and Video Decoder need to support the latest specification. But OES_EGL_image_external extension is already supported by most of the GPU drivers and Video decoders. And one of the performance bottleneck is to converting EGLImage input to TEXTURE_2D to handled by WebGL application. > On Tue, Oct 11, 2016 at 6:54 AM, Byungseon Shin > wrote: > > Hi Florian, Maksims, > > Thanks for the feedback. > > I have updated proposal to explain how application works. > Please see the attached updated proposal. > > To render an update frame, we need to call ext.EGLImageTargetTexture2DOES > to bind newly generated texture buffer from video decoder. > > By supporting OES_EGL_image_external extension, application can use direct > texture when video driver provides EGLImage like OpenGL ES natvie > applications. > > Our proposal is just focusing on extending format of texture compatible > with OpenGL ES extension and does not conflict with previous proposals. > > So, we provides a simplest way to adopt OES_EGL_image_external extension. > > By using references implementations of browser, WebGL renders video more > than 10 times faster than TEXTURE_2D format with Full HD resolution output. > > Kind regards, > Byungseon Shin > > On Tue, Oct 11, 2016 at 4:51 PM Maksims Mihejevs > wrote: > > Worth mentioning that in web there are multiple sources that have > independent redraw mechanics, that includes video as well as canvas > elements. It might make sense to have single extension for providing direct > access to those sources without need to re-upload it. > > I have assumptions that internally in browsers, both video and canvas have > their buffers, so it might lead to very similar implementation as from > internal side as well as from webgl api side. > > > On 11 Oct 2016 12:12 p.m., "Florian B?sch" wrote: > > Am I correct in assuming that once you've called ext. > EGLImageTargetTexture2DOES(ext.TEXTURE_EXTERNAL_OES, videoElement); that > the texture is "bound" to the video and no further calls need to be emitted > (as would presently be the case) for it to stay current? > > If that's the case, I think it's a good idea, it takes some work off web > application programmers to make sure the video is updated, and it should > also in all cases avoid a texture download/reupload. > > There are some other extensions that have attempted to deal with this > problem. > > - > https://www.khronos.org/registry/webgl/extensions/rejected/WEBGL_texture_from_depth_video/ > : this extension is rejected because it introduced a complex behavioral > changes/semantics whose benefits where unclear and the champions of this > extension stopped responding to questions and didn't offer any improvements > on the extension specification. > - > https://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture/ > : this extension is currently in a proposal state and it covers similar > functionality as EGL_image_external. Where it fundamentally differs is that > it also deals with format conversions (YUV 442 to rgb etc.) and accurate > timing for presentation of a video frame. > > Perhaps you could elaborate a little why OES_EGL_image_external would be > preferrable to WEBGL_dynamic_texture, and how the issues that > EGL_image_external does not tackle (timing, format conversion, other > things) would be handled by the web application programmer? > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: From kbr...@ Tue Oct 11 16:48:10 2016 From: kbr...@ (Kenneth Russell) Date: Tue, 11 Oct 2016 16:48:10 -0700 Subject: [Public WebGL] OES_EGL_image_external Extension proposal In-Reply-To: References: Message-ID: Hi Byungseon, Thanks for putting together this extension proposal. In conversations with groups within Google that are trying to do 360 degree video rendering in WebGL, they need more information than your extension would provide; specifically, they would need the exact timestamp of the frame currently being rendered, as well as other per-frame metadata like the current width and height of the texture (for variable bitrate video streams). Mark Callow's WEBGL_dynamic_texture extension proposal https://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture/ provides the controls needed. However, per working group discussions, the current extension's a little too complicated. I think if that extension were simplified a little bit that it would provide all of the performance benefits yours offers, and the controls that are known to be necessary. Do you think you'd be willing to devote some time to either extending your proposal (I can provide specific feedback) or editing down Mark's proposal to handle these use cases? I'd prefer to edit down WEBGL_dynamic_texture, if you're willing. Thanks, -Ken On Tue, Oct 11, 2016 at 8:27 AM, Byungseon Shin wrote: > Hi Corentin, > > On Tue, Oct 11, 2016 at 11:16 PM Corentin Wallez > wrote: > >> The problem with using an EGL_image type of extension is that the texture >> data store becomes shared between the new texture and the object it was >> created from. This is great when the application controls both sides and >> can ensure read and writes are probably synchronized, but in the case of >> video here, one side would write without the application being able to >> control synchronization. >> >> The standard way to do this type of video / texture binding in EGL is to >> use EGL_KHR_stream >> which >> is a much more complex and might not be an improvement over what developers >> can do with the current APIs. >> >> > As in proposal revision#3 and as Florian already mentioned, > Current proposal support "bind once and update implicitly" concept : " > ext.EGLImageTargetTexture2DOES(ext.TEXTURE_EXTERNAL_OES, videoElement); > that the texture is "bound" to the video and no further calls need to be > emitted (as would presently be the case)" > > In a that sense, we could abstract the details implementation into > MediaPlayer Side like synchronizing issues. > > Even EGL_KHR_stream provides more details synchronization issues but still > lot's of GPU driver and Video Decoder need to support the latest > specification. > But OES_EGL_image_external extension is already supported by most of the > GPU drivers and Video decoders. > > And one of the performance bottleneck is to converting EGLImage input to > TEXTURE_2D to handled by WebGL application. > > >> On Tue, Oct 11, 2016 at 6:54 AM, Byungseon Shin >> wrote: >> >> Hi Florian, Maksims, >> >> Thanks for the feedback. >> >> I have updated proposal to explain how application works. >> Please see the attached updated proposal. >> >> To render an update frame, we need to call ext.EGLImageTargetTexture2DOES >> to bind newly generated texture buffer from video decoder. >> >> By supporting OES_EGL_image_external extension, application can use >> direct texture when video driver provides EGLImage like OpenGL ES natvie >> applications. >> >> Our proposal is just focusing on extending format of texture compatible >> with OpenGL ES extension and does not conflict with previous proposals. >> >> So, we provides a simplest way to adopt OES_EGL_image_external extension. >> >> By using references implementations of browser, WebGL renders video more >> than 10 times faster than TEXTURE_2D format with Full HD resolution output. >> >> Kind regards, >> Byungseon Shin >> >> On Tue, Oct 11, 2016 at 4:51 PM Maksims Mihejevs >> wrote: >> >> Worth mentioning that in web there are multiple sources that have >> independent redraw mechanics, that includes video as well as canvas >> elements. It might make sense to have single extension for providing direct >> access to those sources without need to re-upload it. >> >> I have assumptions that internally in browsers, both video and canvas >> have their buffers, so it might lead to very similar implementation as from >> internal side as well as from webgl api side. >> >> >> On 11 Oct 2016 12:12 p.m., "Florian B?sch" wrote: >> >> Am I correct in assuming that once you've called ext.EGLImageTargetTextu >> re2DOES(ext.TEXTURE_EXTERNAL_OES, videoElement); that the texture is >> "bound" to the video and no further calls need to be emitted (as would >> presently be the case) for it to stay current? >> >> If that's the case, I think it's a good idea, it takes some work off web >> application programmers to make sure the video is updated, and it should >> also in all cases avoid a texture download/reupload. >> >> There are some other extensions that have attempted to deal with this >> problem. >> >> - https://www.khronos.org/registry/webgl/extensions/ >> rejected/WEBGL_texture_from_depth_video/ >> >> : this extension is rejected because it introduced a complex behavioral >> changes/semantics whose benefits where unclear and the champions of this >> extension stopped responding to questions and didn't offer any improvements >> on the extension specification. >> - https://www.khronos.org/registry/webgl/extensions/ >> proposals/WEBGL_dynamic_texture/ >> >> : this extension is currently in a proposal state and it covers similar >> functionality as EGL_image_external. Where it fundamentally differs is that >> it also deals with format conversions (YUV 442 to rgb etc.) and accurate >> timing for presentation of a video frame. >> >> Perhaps you could elaborate a little why OES_EGL_image_external would be >> preferrable to WEBGL_dynamic_texture, and how the issues that >> EGL_image_external does not tackle (timing, format conversion, other >> things) would be handled by the web application programmer? >> >> >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From sun...@ Tue Oct 11 18:59:41 2016 From: sun...@ (Byungseon Shin) Date: Wed, 12 Oct 2016 01:59:41 +0000 Subject: [Public WebGL] OES_EGL_image_external Extension proposal In-Reply-To: References: Message-ID: Hi Ken, Thanks for the feedback. Of course, we are willing to devote time and eager to enhance WebGL video rendering performance. We prefer to use our proposal as a base but we could revise dynamic_texture as well. Kind regards, Byungseon Shin On Wed, Oct 12, 2016 at 8:48 AM Kenneth Russell wrote: > Hi Byungseon, > > Thanks for putting together this extension proposal. > > In conversations with groups within Google that are trying to do 360 > degree video rendering in WebGL, they need more information than your > extension would provide; specifically, they would need the exact timestamp > of the frame currently being rendered, as well as other per-frame metadata > like the current width and height of the texture (for variable bitrate > video streams). > > Mark Callow's WEBGL_dynamic_texture extension proposal > https://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture/ > provides the controls needed. However, per working group discussions, the > current extension's a little too complicated. I think if that extension > were simplified a little bit that it would provide all of the performance > benefits yours offers, and the controls that are known to be necessary. > > Do you think you'd be willing to devote some time to either extending your > proposal (I can provide specific feedback) or editing down Mark's proposal > to handle these use cases? I'd prefer to edit down WEBGL_dynamic_texture, > if you're willing. > > Thanks, > > -Ken > > > > On Tue, Oct 11, 2016 at 8:27 AM, Byungseon Shin > wrote: > > Hi Corentin, > > On Tue, Oct 11, 2016 at 11:16 PM Corentin Wallez > wrote: > > The problem with using an EGL_image type of extension is that the texture > data store becomes shared between the new texture and the object it was > created from. This is great when the application controls both sides and > can ensure read and writes are probably synchronized, but in the case of > video here, one side would write without the application being able to > control synchronization. > > The standard way to do this type of video / texture binding in EGL is to > use EGL_KHR_stream > which > is a much more complex and might not be an improvement over what developers > can do with the current APIs. > > > As in proposal revision#3 and as Florian already mentioned, > Current proposal support "bind once and update implicitly" concept : "ext.EGLImageTargetTexture2DOES(ext.TEXTURE_EXTERNAL_OES, > videoElement); that the texture is "bound" to the video and no further > calls need to be emitted (as would presently be the case)" > > In a that sense, we could abstract the details implementation into > MediaPlayer Side like synchronizing issues. > > Even EGL_KHR_stream provides more details synchronization issues but still > lot's of GPU driver and Video Decoder need to support the latest > specification. > But OES_EGL_image_external extension is already supported by most of the > GPU drivers and Video decoders. > > And one of the performance bottleneck is to converting EGLImage input to > TEXTURE_2D to handled by WebGL application. > > > On Tue, Oct 11, 2016 at 6:54 AM, Byungseon Shin > wrote: > > Hi Florian, Maksims, > > Thanks for the feedback. > > I have updated proposal to explain how application works. > Please see the attached updated proposal. > > To render an update frame, we need to call ext.EGLImageTargetTexture2DOES > to bind newly generated texture buffer from video decoder. > > By supporting OES_EGL_image_external extension, application can use direct > texture when video driver provides EGLImage like OpenGL ES natvie > applications. > > Our proposal is just focusing on extending format of texture compatible > with OpenGL ES extension and does not conflict with previous proposals. > > So, we provides a simplest way to adopt OES_EGL_image_external extension. > > By using references implementations of browser, WebGL renders video more > than 10 times faster than TEXTURE_2D format with Full HD resolution output. > > Kind regards, > Byungseon Shin > > On Tue, Oct 11, 2016 at 4:51 PM Maksims Mihejevs > wrote: > > Worth mentioning that in web there are multiple sources that have > independent redraw mechanics, that includes video as well as canvas > elements. It might make sense to have single extension for providing direct > access to those sources without need to re-upload it. > > I have assumptions that internally in browsers, both video and canvas have > their buffers, so it might lead to very similar implementation as from > internal side as well as from webgl api side. > > > On 11 Oct 2016 12:12 p.m., "Florian B?sch" wrote: > > Am I correct in assuming that once you've called ext. > EGLImageTargetTexture2DOES(ext.TEXTURE_EXTERNAL_OES, videoElement); that > the texture is "bound" to the video and no further calls need to be emitted > (as would presently be the case) for it to stay current? > > If that's the case, I think it's a good idea, it takes some work off web > application programmers to make sure the video is updated, and it should > also in all cases avoid a texture download/reupload. > > There are some other extensions that have attempted to deal with this > problem. > > - > https://www.khronos.org/registry/webgl/extensions/rejected/WEBGL_texture_from_depth_video/ > : this extension is rejected because it introduced a complex behavioral > changes/semantics whose benefits where unclear and the champions of this > extension stopped responding to questions and didn't offer any improvements > on the extension specification. > - > https://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture/ > : this extension is currently in a proposal state and it covers similar > functionality as EGL_image_external. Where it fundamentally differs is that > it also deals with format conversions (YUV 442 to rgb etc.) and accurate > timing for presentation of a video frame. > > Perhaps you could elaborate a little why OES_EGL_image_external would be > preferrable to WEBGL_dynamic_texture, and how the issues that > EGL_image_external does not tackle (timing, format conversion, other > things) would be handled by the web application programmer? > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: