From yvo...@ Sun Jul 1 02:56:47 2012 From: yvo...@ (Yvonne Jung) Date: Sun, 1 Jul 2012 11:56:47 +0200 Subject: [Public WebGL] Web3D 2012: Call for Participation Message-ID: <25773551-0783-48D0-BC7B-9F210ABECB12@igd.fraunhofer.de> *** CALL FOR PARTICIPATION --- Web3D 2012 *** Web3D Conference 2012 17th International Conference on 3D Web Technology Sponsored by ACM SIGGRAPH and in cooperation with Eurographics Association August 4 - 5, 2012 L.A. Convention Center, Los Angeles, CA, USA http://www.web3d2012.org/ Mail: web3d2012...@ Registration: http://www.web3d2012.org/registration.html Confirmed Invited Speakers: - Markus Gross (ETH Zurich & Disney Research Zurich, Switzerland) - David J. Kasik (Boing, USA) http://www.web3d2012.org/invitedSpeakers.html Program: The program and the list of accepted papers and posters can be found on http://www.web3d2012.org/program.html Web3D: ------ The 17th ACM International Web3D Conference will address a wide range of research topics related to Web-based 3D Graphics. These topics include: representation and modeling methods, content analysis, rendering, distributed virtual environments, large-scale databases, Web-wide human-computer interaction, as well as innovative tools and applications. On the one hand, authoring and delivering educational contents is one of the main interests of this edition of the conference. The creation of educational materials, such as interactive electronic technical manuals will be supported, as well as the approach to incorporate advanced real-time rendering features into the X3D standard, in a generic way that is independent from the specific rendering method. On the other hand, the conference will particularly focus on novel mobile AR applications, innovative 3D graphics Web applications and novel Medical and eHealth developments, aside from contents over 3D Internet. The annual ACM Web3D Conference is a major event that gathers researchers, developers, entrepreneurs, experimenters, artists, and content creators in a dynamic environment. Attendees can share and explore methods of using, enhancing, and creating new 3D Web and Multimedia technologies, such as WebGL and HTML5, Flash 11/ Stage 3D, X3D, COLLADA, and the MPEG family. The conference highlights capabilities and trends in interactive 3D graphics across a wide range of applications and supports research from mobile devices up to high-end immersive environments. Topics: * Modeling, processing, analysis and rendering of complex geometry, structure, and behaviors * 3D search, shape matching and indexing * Rendering algorithms and standardization and visualization of large data sets * Interaction methods for online 3D content * Interactive 3D graphics for mobile devices * Mixed and Augmented Reality (including standardization aspects) * Agents, animated humanoids, and complex reactive characters * Remote rendering and streaming * Stereo and multi-view visualization of 3D graphic interfaces * High-performance 3D graphics for distributed environments, tele-presence, and 3D broadcasting * Web, multimedia and standards integration and interoperation. Accepted papers and posters will appear in the Web3D 2012 Conference Proceedings, published by ACM Press. General Chairs: - Christophe Mouton, EDF, France - Jorge Posada, Vicomtech, Spain Program Chairs: - Yvonne Jung, Fraunhofer IGD, Germany - Marcio Cabral, University of Sao Paulo, Brazil ----------------------------------------------------------- You are currently subscribed to public_webgl...@ To unsubscribe, send an email to majordomo...@ with the following command in the body of your email: unsubscribe public_webgl ----------------------------------------------------------- From yvo...@ Sun Jul 1 03:23:37 2012 From: yvo...@ (Yvonne Jung) Date: Sun, 1 Jul 2012 12:23:37 +0200 Subject: [Public WebGL] Restricting WebGL exposure of OES_depth_texture In-Reply-To: <551FB9B7-7DA6-428F-B862-DDE62F80445F@gmail.com> References: <4FEC197F.40 30302@igd.fraunhofer.de> <551FB9B7-7DA6-428F-B862-DDE62F80445F@gmail.com> Message-ID: <7E926326-14D7-41A1-8B3B-FF7D1A7B11C4@igd.fraunhofer.de> Hi, thank you for your help! However, I would have liked to use reading back floats exactly for getting rid of such encodings and to improve performance. My use case is simply that I want to improve buffer-based picking by encoding the fragment's position - including an integer object ID - into one floating point RGBA texture to avoid additional calculations and render passes. Are there any plans to include FLOAT readback from an FBO in the near future? Best regards Yvonne Am 28.06.2012 um 15:59 schrieb Carlos Scheidegger: > For what is worth, you can achieve the equivalent of a single-channel FLOAT readback by rendering the bit patterns of the floating point numbers to a 4x8-bit RGBA channel, and then reading the texture. It is horrendously ugly, but it is simple and does the job. > > The version I wrote does not handle denormals, infinities, NaNs or anything that is not a "nice" IEEE 754 value. But I have used this in the past, and it works. > > I don't have a ready-made GLSL program for you, for technical reasons; but the idea can be seen here: > > https://github.com/cscheid/facet/blob/master/src/shade/bits/encode_float.js > > Let me know if you need more specifics. > > Cheers, > -carlos > > On Jun 28, 2012, at 4:44 AM, Yvonne Jung wrote: > >> >> Hi all, >> >> another question concerning depth textures: is it still correct that readPixels is not yet implemented for FLOAT (I got an error when trying to use it and also found a corresponding bug report in the Mozilla bug tracker) and if true, when do you expect to be able to provide such an implementation? >> >> Regards >> Yvonne ----------------------------------------------------------- You are currently subscribed to public_webgl...@ To unsubscribe, send an email to majordomo...@ with the following command in the body of your email: unsubscribe public_webgl ----------------------------------------------------------- From pya...@ Sun Jul 1 04:24:01 2012 From: pya...@ (=?ISO-8859-1?Q?Florian_B=F6sch?=) Date: Sun, 1 Jul 2012 13:24:01 +0200 Subject: [Public WebGL] Restricting WebGL exposure of OES_depth_texture In-Reply-To: <7E926326-14D7-41A1-8B3B-FF7D1A7B11C4@igd.fraunhofer.de> References: <551FB9B7-7DA6-428F-B862-DDE62F80445F@gmail.com> <7E926326-14D7-41A1-8B3B-FF7D1A7B11C4@igd.fraunhofer.de> Message-ID: On Sun, Jul 1, 2012 at 12:23 PM, Yvonne Jung wrote: > thank you for your help! However, I would have liked to use reading back > floats exactly for getting rid of such encodings and to improve > performance. My use case is simply that I want to improve buffer-based > picking by encoding the fragment's position - including an integer object > ID - into one floating point RGBA texture to avoid additional calculations > and render passes. Are there any plans to include FLOAT readback from an > FBO in the near future? > Floating point readback is not supported by vanilla OpenGL ES 2.0. So if that functionality where to be provided, it would have to be an extension. However the OES extension registry (http://www.khronos.org/registry/gles/) does not list any extension that lets you perform floating point readbacks as far as I can see. It is very likely that you cannot render to floating point textures on mobiles, and even if you can, it is certain that you cannot read back from floating point targets. Because the policy is upheld by the Khronos WebGL-WG that no desktop only functionality should be introduced to WebGL, it is therefore impossible to introduce an extension to provide this because it would only work on desktops. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kbr...@ Mon Jul 2 07:01:48 2012 From: kbr...@ (Kenneth Russell) Date: Mon, 2 Jul 2012 10:01:48 -0400 Subject: [Public WebGL] Call for Speakers: WebGL BOF at SIGGRAPH 2012 Message-ID: [cross-posted to webgl-dev-list] At SIGGRAPH 2012, the Khronos Group will host a Birds of a Feather session on WebGL: When: Wednesday, August 8 from 4:00 - 5:00 PM Where: JW Marriott Los Angeles at LA Live Room: Gold Ballroom - Salon 2 As in previous years, we would like this to be an event driven by the WebGL community. If you are developing using WebGL, for example building one of the many WebGL libraries, an application, or cool demos, and would like to share your experience, please send me (kbr at google dot com) a private email. Presentations focused around demonstrations will be highly favored. :) Based on the number of excellent submissions we had last year, we are aiming for a "lightning talk" format this year with approximately 5 minutes per presentation. We ask for and appreciate your flexibility this year as we create and adjust the schedule. Looking forward to seeing you there! -Ken -------------- next part -------------- An HTML attachment was scrubbed... URL: From jda...@ Mon Jul 2 09:09:45 2012 From: jda...@ (John Davis) Date: Mon, 2 Jul 2012 11:09:45 -0500 Subject: [Public WebGL] Re: iOS chrome In-Reply-To: <9A0DADEE-29F8-41D1-959D-56D672E37164@apple.com> References: <206118B4-B62F-4208-ADB5-4C39A4F19A6F@apple.com> <9A0DADEE-29F8-41D1-959D-56D672E37164@apple.com> Message-ID: So opera 12 on android is the only option? On Saturday, June 30, 2012, Dean Jackson wrote: > > On 30/06/2012, at 5:08 PM, Florian B?sch > > wrote: > > On Sat, Jun 30, 2012 at 5:56 AM, Dean Jackson > > wrote: > >> You can file requests for these at bugreporter.apple.com (yes, feature >> requests are "bugs"). You will probably be told it is a duplicate, but >> duplicates help Apple plan features. >> > > Your bug reporter has bugs: *An error has occurred. Please report the > error to Apple Inc. by emailing the error detail to **devbugs...@ 'devbugs...@?subject\x3dA%20Bug%20Reporter%20error%20has%20occurred\x26amp;body\x3d%20Thank%20you%20for%20your%20assistance%20in%20helping%20us%20discover%20and%20isolate%20bugs%20within%20our%20products.%C2%A0Please%20provide%20the%20following%20information%20so%20we%20are%20able%20to%20investigate%20this%20issue%20further.%0A%0A%0A%20-%20Apple%20ID%20used%20as%20Bug%20Reporter%20login%20when%20this%20problem%20occurred:%0A%0A%20-%20Name%20associated%20with%20above%20Apple%20ID:%0A%0A%20-%20Email%20address%20associated%20above%20Apple%20ID:%0A%0A%20-%20Person%20ID%20associated%20with%20above%20Apple%20ID%20(see%20the%20\x26#39;My%20Profile\x26#39;%20section%20within%20Dev%20Center):%0A%0A%20-%20Web%20browser%20(Safari,%20Firefox,%20etc.)%20and%20Version:%0A%0A%20-%20Operating%20system%20and%20Version:%0A%0A%20-%20Date%20and%20time%20zone%20when%20this%20problem%20occurred:%0A%0A%20-%20Reproducibility%20(Always,%20Sometimes,%20Rarely,%20Not%20Reproducible,%20Did%20not%20try):%0A%0A%20-%20Steps%20to%20Reproduce%20or%20other%20supporting%20screenshots,%20logs,%20etc:');> > * > > > I'm fairly confident the people who run that system don't read messages > sent to this list, so could you email the details to devbugs...@ > ? > > Dean > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alo...@ Tue Jul 3 13:28:04 2012 From: alo...@ (Alok Priyadarshi) Date: Tue, 3 Jul 2012 14:28:04 -0600 Subject: [Public WebGL] token length clarification Message-ID: http://www.khronos.org/registry/webgl/specs/latest/#6.17 "WebGL requires support of tokens up to 256 characters in length. Shaders containing tokens longer than 256 characters must fail to compile" Which type of tokens is being implied here - preprocessing-tokens or compiler-tokens? There is a difference between the two. I do not think we want this restriction on preprocessing-tokens, otherwise non-compliant tokens inside excluded blocks would also generate error. If we mean compiler-tokens, then this test is wrong: https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/sdk/tests/conformance/glsl/misc/shader-with-257-character-define.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From kbr...@ Tue Jul 3 13:40:18 2012 From: kbr...@ (Kenneth Russell) Date: Tue, 3 Jul 2012 16:40:18 -0400 Subject: [Public WebGL] token length clarification In-Reply-To: References: Message-ID: Good question. The spec hasn't made this distinction to this point, since the preprocessor and compiler are not separate stages in the WebGL API. Despite the fact that too-long tokens in excluded #if blocks would trigger this restriction, I think it's easier to understand if the restriction is expressed as preprocessor tokens, so that the shader-with-257-character-define.html test is still correct. -Ken On Tue, Jul 3, 2012 at 4:28 PM, Alok Priyadarshi wrote: > http://www.khronos.org/registry/webgl/specs/latest/#6.17 > > "WebGL requires support of tokens up to 256 characters in length. Shaders > containing tokens longer than 256 characters must fail to compile" > > Which type of tokens is being implied here - preprocessing-tokens or > compiler-tokens? There is a difference between the two. I do not think we > want this restriction on preprocessing-tokens, otherwise non-compliant > tokens inside excluded blocks would also generate error. > > If we mean compiler-tokens, then this test is wrong: > https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/sdk/tests/conformance/glsl/misc/shader-with-257-character-define.html > ----------------------------------------------------------- You are currently subscribed to public_webgl...@ To unsubscribe, send an email to majordomo...@ with the following command in the body of your email: unsubscribe public_webgl ----------------------------------------------------------- From kbr...@ Wed Jul 4 12:03:08 2012 From: kbr...@ (Kenneth Russell) Date: Wed, 4 Jul 2012 15:03:08 -0400 Subject: [Public WebGL] Do Typed Array need an isArray method? In-Reply-To: <4FEE0001.1080502@mit.edu> References: <4FEDEE9A.5090207@comcast.net> <4FEDF5B2.4080903@comcast.net> <4FEDF6C1.4010908@mit.edu> <4FEDFD13.4020607@mit.edu> <4FEE0001.1080502@mit.edu> Message-ID: On Fri, Jun 29, 2012 at 3:20 PM, Boris Zbarsky wrote: > On 6/29/12 3:12 PM, Kenneth Russell wrote: >> >> How does this work for DOM nodes? > > > Which "this"? If I have a HTML element, then its proto chain is > HTMLBodyElement.prototype -> HTMLElement.prototype -> Element.prototype -> > Node.prototype (and then possibly to EventTarget.prototype, depending on > which version of which spec you read). And all those interface objects > exist and can be used as the RHS of instanceof. Thanks, that's what I was asking. > For what it's worth, at least some of the SpiderMonkey people I just talked > to about don't want to do this for typed arrays at all. They were proposing > something more like this IDL: > > [NoInterfaceObject] > interface ArrayBufferView { > // stuff > }; > > interface Int32Array { > // stuff > }; > > Int32Array implements ArrayBufferView; > > That would, of course, keep "instanceof ArrayBufferView" not working.... > > Personally, I'm planning to stay out of this whole discussion as much as I > can; I don't actually care which way we go here as long as the end state > involves the spec and the implementations actually agreeing and a test > suite. ;) I agree that the typed array spec at least has to match the current reality. Firefox, Opera, Safari and Chrome all agree that ArrayBufferView is not currently exposed as an interface object, so http://www.khronos.org/registry/typedarray/specs/latest/ and http://www.khronos.org/registry/typedarray/specs/latest/typedarray.idl have been updated making ArrayBufferView [NoInterfaceObject]. https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/sdk/tests/conformance/typedarrays/array-unit-tests.html now verifies this, as well as the Uint8ClampedArray -> Uint8Array inheritance. Please send any feedback to the list. -Ken ----------------------------------------------------------- You are currently subscribed to public_webgl...@ To unsubscribe, send an email to majordomo...@ with the following command in the body of your email: unsubscribe public_webgl ----------------------------------------------------------- From cal...@ Thu Jul 5 00:47:26 2012 From: cal...@ (Mark Callow) Date: Thu, 05 Jul 2012 16:47:26 +0900 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal Message-ID: <4FF5468E.5000603@hicorp.co.jp> I have completed the initial draft of a proposed WEBGL_dynamic_texture extension. The purpose of this is to provide an API for handling textures with dynamic images such as video that can be implemented at the maximum efficiency possible on a given platform, including with zero copies. Please look at the issues list before posting any questions as it contains many of the questions you are likely to have and several answers. You can find the proposal at: http://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture I welcome your comments and help to resolve all the listed issues. Regards -Mark -- ???????????????????????????????????? ???????????????????????????? ??????? ???????????????????????????????????? ????????????????????? ??? NOTE: This electronic mail message may contain confidential and privileged information from HI Corporation. If you are not the intended recipient, any disclosure, photocopying, distribution or use of the contents of the received information is prohibited. If you have received this e-mail in error, please notify the sender immediately and permanently delete this message and all related copies. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pya...@ Thu Jul 5 01:39:53 2012 From: pya...@ (=?ISO-8859-1?Q?Florian_B=F6sch?=) Date: Thu, 5 Jul 2012 10:39:53 +0200 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: <4FF5468E.5000603@hicorp.co.jp> References: <4FF5468E.5000603@hicorp.co.jp> Message-ID: On Thu, Jul 5, 2012 at 9:47 AM, Mark Callow wrote: > I welcome your comments and help to resolve all the listed issues. > I like the functionality. I dislike the use of a separate sampler type as in: uniform samplerExternalOES videoSampler; It means shaders have to be pre-processed according to the type of texture passed in, and that makes it quite a lot harder to support both static and dynamic textures at the same time, as well as writing utilities/libraries around filtering that work regardless. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wil...@ Thu Jul 5 01:52:34 2012 From: wil...@ (William HENNEBOIS) Date: Thu, 5 Jul 2012 10:52:34 +0200 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: <4FF5468E.5000603@hicorp.co.jp> References: <4FF5468E.5000603@hicorp.co.jp> Message-ID: <29C289DA60B5B646ACE5FEC7560D8C88013B03CFB3AB@SAFEX1MAIL2.st.com> Thanks a lot, rom: owner-public_webgl...@ [mailto:owner-public_webgl...@] On Behalf Of Mark Callow Sent: Thursday, July 05, 2012 9:47 AM To: public webgl Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal I have completed the initial draft of a proposed WEBGL_dynamic_texture extension. The purpose of this is to provide an API for handling textures with dynamic images such as video that can be implemented at the maximum efficiency possible on a given platform, including with zero copies. Please look at the issues list before posting any questions as it contains many of the questions you are likely to have and several answers. You can find the proposal at: http://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture I welcome your comments and help to resolve all the listed issues. Regards -Mark -- ???????????????????????????????????????????????????????????????? ???????????????????????????????????????????????????????????????? ??? NOTE: This electronic mail message may contain confidential and privileged information from HI Corporation. If you are not the intended recipient, any disclosure, photocopying, distribution or use of the contents of the received information is prohibited. If you have received this e-mail in error, please notify the sender immediately and permanently delete this message and all related copies. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cal...@ Thu Jul 5 01:54:40 2012 From: cal...@ (Mark Callow) Date: Thu, 05 Jul 2012 17:54:40 +0900 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> Message-ID: <4FF55650.1020301@hicorp.co.jp> On 2012/07/05 17:39, Florian B?sch wrote: > On Thu, Jul 5, 2012 at 9:47 AM, Mark Callow > wrote: > > I welcome your comments and help to resolve all the listed issues. > > I like the functionality. I dislike the use of a separate sampler type > as in: > uniform samplerExternalOES videoSampler; > > It means shaders have to be pre-processed according to the type of > texture passed in, and that makes it quite a lot harder to support > both static and dynamic textures at the same time, as well as writing > utilities/libraries around filtering that work regardless. > If you use the same sampler type the implementation has to figure out at run-time what type of image it is sampling and may have to recompile the shader when switching between yuv and rgb images. It's not wonderful either way but I believe that the majority of applications will know in advance which textures will use dynamic sources. So I think the separate sampler type is the way to go. The underlying OES extension made the same choice. Regards -Mark -- ???????????????????????????????????? ???????????????????????????? ??????? ???????????????????????????????????? ????????????????????? ??? NOTE: This electronic mail message may contain confidential and privileged information from HI Corporation. If you are not the intended recipient, any disclosure, photocopying, distribution or use of the contents of the received information is prohibited. If you have received this e-mail in error, please notify the sender immediately and permanently delete this message and all related copies. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pya...@ Thu Jul 5 02:07:02 2012 From: pya...@ (=?ISO-8859-1?Q?Florian_B=F6sch?=) Date: Thu, 5 Jul 2012 11:07:02 +0200 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: <4FF55650.1020301@hicorp.co.jp> References: <4FF5468E.5000603@hicorp.co.jp> <4FF55650.1020301@hicorp.co.jp> Message-ID: On Thu, Jul 5, 2012 at 10:54 AM, Mark Callow wrote: > If you use the same sampler type the implementation has to figure out at > run-time what type of image it is sampling and may have to recompile the > shader when switching between yuv and rgb images. > I understand the reasoning for the separation in the shader. > It's not wonderful either way but I believe that the majority of > applications will know in advance which textures will use dynamic sources. > So I think the separate sampler type is the way to go. The underlying OES > extension made the same choice. > I think that most code does not have foreknowledge of what kind of source it is going to sample. For instance: - three.js does not know until a sampling source is set, necessitating a shader recompile at that time - kick.js does not know until a sampling source is set - radiapp does not know, and does not contain provisions to produce different shaders, it's current code is based on the copy semantics, therefore its shader infrastructure works regardless. Most frameworks/engines hitherto do not make any provision of having to recompile shaders on-line for sampler types, and most will have a hard time doing that. I am toying with the idea of writing a few tools around generative/dynamic texturing, which often takes the form of processing nodes in a graph (as does radiapp). Because compiling shaders on-line is largely infeasible (due to large possible delays) the shaders would have to be pre-compiled. In the case of a 2-operator node, that would result in 4 shaders instead of one, quadruppling the shader compile time. In case of trinary or tertiary operators, it would 8x or 16x the shader compile time. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pya...@ Thu Jul 5 03:00:17 2012 From: pya...@ (=?ISO-8859-1?Q?Florian_B=F6sch?=) Date: Thu, 5 Jul 2012 12:00:17 +0200 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> <4FF55650.1020301@hicorp.co.jp> Message-ID: I just had a quick word with Cedric Pinson (author of sketchfab and osgjs) and explained to him the semantic of a different sampler type to which he replied: "but having to use a different sampler is less interesting a first thought" So I think it's safe to assume that osgjs will skip supporting this extension for now. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pya...@ Thu Jul 5 05:19:52 2012 From: pya...@ (=?ISO-8859-1?Q?Florian_B=F6sch?=) Date: Thu, 5 Jul 2012 14:19:52 +0200 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> <4FF55650.1020301@hicorp.co.jp> Message-ID: A suggestion to deal with the dynamic uniform typing issue was brought up on #webgl as an API to provide a uniform "hint" at shader compile time, so that at least generating the huge sets of differing shaders is a fairly orderly process and has a "fast" path. If no hint would be provided, the shader would be allowed to recompile on the "slow path". -------------- next part -------------- An HTML attachment was scrubbed... URL: From gma...@ Thu Jul 5 09:51:43 2012 From: gma...@ (=?UTF-8?B?R3JlZ2cgVGF2YXJlcyAo56S+55SoKQ==?=) Date: Thu, 5 Jul 2012 09:51:43 -0700 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> <4FF55650.1020301@hicorp.co.jp> Message-ID: Recompiling shaders on the fly brings up way too many problems. *) The shader may fail to compile but you don't get that error until you try to draw. GL has no such provision for that and programs would have check for errors after draw calls slowing them down tremendously. *) Shaders can get arbitrarily complex and know why to know how complex until draw time uniform sampler2D textures[16]; // Is that 16 2D textures or 16 video textures (which is really 48 textures)? *) The speed of a shader can change drastically from use to use. You'd have to do timing with every combination of inputs. *) Compiling shaders can be slow, currently an application can deal with that at init time (put up a "loading..." message) etc. but if shaders get arbitrarily recompiled at draw time the app has no way to know if and when there will be a significant pause that they need to guide the user through. *) Massively complicating WebGL implementations On Thu, Jul 5, 2012 at 5:19 AM, Florian B?sch wrote: > A suggestion to deal with the dynamic uniform typing issue was brought up > on #webgl as an API to provide a uniform "hint" at shader compile time, so > that at least generating the huge sets of differing shaders is a fairly > orderly process and has a "fast" path. If no hint would be provided, the > shader would be allowed to recompile on the "slow path". > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gma...@ Thu Jul 5 10:04:01 2012 From: gma...@ (=?UTF-8?B?R3JlZ2cgVGF2YXJlcyAo56S+55SoKQ==?=) Date: Thu, 5 Jul 2012 10:04:01 -0700 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: <4FF5468E.5000603@hicorp.co.jp> References: <4FF5468E.5000603@hicorp.co.jp> Message-ID: Looks good Mark. A few comments: *) is naming all the functions dynamicTextureXXX redundant? The functions are called off the extension object so var dynamicTextureExtension = gl.getExtenion("WEBGL_dynamic_texture"); dynamicTextureExtension.setSource(texture, src); might be enough? *) does dynamicTextureGetSource seem un GL like? Maybe getDynamicTextureParameter(param) in case something needs to be added later? *) is the state before calling dynamicTextureAquaireFrame or after calling dynamicTextureReleaseFrame desirable? It means the behavior of the program is implementation dependent. It could be that on some browsers, never calling either function works perfectly and on other browsers or platforms it performs poorly or not at all. Where as if there is no dynamicTextureReleaseFrame and you're required to call dyanicTextureAcquireFrame to see anything at all it seems more likely you'd get consistent behavior across implementations? On Thu, Jul 5, 2012 at 12:47 AM, Mark Callow wrote: > I have completed the initial draft of a proposed WEBGL_dynamic_texture > extension. The purpose of this is to provide an API for handling textures > with dynamic images such as video that can be implemented at the maximum > efficiency possible on a given platform, including with zero copies. > > Please look at the issues list before posting any questions as it contains > many of the questions you are likely to have and several answers. > > You can find the proposal at: > > > http://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture > > I welcome your comments and help to resolve all the listed issues. > > Regards > > -Mark > > > -- > ???????????????????????????????????????????????????????????????? > ???????????????????????????????????????????????????????????????? ??? > > NOTE: This electronic mail message may contain confidential and privileged > information from HI Corporation. If you are not the intended recipient, any > disclosure, photocopying, distribution or use of the contents of the > received information is prohibited. If you have received this e-mail in > error, please notify the sender immediately and permanently delete this > message and all related copies. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pya...@ Thu Jul 5 10:09:07 2012 From: pya...@ (=?ISO-8859-1?Q?Florian_B=F6sch?=) Date: Thu, 5 Jul 2012 19:09:07 +0200 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> Message-ID: So in essence, while claiming dynamic recompilation is too complex to do for WebGL, you fully expect any framework/utility/library to do that now? Despite that they have the exact same problem, they don't know beforehand what flavor of texture will be present, and when they find out, they'll have to recompile. Yeah, 16 samplers that can either be a texture2D or a dynamic texture sure seems like a messy usecase, I'm sure it gets *much* better by having to compile 65536^shaders upon "loading". Are you quite stark mad? -------------- next part -------------- An HTML attachment was scrubbed... URL: From gma...@ Thu Jul 5 10:11:16 2012 From: gma...@ (=?UTF-8?B?R3JlZ2cgVGF2YXJlcyAo56S+55SoKQ==?=) Date: Thu, 5 Jul 2012 10:11:16 -0700 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> Message-ID: A couple more comments * samplerExternalOES or samplerDynamicTextureWEBGL? I'm just wondering if there will be any future conflict. I don't want to end up in a case where we some how have to distinguish between real uses of samplesExternalOES and ones that have to be re-written for WebGL dynamic textures. But maybe that will never come up? * #extension OES_EGL_image_external : enable or #extension WEBGL_dynamic_texture : enable? Similarly, do I need to worry about any future conflicts? On Thu, Jul 5, 2012 at 10:04 AM, Gregg Tavares (??) wrote: > Looks good Mark. > > A few comments: > > *) is naming all the functions dynamicTextureXXX redundant? The functions > are called off the extension object so > > var dynamicTextureExtension = gl.getExtenion("WEBGL_dynamic_texture"); > dynamicTextureExtension.setSource(texture, src); > > might be enough? > > *) does dynamicTextureGetSource seem un GL like? > > Maybe getDynamicTextureParameter(param) in case something needs to be > added later? > > *) is the state before calling dynamicTextureAquaireFrame or after calling > dynamicTextureReleaseFrame desirable? > > It means the behavior of the program is implementation dependent. It could > be that on some browsers, never calling either function works perfectly and > on other browsers or platforms it performs poorly or not at all. Where as > if there is no dynamicTextureReleaseFrame and you're required to call > dyanicTextureAcquireFrame to see anything at all it seems more likely you'd > get consistent behavior across implementations? > > > > > > > > > > > > On Thu, Jul 5, 2012 at 12:47 AM, Mark Callow wrote: > >> I have completed the initial draft of a proposed WEBGL_dynamic_texture >> extension. The purpose of this is to provide an API for handling textures >> with dynamic images such as video that can be implemented at the maximum >> efficiency possible on a given platform, including with zero copies. >> >> Please look at the issues list before posting any questions as it >> contains many of the questions you are likely to have and several answers. >> >> You can find the proposal at: >> >> >> http://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture >> >> I welcome your comments and help to resolve all the listed issues. >> >> Regards >> >> -Mark >> >> >> -- >> ???????????????????????????????????????????????????????????????? >> ???????????????????????????????????????????????????????????????? ??? >> >> NOTE: This electronic mail message may contain confidential and >> privileged information from HI Corporation. If you are not the intended >> recipient, any disclosure, photocopying, distribution or use of the >> contents of the received information is prohibited. If you have received >> this e-mail in error, please notify the sender immediately and permanently >> delete this message and all related copies. >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gma...@ Thu Jul 5 10:16:21 2012 From: gma...@ (=?UTF-8?B?R3JlZ2cgVGF2YXJlcyAo56S+55SoKQ==?=) Date: Thu, 5 Jul 2012 10:16:21 -0700 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> Message-ID: I don't expect most libraries will handle it automatically. Most will require the user to specify upfront what they want. but yes, if they want to support dynamic textures I do expect them to handle it. Only they know when is the appropriate time to compile and when to fail. On top of that they have to call dynamicTextureAcquireFrame to use these textures so it's not like they can magically do nothing and have it just work. Those libraries that do want to type handle it automatically are already most likely handling it dynamically. They generate shaders based on number of lights and other material parameters. The can just as easily add dynamic textures as an option to their generator. On Thu, Jul 5, 2012 at 10:09 AM, Florian B?sch wrote: > So in essence, while claiming dynamic recompilation is too complex to do > for WebGL, you fully expect any framework/utility/library to do that now? > Despite that they have the exact same problem, they don't know beforehand > what flavor of texture will be present, and when they find out, they'll > have to recompile. > > Yeah, 16 samplers that can either be a texture2D or a dynamic texture sure > seems like a messy usecase, I'm sure it gets *much* better by having to > compile 65536^shaders upon "loading". Are you quite stark mad? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pya...@ Thu Jul 5 10:19:06 2012 From: pya...@ (=?ISO-8859-1?Q?Florian_B=F6sch?=) Date: Thu, 5 Jul 2012 19:19:06 +0200 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> Message-ID: On Thu, Jul 5, 2012 at 7:16 PM, Gregg Tavares (??) wrote: > I don't expect most libraries will handle it automatically. Most will > require the user to specify upfront what they want. > > but yes, if they want to support dynamic textures I do expect them to > handle it. Only they know when is the appropriate time to compile and when > to fail. On top of that they have to call dynamicTextureAcquireFrame to use > these textures so it's not like they can magically do nothing and have it > just work. > > Those libraries that do want to type handle it automatically are already > most likely handling it dynamically. They generate shaders based on number > of lights and other material parameters. The can just as easily add dynamic > textures as an option to their generator. > The only remotely reasonable sane way that this extension can be "handled" by frameworks is creating ghost textures, creating an FBO, then doing a copy shader, that just uses this one sampler, copies it to an RGB texture and then drop in that RGB texture in place of this broken piece of mess. So that's fine, we can do that, no problem, I just thought, you know, maybe don't make people jump trough complicated mad hoops just to make it possible to write a sane library/framework/toolkit. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pya...@ Thu Jul 5 10:26:25 2012 From: pya...@ (=?ISO-8859-1?Q?Florian_B=F6sch?=) Date: Thu, 5 Jul 2012 19:26:25 +0200 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> Message-ID: On Thu, Jul 5, 2012 at 7:19 PM, Florian B?sch wrote: > > On Thu, Jul 5, 2012 at 7:16 PM, Gregg Tavares (??) wrote: > >> but yes, if they want to support dynamic textures I do expect them to >> handle it. Only they know when is the appropriate time to compile and when >> to fail. On top of that they have to call dynamicTextureAcquireFrame to use >> these textures so it's not like they can magically do nothing and have it >> just work. >> > Ohyeah, and one other thing. Calling extra functions and doing hoop-jumping that does not require a different fracking shader for every possible sampler source is perfectly fine. That is a piece of functionality that can be nicely encapsulated. Dealing with every possible combination of shaders however, is *not* fine. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gma...@ Thu Jul 5 10:39:06 2012 From: gma...@ (=?UTF-8?B?R3JlZ2cgVGF2YXJlcyAo56S+55SoKQ==?=) Date: Thu, 5 Jul 2012 10:39:06 -0700 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> Message-ID: On Thu, Jul 5, 2012 at 10:19 AM, Florian B?sch wrote: > > > On Thu, Jul 5, 2012 at 7:16 PM, Gregg Tavares (??) wrote: > >> I don't expect most libraries will handle it automatically. Most will >> require the user to specify upfront what they want. >> >> but yes, if they want to support dynamic textures I do expect them to >> handle it. Only they know when is the appropriate time to compile and when >> to fail. On top of that they have to call dynamicTextureAcquireFrame to use >> these textures so it's not like they can magically do nothing and have it >> just work. >> >> Those libraries that do want to type handle it automatically are already >> most likely handling it dynamically. They generate shaders based on number >> of lights and other material parameters. The can just as easily add dynamic >> textures as an option to their generator. >> > > The only remotely reasonable sane way that this extension can be "handled" > by frameworks is creating ghost textures, creating an FBO, then doing a > copy shader, that just uses this one sampler, copies it to an RGB texture > and then drop in that RGB texture in place of this broken piece of mess. So > that's fine, we can do that, no problem, I just thought, you know, maybe > don't make people jump trough complicated mad hoops just to make it > possible to write a sane library/framework/toolkit. > The only point of this extension is to avoid the copy. To avoid the copy videos are effectively triple buffered. Video usually has 3 textures, Y, U and V. It has those double buffered so it can be filling out one while displaying the other. This extension basically adds a 3rd buffer. Calling dynamicTextureAcquireFrame takes one of those buffers (sets of 3 textures). No copy required. On other words. The video decoder is using frame 1: fill in setA, draw with setB frame 2: fill in setB, draw with setA frame 3: fill in setA, draw with setB frame 4: fill in setB, draw with setA User calls dyanmicTextureAcquireFrame and is given setB. Video is now doing this frame 5: fill in setC, draw with setB frame 6: fill in setA, draw with setC User calls dyanmicTextureAcquireFrame and is given setA. Video is now doing this frame 7: fill in setB, draw with setA frame 6: fill in setC, draw with setB etc... Zero copy is the whole point of this extension. If you want the copy path just call texImage2D(....., HTMLVideoElement). Browsers are already working on making that path use an FBO to speed up the copy. But it's still a copy. Copying 1920x1080p video at 60hz is a much slower and battery intensive operation than not copying at all. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pya...@ Thu Jul 5 10:43:25 2012 From: pya...@ (=?ISO-8859-1?Q?Florian_B=F6sch?=) Date: Thu, 5 Jul 2012 19:43:25 +0200 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> Message-ID: On Thu, Jul 5, 2012 at 7:39 PM, Gregg Tavares (??) wrote: > Zero copy is the whole point of this extension. If you want the copy path > just call texImage2D(....., HTMLVideoElement). Browsers are already working > on making that path use an FBO to speed up the copy. But it's still a copy. > Copying 1920x1080p video at 60hz is a much slower and battery intensive > operation than not copying at all. > The only way this texture type can be accommodated without breaking all existing shaders and forcing everybody to adopt shader preprocessors as well as forcing everybody to define their sampler types up front for this flavor is to *copy* so we can get it into a texture that we can drop into our existing frameworks/libraries/utilities that just works. If the aim of this extension is to get rid of the copy, than that is mostly a failure, except for border-use cases where people incidentially have a friendly and simple enough app structure and framework so that works. -------------- next part -------------- An HTML attachment was scrubbed... URL: From toj...@ Thu Jul 5 10:52:07 2012 From: toj...@ (Brandon Jones) Date: Thu, 5 Jul 2012 10:52:07 -0700 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: <4FF55650.1020301@hicorp.co.jp> References: <4FF5468E.5000603@hicorp.co.jp> <4FF55650.1020301@hicorp.co.jp> Message-ID: On Thu, Jul 5, 2012 at 1:54 AM, Mark Callow wrote: > > If you use the same sampler type the implementation has to figure out at > run-time what type of image it is sampling and may have to recompile the > shader when switching between yuv and rgb images. > But as Florian mentioned most frameworks will be forced to do that anyway. Wouldn't it be preferable to have that happen in native code? It's not wonderful either way but I believe that the majority of > applications will know in advance which textures will use dynamic sources. > That feels like some sort of hinting system would be more appropriate. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cma...@ Thu Jul 5 14:02:50 2012 From: cma...@ (Chris Marrin) Date: Thu, 05 Jul 2012 14:02:50 -0700 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: <4FF5468E.5000603@hicorp.co.jp> References: <4FF5468E.5000603@hicorp.co.jp> Message-ID: <1DFF71A7-51BD-4EBD-9D26-83E3D8357C15@apple.com> On Jul 5, 2012, at 12:47 AM, Mark Callow wrote: > I have completed the initial draft of a proposed WEBGL_dynamic_texture extension. The purpose of this is to provide an API for handling textures with dynamic images such as video that can be implemented at the maximum efficiency possible on a given platform, including with zero copies. > > Please look at the issues list before posting any questions as it contains many of the questions you are likely to have and several answers. > > You can find the proposal at: > > http://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture > I welcome your comments and help to resolve all the listed issues. > Good first pass, Mark. Thanks. There are a couple of inconsistencies: 1) The IDL says "dynamicTextureAcquireImage" and the description says "dynamicTextureAcquireFrame". Same for release. 2) dynamicTextureSetSource passes the texture in the IDL and description, but not in the example. Given the nature of OpenGL, seems like you should not pass the texture, but rather use the currently bound texture? Same would be true of all the other API calls. I am also concerned about timestamping. iOS and OSX has the concept of image queues, where a series of images are stored, each with a timestamp of when they should be displayed. So, for instance, it might contain several video frames, each with a timestamp that is 1/30th of a second apart (assuming a 30hz frame rate). This allows the decoder to place images in the queue at a different (and possibly non-uniform) rate, relative to the times at which each image is displayed. Your proposal doesn't deal with timestamps, so it's possible to get out of sync with the decode rate, causing strobing or other undesirable effects. Maybe this can be solved by passing the desired timestamp via dynamicTextureAcquireImage? When passed, the frame with the closest timestamp would be acquired and locked. You'd probably also want to return the actual timestamp acquired to allow for throttling. Or maybe it would be better to return the timestamp of the next frame after the one acquired. That would make it easy to sync rendering with the video frame rate. Either way, there would need to be rules about frame availability. If I ask for a frame in the distant past, would it fail, or give me the oldest frame it has. After I release a frame with a given timestamp, is it and all older frames no longer available? Or is the lifetime of a frame something that is hidden. At any rate, I think something like this will be needed to deal with timed media. ----- ~Chris Marrin cmarrin...@ -------------- next part -------------- An HTML attachment was scrubbed... URL: From kos...@ Thu Jul 5 14:54:51 2012 From: kos...@ (David Sheets) Date: Thu, 5 Jul 2012 14:54:51 -0700 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: <4FF5468E.5000603@hicorp.co.jp> References: <4FF5468E.5000603@hicorp.co.jp> Message-ID: On Thu, Jul 5, 2012 at 12:47 AM, Mark Callow wrote: > I have completed the initial draft of a proposed WEBGL_dynamic_texture > extension. The purpose of this is to provide an API for handling textures > with dynamic images such as video that can be implemented at the maximum > efficiency possible on a given platform, including with zero copies. > > Please look at the issues list before posting any questions as it contains > many of the questions you are likely to have and several answers. > > You can find the proposal at: > > http://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_dynamic_texture > > I welcome your comments and help to resolve all the listed issues. This extension looks great, Mark. Thank you! A few questions: 1. Can the source interface be standardized to allow, e.g., WebGL FBO-based dynamic texture sources? I would like to construct generic WebGL rendering modules that expect something satisfying a dynamic texture interface that isn't necessarily an HTML Canvas element. I believe this involves defining the dual to the IDL to provide an object with 4 functions like onSet, onUnset, onAcquire, and onRelease. 2. Is there any performance penalty for using this functionality with static textures vs the present texturing functions? If not, is it possible to formulate and include a concise description of the WebGL core texture functionality in WEBGL_dynamic_texture terms? 3. samplerExternalOES is a new abstract GLSL type (no constructors). How would you like to declare this? I see right now, it is called a "" without HTML generation. Perhaps "" with corresponding boilerplate text? Would you like me to draft this tag? 4. doesn't exist but is referenced from Dependencies. Where is 1.0.1 hosted? 5. If WEBGL_dynamic_texture is the name of the WebGL extension which mirrors but restricts OES_EGL_image_external and OES_EGL_image_external is still used in GLSL for cross compat reasons, should this connection be made more prominent in the extension spec text? The spec document currently has all the semantic information necessary to extract this link. I wonder if using WEBGL_dynamic_texture in JS and OES_EGL_image_external in GLSL otherwise appears to casual readers as an OES => WebGL oversight. Perhaps WebGL could create a GLSL extension alias "WEBGL_dynamic_texture"? Do you foresee any more extension name contexts besides JS and GLSL? Cheers, David > Regards > > -Mark > > > -- > ???????????????????????????????????????????????????????????????? > ???????????????????????????????????????????????????????????????? ??? > > NOTE: This electronic mail message may contain confidential and privileged > information from HI Corporation. If you are not the intended recipient, any > disclosure, photocopying, distribution or use of the contents of the > received information is prohibited. If you have received this e-mail in > error, please notify the sender immediately and permanently delete this > message and all related copies. ----------------------------------------------------------- You are currently subscribed to public_webgl...@ To unsubscribe, send an email to majordomo...@ with the following command in the body of your email: unsubscribe public_webgl ----------------------------------------------------------- From cal...@ Thu Jul 5 19:14:37 2012 From: cal...@ (Mark Callow) Date: Fri, 06 Jul 2012 11:14:37 +0900 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> Message-ID: <4FF64A0D.80309@hicorp.co.jp> On 06/07/2012 02:43, Florian B?sch wrote: > On Thu, Jul 5, 2012 at 7:39 PM, Gregg Tavares (??) > wrote: > > Zero copy is the whole point of this extension. If you want the > copy path just call texImage2D(....., HTMLVideoElement). Browsers > are already working on making that path use an FBO to speed up the > copy. But it's still a copy. Copying 1920x1080p video at 60hz is a > much slower and battery intensive operation than not copying at all. > > The only way this texture type can be accommodated without breaking > all existing shaders and forcing everybody to adopt shader > preprocessors as well as forcing everybody to define their sampler > types up front for this flavor is to *copy* so we can get it into a > texture that we can drop into our existing > frameworks/libraries/utilities that just works. If the aim of this > extension is to get rid of the copy, than that is mostly a failure, > except for border-use cases where people incidentially have a friendly > and simple enough app structure and framework so that works. I strongly disagree that frameworks can't handle this. As long as the framework provides a way for the application to tell it what type of texture image it wants to use, many frameworks will be able to handle the separate sampler type without major problem. Existing frameworks of course have no mechanism for the app. to indicate the type of texture image so they will require modification. I also strongly disagree that the cases where a framework (designed for handling dynamic images) would not need to copy are "border-use cases." The impetus for drafting this proposal was a discussion with a set-top box maker who needs to avoid the copy. (Isn't set-top box a ridiculously out-dated expression; who has a TV set any more with a top on which anything can be placed?) Due to the architecture of OpenGL, the only time a check and recompile can be done, if a single sampler type is used, is during the draw call. I don't see any way to change that as uniform values can be changed at any time. Such a check would slow down every application. Even if a single sampler type is used, the end result will be multiple program binaries that need to be stored so it does not save a whole lot compared to an app. or framework providing multiple shaders up front. I also think it is likely that you'll want a different fragment shader for your video textures anyway, regardless of the sampler type. Regards -Mark -- ???????????????????????????????????? ???????????????????????????? ??????? ???????????????????????????????????? ????????????????????? ??? NOTE: This electronic mail message may contain confidential and privileged information from HI Corporation. If you are not the intended recipient, any disclosure, photocopying, distribution or use of the contents of the received information is prohibited. If you have received this e-mail in error, please notify the sender immediately and permanently delete this message and all related copies. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cal...@ Thu Jul 5 19:28:59 2012 From: cal...@ (Mark Callow) Date: Fri, 06 Jul 2012 11:28:59 +0900 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> Message-ID: <4FF64D6B.4050405@hicorp.co.jp> On 06/07/2012 02:04, Gregg Tavares (??) wrote: > Looks good Mark. > > A few comments: > > *) is naming all the functions dynamicTextureXXX redundant? The > functions are called off the extension object so > > var dynamicTextureExtension = gl.getExtenion("WEBGL_dynamic_texture"); > dynamicTextureExtension.setSource(texture, src); > > might be enough? If it remains solely an extension then yes but I was considering the cases of going directly into core and eventually going into core. Do you want the method names to change then? Handling extensions that move into core is something I don't think we've discussed. In native OpenGL it is easy to provide undecorated names when and extension moves into core while also maintaining the decorated names for a while. > > *) does dynamicTextureGetSource seem un GL like? > > Maybe getDynamicTextureParameter(param) in case something needs to be > added later? I don't see a need. There's only one texture type so getTextureParameter() can be used. The un-GL-like point about dynamicTexture[GS]etSource is the direct access to the texture object instead of having to bind it first. If we want to continue the direct access direction and expect more dynamic texture related parameters getDynamicTextureParameter(tex, param) would be better. > > *) is the state before calling dynamicTextureAquaireFrame or after > calling dynamicTextureReleaseFrame desirable? > > It means the behavior of the program is implementation dependent. It > could be that on some browsers, never calling either function works > perfectly and on other browsers or platforms it performs poorly or not > at all. Where as if there is no dynamicTextureReleaseFrame and you're > required to call dyanicTextureAcquireFrame to see anything at all it > seems more likely you'd get consistent behavior across implementations? Are you saying that between *ReleaseFrame and *AcquireFrame the sampler should return (0,0,0,1)? API and spec-wise that's fine. I'm not sure how to implement that on top of OES_EGL_image_external. If it's implementable, I'd be in favour. Regards -Mark -- ???????????????????????????????????? ???????????????????????????? ??????? ???????????????????????????????????? ????????????????????? ??? NOTE: This electronic mail message may contain confidential and privileged information from HI Corporation. If you are not the intended recipient, any disclosure, photocopying, distribution or use of the contents of the received information is prohibited. If you have received this e-mail in error, please notify the sender immediately and permanently delete this message and all related copies. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cal...@ Thu Jul 5 19:45:43 2012 From: cal...@ (Mark Callow) Date: Fri, 06 Jul 2012 11:45:43 +0900 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> Message-ID: <4FF65157.30607@hicorp.co.jp> On 06/07/2012 02:11, Gregg Tavares (??) wrote: > A couple more comments > > * samplerExternalOES or samplerDynamicTextureWEBGL? > > I'm just wondering if there will be any future conflict. I don't want > to end up in a case where we some how have to distinguish between real > uses of samplesExternalOES and ones that have to be re-written for > WebGL dynamic textures. > > But maybe that will never come up? > > * #extension OES_EGL_image_external : enable or #extension > WEBGL_dynamic_texture : enable? > > Similarly, do I need to worry about any future conflicts? > Conflicts seem unlikely. OES_EGL_image_external is not going to change. The functionality exposed in WEBGL_dynamic_texture is identical. I don't have any strong feeling about it which is why the issue is unresolved. Regards -Mark -- ???????????????????????????????????? ???????????????????????????? ??????? ???????????????????????????????????? ????????????????????? ??? NOTE: This electronic mail message may contain confidential and privileged information from HI Corporation. If you are not the intended recipient, any disclosure, photocopying, distribution or use of the contents of the received information is prohibited. If you have received this e-mail in error, please notify the sender immediately and permanently delete this message and all related copies. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cal...@ Thu Jul 5 20:01:02 2012 From: cal...@ (Mark Callow) Date: Fri, 06 Jul 2012 12:01:02 +0900 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> <4FF55650.1020301@hicorp.co.jp> Message-ID: <4FF654EE.9010809@hicorp.co.jp> On 06/07/2012 02:52, Brandon Jones wrote: > On Thu, Jul 5, 2012 at 1:54 AM, Mark Callow > wrote: > > > If you use the same sampler type the implementation has to figure > out at run-time what type of image it is sampling and may have to > recompile the shader when switching between yuv and rgb images. > > > But as Florian mentioned most frameworks will be forced to do that > anyway. Wouldn't it be preferable to have that happen in native code? What the framework needs to do is a little different. It needs the application to tell it that it wants to use a dynamic image source and then generate or select an appropriate shader. All this can be done before the first draw call. As I mentioned earlier in the thread with the OpenGL architecture there is nowhere other than the draw call where the check and re-compilation could happen. > > It's not wonderful either way but I believe that the majority of > applications will know in advance which textures will use dynamic > sources. > > > That feels like some sort of hinting system would be more appropriate. Possibly something like setSamplerVariableAsExternalSampler(program, "my_sampler_variable", GL_TRUE);? IIt would have to be called before shader compilation. I'm not a WebGL implementer so I'm not sure how much that might reduce the complexity vs. having nothing. I think it would still impose a requirement for draw time checks that would reduce performance for every application. Regards -Mark -- ???????????????????????????????????? ???????????????????????????? ??????? ???????????????????????????????????? ????????????????????? ??? NOTE: This electronic mail message may contain confidential and privileged information from HI Corporation. If you are not the intended recipient, any disclosure, photocopying, distribution or use of the contents of the received information is prohibited. If you have received this e-mail in error, please notify the sender immediately and permanently delete this message and all related copies. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cal...@ Thu Jul 5 20:28:58 2012 From: cal...@ (Mark Callow) Date: Fri, 06 Jul 2012 12:28:58 +0900 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: <1DFF71A7-51BD-4EBD-9D26-83E3D8357C15@apple.com> References: <4FF5468E.5000603@hicorp.co.jp> <1DFF71A7-51BD-4EBD-9D26-83E3D8357C15@apple.com> Message-ID: <4FF65B7A.9010705@hicorp.co.jp> On 06/07/2012 06:02, Chris Marrin wrote: > > There are a couple of inconsistencies: > > 1) The IDL says "dynamicTextureAcquireImage" and the description says > "dynamicTextureAcquireFrame". Same for release. Fixed. Thanks. The search/replace in my XML editor when I decided to change the name, missed these because they are element attributes. > > 2) dynamicTextureSetSource passes the texture in the IDL and > description, but not in the example. Also fixed. > Given the nature of OpenGL, seems like you should not pass the > texture, but rather use the currently bound texture? Same would be > true of all the other API calls. Yeah. See issues #3, #4 & #9, currently unresolved. I was thinking that an app. might have dynamic textures on several texture units and doing activeTexture/bindTexture on each in order to call acquireImage was a bit much. This led me to pass to acquireImage. Once I'd done that, I decided to make setSource direct access as well. > > I am also concerned about timestamping. ... > > Your proposal doesn't deal with timestamps, so it's possible to get > out of sync with the decode rate, causing strobing or other > undesirable effects. How do you handle this today when an HTMLVideoElement is passed to texImage2D? I'm not sure that strobing, etc. will really be a problem. Basically the images are triple buffered, as Gregg described upthread, and you will always get the latest available frame. The worst that will happen is a few dropped frames. If the app. is drawing so fast it gets the same video frame more than once in succession, I don't think that is a problem. But you bring up a good point and this definitely needs consideration. Regards -Mark -- ???????????????????????????????????? ???????????????????????????? ??????? ???????????????????????????????????? ????????????????????? ??? NOTE: This electronic mail message may contain confidential and privileged information from HI Corporation. If you are not the intended recipient, any disclosure, photocopying, distribution or use of the contents of the received information is prohibited. If you have received this e-mail in error, please notify the sender immediately and permanently delete this message and all related copies. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cal...@ Thu Jul 5 20:57:22 2012 From: cal...@ (Mark Callow) Date: Fri, 06 Jul 2012 12:57:22 +0900 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> Message-ID: <4FF66222.8080702@hicorp.co.jp> On 06/07/2012 06:54, David Sheets wrote: > 1. Can the source interface be standardized to allow, e.g., WebGL > FBO-based dynamic texture sources? I would like to construct generic > WebGL rendering modules that expect something satisfying a dynamic > texture interface that isn't necessarily an HTML Canvas element. I > believe this involves defining the dual to the IDL to provide an > object with 4 functions like onSet, onUnset, onAcquire, and onRelease. The point about this extension is that the texture images are updated outside the control of WebGL and the WebGL application. The case you describe is wholly within the GL so I do not see how {acquire,release}Image bring any benefit. Currently all you need do is rebind the texture from the FBO to the appropriate texture target. I do not understand "dual to the IDL ...". > 2. Is there any performance penalty for using this functionality with > static textures vs the present texturing functions? Why would you want to do that? > If not, is it > possible to formulate and include a concise description of the WebGL > core texture functionality in WEBGL_dynamic_texture terms? The core texture functionality remains unchanged. If you bind the texture to TEXTURE2D or one of the other existing targets, data is sampled from the regular image store that is set via tex{,Sub,}Image* or copyTex{,Sub}Image. > 3. samplerExternalOES is a new abstract GLSL type (no constructors). > How would you like to declare this? I see right now, it is called a > "" without HTML generation. Perhaps "" > with corresponding boilerplate text? Would you like me to draft this > tag? Yes please. I hadn't even noticed that using meant the type didn't appear in the generate HTML. > 4. doesn't exist > but is referenced from Dependencies. Where is 1.0.1 hosted? Fixed. It should have been 1.0. I was confusing it with the CTS version. > 5. If WEBGL_dynamic_texture is the name of the WebGL extension which > mirrors but restricts OES_EGL_image_external and > OES_EGL_image_external is still used in GLSL for cross compat reasons, > should this connection be made more prominent in the extension spec > text? The spec document currently has all the semantic information > necessary to extract this link. I wonder if using > WEBGL_dynamic_texture in JS and OES_EGL_image_external in GLSL > otherwise appears to casual readers as an OES => WebGL oversight. > Perhaps WebGL could create a GLSL extension alias > "WEBGL_dynamic_texture"? See issue #5. How would you make the connection "more prominent?" > Do you foresee any more extension name > contexts besides JS and GLSL? No. Regards -Mark -- ???????????????????????????????????? ???????????????????????????? ??????? ???????????????????????????????????? ????????????????????? ??? NOTE: This electronic mail message may contain confidential and privileged information from HI Corporation. If you are not the intended recipient, any disclosure, photocopying, distribution or use of the contents of the received information is prohibited. If you have received this e-mail in error, please notify the sender immediately and permanently delete this message and all related copies. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jba...@ Thu Jul 5 21:01:35 2012 From: jba...@ (John Bauman) Date: Thu, 5 Jul 2012 21:01:35 -0700 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: <4FF55650.1020301@hicorp.co.jp> References: <4FF5468E.5000603@hicorp.co.jp> <4FF55650.1020301@hicorp.co.jp> Message-ID: On Thu, Jul 5, 2012 at 1:54 AM, Mark Callow wrote: > On 2012/07/05 17:39, Florian B?sch wrote: > > On Thu, Jul 5, 2012 at 9:47 AM, Mark Callow wrote: > >> I welcome your comments and help to resolve all the listed issues. >> > I like the functionality. I dislike the use of a separate sampler type as > in: > uniform samplerExternalOES videoSampler; > > It means shaders have to be pre-processed according to the type of > texture passed in, and that makes it quite a lot harder to support both > static and dynamic textures at the same time, as well as writing > utilities/libraries around filtering that work regardless. > > If you use the same sampler type the implementation has to figure out > at run-time what type of image it is sampling and may have to recompile the > shader when switching between yuv and rgb images. It's not wonderful either > way but I believe that the majority of applications will know in advance > which textures will use dynamic sources. So I think the separate sampler > type is the way to go. The underlying OES extension made the same choice. > Presumably canvases (and many image elements) will be in RGB, so won't implementations have to recompile shaders anyway? > > > Regards > > -Mark > -- > ???????????????????????????????????????????????????????????????? > ???????????????????????????????????????????????????????????????? ??? > > NOTE: This electronic mail message may contain confidential and privileged > information from HI Corporation. If you are not the intended recipient, any > disclosure, photocopying, distribution or use of the contents of the > received information is prohibited. If you have received this e-mail in > error, please notify the sender immediately and permanently delete this > message and all related copies. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cal...@ Thu Jul 5 21:07:49 2012 From: cal...@ (Mark Callow) Date: Fri, 06 Jul 2012 13:07:49 +0900 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> <4FF55650.1020301@hicorp.co.jp> Message-ID: <4FF66495.9080801@hicorp.co.jp> On 06/07/2012 13:01, John Bauman wrote: > > Presumably canvases (and many image elements) will be in RGB, so won't > implementations have to recompile shaders anyway? I realized this just a few minutes before I saw your message. We may have to make this extension video only unless we can figure out a better way. Regards -Mark -- ???????????????????????????????????? ???????????????????????????? ??????? ???????????????????????????????????? ????????????????????? ??? NOTE: This electronic mail message may contain confidential and privileged information from HI Corporation. If you are not the intended recipient, any disclosure, photocopying, distribution or use of the contents of the received information is prohibited. If you have received this e-mail in error, please notify the sender immediately and permanently delete this message and all related copies. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pya...@ Fri Jul 6 01:12:58 2012 From: pya...@ (=?ISO-8859-1?Q?Florian_B=F6sch?=) Date: Fri, 6 Jul 2012 10:12:58 +0200 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: <4FF64A0D.80309@hicorp.co.jp> References: <4FF5468E.5000603@hicorp.co.jp> <4FF64A0D.80309@hicorp.co.jp> Message-ID: A set-top box maker who intends to write a custom app specifically targeted at displaying video is just one of those border use-cases. I know that there are insurmountable obstacles in making this thing work without some kind of shader semantic change. I wish there wasn't. Because if there was not, you could enjoy nearly instant support by any 3rd party code out there. I would wish to be copy-less to be as popular as possible. I've run personally into problems with copying in a previous project (too slow). But I just don't see how this will be uptaken if it breaks everything. On Fri, Jul 6, 2012 at 4:14 AM, Mark Callow wrote: > I strongly disagree that frameworks can't handle this. As long as the > framework provides a way for the application to tell it what type of > texture image it wants to use, many frameworks will be able to handle the > separate sampler type without major problem. Existing frameworks of course > have no mechanism for the app. to indicate the type of texture image so > they will require modification. > > I also strongly disagree that the cases where a framework (designed for > handling dynamic images) would not need to copy are "border-use cases." The > impetus for drafting this proposal was a discussion with a set-top box > maker who needs to avoid the copy. (Isn't set-top box a ridiculously > out-dated expression; who has a TV set any more with a top on which > anything can be placed?) > > Due to the architecture of OpenGL, the only time a check and recompile can > be done, if a single sampler type is used, is during the draw call. I don't > see any way to change that as uniform values can be changed at any time. > Such a check would slow down every application. > > Even if a single sampler type is used, the end result will be multiple > program binaries that need to be stored so it does not save a whole lot > compared to an app. or framework providing multiple shaders up front. I > also think it is likely that you'll want a different fragment shader for > your video textures anyway, regardless of the sampler type. > > > Regards > > -Mark > -- > ???????????????????????????????????????????????????????????????? > ???????????????????????????????????????????????????????????????? ??? > > NOTE: This electronic mail message may contain confidential and privileged > information from HI Corporation. If you are not the intended recipient, any > disclosure, photocopying, distribution or use of the contents of the > received information is prohibited. If you have received this e-mail in > error, please notify the sender immediately and permanently delete this > message and all related copies. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cal...@ Fri Jul 6 03:19:50 2012 From: cal...@ (Mark Callow) Date: Fri, 06 Jul 2012 19:19:50 +0900 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: References: <4FF5468E.5000603@hicorp.co.jp> <4FF64A0D.80309@hicorp.co.jp> Message-ID: <4FF6BBC6.8020308@hicorp.co.jp> On 06/07/2012 17:12, Florian B?sch wrote: > A set-top box maker who intends to write a custom app specifically > targeted at displaying video is just one of those border use-cases. Come off it. Any application is a custom app specifically targeted at something. > > I know that there are insurmountable obstacles in making this thing > work without some kind of shader semantic change. I wish there wasn't. > Because if there was not, you could enjoy nearly instant support by > any 3rd party code out there. I would wish to be copy-less to be as > popular as possible. I've run personally into problems with copying in > a previous project (too slow). But I just don't see how this will be > uptaken if it breaks everything. How do/will these frameworks handle the many sampler types in other versions of GLSL? There's a different type of sampler for every texture target. Regards -Mark -- ???????????????????????????????????? ???????????????????????????? ??????? ???????????????????????????????????? ????????????????????? ??? NOTE: This electronic mail message may contain confidential and privileged information from HI Corporation. If you are not the intended recipient, any disclosure, photocopying, distribution or use of the contents of the received information is prohibited. If you have received this e-mail in error, please notify the sender immediately and permanently delete this message and all related copies. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pya...@ Fri Jul 6 04:08:53 2012 From: pya...@ (=?ISO-8859-1?Q?Florian_B=F6sch?=) Date: Fri, 6 Jul 2012 13:08:53 +0200 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: <4FF6BBC6.8020308@hicorp.co.jp> References: <4FF5468E.5000603@hicorp.co.jp> <4FF64A0D.80309@hicorp.co.jp> <4FF6BBC6.8020308@hicorp.co.jp> Message-ID: On Fri, Jul 6, 2012 at 12:19 PM, Mark Callow wrote: > How do/will these frameworks handle the many sampler types in other > versions of GLSL? There's a different type of sampler for every texture > target. > sampler1D, sampler2D, sampler3D, samplerCube, sampler2DRect, sampler1DShadow, sampler2DShadow, sampler2DRectShadow, sampler2DMS and samplerCubeArrayShadow cannot be used in place of the other. Each of these brings a distinct functionality/semantic inside shaders that differs in how to use them. For instance, you would not be trying to drop in a samplerCube in place of a sampler2D. The sampler*Array vs. the sampler* functions are serviced by the same underlying sampler* type, sampler*Array is just an array of them, hence you would not try to drop in an array in place of a single sampler. isampler* and usampler* can be dropped in "in place" of floating point samplers, however the code using these is well aware that he only makes a convenience conversion to avoid a cast, this has no bearing outside of the shader. OpenGL hitherto contains *no* single sampler type that uses the exact same semantic yet only introduces the type to distinguish between an opaque internal conversion process. All types represent semantic concepts transparently and as such will not prompt variant compiling because "variants" would not make sense in their context. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kos...@ Fri Jul 6 14:04:38 2012 From: kos...@ (David Sheets) Date: Fri, 6 Jul 2012 14:04:38 -0700 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: <4FF66222.8080702@hicorp.co.jp> References: <4FF5468E.5000603@hicorp.co.jp> <4FF66222.8080702@hicorp.co.jp> Message-ID: On Thu, Jul 5, 2012 at 8:57 PM, Mark Callow wrote: > >> On 06/07/2012 06:54, David Sheets wrote: >> >> 1. Can the source interface be standardized to allow, e.g., WebGL >> FBO-based dynamic texture sources? I would like to construct generic >> WebGL rendering modules that expect something satisfying a dynamic >> texture interface that isn't necessarily an HTML Canvas element. I >> believe this involves defining the dual to the IDL to provide an >> object with 4 functions like onSet, onUnset, onAcquire, and onRelease. > > The point about this extension is that the texture images are updated > outside the control of WebGL and the WebGL application. The case you > describe is wholly within the GL so I do not see how {acquire,release}Image > bring any benefit. Currently all you need do is rebind the texture from the > FBO to the appropriate texture target. The variants of dynamicTextureSetSource imply that HTMLCanvasElement, HTMLImageElement, and HTMLVideoElement share an interface supporting dynamicTextureAcquireImage and dynamicTextureReleaseImage. What is this interface? > I do not understand "dual to the IDL ...". This would be the common texture source interface that each texture source implementation supplies to support the WEBGL_dynamic_texture IDL. WEBGL_dynamic_texture introduces new texture pipeline operations. Can both sides of this interface be standardized in this extension? >> 2. Is there any performance penalty for using this functionality with >> static textures vs the present texturing functions? > > Why would you want to do that? If there is no penalty, then WEBGL_dynamic_texture defines a superset of the core texture functionality. By using only WEBGL_dynamic_texture, fewer distinct paths are exercised, fewer special cases are necessary, and the texture interfaces are unified. By defining the present texture functions in terms of WEBGL_dynamic_texture, WEBGL_dynamic_texture is understood to expose previously implicit locking to the application. Future implementors are then able to directly implement WEBGL_dynamic_texture and provide the core WebGL texture functions as a subset of this interface in a standard way. It also means that: #define sampler2D samplerExternalOES in the shader preamble solves Florian's problem if the app developer always uses WEBGL_dynamic_texture even for static textures. >> If not, is it >> possible to formulate and include a concise description of the WebGL >> core texture functionality in WEBGL_dynamic_texture terms? > > The core texture functionality remains unchanged. If you bind the texture to > TEXTURE2D or one of the other existing targets, data is sampled from the > regular image store that is set via tex{,Sub,}Image* or copyTex{,Sub}Image. Is the core TEXTURE2D functionality a subset of WEBGL_dynamic_texture's functionality? >> 3. samplerExternalOES is a new abstract GLSL type (no constructors). >> How would you like to declare this? I see right now, it is called a >> "" without HTML generation. Perhaps "" >> with corresponding boilerplate text? Would you like me to draft this >> tag? > > Yes please. I hadn't even noticed that using meant the type didn't > appear in the generate HTML. The patch is attached to the bug . >> 4. doesn't exist >> but is referenced from Dependencies. Where is 1.0.1 hosted? > > Fixed. It should have been 1.0. I was confusing it with the CTS version. What do you mean by "the amended OpenGL ES 2.0 API specification" in overview bullet 2? What is the URL of this document? Does it have a versioning scheme? >> 5. If WEBGL_dynamic_texture is the name of the WebGL extension which >> mirrors but restricts OES_EGL_image_external and >> OES_EGL_image_external is still used in GLSL for cross compat reasons, >> should this connection be made more prominent in the extension spec >> text? The spec document currently has all the semantic information >> necessary to extract this link. I wonder if using >> WEBGL_dynamic_texture in JS and OES_EGL_image_external in GLSL >> otherwise appears to casual readers as an OES => WebGL oversight. >> Perhaps WebGL could create a GLSL extension alias >> "WEBGL_dynamic_texture"? > > See issue #5. How would you make the connection "more prominent?" Perhaps some specific generated text that explicitly states the usable extension names in JS and in GLSL? OES_standard_derivatives has a similar but less severe discrepancy between OES_standard_derivatives and GL_OES_standard_derivatives. I propose making extension names in GLSL identical to those in JS and providing GLSL extension aliases for those legacy names that do not match. The legacy names would then be deprecated and the extension specification XML and HTML documents would contain information sufficient to transform a legacy shader into a modern shader. Aside: using URIs to identify extensions would enable the construction of generic shader analysis tools to perform this update and future updates like this automatically. Alternately, a programmatic means to retrieve GLSL extension names from JS extension names at run-time could be specified. Regards, David >> Do you foresee any more extension name >> contexts besides JS and GLSL? > > No. > > Regards > > -Mark > > -- > ???????????????????????????????????????????????????????????????? > ???????????????????????????????????????????????????????????????? ??? > > NOTE: This electronic mail message may contain confidential and privileged > information from HI Corporation. If you are not the intended recipient, any > disclosure, photocopying, distribution or use of the contents of the > received information is prohibited. If you have received this e-mail in > error, please notify the sender immediately and permanently delete this > message and all related copies. > > > ----------------------------------------------------------- You are currently subscribed to public_webgl...@ To unsubscribe, send an email to majordomo...@ with the following command in the body of your email: unsubscribe public_webgl ----------------------------------------------------------- From cma...@ Fri Jul 6 14:17:17 2012 From: cma...@ (Chris Marrin) Date: Fri, 06 Jul 2012 14:17:17 -0700 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: <4FF65B7A.9010705@hicorp.co.jp> References: <4FF5468E.5000603@hicorp.co.jp> <1DFF71A7-51BD-4EBD-9D26-83E3D8357C15@apple.com> <4FF65B7A.9010705@hicorp.co.jp> Message-ID: <6BB8F10C-535A-42C0-AA9E-C8CB1D093383@apple.com> On Jul 5, 2012, at 8:28 PM, Mark Callow wrote: > On 06/07/2012 06:02, Chris Marrin wrote: >> >> There are a couple of inconsistencies: >> >> 1) The IDL says "dynamicTextureAcquireImage" and the description says "dynamicTextureAcquireFrame". Same for release. > Fixed. Thanks. The search/replace in my XML editor when I decided to change the name, missed these because they are element attributes. >> >> 2) dynamicTextureSetSource passes the texture in the IDL and description, but not in the example. > Also fixed. >> Given the nature of OpenGL, seems like you should not pass the texture, but rather use the currently bound texture? Same would be true of all the other API calls. > Yeah. See issues #3, #4 & #9, currently unresolved. > > I was thinking that an app. might have dynamic textures on several texture units and doing activeTexture/bindTexture on each in order to call acquireImage was a bit much. This led me to pass to acquireImage. Once I'd done that, I decided to make setSource direct access as well. But it wouldn't be any more work that the texImage2D you'd be doing today without using dynamic textures. I think it would be best to avoid changing the OpenGL API model with this extension. >> >> I am also concerned about timestamping. ... >> >> Your proposal doesn't deal with timestamps, so it's possible to get out of sync with the decode rate, causing strobing or other undesirable effects. > How do you handle this today when an HTMLVideoElement is passed to texImage2D? Poorly :-) The only thing you can do is get the image that happens to be the current frame at the moment you get it. It makes for very poor playback behavior, even ignoring all the other overhead that makes today's scheme so bad. > > I'm not sure that strobing, etc. will really be a problem. Basically the images are triple buffered, as Gregg described upthread, and you will always get the latest available frame. The worst that will happen is a few dropped frames. If the app. is drawing so fast it gets the same video frame more than once in succession, I don't think that is a problem. Whether or not images are "triple buffered" is an implementation detail. Video decoders tend to be very non-linear in their behavior. It might take a lot longer to decode an I-frame than a B or P frame. Some decoders buffer several frames, so decoding and expensive frame can happen while the client is consuming several inexpensive frames. The ImageQueues used in OSX and iOS allows the decoder and consumer to be very independent. It allows the decoder to get ahead of the renderer to provide glitch free video playback. But it also allows the system to throw away decoded frames that have become stale without being consumed because the renderer is not able to keep up with the frame rate for some reason. This would result in a reduced frame rate, but the video would still play at the correct rate. I think our goal should be to allow full frame rate playback of perfectly synced and paced video in a 3D scene. But to do that we need to do better than just letting the video decoder give us whatever frame happens to be ready. The renderer really needs full awareness of media timing, which needs to be communicated in both directions between the media provider and the renderer. ----- ~Chris Marrin cmarrin...@ -------------- next part -------------- An HTML attachment was scrubbed... URL: From pya...@ Fri Jul 6 14:27:23 2012 From: pya...@ (=?ISO-8859-1?Q?Florian_B=F6sch?=) Date: Fri, 6 Jul 2012 23:27:23 +0200 Subject: [Public WebGL] WEBGL_dynamic_texture extension proposal In-Reply-To: <6BB8F10C-535A-42C0-AA9E-C8CB1D093383@apple.com> References: <4FF5468E.5000603@hicorp.co.jp> <1DFF71A7-51BD-4EBD-9D26-83E3D8357C15@apple.com> <4FF65B7A.9010705@hicorp.co.jp> <6BB8F10C-535A-42C0-AA9E-C8CB1D093383@apple.com> Message-ID: On Fri, Jul 6, 2012 at 11:17 PM, Chris Marrin wrote: > I think our goal should be to allow full frame rate playback of perfectly > synced and paced video in a 3D scene. But to do that we need to do better > than just letting the video decoder give us whatever frame happens to be > ready. The renderer really needs full awareness of media timing, which > needs to be communicated in both directions between the media provider and > the renderer. > The