From ste...@ Sun Jan 3 04:51:59 2010 From: ste...@ (stephen white) Date: Sun, 3 Jan 2010 23:21:59 +1030 Subject: [Public WebGL] Shaders, glReadPixels and speed Message-ID: <71E7F509-6F11-43C8-AEB9-672B59944A26@adam.com.au> I am finding that WebGL has a per pixel performance that is amazing, but a per item performance that is very slow. The speed between the vertex and fragment shaders is unhampered, but things bog down in two main areas: * Transferring graphics back to the browser instead of drawing directly to screen. This makes a full screen 30x30 textured sphere slower than the same thing in Flash set to wmode=opaque. WebGL needs an equivalent wmode. * Javascript performing matrix math operations then feeding it to the card variable by variable. The maths is optimised by the JIT, but the bridging cost is too large to help by having a C based implementation. PROPOSAL: WebGL needs to specify a geometry shader API, with a fallback software implementation as a requirement. I know geometry shaders are not in OpenGL ES, however there is a increased emphasis on the need for geometry shaders as a result of interfacing to Javascript. C level code can manage with vertex/ fragment shaders alone as the CPU will be fully utilised. Javascript does not have that luxury, therefore it needs geometry shaders. Geometry shaders can be mostly implemented with vertex/fragment shaders, so the overhead would be minimised on platforms without hardware support. The flexibility of geometry shaders also forms the basis for writing hardware accelerated scenegraph engines, which would be WebGL's answer to Google's O3D and their amazing island. -- steve...@ ----------------------------------------------------------- You are currently subscribe to public_webgl...@ To unsubscribe, send an email to majordomo...@ with the following command in the body of your email: From hao...@ Sun Jan 3 07:11:02 2010 From: hao...@ (Hao Nguyen) Date: Sun, 3 Jan 2010 10:11:02 -0500 Subject: [Public WebGL] WebGLAttributeLocation instead of GLint? Message-ID: <85f6b291001030711w244aaafai283ae53980283497@mail.gmail.com> I think it would be more intuitive to have vertex attribute index to be an interface like WebGLAttributeLocation (similar to WebGLUniformLocation) rather than using an GLint. After all, we are not modifying the value of that integer anyway. Then, we can get rid of bindAttribLocation(), and have the context implementation binds a unique index to an attribute variable when getAttribLocation() is called for the first time. ----------------------------------------------------------- You are currently subscribe to public_webgl...@ To unsubscribe, send an email to majordomo...@ with the following command in the body of your email: From kbr...@ Sun Jan 3 10:54:39 2010 From: kbr...@ (Kenneth Russell) Date: Sun, 3 Jan 2010 13:54:39 -0500 Subject: [Public WebGL] Shaders, glReadPixels and speed In-Reply-To: <71E7F509-6F11-43C8-AEB9-672B59944A26@adam.com.au> References: <71E7F509-6F11-43C8-AEB9-672B59944A26@adam.com.au> Message-ID: <921bb11001031054v3765a3e2x5b7ef9a372b6ae51@mail.gmail.com> On Sun, Jan 3, 2010 at 7:51 AM, stephen white wrote: > I am finding that WebGL has a per pixel performance that is amazing, but a > per item performance that is very slow. The speed between the vertex and > fragment shaders is unhampered, but things bog down in two main areas: > > * Transferring graphics back to the browser instead of drawing directly to > screen. This makes a full screen 30x30 textured sphere slower than the same > thing in Flash set to wmode=opaque. WebGL needs an equivalent wmode. What browser and operating system are you using? Safari on Mac OS X currently has the advantage of hardware accelerated compositing, but all browser vendors are working on adding this, and it will eliminate the glReadPixels currently needed to put WebGL on the screen. Adding a wmode=opaque will be a temporary workaround which we don't want to have to support forever. > * Javascript performing matrix math operations then feeding it to the card > variable by variable. The maths is optimised by the JIT, but the bridging > cost is too large to help by having a C based implementation. Could you please provide a test case which demonstrates the performance problem? If you are finding that all of your time is spent in uniformMatrix4fv calls, then you should be seeing the same performance issue in C. I can only speak for Google but we have already done some optimization of data transfer from JavaScript to WebGL. More work is certainly needed but the raw speed of sending down floating point numbers via WebGLFloatArray is already pretty good. > PROPOSAL: WebGL needs to specify a geometry shader API, with a fallback > software implementation as a requirement. I don't think a requirement of a software fallback is viable. As far as I can tell the entire vertex and geometry shader path would have to fall back to software, and this would kill performance. In a future revision of the WebGL spec we could consider having a capability bit which would allow the use of geometry shaders on capable hardware, but I don't think that we should emulate the functionality. -Ken > I know geometry shaders are not in OpenGL ES, however there is a increased > emphasis on the need for geometry shaders as a result of interfacing to > Javascript. ?C level code can manage with vertex/fragment shaders alone as > the CPU will be fully utilised. Javascript does not have that luxury, > therefore it needs geometry shaders. > > Geometry shaders can be mostly implemented with vertex/fragment shaders, so > the overhead would be minimised on platforms without hardware support. The > flexibility of geometry shaders also forms the basis for writing hardware > accelerated scenegraph engines, which would be WebGL's answer to Google's > O3D and their amazing island. > > -- > ?steve...@ > > ----------------------------------------------------------- > You are currently subscribe to public_webgl...@ > To unsubscribe, send an email to majordomo...@ with > the following command in the body of your email: > > ----------------------------------------------------------- You are currently subscribe to public_webgl...@ To unsubscribe, send an email to majordomo...@ with the following command in the body of your email: From ste...@ Sun Jan 3 21:00:06 2010 From: ste...@ (stephen white) Date: Mon, 4 Jan 2010 15:30:06 +1030 Subject: [Public WebGL] Shaders, glReadPixels and speed In-Reply-To: <921bb11001031054v3765a3e2x5b7ef9a372b6ae51@mail.gmail.com> References: <71E7F509-6F11-43C8-AEB9-672B59944A26@adam.com.au> <921bb11001031054v3765a3e2x5b7ef9a372b6ae51@mail.gmail.com> Message-ID: On 04/01/2010, at 5:24 AM, Kenneth Russell wrote: > What browser and operating system are you using? Safari on Mac OS X Mac OS X 10.5, which probably lacks the hardware composition in the browser (despite being nightly). It is good to know that all browsers are moving towards hardware compositing to address this bottleneck. > needed but the raw speed of sending down floating point numbers via > WebGLFloatArray is already pretty good. I was referring more to the step by step nature of needing to feed variables through, which leads to this kind of analysis: Gregg Tavares writes: >> 1 call to for each matrix you want to pass to the shader (usually 1 >> to 4 matrices) >> 1 call for each color parameter ( for phong the minimum would be 2, >> color >> and shininess though most phong shaders have 5, emissive, ambient, >> diffuse, >> specular, shininess) >> 1 call to setup position vertices >> 1 call to setup normals >> If it's textured you'll need another call to supply UVs >> 1 call to finally draw the object >> and then possibly a few calls to restore GL state. >> >> That a minimum of 5 calls and in this case a maximum of 13 per >> object, per >> frame. JavaScript is going to have a tough time doing that for more >> than a >> few objects and keep 60 or even 30hz. Rather than having a custom engine like O3D, it seems less controversial to add in a standard GL_ARB_geometry_shader4 so that WebGL is capable of displaying and animating 3d graphics to an equivalent level. The shaders have the vector and matrix functionality that is a painful library in Javascript, so is there a way of pushing the code across to that domain? Use a fragment shader and render vertex data to a texture? -- steve...@ ----------------------------------------------------------- You are currently subscribe to public_webgl...@ To unsubscribe, send an email to majordomo...@ with the following command in the body of your email: From vla...@ Sun Jan 3 21:20:05 2010 From: vla...@ (Vladimir Vukicevic) Date: Sun, 03 Jan 2010 21:20:05 -0800 Subject: [Public WebGL] Shaders, glReadPixels and speed In-Reply-To: References: <71E7F509-6F11-43C8-AEB9-672B59944A26@adam.com.au> <921bb11001031054v3765a3e2x5b7ef9a372b6ae51@mail.gmail.com> Message-ID: <4B417A85.1050302@mozilla.com> On 1/3/2010 9:00 PM, stephen white wrote: > I was referring more to the step by step nature of needing to feed > variables through, which leads to this kind of analysis: > > Gregg Tavares writes: >>> 1 call to for each matrix you want to pass to the shader (usually 1 >>> to 4 matrices) >>> 1 call for each color parameter ( for phong the minimum would be 2, >>> color >>> and shininess though most phong shaders have 5, emissive, ambient, >>> diffuse, >>> specular, shininess) >>> 1 call to setup position vertices >>> 1 call to setup normals >>> If it's textured you'll need another call to supply UVs >>> 1 call to finally draw the object >>> and then possibly a few calls to restore GL state. >>> >>> That a minimum of 5 calls and in this case a maximum of 13 per >>> object, per >>> frame. JavaScript is going to have a tough time doing that for more >>> than a >>> few objects and keep 60 or even 30hz. > > Rather than having a custom engine like O3D, it seems less > controversial to add in a standard GL_ARB_geometry_shader4 so that > WebGL is capable of displaying and animating 3d graphics to an > equivalent level. > > The shaders have the vector and matrix functionality that is a painful > library in Javascript, so is there a way of pushing the code across to > that domain? Use a fragment shader and render vertex data to a texture? Are the calls that Gregg outlined above actually showing up as a bottleneck? If they are, that's a performance bug that I'm pretty sure we'll all be very motivated to fix; geometry shaders don't really replace much of that work, and emulating them in software and/or trying to do a bolt-on to GL ES seems like a bad way to fix these problems. As Ken mentioned, the best thing to do is to send along some benchmarks showing bottlenecks that we can jump on. For Firefox, we're moving to hardware compositing for WebGL in the next month or so (hopefully next few weeks actually, depending on when some patches land), and I have some patches in the queue that should significantly speed up all WebGL calls and the WebGL array types. But again, any benchmarks/testcases showing problems would be very welcome. - Vlad ----------------------------------------------------------- You are currently subscribe to public_webgl...@ To unsubscribe, send an email to majordomo...@ with the following command in the body of your email: From ste...@ Mon Jan 4 00:54:59 2010 From: ste...@ (stephen white) Date: Mon, 4 Jan 2010 19:24:59 +1030 Subject: [Public WebGL] Shaders, glReadPixels and speed In-Reply-To: <4B417A85.1050302@mozilla.com> References: <71E7F509-6F11-43C8-AEB9-672B59944A26@adam.com.au> <921bb11001031054v3765a3e2x5b7ef9a372b6ae51@mail.gmail.com> <4B417A85.1050302@mozilla.com> Message-ID: On 04/01/2010, at 3:50 PM, Vladimir Vukicevic wrote: > Are the calls that Gregg outlined above actually showing up as a > bottleneck? If they are, that's a performance bug that I'm pretty > sure we'll all be very motivated to fix; geometry shaders don't > really replace much of that work, and emulating them in software and/ > or trying to do a bolt-on to GL ES seems like a bad way to fix these > problems. As Ken mentioned, the best thing to do is to send along > some benchmarks showing bottlenecks that we can jump on. I'll upgrade my OS as the glReadPixels bottleneck is dominating my results at the moment. I'll keep this area in mind as I experiment more, and it seems readily testable by creating thousands of independently animated objects. If WebGL is expected to do the same level of graphics as the Beach demo, then I agree that there isn't a problem. :) -- steve...@ ----------------------------------------------------------- You are currently subscribe to public_webgl...@ To unsubscribe, send an email to majordomo...@ with the following command in the body of your email: From cma...@ Mon Jan 4 08:15:58 2010 From: cma...@ (Chris Marrin) Date: Mon, 04 Jan 2010 08:15:58 -0800 Subject: [Public WebGL] WebGLAttributeLocation instead of GLint? In-Reply-To: <85f6b291001030711w244aaafai283ae53980283497@mail.gmail.com> References: <85f6b291001030711w244aaafai283ae53980283497@mail.gmail.com> Message-ID: <2DD5B4CC-A990-4C4F-801B-942E812A970F@apple.com> On Jan 3, 2010, at 7:11 AM, Hao Nguyen wrote: > I think it would be more intuitive to have vertex attribute index to > be an interface like WebGLAttributeLocation (similar to > WebGLUniformLocation) rather than using an GLint. After all, we are > not modifying the value of that integer anyway. Then, we can get rid > of bindAttribLocation(), and have the context implementation binds a > unique index to an attribute variable when getAttribLocation() is > called for the first time. The reason we didn't do this is because uniform location and attrib indexes are fundamentally different. Uniforms live in the Program object and are accessed using abstract tokens generated by the Program itself. Attributes live in the OpenGL state and really are accessed using numeric indexes. So, for instance, you can ask for index+1 and get the next attribute location in the state. But you can't ask for location+1 and have it be meaningful. In fact, we created WebGLUniformLocation to avoid authors creating a bad access by using location ids not generated by the program object. ----- ~Chris cmarrin...@ ----------------------------------------------------------- You are currently subscribe to public_webgl...@ To unsubscribe, send an email to majordomo...@ with the following command in the body of your email: From cma...@ Mon Jan 4 10:13:07 2010 From: cma...@ (Chris Marrin) Date: Mon, 04 Jan 2010 10:13:07 -0800 Subject: [Public WebGL] Shaders, glReadPixels and speed In-Reply-To: References: <71E7F509-6F11-43C8-AEB9-672B59944A26@adam.com.au> <921bb11001031054v3765a3e2x5b7ef9a372b6ae51@mail.gmail.com> Message-ID: <36B8EB6F-294C-44F1-90FA-685DB938ACCB@apple.com> On Jan 3, 2010, at 9:00 PM, stephen white wrote: > On 04/01/2010, at 5:24 AM, Kenneth Russell wrote: >> What browser and operating system are you using? Safari on Mac OS X > > > Mac OS X 10.5, which probably lacks the hardware composition in the browser (despite being nightly). It is good to know that all browsers are moving towards hardware compositing to address this bottleneck. The WebKit nightly does do hardware compositing. The problem is that you are reading back the pixels, which is slow in all GL implementations. Are you doing this because you need to use the image as a source for some other operation (like 2D rendering)? If there were some way you could avoid that read, your content would run much faster. > >> needed but the raw speed of sending down floating point numbers via >> WebGLFloatArray is already pretty good. > > I was referring more to the step by step nature of needing to feed variables through, which leads to this kind of analysis: > > Gregg Tavares writes: >>> 1 call to for each matrix you want to pass to the shader (usually 1 to 4 matrices) >>> 1 call for each color parameter ( for phong the minimum would be 2, color >>> and shininess though most phong shaders have 5, emissive, ambient, diffuse, >>> specular, shininess) >>> 1 call to setup position vertices >>> 1 call to setup normals >>> If it's textured you'll need another call to supply UVs >>> 1 call to finally draw the object >>> and then possibly a few calls to restore GL state. >>> >>> That a minimum of 5 calls and in this case a maximum of 13 per object, per >>> frame. JavaScript is going to have a tough time doing that for more than a >>> few objects and keep 60 or even 30hz. I don't agree with this analysis. GLES is a very efficient API. 13 calls per primitive is an extremely small number. I have seen demos with at least a couple thousand calls per frame making over 50fps in WebKit. We should do everything possible to reduce that, so things like native matrix calls and geometry shaders are good suggestions for the future. Another big win I think is the use of uniform buffers, which is a proposed GL extension. This allows a set of uniforms to be sent down in one call. It would be pretty easy to emulate in software and still get the benefits of reducing the JS overhead. > > Rather than having a custom engine like O3D, it seems less controversial to add in a standard GL_ARB_geometry_shader4 so that WebGL is capable of displaying and animating 3d graphics to an equivalent level. > > The shaders have the vector and matrix functionality that is a painful library in Javascript, so is there a way of pushing the code across to that domain? Use a fragment shader and render vertex data to a texture? We've discussed both a native matrix library and geometry shaders as future additions to WebGL. The first needs test cases to prove that matrix math and transfer is a bottleneck. The latter needs to wait for OpenGL ES to add them. ----- ~Chris cmarrin...@ ----------------------------------------------------------- You are currently subscribe to public_webgl...@ To unsubscribe, send an email to majordomo...@ with the following command in the body of your email: From cal...@ Mon Jan 4 21:21:11 2010 From: cal...@ (Mark Callow) Date: Tue, 05 Jan 2010 14:21:11 +0900 Subject: [Public WebGL] Some WebGL draft feedback In-Reply-To: References: <4B2EFEAB.4000402@hicorp.co.jp> <232D3496-5733-48BA-98AA-A1EF5F0DA3AE@apple.com> <4B2FF8F2.7080506@mozilla.com> <7E7BE0B9-573C-467B-A85A-5692826E9A2F@apple.com> <35523F5C-763A-4F95-9539-66CA9B91B472@apple.com> Message-ID: <4B42CC47.4070101@hicorp.co.jp> Gregg Tavares wrote: > > ... [snip] ... > I don't follow the logic here? Which OS doesn't allow someone to use > any API they want to draw to a window? Which system doesn't allow you > to use any code you want to draw to a rectangle of pixels? Maybe I'm > reading something into it but it seems like allowing multiple contexts > as the normal thing to do and not allowing multiple contexts is only > being advocated because it's hard for WebGL which doesn't seem like it > should be the deciding factor. Rather then limit the usefulness of > canvas by making it not allowed to have multiple contexts because of > WebGL, if WebGL can't share then it doesn't belong in canvas. > ... [snip] ... Sure you can use any API you want to draw to a window. If you start using multiple APIs for simultaneous drawing to the same window, which is what you are requesting here, it is entirely conceivable that the OS may not support it. When supported, at a minimum the application has to take responsibility for finding a suitable pixel format and synchronizing the drawing of those different APIs. Regards -Mark -------------- next part -------------- A non-text attachment was scrubbed... Name: callow_mark.vcf Type: text/x-vcard Size: 398 bytes Desc: not available URL: From cal...@ Mon Jan 4 21:32:25 2010 From: cal...@ (Mark Callow) Date: Tue, 05 Jan 2010 14:32:25 +0900 Subject: [Public WebGL] Some WebGL draft feedback In-Reply-To: <7D4BC4A3-156F-46D3-AC42-E3C9F8AD96DD@apple.com> References: <4B2EFEAB.4000402@hicorp.co.jp> <232D3496-5733-48BA-98AA-A1EF5F0DA3AE@apple.com> <4B2FF8F2.7080506@mozilla.com> <7E7BE0B9-573C-467B-A85A-5692826E9A2F@apple.com> <35523F5C-763A-4F95-9539-66CA9B91B472@apple.com> <90B8236C088D42F2A8483A1AA3CB7235@hp8510w> <7D4BC4A3-156F-46D3-AC42-E3C9F8AD96DD@apple.com> Message-ID: <4B42CEE9.1080102@hicorp.co.jp> Hi Chris, All, Perhaps we should propose and register some mime-types. Can you do a 3 level mime-type as in text/shader/{vertex,fragment} or text/glsles/{vertex,fragment}? Is it useful at the Javascript level to be able to identify vertex vs. fragment shaders by mime-type or would text/glsles be sufficient? Regards -Mark Chris Marrin wrote: > > On Dec 23, 2009, at 12:34 PM, Carl van Heezik wrote: >> Chris, >> >> I have some questions about the