[Public WebGL] the commit API on the OffscreenCanvas

Gregg Tavares [email protected]
Wed Jul 4 23:33:12 PDT 2018


I'm not sure where to bring this up but I've been trying for a couple of
weeks in other places and getting zero feedback sooo I am hoping you guys
in charge of things will take a few minutes and read this and take some
time to thoughtfully respond.

It's possible I don't understand how OffscreenCanvas is supposed to work.
I've read the spec and written several tests and a couple of short examples
and this is my understanding.

There are basically 2 ways to use it. They are documented at MDN.

https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas

One is listed as  "*Synchronous display of frames produced by an
OffscreenCanvas*". It involves using
"offscreenCanvas.transferToImageBitmap" inside the worker, transferring
that bitmap back to the main thread, and calling
bitmapContext.transferImageBitmap. This API makes sense to me. If you want
to synchronize DOM updates with WebGL updates then you need to make sure
both get updated at the same time. Like say you have an HTML label over a
moving 3D object.

The other is listed as "*Asynchronous display of frames produced by an
OffscreenCanvas*". In that case you just call `*gl.commit*` inside the
worker and the canvas back on the page will be updated. This is arguably
the more common use case. The majority of WebGL and three.js apps etc would
use this method. The example on MDN shows sending a message to the worker
each time you want it to render. Testing that on Chrome seems to work but
it currently has a significant performance penalty.

Recently 2 more things were added. One is that *requestAnimationFrame* was
added to workers. The other is the *commit as been changed to be a
synchronous* function. The worker freezes until the frame has been
displayed.

It's these last 2 things I don't understand.

*First:* given that rAF is now available in workers I would think this is
valid code

      // in worker
      function loop() {
          render();
          requestAnimationFrame(loop);
          gl.commit();
      }
      loop();

      onmessage = function() {
          // get messages related to say camera position or
          // window size or mouse position etc to affect rendering
      };

Unfortunately testing it out in Chrome this doesn't work. The `onmessage`
callback is never called regardless of how many messages are sent. I filed
a bug. Was told "WONTFIX: working as intended"

Really? Is that really the intent of the spec? Apple? Mozilla? Microsoft?
Do you agree that the code above is not a supported use case and is working
as intended?

*Second:* other events and callbacks don't work

      // in worker
      fetch('someimage.png', {mode:'cors'}).then(function(response) {
         return response.blob();
      }).then(function(blob) {
         return createImageBitmap(response.blob());
      }).then(function(bitmap) {
         gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA,
gl.UNSIGNED_BYTE, bitmap);
      });

      function loop() {
          render();
          requestAnimationFrame(loop);
          gl.commit();
      }
      loop();

This also does not work. The *fetch* response never comes. My guess is this
is because in Chrome *commit* blocks and rAF event gets pushed to the top
of the event queue so no other events ever get processed. The spec has
nothing to say about this. Is this supposed to work? It seems like a valid
use case. Note that switching the end of loop to

          gl.commit();
          requestAnimationFrame(loop);

also does not work.

Is that correct that it should not work? I guess I don't really understand
the point of having rAF in worker if these use cases are not supposed to
work. Are they? If they are not supposed to work can someone please explain
rAF's use case in a worker?

*Third*, according to various comments around the specs one use case is a
spin loop on gl.commit for webassembly ports. Effectively this is supposed
to work

      while(true) {
         render();
         gl.commit();
      }

But I don't understand how this is useful given that no events come in if
you do that. You can't communicate with the worker. The worker can't load
files or call fetch or get a websocket message or receive input passed in
from the main thread or do anything except render.

Maybe people are thinking SharedArrayBuffers are a way to pass in data to
such a loop but really? How would you pass in an image? As it is you'd have
write your own decoder since you can't get the raw data losslessly out of
an image from any web APIs and you can't transfer images into the worker
(since it's not listening for messages) then you'd need to some how parse
the image yourself and copy it into a sharedarraybuffer. That would a very
slow jank inducing process in the main thread so now it seems like the spec
is saying to use a gl.commit spin loop you need 2 workers, one for
rendering, one for loading images and other things and then you need 1 or
more SharedArrayBuffers and you have to implement a bunch of
synchronization stuff just so you can use WebGL in a worker using this
pattern mentioned in the spec?

Is that really the intent? Is there something I'm missing? This seems like
a platform breaking API. Use it and the entire rest of the platform becomes
unusable without major amounts of code.

If I'm wrong I'm happy to be corrected.

*Four:* Non front tabs: rAF is currently not delivered if the page is not
the front tab which is great but rAF is an event so even when rAF stops
firing because the page is not on the front tab other events still arrive
(fetch, onmessage, XHR, websockets, etc...) This means even though your
page doesn't get a rAF callback it can still process incoming data (like
your chat app's messages).

How is that supposed to work with `gl.commit` loops? It's not the front tab
so you want to block the commit so the worker doesn't spin and waste time.
If the worker locks then that seems to have implications for all other
associated workers and the main thread. If you're using Atomics to sync up
things suddenly they'll fail indefinitely even more complicating all the
code you have to write to use this feature.



Chrome has already committed to shipping the API. The code as been
committed so if nothing changes it will ship automatically in a few weeks
with all the issues mentioned above not behind a flag but live so it seems
important to understand how to use this and if all these issues were
considered and what their solutions are.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://khronos.org/pipermail/public_webgl_khronos.org/attachments/20180705/e298ecae/attachment.html>


More information about the public_webgl mailing list