[Public WebGL] y-orientation for texImage2D from HTML elements

Ben Adams [email protected]
Fri Dec 7 05:21:20 PST 2012


Hi Ken

A plethora of WebGL libraries, applications, and conformance tests
> have been developed without this issue having come up before. This
> indicates to me that there is no widespread confusion among WebGL
> developers.


Not necessarily arguing for changes; but for examples of confusion:

Render-To-Texture Example Renders Upside-Down

https://github.com/mrdoob/three.js/issues/2164

Sprites are positioned upside down

https://github.com/mrdoob/three.js/issues/277

"I flipped the tilemap (tilemap.flipY = true), unflipped the tileset
(tileset.flipY
> = false) and then modified the mapmaker.html (that Toji wrote that will
> create these tilemaps) to draw each tile *upside down on the sprite sheet
> (tileset)*.


> I would MUCH prefer a different answer that actually fixes the problem
> instead of working around it like this, but for now this is my solution."

http://stackoverflow.com/questions/13013658/three-js-custom-shader/13087410#13087410


Though perhaps most worryingly (though may be driver bugs?):

> "But I ran into this problem with y-flipped textures, recently.



Seems to be an issue with Ati+OpenGL+Certain drivers as far as I can tell.
> I can´t recreate on my ati-windows-machine using the opengl-switch for
> example, but people with osx and ati reported it frequently.



Trying to think of a way to detect and work around this. I was thinking,
> what if I create like a small canvas and render one sprite/particle that is
> red on the top half and green on the bottom half. And then I guess draw
> that to a 2dcanvas and pick the color at the top to see if it´s flipped or
> not. That way I would know if I should flip or not from there on.



Would something like that work? Seems like a kinda hacky way :) Anyone got
> a better idea?"

 https://github.com/mrdoob/three.js/issues/277#issuecomment-3989529

Cheers
Ben

On Fri, Dec 7, 2012 at 3:02 AM, Kenneth Russell <[email protected]> wrote:

>
> A plethora of WebGL libraries, applications, and conformance tests
> have been developed without this issue having come up before. This
> indicates to me that there is no widespread confusion among WebGL
> developers.
>
> I don't see any motivation to make changes in this area, and strongly
> disagree with the suggestion that a future version of WebGL intended
> to be backward compatible change the current behavior. If you have a
> concrete use case in mind, please describe it.
>
> -Ken
>
>
> On Thu, Dec 6, 2012 at 6:49 PM, Jeff Gilbert <[email protected]> wrote:
> > There is clearly developer confusion in this regard, since many think
> that texture coordinates start in the top-left. They don't. The GLES2 spec
> is clear about this, even. But people think they do because when they
> upload an image, it's upside down if you put 0,0 in the bottom-left. Then
> they try copyTexImage or readPixels, and everything appears upside-down.
> How is this behavior not confusing?
> >
> > WebGL's UNPACK_FLIP_Y_WEBGL behavior appears consistent in theory, but
> falls down in practice. For an app that uses 0,0=bottom-left coords, the
> behavior of texImage(HTML element) is never consistent with
> texImage(TypedArray). Since internally, GL textures are 0,0=bottom-left,
> this makes this behavior confusing.
> >
> > It's already come up in this thread how "copyTexImage is backwards",
> even though it isn't, so there's very clearly some developer confusion in
> this area. Don't misunderstand people getting something to work eventually
> with nothing being wrong. If people made websites for IE6, I'm pretty sure
> they'll eventually figure out how to make due with anything.
> >
> > OpenGL internally uses 0,0=bottom-left, so why is it so hard to use
> WebGL this way. Why do texImage2D calls need to be interleaved with
> twiddling of UNPACK_FLIP_Y_WEBGL to get it to work.
> >
> > While there is a very good reason to want to flip HTML element uploads,
> there is much less of a reason to want to flip TypedArray uploads.
> Regardless of whether your app is 0,0 at the top-left or bottom-left,
> upload the data in top-to-bottom or bottom-to-top respectively, set 0,0 to
> top-left or bottom-left respectively, and your data will be right-side up.
> (Both with UNPACK_FLIP_Y_WEBGL=false) However, for HTML elements, top-left
> apps need UNPACK_FLIP_Y_WEBGL=false, and bottom-left apps need
> UNPACK_FLIP_Y_WEBGL=true.
> > The only usecase for TypedArray uploads with UNPACK_FLIP_Y_WEBGL=true is
> if the data you have is backwards. (That is, UNPACK_FLIP_Y_WEBGL=true is
> only useful for uploading bottom-to-top data in a 0,0=top-left app, and
> vice-versa) Why your data is already wrong inside your own app is anyone's
> guess.
> >
> > A form of UNPACK_FLIP_Y_WEBGL that only affected HTML element uploads is
> needed to provide a level of (texImage) consistency for 0,0=bottom-left
> apps that 0,0=top-left apps already enjoy. The choice to have
> UNPACK_FLIP_Y_WEBGL modify both texImage2D behaviors, while seemingly more
> consistent, actually makes things more convoluted than just having it
> affect HTML element uploads.
> >
> > ----- Original Message -----
> > From: "Kenneth Russell" <[email protected]>
> > To: "Gregg Tavares (社用)" <[email protected]>
> > Cc: "Jeff Gilbert" <[email protected]>, "public webgl" <
> [email protected]>
> > Sent: Thursday, December 6, 2012 6:06:04 PM
> > Subject: Re: [Public WebGL] y-orientation for texImage2D from HTML
> elements
> >
> > On Thu, Dec 6, 2012 at 5:44 PM, Gregg Tavares (社用) <[email protected]>
> wrote:
> >>
> >>
> >>
> >> On Fri, Dec 7, 2012 at 10:27 AM, Jeff Gilbert <[email protected]>
> wrote:
> >>>
> >>> I am not saying people should not have the option (or even the
> default) of
> >>> having texImage2D of an HTML element be y-flipped to match their
> upside-down
> >>> tex coords, but it should not make it harder for projects that use 0,0
> as
> >>> the bottom-left, as GL does natively. It should at least be possible
> to get
> >>> consistent behavior such that an app can upload TypedArrays bottom-row
> first
> >>> while *also* having texImage on HTML elements get uploaded
> bottom-row-first.
> >>> This is not currently possible.
> >>>
> >>> It is also a little crazy that texImage2D(gl.canvas) does not reduce to
> >>> the same operation as copyTexImage2D.
> >>
> >>
> >> Why is this crazy? Canvas started as a 2D software implementation with
> >> top-row first. It's still top row first.
> >>
> >>>
> >>>
> >>> I understand that people are used to passing image data directly to
> >>> texImage2D as top-row-first, and just dealing with the flipped u/v
> >>> coords[1], but this choice should not be force on everyone as it is
> now.
> >>> UNPACK_FLIP_Y_WEBGL as it is today does not solve this, as to get
> consistent
> >>> bottom-row-first behavior, an app needs to use
> UNPACK_FLIP_Y_WEBGL=false for
> >>> TypedArray uploads, and UNPACK_FLIP_Y_WEBGL=true for HTML element
> uploads
> >>
> >>
> >> I don't see that as a problem. Most game devs flip their images in their
> >> asset pipeline since there's no UNPACK_FLIP_Y on real GL. In that case
> their
> >> ported to WebGL apps just work. They load their flipped at build time
> >> images, set no flags and get the same results they got on GL.
> >>
> >>
> >>>
> >>> .
> >>>
> >>> WebGL is *not* working the same way as OpenGL here, because OpenGL
> *only*
> >>> accepts raw bytes. If you upload data top-row-first, and use uv 0,0 at
> the
> >>> top-left, it's right-side-up. If you upload data bottom-row-first
> (like the
> >>> texImage2D spec says), and use uv 0,0 at the bottom-left, it's also
> >>> right-side-up.
> >>
> >>
> >> WebGL is providing 2 things. 1) GL. 2) image loaders. Both are behaving
> as
> >> most do. GL is doing bottom row first. The Image loaders are doing
> top-row
> >> first.
> >>
> >>>
> >>>
> >>> WebGL introduces loading from images directly. This means WebGL has to
> >>> make a choice over what order to upload the rows. Whereas texImage2D on
> >>> TypedArrays doesn't really matter, texImage2D on HTML elements can't
> >>> automatically handle both. We have half the solution in the form of
> >>> UNPACK_FLIP_Y_WEBGL, but it doesn't provide a solution to making WebGL
> >>> consistent for both choices of origin location.
> >>>
> >>> [1] This also tends to create the appearance that CopyTexImage and/or
> >>> ReadPixels are backwards, even though they're not.
> >>
> >>
> >> As has always been the case. I know of no OS who's windowing system is
> >> bottom first. Every OS provides ways to draw to a window and read from a
> >> window. AFAIK all of those are top-row first. GL is bottom-row first so
> this
> >> problem exists everywhere not just WebGL in browser. Browsers, like
> OSes are
> >> top-row first.
> >
> > For historical reference, *all* image loading libraries for OpenGL
> > which have ever existed have an option to vertically flip the incoming
> > image, precisely because the first byte of the image in all modern
> > image formats is at the upper-left corner.
> >
> > HTML's coordinate systems assume that (0, 0) is the upper left corner
> > of images; e.g. CanvasRenderingContext2D.drawImage. WebGL's
> > UNPACK_FLIP_Y parameter is both convenient and consistent as it
> > applies not only to the upload paths taking HTML elements but also
> > applies to those taking Typed Arrays. (It didn't always, in early
> > WebGL implementations.)
> >
> > I don't believe there is any developer confusion in this area and do
> > not think any new tunable parameters should be added in this area.
> >
> > -Ken
> >
> >
> >>>
> >>>
> >>> ----- Original Message -----
> >>> From: "Gregg Tavares (社用)" <[email protected]>
> >>> To: "Jeff Gilbert" <[email protected]>
> >>> Sent: Thursday, December 6, 2012 5:01:18 PM
> >>> Subject: Re: [Public WebGL] y-orientation for texImage2D from HTML
> >>> elements
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Fri, Dec 7, 2012 at 9:51 AM, Jeff Gilbert < [email protected] >
> >>> wrote:
> >>>
> >>>
> >>> It most certainly is a bug that it is not specified.
> >>> My point is we never specify that data is uploaded to texImage2D from
> HTML
> >>> elements top-row-first. Just because we're 'used to' talking about
> image
> >>> data top-row-first doesn't mean we shouldn't process this data into
> GL's
> >>> coord system. The reasonable default, if we're doing the legwork of
> >>> uploading HTML element contents, is to upload them (at least by
> default) in
> >>> the orientation GL expects: bottom-row-first.
> >>>
> >>>
> >>> No that is not a reasonable default. A reasonable default is to upload
> the
> >>> same as GL programmers expect. GL programmers expect that image
> libraries
> >>> load top row first and GL expects bottom row first and adjust their
> code
> >>> accordingly. Many people are porting GL code. WebGL should not be
> different
> >>> here.
> >>>
> >>>
> >>> If you want to specify everything top-row-first, we should use our
> UNPACK
> >>> boolean for that.
> >>>
> >>> As it stands, it works consistently only if you pretend that for some
> >>> reason, OpenGL textures coords are 0,0=top-left. That's the way we
> upload
> >>> HTML elements, and if you pass texImage TypedArray data in
> top-row-first
> >>> order, it'll upload upside-down, but since you y-flip the texture
> coords
> >>> (that is, use 0,0=top-left), now it's correctly oriented.
> >>>
> >>> One of the key issues is there's no way to configure UNPACK such that
> >>> texImage2D's upload orientation always matches GL's coord system.
> >>>
> >>> CopyTexImage isn't backwards, it matches GL's coordinate system.
> >>> CopyTexImage, ReadPixels, and TexImage all specify that they are
> >>> bottom-row-first. Scissor and Viewport, being in window space, are also
> >>> 0,0=bottom-left. The idea that textures are 0,0=top-left goes against
> >>> OpenGL's 0,0=bottom-left coord system. (actually -1,-1=bottom-left,
> for clip
> >>> space) CopyTexImage matches the behavior of calling texImage2D on data
> >>> retrieved from readPixels. It would *ideally* match calling texImage2D
> on
> >>> our canvas, but it doesn't. Instead, we readPixels data down, y-flip
> it,
> >>> then reupload it. Ideally we would just call CopyTexImage instead of
> >>> ReadPixels+TexImage, but it doesn't looks like we even can. Instead, we
> >>> would have to CopyTexImage, create a new texture, wrap it in a
> framebuffer,
> >>> and draw a full-screen quad with the original copyTexImage texture,
> but with
> >>> y coords flipped.
> >>>
> >>> TexImage2D with both TypedArrays and canvas/image/video elements is
> only
> >>> consistent for u/v 0,0=top-left. But then it's no longer consistent
> with
> >>> CopyTexImage or ReadPixels. Now we're in y-flip hell.
> >>>
> >>> Proposal:
> >>> Since basically everyone uses texImage2D(<image>) for content
> delivery, I
> >>> don't think we can afford to break compatibility there. What I propose
> is
> >>> adding a new UNPACK_HTML_FLIP_Y_WEBGL, which defaults to `true`, which
> >>> specifies that WebGL will upload HTML elements top-row-first into
> >>> texImage2D, as we do today. However, with `false`, we would upload
> elements
> >>> bottom-row-first, such that texImage2D(gl.canvas) has the same result
> as
> >>> copyTexImage2D.
> >>>
> >>>
> >>>
> >>> Disagree. No change needed. WebGL works the same as OpenGL.
> >>> UNPACK_FLIP_Y_WEBGL is there as a convenience only because unlike GL we
> >>> don't expect web developers to supply their own image loaders (though
> some
> >>> do) so we've provided the option to tell to the browser to flip the
> image
> >>> before providing it to the driver.
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> -Jeff
> >>>
> >>>
> >>>
> >>> ----- Original Message -----
> >>> From: "Gregg Tavares (社用)" < [email protected] >
> >>> To: "Jeff Gilbert" < [email protected] >
> >>> Cc: "public webgl" < [email protected] >
> >>> Sent: Thursday, December 6, 2012 12:35:14 AM
> >>> Subject: Re: [Public WebGL] y-orientation for texImage2D from HTML
> >>> elements
> >>>
> >>>
> >>>
> >>> This isn't a bug. it's by design.
> >>>
> >>>
> >>> By default WebGL does the same as GL. It doesn't flip anything. The
> "data"
> >>> pointer passed to texImage2D represents the bottom left corner of the
> >>> texture. Whether that's an ArrayBuffer, an Image, a Canvas, a Video.
> For
> >>> Image, Canvas, and Video that means by default they'be upside down
> (which is
> >>> the same as GL).
> >>>
> >>>
> >>> CopyTexImage2D in GL has always been "effectively" backward relative to
> >>> TexImage2D when talking about images because most (all?) image
> libraries
> >>> load images with the first pixel in the top, left and GL expects the
> first
> >>> pixel is the bottom, left. Since CopyTexImage is copying internally it
> >>> appears to have different behavior because it's matching src data 0x0 =
> >>> bottom, left, dst data 0x0 = bottom left where as TexImage2D, when
> passing
> >>> standard image data, is src data 0x0 = top,left, dest = bottom, left
> >>>
> >>>
> >>> If you want WebGL to flip the data for you call
> >>> gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, true);
> >>>
> >>>
> >>> There is no PACK_FLIP_Y so readPixels will always return the
> bottom,left
> >>> as the first data in the array buffer.
> >>>
> >>>
> >>> As for matching implementations it's all tested in the conformance
> tests
> >>> AFAIK.
> >>>
> >>>
> >>>
> >>> On Thu, Dec 6, 2012 at 4:59 PM, Jeff Gilbert < [email protected] >
> >>> wrote:
> >>>
> >>>
> >>>
> >>> It seems backwards (as implemented) and underdefined in the spec. It
> looks
> >>> like we all upload HTML element texImage data with 0,0=top-left,
> making it
> >>> effectively upside-down.
> >>>
> >>> The following two lines yield different results:
> >>> [1] gl.copyTexImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 0, 0,
> >>> gl.drawingBufferWidth, gl.drawingBufferHeight, 0);
> >>> [2] gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE,
> >>> gl.canvas);
> >>>
> >>> [1] matches this: [3]
> >>> var data = new Uint8Array(gl.drawingBufferWidth *
> gl.drawingBufferHeight *
> >>> 4);
> >>> gl.readPixels(0, 0, gl.drawingBufferWidth, gl.drawingBufferHeight,
> >>> gl.RGBA, gl.UNSIGNED_BYTE, data);
> >>> gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.drawingBufferWidth,
> >>> gl.drawingBufferHeight, 0, gl.RGBA, gl.UNSIGNED_BYTE, data);
> >>>
> >>> UNPACK_FLIP_Y_WEBGL=true makes [2] the same as [1], but now [3] is
> >>> upside-down.
> >>>
> >>> This is Mozilla bug 818810:
> >>> https://bugzilla.mozilla.org/show_bug.cgi?id=818810
> >>>
> >>> Testcase showing inconsistent behavior:
> >>> https://bugzilla.mozilla.org/attachment.cgi?id=689110
> >>>
> >>> As I mention in the bug, it doesn't look like y-orientation of
> texImage2D
> >>> of HTML elements is well-specified in the spec. So far everyone
> appears to
> >>> have implemented it the same, though.
> >>>
> >>> My suggestion was adding an UNPACK_ pixelStore option that allows us to
> >>> get OpenGL-style 0,0=bottom-left texImage2D uploads of HTML elements.
> >>>
> >>> -----------------------------------------------------------
> >>> You are currently subscribed to [email protected] .
> >>> To unsubscribe, send an email to [email protected] with
> >>> the following command in the body of your email:
> >>> unsubscribe public_webgl
> >>> -----------------------------------------------------------
> >>>
> >>>
> >>>
> >>
>
> -----------------------------------------------------------
> You are currently subscribed to [email protected]
> To unsubscribe, send an email to [email protected] with
> the following command in the body of your email:
> unsubscribe public_webgl
> -----------------------------------------------------------
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://khronos.org/pipermail/public_webgl_khronos.org/attachments/20121207/88e9c4f1/attachment.html>


More information about the public_webgl mailing list