[Public WebGL] y-orientation for texImage2D from HTML elements

Gregg Tavares (社用) [email protected]
Sat Dec 8 02:29:08 PST 2012


On Fri, Dec 7, 2012 at 10:21 PM, Ben Adams <[email protected]>wrote:

> Hi Ken
>
> A plethora of WebGL libraries, applications, and conformance tests
>> have been developed without this issue having come up before. This
>> indicates to me that there is no widespread confusion among WebGL
>> developers.
>
>
> Not necessarily arguing for changes; but for examples of confusion:
>
> Render-To-Texture Example Renders Upside-Down
>
> https://github.com/mrdoob/three.js/issues/2164
>
> Sprites are positioned upside down
>
> https://github.com/mrdoob/three.js/issues/277
>


This isn't unique to WebGL

https://www.google.com/search?q=opengl+textures+upside+down&oq=opengl+textures+upside+down&sugexp=chrome,mod=1&sourceid=chrome&ie=UTF-8

It seems nice that once you learned the rules for OpenGL the same rules
apply for WebGL and visa versa


>
> "I flipped the tilemap (tilemap.flipY = true), unflipped the tileset (tileset.flipY
>> = false) and then modified the mapmaker.html (that Toji wrote that will
>> create these tilemaps) to draw each tile *upside down on the sprite
>> sheet (tileset)*.
>
>
>> I would MUCH prefer a different answer that actually fixes the problem
>> instead of working around it like this, but for now this is my solution.
>> "
>
>
> http://stackoverflow.com/questions/13013658/three-js-custom-shader/13087410#13087410
>
>
> Though perhaps most worryingly (though may be driver bugs?):
>
>> "But I ran into this problem with y-flipped textures, recently.
>
>
>
> Seems to be an issue with Ati+OpenGL+Certain drivers as far as I can tell.
>> I can´t recreate on my ati-windows-machine using the opengl-switch for
>> example, but people with osx and ati reported it frequently.
>
>
>
> Trying to think of a way to detect and work around this. I was thinking,
>> what if I create like a small canvas and render one sprite/particle that is
>> red on the top half and green on the bottom half. And then I guess draw
>> that to a 2dcanvas and pick the color at the top to see if it´s flipped or
>> not. That way I would know if I should flip or not from there on.
>
>
>
> Would something like that work? Seems like a kinda hacky way :) Anyone got
>> a better idea?"
>
>  https://github.com/mrdoob/three.js/issues/277#issuecomment-3989529
>
> Cheers
> Ben
>
>
> On Fri, Dec 7, 2012 at 3:02 AM, Kenneth Russell <[email protected]> wrote:
>
>>
>> A plethora of WebGL libraries, applications, and conformance tests
>> have been developed without this issue having come up before. This
>> indicates to me that there is no widespread confusion among WebGL
>> developers.
>>
>> I don't see any motivation to make changes in this area, and strongly
>> disagree with the suggestion that a future version of WebGL intended
>> to be backward compatible change the current behavior. If you have a
>> concrete use case in mind, please describe it.
>>
>> -Ken
>>
>>
>> On Thu, Dec 6, 2012 at 6:49 PM, Jeff Gilbert <[email protected]>
>> wrote:
>> > There is clearly developer confusion in this regard, since many think
>> that texture coordinates start in the top-left. They don't. The GLES2 spec
>> is clear about this, even. But people think they do because when they
>> upload an image, it's upside down if you put 0,0 in the bottom-left. Then
>> they try copyTexImage or readPixels, and everything appears upside-down.
>> How is this behavior not confusing?
>> >
>> > WebGL's UNPACK_FLIP_Y_WEBGL behavior appears consistent in theory, but
>> falls down in practice. For an app that uses 0,0=bottom-left coords, the
>> behavior of texImage(HTML element) is never consistent with
>> texImage(TypedArray). Since internally, GL textures are 0,0=bottom-left,
>> this makes this behavior confusing.
>> >
>> > It's already come up in this thread how "copyTexImage is backwards",
>> even though it isn't, so there's very clearly some developer confusion in
>> this area. Don't misunderstand people getting something to work eventually
>> with nothing being wrong. If people made websites for IE6, I'm pretty sure
>> they'll eventually figure out how to make due with anything.
>> >
>> > OpenGL internally uses 0,0=bottom-left, so why is it so hard to use
>> WebGL this way. Why do texImage2D calls need to be interleaved with
>> twiddling of UNPACK_FLIP_Y_WEBGL to get it to work.
>> >
>> > While there is a very good reason to want to flip HTML element uploads,
>> there is much less of a reason to want to flip TypedArray uploads.
>> Regardless of whether your app is 0,0 at the top-left or bottom-left,
>> upload the data in top-to-bottom or bottom-to-top respectively, set 0,0 to
>> top-left or bottom-left respectively, and your data will be right-side up.
>> (Both with UNPACK_FLIP_Y_WEBGL=false) However, for HTML elements, top-left
>> apps need UNPACK_FLIP_Y_WEBGL=false, and bottom-left apps need
>> UNPACK_FLIP_Y_WEBGL=true.
>> > The only usecase for TypedArray uploads with UNPACK_FLIP_Y_WEBGL=true
>> is if the data you have is backwards. (That is, UNPACK_FLIP_Y_WEBGL=true is
>> only useful for uploading bottom-to-top data in a 0,0=top-left app, and
>> vice-versa) Why your data is already wrong inside your own app is anyone's
>> guess.
>> >
>> > A form of UNPACK_FLIP_Y_WEBGL that only affected HTML element uploads
>> is needed to provide a level of (texImage) consistency for 0,0=bottom-left
>> apps that 0,0=top-left apps already enjoy. The choice to have
>> UNPACK_FLIP_Y_WEBGL modify both texImage2D behaviors, while seemingly more
>> consistent, actually makes things more convoluted than just having it
>> affect HTML element uploads.
>> >
>> > ----- Original Message -----
>> > From: "Kenneth Russell" <[email protected]>
>> > To: "Gregg Tavares (社用)" <[email protected]>
>> > Cc: "Jeff Gilbert" <[email protected]>, "public webgl" <
>> [email protected]>
>> > Sent: Thursday, December 6, 2012 6:06:04 PM
>> > Subject: Re: [Public WebGL] y-orientation for texImage2D from HTML
>> elements
>> >
>> > On Thu, Dec 6, 2012 at 5:44 PM, Gregg Tavares (社用) <[email protected]>
>> wrote:
>> >>
>> >>
>> >>
>> >> On Fri, Dec 7, 2012 at 10:27 AM, Jeff Gilbert <[email protected]>
>> wrote:
>> >>>
>> >>> I am not saying people should not have the option (or even the
>> default) of
>> >>> having texImage2D of an HTML element be y-flipped to match their
>> upside-down
>> >>> tex coords, but it should not make it harder for projects that use
>> 0,0 as
>> >>> the bottom-left, as GL does natively. It should at least be possible
>> to get
>> >>> consistent behavior such that an app can upload TypedArrays
>> bottom-row first
>> >>> while *also* having texImage on HTML elements get uploaded
>> bottom-row-first.
>> >>> This is not currently possible.
>> >>>
>> >>> It is also a little crazy that texImage2D(gl.canvas) does not reduce
>> to
>> >>> the same operation as copyTexImage2D.
>> >>
>> >>
>> >> Why is this crazy? Canvas started as a 2D software implementation with
>> >> top-row first. It's still top row first.
>> >>
>> >>>
>> >>>
>> >>> I understand that people are used to passing image data directly to
>> >>> texImage2D as top-row-first, and just dealing with the flipped u/v
>> >>> coords[1], but this choice should not be force on everyone as it is
>> now.
>> >>> UNPACK_FLIP_Y_WEBGL as it is today does not solve this, as to get
>> consistent
>> >>> bottom-row-first behavior, an app needs to use
>> UNPACK_FLIP_Y_WEBGL=false for
>> >>> TypedArray uploads, and UNPACK_FLIP_Y_WEBGL=true for HTML element
>> uploads
>> >>
>> >>
>> >> I don't see that as a problem. Most game devs flip their images in
>> their
>> >> asset pipeline since there's no UNPACK_FLIP_Y on real GL. In that case
>> their
>> >> ported to WebGL apps just work. They load their flipped at build time
>> >> images, set no flags and get the same results they got on GL.
>> >>
>> >>
>> >>>
>> >>> .
>> >>>
>> >>> WebGL is *not* working the same way as OpenGL here, because OpenGL
>> *only*
>> >>> accepts raw bytes. If you upload data top-row-first, and use uv 0,0
>> at the
>> >>> top-left, it's right-side-up. If you upload data bottom-row-first
>> (like the
>> >>> texImage2D spec says), and use uv 0,0 at the bottom-left, it's also
>> >>> right-side-up.
>> >>
>> >>
>> >> WebGL is providing 2 things. 1) GL. 2) image loaders. Both are
>> behaving as
>> >> most do. GL is doing bottom row first. The Image loaders are doing
>> top-row
>> >> first.
>> >>
>> >>>
>> >>>
>> >>> WebGL introduces loading from images directly. This means WebGL has to
>> >>> make a choice over what order to upload the rows. Whereas texImage2D
>> on
>> >>> TypedArrays doesn't really matter, texImage2D on HTML elements can't
>> >>> automatically handle both. We have half the solution in the form of
>> >>> UNPACK_FLIP_Y_WEBGL, but it doesn't provide a solution to making WebGL
>> >>> consistent for both choices of origin location.
>> >>>
>> >>> [1] This also tends to create the appearance that CopyTexImage and/or
>> >>> ReadPixels are backwards, even though they're not.
>> >>
>> >>
>> >> As has always been the case. I know of no OS who's windowing system is
>> >> bottom first. Every OS provides ways to draw to a window and read from
>> a
>> >> window. AFAIK all of those are top-row first. GL is bottom-row first
>> so this
>> >> problem exists everywhere not just WebGL in browser. Browsers, like
>> OSes are
>> >> top-row first.
>> >
>> > For historical reference, *all* image loading libraries for OpenGL
>> > which have ever existed have an option to vertically flip the incoming
>> > image, precisely because the first byte of the image in all modern
>> > image formats is at the upper-left corner.
>> >
>> > HTML's coordinate systems assume that (0, 0) is the upper left corner
>> > of images; e.g. CanvasRenderingContext2D.drawImage. WebGL's
>> > UNPACK_FLIP_Y parameter is both convenient and consistent as it
>> > applies not only to the upload paths taking HTML elements but also
>> > applies to those taking Typed Arrays. (It didn't always, in early
>> > WebGL implementations.)
>> >
>> > I don't believe there is any developer confusion in this area and do
>> > not think any new tunable parameters should be added in this area.
>> >
>> > -Ken
>> >
>> >
>> >>>
>> >>>
>> >>> ----- Original Message -----
>> >>> From: "Gregg Tavares (社用)" <[email protected]>
>> >>> To: "Jeff Gilbert" <[email protected]>
>> >>> Sent: Thursday, December 6, 2012 5:01:18 PM
>> >>> Subject: Re: [Public WebGL] y-orientation for texImage2D from HTML
>> >>> elements
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> On Fri, Dec 7, 2012 at 9:51 AM, Jeff Gilbert < [email protected] >
>> >>> wrote:
>> >>>
>> >>>
>> >>> It most certainly is a bug that it is not specified.
>> >>> My point is we never specify that data is uploaded to texImage2D from
>> HTML
>> >>> elements top-row-first. Just because we're 'used to' talking about
>> image
>> >>> data top-row-first doesn't mean we shouldn't process this data into
>> GL's
>> >>> coord system. The reasonable default, if we're doing the legwork of
>> >>> uploading HTML element contents, is to upload them (at least by
>> default) in
>> >>> the orientation GL expects: bottom-row-first.
>> >>>
>> >>>
>> >>> No that is not a reasonable default. A reasonable default is to
>> upload the
>> >>> same as GL programmers expect. GL programmers expect that image
>> libraries
>> >>> load top row first and GL expects bottom row first and adjust their
>> code
>> >>> accordingly. Many people are porting GL code. WebGL should not be
>> different
>> >>> here.
>> >>>
>> >>>
>> >>> If you want to specify everything top-row-first, we should use our
>> UNPACK
>> >>> boolean for that.
>> >>>
>> >>> As it stands, it works consistently only if you pretend that for some
>> >>> reason, OpenGL textures coords are 0,0=top-left. That's the way we
>> upload
>> >>> HTML elements, and if you pass texImage TypedArray data in
>> top-row-first
>> >>> order, it'll upload upside-down, but since you y-flip the texture
>> coords
>> >>> (that is, use 0,0=top-left), now it's correctly oriented.
>> >>>
>> >>> One of the key issues is there's no way to configure UNPACK such that
>> >>> texImage2D's upload orientation always matches GL's coord system.
>> >>>
>> >>> CopyTexImage isn't backwards, it matches GL's coordinate system.
>> >>> CopyTexImage, ReadPixels, and TexImage all specify that they are
>> >>> bottom-row-first. Scissor and Viewport, being in window space, are
>> also
>> >>> 0,0=bottom-left. The idea that textures are 0,0=top-left goes against
>> >>> OpenGL's 0,0=bottom-left coord system. (actually -1,-1=bottom-left,
>> for clip
>> >>> space) CopyTexImage matches the behavior of calling texImage2D on data
>> >>> retrieved from readPixels. It would *ideally* match calling
>> texImage2D on
>> >>> our canvas, but it doesn't. Instead, we readPixels data down, y-flip
>> it,
>> >>> then reupload it. Ideally we would just call CopyTexImage instead of
>> >>> ReadPixels+TexImage, but it doesn't looks like we even can. Instead,
>> we
>> >>> would have to CopyTexImage, create a new texture, wrap it in a
>> framebuffer,
>> >>> and draw a full-screen quad with the original copyTexImage texture,
>> but with
>> >>> y coords flipped.
>> >>>
>> >>> TexImage2D with both TypedArrays and canvas/image/video elements is
>> only
>> >>> consistent for u/v 0,0=top-left. But then it's no longer consistent
>> with
>> >>> CopyTexImage or ReadPixels. Now we're in y-flip hell.
>> >>>
>> >>> Proposal:
>> >>> Since basically everyone uses texImage2D(<image>) for content
>> delivery, I
>> >>> don't think we can afford to break compatibility there. What I
>> propose is
>> >>> adding a new UNPACK_HTML_FLIP_Y_WEBGL, which defaults to `true`, which
>> >>> specifies that WebGL will upload HTML elements top-row-first into
>> >>> texImage2D, as we do today. However, with `false`, we would upload
>> elements
>> >>> bottom-row-first, such that texImage2D(gl.canvas) has the same result
>> as
>> >>> copyTexImage2D.
>> >>>
>> >>>
>> >>>
>> >>> Disagree. No change needed. WebGL works the same as OpenGL.
>> >>> UNPACK_FLIP_Y_WEBGL is there as a convenience only because unlike GL
>> we
>> >>> don't expect web developers to supply their own image loaders (though
>> some
>> >>> do) so we've provided the option to tell to the browser to flip the
>> image
>> >>> before providing it to the driver.
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> -Jeff
>> >>>
>> >>>
>> >>>
>> >>> ----- Original Message -----
>> >>> From: "Gregg Tavares (社用)" < [email protected] >
>> >>> To: "Jeff Gilbert" < [email protected] >
>> >>> Cc: "public webgl" < [email protected] >
>> >>> Sent: Thursday, December 6, 2012 12:35:14 AM
>> >>> Subject: Re: [Public WebGL] y-orientation for texImage2D from HTML
>> >>> elements
>> >>>
>> >>>
>> >>>
>> >>> This isn't a bug. it's by design.
>> >>>
>> >>>
>> >>> By default WebGL does the same as GL. It doesn't flip anything. The
>> "data"
>> >>> pointer passed to texImage2D represents the bottom left corner of the
>> >>> texture. Whether that's an ArrayBuffer, an Image, a Canvas, a Video.
>> For
>> >>> Image, Canvas, and Video that means by default they'be upside down
>> (which is
>> >>> the same as GL).
>> >>>
>> >>>
>> >>> CopyTexImage2D in GL has always been "effectively" backward relative
>> to
>> >>> TexImage2D when talking about images because most (all?) image
>> libraries
>> >>> load images with the first pixel in the top, left and GL expects the
>> first
>> >>> pixel is the bottom, left. Since CopyTexImage is copying internally it
>> >>> appears to have different behavior because it's matching src data 0x0
>> =
>> >>> bottom, left, dst data 0x0 = bottom left where as TexImage2D, when
>> passing
>> >>> standard image data, is src data 0x0 = top,left, dest = bottom, left
>> >>>
>> >>>
>> >>> If you want WebGL to flip the data for you call
>> >>> gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, true);
>> >>>
>> >>>
>> >>> There is no PACK_FLIP_Y so readPixels will always return the
>> bottom,left
>> >>> as the first data in the array buffer.
>> >>>
>> >>>
>> >>> As for matching implementations it's all tested in the conformance
>> tests
>> >>> AFAIK.
>> >>>
>> >>>
>> >>>
>> >>> On Thu, Dec 6, 2012 at 4:59 PM, Jeff Gilbert < [email protected] >
>> >>> wrote:
>> >>>
>> >>>
>> >>>
>> >>> It seems backwards (as implemented) and underdefined in the spec. It
>> looks
>> >>> like we all upload HTML element texImage data with 0,0=top-left,
>> making it
>> >>> effectively upside-down.
>> >>>
>> >>> The following two lines yield different results:
>> >>> [1] gl.copyTexImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 0, 0,
>> >>> gl.drawingBufferWidth, gl.drawingBufferHeight, 0);
>> >>> [2] gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA,
>> gl.UNSIGNED_BYTE,
>> >>> gl.canvas);
>> >>>
>> >>> [1] matches this: [3]
>> >>> var data = new Uint8Array(gl.drawingBufferWidth *
>> gl.drawingBufferHeight *
>> >>> 4);
>> >>> gl.readPixels(0, 0, gl.drawingBufferWidth, gl.drawingBufferHeight,
>> >>> gl.RGBA, gl.UNSIGNED_BYTE, data);
>> >>> gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.drawingBufferWidth,
>> >>> gl.drawingBufferHeight, 0, gl.RGBA, gl.UNSIGNED_BYTE, data);
>> >>>
>> >>> UNPACK_FLIP_Y_WEBGL=true makes [2] the same as [1], but now [3] is
>> >>> upside-down.
>> >>>
>> >>> This is Mozilla bug 818810:
>> >>> https://bugzilla.mozilla.org/show_bug.cgi?id=818810
>> >>>
>> >>> Testcase showing inconsistent behavior:
>> >>> https://bugzilla.mozilla.org/attachment.cgi?id=689110
>> >>>
>> >>> As I mention in the bug, it doesn't look like y-orientation of
>> texImage2D
>> >>> of HTML elements is well-specified in the spec. So far everyone
>> appears to
>> >>> have implemented it the same, though.
>> >>>
>> >>> My suggestion was adding an UNPACK_ pixelStore option that allows us
>> to
>> >>> get OpenGL-style 0,0=bottom-left texImage2D uploads of HTML elements.
>> >>>
>> >>> -----------------------------------------------------------
>> >>> You are currently subscribed to [email protected] .
>> >>> To unsubscribe, send an email to [email protected] with
>> >>> the following command in the body of your email:
>> >>> unsubscribe public_webgl
>> >>> -----------------------------------------------------------
>> >>>
>> >>>
>> >>>
>> >>
>>
>> -----------------------------------------------------------
>> You are currently subscribed to [email protected]
>> To unsubscribe, send an email to [email protected] with
>> the following command in the body of your email:
>> unsubscribe public_webgl
>> -----------------------------------------------------------
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://khronos.org/pipermail/public_webgl_khronos.org/attachments/20121208/68cff738/attachment.html>


More information about the public_webgl mailing list