Sent: 2012-01-28 06:10:15 UTC (4 months ago)

From: Martin Nordholts

GIMP's future internal image data format

permalink

I don't want to hijack Alexandre's thread with the interesting
discussion that started therein, so here's a new thread for it.

2012/1/27 Bogdan Szczurek :
> W dniu 12-01-27 10:16, Alexandre Prokoudine pisze:
>> On Fri, Jan 27, 2012 at 11:01 AM, Martin Nordholts wrote:
>>> Images shall always be composed in 32-bit floating point
>>> RGBA
>
> Which RGB? Is it scRGB of GEGL "guts"? :)

Hi thebodzio

In the end, yes, GIMP's native image data format will likely be 32 bit
floating point per component/channel, pre-multiplied linear light RGBA
with the same primaries and white point as scRGB (and consequently
sRGB).

>>> and then have suitable filters and export mechanisms to deal with
>>> grayscale and indexed images.
>
>

Sent: 2012-01-28 13:33:36 UTC (4 months ago)

From: jcupitt@gmail.com

GIMP's future internal image data format

permalink

On 28 January 2012 06:10, Martin Nordholts wrote:
> 2. Make GIMP clever. If GIMP encounters a tile with only values 0.0
> and 1.0, the 32 bpc data can be transparently, i.e. without the user
> noticing, replaced with 1 bpc data. As soon as more bits of precision
> is required to avoid loss of data, GIMP can transparently convert the
> tile back to RGBA float. The same kind of optimization can be done for
> completely black, white and transparent tiles too.

This sounds nice, though the problem with this approach is that you
need to scan each tile to work out what format it can be compressed
to.

It's probably quicker to work out what format the various
intermediates should be in by static analysis of the pipeline of
operations. For example, a threshold operation can only ever output 0
and 1 (over threshold, under threshold) so you can always have a 1-bit
image coming out of it. If the next operation in the pipeline is
something like dilate, which works on binary images, you can directly
join the two operations up.

Another useful and related optimisation is to delay the conversion
from u8 to linear float as long as possible. You keep the original u8
image about plus the u8 to float LUT. For many unary pointwise
operations on the image (like threshold or level adjust) you can
simply manipulate the LUT and not touch the image at all.

John

Sent: 2012-01-28 14:37:35 UTC (4 months ago)

From: Øyvind Kolås

GIMP's future internal image data format

permalink

On Sat, Jan 28, 2012 at 1:33 PM, wrote:
> On 28 January 2012 06:10, Martin Nordholts wrote:
>> 2. Make GIMP clever. If GIMP encounters a tile with only values 0.0
>> and 1.0, the 32 bpc data can be transparently, i.e. without the user
>> noticing, replaced with 1 bpc data. As soon as more bits of precision
>> is required to avoid loss of data, GIMP can transparently convert the
>> tile back to RGBA float. The same kind of optimization can be done for
>> completely black, white and transparent tiles too.
>
> This sounds nice, though the problem with this approach is that you
> need to scan each tile to work out what format it can be compressed
> to.

Instead of replacing tiles with 1bpc data, one can either compress the
tile contents (trading of cpu use for io amount) or do a more generic
de-duplication through hashing at runtime (based on an idle job
combined with tracking revision numbers on tiles). Both are things
that fit well in the GeglBuffer architecture, though the runtime dedup
could have a rather large runtime cost.

Such streaming compression of tile data as it is saved/loaded wouldn't
be much slower than the current implementation. Though it likely be
quite a bit slower than a mmap backed thing with a threaded tile
writer.. (the hdd eating monster consumed a hdd with an almost working
such implementation last year).

/

Sent: 2012-01-28 19:22:53 UTC (4 months ago)

From: Bogdan Szczurek

GIMP's future internal image data format

permalink

> I don't want to hijack Alexandre's thread with the interesting
> discussion that started therein, so here's a new thread for it.

I think you did a good thing by creating this thread.

> 2012/1/27 Bogdan Szczurek:
>> W dniu 12-01-27 10:16, Alexandre Prokoudine pisze:
>>> On Fri, Jan 27, 2012 at 11:01 AM, Martin Nordholts wrote:
>>>> Images shall always be composed in 32-bit floating point
>>>> RGBA
>>
>> Which RGB? Is it scRGB of GEGL "guts"? :)
>
> Hi thebodzio
>
> In the end, yes, GIMP's native image data format will likely be 32 bit
> floating point per component/channel, pre-multiplied linear light RGBA
> with the same primaries and white point as scRGB (and consequently
> sRGB).

I thought so, but wasn't sure. Anyway

Sent: 2012-01-28 19:23:32 UTC (4 months ago)

From: Bogdan Szczurek

GIMP's future internal image data format

permalink

W dniu 12-01-28 15:37,

Sent: 2012-01-28 19:47:11 UTC (4 months ago)

From: Øyvind Kolås

GIMP's future internal image data format

permalink

On Sat, Jan 28, 2012 at 7:23 PM, Bogdan Szczurek wrote:
> W dniu 12-01-28 15:37,

Sent: 2012-01-28 21:44:43 UTC (4 months ago)

From: Bogdan Szczurek

GIMP's future internal image data format

permalink

W dniu 12-01-28 20:47,

Sent: 2012-01-30 15:55:55 UTC (4 months ago)

From: Martin Nordholts

GIMP's future internal image data format

permalink

2012/1/28 Bogdan Szczurek :
> Now

Sent: 2012-01-30 23:31:34 UTC (4 months ago)

From: Bogdan Szczurek

GIMP's future internal image data format

permalink

W dniu 12-01-30 16:55, Martin Nordholts pisze:
> 2012/1/28 Bogdan Szczurek:
>> Now

Sent: 2012-01-31 06:18:56 UTC (4 months ago)

From: Martin Nordholts

GIMP's future internal image data format

permalink

2012/1/30 Nathan Summers :
> Big images require lots of ram, but in my mind a high-quality photo
> editing program doesn't limit the size of how big an image you can
> edit with it to much smaller than how much you would normally be able
> to fit into ram.

Absolutely, but you'll still want big amounts of RAM when you work
with big images.

> Given the wide variety of input sources, output formats, and rendering
> targets that the vision contemplates, I wouldn't be so bold as to call
> image modes an unnecessary burden.

Sent: 2012-01-31 12:11:07 UTC (4 months ago)

From: Bogdan Szczurek

GIMP's future internal image data format

permalink

12-01-31 07:18, Martin Nordholts:
> 2012/1/30 Nathan Summers:
>> Big images require lots of ram, but in my mind a high-quality photo
>> editing program doesn't limit the size of how big an image you can
>> edit with it to much smaller than how much you would normally be able
>> to fit into ram.
>
> Absolutely, but you'll still want big amounts of RAM when you work
> with big images.

At least we all agree about that ;D

>> Given the wide variety of input sources, output formats, and rendering
>> targets that the vision contemplates, I wouldn't be so bold as to call
>> image modes an unnecessary burden. While it's true that it's often
>> (but not always) desirable to work in the highest precision possible,
>> getting an effect correct down to the 22nd binary digit isn't always
>> the top priority of the user.
>
> I really don't understand why someone would want to deal rounding
> errors. If you want them to create an artistic effect, you should use
> a filter that simulates it. I think it is safe to say that most users
> of a high-end photo manipulation program don't want to deal with
> rounding errors. To then force them to make a choice about it would be
> a disfavor and unnecessary burden.

I agree that having more precision is more favourable mathematically.
But is it so "visually"? I mean we'd have more precisely determined
color components but will it really "show"? And if it will, then how
much? Just a couple of questions, that, I believe, can't be answered
right now. I said it before, I say it again: let's wait and see :). Real
world application will for sure provide us with "proofs", while
continuing this could unfold into battle of "I thinks" and "IMHOs" not
"knows".

>> Given that the product vision you linked to explicitly included
>> targets like application icons and web pages, and I know of no
>> standard application icon or web page graphic format that even
>> optionally supports HDR, it's not far-fetched to say that a program
>> that targets icons and web page graphics that has the ability to work
>> in the native bit depth of those kinds of graphics can still be called
>> high-end.
>
> In the case of application icons and web page graphics, the extra bits
> of precision would be used to get rid of rounding errors when you
> stack layers and effects on top of each other.
>
>
>> Punting a decision on how to support non-RGB workflows until after you
>> have code for RGB workflows is pretty much a recipe for coding
>> yourself into a corner, especially if you toss out the existing
>> support for non-RGB workflows (i.e. indexed and grayscale) in the
>> process.
>
> Well, I don't think so. The strategies for dealing with RGB and CMYK
> are different enough for them to co-exist without stepping on each
> other's feet. As long as we don't implement RGB naively and
> short-sighted, we will be able to add CMYK support later without many
> conflicts with RGB.

A focal point is needed and if it had to be RGB

Sent: 2012-01-31 12:34:21 UTC (4 months ago)

From: Tobias Ellinghaus

GIMP's future internal image data format

permalink

Am Dienstag, 31. Januar 2012 schrub Bogdan Szczurek:

[...]

> non-square pixels

GIMP has that for ages.

[...]

Sent: 2012-01-31 13:05:37 UTC (4 months ago)

From: Bogdan Szczurek

GIMP's future internal image data format

permalink

> Am Dienstag, 31. Januar 2012 schrub Bogdan Szczurek:
>
> [...]
>
>> non-square pixels
>
> GIMP has that for ages.

Still, besides information about "uneven" horizontal and vertical
resolutions not stored within image itself. So designer have to
remember his image has non-square pixels and to check if "Dot for dot"
is off. AFAIK there's not built-in mechanism for setting pixel edges
ratio itself, which, IMHO, makes more sense than "artificial distorting"
pixels using resolution settings. The same effect, but convenience and
semantical correctness do differ.

Besides, when I said "non-square" I meant *non-square* per se (hexagonal
e.g.), not only "rectangular". I don't know what good could come out of
that, but who knows? It's just interesting, don't you think? :)

My best!
thebodzio

Sent: 2012-01-31 14:48:35 UTC (4 months ago)

From: gespertino@gmail.com

GIMP's future internal image data format

permalink

2012/1/31 Bogdan Szczurek

> I agree that having more precision is more favourable mathematically. But
> is it so "visually"? I mean we'd have more precisely determined color
> components but will it really "show"? And if it will, then how much? Just a
> couple of questions, that, I believe, can't be answered right now. I said
> it before, I say it again: let's wait and see :). Real world application
> will for sure provide us with "proofs", while continuing this could unfold
> into battle of "I thinks" and "IMHOs" not "knows".

The case of 8 bit being insufficient and the rounding errors caused by
working with such low precision is more common that one would think.
I used to think that it wasn't so critical, but after a couple of simple
tests you might change your mind.
- Gradients. Go from 30% gray to white. Dithering will make it more
pleasing visually, but adjust contrast and you'll see immediately you're
short of shades to work with.
- Create a black shape on white background. Apply 50 px gaussian blur.
Ajdust curves and you'll see banding immediately.
- Painting with soft edge brushes. Banding appears almost instantly when
you make a couple of overlapped strokes.

Grayscale with reduced dynamic range is almost impossible to work with
using 8bpc and the destructive effects will show even with slight
adjustments.
I use to design some brochures for a company that sales cars, and everytime
I have to touch up an image of a car (mostly in silver colors) I have to
duplicate, blur and diher patches to avoid banding. And I don't do extreme
manipulations there.
Another example is banding in skies in landscape photos.

Here I have to quote my friend Troy Sobotka who's usually a jerk towards
GIMP :-p but makes a point:
http://troy-sobotka.blogspot.com/2011/01/bit-depth-and-confusion.html

I'm not worried about the excess of data. That overhead is useful and will
be discarded anyway if you save to a 8bpc format, so I don't see the
problem there.
I just wonder if performance won't be substancially degraded when such
amount of data is handled.
AFAIK other applications as Photoshop or Mypaint are using 15bpc for
working with 8bpc images, leaving higher precision (photoshop) for higher
bitdepth "modes".
I'm sure devs are considering this problem, but my concern is what will
happen with gimp right after GEGL is fully integrated and before it gets
optimizations to make its performance really usable.
What's the plan? Realeasing GIMP with high bit depth when it's ready or
hold the realease until its fast enough?

A focal point is needed and if it had to be RGB… well… so be it. Meantime
> why not to think a little bit about the future? :) Why not dwell for moment
> in a world of non-square pixels, generic substractive and additive color
> models (who said RGB is *the best* and *ultimate* solution?),
> non-orthogonal pixel grids… dreams, dreams… :) That's it for the
> (hopefully) "inceptive" off-topic :)
>

We had this conversation before, and I keep my stand about CMYK. If we can
have a CMYK "projection" like Pippin and Peter suggested some time ago, and
if we have control over layers to make them "project" directly to pure CMYK
primaries or spot channels, then we can have a good CMYK/spot workflow
without having to deal with "modes"
For example: Imagine you can fill an RGBA layer with cyan and you can "tag"
that layer to tell gimp it will only separate to cyan in the CMYK
projection. That would address the need of spot channels (and CMYK
primaries, which are used as spot channels as soon as you decide to define
them manually).
The other situation to solve to get proper CMYK would be controlling the
black generation and that's it.
Color managed conversion from RGB to CMYK with control of the primaries is
all that you need to get proper CMYK, and that wouldn't require a special
"mode".

Sent: 2012-01-31 17:25:14 UTC (4 months ago)

From: Bogdan Szczurek

GIMP's future internal image data format

permalink

W dniu 12-01-31 15:48, gespertino@gmail.com pisze:
> 2012/1/31 Bogdan Szczurek >
>
> I agree that having more precision is more favourable
> mathematically. But is it so "visually"? I mean we'd have more
> precisely determined color components but will it really "show"? And
> if it will, then how much? Just a couple of questions, that, I
> believe, can't be answered right now. I said it before, I say it
> again: let's wait and see :). Real world application will for sure
> provide us with "proofs", while continuing this could unfold into
> battle of "I thinks" and "IMHOs" not "knows".
>
>
> The case of 8 bit being insufficient and the rounding errors caused by
> working with such low precision is more common that one would think.
> I used to think that it wasn't so critical, but after a couple of simple
> tests you might change your mind.
> - Gradients. Go from 30% gray to white. Dithering will make it more
> pleasing visually, but adjust contrast and you'll see immediately you're
> short of shades to work with.
> - Create a black shape on white background. Apply 50 px gaussian blur.
> Ajdust curves and you'll see banding immediately.
> - Painting with soft edge brushes. Banding appears almost instantly when
> you make a couple of overlapped strokes.

You're right. I've tried to give some smart*ss answer to that, but
failed :).

> Grayscale with reduced dynamic range is almost impossible to work with
> using 8bpc and the destructive effects will show even with slight
> adjustments.
> I use to design some brochures for a company that sales cars, and
> everytime I have to touch up an image of a car (mostly in silver colors)
> I have to duplicate, blur and diher patches to avoid banding. And I
> don't do extreme manipulations there.
> Another example is banding in skies in landscape photos.

It's not always so bad. I do agree however that the less precise
information we have

Sent: 2012-02-03 19:35:13 UTC (3 months ago)

From: Tobias Oelgarte

GIMP's future internal image data format

permalink

Am 31.01.2012 18:25, schrieb Bogdan Szczurek:
> W dniu 12-01-31 15:48, gespertino@gmail.com pisze:
>> 2012/1/31 Bogdan Szczurek > >
>>
>> I agree that having more precision is more favourable
>> mathematically. But is it so "visually"? I mean we'd have more
>> precisely determined color components but will it really "show"? And
>> if it will, then how much? Just a couple of questions, that, I
>> believe, can't be answered right now. I said it before, I say it
>> again: let's wait and see :). Real world application will for sure
>> provide us with "proofs", while continuing this could unfold into
>> battle of "I thinks" and "IMHOs" not "knows".
>>
>>
>> The case of 8 bit being insufficient and the rounding errors caused by
>> working with such low precision is more common that one would think.
>> I used to think that it wasn't so critical, but after a couple of simple
>> tests you might change your mind.
>> - Gradients. Go from 30% gray to white. Dithering will make it more
>> pleasing visually, but adjust contrast and you'll see immediately you're
>> short of shades to work with.
>> - Create a black shape on white background. Apply 50 px gaussian blur.
>> Ajdust curves and you'll see banding immediately.
>> - Painting with soft edge brushes. Banding appears almost instantly when
>> you make a couple of overlapped strokes.
>
> You're right. I've tried to give some smart*ss answer to that, but
> failed :).
>
>> Grayscale with reduced dynamic range is almost impossible to work with
>> using 8bpc and the destructive effects will show even with slight
>> adjustments.
>> I use to design some brochures for a company that sales cars, and
>> everytime I have to touch up an image of a car (mostly in silver colors)
>> I have to duplicate, blur and diher patches to avoid banding. And I
>> don't do extreme manipulations there.
>> Another example is banding in skies in landscape photos.
>
> It's not always so bad. I do agree however that the less precise
> information we have

Sent: 2012-02-05 00:07:57 UTC (3 months ago)

From: Bogdan Szczurek

GIMP's future internal image data format

permalink

-- cut for brevity's sake :) --

> As an artist that does a lot of hand drawing i noticed the problems of
> the 8 bit limit very quickly. Just use the smudge tool and you will see
> any color, but not the intended colors (rounding errors=new colors).
> Same goes for the blur brush. Ever tried to to blur larger areas with
> it? Guess not, since it doesn't work. If the gradient is flat enough it
> won't blur anymore, since it is just one step away from actually
> switching a bit in the result. That are very common cases that really
> would profit from higher color resolutions.

Naah ;) you've misunderstood me

Sent: 2012-02-07 17:59:30 UTC (3 months ago)

From: Nathan Summers

GIMP's future internal image data format

permalink

On Sat, Feb 4, 2012 at 7:07 PM, Bogdan Szczurek wrote:
> -- cut for brevity's sake :) --
>
>
>> As an artist that does a lot of hand drawing i noticed the problems of
>> the 8 bit limit very quickly. Just use the smudge tool and you will see
>> any color, but not the intended colors (rounding errors=new colors).
>> Same goes for the blur brush. Ever tried to to blur larger areas with
>> it? Guess not, since it doesn't work. If the gradient is flat enough it
>> won't blur anymore, since it is just one step away from actually
>> switching a bit in the result. That are very common cases that really
>> would profit from higher color resolutions.
>
> Naah ;) you've misunderstood me

Sent: 2012-02-07 20:43:53 UTC (3 months ago)

From: Martin Renold

GIMP's future internal image data format

permalink

On Tue, Jan 31, 2012 at 06:25:14PM +0100, Bogdan Szczurek wrote:
> W dniu 12-01-31 15:48, gespertino@gmail.com pisze:
> >Here I have to quote my friend Troy Sobotka who's usually a jerk towards
> >GIMP :-p but makes a point:
> >http://troy-sobotka.blogspot.com/2011/01/bit-depth-and-confusion.html
>
> As much as I agree with general idea I want to point out two things
> about image comparing 8 bit dithered and 15 bit outputs (the square
> regions).

My point was to show the dithering noise that I got rid of by switching from
8bpc to 15bpc. It wasn't really meant for scientific analysis ;-)

> First: they're not of the same dynamic range. The upper one is "brighter",
> "wider"

Well, I just opened both MyPaint versions and painted roughly the same
(large) number of circles with a faint brush. The right box is a
magnification of the left box (which is a section of the full circle).

> and "not grayscale" like the bottom one. It makes its "ugliness" more
> apparent.

Both images were made with a pure black brush painting over pure white. The
color that you see comes from the dithering. I used a different random
number to dither each color channel. This actually improves the perceived
smoothness of the result.

> Second: why to use dithering anyway since internally you're using sRGB,
> monitor "should be" at least sRGB?… Do you have any other examples maybe?

Dithering was done per dab. When you stamp a dab, 8bpc over 8bpc, you get a
16bpc intermediate result. When scaling this back to 8bpc I used dithering
instead of simple rounding. If I had done rounding, I would have gotten
banding instead of noise. Same problem, different manifestation.

Martin Renold

Sent: 2012-02-08 00:50:06 UTC (3 months ago)

From: Bogdan Szczurek

GIMP's future internal image data format

permalink

W dniu 12-02-07 18:59, Nathan Summers pisze:
> On Sat, Feb 4, 2012 at 7:07 PM, Bogdan Szczurek wrote:
>> -- cut for brevity's sake :) --
>>
>>
>>> As an artist that does a lot of hand drawing i noticed the problems of
>>> the 8 bit limit very quickly. Just use the smudge tool and you will see
>>> any color, but not the intended colors (rounding errors=new colors).
>>> Same goes for the blur brush. Ever tried to to blur larger areas with
>>> it? Guess not, since it doesn't work. If the gradient is flat enough it
>>> won't blur anymore, since it is just one step away from actually
>>> switching a bit in the result. That are very common cases that really
>>> would profit from higher color resolutions.
>>
>> Naah ;) you've misunderstood me

Sent: 2012-02-08 01:06:51 UTC (3 months ago)

From: Bogdan Szczurek

GIMP's future internal image data format

permalink

W dniu 12-02-07 21:43, Martin Renold pisze:
> On Tue, Jan 31, 2012 at 06:25:14PM +0100, Bogdan Szczurek wrote:
>> W dniu 12-01-31 15:48, gespertino@gmail.com pisze:
>>> Here I have to quote my friend Troy Sobotka who's usually a jerk towards
>>> GIMP :-p but makes a point:
>>> http://troy-sobotka.blogspot.com/2011/01/bit-depth-and-confusion.html
>>
>> As much as I agree with general idea I want to point out two things
>> about image comparing 8 bit dithered and 15 bit outputs (the square
>> regions).
>
> My point was to show the dithering noise that I got rid of by switching from
> 8bpc to 15bpc. It wasn't really meant for scientific analysis ;-)

I got your point :). I just expected something "stricter" :). As said
earlier, I'm not trying to invalidate the reasoning behind 15 bits in
this case.

>> First: they're not of the same dynamic range. The upper one is "brighter",
>> "wider"
>
> Well, I just opened both MyPaint versions and painted roughly the same
> (large) number of circles with a faint brush. The right box is a
> magnification of the left box (which is a section of the full circle).

I thought so.

>> and "not grayscale" like the bottom one. It makes its "ugliness" more
>> apparent.
>
> Both images were made with a pure black brush painting over pure white. The
> color that you see comes from the dithering. I used a different random
> number to dither each color channel. This actually improves the perceived
> smoothness of the result.

I'm OK with that, however in this case I've got the impression that
colors "sticking out" of neutral gray are actually "emphasizing"
visibility of the dithering (I've did a comparison myself).

>> Second: why to use dithering anyway since internally you're using sRGB,
>> monitor "should be" at least sRGB?… Do you have any other examples maybe?
>
> Dithering was done per dab. When you stamp a dab, 8bpc over 8bpc, you get a
> 16bpc intermediate result. When scaling this back to 8bpc I used dithering
> instead of simple rounding. If I had done rounding, I would have gotten
> banding instead of noise. Same problem, different manifestation.

Got it :)!

My best!
thebodzio