I've been working on porting the Gimp lcms.c plug-in from using LittleCMS version 1 to using LittleCMS version 2. This will make possible high bit-depth ICC profile conversions.

I'm not a super-experienced c-coder, nor have I ever worked with the lcms engine before. So I'm learning as I go. So far I've: (1)compiled the existing Gimp lcms plug-in, (2)modified the existing plug-in to use lcms2.h and attempted to compile it (knowing it would fail)
(3)tracked down all the errors and warnings that result from the differences between LittleCMS version 1 and LittleCMS version 2

My next steps will be to add a whole bunch of "print to screen" commands to the current Gimp lcms.c plug-in so I can get a better handle on the flow of the code, and then start rewriting (or perhaps writing from scratch) the plug-in to use the LittleCMS v2 engine.

If anyone is curious, I've documented what I've done so far:

http://ninedegreesbelow.com/temp/gimp-lcms-1.html http://ninedegreesbelow.com/temp/gimp-lcms-2.html

Comments, input is more than welcome.

I have a couple of big-picture questions:

Will Gimp be using as its internal working space some variant of Microsoft's scRGB? What about:

Wikipedia: http://en.wikipedia.org/wiki/Scrgb "scRGB is a wide color gamut RGB (Red Green Blue) color space created by Microsoft and HP that uses the same color primaries and white/black points as the sRGB color space but allows coordinates below zero and greater than one. . . . the cost of maintaining compatibility with sRGB is that approximately 80% of the scRGB color space consists of imaginary colors."

"Two encodings are defined for the individual primaries: a linear 16 bit per channel encoding and a nonlinear 12 bit per channel encoding. . . The 16 bit scRGB(16) encoding is the linear RGB channels converted by 8192 x + 4096. Compared to 8-bit sRGB this ranges from about 1/2 the color resolution near 0.0 to more than 10 times the color resolution
near 1.0."

That last sentence is a bit worrisome - losing resolution in the shadow areas.

It seems to me that you will always need ICC profiles, to convert an image from whatever ICC color space profile it happens to be in, to your internal working space, and from your internal working space to the monitor profile, and from your internal working space back out to whatever color space profile the person wants to use to upon exporting to a non-xcf file. Yes? No?

Elle

Hi, John,

By "native space" do you mean whatever color space the image happens to be in when you open it? As per the jpegs generated by my point-and-shoot are all in the sRGB color space? Or if someone hands me a file in the AdobeRGB color space, then that file's "native" color space is AdobeRGB? If this is what you mean by "native color space", then I agree with you 100% - converting from one color space to another without a good reason is pointless and can potentially have bad effects on image integrity.

The "internal working space" I was asking about is something else altogether. I am under the impression that Gimp either already does, or is planning to, use a very large internal working space, which internal working space will be completely beyond the end user's control.

So the admittedly ambiguous phrase I used, "your internal working space", was referring to Gimp's internal working space, if in fact Gimp either already uses, or is planning to use such a thing. I wasn't referring to whatever working space a user might have specified in Gimp's color management settings, but I can see how my wording was ambiguous.

I was hoping for some very brief answers to the following questions:

1. Is my impression correct? Is Gimp planning to use a very large internal working space for all image manipulation? If the answer is "No", ignore questions 2, 3, and 4.

2. If so, is that color space something like scRGB mentioned in the Wikipedia article?

3. If so, what about shadow detail (a problem mentioned in the Wikipedia article)?

4. If so, will there still be a need for ICC profile conversions (I can't imagine what the alternative might be, but that doesn't mean there isn't an alternative) from the image's "native space" (if I understand your usage of the phrase correctly) to Gimp's very large internal working space (and back out to whatever output space the user wants upon exporting the image, and also to display the image on the monitor screen)?

Elle

On 7/19/12, John Harris wrote:

Elle, I am not a programmer but I am a color professional. so here is my input regarding your last question:

It seems to me that you will always need ICC profiles, to convert an image from whatever ICC color space profile it happens to be in, to your internal working space, and from your internal working space to the monitor profile, and from your internal working space back out to whatever color space profile the person wants to use to upon exporting to a non-xcf file. Yes? No?

"always" is a bit strong. Some individuals, such as myself, prefer to do as little profile swapping as possible, and in some cases work in complete native spaces during the entire flow. Display calibration excluded.

I rarely take an incoming file from it's native space to some other working space unless the file needs extreme adjustments. So converting from native to working is not a given. There will be some situations where I will send the image file directly to the output device without a conversion to the output profile. An output profile should be able to be used in soft-proofing without the need for conversion to profile. The Adobe workflow refers to this as "Preserve RGB". Having this option allows me and other users to proof the output on screen and never convert the file to any profile, thus keeping the working file in it's fullest integrity, free of banding, loss of color fidelity and other profile related issues that can result from remapping.

There may also be times when I will maintain the files native space but print using a convert to profile flow, and there are times when I DO convert from the native space to a larger space. The latter is often the case when the file needs major adjustments for density and contrast. I may also go from a larger space, such as ProRGB or Adobe1998 down to sRGB if I am working a file that will only see usage on the web and never go to wide gamut print.

Feel free to reach out to me with specific questions that might be off-topic.

On 07/19/2012 11:12 AM, Elle Stone wrote:

I've been working on porting the Gimp lcms.c plug-in from using LittleCMS version 1 to using LittleCMS version 2. This will make possible high bit-depth ICC profile conversions.

I'm not a super-experienced c-coder, nor have I ever worked with the lcms engine before. So I'm learning as I go. So far I've: (1)compiled the existing Gimp lcms plug-in, (2)modified the existing plug-in to use lcms2.h and attempted to compile it (knowing it would fail)
(3)tracked down all the errors and warnings that result from the differences between LittleCMS version 1 and LittleCMS version 2

My next steps will be to add a whole bunch of "print to screen" commands to the current Gimp lcms.c plug-in so I can get a better handle on the flow of the code, and then start rewriting (or perhaps writing from scratch) the plug-in to use the LittleCMS v2 engine.

If anyone is curious, I've documented what I've done so far:

http://ninedegreesbelow.com/temp/gimp-lcms-1.html http://ninedegreesbelow.com/temp/gimp-lcms-2.html

Comments, input is more than welcome.

I have a couple of big-picture questions:

Will Gimp be using as its internal working space some variant of Microsoft's scRGB? What about:

Wikipedia: http://en.wikipedia.org/wiki/Scrgb "scRGB is a wide color gamut RGB (Red Green Blue) color space created by Microsoft and HP that uses the same color primaries and white/black points as the sRGB color space but allows coordinates below zero and greater than one. . . . the cost of maintaining compatibility with sRGB is that approximately 80% of the scRGB color space consists of imaginary colors."

"Two encodings are defined for the individual primaries: a linear 16 bit per channel encoding and a nonlinear 12 bit per channel encoding. . . The 16 bit scRGB(16) encoding is the linear RGB channels converted by 8192 x + 4096. Compared to 8-bit sRGB this ranges from about 1/2 the color resolution near 0.0 to more than 10 times the color resolution
near 1.0."

That last sentence is a bit worrisome - losing resolution in the shadow areas.

It seems to me that you will always need ICC profiles, to convert an image from whatever ICC color space profile it happens to be in, to your internal working space, and from your internal working space to the monitor profile, and from your internal working space back out to whatever color space profile the person wants to use to upon exporting to a non-xcf file. Yes? No?

Elle

On 7/20/12, yvind Kols wrote:

As we move along towards embracing a fully GEGLified workflow, different things will happen at different times. The initial thing to reach for is likely to support the 2.8 era manual colormanagement. With a fully GEGLified system there will still be the need to use ICC based conversions to pixel formats efficiently dealt with by babl/GEGL/GIMP as well as do final transforms for display / export.

Thanks! Now I have a better idea of the big picture regarding the lcms plug-in. I've had a chance to dig into the code and see what the different functions do and when they are called. It looks like converting the lcms.c plug-in to use lcms2 will be pretty easy to do. Which means really high bit depth conversions should also be not too difficult. But one step at a time.

Elle

http://ninedegreesbelow.com
Articles and tutorials on open source digital imaging and photography
_______________________________________________
gimp-developer-list mailing list
gimp-developer-list@gnome.org
https://mail.gnome.org/mailman/listinfo/gimp-developer-list

On 7/20/12, Elle Stone wrote:

On 7/20/12, yvind Kols wrote:

As we move along towards embracing a fully GEGLified workflow, different things will happen at different times. The initial thing to reach for is likely to support the 2.8 era manual colormanagement. With a fully GEGLified system there will still be the need to use ICC based conversions to pixel formats efficiently dealt with by babl/GEGL/GIMP as well as do final transforms for display / export.

Thanks! Now I have a better idea of the big picture regarding the lcms plug-in. I've had a chance to dig into the code and see what the different functions do and when they are called. It looks like converting the lcms.c plug-in to use lcms2 will be pretty easy to do. Which means really high bit depth conversions should also be not too difficult. But one step at a time.

Just an update. I rewrote the existing Gimp lcms.c plug-in to use lcm2. It seems to work just like the current Gimp plug-in. If anyone is interested in testing it, I can supply the code.

As explained here: http://ninedegreesbelow.com/temp/gimp-lcms-3.html (see summary at bottom of page), there are some coding issues to be resolved before attempting to make the lcms2 plug-in do high-bit-depth color conversions. So dealing with the coding issues is my next step.

Also, has anyone seen this Ubuntu bug report? Completely replace lcms1 by lcms2 in Ubuntu https://bugs.launchpad.net/ubuntu/+source/lcms/+bug/885324

Kind regards, Elle

gimp-developer-list mailing list
gimp-developer-list@gnome.org
https://mail.gnome.org/mailman/listinfo/gimp-developer-list

Yes, it looks like both of them are concerned with exactly what I've been trying to do. Sigh.

As I'm working with Gimp 2.9 from git,

and have no clue how Gimp development works, and only got involved because I really want Gimp color conversion to use high bit-depth, and my impression was that making this happen is not exactly a high priority item for Gimp and right now Gimp from git is doing 8-bit color conversions,

may I ask why the Gimp from git that I'm using (which is really recent, July 13, 2012) is still using lcms1 if there are patches for lcms2?

Elle

On 7/24/12, Michael Schumacher wrote:

On 24.07.2012 17:47, Elle Stone wrote:

Just an update. I rewrote the existing Gimp lcms.c plug-in to use lcm2. It seems to work just like the current Gimp plug-in. If anyone is interested in testing it, I can supply the code.

Also, has anyone seen this Ubuntu bug report? Completely replace lcms1 by lcms2 in Ubuntu https://bugs.launchpad.net/ubuntu/+source/lcms/+bug/885324

Hi Elle,

there are two bug reports in Bugzilla about changes in GIMP related to lcms2:

This one introduces lcms2 to get support for ICC V4 profiles: https://bugzilla.gnome.org/show_bug.cgi?id=662739

That one has been created to move everything to lcms2 to counter some performance problems:
https://bugzilla.gnome.org/show_bug.cgi?id=675558

I'm not sure if you are aware of them. Both have patches attached, and you should check if any of them - in particular the latter - is a duplication of your efforts.

HTH, Michael
_______________________________________________ gimp-developer-list mailing list
gimp-developer-list@gnome.org
https://mail.gnome.org/mailman/listinfo/gimp-developer-list

How does one apply a patch? What does "apply a patch" mean?

On 7/24/12, Michael Schumacher wrote:

On 24.07.2012 23:00, Elle Stone wrote:

may I ask why the Gimp from git that I'm using (which is really recent, July 13, 2012) is still using lcms1 if there are patches for lcms2?

Because no one has tried and applied the patches yet.

Perhaps what I really should be asking is what is the process by which an existing patch to the Gimp code is applied so that everyone can use it?

In particular, by what process does Gimp get patched to use lcms2? ( I kinda doubt that the Gimp developers are going to let Elle Stone patch Gimp, as the chances are very great that Elle Stone would break Gimp in the process.)

I don't need to patch my own copy of Gimp. It's already using lcms2.

On 7/24/12, Michael Schumacher wrote:

On 24.07.2012 23:16, Elle Stone wrote:

How does one apply a patch? What does "apply a patch" mean?

You use the patch program to apply the changes that are contained in the attachments to the bug report to the GIMP code.

The usage example from the Wikipedia article about patch should do: http://en.wikipedia.org/wiki/Patch_%28Unix%29

patch < foo.diff

This could be executed in the top directory of your GIMP source tree.

There are parameters to adjust for differences in the diff files regarding both your and the patch author's systems, see http://linux.die.net/man/1/patch - -p is the most commonly used parameter to get rid of leading folders in the paths used in the diff file, for example.

--
HTH,
Michael
_______________________________________________ gimp-developer-list mailing list
gimp-developer-list@gnome.org
https://mail.gnome.org/mailman/listinfo/gimp-developer-list

Jernej, thanks! That is a very clear explanation of how to use patch.

So please, just go ahead, look at the patches in bugzilla, pick the best stuff from your patches and the bugzilla ones, and all will be fine.

OK, I'm working on it. And I promise to keep communicating actively. I'm making two Gimp from git installations, one for the existing lcms2 patches (thank you! Michael Schumacher, for pointing the two patches out to me) and one for the lcms2 code that I'm writing.

On a related note, I reported a bug back in 2010: https://bugzilla.gnome.org/show_bug.cgi?id=620781 (I changed my website domain, the link I gave in the bug report is no longer available). That bug still persists in Gimp 2.8/2.9 if the sRGB to which one is converting to, starting with a linear gamma image, is "sRGB built-in". However, using my lcms.c plugin using lcms2.h, that bug disappears. I need to do some more testing, but there is a more subtle bug, also involving conversions to and from a linear gamma color space, that also seems to no longer be a problem when using my lcms2 plug-in.

We are short on
resources, and naturally active community members are harder to forget, as opposed to patches sitting in bugzilla.

It always surprises me how few people are working on a project as big, important, and widely used as Gimp.

On 7/24/12, Michael Schumacher wrote:

there are two bug reports in Bugzilla about changes in GIMP related to lcms2:

This one introduces lcms2 to get support for ICC V4 profiles: https://bugzilla.gnome.org/show_bug.cgi?id=662739

That one has been created to move everything to lcms2 to counter some performance problems:
https://bugzilla.gnome.org/show_bug.cgi?id=675558

I'm not sure if you are aware of them. Both have patches attached, and you should check if any of them - in particular the latter - is a duplication of your efforts.

Patches compared:
I've checked both patches, line for line comparing each patch to the code changes that I've made so far to convert the Gimp lcms.c plug-in to use lcms2.

Both patches cover *all* the source code files that are affected by a port from lcms to lcms2 (not just the lcms.c file itself). So far I've only worked on the actual lcms.c plug-in code. Before working on the other files myself, I want to add "printf" statements to the other relevant source code files so I can get a better idea of how all of Gimp color management hangs together. In particular, I want to track down where/why Gimp doesn't offer the user the option of using or not using black point compensation with the user-chosen monitor profile/conversion intent.

Both patches are similar to what I've written so far. The second patch lcms.c code is very, very close to what I've written and the first patch already does some of what was on my "to do" list:

1. Retrieving textual information: The first patch code replaces the three existing information collecting functons (that all actually produce the same information), and adds code that collects additional information (model, copyright, etc). The second patch and my own code both use a "three to one" workaround (keeping more of the original code intact, not necessarily a plus, but perhaps a safer first move). Also the first patch directly uses the lcms2 UTF8 information, rather than getting the ascii information and passing it to Gimp to change to UTF8 - a definite plus, it seems to me.

2. Error Handling (bad profiles, lcms errors in conversions, etc): The first patch keeps the existing non-functioning error code reporting intact. The second patch and my own code both eliminate the non-functioning error code. Ideally there should be some error code reporting/some way to detect and deal with bad profiles. But based on testing as well as examining the code, the existing code doesn't do anything at all. I have located examples of functioning lcms1 and presumably functioning lcms2 error handling, but haven't tried to incorporate same.

3. Checksum: Both patches update to lcms2 some MD5 code that I've temporarily commented out. I commented the code out partly because I hadn't found the right code to replace the old code, and mostly because I don't understand the point of calculating the MD5 of the profile header (to make sure you still have the profile you just opened??). See functions "lcms_calculate_checksum" and the no doubt related "lcms_sRGB_checksum". What is a "digest"? Why calculate the MD5 of the profile header?

The next steps: Based on the code, both patches look like they would probably work just fine. My next step is to actually apply the patches (separately, of course) and see if/how well the patches work in practice. And then pick and choose the best bits of each patch. It will be several days (at least) before I finish this next step.

Bug fix? enhancement? It seems to me that porting to use lcms2 is both a fix for some bugs (wrong results when converting from a linear gamma to the Gimp sRGB built in profile; inability to read V4 profiles, which are becoming more and more common), and also an enhancement in terms of speed, based on reports by others. And of course lcms2 is a prerequisite for 32-bit floating point ICC profile conversions.

Alexandre, thanks! for the link to the upcoming lcms2 changes regarding supported floating point variants.

I posted an update to my effort to port the lcms plugin to lcms2 and get it to do high-bit-depth color conversions:

http://ninedegreesbelow.com/temp/gimp-lcms-4.html

Summary of current state of plugin:

It uses lcms2. It does high bit depth conversions, but some options are not available yet. It still uses deprecated functions.

Black point compensation flag has been temporarily removed (easy to add back in, just haven't done it yet).

The code doesn't check for an alpha channel, and won't work if the image has an alpha channel (easy to add, just haven't done it yet).

The code will convert at 8-bit integer, 16-bit integer and 32-bit floating point (only, so far).

The ICC profile conversion is seemingly correct only if the starting profile is a linear gamma version of sRGB and the ending profile is the Gimp built-in sRGB, or vice versa.

Upon converting to any other ICC profile, the colors come out all wrong (confirm by opening in another application and eye-droppering). ***However***, upon:

*exporting the odd-looking image anyway (and confirming that the numbers are wrong in another imaging application) *closing and reopening Gimp (not a necessary step, but eliminates the possibility of "right values" somehow being in memory) *reopening the exported image (which still looks odd) *and then converting it to built-in sRGB or to linear sRGB

. . . then the colors are magically right. Which is both bizarre and wrong.

Elle

On 8/13/12, Elle Stone wrote:

Upon converting to any other ICC profile, the colors come out all wrong . . . then the colors are magically right. Which is both bizarre and wrong.

A coding error/oversight on my part is the answer to the "magically right": the new profile is embedded even if the conversion didn't really happen.

Elle

My lcms2 plug-in now does do correct ICC profile conversions, from any RGB color space, to any RGB color space, at 8-bit integer, 16-bit integer, and 32-bit floating point.

However, to get it to work, I had to modify the babl/babl/util.h file.

After modifying the babl/babl/util.h file, 16-bit tiffs open correctly. As you might recall, with the default util.h file, 16-bit tiffs open with a mysterious and erroneous "gamma=2.2 correction" having been applied. This tiff issue has nothing to do with the lcms plug-in.

I realize that modifying a babl file might be a controversial move on my part. But I would ask you to look at this web page before dismissing what I've done:
http://ninedegreesbelow.com/temp/gimp-lcms-5.html

The modified babl/babl/util.h code can be found here: http://ninedegreesbelow.com/temp/gimp-lcms-4.html

Elle

On 8/27/12, yvind Kols wrote:

On Mon, Aug 27, 2012 at 4:08 PM, Elle Stone wrote:

My lcms2 plug-in now does do correct ICC profile conversions, from any RGB color space, to any RGB color space, at 8-bit integer, 16-bit integer, and 32-bit floating point.

However, to get it to work, I had to modify the babl/babl/util.h file.

This is great news, since it means that the code works as intenended but probably with slightly different intentions.

Could you clarify? which code works with which intentions?

After modifying the babl/babl/util.h file, 16-bit tiffs open correctly. As you might recall, with the default util.h file, 16-bit tiffs open with a mysterious and erroneous "gamma=2.2 correction" having been applied. This tiff issue has nothing to do with the lcms plug-in.

Data stored for higher bitdepths per pixel than 8 are assumed to be stored with a linear gamma ramp,

ICC V2 matrix profiles (simplest case here) have two parts that are relevant to the present discussion:

1. They have RGB/XYZ primaries, the chromaticity coordinates that "interpret" if you will the chroma part of the image RGB values.

2. They have a tone response curve (TRC) that determines how fast the image RGB tonality approches maximum white, starting from minimum black, as the RGB values go from 0 to whatever maximum value is allowed by the bit-depth of the image.

Also, the sRGB V2 matrix ICC profile has a 1024 point TRC that is contained in the sRGB profile TRC tags; this sRGB TRC can be exported and then imported into another ICC profile to change the other profile's TRC, to create a variant of the other profile that has the other profile's primaries, but the sRGB TRC.

The *only* time the default babl code properly converts an image from one ICC profile to another is if two conditions are met:

1. The primaries are the same in the source and destination profiles. 2. The TRCs of the two profiles are either linear gamma or the sRGB TRC.

So if I create a scarse prophoto variant that has a TRC equal to the sRGB TRC instead of being equal to gamma=1.8, and a second variant that has a linear gamma TRC, I can use the default babl code to convert from linear prophoto to the special prophoto variant with the sRGB TRC and back again.

If I create a variant of ClayRGB that has a linear gamma TRC and a second variant that has the sRGB TRC, I can use the default babl code to convert from linear gamma ClayRGB to the special ClayRGB variant with the sRGB TRC and back again.

All other ICC profile conversions are wrong if done with the default babl code, but correct if I modify the /babl/babl/util.h file.

In particular, if I keep the TRC as linear gamma for the source and destination profile, but the primaries vary, the colors are wrong.

And if I keep the primaries the same for the source and destination profile, but the TRCs are anything other than linear gamma and the sRGB TRC, the tonality comes out wrong.

And if both the primaries and the TRC vary, then everything is wrong. Unless I modify the util.h file. Then everything is right.

a higher level code where you
probably could do a similar tweak without changing core babl behavior could be in ... digging through GIMP code ... gimp/app/gegl/gimp-babl.c which contains code determining the data expected to be managed, for all arbitrary profile source imagery it would be recommended to store layer data in floating point.

I don't think there is any way to tweak the Gimp code to undo the effect of the util.h code that converts "something" in the background from a linear gamma TRC to the regular sRGB TRC and back. But perhaps if I had a better understanding of what the default babl code in babl/babl/base/util.h, and also in fast-float.c and float.c, is supposed to accomplish, it would help. Can anyone give a summary?

/yvind K.

Elle

gimp-developer-list mailing list
gimp-developer-list@gnome.org
https://mail.gnome.org/mailman/listinfo/gimp-developer-list

If anyone would like to try the lcms2 high bit depth plug-in, I put up instructions on compiling Gimp from git and the lcms2 plug-in, plus babl and gegl, with the modified babl/babl/util.h file:

http://ninedegreesbelow.com/temp/gimp-lcms-6.html