Sign up now! · Forgot password?
RSS/Atom feed Twitter

AI algorithms in GIMP

This discussion is connected to the gimp-developer-list.gnome.org mailing list which is provided by the GIMP developers and not related to gimpusers.com.

This is a read-only list on gimpusers.com so this discussion thread is read-only, too.

8 of 10 messages available
Toggle history

Please log in to manage your subscriptions.

CAF9cbAQP5Juht1mLL+28wtXPum... 25 Jan 18:21
  CAFgjPJ_zXY8kpw=x1EbhOhiqSg... 25 Jan 18:21
   AI algorithms in GIMP Maitraya Bhattacharyya via gimp-developer-list 21 Jan 13:53
    AI algorithms in GIMP Elle Stone 21 Jan 15:01
     AI algorithms in GIMP Martin Marmsoler via gimp-developer-list 21 Jan 15:21
      AI algorithms in GIMP Maitraya Bhattacharyya via gimp-developer-list 21 Jan 15:40
     AI algorithms in GIMP Elle Stone 21 Jan 15:42
      AI algorithms in GIMP Maitraya Bhattacharyya via gimp-developer-list 21 Jan 15:47
       AI algorithms in GIMP Maitraya Bhattacharyya via gimp-developer-list 04 Feb 11:50
    AI algorithms in GIMP Liam R E Quin 21 Jan 21:39
Maitraya Bhattacharyya via gimp-developer-list
2019-01-21 13:53:01 UTC (2 months ago)

AI algorithms in GIMP

Hi guys,

I think Casey's suggestion of super resolution is an excellent idea and would be useful to a lot of people using GIMP. The two algorithms which have been doing quite well are:

1. SRGAN: The relevant paper is at https://arxiv.org/abs/1609.04802 . This has yielded some very nice results so far and is now considered to be state-of-the-art in the computer vision community. Implementing this will be my first priority.

2. SRResNet: The paper is at https://arxiv.org/abs/1609.04802 . This algorithm also gives quite good results.

Now to train these neural nets, I will have to prepare a dataset consisting of low resolution and high resolution images and divide them into training, testing and validation sets. I'll try to write a script which pulls high-resolution CC-A/CC0/CC-A-SA images from the internet and downscale them in python to prepare their low resolution counterparts. I only hope that my GPU has enough memory to train the model, otherwise I am done for.

What I would like to do is create a python script (most of these libraries are available in Python) and then pass the image along with the scaling factor into this from inside GIMP. I've never worked with GEGL before but I will try finding out how to use this.

Since I am working on this in my free time and have not implemented neural nets much, this might take some time for me to learn. I'll try keeping you guys informed.

Any suggestions are welcome.

Thanks, Maitraya.

On Sun, Jan 20, 2019 at 12:41 AM Jehan Pagès wrote:

Hi!

On Thu, Jan 17, 2019 at 12:14 PM Maitraya Bhattacharyya via gimp-developer-list wrote:

Dear devs,

I have recently joined the mailing list because I wanted to contribute my two pennies to GIMP development (since I use it for my work). I had a look at the proposed plan for GIMP and wondered if people would be interested in
including some popular AI algorithms for several image processing tasks.

We are definitely interested by any AI algorithms. At least I am.

I would be interested in writing implementations of some of these

algorithms into gimp if someone can commit to writing a frontend/GUI for it.

The hard point here is "if someone can commit to […]". It's a bit hard to commit without knowing much (unless you are paid, then you have no choice ;p). Usually it's the other way around: you propose something. It doesn't have to be with a great GUI or whatever. Then if we like what we see, we will definitely add our own 2 cents to the pool. This is usually how most features are done around here, when someone contributes a patch with a very cool idea, then we review and often fix/change what we think is needed (sometimes just a bit, sometimes deeply).

I often wrote crap GUI myself and others came to the rescue with ideas and code. :-)

It would be great if we can make a list of these algorithms to implement and rank them according to priority.

I would suggest to *not do that*. :-) Basically we are not a company, we don't sell GIMP and don't have huge plans for the next decade. Well "officially", we do have a roadmap and such, but if you follow GIMP development, you'd see it is more flexible and experimental than some rigid plan.
Making a huge list with big plans for the future "might" be just the way to spend a lot of time and kill your project in the end.

Instead I would propose **you** just select **one** such algorithm which you find is great and even irrefusable since it would be so fucking awesome and useful! Then you implement and propose it and we will be so amazed that we just have to include it and do a proper GUI for it. That sounds like the best plan.

From there, you can go on with more awesome ideas. :-) You may even be able to start doing more organized work with a list of algorithms after, etc. But for a first patch, I would suggest you just take the lead.

As for my background, I am a theoretical physicist making simulations for HPCs (in C/C++) and interpreting their data (in Python). I have a

reasonable workstation to train neural nets, if necessary. Be warned that I

There is only a single very important part about AI algorithms which need training: we will want the code to train, the data, etc. everything as free software/Open Data and properly documented. I have worked with trained algorithms in Free Software where the trained data is just dropped as-is, and once the original author disappears, this is unmaintainable. In particular it cannot be improved or fixed or nothing, because we don't have the code to re-generate the neural networks (or alike). This can only be a recipe for failure long-term.

So AI in GIMP? Yeah definitely! But it has to be reproducible generated data, with the whole training infrastructure available and properly documented and the input data under Libre license as well.

have never written a GUI software in my life and I don't know the GIMP codebase at all. I envision these to be standalone scripts which can be called in from the GIMP interface.

Cool. Depending on the idea, it may be interesting to rather implement it directly as a GEGL operation (GEGL is our graphics engine). Maybe you don't even need to make a GUI then. Just make a GEGL op, and when it is done, run it with examples to show us how good it is, and we might just get into the game to make it a proper GUI.

Please let me know what you think.

And here you are! I hope we will see soon a lot of baby robots in our code. ;-)

Jehan

Cheerio,
Maitraya.

Senior Research Fellow Center for Excellence in Space Sciences India Indian Institute of Science Education and Research Kolkata _______________________________________________ gimp-developer-list mailing list
List address: gimp-developer-list@gnome.org List membership:
https://mail.gnome.org/mailman/listinfo/gimp-developer-list List archives: https://mail.gnome.org/archives/gimp-developer-list

--
ZeMarmot open animation film
http://film.zemarmot.net
Liberapay: https://liberapay.com/ZeMarmot/ Patreon: https://patreon.com/zemarmot Tipeee: https://www.tipeee.com/zemarmot

Elle Stone
2019-01-21 15:01:15 UTC (2 months ago)

AI algorithms in GIMP

On 1/21/19 8:53 AM, Maitraya Bhattacharyya via gimp-developer-list wrote:

super resolution

I don't know if this is the same "super resolution", but FWIW the topic has come up several times on discuss.pixls.us:

https://discuss.pixls.us/search?q=%20superresolution

Best regards, Elle Stone

https://ninedegreesbelow.com
Color management and free/libre photography
Martin Marmsoler via gimp-developer-list
2019-01-21 15:21:12 UTC (2 months ago)

AI algorithms in GIMP

Hi,

I already tried the first one, but it has not the quality I expected. Did anybody tried the second one? The second one looks quite good :)

[1] https://github.com/alexjc/neural-enhance [2] https://github.com/tensorlayer/srgan

Best regards,

Martin

Am Mo., 21. Jan. 2019 um 16:07 Uhr schrieb Elle Stone < ellestone@ninedegreesbelow.com>:

On 1/21/19 8:53 AM, Maitraya Bhattacharyya via gimp-developer-list wrote:

super resolution

I don't know if this is the same "super resolution", but FWIW the topic has come up several times on discuss.pixls.us:

https://discuss.pixls.us/search?q=%20superresolution

Best regards, Elle Stone
--
https://ninedegreesbelow.com
Color management and free/libre photography _______________________________________________ gimp-developer-list mailing list
List address: gimp-developer-list@gnome.org List membership:
https://mail.gnome.org/mailman/listinfo/gimp-developer-list List archives: https://mail.gnome.org/archives/gimp-developer-list

Maitraya Bhattacharyya via gimp-developer-list
2019-01-21 15:40:50 UTC (2 months ago)

AI algorithms in GIMP

Hi Martin,

Thanks for the second link on Tensorlayer. if the license of the library is compatible with GPL, there is no reason we cannot used their VGG19 network but I don't think it is! I will still need a few weeks to create the image dataset and then train this.

Thanks, Maitraya.

On Mon, Jan 21, 2019 at 8:51 PM Martin Marmsoler wrote:

Hi,

I already tried the first one, but it has not the quality I expected. Did anybody tried the second one? The second one looks quite good :)

[1] https://github.com/alexjc/neural-enhance [2] https://github.com/tensorlayer/srgan

Best regards,

Martin

Am Mo., 21. Jan. 2019 um 16:07 Uhr schrieb Elle Stone < ellestone@ninedegreesbelow.com>:

On 1/21/19 8:53 AM, Maitraya Bhattacharyya via gimp-developer-list wrote:

super resolution

I don't know if this is the same "super resolution", but FWIW the topic has come up several times on discuss.pixls.us:

https://discuss.pixls.us/search?q=%20superresolution

Best regards, Elle Stone
--
https://ninedegreesbelow.com
Color management and free/libre photography _______________________________________________ gimp-developer-list mailing list
List address: gimp-developer-list@gnome.org List membership:
https://mail.gnome.org/mailman/listinfo/gimp-developer-list List archives: https://mail.gnome.org/archives/gimp-developer-list

Elle Stone
2019-01-21 15:42:19 UTC (2 months ago)

AI algorithms in GIMP

On 1/21/19 10:01 AM, Elle Stone wrote:

I don't know if this is the same "super resolution", but FWIW the topic has come up several times on discuss.pixls.us:

https://discuss.pixls.us/search?q=%20superresolution

My apologies, I wasn't very clear. discuss.pixls.us is a discussion forum for users and developers of free/libre software.

Possibly there is already some sort of super-resolution algorithm implementation for one or another free-libre softwares, in which case there is also free/libre code available. But I didn't read any of the links to see what they are actually about.

Pat David (the person who runs the forum) might know whether there's any actual implementation with code that might overlap with/be useful for what Maitraya Bhattacharyya is proposing to do.

https://ninedegreesbelow.com
Color management and free/libre photography
Maitraya Bhattacharyya via gimp-developer-list
2019-01-21 15:47:07 UTC (2 months ago)

AI algorithms in GIMP

Hi Elle,

I looked at the posts on discuss.pixls.us, and it seems that their approach is quite different to what we are trying to achieve. They are mostly trying to use multiple images or use traditional methods which don't yield very good results.

Thanks,
Maitraya.

On Mon, Jan 21, 2019 at 9:12 PM Elle Stone wrote:

On 1/21/19 10:01 AM, Elle Stone wrote:

I don't know if this is the same "super resolution", but FWIW the topic has come up several times on discuss.pixls.us:

https://discuss.pixls.us/search?q=%20superresolution

My apologies, I wasn't very clear. discuss.pixls.us is a discussion forum for users and developers of free/libre software.

Possibly there is already some sort of super-resolution algorithm implementation for one or another free-libre softwares, in which case there is also free/libre code available. But I didn't read any of the links to see what they are actually about.

Pat David (the person who runs the forum) might know whether there's any actual implementation with code that might overlap with/be useful for what Maitraya Bhattacharyya is proposing to do.

-- https://ninedegreesbelow.com
Color management and free/libre photography

Liam R E Quin
2019-01-21 21:39:12 UTC (2 months ago)

AI algorithms in GIMP

On Mon, 2019-01-21 at 19:23 +0530, Maitraya Bhattacharyya via gimp- developer-list wrote:
[...]

Now to train these neural nets, I will have to prepare a dataset consisting
of low resolution and high resolution images and divide them into training,
testing and validation sets.

If it helps, i have high resolution versions of most of the images on https://www.fromoldbooks.org/ - usually i sell them, but i can donate some if would be of use. Most are engravings, up to 20,000 pixels on a side in many cases.

This leads me to wonder whether we could make a “libre” image training set.

It also leads me to wonder whether a neural net could be trained to clean up scanned images.

Liam (slave ankh)

Liam Quin, https://www.delightfulcomputing.com/
Available for XML/Document/Information Architecture/XSLT/
XSL/XQuery/Web/Text Processing/A11Y training, work & consulting.
Barefoot Web-slave for vintage clipart http://www.fromoldbooks.org/
Maitraya Bhattacharyya via gimp-developer-list
2019-02-04 11:50:25 UTC (about 1 month ago)

AI algorithms in GIMP

Hi guys,

I have started working on the super-resolution plugin after going through some tutorials on GANs. I will try to put some kind of update on my blog in a month or so.

Thanks,
Maitraya.

On Mon, Jan 21, 2019 at 9:17 PM Maitraya Bhattacharyya < maitraya.linux@gmail.com> wrote:

Hi Elle,

I looked at the posts on discuss.pixls.us, and it seems that their approach is quite different to what we are trying to achieve. They are mostly trying to use multiple images or use traditional methods which don't yield very good results.

Thanks, Maitraya.

On Mon, Jan 21, 2019 at 9:12 PM Elle Stone wrote:

On 1/21/19 10:01 AM, Elle Stone wrote:

I don't know if this is the same "super resolution", but FWIW the topic has come up several times on discuss.pixls.us:

https://discuss.pixls.us/search?q=%20superresolution

My apologies, I wasn't very clear. discuss.pixls.us is a discussion forum for users and developers of free/libre software.

Possibly there is already some sort of super-resolution algorithm implementation for one or another free-libre softwares, in which case there is also free/libre code available. But I didn't read any of the links to see what they are actually about.

Pat David (the person who runs the forum) might know whether there's any actual implementation with code that might overlap with/be useful for what Maitraya Bhattacharyya is proposing to do.

-- https://ninedegreesbelow.com
Color management and free/libre photography