Introduction to GEGL's Architecture & GPU-support Architecture #2

On Sat, May 9, 2009 at 9:31 PM, utkarsh shukla <utk.shukla@gmail.com> wrote:
> I am not at all sure that CUDA is an option at all or even OpenCL can be
> used. They are a more higher level api based implementations tweeked for
> computing. The thing for GEGL is quite a low level. You have to start from
> the extreme basic and the best way to start is using GLSL (makes more sence
> than HLSL or Cg). CUDA or OpenCL are all together different concepts and
> they dont let the user see any shaders or other implementations. It can
> easily be done using GLSL.

In OpenCL, we can still use image primitives and kernels (shaders).
So I'm not quite sure that OpenCL isn't suitable for our problem. I
haven't really looked into CUDA, so I have no opinion regarding its
suitability.

> I applied for GSOC but had not got it but still I am very very interested it
> this project particular. I just wish to take part in it and just wish if you
> keep on posting on the mailing list.

I agree that this topic is really interesting. I'd really appreciate
help code-wise and architecture-wise. I still have one architecture
article to post, please look forward to it! Also, if you're positive
that you want to help, you can start by hanging around #gegl in IRC.
I'll be there later (not now, 'coz I don't have net connectivity at
home).

Later! :)

Hi all,


It's me again. Over the past weeks, I have been silently digging into
GEGL's source code, trying to understand how everything works. This,
while researching about OpenGL and GObjects. Though I am quite
familiar with C, I didn't know how GObject layers on top of C to
provide OOP features. To make things worse, my previous experience
with OpenGL is very minimal. When I say 'experience', I meant that I
only knew OpenGL through playing games. And playing games doesn't
really help with programming, right? :) Let's just hope that I
resist the urge to play during the whole of GSoC's duration. :p

Of course, I disclaim all views that I am acting like a GEGL expert
given that I am now talking about GEGL's architecture. All that is
written in these articles reflect what I have read so far. If you
have anything to correct, please do. Even if I have all the time in
the world to read, I still am human. So pardon the mistakes and keep
the criticisms constructive. :)

>From the GEGL website[1]: GEGL is a graph-based image processing
framework. Okay. Now what does graph-based mean and how does it
relate to graphics manipulation? I have explained in the previous
article that GEGL uses operations to transform pixel values. In GEGL,
an operation is attached to a GEGL node.

Nodes are connected to other nodes through edges (represented by GEGL
pads[2]) that either come-in or come-out from the current node. Pixel
data come in through the node's input pads and come out through the
same node's output pads. Pixel data must first be processed by a
node's operation before it becomes available in the output pad. And
pixel data must first be available from all the node's input pads
before the same node can manipulate the pixels. A node's input pads
are connected to other nodes' output pads and vice versa. All you
ever need is to imagine and you'll see that this forms a graph of
nodes that inter-depend on and from one another. Perhaps, it should
help your imagination if you also know that there are source nodes and
sink nodes that do nothing but provide and accept pixel data,
respectively. :p Also, note that nodes may not be configured such
that a cycle is introduced. This is why GEGL is also DAG (Directed
Acyclic Graph) based.

Each GEGL node makes use of GEGL buffers[3] to cache image data from
the previous nodes in the composition. It is an instance of this
buffer that the node modifies with its operation[4]. The node
provides the relevant pixels to its operation by using a buffer
iterator that iterates over chunks of data within the input and output
buffers.

The second architecture is a scheme to let our GEGL nodes take care of
the image transfer from main memory to GPU memory (image to OpenGL
texture). The operations will implement a new processor function[4]
that will expect texture parameters rather than pixel data arrays[5].
GEGL will decide to transfer the image to GPU memory and use the GPU
processor function depending on whether or not OpenGL is supported by
the current operating system GEGL is running.

Image transfer from main memory to GPU memory will be accomplished by
implementing a buffer iterator that will return textures instead of
pixel arrays. These GPU buffer iterators will keep a pool of textures
that will be reused as textures are requested and committed to the
buffers. As with the GPU processor function, a GPU buffer iterator
will only be used when OpenGL is actually supported by the current
operating system and that the current operation has GPU-support.

The advantage over the previous architecture is that GEGL is now able
to take full control over the OpenGL state machine[6]. We are now
able to initialize and destroy OpenGL contexts through gegl_init and
gegl_exit, respectively. This will remove unnecessary code
repetition. Moreover, OpenGL context initialization is CPU expensive.

Though we now have a pretty complex architecture, we still aren't
making full use of the GPU. Why? You'll see why when I explain GEGL
buffers. That's next. Also in the next installment, We will look at
ways of integrating GPU-support into GEGL buffers to really extract
the GPU's (evil) power.

Until next time! :)


Kind regards,
Daerd

P.S. Sorry for the really lengthy explanation. I know that pretty
much most of you know about these. These are just notes to explain
things better. Or just consider this to be representative of what I
currently know. (Really, I'm just making excuses. Hehe.. :) Please
correct me where I am wrong.


[1] http://www.gegl.org/

[2] From the GEGL website, a pad is the part of a node that exchanges
image content. The place where image "pipes" are used to connect the
various operations in the composition.

[3] Look forward to the next article, as I explain GEGL buffer's architecture.

[4] Not to be confused with GEGL processors. GEGL uses processor
functions to implement alternative operation executions. For example,
when a machine supports SIMD CPU extensions, GEGL will use the
processor function for SIMD when it is available from the current
operation.

[5] Though I really don't know if it's possible to create a processor
function whose signature is slightly different from the default
processor function. I need your wisdom here, GEGL gods. ;)

[6] I didn't tell you that OpenGL is a state machine. But, really, it
is. See OpenGL's Wikipedia entry, if you're interested.

Hello,

Not being a GEGL or OpenGL developer, I would nevertheless being interested
in how acceleration of filters etc. can basically work with OpenGL (with
matrix operations on buffers? Can you do other operations, too?) because I
always thought OpenGL was an API for accelerated (3D but also 2D, considering
it as a plane in 3D space) rendering, but not for calculating (like it is
required with many filters). Wouldn't it, from a technical view, be more
useful to implement CUDA (http://www.nvidia.com/cuda/) support? I heard that
there is a similar API for ATI. I also know that there is a GIMP plugin using
CUDA (it has won the first place of a CUDA challenge:
http://www.nvidia.co.uk/object/io_1222782056939.html).

As far as I have seen taking a quick look at the GEGL source code, the
hardest task seems to be changing the single-threaded algorithm
while (more_work) render();
to
for (i = 0; i < work_packets; i++)
create_thread(render_packets);
where the packets are rectangles.

This would introduce a parallelism which would allow to
1) use all CPUs (would be very useful too, especially if you don't have 3D
graphics acceleration available, but more than 1 CPU)
2) use CUDA to process the threads directly using GPU shaders (in parallel).

However, I have to say that I have no GEGL experience and little OpenGL and
CUDA experience, so maybe I have some wrong assumptions, so feel free to
correct me if I just wrote nonsense.

On Fri, May 8, 2009 at 4:21 PM, Richard H. <forums@gimpusers.com> wrote:
> Hello,
>
> Not being a GEGL or OpenGL developer, I would nevertheless being interested
> in how acceleration of filters etc. can basically work with OpenGL (with
> matrix operations on buffers? Can you do other operations, too?) because I
> always thought OpenGL was an API for accelerated (3D but also 2D, considering
> it as a plane in 3D space) rendering, but not for calculating (like it is
> required with many filters). Wouldn't it, from a technical view, be more
> useful to implement CUDA (http://www.nvidia.com/cuda/) support? I heard that
> there is a similar API for ATI. I also know that there is a GIMP plugin using
> CUDA (it has won the first place of a CUDA challenge:
> http://www.nvidia.co.uk/object/io_1222782056939.html).

As far as I am concerned, CUDA and ATI's Stream SDK aren't an option
because they are video card specific. I haven't really looked into
them but I'm sure they are easy to use for computations, but the same
computations can be accomplished through OpenGL through multiple
rendering/computation passes. Moreover, we decided to go with OpenGL
instead of a cross-platform API like OpenCL because implementations
for the latter are still either very young or non-existent for most
major platforms[1].

> As far as I have seen taking a quick look at the GEGL source code, the
> hardest task seems to be changing the single-threaded algorithm
>  while (more_work) render();
> to
>  for (i = 0; i < work_packets; i++)
>    create_thread(render_packets);
> where the packets are rectangles.
>
> This would introduce a parallelism which would allow to
> 1) use all CPUs (would be very useful too, especially if you don't have 3D
> graphics acceleration available, but more than 1 CPU)
> 2) use CUDA to process the threads directly using GPU shaders (in parallel).

Sorry, but I'm not really familiar with CUDA's threading model.
Furthermore, GEGL doesn't really take care of thread creation and
management. As far as I can see, your code's intent is to divide the
tasks into smaller packets of rectangle. GEGL already has GEGL
processors that can be used to accomplish this[2]. In fact,
internally, GEGL uses tiles to divide the image into smaller packets
for on demand rendering and caching.

[1] Mac OS X is slated to include OpenCL in Snow Leopard. I suspect
OpenCL in Windows will be there as soon as drivers support the
standard. For GNU/Linux, on the other hand, support will come in the
form of a Gallium3D state tracker. See this article on Phoronix for
more info: http://www.phoronix.com/scan.php?page=news_item&px=NzAzMw.

[2] See http://gegl.org/api.html#GeglProcessor for more info.

Also, please see http://gpgpu.org/ for more info regarding doing
computations using OpenGL.

Hello,

Thanks for your answer.

>As far as I am concerned, CUDA and ATI's Stream SDK aren't an option
>because they are video card specific. I haven't really looked into
>them but I'm sure they are easy to use for computations, but the same
>computations can be accomplished through OpenGL through multiple
>rendering/computation passes. Moreover, we decided to go with OpenGL
>instead of a cross-platform API like OpenCL because implementations
>for the latter are still either very young or non-existent for most
>major platforms[1].

Yes, I hope that all needed operations can be done via OpenGL, too because I
have already experienced that CUDA sometimes makes problems even on supported
platforms if you don't have exactly the same compiler options as the samples
etc. On the other side, it's an additional unnecessary layer because the
OpenGL computations use the same things you can directly access via CUDA or
equivalent SDKs (if I have interpreted the infos I read about CUDA correctly).
This layer is optimized for rendering and not for computing, so - from a pure
simplicistic view - CUDA would be easier and more performant than OpenGL.

I have never heard of OpenCL, but it seems to be very interesting (and not
proprietary). I hope that it will be available for the most important
platforms, soon.

>Sorry, but I'm not really familiar with CUDA's threading model.
Basically it's just the same as normal CPU threads, which would IMO be
important to support, too. Many (especially Linux) systems don't have graphics
acceleration available, but more than one CPU. Today you get already 8 cores,
i.e. filters would be ~8 times faster if they utilised all CPUs (for at least
8 available rectangles).

>Furthermore, GEGL doesn't really take care of thread creation and
>management.

I have seen this. But isn't that bad? My opionion is that it should.

> As far as I can see, your code's intent is to divide the
>tasks into smaller packets of rectangle. GEGL already has GEGL
>processors that can be used to accomplish this[2]. In fact,
>internally, GEGL uses tiles to divide the image into smaller packets
>for on demand rendering and caching.

Yes I have found the relevant code sections in the GEGL source (at least I
think that I have ;) ), and they seem to have the linear structure I have
mentioned in my first email. I mean that gegl_processor_work() returns a
boolean telling you if there is some more work to do, in which case you have
to call it again. I think this is not usable for parallelism, because you
always have to wait until the current packet (processed by
gegl_processor_work()) is finished until you know if you have to call it
again. For parallelism, a better approach would be that gegl_process_work()
does everything at once, for instance by moving the outer while loop over the
rectangles into in inner for loop.

I just wonder if that would help for the pure OpenGL approach, too. But I
think that processing with OpenGL _and_ multithreading would make the best of
pure OpenGL acceleration as well.

I will have a look at the articles on gpgpu.org, too, seems to be a very
interesting site, thanks.

I am not at all sure that CUDA is an option at all or even OpenCL can be
used. They are a more higher level api based implementations tweeked for
computing. The thing for GEGL is quite a low level. You have to start from
the extreme basic and the best way to start is using GLSL (makes more sence
than HLSL or Cg). CUDA or OpenCL are all together different concepts and
they dont let the user see any shaders or other implementations. It can
easily be done using GLSL.

I applied for GSOC but had not got it but still I am very very interested it
this project particular. I just wish to take part in it and just wish if you
keep on posting on the mailing list.

On Sat, May 9, 2009 at 6:05 PM, Richard H. <forums@gimpusers.com> wrote:

> Hello,
>
> Thanks for your answer.
>
> >As far as I am concerned, CUDA and ATI's Stream SDK aren't an option
> >because they are video card specific. I haven't really looked into
> >them but I'm sure they are easy to use for computations, but the same
> >computations can be accomplished through OpenGL through multiple
> >rendering/computation passes. Moreover, we decided to go with OpenGL
> >instead of a cross-platform API like OpenCL because implementations
> >for the latter are still either very young or non-existent for most
> >major platforms[1].
>
> Yes, I hope that all needed operations can be done via OpenGL, too because
> I
> have already experienced that CUDA sometimes makes problems even on
> supported
> platforms if you don't have exactly the same compiler options as the
> samples
> etc. On the other side, it's an additional unnecessary layer because the
> OpenGL computations use the same things you can directly access via CUDA or
> equivalent SDKs (if I have interpreted the infos I read about CUDA
> correctly).
> This layer is optimized for rendering and not for computing, so - from a
> pure
> simplicistic view - CUDA would be easier and more performant than OpenGL.
>
> I have never heard of OpenCL, but it seems to be very interesting (and not
> proprietary). I hope that it will be available for the most important
> platforms, soon.
>
> >Sorry, but I'm not really familiar with CUDA's threading model.
> Basically it's just the same as normal CPU threads, which would IMO be
> important to support, too. Many (especially Linux) systems don't have
> graphics
> acceleration available, but more than one CPU. Today you get already 8
> cores,
> i.e. filters would be ~8 times faster if they utilised all CPUs (for at
> least
> 8 available rectangles).
>
> >Furthermore, GEGL doesn't really take care of thread creation and
> >management.
>
> I have seen this. But isn't that bad? My opionion is that it should.
>
> > As far as I can see, your code's intent is to divide the
> >tasks into smaller packets of rectangle. GEGL already has GEGL
> >processors that can be used to accomplish this[2]. In fact,
> >internally, GEGL uses tiles to divide the image into smaller packets
> >for on demand rendering and caching.
>
> Yes I have found the relevant code sections in the GEGL source (at least I
> think that I have ;) ), and they seem to have the linear structure I have
> mentioned in my first email. I mean that gegl_processor_work() returns a
> boolean telling you if there is some more work to do, in which case you
> have
> to call it again. I think this is not usable for parallelism, because you
> always have to wait until the current packet (processed by
> gegl_processor_work()) is finished until you know if you have to call it
> again. For parallelism, a better approach would be that gegl_process_work()
> does everything at once, for instance by moving the outer while loop over
> the
> rectangles into in inner for loop.
>
> I just wonder if that would help for the pure OpenGL approach, too. But I
> think that processing with OpenGL _and_ multithreading would make the best
> of
> pure OpenGL acceleration as well.
>
> I will have a look at the articles on gpgpu.org, too, seems to be a very
> interesting site, thanks.
>
> --
> Richard H. (via www.gimpusers.com)
> _______________________________________________
> Gegl-developer mailing list
> Gegl-developer@lists.XCF.Berkeley.EDU
> https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
>

> Thanks for your answer.

No problem. :)

> Yes, I hope that all needed operations can be done via OpenGL, too because I
> have already experienced that CUDA sometimes makes problems even on supported
> platforms if you don't have exactly the same compiler options as the samples
> etc. On the other side, it's an additional unnecessary layer because the
> OpenGL computations use the same things you can directly access via CUDA or
> equivalent SDKs (if I have interpreted the infos I read about CUDA correctly).
> This layer is optimized for rendering and not for computing, so - from a pure
> simplicistic view - CUDA would be easier and more performant than OpenGL.
>
> I have never heard of OpenCL, but it seems to be very interesting (and not
> proprietary). I hope that it will be available for the most important
> platforms, soon.

Really, my only concern with CUDA (and Stream SDK) is that they are
too vendor-specific (I'd loved to be proven wrong though). So,
implementing GPU-support through them is a no-go, afaic. I'm not
willing to implement two GPU-enabled GEGL branches, one for NVIDIA and
one for ATI video cards. That would be too much hassle and would not
fit sensibly into GSoC's timeline of 3 months. I am willing to
develop for GIMP and GEGL after GSoC but I would prefer my targets
realistic. I really do agree that coding against CUDA and similar
APIs is easier than coding against OpenGL, I'm just not convinced that
the latter is an 'unnecessary' investment given our particular case.
The only API that I would really approve of is OpenCL, but as I have
previously said, OpenCL implementations are still in their infancy.

Nevertheless, thank you for your concern. We are of the same spirit.
I just hope that we have a mature, non-proprietary, vendor-neutral,
platform-agnostic solution here and now. But I'm forced to use OpenGL
as it is what is currently available, imho. But rest assured, I will
commit myself into porting my OpenGL implementation into OpenCL as
soon as implementations for the latter becomes available. I also hope
that I'd have help by that time. :)

> Basically it's just the same as normal CPU threads, which would IMO be
> important to support, too. Many (especially Linux) systems don't have graphics
> acceleration available, but more than one CPU. Today you get already 8 cores,
> i.e. filters would be ~8 times faster if they utilised all CPUs (for at least
> 8 available rectangles).

Implementing multi-threading in GEGL is out of my scope and I'm not
even sure if it's in GEGL's scope. GEGL is pretty low-level and
threading can be implemented on top of GEGL by the relevant client
(i.e. GIMP). Furthermore, be aware that threading doesn't really map
into multiple cores and not using threads doesn't really mean that the
code will not be parallelized[1].

Currently, the changes that I propose to introduce GPU-support to GEGL
will be added to GEGL's existing code paths so that when OpenGL and/or
hardware acceleration isn't available, we will be able to fallback to
plain software rendering (using the CPU).

> Yes I have found the relevant code sections in the GEGL source (at least I
> think that I have ;) ), and they seem to have the linear structure I have
> mentioned in my first email. I mean that gegl_processor_work() returns a
> boolean telling you if there is some more work to do, in which case you have
> to call it again. I think this is not usable for parallelism, because you
> always have to wait until the current packet (processed by
> gegl_processor_work()) is finished until you know if you have to call it
> again. For parallelism, a better approach would be that gegl_process_work()
> does everything at once, for instance by moving the outer while loop over the
> rectangles into in inner for loop.

I'm not really an expert with regards to how GEGL uses threading
frameworks (if at all) to parallelize tile/rectangle-level operations.
My work will be concerned with accelerating pixel-level operations by
parallelizing each pixel operation in the GPU. The key in my work is
that all pixel operations should (theoretically) work in parallel.
I'm not in the position to think about parallelizing tiles/rectangles
though I suspect that parallelization in that area is limited because
of GEGL's complex graph dependencies. I'd appreciate it if a GEGL
expert steps up and clarify these issues.

[1] GCC, for example, will later provide automatic parallelization
through the Graphite framework. Please see:
http://blog.internetnews.com/skerner/2009/04/gcc-44-improves-open-source-co.html
and http://www.phoronix.com/scan.php?page=news_item&px=NzIyMQ.

I believe you are much off the track and you havent understood what you are
supossed to do. It is not that we are forced to use OpenGl but it is that
OpenGl is the best option. What you are telling is that OpenCl or Cuda can
be used but that is purely wrong. Why you have to complicate procedures by
using the OpenCL abstraction layer, of which there is no need as the things
can be easily done using OpenGl. Infact I am sure you havent understood the
whole of the problem at all. You feel for the support of the GPU since there
is an API called OpenCL exists so you can user it. But it is not that way.
You dont understand what OpenCL or CUDA is and I am sure you havent worked
much upon it. Its a compute platform it provides API lot different that what
is needed by GEGL. Infact as I previously said it is no way an option. I am
sure you misunderstood the thing from the basics. You can create a minimap
and use it to distribute the workload of the threads to either CPU using
software rendering or Gpu using Hardware rendering or even to a network or
something. I dont know why you are running out of the track.

Since you are Officially accepted for this project and I was Officially
rejected so I expected you might had a well plan but you idea of using
OpenCL or CUDA seems to me extremely Illogical. It would just not cause the
increase the speed as can be got with basic implementation. This project is
not a big and difficult task.


> Really, my only concern with CUDA (and Stream SDK) is that they are
> too vendor-specific (I'd loved to be proven wrong though). So,
> implementing GPU-support through them is a no-go, afaic. I'm not
> willing to implement two GPU-enabled GEGL branches, one for NVIDIA and
> one for ATI video cards. That would be too much hassle and would not
> fit sensibly into GSoC's timeline of 3 months. I am willing to
> develop for GIMP and GEGL after GSoC but I would prefer my targets
> realistic. I really do agree that coding against CUDA and similar
> APIs is easier than coding against OpenGL, I'm just not convinced that
> the latter is an 'unnecessary' investment given our particular case.
> The only API that I would really approve of is OpenCL, but as I have
> previously said, OpenCL implementations are still in their infancy.

I really really feel that have you never been exposed to shaders or ever
used GLSL or HLSL , because what I feel you dont have the understanding of
the topic and it would really be too difficult for you to complete this.
I am working upon GPU for last 2.5 years and upon things like cloud
computing since last year.



On Sun, May 10, 2009 at 4:32 PM, Jerson Michael Perpetua <
jersonperpetua@gmail.com> wrote:

> > Thanks for your answer.
>
> No problem. :)
>
> > Yes, I hope that all needed operations can be done via OpenGL, too
> because I
> > have already experienced that CUDA sometimes makes problems even on
> supported
> > platforms if you don't have exactly the same compiler options as the
> samples
> > etc. On the other side, it's an additional unnecessary layer because the
> > OpenGL computations use the same things you can directly access via CUDA
> or
> > equivalent SDKs (if I have interpreted the infos I read about CUDA
> correctly).
> > This layer is optimized for rendering and not for computing, so - from a
> pure
> > simplicistic view - CUDA would be easier and more performant than OpenGL.
> >
> > I have never heard of OpenCL, but it seems to be very interesting (and
> not
> > proprietary). I hope that it will be available for the most important
> > platforms, soon.
>
> Really, my only concern with CUDA (and Stream SDK) is that they are
> too vendor-specific (I'd loved to be proven wrong though). So,
> implementing GPU-support through them is a no-go, afaic. I'm not
> willing to implement two GPU-enabled GEGL branches, one for NVIDIA and
> one for ATI video cards. That would be too much hassle and would not
> fit sensibly into GSoC's timeline of 3 months. I am willing to
> develop for GIMP and GEGL after GSoC but I would prefer my targets
> realistic. I really do agree that coding against CUDA and similar
> APIs is easier than coding against OpenGL, I'm just not convinced that
> the latter is an 'unnecessary' investment given our particular case.
> The only API that I would really approve of is OpenCL, but as I have
> previously said, OpenCL implementations are still in their infancy.
>
>
> Nevertheless, thank you for your concern. We are of the same spirit.
> I just hope that we have a mature, non-proprietary, vendor-neutral,
> platform-agnostic solution here and now. But I'm forced to use OpenGL
> as it is what is currently available, imho. But rest assured, I will
> commit myself into porting my OpenGL implementation into OpenCL as
> soon as implementations for the latter becomes available. I also hope
> that I'd have help by that time. :)
>
> > Basically it's just the same as normal CPU threads, which would IMO be
> > important to support, too. Many (especially Linux) systems don't have
> graphics
> > acceleration available, but more than one CPU. Today you get already 8
> cores,
> > i.e. filters would be ~8 times faster if they utilised all CPUs (for at
> least
> > 8 available rectangles).
>
> Implementing multi-threading in GEGL is out of my scope and I'm not
> even sure if it's in GEGL's scope. GEGL is pretty low-level and
> threading can be implemented on top of GEGL by the relevant client
> (i.e. GIMP). Furthermore, be aware that threading doesn't really map
> into multiple cores and not using threads doesn't really mean that the
> code will not be parallelized[1].
>
> Currently, the changes that I propose to introduce GPU-support to GEGL
> will be added to GEGL's existing code paths so that when OpenGL and/or
> hardware acceleration isn't available, we will be able to fallback to
> plain software rendering (using the CPU).
>
> > Yes I have found the relevant code sections in the GEGL source (at least
> I
> > think that I have ;) ), and they seem to have the linear structure I have
> > mentioned in my first email. I mean that gegl_processor_work() returns a
> > boolean telling you if there is some more work to do, in which case you
> have
> > to call it again. I think this is not usable for parallelism, because you
> > always have to wait until the current packet (processed by
> > gegl_processor_work()) is finished until you know if you have to call it
> > again. For parallelism, a better approach would be that
> gegl_process_work()
> > does everything at once, for instance by moving the outer while loop over
> the
> > rectangles into in inner for loop.
>
> I'm not really an expert with regards to how GEGL uses threading
> frameworks (if at all) to parallelize tile/rectangle-level operations.
> My work will be concerned with accelerating pixel-level operations by
> parallelizing each pixel operation in the GPU. The key in my work is
> that all pixel operations should (theoretically) work in parallel.
> I'm not in the position to think about parallelizing tiles/rectangles
> though I suspect that parallelization in that area is limited because
> of GEGL's complex graph dependencies. I'd appreciate it if a GEGL
> expert steps up and clarify these issues.
>
> [1] GCC, for example, will later provide automatic parallelization
> through the Graphite framework. Please see:
>
> http://blog.internetnews.com/skerner/2009/04/gcc-44-improves-open-source-co.html
> and http://www.phoronix.com/scan.php?page=news_item&px=NzIyMQ.
> _______________________________________________
> Gegl-developer mailing list
> Gegl-developer@lists.XCF.Berkeley.EDU
> https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
>

Hello,


Sorry that you feel that way. I did tell the GIMP and GEGL
developer's during proposal period that I had no previous experience
with OpenGL. Moreover, you can read from my mailing list posts that I
admit to being a beginner in OpenGL, GObjects and GEGL. I can't
really speculate why they accepted me against someone experienced like
you. But one thing's for sure, I can and am willing to learn.

As to me being off-track, maybe we are approaching the same problem in
different perspectives? What I've written in my GSoC proposal is a
very murky way of parallelizing pixel operations by loading buffer
image data into OpenGL textures and using those textures inside
shaders. It's now extended into off-loading pixel data copy
operations and conversions to the GPU, so that the CPU can save
several cycles to do other tasks. In my view, OpenCL and similar APIs
aren't very different with respect to how I am envisioning to use
OpenGL for GEGL. I don't see why I can't use those so called
computing languages. Sure, I'm not positive that all operations could
be rewritten into shaders. What I'm sure is that the very basic of
them I could implement in time for GSoC.

I'm curious about your mention of minimaps though. I haven't really
heard about them being used in GPGPU techniques. The only thing I
know about minimaps is that they are "miniature map[s], often placed
in a corner of the screen in computer games to aid in reorientation"
[from Wikipedia]. Maybe you can shed some light into this dark mind?


Respectfully yours,
Daerd

Sorry from my side if you felt something. I did not mean to hurt you.
Obviously you must be better than me.
Well by the minimap I mean you create a mapping of what part of buffer data
is where, wether it is processed in CPU or in GPU or over the network. By
this way You reduce the tranfer of data. Basically in GPU the costliest
thing is the Data transfer than the processing.
I dont feel such thing is used frequently but if you want to make a unified
memory architechture then it would be much better. This thing will easily
fit into the GEGL api.
I dint said OpenCL can not be used I just said OpenGl can do all the things
needed and OpenCL or CUDA although can do this but they would be extra
burden. OpenCl and CUDA use the Unified Compute architechture but I feel
Gimp has much more worries of Data Transfer than the computation.
CUDA or OpenCL is used best in places where large computation and less
transfer is needed. I hope you understand. By this way of mapping you not
only save the Data Tranfer but the architechture is much more usable even of
you try Cloud Computing.


On Mon, May 11, 2009 at 4:03 PM, Jerson Michael Perpetua <
jersonperpetua@gmail.com> wrote:

> Hello,
>
>
> Sorry that you feel that way. I did tell the GIMP and GEGL
> developer's during proposal period that I had no previous experience
> with OpenGL. Moreover, you can read from my mailing list posts that I
> admit to being a beginner in OpenGL, GObjects and GEGL. I can't
> really speculate why they accepted me against someone experienced like
> you. But one thing's for sure, I can and am willing to learn.
>
> As to me being off-track, maybe we are approaching the same problem in
> different perspectives? What I've written in my GSoC proposal is a
> very murky way of parallelizing pixel operations by loading buffer
> image data into OpenGL textures and using those textures inside
> shaders. It's now extended into off-loading pixel data copy
> operations and conversions to the GPU, so that the CPU can save
> several cycles to do other tasks. In my view, OpenCL and similar APIs
> aren't very different with respect to how I am envisioning to use
> OpenGL for GEGL. I don't see why I can't use those so called
> computing languages. Sure, I'm not positive that all operations could
> be rewritten into shaders. What I'm sure is that the very basic of
> them I could implement in time for GSoC.
>
> I'm curious about your mention of minimaps though. I haven't really
> heard about them being used in GPGPU techniques. The only thing I
> know about minimaps is that they are "miniature map[s], often placed
> in a corner of the screen in computer games to aid in reorientation"
> [from Wikipedia]. Maybe you can shed some light into this dark mind?
>
>
> Respectfully yours,
> Daerd
>

On Tue, May 12, 2009 at 2:22 AM, utkarsh shukla <utk.shukla@gmail.com> wrote:
> Sorry from my side if you felt something. I did not mean to hurt you.
> Obviously you must be better than me.

It's okay, no harm taken. :)

> Well by the minimap I mean you create a mapping of what part of buffer data
> is where, wether it is processed in CPU or in GPU or over the  network. By
> this way You reduce the tranfer of data. Basically in GPU the costliest
> thing is the Data transfer than the processing.
> I dont feel such thing is used frequently but if you want to make a unified
> memory architechture then it would be much better. This thing will easily
> fit into the GEGL api.

I see. This minimap thing greatly interests me. I have been
searching through Google for anything related to the minimaps you were
talking about, but to no avail. All the results seem to refer to game
minimaps only. Do you have, by any chance, some helpful references
for them?

Actually, it's kinda funny that this appeared in our conversation.
Richard T. and I, in a different conversation, were just talking about
tile/rectangle-level parallelization. Actually, as it currently
stands, GEGL doesn't seem to support concurrent operations on
different tiles/rectangles. Reading through the source, especially
through GeglBuffer's source, I was made to assume that the buffer
doesn't really need to execute different operations on
tiles/rectangles in different devices (e.g. GPUs, CPUs, etc.) because
support for such isn't really there in the sources yet.

What I had in mind for GPU-support was to store all cached tiles in
GPU memory, relying in the cache to actually minimize CPU to GPU
transfers (and vice versa). I wasn't able to think of a setup wherein
different tiles could be stored in and processed by different devices.
Perhaps, I could better explain this if you understand GEGL's buffer
architecture. I'm going to post another article for that, but had
originally planned to include some of my proposed changes for
GPU-support. I'm going to post the article anyway without my proposed
changes. I think it might help.

Setting all that aside, I still don't know if introducing
tile/rectangle-level parallelization is something that I could take
on. Minimaps are indeed interesting, If my basic understanding of
them is correct. But I would have to set that aside as of now..
Unless someone steps-up to implement them? :) Then maybe we could
implement both tile/rectangle-level and pixel-level parallelization
all at the same time... Sweetness. :)


Kind regards,
Daerd

Hello,

>Implementing multi-threading in GEGL is out of my scope and I'm not
>even sure if it's in GEGL's scope.

I understand that as your project's task is adding OpenGL support. In my
opinion, multi-CPU support would be more important, but that is of course
another big task/project.

> GEGL is pretty low-level and
>threading can be implemented on top of GEGL by the relevant client
>(i.e. GIMP).

I don't understand this. How should GIMP provide threading support for GEGL
processors? I mean, if you for instance scale a picture using GEGL, it is one
GEGL operation. How can GIMP create multiple threads for this?

> Furthermore, be aware that threading doesn't really map
>into multiple cores and not using threads doesn't really mean that the
>code will not be parallelized[1].

I had a look at Graphite/autopar and can say: Wow! Splitting loops to
multiple threads automatically
[http://gcc.gnu.org/ml/gcc/2009-03/msg00239.html] is a good thing, but I think
it's not true that threads don't map into multiple cores automatically. That's
what threads (or processes, when threads are processes with same address
space) are for.

>I'm not really an expert with regards to how GEGL uses threading
>frameworks (if at all) to parallelize tile/rectangle-level operations.

I think it doesn't use threads at all and is designed for single-threaded use
only... :/

> My work will be concerned with accelerating pixel-level operations by
>parallelizing each pixel operation in the GPU. The key in my work is
>that all pixel operations should (theoretically) work in parallel.

That would indeed be very nice to have.

> I don't understand this. How should GIMP provide threading support for GEGL
> processors? I mean, if you for instance scale a picture using GEGL, it is one
> GEGL operation. How can GIMP create multiple threads for this?

Yes, you are right. Probably just my confused mind, but I was
actually thinking about setting-up different threads for different
graphs. Sorry about that. :)

Thanks for the patience.

Be aware that there is another conversation I'm having in this same
list that has something to do with tile/rectangle-level
parallelization (or so I think). You might be interested to look into
utkarsh's messages. He seem to be an expert in this.


Kind regards,
Daerd

On Mon, May 11, 2009 at 7:49 PM, Richard H. <forums@gimpusers.com> wrote:
> Hello,
>
>>Implementing multi-threading in GEGL is out of my scope and I'm not
>>even sure if it's in GEGL's scope.
>
> I understand that as your project's task is adding OpenGL support. In my
> opinion, multi-CPU support would be more important, but that is of course
> another big task/project.

GEGL has been designed to have an API that will allows using it
without concern of whether it internally is single threaded (like its
current incarnation), threaded, multi-process, multi-host or
multi-process with parallel tasks driving GeglProcessors for different
subregions, or perhaps even with the workload divided in a stream
processing like manner.

>> GEGL is pretty low-level and
>>threading can be implemented on top of GEGL by the relevant client
>>(i.e. GIMP).

Nope,. see above.

>> My work will be concerned with accelerating pixel-level operations by
>>parallelizing each pixel operation in the GPU.  The key in my work is
>>that all pixel operations should (theoretically) work in parallel.
>
> That would indeed be very nice to have.

A GPU based backend should be possible to integrate with some of the
above outlined possible approaches of dividing labour, ideally it
would be possible to use GEGL and let GEGL itself figured out how to
best make use computational resources available.

/Øyvind K.