API change for samplers to pass inverse Jacobian data to samplers

On Mon, Jun 29, 2009 at 4:47 PM, Martin Nordholts<enselic@gmail.com> wrote:
> It'll be fine to put it in gegl-matrix.c

I have added the typedef, but now I have the issue of adding this
GeglMatrix2 as a property of the sampler. I notice that there are
g_object_class_install_property ( ) lines that use
g_param_spec_pointer () and g_param_spec_object () to specify a
GParamSpec.

What is the correct method call for a GeglMatrix2?



Adam Turcotte

Hi,

On Thu, 2009-07-02 at 16:27 -0400, Adam Turcotte wrote:

> I have added the typedef, but now I have the issue of adding this
> GeglMatrix2 as a property of the sampler. I notice that there are
> g_object_class_install_property ( ) lines that use
> g_param_spec_pointer () and g_param_spec_object () to specify a
> GParamSpec.
>
> What is the correct method call for a GeglMatrix2?

You should make GeglMatrix2 a boxed type and use g_param_spec_boxed()
for it. gegl-utils.c has code that does this for GeglRectangle.


Sven

On 07/02/2009 10:34 PM, Sven Neumann wrote:
> Hi,
>
> On Thu, 2009-07-02 at 16:27 -0400, Adam Turcotte wrote:
>
>
>> I have added the typedef, but now I have the issue of adding this
>> GeglMatrix2 as a property of the sampler. I notice that there are
>> g_object_class_install_property ( ) lines that use
>> g_param_spec_pointer () and g_param_spec_object () to specify a
>> GParamSpec.
>>
>> What is the correct method call for a GeglMatrix2?
>>
>
> You should make GeglMatrix2 a boxed type and use g_param_spec_boxed()
> for it. gegl-utils.c has code that does this for GeglRectangle.
>

Or he can just do as GeglProcessor does for its GeglRectangle property
and use g_param_spec_pointer() and copy the data on a set/get.

/ Martin

Hi,

On Thu, 2009-07-02 at 22:43 +0200, Martin Nordholts wrote:

> > You should make GeglMatrix2 a boxed type and use g_param_spec_boxed()
> > for it. gegl-utils.c has code that does this for GeglRectangle.
> >
>
> Or he can just do as GeglProcessor does for its GeglRectangle property
> and use g_param_spec_pointer() and copy the data on a set/get.

Better fix GeglProcessor then and make it use the boxed GeglRectangle
type properly.


Sven

On Thu, Jul 2, 2009 at 4:34 PM, Sven Neumann<sven@gimp.org> wrote:
> You should make GeglMatrix2 a boxed type and use g_param_spec_boxed()
> for it. gegl-utils.c has code that does this for GeglRectangle.

Should the GType be declared inside gegl-matrix.h?

Should the required methods for this new Boxed Type be included in gegl-utils.c?

Thanks!




Adam Turcotte

Hi,

On Thu, 2009-07-02 at 22:14 -0400, Adam Turcotte wrote:
> On Thu, Jul 2, 2009 at 4:34 PM, Sven Neumann<sven@gimp.org> wrote:
> > You should make GeglMatrix2 a boxed type and use g_param_spec_boxed()
> > for it. gegl-utils.c has code that does this for GeglRectangle.
>
> Should the GType be declared inside gegl-matrix.h?
>
> Should the required methods for this new Boxed Type be included in gegl-utils.c?

Keep them all in a single place. The GeglRectangle code should actually
be moved out of gegl-utils.[ch] to new files gegl-rectangle.[ch].


Sven

In working on an implementation of Exact Area Box Filtering (modified
so it does well both upsampling, downsampling, and actually, for
arbitrary warps), which is particularly well-suited for downsampling,
I have determined that the sampler should receive the inverse of the
Jacobian matrix from the (currently, affine) transformation that is
calling it. This inverse need not be exact--it can be an
approximation, though the more accurate the approximation, the better
the results. The motivation for this modification is that future
developments such as warps, perspective, and other transformations of
the same type will also be able to rely on the new sampler and use it
to maximal effect.

I propose modifying each affine transformation's create_matrix ()
method to generate both the Jacobian and (an approximation of) the
inverse Jacobian. In addition, I will need to expand the OpAffine
struct to include an additional GeglMatrix3 (call it matrix_inverse or
inverse_jacobian?) to store this data.

Now, I will also require passing this new matrix to the sampler. This
appears to necessitate a slight API change, though the pre-existing
samplers will simply ignore this new information. This expansion
encourages further development of methods that take advantage of this
inverse Jacobian data and sets a framework for future transformations
that may also want to pass inverse Jacobian data to samplers that use
it.

My question at the moment is: what is the best way to pass this data
to the sampler?

Note: Passing the Jacobian matrix itself may be better, but Nicolas is
of the opinion that degenerate cases are easier to handle if the
inverse is passed.


Adam Turcotte

In the case of affine transformations, there is no need to approximate
the inverse Jacobian: computing it exactly is trivial. For warps which
follow a complex algorithm, it may however be necessary to
approximate.

Nicolas Robidoux

Adam Turcotte writes:
> In addition, I will need to expand the OpAffine struct to include an
> additional GeglMatrix3 (call it matrix_inverse or inverse_jacobian?)
> to store this data.

GeglMatrix3? Is this a 3x3 matrix type?

(Ignore the following if it is not.)

Maybe the matrix created by the create_matrix () method is not really
the "plain Jacobian matrix" even for affine transformations.

I understand that there are convenient representations of common image
transforms in terms of 3x3 matrices, but I wonder if using a 3x3 to
store either the Jacobian or its inverse just confuses things, at
least for our purposes.

My suggestion: If create_matrix () returns a 3x3, keep the inverse
Jabocian computation out of create_matrix ().

Because I am far from certain that there would ever be a need to do
matrix operations on the inverse Jacobian, I actually am not even sure
that it is necessary to store it in an explicit matrix type. For
documentation sake? Can anyone foresee a use for the inverse Jacobian
matrix being stored in a 3x3?

Nicolas Robidoux

Adam:

(Ignore if my last email about GeglMatrix3 was off the mark.)

Is it that affine transformations, stored in 3x3 matrices, get
simplified when they are chained?

If so, where is the final overall 3x3 representation?

Hopefully, extracting the linear part from it to compute the inverse
Jacobian could be done there.

Nicolas Robidoux

Because, AFAIK, representing transformations as 3x3 matrices is only
natural for affine, perspective and combinations, but not for more
general warps, it is my opinion that what should be passed to samplers
(so that those which may downsample use it) should either be an array
with four double entries or an object of a type which could be named
GeglMatrix2.

Is it really worth it to create (name) a new GEGL type just for this
purpose? A plain array sure seems more expedient.

Nicolas Robidoux

On 06/29/2009 03:16 PM, Nicolas Robidoux wrote:
> a type which could be named
> GeglMatrix2.
>
> Is it really worth it to create (name) a new GEGL type just for this
> purpose? A plain array sure seems more expedient.
>

Since GeglMatrix3 exists, GeglMatrix2 also should exist for consistency.

Personally I don't see the point in duplicating a matrix lib in every
software, i.e. I question the introduction of GeglMatrix3 in the first
place, I mean there certainly must be high quality libs for matrix
operations out there, but that's a different story...

/ Martin

Martin Nordhotls wrote:
> Personally I don't see the point in duplicating a matrix lib in
> every software, i.e. I question the introduction of GeglMatrix3 in
> the first place, I mean there certainly must be high quality libs
> for matrix operations out there, but that's a different story...

IMHO, when all you are dealing with are 3x3 matrices (which are not
arbitrary 3x3, since they represent affine + perspective
transformations), it is likely that a general purpose matrix library
would be slower. Even more so with 2x2.

So, my vote is actually for "not library."

Nicolas Robidoux

On 06/29/2009 09:53 PM, Nicolas Robidoux wrote:
> IMHO, when all you are dealing with are 3x3 matrices (which are not
> arbitrary 3x3, since they represent affine + perspective
> transformations), it is likely that a general purpose matrix library
> would be slower. Even more so with 2x2.
>
> So, my vote is actually for "not library."
>

I'd expect any high quality matrix library to provide optimized routines
and data structures for 3x3 matrices.

/ Martin

On Mon, Jun 29, 2009 at 3:44 PM, Martin Nordholts<enselic@gmail.com> wrote:
> Since GeglMatrix3 exists, GeglMatrix2 also should exist for consistency.

Should GeglMatrix2 also reside within gegl-matrix.c, or should it be
separate? I can quickly create GeglMatrix2 for the purposes of passing
Jacobian information.



Adam Turcotte

On 06/29/2009 10:42 PM, Adam Turcotte wrote:
> Should GeglMatrix2 also reside within gegl-matrix.c

It'll be fine to put it in gegl-matrix.c

/ Martin

Adam:

At this point I think that all that is needed is:

A function which computes the 2x2 inverse Jacobian from the 3x3 and
puts it in a 2x2. Given that right now only affine functions are
supported in GEGL this is the inverse of the top-left 2x2. The source
code needs to make a note that when perspective is supported the
Jacobian and its inverse will not be solely defined by the top left
2x2 submatrix. (It would make sense to fix the code before the end of
the Summer so that the inverse Jacobian is correct when perspective is
implemented in GEGL.)

Nicolas Robidoux

On 06/25/2009 10:50 PM, Adam Turcotte wrote:
> My question at the moment is: what is the best way to pass this data
> to the sampler?
>

I feel bad about you not getting a reply, so I want to give a reply even
though it's not a super helpful one:

Use whatever way you can come up with and that works for you. If it
turns out we don't like your approach for whatever reason, we'll take it
from there.

Best regards,
Martin