postgres-xc-developers Mailing List for Postgres-XC

On Fri, Mar 30, 2012 at 8:05 PM, Andrei Martsinchyk <
and...@gm...> wrote:

> Hi Michael,
>
> Just found the reason why referential integrity check does not work on
> datanodes.
> Here is the code in RI_FKey_Check (ri_triggers.c):
>
> #ifdef PGXC
>  /*
>  * XXX Referential integrity is not yet supported. We just come out of the
>  * function without actually performing any integrity checks. This must be
>  * fixed when we support local/global referential integrities.
>  */
> return PointerGetDatum(NULL);
> #endif
>
> I guess it will work if these lines will be removed.
>
Hum, OK. It would be nice to make a try, but XC does not support yet local
and global referential integrity.
So perhaps in the particular case of the patch it will work, but I would
except side effects.
Could you update your patch with those lines deleted?
-- 
Michael Paquier
http://michael.otacoo.com

Hi Michael,

Just found the reason why referential integrity check does not work on
datanodes.
Here is the code in RI_FKey_Check (ri_triggers.c):

#ifdef PGXC
/*
 * XXX Referential integrity is not yet supported. We just come out of the
 * function without actually performing any integrity checks. This must be
 * fixed when we support local/global referential integrities.
 */
return PointerGetDatum(NULL);
#endif

I guess it will work if these lines will be removed.

23 марта 2012 г. 3:32 пользователь Michael Paquier <
mic...@gm...> написал:

>
>
> On Thu, Mar 22, 2012 at 5:50 PM, Andrei Martsinchyk <
> and...@gm...> wrote:
>
>> Hi Michael,
>>
>> So you find "A tuple cannot be fetched from Coordinator" error is better
>> then "could not read block 0 in file "base/12052/32771": read only 0 of
>> 8192 bytes" ?
>> I think both are equally misty for the end user.
>> I have reported the bug some time ago (ID: 3485997) and proposed a patch,
>> that fixes the problem.
>> The idea is to skip AFTER UPDATE/DELETE triggers if target relation is
>> remote. The triggers enforcing the foreign key are still fired on the
>> datanodes.
>> Is something wrong with my patch?
>>
> Your patch does not protect data consistency on foreign constraints, which
> is perhaps worse than the bug referenced.
> With this test case:
>
> create table x1 (id int, value timestamp);
> create table x2 (id int, value timestamp);
> alter table x1 add primary key (id);
> alter table x2 add foreign key (id) references x1(id);
> insert into x1 values (1, now());
> insert into x2 values (2, now());
>
> XC inserts the tuple.
>
> Postgres returns this error:
> ERROR:  insert or update on table "x2" violates foreign key constraint
> "x2_id_fkey"
> DETAIL:  Key (id)=(2) is not present in table "x1".
>
>
>> 22 марта 2012 г. 9:31 пользователь Michael Paquier <
>> mic...@gm...> написал:
>>
>> >
>> > Hi,
>> >
>> > Please find attached a patch that handles smoothly the following case:
>> > postgres=# create table x1 (id int, value timestamp);
>> > CREATE TABLE
>> > postgres=# create table x2 (id int, value timestamp);
>> > CREATE TABLE
>> > postgres=# alter table x1 add primary key (id);
>> > NOTICE: ALTER TABLE / ADD PRIMARY KEY will create implicit index
>> "x1_pkey" for table "x1"
>> > ALTER TABLE
>> > postgres=# alter table x2 add foreign key (id) references x1(id);
>> > ALTER TABLE
>> > postgres=# insert into x1 values (1, now());
>> > INSERT 0 1
>> > postgres=# insert into x2 values (1, now());
>> > INSERT 0 1
>> > postgres=# update x1 set value=now() where id=1;
>> > ERROR: could not read block 0 in file "base/12052/32768": read only 0
>> of 8192 bytes
>> > postgres=# update x2 set value=now() where id=1;
>> > ERROR: could not read block 0 in file "base/12052/32771": read only 0
>> of 8192 bytes
>> >
>> > What happens here is that the update query tries to fetch a tuple on
>> Coordinator to check foreign key consistency by using a trigger mechanism.As
>> > As we do not target that in 1.0, I propose simply to block such cases
>> with the patch attached.
>> > Opinions?
>> > --
>> > Michael Paquier
>> > http://michael.otacoo.com
>> >
>> >
>> ------------------------------------------------------------------------------
>> > This SF email is sponsosred by:
>> > Try Windows Azure free for 90 days Click Here
>> > http://p.sf.net/sfu/sfd2d-msazure
>> > _______________________________________________
>> > Postgres-xc-developers mailing list
>> > Pos...@li...
>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>> >
>>
>>
>>
>> --
>> Best regards,
>> Andrei Martsinchyk                   mailto:and...@gm...
>>
>
>
>
> --
> Michael Paquier
> http://michael.otacoo.com
>



-- 
Best regards,
Andrei Martsinchyk                   mailto:and...@gm...

I wonder why ctid is involved here.   This might have been introduced by very old code and may need to look into it.  Not a simple work but this may be related to many other potential problems.

Regards;
---
Koichi Suzuki

On Fri, 30 Mar 2012 11:09:10 +0900
Michael Paquier <mic...@gm...> wrote:

> Hi Ashutosh,
> 
> I have been working around the area of remote DELETE and remote UPDATE
> planning. I am trying to have a general support to allow users to use
> freely rules and joins on those queries.
> A first step of my work was to have a look at what was happening for
> DELETE, and I found that the target list generated in rewriteHandler is not
> correct because it fetched some TLEs that are not always from this relation.
> It resulted in the commit I did yesterday which was a quick fix, and you
> were correct, it is possible to use the varno in the TLE's expressions, so
> setting origtbl is not necessary.
> 
> However, before doing anything else, I have been digging deeply in the code.
> Just by having a look at the code now, I can say the following things
> regarding remote UPDATE and DELETE:
> 1) The code in create_remotedelete_plan is correct. But...
> I saw some issues with the materialization target list at the bottom of
> remote DELETE node in planning.
> Let's take a simple example:
> create table aa (a int, b int);
> create table bb (a int, b int);
> 
> And here is a resulting plan:
> postgres=# explain verbose delete from aa using bb where bb.a = aa.a and
> bb.b = aa.b;
> 
> QUERY PLAN
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>  Delete on public.aa  (cost=0.00..0.00 rows=0 width=0)
>    Node/s: dn1, dn2
>    Remote query: DELETE FROM public.aa WHERE a = $1 AND b = $2 AND ctid =
> $3 AND $4 = $4
>    ->  Result  (cost=0.00..0.00 rows=0 width=0)
>          Output: "__REMOTE_JOIN_QUERY__".a_1_1_1,
> "__REMOTE_JOIN_QUERY__".b_1_2_1, "__REMOTE_JOIN_QUERY__".ctid_1_1_1,
> "__REMOTE_JOIN_QUERY__".ctid_2_1_1
>          ->  Materialize  (cost=0.00..0.00 rows=0 width=0)
>                Output: "__REMOTE_JOIN_QUERY__".a_1_1_1,
> "__REMOTE_JOIN_QUERY__".b_1_2_1, "__REMOTE_JOIN_QUERY__".ctid_1_1_1,
> "__REMOTE_JOIN_QUERY__".ctid_2_1_1
>                ->  Data Node Scan on "__REMOTE_JOIN_QUERY__"
> (cost=0.00..0.00 rows=1000 width=14)
>                      Output: "__REMOTE_JOIN_QUERY__".a_1_1_1,
> "__REMOTE_JOIN_QUERY__".b_1_2_1, "__REMOTE_JOIN_QUERY__".ctid_1_1_1,
> "__REMOTE_JOIN_QUERY__".ctid_2_1_1
>                      Node/s: dn1, dn2
>                      Remote query: SELECT in_1.a AS a_1_1_1 ...
> 
> The output node on top of __REMOTE_JOIN_QUERY__ is obviously incorrect.
> What we get here is "aa.a,aa.b, aa.ctid, bb.ctid", this is incorrect
> because the ctid of the second relation involved in the join should not be
> included in the final target list to generate a correct remote DELETE.
> I suppose there is an error at the top of create_remotejoin_plan, but it
> will be helpful if you could direct me where this could happen.
> 
> For the time being, I added in master branch a trick to bypass parameters
> that are not from the relation (here refer to the "$4 = $4"), but this is
> definitely incorrect.
> 
> 2) the code in create_remoteupdate_plan is incorrect.
> This code needs to be definitely reworked. Postgres has already some
> mechanisms to pre-generate a target list, but what we should do is
> something like that to complete it:
> 
> Based on the example above (This is based on current master with my patch
> but is incorrect):
> postgres=# explain verbose update bb set b = 2 where b = 5;
>                                      QUERY
> PLAN
> ------------------------------------------------------------------------------------
>  Update on public.bb  (cost=0.00..0.00 rows=1000 width=14)
>    Node/s: dn1, dn2
>    Node expr: bb.a
>    Remote query: UPDATE public.bb SET b = $2 WHERE b = $3 AND ctid = $4 AND
> a = $1
>    ->  Result  (cost=0.00..0.00 rows=1000 width=14)
>          Output: a, 2, b, ctid
>          ->  Data Node Scan on bb  (cost=0.00..0.00 rows=1000 width=14)
>                Output: a, b, ctid
>                Node/s: dn1, dn2
>                Remote query: SELECT a, b, ctid FROM ONLY bb WHERE (b = 5)
> In this case the target list that we define in rewriter is "b, ctid". I
> added in rewriteHandler a function to scan the vars from parse tree and to
> add in target list some new conditions are ctid is not always consistent in
> cluster.
> Thanks to Postgres, this target list is completed by the new values to set
> for the update query, here on plan it becomes "new.a, new.b, targetlist.b,
> targetlist.ctid".
> What I am planning to do is to generate the UPDATE query like this:
> UPDATE bb SET a = $1, b = $2 WHERE b = $3, ctid = $4;
> 
> You can see that for current planner the query generated is incorrect.
> I believe that UPDATE has the same problems as DELETE regarding the target
> list received from remote join plan, so my first focus will be to fix the
> final target list to insure correct outputs.
> Then the rest of the code will be simple to implement as I am already
> experienced in this area.
> 
> I am adding to this email a WIP patch I used for my investigations.
> Regards,
> -- 
> Michael Paquier
> http://michael.otacoo.com

There are a couple of things I want to discuss about in this area, and it
is related to general query planning.
I'll send an email soon.

On Thu, Mar 29, 2012 at 3:42 PM, Ashutosh Bapat <
ash...@en...> wrote:

> Hi Michael,
> Was this patch discussed on the mailing list? I did not see any relevant
> discussion.
>
> There can a possibility that resorigtbl is set the result table but it's
> not added in rewriting. In this case, would there be erroneous column
> updation? Instead of resorigtbl can you use some attribute inside the Var
> node like varno?
>
> On Thu, Mar 29, 2012 at 10:01 AM, Michael Paquier <
> mic...@us...> wrote:
>
>> Project "Postgres-XC".
>>
>> The branch, master has been updated
>>       via  8427ea322eabca18fa21f286c2284aa4ffa105a8 (commit)
>>      from  0f11b36d79827bd8b2ed66a31e074e1a8a86fba3 (commit)
>>
>>
>> - Log -----------------------------------------------------------------
>>
>> http://postgres-xc.git.sourceforge.net/git/gitweb.cgi?p=postgres-xc/postgres-xc;a=commitdiff;h=8427ea322eabca18fa21f286c2284aa4ffa105a8
>>
>> commit 8427ea322eabca18fa21f286c2284aa4ffa105a8
>> Author: Michael P <mi...@ot...>
>> Date:   Thu Mar 29 13:32:53 2012 +0900
>>
>>    Correct remote DELETE planning for multiple relations
>>
>>    In case several relations were involved in a remote DELETE (rules,
>>    join, etc.), remote query was not correctly generated because it
>>    took into account not only its target entries, but also the ones
>>    from other relations.
>>    A reference of the original table is added in target entries and
>>    this is compared to the relation whose remote DELETE is generated
>>    to insure that all the operations are consistent.
>>
>>    This had consequences like bug 3512657, but also on the usage of
>>    rules on DELETE.
>>
>> M       src/backend/optimizer/plan/createplan.c
>> M       src/backend/rewrite/rewriteHandler.c
>>
>> -----------------------------------------------------------------------
>>
>> Summary of changes:
>>  src/backend/optimizer/plan/createplan.c |   13 ++++++++++---
>>  src/backend/rewrite/rewriteHandler.c    |    9 +++++++++
>>  2 files changed, 19 insertions(+), 3 deletions(-)
>>
>>
>> hooks/post-receive
>> --
>> Postgres-XC
>>
>>
>> ------------------------------------------------------------------------------
>> This SF email is sponsosred by:
>> Try Windows Azure free for 90 days Click Here
>> http://p.sf.net/sfu/sfd2d-msazure
>> _______________________________________________
>> Postgres-xc-committers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-committers
>>
>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EntepriseDB Corporation
> The Enterprise Postgres Company
>
>
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> Postgres-xc-committers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-committers
>
>


-- 
Michael Paquier
http://michael.otacoo.com

On Thu, Mar 29, 2012 at 2:49 PM, Ashutosh Bapat <
ash...@en...> wrote:

> Sorry,
> Here's the correct one,

Thanks. Sorry for my late reply.
It looks good to go, I don't get any failures.
-- 
Michael Paquier
http://michael.otacoo.com