postgres-xc-developers Mailing List for Postgres-XC

I was surprised to see in SF that comment posting is allowed by default as
an option.
This has been changed to allow only users logged in to post comments.

I also marked the bug report full of spams as deleted and create a new
identic one.

The origin of this problem was an attack directly made of SF servers.
It looks that other projects may have been spammed as well.
Thanks for your report.
-- 
Michael Paquier
http://michaelpq.users.sourceforge.net

I have asked Benny to add related test cases in the system as a separate
patch. He will add them after holidays in China.

On Fri, Jan 28, 2011 at 6:54 PM, Mason <ma...@us...>wrote:

> On Thu, Jan 27, 2011 at 8:26 PM, Michael Paquier
> <mic...@gm...> wrote:
> > I committed the two patches after making some tests and correcting a bit
> of
> > typo in it.
> > Thanks a lot! It is extremely helpful.
>
> How does this affect pg_regress? Do the cases that set up data this
> way now work ok?
>
> Thanks,
>
> Mason
>
>
> > Regards,
> > --
> > Michael Paquier
> > http://michaelpq.users.sourceforge.net
> >
> >
> >
> ------------------------------------------------------------------------------
> > Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
> > Finally, a world-class log management solution at an even better
> price-free!
> > Download using promo code Free_Logger_4_Dev2Dev. Offer expires
> > February 28th, so secure your free ArcSight Logger TODAY!
> > http://p.sf.net/sfu/arcsight-sfd2d
> > _______________________________________________
> > Postgres-xc-developers mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >
> >
>
>
> ------------------------------------------------------------------------------
> Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
> Finally, a world-class log management solution at an even better
> price-free!
> Download using promo code Free_Logger_4_Dev2Dev. Offer expires
> February 28th, so secure your free ArcSight Logger TODAY!
> http://p.sf.net/sfu/arcsight-sfd2d
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

On Thu, Jan 27, 2011 at 8:26 PM, Michael Paquier
<mic...@gm...> wrote:
> I committed the two patches after making some tests and correcting a bit of
> typo in it.
> Thanks a lot! It is extremely helpful.

How does this affect pg_regress? Do the cases that set up data this
way now work ok?

Thanks,

Mason


> Regards,
> --
> Michael Paquier
> http://michaelpq.users.sourceforge.net
>
>
> ------------------------------------------------------------------------------
> Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
> Finally, a world-class log management solution at an even better price-free!
> Download using promo code Free_Logger_4_Dev2Dev. Offer expires
> February 28th, so secure your free ArcSight Logger TODAY!
> http://p.sf.net/sfu/arcsight-sfd2d
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>

Dears,

The bug reported by Abbas as follows:
create table students(rno integer, class int, pos int) distribute by
round robin;
create rule skip_students as on insert to students do instead nothing;
insert into students(rno, class, pos) values (1, 10, 5), (2, 10, 6),
(3, 10, 7), (4, 10, 8)
crashes the coordinator with segmentation fault.

The enclosure is the patch for the above bug.

Because Postgres-XC declares that it doesn't support rule by now, I
didn't pay much attention on rule test. Thank Abbas for your test.

BTW, the test cases will be submitted after my holiday is over.

Thanks again.

Regards,
Benny

Hi all,

The enclosure is the patch for bug as follows:
create table pupils(rno integer, class int, pos int default 2345)
distribute by round robin;
insert into pupils (rno, class) values (44,55), (3,4)
complains
ERROR:  INSERT has more target columns than expressions
where as
insert into pupils (rno, class, pos) values (44,55, DEFAULT), (3,4, DEFAULT)
works fine.

This bug I fixed in my earliest patch. But I didn't notice the newest
code has changed the old logic. I fixed it in my patch.

Regards.
Benny

I committed the two patches after making some tests and correcting a bit of
typo in it.
Thanks a lot! It is extremely helpful.
Regards,
-- 
Michael Paquier
http://michaelpq.users.sourceforge.net

Hi Abbas,

2011/1/25 Abbas Butt <abb...@en...>:
> Hi
> Attached please find a modified patch that applies correctly to the current
> state of the repository.
> I made some modifications in the patch other than those which were to make
> the patch apply
>
> In the function QueryRewrite there is an assert
>     Assert(!foundOriginalQuery);
> It fails with multi insert test cases. This is understandable because in our
> case
> multiple queries are supposed to set the command result tag, and
> HandleCmdComplete
> is then supposed to merge these multiple results in to one consolidated
> result.
> Therefore I have put an ifndef on that assertion.
>
> Other than that I have added ifdef PGXC to a couple of places where it was
> missed.
>
> Here is a paragraph describing the patch that can be used as commit message.
> ===========================================
> The patch implements multiple insert syntax in PGXC.
> Multiple insert means using a single insert statement to insert
> multiple rows into a table using the syntax e.g.
>
> insert into students(rno, class, pos) values (1, 10, 5), (2, 10, 6), (3, 10,
> 7), (4, 10, 8);
>
> Without the patch statements like these pass, but actually do not insert any
> thing in the table.
>
> Here is how the patch works, can be used as commit log message.
>
> The main code changes are in re-writer.
> The patch checks to see if the insert statement has more than one sets in
> the provided list of values
> (FOUR in the above example), and in that case rewrites the insert statement.
> The insert rewriter seperates the sets in the provided list of values into
> independent lists
> depending on the distribution of the table, the distribution column and the
> value provided for
> the distribution column.
> Next the main re-writer is seperated into two possible paths, one without a
> for loop and
> if we have a seperated list of insert values, we run a for loop on the list
> and create an insert statement for each of the data nodes providing it that
> sub-group of the original list that is supposed to run on this particular
> data node.
>
> Main work is done now, all that is left is to handle multiple command result
> tags from
> the data nodes. HandleCmdComplete does this, it simply keeps adding into the
> insert row count
> untill all data nodes are done.
> ===========================================
>
> Here are my comments on the code.
>
> 1) The patch adds many new functions in the code base.
>    Each new function added into the code base should be
>    accompanied by a comment header
>    - a small para describiong the logic/purpose
>    - one line parameter detail
>    Benny, After we commit the base version of the patch
>    could you please add these function headers to the code base.
>
> 2) Each patch submitted should be accompanied by a paragraph
>    describing the patch similar to the one I have written above.
>
> 3) Also it would be helpful to attach a few test cases covering
>    the corner cases.
>
> I have completed the basic testing and here are my results
>
> Multiple insert works for tables distributed by round robin or hash
> but it does NOT work for replicated tables.

The encloser is the patch for this bug. It's should be a bug in later
committed code.

Regards,
Benny


> I have yet to test other possible cases that I will finish today.
>
> End Result:
> ==========
> Check in the attached patch and fix the issues as a seperate patch.
>
> Regards
> Abbas
>
>
> On Mon, Jan 17, 2011 at 2:14 PM, Abbas Butt <abb...@en...>
> wrote:
>>
>> OK sure I will try after finishing my regular to do list for today
>>
>> On Mon, Jan 17, 2011 at 1:51 PM, Michael Paquier
>> <mic...@gm...> wrote:
>>>
>>> Hi Abbas,
>>>
>>> It's Michael.
>>> I am not at the company today, but just to let you know that I received
>>> the multi INSERT patch from Benny, the Chinese student.
>>>
>>> If you have time, it may be cool to have a look at it.
>>> Thanks,
>>>
>>> --
>>> Michael Paquier
>>> http://michaelpq.users.sourceforge.net
>>>
>>
>
>

Hi Abbas,

2011/1/25 Abbas Butt <abb...@en...>:
> Hi
> Attached please find a modified patch that applies correctly to the current
> state of the repository.
> I made some modifications in the patch other than those which were to make
> the patch apply
>
> In the function QueryRewrite there is an assert
>     Assert(!foundOriginalQuery);
> It fails with multi insert test cases. This is understandable because in our
> case
> multiple queries are supposed to set the command result tag, and
> HandleCmdComplete
> is then supposed to merge these multiple results in to one consolidated
> result.
> Therefore I have put an ifndef on that assertion.
>
> Other than that I have added ifdef PGXC to a couple of places where it was
> missed.
>
> Here is a paragraph describing the patch that can be used as commit message.
> ===========================================
> The patch implements multiple insert syntax in PGXC.
> Multiple insert means using a single insert statement to insert
> multiple rows into a table using the syntax e.g.
>
> insert into students(rno, class, pos) values (1, 10, 5), (2, 10, 6), (3, 10,
> 7), (4, 10, 8);
>
> Without the patch statements like these pass, but actually do not insert any
> thing in the table.
>
> Here is how the patch works, can be used as commit log message.
>
> The main code changes are in re-writer.
> The patch checks to see if the insert statement has more than one sets in
> the provided list of values
> (FOUR in the above example), and in that case rewrites the insert statement.
> The insert rewriter seperates the sets in the provided list of values into
> independent lists
> depending on the distribution of the table, the distribution column and the
> value provided for
> the distribution column.
> Next the main re-writer is seperated into two possible paths, one without a
> for loop and
> if we have a seperated list of insert values, we run a for loop on the list
> and create an insert statement for each of the data nodes providing it that
> sub-group of the original list that is supposed to run on this particular
> data node.
>
> Main work is done now, all that is left is to handle multiple command result
> tags from
> the data nodes. HandleCmdComplete does this, it simply keeps adding into the
> insert row count
> untill all data nodes are done.
> ===========================================
>
> Here are my comments on the code.
>
> 1) The patch adds many new functions in the code base.
>    Each new function added into the code base should be
>    accompanied by a comment header
>    - a small para describiong the logic/purpose
>    - one line parameter detail
>    Benny, After we commit the base version of the patch
>    could you please add these function headers to the code base.
>
> 2) Each patch submitted should be accompanied by a paragraph
>    describing the patch similar to the one I have written above.
>
I have added comments at the head of each function I added.

Regads,
Benny

> 3) Also it would be helpful to attach a few test cases covering
>    the corner cases.
>
> I have completed the basic testing and here are my results
>
> Multiple insert works for tables distributed by round robin or hash
> but it does NOT work for replicated tables.
>
> I have yet to test other possible cases that I will finish today.
>
> End Result:
> ==========
> Check in the attached patch and fix the issues as a seperate patch.
>
> Regards
> Abbas
>
>
> On Mon, Jan 17, 2011 at 2:14 PM, Abbas Butt <abb...@en...>
> wrote:
>>
>> OK sure I will try after finishing my regular to do list for today
>>
>> On Mon, Jan 17, 2011 at 1:51 PM, Michael Paquier
>> <mic...@gm...> wrote:
>>>
>>> Hi Abbas,
>>>
>>> It's Michael.
>>> I am not at the company today, but just to let you know that I received
>>> the multi INSERT patch from Benny, the Chinese student.
>>>
>>> If you have time, it may be cool to have a look at it.
>>> Thanks,
>>>
>>> --
>>> Michael Paquier
>>> http://michaelpq.users.sourceforge.net
>>>
>>
>
>

I am not sure if this is going to be enough. Earlier in the code, are
all messages consumed on all connections when there is an error?

We had a problem in the past where if there was an error, we put the
connections back in the pool and then buffered messages were consumed
on subsequent requests.  This also affected connections where there
were no errors; the connection seemed fine, so we just threw it back
in the pool because one of the other connections had a problem. Next
time it was grabbed from the pool, the current request would consume
messages from the previous request.

If you can verify that these cases are handled properly, the patch may
be fine and can be committed. Otherwise, this may help with the
current bug, but it may cause other problems.

I did have code to try and consume these extra messages which seemed
to be working. Other changes in connection handling done later seemed
to break it, and then this code got removed in a later patch.
Admittedly, the code should not really be needed if handling is good
elsewhere. Anyway, I would verify that other error cases like data
node crashes are handled properly before committing this, and keep
this in mind in case other errors suddenly appear, like unexpected
messages from data nodes or a mismatch of columns in DataRow messages,
etc.

On a related matter -

I am also adding that long term when support for savepoints is added,
it is not enough to just flag the connections as bad for disposal (or
force a rollback right away)-- we want the application to be able to
rollback to an earlier savepoint if possible. So, even after an error,
we want to gracefully handle all of the messages on all connections
involved in the statement and also not issue a ROLLBACK internally.
That way, rollback to a savepoint is possible.  Actually, implementing
savepoints might be one way of forcing us to make sure that connection
handling is done properly.

Regards,

Mason

Hi all,

The enclosure is a  patch for cleaning the coordinator connections
state when an error occurred on any of datanodes.

When we executed statements like follows:
create table aa(a int);
1.alter table aa rename  b to bb;
ERROR: relation "aa" does not exist

2.alter table aa rename  b to bb;
ERROR: Could not begin transaction on Coordinators

As you see, the error messages are different. After checked the source
code, we found it needs to clean the coordinators state when error
occured on datanodes.

Your suggestions will be appreciated.

Regards,
Benny

Hi all,

A lot of patches are being submitted through the mailing list,
what is kind of nice because people show interest in XC.

However, due to a lack of support to patch submitters [?],
it has become kind of difficult to follow which patch is being submitted,
which one is in review, who is the reviewer, etc.

I'd like to use this thread to centralize the reviews.
It would be nice that comments or discussions related to a special patch are
made on another thread.

So, please find attached the list of current patches waiting for review
(creepy table... btw).
I am also attaching the patches themselves.

I counted not less than 7 patches until now.
Perhaps I am missing some patches, so feel free to complete my list.

I am myself planning to review some of those patches in short term.
-- 
Michael Paquier
http://michaelpq.users.sourceforge.net

Hi all,

As we are beginning the implementation of HA features for Postgres-XC,
I have created a new branch called ha_support that contains all the
functionalities linked to HA.

This is not put in master branch yet because there are risks of bugs linked
to HA,
and I wanted to make a distinction between HA and SQL related stuff.
First implementation of Datanode Mirroring will also be made in this branch.

As soon as this branch will be considered as stable, it will be merged with
master.
Regards,

-- 
Michael Paquier
http://michaelpq.users.sourceforge.net

Koichi san,
Thank you for the reply.


Koichi Suzuki san wrote:
> Practically, it is just for internal use.   When coordinator receives 
> statements using sequence, it first looks up coordinator catalog and 
> go to GTM only when it is registered.   So this will not cause any 
> problem.
>
> Call to gtm is not exposed to XC application.   It is only called from 
> within coordinator and datanode internals.


Yes, I know well.
I am carefully examining details of the gtm's specification.
Because, I'm making gtm and gtm_proxy implemented in Java as you already 
know.


Regards,
    suzuki hironobu

Thank you very much for pointing out the issue.

I will put this to bug track list of the project.

Practically, it is just for internal use.   When coordinator receives 
statements using sequence, it first looks up coordinator catalog and go 
to GTM only when it is registered.   So this will not cause any problem.

Call to gtm is not exposed to XC application.   It is only called from 
within coordinator and datanode internals.

Thank you again;
---
Koichi Suzuki

(2011年01月13日 12:31), Suzuki Hironobu wrote:
> Dear developers,
>
> I found a gtm's bug.
>
> Two functions in gtm_seq.c (ProcessSequenceGetCurrentCommand and
> ProcessSequenceGetNextCommand) do not return the error even if
> sequence_key is not registered.
> In this case, these functions return 22 (= EINVAL) as a normal value.
> Therefore, gtm also returns 22 to proxy and coordinator.
>
>
> Regards,
> suzuki hironobu
>
>
> P.S.
> The simplest bugfix patch is as follows:
>
> diff -crN pgxc.org/src/gtm/main/gtm_seq.c pgxc/src/gtm/main/gtm_seq.c
> *** pgxc.org/src/gtm/main/gtm_seq.c 2011-01-13 11:38:55.000000000 +0900
> --- pgxc/src/gtm/main/gtm_seq.c 2011-01-13 11:45:48.000000000 +0900
> ***************
> *** 532,535 ****
> if (seqinfo == NULL)
> {
> ! ereport(LOG,
> (EINVAL,
> --- 532,535 ----
> if (seqinfo == NULL)
> {
> ! ereport(ERROR,
> (EINVAL,
> ***************
> *** 596,599 ****
> if (seqinfo == NULL)
> {
> ! ereport(LOG,
> (EINVAL,
> --- 596,599 ----
> if (seqinfo == NULL)
> {
> ! ereport(ERROR,
> (EINVAL,
>
>
>
>
> ------------------------------------------------------------------------------
> Protect Your Site and Customers from Malware Attacks
> Learn about various malware tactics and how to avoid them. Understand
> malware threats, the impact they can have on your business, and how you
> can protect your company and customers by using code signing.
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

dear all,

attached patch fix 2 problems:

1. make -j broken
2. install pgxc headers when make install, utils/rel.h and other headers
depend on it. 

Hi Andrei,

Thank you very much. I understand now.

Regards,
Benny

2011/1/13 Andrei.Martsinchyk <and...@en...>:
> Benny,
>
> OK, the distributed constraint restriction still in force.
> Partitioning of master and slave tables should guarantee that if
> master record ever presents in the table it must be on the same data
> node as the slave record. Constraint checker only sees local data
> node, it can not look up other datanodes.
>
> 2011/1/13 xiong wang <wan...@gm...>:
>> Hi,
>> I am sorry that I took a wrong case as an example.
>> The right example is like:
>> postgres=# create table xyz(x int, y int, foreign key(y) references
>> shench(a)) distribute by hash(x);//not by hash(a)
>> ERROR:  Hash distributed table must include distribution column in index
>>
>> My question is why hash distributed key "x" must be included in foreign keys.
>>
>> Many thanks for your reply.
>>
>> Regards,
>> Benny
>>
>> 2011/1/13 Andrei.Martsinchyk <and...@en...>:
>>> Benny,
>>>
>>> The column specified in hash() clause must be from table being
>>> created. The specified (a) column is from other table. It looks like
>>> it just output incorrect error message.
>>> The note about distributed constraint is correct.
>>> In your example one of two conditions must be met:
>>> Table xyz must be hash distributed by y AND shench table must be hash
>>> distributed by a AND columns y and a must be of the same data type;
>>> or
>>> shench table must be replicated.
>>>
>>> 2011/1/13 Koichi Suzuki <ko...@in...>:
>>>> Unfortunately, reference integrity is not fully supported by XC yet,
>>>> because we have to check existing of the matching rows in different data
>>>> nodes, which is not simple, as in the case of global unique constraint
>>>> for non-distribution columns.
>>>>
>>>> So I think this is one of the issue we have.
>>>>
>>>> Any further thoughts?
>>>>
>>>> ---
>>>> Koichi
>>>>
>>>> (2011年01月13日 15:53), xiong wang wrote:
>>>>> Dears,
>>>>>
>>>>> When We are fixing the bug relative with the foreign key, there is
>>>>> something we are confused. The problem as follows:
>>>>> postgres=# create table xyz(x int, y int, foreign key(y) references
>>>>> shench(a)) distribute by hash(a);
>>>>> ERROR:  Hash distributed table must include distribution column in index
>>>>>
>>>>> I don't know why Hash distributed column must be included in foreign
>>>>> constraint. Could you explain something?
>>>>>
>>>>> Thanks a lot.
>>>>>
>>>>> Regards,
>>>>> Benny
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Protect Your Site and Customers from Malware Attacks
>>>>> Learn about various malware tactics and how to avoid them. Understand
>>>>> malware threats, the impact they can have on your business, and how you
>>>>> can protect your company and customers by using code signing.
>>>>> http://p.sf.net/sfu/oracle-sfdevnl
>>>>> _______________________________________________
>>>>> Postgres-xc-developers mailing list
>>>>> Pos...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Protect Your Site and Customers from Malware Attacks
>>>> Learn about various malware tactics and how to avoid them. Understand
>>>> malware threats, the impact they can have on your business, and how you
>>>> can protect your company and customers by using code signing.
>>>> http://p.sf.net/sfu/oracle-sfdevnl
>>>> _______________________________________________
>>>> Postgres-xc-developers mailing list
>>>> Pos...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>
>>>
>>>
>>>
>>> --
>>> Andrei Martsinchyk
>>>
>>> EntepriseDB Corporation
>>> The Enterprise Postgres Company
>>>
>>> Website: www.enterprisedb.com
>>> EnterpriseDB Blog: http://blogs.enterprisedb.com/
>>> Follow us on Twitter: http://www.twitter.com/enterprisedb
>>>
>>> This e-mail message (and any attachment) is intended for the use of
>>> the individual or entity to whom it is addressed. This message
>>> contains information from EnterpriseDB Corporation that may be
>>> privileged, confidential, or exempt from disclosure under applicable
>>> law. If you are not the intended recipient or authorized to receive
>>> this for the intended recipient, any use, dissemination, distribution,
>>> retention, archiving, or copying of this communication is strictly
>>> prohibited. If you have received this e-mail in error, please notify
>>> the sender immediately by reply e-mail and delete this message.
>>>
>>
>
>
>
> --
> Andrei Martsinchyk
>
> EntepriseDB Corporation
> The Enterprise Postgres Company
>
> Website: www.enterprisedb.com
> EnterpriseDB Blog: http://blogs.enterprisedb.com/
> Follow us on Twitter: http://www.twitter.com/enterprisedb
>
> This e-mail message (and any attachment) is intended for the use of
> the individual or entity to whom it is addressed. This message
> contains information from EnterpriseDB Corporation that may be
> privileged, confidential, or exempt from disclosure under applicable
> law. If you are not the intended recipient or authorized to receive
> this for the intended recipient, any use, dissemination, distribution,
> retention, archiving, or copying of this communication is strictly
> prohibited. If you have received this e-mail in error, please notify
> the sender immediately by reply e-mail and delete this message.
>

Benny,

OK, the distributed constraint restriction still in force.
Partitioning of master and slave tables should guarantee that if
master record ever presents in the table it must be on the same data
node as the slave record. Constraint checker only sees local data
node, it can not look up other datanodes.

2011/1/13 xiong wang <wan...@gm...>:
> Hi,
> I am sorry that I took a wrong case as an example.
> The right example is like:
> postgres=# create table xyz(x int, y int, foreign key(y) references
> shench(a)) distribute by hash(x);//not by hash(a)
> ERROR:  Hash distributed table must include distribution column in index
>
> My question is why hash distributed key "x" must be included in foreign keys.
>
> Many thanks for your reply.
>
> Regards,
> Benny
>
> 2011/1/13 Andrei.Martsinchyk <and...@en...>:
>> Benny,
>>
>> The column specified in hash() clause must be from table being
>> created. The specified (a) column is from other table. It looks like
>> it just output incorrect error message.
>> The note about distributed constraint is correct.
>> In your example one of two conditions must be met:
>> Table xyz must be hash distributed by y AND shench table must be hash
>> distributed by a AND columns y and a must be of the same data type;
>> or
>> shench table must be replicated.
>>
>> 2011/1/13 Koichi Suzuki <ko...@in...>:
>>> Unfortunately, reference integrity is not fully supported by XC yet,
>>> because we have to check existing of the matching rows in different data
>>> nodes, which is not simple, as in the case of global unique constraint
>>> for non-distribution columns.
>>>
>>> So I think this is one of the issue we have.
>>>
>>> Any further thoughts?
>>>
>>> ---
>>> Koichi
>>>
>>> (2011年01月13日 15:53), xiong wang wrote:
>>>> Dears,
>>>>
>>>> When We are fixing the bug relative with the foreign key, there is
>>>> something we are confused. The problem as follows:
>>>> postgres=# create table xyz(x int, y int, foreign key(y) references
>>>> shench(a)) distribute by hash(a);
>>>> ERROR:  Hash distributed table must include distribution column in index
>>>>
>>>> I don't know why Hash distributed column must be included in foreign
>>>> constraint. Could you explain something?
>>>>
>>>> Thanks a lot.
>>>>
>>>> Regards,
>>>> Benny
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Protect Your Site and Customers from Malware Attacks
>>>> Learn about various malware tactics and how to avoid them. Understand
>>>> malware threats, the impact they can have on your business, and how you
>>>> can protect your company and customers by using code signing.
>>>> http://p.sf.net/sfu/oracle-sfdevnl
>>>> _______________________________________________
>>>> Postgres-xc-developers mailing list
>>>> Pos...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Protect Your Site and Customers from Malware Attacks
>>> Learn about various malware tactics and how to avoid them. Understand
>>> malware threats, the impact they can have on your business, and how you
>>> can protect your company and customers by using code signing.
>>> http://p.sf.net/sfu/oracle-sfdevnl
>>> _______________________________________________
>>> Postgres-xc-developers mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>
>>
>>
>>
>> --
>> Andrei Martsinchyk
>>
>> EntepriseDB Corporation
>> The Enterprise Postgres Company
>>
>> Website: www.enterprisedb.com
>> EnterpriseDB Blog: http://blogs.enterprisedb.com/
>> Follow us on Twitter: http://www.twitter.com/enterprisedb
>>
>> This e-mail message (and any attachment) is intended for the use of
>> the individual or entity to whom it is addressed. This message
>> contains information from EnterpriseDB Corporation that may be
>> privileged, confidential, or exempt from disclosure under applicable
>> law. If you are not the intended recipient or authorized to receive
>> this for the intended recipient, any use, dissemination, distribution,
>> retention, archiving, or copying of this communication is strictly
>> prohibited. If you have received this e-mail in error, please notify
>> the sender immediately by reply e-mail and delete this message.
>>
>



-- 
Andrei Martsinchyk

EntepriseDB Corporation
The Enterprise Postgres Company

Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb

This e-mail message (and any attachment) is intended for the use of
the individual or entity to whom it is addressed. This message
contains information from EnterpriseDB Corporation that may be
privileged, confidential, or exempt from disclosure under applicable
law. If you are not the intended recipient or authorized to receive
this for the intended recipient, any use, dissemination, distribution,
retention, archiving, or copying of this communication is strictly
prohibited. If you have received this e-mail in error, please notify
the sender immediately by reply e-mail and delete this message.

Hi,
I am sorry that I took a wrong case as an example.
The right example is like:
postgres=# create table xyz(x int, y int, foreign key(y) references
shench(a)) distribute by hash(x);//not by hash(a)
ERROR:  Hash distributed table must include distribution column in index

My question is why hash distributed key "x" must be included in foreign keys.

Many thanks for your reply.

Regards,
Benny

2011/1/13 Andrei.Martsinchyk <and...@en...>:
> Benny,
>
> The column specified in hash() clause must be from table being
> created. The specified (a) column is from other table. It looks like
> it just output incorrect error message.
> The note about distributed constraint is correct.
> In your example one of two conditions must be met:
> Table xyz must be hash distributed by y AND shench table must be hash
> distributed by a AND columns y and a must be of the same data type;
> or
> shench table must be replicated.
>
> 2011/1/13 Koichi Suzuki <ko...@in...>:
>> Unfortunately, reference integrity is not fully supported by XC yet,
>> because we have to check existing of the matching rows in different data
>> nodes, which is not simple, as in the case of global unique constraint
>> for non-distribution columns.
>>
>> So I think this is one of the issue we have.
>>
>> Any further thoughts?
>>
>> ---
>> Koichi
>>
>> (2011年01月13日 15:53), xiong wang wrote:
>>> Dears,
>>>
>>> When We are fixing the bug relative with the foreign key, there is
>>> something we are confused. The problem as follows:
>>> postgres=# create table xyz(x int, y int, foreign key(y) references
>>> shench(a)) distribute by hash(a);
>>> ERROR:  Hash distributed table must include distribution column in index
>>>
>>> I don't know why Hash distributed column must be included in foreign
>>> constraint. Could you explain something?
>>>
>>> Thanks a lot.
>>>
>>> Regards,
>>> Benny
>>>
>>> ------------------------------------------------------------------------------
>>> Protect Your Site and Customers from Malware Attacks
>>> Learn about various malware tactics and how to avoid them. Understand
>>> malware threats, the impact they can have on your business, and how you
>>> can protect your company and customers by using code signing.
>>> http://p.sf.net/sfu/oracle-sfdevnl
>>> _______________________________________________
>>> Postgres-xc-developers mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Protect Your Site and Customers from Malware Attacks
>> Learn about various malware tactics and how to avoid them. Understand
>> malware threats, the impact they can have on your business, and how you
>> can protect your company and customers by using code signing.
>> http://p.sf.net/sfu/oracle-sfdevnl
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>
>
>
> --
> Andrei Martsinchyk
>
> EntepriseDB Corporation
> The Enterprise Postgres Company
>
> Website: www.enterprisedb.com
> EnterpriseDB Blog: http://blogs.enterprisedb.com/
> Follow us on Twitter: http://www.twitter.com/enterprisedb
>
> This e-mail message (and any attachment) is intended for the use of
> the individual or entity to whom it is addressed. This message
> contains information from EnterpriseDB Corporation that may be
> privileged, confidential, or exempt from disclosure under applicable
> law. If you are not the intended recipient or authorized to receive
> this for the intended recipient, any use, dissemination, distribution,
> retention, archiving, or copying of this communication is strictly
> prohibited. If you have received this e-mail in error, please notify
> the sender immediately by reply e-mail and delete this message.
>

Hi Koichi.

Thank your for your reply.

I consider it will make sense if referenced key must be included in an
index in your view, but it doesn't need to constraint hash parition
key must be included in foreign keys. That's what I am confused.

Thanks again.
Regards,
Benny

2011/1/13 Koichi Suzuki <ko...@in...>:
> Unfortunately, reference integrity is not fully supported by XC yet, because
> we have to check existing of the matching rows in different data nodes,
> which is not simple, as in the case of global unique constraint for
> non-distribution columns.
>
> So I think this is one of the issue we have.
>
> Any further thoughts?
>
> ---
> Koichi
>
> (2011年01月13日 15:53), xiong wang wrote:
>>
>> Dears,
>>
>> When We are fixing the bug relative with the foreign key, there is
>> something we are confused. The problem as follows:
>> postgres=# create table xyz(x int, y int, foreign key(y) references
>> shench(a)) distribute by hash(a);
>> ERROR:  Hash distributed table must include distribution column in index
>>
>> I don't know why Hash distributed column must be included in foreign
>> constraint. Could you explain something?
>>
>> Thanks a lot.
>>
>> Regards,
>> Benny
>>
>>
>> ------------------------------------------------------------------------------
>> Protect Your Site and Customers from Malware Attacks
>> Learn about various malware tactics and how to avoid them. Understand
>> malware threats, the impact they can have on your business, and how you
>> can protect your company and customers by using code signing.
>> http://p.sf.net/sfu/oracle-sfdevnl
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>
>

Benny,

The column specified in hash() clause must be from table being
created. The specified (a) column is from other table. It looks like
it just output incorrect error message.
The note about distributed constraint is correct.
In your example one of two conditions must be met:
Table xyz must be hash distributed by y AND shench table must be hash
distributed by a AND columns y and a must be of the same data type;
or
shench table must be replicated.

2011/1/13 Koichi Suzuki <ko...@in...>:
> Unfortunately, reference integrity is not fully supported by XC yet,
> because we have to check existing of the matching rows in different data
> nodes, which is not simple, as in the case of global unique constraint
> for non-distribution columns.
>
> So I think this is one of the issue we have.
>
> Any further thoughts?
>
> ---
> Koichi
>
> (2011年01月13日 15:53), xiong wang wrote:
>> Dears,
>>
>> When We are fixing the bug relative with the foreign key, there is
>> something we are confused. The problem as follows:
>> postgres=# create table xyz(x int, y int, foreign key(y) references
>> shench(a)) distribute by hash(a);
>> ERROR:  Hash distributed table must include distribution column in index
>>
>> I don't know why Hash distributed column must be included in foreign
>> constraint. Could you explain something?
>>
>> Thanks a lot.
>>
>> Regards,
>> Benny
>>
>> ------------------------------------------------------------------------------
>> Protect Your Site and Customers from Malware Attacks
>> Learn about various malware tactics and how to avoid them. Understand
>> malware threats, the impact they can have on your business, and how you
>> can protect your company and customers by using code signing.
>> http://p.sf.net/sfu/oracle-sfdevnl
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>
>
> ------------------------------------------------------------------------------
> Protect Your Site and Customers from Malware Attacks
> Learn about various malware tactics and how to avoid them. Understand
> malware threats, the impact they can have on your business, and how you
> can protect your company and customers by using code signing.
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>



-- 
Andrei Martsinchyk

EntepriseDB Corporation
The Enterprise Postgres Company

Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb

This e-mail message (and any attachment) is intended for the use of
the individual or entity to whom it is addressed. This message
contains information from EnterpriseDB Corporation that may be
privileged, confidential, or exempt from disclosure under applicable
law. If you are not the intended recipient or authorized to receive
this for the intended recipient, any use, dissemination, distribution,
retention, archiving, or copying of this communication is strictly
prohibited. If you have received this e-mail in error, please notify
the sender immediately by reply e-mail and delete this message.

Unfortunately, reference integrity is not fully supported by XC yet, 
because we have to check existing of the matching rows in different data 
nodes, which is not simple, as in the case of global unique constraint 
for non-distribution columns.

So I think this is one of the issue we have.

Any further thoughts?

---
Koichi

(2011年01月13日 15:53), xiong wang wrote:
> Dears,
>
> When We are fixing the bug relative with the foreign key, there is
> something we are confused. The problem as follows:
> postgres=# create table xyz(x int, y int, foreign key(y) references
> shench(a)) distribute by hash(a);
> ERROR:  Hash distributed table must include distribution column in index
>
> I don't know why Hash distributed column must be included in foreign
> constraint. Could you explain something?
>
> Thanks a lot.
>
> Regards,
> Benny
>
> ------------------------------------------------------------------------------
> Protect Your Site and Customers from Malware Attacks
> Learn about various malware tactics and how to avoid them. Understand
> malware threats, the impact they can have on your business, and how you
> can protect your company and customers by using code signing.
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

Dears,

When We are fixing the bug relative with the foreign key, there is
something we are confused. The problem as follows:
postgres=# create table xyz(x int, y int, foreign key(y) references
shench(a)) distribute by hash(a);
ERROR:  Hash distributed table must include distribution column in index

I don't know why Hash distributed column must be included in foreign
constraint. Could you explain something?

Thanks a lot.

Regards,
Benny

Hi all,

Please see attached a patch that would need review.
It concerns an HA feature for 2PC transactions.

Basically, when there is a 2PC transaction in the cluster, we save in static
2PC structure come additional information:
- Coordinator number from which PREPARE was issued
- if the prepared transaction contained DDL (for Coordinator commit)
- if 2PC is explicit or implicit
- the list of datanodes where transaction PREPARE has been done.

I believe such additional information would help to recover transaction data
in case of a node crash.

If you want to try...
1) For sequences:
begin;
create sequence toto6;
prepare transaction 'i';
-- results
select * from pg_prepared_xacts;
 transaction | gid |           prepared            |  owner  | database |
is_ddl | is_implicit | coordnum | nodelist
-------------+-----+-------------------------------+---------+----------+--------+-------------+----------+----------
        1054 | i   | 2011-01-13 15:10:09.717025+09 | michael | dbt1     |
t      | f           |        1 | n

Note: For prepared transactions involving only sequence DDL, node list is
set at 'n'.

2) general example:
create table test(a int);
begin;
insert into test values (1);
prepare transaction 'hoge';
-- results, connect to datanode 1...
select * from pg_prepared_xacts;
WARNING:  Do not have a GTM snapshot available -- this is normal
 transaction | gid  |           prepared            |  owner  | database |
is_ddl | is_implicit | coordnum | nodelist
-------------+------+-------------------------------+---------+----------+--------+-------------+----------+----------
        1063 | hoge | 2011-01-13 15:22:07.770166+09 | michael | dbt1     |
f      | f           |        1 | 1,2
This was done with 2 Coordinators and 2 Datanodes.

For sure, this impacts performance, I am currently investigating how much
and if it is possible to optimize it.

Thanks,
-- 
Michael Paquier
http://michaelpq.users.sourceforge.net

Dear developers,

I found a gtm's bug.

Two functions in gtm_seq.c (ProcessSequenceGetCurrentCommand and
ProcessSequenceGetNextCommand) do not return the error even if
sequence_key is not registered.
In this case, these functions return 22 (= EINVAL) as a normal value.
Therefore, gtm also returns 22 to proxy and coordinator.


Regards,
suzuki hironobu


P.S.
The simplest bugfix patch is as follows:

diff -crN pgxc.org/src/gtm/main/gtm_seq.c pgxc/src/gtm/main/gtm_seq.c
*** pgxc.org/src/gtm/main/gtm_seq.c 2011-01-13 11:38:55.000000000 +0900
--- pgxc/src/gtm/main/gtm_seq.c 2011-01-13 11:45:48.000000000 +0900
***************
*** 532,535 ****
if (seqinfo == NULL)
{
! ereport(LOG,
(EINVAL,
--- 532,535 ----
if (seqinfo == NULL)
{
! ereport(ERROR,
(EINVAL,
***************
*** 596,599 ****
if (seqinfo == NULL)
{
! ereport(LOG,
(EINVAL,
--- 596,599 ----
if (seqinfo == NULL)
{
! ereport(ERROR,
(EINVAL,

Hi Michael,

I noticed you added IsConnFromCoord() after macro IS_PGXC_COORDINATOR
in all your committed code today and yesterday. I doubt whether it's
right.
In this patch,
                                     if (IS_PGXC_COORDINATOR &&
IsConnFromCoord())  //I consider IsConnFromCoord() should not be added
here because the coordinator which is connected by psql cann't execute
the code below.
			{
				RemoteQueryExecType remoteExecType = EXEC_ON_ALL_NODES;
				RenameStmt *stmt = (RenameStmt *) parsetree;

				if (stmt->renameType == OBJECT_SEQUENCE)
					remoteExecType = EXEC_ON_COORDS;
				else if (stmt->renameType == OBJECT_TABLE)
				{
					Oid relid = RangeVarGetRelid(stmt->relation, false);

					if (get_rel_relkind(relid) == RELKIND_SEQUENCE)
						remoteExecType = EXEC_ON_COORDS;
				}

				ExecUtilityStmtOnNodes(queryString, NULL, false,
remoteExecType);//There is "if (!IsConnFromCoord())" clause in this
function. It can control the execution flow.
			}

Regards,
Benny

> Hi,
>
> Just to let you know that this patch has been committed in git repository
> with this ID:
> 28da86b88e667b5e45e6deef831cc839ccba2703
>
> The logic of the patch was OK, I don't really have comments.
> Well 1 perhaps...
> Be sure to respect the 4-space tab :)
> This is a PostgreSQL convention.
>
> Regards,
>
> --
> Michael Paquier
> http://michaelpq.users.sourceforge.net
>
>