postgres-xc-developers Mailing List for Postgres-XC

Some scenarios, user would have to be careful about assuming that the
query would be run at the datanode. If a query has some non-shippable
expressions, the query might not get completely shipped. Or if the
query does not have tables, the user would use execute direct. I think
we need to keep both ways.

On 29 February 2012 11:50, David E. Wheeler <da...@ju...> wrote:
> On Feb 28, 2012, at 10:14 PM, Ashutosh Bapat wrote:
>
>> SELECT * FROM dn1.table;
>>
>> Instead of mixing with schema qualification syntax, we can also use SELECT * from table@dn1.
>
> +1, although I think @ is allowed to be used for custom operators.
>
>  http://www.postgresql.org/docs/current/static/sql-createoperator.html
>
> So that could cause problems. But maybe not in the context of a table, where such an operator perhaps could not be used?
>
> Best,
>
> David
>
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

On Feb 28, 2012, at 10:14 PM, Ashutosh Bapat wrote:

> SELECT * FROM dn1.table;
> 
> Instead of mixing with schema qualification syntax, we can also use SELECT * from table@dn1.

+1, although I think @ is allowed to be used for custom operators.

  http://www.postgresql.org/docs/current/static/sql-createoperator.html

So that could cause problems. But maybe not in the context of a table, where such an operator perhaps could not be used?

Best,

David

On Wed, Feb 29, 2012 at 11:44 AM, Ashutosh Bapat
<ash...@en...> wrote:
>
>
> On Wed, Feb 29, 2012 at 11:17 AM, Pavan Deolasee
> <pav...@en...> wrote:
>>
>> On Wed, Feb 29, 2012 at 11:03 AM, Koichi Suzuki <koi...@gm...>
>> wrote:
>> > I agree that the suggestion looks better.
>> >
>> > With this modification, EXECUTE DIRECT will receive parsed tree, not a
>> > text which should go straight to the target node.   EXECTE DIRECT
>> > should rebuild the statement.   May the order of the predicate or
>> > nesting parenthesis be different from the original input?   I hope
>> > not.
>>
>> Just for the record, another mechanism that we had discussed in the
>> past is to have some kind of schema for every node in the cluster. So
>> user can schema qualify the objects to force run a query on a
>> particular node. For example,
>>
>> SELECT * FROM dn1.table;
>
>
> Instead of mixing with schema qualification syntax, we can also use SELECT *
> from table@dn1.
>

We can do that. But that gets a bit tricky with function invokation
etc. Also, its probably better to use existing syntax unless we
conclude that it can't be done for some reason.

>>
>>
>> This would return all the tuples from the partition on data node dn1.
>> So this would be equivalent to running:
>>
>> EXECUTE DIRECT ON dn1 'SELECT * FROM table';
>>
>> Once we add the capability of qualify the objects with nodes, user can
>> write many powerful queries for administrative as well as other
>> purposes.
>>
>
> Although, this means that such a query has to go through planner, which can
> be non-performant. With EXEC DIRECT kind of syntax, we know that the query
> does not need any planning at coordinator.
>

Yeah. But then these are mostly administrative or special purpose
queries. We don't expect users to use them a lot. So some additional
planning cost at the coordinator may be a non-issue.

Thanks,
Pavan

-- 
Pavan Deolasee
EnterpriseDB http://www.enterprisedb.com

On Feb 28, 2012, at 9:47 PM, Pavan Deolasee wrote:

> Just for the record, another mechanism that we had discussed in the
> past is to have some kind of schema for every node in the cluster. So
> user can schema qualify the objects to force run a query on a
> particular node. For example,
> 
> SELECT * FROM dn1.table;
> 
> This would return all the tuples from the partition on data node dn1.
> So this would be equivalent to running:

That's awesome, but what if table is in a schema?

I think you'd have to add another level of path, e.g.,

    SELECT * FROM dn1.public.table;

Best,

David

On Wed, Feb 29, 2012 at 11:17 AM, Pavan Deolasee <
pav...@en...> wrote:

> On Wed, Feb 29, 2012 at 11:03 AM, Koichi Suzuki <koi...@gm...>
> wrote:
> > I agree that the suggestion looks better.
> >
> > With this modification, EXECUTE DIRECT will receive parsed tree, not a
> > text which should go straight to the target node.   EXECTE DIRECT
> > should rebuild the statement.   May the order of the predicate or
> > nesting parenthesis be different from the original input?   I hope
> > not.
>
> Just for the record, another mechanism that we had discussed in the
> past is to have some kind of schema for every node in the cluster. So
> user can schema qualify the objects to force run a query on a
> particular node. For example,
>
> SELECT * FROM dn1.table;
>

Instead of mixing with schema qualification syntax, we can also use SELECT
* from table@dn1.


>
> This would return all the tuples from the partition on data node dn1.
> So this would be equivalent to running:
>
> EXECUTE DIRECT ON dn1 'SELECT * FROM table';
>
> Once we add the capability of qualify the objects with nodes, user can
> write many powerful queries for administrative as well as other
> purposes.
>
>
Although, this means that such a query has to go through planner, which can
be non-performant. With EXEC DIRECT kind of syntax, we know that the query
does not need any planning at coordinator.


> Thanks,
> Pavan
>
> --
> Pavan Deolasee
> EnterpriseDB http://www.enterprisedb.com
>



-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

On Wed, Feb 29, 2012 at 11:28 AM, David E. Wheeler <da...@ki...> wrote:
> On Feb 28, 2012, at 9:47 PM, Pavan Deolasee wrote:
>
>> Just for the record, another mechanism that we had discussed in the
>> past is to have some kind of schema for every node in the cluster. So
>> user can schema qualify the objects to force run a query on a
>> particular node. For example,
>>
>> SELECT * FROM dn1.table;
>>
>> This would return all the tuples from the partition on data node dn1.
>> So this would be equivalent to running:
>
> That's awesome, but what if table is in a schema?
>
> I think you'd have to add another level of path, e.g.,
>
>    SELECT * FROM dn1.public.table;
>

Right. The node qualification will be one level above schema. I think
we may need to also think about the order in which we match nodes vs
schema, especially if user creates schema with the same name as node.
For example,

- SELECT * FROM dn1.test => if there is no schema with name dn1, we
try to match it to a node name and if found, run the query on that
node. The default name resolution mechanism will kick in node dn1

- SELECT * FROM dn1.test => if there is a schema with name dn1 and
there is also a node with the same name, the schema take precedence.
So the query runs on all the nodes

- SELECT * FROM dn1.dn1.test => the first dn1 will qualify node name
while the next dn1 will qualify schema name. So the query runs on dn1
and table named "test" in schema dn1.

Thanks,
Pavan

-- 
Pavan Deolasee
EnterpriseDB http://www.enterprisedb.com

On Wed, Feb 29, 2012 at 11:03 AM, Koichi Suzuki <koi...@gm...> wrote:
> I agree that the suggestion looks better.
>
> With this modification, EXECUTE DIRECT will receive parsed tree, not a
> text which should go straight to the target node.   EXECTE DIRECT
> should rebuild the statement.   May the order of the predicate or
> nesting parenthesis be different from the original input?   I hope
> not.

Just for the record, another mechanism that we had discussed in the
past is to have some kind of schema for every node in the cluster. So
user can schema qualify the objects to force run a query on a
particular node. For example,

SELECT * FROM dn1.table;

This would return all the tuples from the partition on data node dn1.
So this would be equivalent to running:

EXECUTE DIRECT ON dn1 'SELECT * FROM table';

Once we add the capability of qualify the objects with nodes, user can
write many powerful queries for administrative as well as other
purposes.

Thanks,
Pavan

-- 
Pavan Deolasee
EnterpriseDB http://www.enterprisedb.com

On 29 February 2012 11:03, Koichi Suzuki <koi...@gm...> wrote:
> I agree that the suggestion looks better.
>
> With this modification, EXECUTE DIRECT will receive parsed tree, not a
> text which should go straight to the target node.   EXECTE DIRECT
> should rebuild the statement.

We don't have to rebuild the sql string. We can easily save the sql
string snippet in the grammar file.

> May the order of the predicate or
> nesting parenthesis be different from the original input?   I hope
> not.
>
> Regards;
> --------
> Koichi Suzuki
>
>
>
> 2012/2/29 Ashutosh Bapat <ash...@en...>:
>> If that's possible +1. It will also simplify Execute Direct handling, which
>> needs an extra parsing step.
>>
>>
>> On Wed, Feb 29, 2012 at 6:11 AM, David E. Wheeler <da...@ju...>
>> wrote:
>>>
>>> PGXCers,
>>>
>>> Michael Paquier pointed out EXECUTE DIRECT to me today, which is quite
>>> handy. I'm wondering, however, if it might not be too late to change the
>>> parser to allow a raw query to be passed to it, rather than a string. That
>>> is, instead of this:
>>>
>>>    execute direct on node dn1 'select * from nodes';
>>>
>>> I'd like to do this:
>>>
>>>    execute direct on node dn1 select * from nodes;
>>>
>>> Note the lack of single quotes. It seems to me that there is a precedent
>>> in EXPLAIN for this. If it needs to support more parameters, perhaps they
>>> could be included in parentheses, again following EXPLAIN:
>>>
>>>    execute direct (node nd1, coordinator coord2) SELECT * FROM nodes;
>>>
>>> Thoughts?
>>>
>>> Thanks,
>>>
>>> David
>>>
>>> ------------------------------------------------------------------------------
>>> Virtualization & Cloud Management Using Capacity Planning
>>> Cloud computing makes use of virtualization - but cloud computing
>>> also focuses on allowing computing to be delivered as a service.
>>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>>> _______________________________________________
>>> Postgres-xc-developers mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>>
>>
>>
>> --
>> Best Wishes,
>> Ashutosh Bapat
>> EntepriseDB Corporation
>> The Enterprise Postgres Company
>>
>>
>> ------------------------------------------------------------------------------
>> Virtualization & Cloud Management Using Capacity Planning
>> Cloud computing makes use of virtualization - but cloud computing
>> also focuses on allowing computing to be delivered as a service.
>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers

On Feb 28, 2012, at 9:33 PM, Koichi Suzuki wrote:

> I agree that the suggestion looks better.
> 
> With this modification, EXECUTE DIRECT will receive parsed tree, not a
> text which should go straight to the target node.   EXECTE DIRECT
> should rebuild the statement.   May the order of the predicate or
> nesting parenthesis be different from the original input?   I hope
> not.

Is that what EXPLAIN does?

Best,

David

I agree that the suggestion looks better.

With this modification, EXECUTE DIRECT will receive parsed tree, not a
text which should go straight to the target node.   EXECTE DIRECT
should rebuild the statement.   May the order of the predicate or
nesting parenthesis be different from the original input?   I hope
not.

Regards;
--------
Koichi Suzuki



2012/2/29 Ashutosh Bapat <ash...@en...>:
> If that's possible +1. It will also simplify Execute Direct handling, which
> needs an extra parsing step.
>
>
> On Wed, Feb 29, 2012 at 6:11 AM, David E. Wheeler <da...@ju...>
> wrote:
>>
>> PGXCers,
>>
>> Michael Paquier pointed out EXECUTE DIRECT to me today, which is quite
>> handy. I'm wondering, however, if it might not be too late to change the
>> parser to allow a raw query to be passed to it, rather than a string. That
>> is, instead of this:
>>
>>    execute direct on node dn1 'select * from nodes';
>>
>> I'd like to do this:
>>
>>    execute direct on node dn1 select * from nodes;
>>
>> Note the lack of single quotes. It seems to me that there is a precedent
>> in EXPLAIN for this. If it needs to support more parameters, perhaps they
>> could be included in parentheses, again following EXPLAIN:
>>
>>    execute direct (node nd1, coordinator coord2) SELECT * FROM nodes;
>>
>> Thoughts?
>>
>> Thanks,
>>
>> David
>>
>> ------------------------------------------------------------------------------
>> Virtualization & Cloud Management Using Capacity Planning
>> Cloud computing makes use of virtualization - but cloud computing
>> also focuses on allowing computing to be delivered as a service.
>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EntepriseDB Corporation
> The Enterprise Postgres Company
>
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

If that's possible +1. It will also simplify Execute Direct handling, which
needs an extra parsing step.

On Wed, Feb 29, 2012 at 6:11 AM, David E. Wheeler <da...@ju...>wrote:

> PGXCers,
>
> Michael Paquier pointed out EXECUTE DIRECT to me today, which is quite
> handy. I'm wondering, however, if it might not be too late to change the
> parser to allow a raw query to be passed to it, rather than a string. That
> is, instead of this:
>
>    execute direct on node dn1 'select * from nodes';
>
> I'd like to do this:
>
>    execute direct on node dn1 select * from nodes;
>
> Note the lack of single quotes. It seems to me that there is a precedent
> in EXPLAIN for this. If it needs to support more parameters, perhaps they
> could be included in parentheses, again following EXPLAIN:
>
>    execute direct (node nd1, coordinator coord2) SELECT * FROM nodes;
>
> Thoughts?
>
> Thanks,
>
> David
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>



-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

> I could not reproduce this on my end. How many coordinators/data nodes
> you are running ? Also, does it happen every time for you or once in a
> while ?
>
OK, I have more precisions here.
It happens only after running regressions with 1 Coordinator and 2
Datanodes. On a fresh cluster it runs OK.
It looks that something got messed up. It may be related to the issues you
saw when you ran regressions several times in a row.
This is not reproducible on master, but I doubt it is directly related to
your patch.

Hmm.. this looks a bit weird. There should not have been more than 4
> connections because there are 5 Co. Seems like there is a bug in
> maintaining the connection list involved in the transaction. I may
> find difficult to reproduce this because I can't run so many
> coordinators and data nodes. But let me look at the code again and see
> if I can spot something.
>
I have run in the past such clusters even on small laptop. So it should be
OK if shared_buffers of each node is small.
However, I don't see this issue anymore today...
OK so let's do that. Once Sudou-san is back, I'll see with him for the
performance tests and if it works OK, let's commit this feature.
Regards,
-- 
Michael Paquier
http://michael.otacoo.com

PGXCers,

Michael Paquier pointed out EXECUTE DIRECT to me today, which is quite handy. I'm wondering, however, if it might not be too late to change the parser to allow a raw query to be passed to it, rather than a string. That is, instead of this:

    execute direct on node dn1 'select * from nodes';

I'd like to do this:

    execute direct on node dn1 select * from nodes;

Note the lack of single quotes. It seems to me that there is a precedent in EXPLAIN for this. If it needs to support more parameters, perhaps they could be included in parentheses, again following EXPLAIN:

    execute direct (node nd1, coordinator coord2) SELECT * FROM nodes;

Thoughts?

Thanks,

David

On Feb 27, 2012, at 5:49 PM, Koichi Suzuki wrote:

> As we got a report from David, we should consider to include *.c file
> generated from *.l and *.y file in release tar balls, as done in
> vanilla PostgreSQL.    XC has specific handling of sgml docs through
> *.sgmlin, to make merge easier from PostgreSQL docs.   Although *.sgml
> can be generated from *.sgmlin using makesgml as found in
> doc-xc/tools, it may help users if we have *.sgml (for XC, English)
> files in tarball.
> 
> Any thoughts?

I chatted with Andrew Dunstan about it yesterday. He said the distcheck target should handle it, and if not it can be patched to do so.

Best,

David

To make GTM active and reconnect from Apps, you need
coordinators/datanodes to connect GTM via gtm_proxy.    You may need
to tweak err_wait_count, err_wait_idle and err_wait_interval
configuration parameters for gtm_proxy to wait for "reconnect" command
form gtm_ctl.  (As a default, they're all set to zro).

Then, when you kill gtm, by gtm_ctl or whatsoever, you should issue
"promote" command to gtm standby via gtm_ctl, such as,
gtm_ctl promote -S gtm -D working_dir
This makes gtm standby to run as active gtm.

Next, you should tell all the gtm_proxies to reconnect to the new gtm such as
gtm_ctl reconnect -S gtm_proxy -D working_dir -o "-s new_gtm_hostname
-t new_gtm_port_number"

They will be found in the reference manual published at
http://postgres-xc.sourceforge.net/docs/0_9_7/

These procedure has not been made automatically.   This is an issue of
integration with HA middleware.   Writing RA for specific HA
middleware is regarded as separate project.   I welcome efforts to
write RAs (resource agents) for different HA middleware.

Best Regards;
----------
Koichi Suzuki



2012/2/29 Krzysztof Nowicki <krz...@gm...>:
> Hi,
>
> I tried a few times get reconnection to gtm standby server after gtm
> active failure but unfortunately I didn't succeed. When I kill or
> shutdown process of gtm active server the standby not catch traffic
> and on application side I get java.sql.SQLException: No active
> databases in cluster dataSource.
>
> For standby configuration I use only:
>
> nodename = 'gtm_standby'                                # Specifies
> the node name.
> port = 6666                             # Port number of this GTM.
> startup = STANDBY                               # Start mode. ACT/STANDBY.
>
> active_host = '10.178.228.145'                  # Listen address of active GTM.
> active_port = 6666                              # Port number of active GTM.
>
> Connection between GTM's seems to work:
>
> 815744:2012-02-28 15:40:23.510 UTC -LOG:  GTM standby is active. Going
> to connect.
> LOCATION:  gtm_standby_connect_to_standby_int, gtm_standby.c:390
> 1:140727810815744:2012-02-28 15:40:23.512 UTC -LOG:  Connection
> established with GTM standby. - 0x13d64b8
> LOCATION:  gtm_standby_connect_to_standby_int, gtm_standby.c:405
> 1:140727766959872:2012-02-28 15:40:23.512 UTC -LOG:  Sending
> transaction id 136318
> LOCATION:  ProcessBeginTransactionGetGXIDCommand, gtm_txn.c:1164
> 1:140727766959872:2012-02-28 15:40:23.513 UTC -LOG:  calling
> begin_transaction() for standby GTM 0x7ffdb40008d0.
> LOCATION:  ProcessBeginTransactionGetGXIDCommand, gtm_txn.c:1172
> 1:140727766959872:2012-02-28 15:40:23.513 UTC -LOG:  Committing
> transaction id 136318
> LOCATION:  ProcessCommitTransactionCommand, gtm_txn.c:1583
> 1:140727766959872:2012-02-28 15:40:23.513 UTC -LOG:  calling
> commit_transaction() for standby GTM 0x7ffdb40008d0.
> LOCATION:  ProcessCommitTransactionCommand, gtm_txn.c:1603
> 1:140727766959872:2012-02-28 15:40:23.513 UTC -LOG:
> commit_transaction() rc=0 done.
> LOCATION:  ProcessCommitTransactionCommand, gtm_txn.c:1615
> 1:140727766959872:2012-02-28 15:40:23.514 UTC -LOG:  Cleaning up thread state
> LOCATION:  GTM_ThreadCleanup, gtm_thread.c:265
> 1:140727766959872:2012-02-28 15:40:23.514 UTC -LOG:  Closing a
> connection to the GTM standby
>
> Although on standby server I often get error:
>
>  UTC -ERROR:  insufficient data left in message
>
> Is there any tutorial how GTM Standby should be configured and how it
> is working?
>
> Br,
>
> Kris Nowicki
>
> ------------------------------------------------------------------------------
> Keep Your Developer Skills Current with LearnDevNow!
> The most comprehensive online learning library for Microsoft developers
> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
> Metro Style Apps, more. Free future releases when you subscribe now!
> http://p.sf.net/sfu/learndevnow-d2d
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers

Hi,

I tried a few times get reconnection to gtm standby server after gtm
active failure but unfortunately I didn't succeed. When I kill or
shutdown process of gtm active server the standby not catch traffic
and on application side I get java.sql.SQLException: No active
databases in cluster dataSource.

For standby configuration I use only:

nodename = 'gtm_standby'                                # Specifies
the node name.
port = 6666                             # Port number of this GTM.
startup = STANDBY                               # Start mode. ACT/STANDBY.

active_host = '10.178.228.145'                  # Listen address of active GTM.
active_port = 6666                              # Port number of active GTM.

Connection between GTM's seems to work:

815744:2012-02-28 15:40:23.510 UTC -LOG:  GTM standby is active. Going
to connect.
LOCATION:  gtm_standby_connect_to_standby_int, gtm_standby.c:390
1:140727810815744:2012-02-28 15:40:23.512 UTC -LOG:  Connection
established with GTM standby. - 0x13d64b8
LOCATION:  gtm_standby_connect_to_standby_int, gtm_standby.c:405
1:140727766959872:2012-02-28 15:40:23.512 UTC -LOG:  Sending
transaction id 136318
LOCATION:  ProcessBeginTransactionGetGXIDCommand, gtm_txn.c:1164
1:140727766959872:2012-02-28 15:40:23.513 UTC -LOG:  calling
begin_transaction() for standby GTM 0x7ffdb40008d0.
LOCATION:  ProcessBeginTransactionGetGXIDCommand, gtm_txn.c:1172
1:140727766959872:2012-02-28 15:40:23.513 UTC -LOG:  Committing
transaction id 136318
LOCATION:  ProcessCommitTransactionCommand, gtm_txn.c:1583
1:140727766959872:2012-02-28 15:40:23.513 UTC -LOG:  calling
commit_transaction() for standby GTM 0x7ffdb40008d0.
LOCATION:  ProcessCommitTransactionCommand, gtm_txn.c:1603
1:140727766959872:2012-02-28 15:40:23.513 UTC -LOG:
commit_transaction() rc=0 done.
LOCATION:  ProcessCommitTransactionCommand, gtm_txn.c:1615
1:140727766959872:2012-02-28 15:40:23.514 UTC -LOG:  Cleaning up thread state
LOCATION:  GTM_ThreadCleanup, gtm_thread.c:265
1:140727766959872:2012-02-28 15:40:23.514 UTC -LOG:  Closing a
connection to the GTM standby

Although on standby server I often get error:

 UTC -ERROR:  insufficient data left in message

Is there any tutorial how GTM Standby should be configured and how it
is working?

Br,

Kris Nowicki

Hi Michael,

Thanks for testing the patch. I am happy that regression is better
now. As I discussed on the call, I know there is another bug lingering
around the serializable xact.

On Tue, Feb 28, 2012 at 9:37 AM, Michael Paquier
<mic...@gm...> wrote:

>
> It happens for the test case rowtypes, and I extracted the test case:
> BEGIN;
> CREATE TABLE price (
>     id SERIAL PRIMARY KEY,
>     active BOOLEAN NOT NULL,
>     price NUMERIC
> );
> CREATE TYPE price_input AS (
>     id INTEGER,
>     price NUMERIC
> );
> CREATE TYPE price_key AS (
>     id INTEGER
> );
> CREATE FUNCTION price_key_from_table(price) RETURNS price_key AS $$
>     SELECT $1.id
> $$ LANGUAGE SQL;
> CREATE FUNCTION price_key_from_input(price_input) RETURNS price_key AS $$
>     SELECT $1.id
> $$ LANGUAGE SQL;
> insert into price values (1,false,42), (10,false,100), (11,true,17.99);
> UPDATE price
>     SET active = true, price = input_prices.price
>     FROM unnest(ARRAY[(10, 123.00), (11, 99.99)]::price_input[])
> input_prices
>     WHERE price_key_from_table(price.*) =
> price_key_from_input(input_prices.*);
> select * from price;
> rollback; --assertion error here
>

I could not reproduce this on my end. How many coordinators/data nodes
you are running ? Also, does it happen every time for you or once in a
while ?

> 2) A coordinator crash with 5Co/5Dn
> postgres=# psql -X -c "create database toto" postgres
> postgres-# \q
> michael@boheme:~/pgsql $ psql -X -c "create database toto" postgres
> CREATE DATABASE
> michael@boheme:~/pgsql $ psql toto
>
> psql (9.1beta2)
> Type "help" for help.
> toto=# create table aa (a int);
> server closed the connection unexpectedly
>     This probably means the server terminated abnormally
>     before or while processing the request.
> The connection to the server was lost. Attempting reset: Failed.
> !> \q
>
> #0  0x000000000065127a in pgxc_node_remote_prepare (prepareGID=0x2076380
> "T10056") at execRemote.c:1705
> 1705            if (connections[i]->state != DN_CONNECTION_STATE_IDLE)
> (gdb) bt
> #0  0x000000000065127a in pgxc_node_remote_prepare (prepareGID=0x2076380
> "T10056") at execRemote.c:1705
> #1  0x0000000000657008 in PrePrepare_Remote (prepareGID=0x2076380 "T10056",
> localNode=1 '\001', implicit=1 '\001') at execRemote.c:4561
> #2  0x00000000004b91dc in PrepareTransaction () at xact.c:2353
> #3  0x00000000004b8c54 in CommitTransaction () at xact.c:2005
> #4  0x00000000004b9a6a in CommitTransactionCommand () at xact.c:2897
> #5  0x0000000000753f8e in finish_xact_command () at postgres.c:2572
> #6  0x00000000007518e9 in exec_simple_query (query_string=0x1fcd6c8 "create
> table aa (a int);") at postgres.c:1136
> #7  0x0000000000755f10 in PostgresMain (argc=2, argv=0x1f11de8,
> username=0x1f11c30 "michael") at postgres.c:4155
> #8  0x00000000006fc1ea in BackendRun (port=0x1f40e10) at postmaster.c:3763
> #9  0x00000000006fb86a in BackendStartup (port=0x1f40e10) at
> postmaster.c:3448
> #10 0x00000000006f87b6 in ServerLoop () at postmaster.c:1539
> #11 0x00000000006f7f57 in PostmasterMain (argc=7, argv=0x1f0eba0) at
> postmaster.c:1200
> #12 0x0000000000662b39 in main (argc=7, argv=0x1f0eba0) at main.c:199
> (gdb) p i
> $2 = 4
> (gdb) p connections
> $3 = (PGXCNodeHandle **) 0x1f2ca40
> (gdb) p connections[0]
> $4 = (PGXCNodeHandle *) 0x2007638
> (gdb) p *connections[0]
> $5 = {nodeoid = 16384, sock = 31, transaction_status = 84 'T', state =
> DN_CONNECTION_STATE_QUERY, combiner = 0x0, error = 0x0, outBuffer =
> 0x2019568 "Q", outSize = 16384, outEnd = 0,
>   inBuffer = 0x201d5a8 "C", inSize = 16384, inStart = 24, inEnd = 24,
> inCursor = 24}
> (gdb) p *connections[1]
> $6 = {nodeoid = 16385, sock = 41, transaction_status = 84 'T', state =
> DN_CONNECTION_STATE_QUERY, combiner = 0x0, error = 0x0, outBuffer =
> 0x20215e8 "Q", outSize = 16384, outEnd = 0,
>   inBuffer = 0x2025628 "C", inSize = 16384, inStart = 24, inEnd = 24,
> inCursor = 24}
> (gdb) p *connections[2]
> $7 = {nodeoid = 16386, sock = 42, transaction_status = 84 'T', state =
> DN_CONNECTION_STATE_QUERY, combiner = 0x0, error = 0x0, outBuffer =
> 0x2029668 "Q", outSize = 16384, outEnd = 0,
>   inBuffer = 0x202d6a8 "C", inSize = 16384, inStart = 24, inEnd = 24,
> inCursor = 24}
> (gdb) p *connections[3]
> $8 = {nodeoid = 16387, sock = 43, transaction_status = 84 'T', state =
> DN_CONNECTION_STATE_QUERY, combiner = 0x0, error = 0x0, outBuffer =
> 0x20316e8 "Q", outSize = 16384, outEnd = 0,
>   inBuffer = 0x2035728 "C", inSize = 16384, inStart = 24, inEnd = 24,
> inCursor = 24}
> (gdb) p *connections[4]
> Cannot access memory at address 0x100000001
>

Hmm.. this looks a bit weird. There should not have been more than 4
connections because there are 5 Co. Seems like there is a bug in
maintaining the connection list involved in the transaction. I may
find difficult to reproduce this because I can't run so many
coordinators and data nodes. But let me look at the code again and see
if I can spot something.

Thanks for your efforts.

Thanks,
Pavan

-- 
Pavan Deolasee
EnterpriseDB http://www.enterprisedb.com

Thanks for your input.
This looks good for commit.

On Tue, Feb 28, 2012 at 2:05 PM, Amit Khandekar <
ami...@en...> wrote:

> I had not committed the doc changes for this. Attached is a patch that
> changes the doc in the following places:
>
> For set_config(), it is now mentioned that set_config behaves like SET,
> and propagates the parameter value to all nodes.
>
> For advisory locks, removed earlier comments about their limitation of
> local node scope, and added explicit mention that it is now cluster-aware.
>
> -Amit
>
>
> On 7 February 2012 08:00, Amit Khandekar <ami...@en...>wrote:
>
>> Attached is the patch that implements transaction level lock
>> functions. As mentioned, keeping these functions at the coordinator
>> level sufficed; pushing them further to data nodes was not required
>> because coordinators are aware of the transactions.
>>
>> On 1 February 2012 17:30, Amit Khandekar
>> <ami...@en...> wrote:
>> >
>> > Now that  session level locks implementation is checked in, here are
>> some notes on implementation of transaction level locks ... (Just archiving
>> my thoughts here).
>> >
>> > Transaction level locks do not have an explicit way to unlock. They are
>> released once the transaction ends.
>> >
>> > Similar to session level locks, we need to propagate the
>> pg_advisory_xact_lock function to other nodes to make them cluster-aware.
>> But should we propagate to data nodes or coordinators?
>> >
>> > I was earlier of the opinion that we should propagate on data nodes,
>> but now I feel the propagation can be kept to coordinators because they are
>> aware of the transactions. I realized that besides datanodes, even the
>> coordinator keeps a local transaction active until user ends the
>> transaction.
>> >
>> > Is it necessary to  propagate to data nodes also, besides coordinators?
>> I don't think it is necessary. Because all advisory lock commands have to
>> go through coordinators. And these locks do not share the same space with
>> implicit database object locks which are held on datanodes when a table is
>> referenced in a query.
>> >
>> > The LOCK <table> command is implemented by propagating to both
>> coordinator and data nodes, which is necessary because the lock held on a
>> table using LOCK command shares the same space with implicit table locks
>> held when a query accesses tables. So, when a query like "select * from
>> tab1" is executed in a transaction on data node, it should wait for a lock
>> held on the table using LOCK tab1. And for that, this explicit lock should
>> be propagated onto data nodes, otherwise the query won't see this lock.
>> >
>> >
>> > Will the coordinator-to-coordinator connection pooling have the same
>> unlocking issues like session level locks? No, because the the transaction
>> locks are never unlocked explicitly.
>> >
>> > Will the coord-to-coord pooler connection keep the lock active even
>> after the user ends transaction ? No, because when a user transaction ends,
>> the pooler connection also ends its transaction although it does not end
>> the connection. When there are two simultaneous transactions from the same
>> coordinator, the coordinator creates two pooler connections to the remote
>> coordinator. The advisory locks on the two transactions will be held on
>> these two pooler connections until the transactions end.
>> >
>> > So effectively, there does not look to be much more work for
>> transaction level locks except that they keep all the coord-coord locks
>> active, they don't unlock the remote locks unlike session-level locks.
>> Attached patch does this for a sample function. All synchronization issues
>> considered for session level locks apply here also.
>> >
>> >
>> > Will send the complete patch after more testing.
>> >
>> >
>> > On 30 January 2012 17:25, Amit Khandekar <
>> ami...@en...> wrote:
>> >>
>> >> On 20 January 2012 17:50, Amit Khandekar
>> >> <ami...@en...> wrote:
>> >> > Before going into implementation issues that I thought of on
>> advisory locks
>> >> > , here's a PG doc snippet briefly explaining what are they:
>> >> >
>> >> > "There are two different types of advisory locks in PostgreSQL:
>> session
>> >> > level and transaction level. Once acquired, a session level advisory
>> lock is
>> >> > held until explicitly released or the session ends. Unlike standard
>> locks,
>> >> > session level advisory locks do not honor transaction semantics: a
>> lock
>> >> > acquired during a transaction that is later rolled back will still
>> be held
>> >> > following the rollback, and likewise an unlock is effective even if
>> the
>> >> > calling transaction fails later. The same session level lock can be
>> acquired
>> >> > multiple times by its owning process: for each lock request there
>> must be a
>> >> > corresponding unlock request before the lock is actually released.
>> (If a
>> >> > session already holds a given lock, additional requests will always
>> succeed,
>> >> > even if other sessions are awaiting the lock.) Transaction level
>> locks on
>> >> > the other hand behave more like regular locks; they are automatically
>> >> > released at the end of the transaction, and can not be explicitly
>> unlocked."
>> >> >
>> >> >
>> >> > Implementation notes for session level locks.
>> >> > ----------------------------
>> >> >
>> >> > In a nutshell this is how they work :
>> >> >
>> >> > Application 1 calls select pg_advisory_lock(100)
>> >> > Application 2 calls select pg_advisory_lock(100)  and waits for app
>> 1 to
>> >> > call select pg_advisory_unlock(100) or to end the session.
>> >> >
>> >> >
>> >> > Goal is to make sure that when a client from coordinator C1 locks on
>> a key,
>> >> > another client from coordinator C2 should wait on the lock. THe
>> >> > advisory_locks calls from the session get stacked, that means the
>> resource
>> >> > won't get released until it is unlocked as many times as it has
>> locked. So
>> >> > pg_advisory_lock() 3 times should be followed by
>> pg_advisory_unlock() 3
>> >> > times for the resource to be freed for other sessions.
>> >> >
>> >> > One simple way is to just send a parallel 'exec direct $$select
>> >> > pg_advisory_lock()$$ '  on all coordinators.
>> >> >
>> >> > Sequence of actions:
>> >> >
>> >> > 1. Client1 from C1 calls :
>> >> > select pg_advisory_lock(100)
>> >> > THis call will be propogated to all coords (C1 and C2) so there will
>> be
>> >> > locks held on both C1 and C2. C1 lock is native lock, and the lock
>> on C2 is
>> >> > through C1's pooler connection to C2.
>> >> >
>> >> > 2. Next client2 from C2 calls :
>> >> > select pg_advisory_lock(100)
>> >> > This will again call pg_advisory_lock() on C1 and C2. Both of these
>> calls
>> >> > would wait because client1 has held locks. so effectively the
>> client2 will
>> >> > wait on this call.
>> >> >
>> >> > 3a. Next client1 calls pg_advisory_unlock().
>> >> >     This will call pg_advisory_unlock() on C1 and C2, unlocking the
>> earlier
>> >> > held locks.
>> >> >     THis will in turn make client2 return from its
>> pg_advisory_lock() call.
>> >> > OR
>> >> >
>> >> > 3b. Client 1 exits without calling pg_advisory_unlock().
>> >> > This will automatically release the lock held on the native
>> coordinator C1.
>> >> > But the one held on C2 will not be released because that lock is
>> held on the
>> >> > pooler session to C2. So here we are in trouble. This C2 lock will
>> be held
>> >> > there permanently until the pooler exists or someone explicitly calls
>> >> > pg_advisory_unlock, and so client2 waiting on this lock will hang
>> >> > indefintely.
>> >> >
>> >> > To handle this issue with pooler connection, the pg_advisory_lock()
>> >> > definition should immediately unlock all locks except the native
>> coordinator
>> >> > lock. But still these lock-unlock calls on remote coordinator are
>> necessary
>> >> > because it should get a chance to wait for a resource in case already
>> >> > someone else has held the resource from some other coordinator.
>> >> >
>> >> > pg_advisory_lock ()
>> >> > {
>> >> >     /*
>> >> >      * Go on locking on each coordinator. Keep on unlocking the
>> previous one
>> >> >      * each time a new lock is held. Don't unlock the native
>> coordinator.
>> >> >      * After finishing all coordinators, ultimately only the native
>> >> > coordinator
>> >> >      * would be held, but still we will have scanned all
>> coordinators to
>> >> > make
>> >> >      * sure no one else is already grabbed the lock.
>> >> >      */
>> >> >     for (i = 0; i < last_coordinator_index; i++)
>> >> >     {
>> >> >         Call pg_advisory_lock() on coordinator[i];
>> >> >         if (i > 0 && !is_native(coordinator[i-1]) )
>> >> >             Call pg_advisory_unlock() on coordinator[i-1];
>> >> >     }
>> >> >
>> >> >
>> >> > }
>> >> >
>> >> >     Note that the order of locking all coordinators must strictly be
>> common
>> >> > to all. If client1 from C1 locks C1 first and then C2, and in
>> parallel
>> >> > client2 from C2 locks C2 first and then C1, there would arise the
>> typical
>> >> > synchronisation issues. For e.g., simultaneously client1 and client2
>> call
>> >> > pg_advisory_lock() on C1 and C2 respectively. They won't wait
>> because they
>> >> > are on different coordinators. NExt, client1 and client2 both would
>> >> > simultaneously try to lock on C2 and C1 resp. , and both would wait
>> on each
>> >> > other. This deadlock would not arise if both lock with a common
>> order, say
>> >> > C1 then C2.
>> >> >
>> >> > And that's the reason lock function cannot be propogated to all
>> coordinators
>> >> > in parallel because that way we cannot gurantee the order of locking.
>> >> >
>> >> >
>> >>
>> >> Based on the implementation notes, attached is a patch that covers
>> >> session level advisory locks. A common function pgxc_advisory_locks()
>> >> is used to implement the common algorithm used by all variants of
>> >> pg_advisory_lock function. The algorithm is based keeping in mind the
>> >> above issue that I had discussed.
>> >>
>> >> Additionally, I had to tweak a bit the existing LockAcquire() function
>> >> in locks.c. When a user from the same session calls advisory lock a
>> >> second time, then it should not propogate the call to all
>> >> coordinators, rather it should just increment the lock reference
>> >> locally. So it was needed to check if the lock already exists on the
>> >> same session, and then increment it, else just return without locking.
>> >> This specific behaviour was not present in existing LockAcquire()
>> >> function, so I made it support the same.
>> >>
>> >> Tweaked pgxc_execute_on_nodes() function that was written for object
>> >> size functions so that it can be reused for advisory lock functions.
>> >>
>> >> I was planning to include both session level and transaction level
>> >> locking functions, but looks like transaction level functions have to
>> >> be handled differently, so I am working on it, and will send it using
>> >> a separate patch.
>> >>
>> >>
>> >> >
>> >> > Implementation notes for transaction  level locks.
>> >> > --------------------------------------------------------
>> >> >
>> >> > Yet to think on this, but for this, I think we need to propogate the
>> calls
>> >> > to datanode rather than coordinator. Because datanodes are the ones
>> who
>> >> > handle transactions. These locks cannot be unlocked. They only
>> unlock at end
>> >> > of transmission.
>> >
>> >
>>
>
>
>
> ------------------------------------------------------------------------------
> Keep Your Developer Skills Current with LearnDevNow!
> The most comprehensive online learning library for Microsoft developers
> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
> Metro Style Apps, more. Free future releases when you subscribe now!
> http://p.sf.net/sfu/learndevnow-d2d
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>


-- 
Michael Paquier
http://michael.otacoo.com

I had not committed the doc changes for this. Attached is a patch that
changes the doc in the following places:

For set_config(), it is now mentioned that set_config behaves like SET, and
propagates the parameter value to all nodes.

For advisory locks, removed earlier comments about their limitation of
local node scope, and added explicit mention that it is now cluster-aware.

-Amit


On 7 February 2012 08:00, Amit Khandekar <ami...@en...>wrote:

> Attached is the patch that implements transaction level lock
> functions. As mentioned, keeping these functions at the coordinator
> level sufficed; pushing them further to data nodes was not required
> because coordinators are aware of the transactions.
>
> On 1 February 2012 17:30, Amit Khandekar
> <ami...@en...> wrote:
> >
> > Now that  session level locks implementation is checked in, here are
> some notes on implementation of transaction level locks ... (Just archiving
> my thoughts here).
> >
> > Transaction level locks do not have an explicit way to unlock. They are
> released once the transaction ends.
> >
> > Similar to session level locks, we need to propagate the
> pg_advisory_xact_lock function to other nodes to make them cluster-aware.
> But should we propagate to data nodes or coordinators?
> >
> > I was earlier of the opinion that we should propagate on data nodes, but
> now I feel the propagation can be kept to coordinators because they are
> aware of the transactions. I realized that besides datanodes, even the
> coordinator keeps a local transaction active until user ends the
> transaction.
> >
> > Is it necessary to  propagate to data nodes also, besides coordinators?
> I don't think it is necessary. Because all advisory lock commands have to
> go through coordinators. And these locks do not share the same space with
> implicit database object locks which are held on datanodes when a table is
> referenced in a query.
> >
> > The LOCK <table> command is implemented by propagating to both
> coordinator and data nodes, which is necessary because the lock held on a
> table using LOCK command shares the same space with implicit table locks
> held when a query accesses tables. So, when a query like "select * from
> tab1" is executed in a transaction on data node, it should wait for a lock
> held on the table using LOCK tab1. And for that, this explicit lock should
> be propagated onto data nodes, otherwise the query won't see this lock.
> >
> >
> > Will the coordinator-to-coordinator connection pooling have the same
> unlocking issues like session level locks? No, because the the transaction
> locks are never unlocked explicitly.
> >
> > Will the coord-to-coord pooler connection keep the lock active even
> after the user ends transaction ? No, because when a user transaction ends,
> the pooler connection also ends its transaction although it does not end
> the connection. When there are two simultaneous transactions from the same
> coordinator, the coordinator creates two pooler connections to the remote
> coordinator. The advisory locks on the two transactions will be held on
> these two pooler connections until the transactions end.
> >
> > So effectively, there does not look to be much more work for transaction
> level locks except that they keep all the coord-coord locks active, they
> don't unlock the remote locks unlike session-level locks. Attached patch
> does this for a sample function. All synchronization issues considered for
> session level locks apply here also.
> >
> >
> > Will send the complete patch after more testing.
> >
> >
> > On 30 January 2012 17:25, Amit Khandekar <
> ami...@en...> wrote:
> >>
> >> On 20 January 2012 17:50, Amit Khandekar
> >> <ami...@en...> wrote:
> >> > Before going into implementation issues that I thought of on advisory
> locks
> >> > , here's a PG doc snippet briefly explaining what are they:
> >> >
> >> > "There are two different types of advisory locks in PostgreSQL:
> session
> >> > level and transaction level. Once acquired, a session level advisory
> lock is
> >> > held until explicitly released or the session ends. Unlike standard
> locks,
> >> > session level advisory locks do not honor transaction semantics: a
> lock
> >> > acquired during a transaction that is later rolled back will still be
> held
> >> > following the rollback, and likewise an unlock is effective even if
> the
> >> > calling transaction fails later. The same session level lock can be
> acquired
> >> > multiple times by its owning process: for each lock request there
> must be a
> >> > corresponding unlock request before the lock is actually released.
> (If a
> >> > session already holds a given lock, additional requests will always
> succeed,
> >> > even if other sessions are awaiting the lock.) Transaction level
> locks on
> >> > the other hand behave more like regular locks; they are automatically
> >> > released at the end of the transaction, and can not be explicitly
> unlocked."
> >> >
> >> >
> >> > Implementation notes for session level locks.
> >> > ----------------------------
> >> >
> >> > In a nutshell this is how they work :
> >> >
> >> > Application 1 calls select pg_advisory_lock(100)
> >> > Application 2 calls select pg_advisory_lock(100)  and waits for app 1
> to
> >> > call select pg_advisory_unlock(100) or to end the session.
> >> >
> >> >
> >> > Goal is to make sure that when a client from coordinator C1 locks on
> a key,
> >> > another client from coordinator C2 should wait on the lock. THe
> >> > advisory_locks calls from the session get stacked, that means the
> resource
> >> > won't get released until it is unlocked as many times as it has
> locked. So
> >> > pg_advisory_lock() 3 times should be followed by pg_advisory_unlock()
> 3
> >> > times for the resource to be freed for other sessions.
> >> >
> >> > One simple way is to just send a parallel 'exec direct $$select
> >> > pg_advisory_lock()$$ '  on all coordinators.
> >> >
> >> > Sequence of actions:
> >> >
> >> > 1. Client1 from C1 calls :
> >> > select pg_advisory_lock(100)
> >> > THis call will be propogated to all coords (C1 and C2) so there will
> be
> >> > locks held on both C1 and C2. C1 lock is native lock, and the lock on
> C2 is
> >> > through C1's pooler connection to C2.
> >> >
> >> > 2. Next client2 from C2 calls :
> >> > select pg_advisory_lock(100)
> >> > This will again call pg_advisory_lock() on C1 and C2. Both of these
> calls
> >> > would wait because client1 has held locks. so effectively the client2
> will
> >> > wait on this call.
> >> >
> >> > 3a. Next client1 calls pg_advisory_unlock().
> >> >     This will call pg_advisory_unlock() on C1 and C2, unlocking the
> earlier
> >> > held locks.
> >> >     THis will in turn make client2 return from its pg_advisory_lock()
> call.
> >> > OR
> >> >
> >> > 3b. Client 1 exits without calling pg_advisory_unlock().
> >> > This will automatically release the lock held on the native
> coordinator C1.
> >> > But the one held on C2 will not be released because that lock is held
> on the
> >> > pooler session to C2. So here we are in trouble. This C2 lock will be
> held
> >> > there permanently until the pooler exists or someone explicitly calls
> >> > pg_advisory_unlock, and so client2 waiting on this lock will hang
> >> > indefintely.
> >> >
> >> > To handle this issue with pooler connection, the pg_advisory_lock()
> >> > definition should immediately unlock all locks except the native
> coordinator
> >> > lock. But still these lock-unlock calls on remote coordinator are
> necessary
> >> > because it should get a chance to wait for a resource in case already
> >> > someone else has held the resource from some other coordinator.
> >> >
> >> > pg_advisory_lock ()
> >> > {
> >> >     /*
> >> >      * Go on locking on each coordinator. Keep on unlocking the
> previous one
> >> >      * each time a new lock is held. Don't unlock the native
> coordinator.
> >> >      * After finishing all coordinators, ultimately only the native
> >> > coordinator
> >> >      * would be held, but still we will have scanned all coordinators
> to
> >> > make
> >> >      * sure no one else is already grabbed the lock.
> >> >      */
> >> >     for (i = 0; i < last_coordinator_index; i++)
> >> >     {
> >> >         Call pg_advisory_lock() on coordinator[i];
> >> >         if (i > 0 && !is_native(coordinator[i-1]) )
> >> >             Call pg_advisory_unlock() on coordinator[i-1];
> >> >     }
> >> >
> >> >
> >> > }
> >> >
> >> >     Note that the order of locking all coordinators must strictly be
> common
> >> > to all. If client1 from C1 locks C1 first and then C2, and in parallel
> >> > client2 from C2 locks C2 first and then C1, there would arise the
> typical
> >> > synchronisation issues. For e.g., simultaneously client1 and client2
> call
> >> > pg_advisory_lock() on C1 and C2 respectively. They won't wait because
> they
> >> > are on different coordinators. NExt, client1 and client2 both would
> >> > simultaneously try to lock on C2 and C1 resp. , and both would wait
> on each
> >> > other. This deadlock would not arise if both lock with a common
> order, say
> >> > C1 then C2.
> >> >
> >> > And that's the reason lock function cannot be propogated to all
> coordinators
> >> > in parallel because that way we cannot gurantee the order of locking.
> >> >
> >> >
> >>
> >> Based on the implementation notes, attached is a patch that covers
> >> session level advisory locks. A common function pgxc_advisory_locks()
> >> is used to implement the common algorithm used by all variants of
> >> pg_advisory_lock function. The algorithm is based keeping in mind the
> >> above issue that I had discussed.
> >>
> >> Additionally, I had to tweak a bit the existing LockAcquire() function
> >> in locks.c. When a user from the same session calls advisory lock a
> >> second time, then it should not propogate the call to all
> >> coordinators, rather it should just increment the lock reference
> >> locally. So it was needed to check if the lock already exists on the
> >> same session, and then increment it, else just return without locking.
> >> This specific behaviour was not present in existing LockAcquire()
> >> function, so I made it support the same.
> >>
> >> Tweaked pgxc_execute_on_nodes() function that was written for object
> >> size functions so that it can be reused for advisory lock functions.
> >>
> >> I was planning to include both session level and transaction level
> >> locking functions, but looks like transaction level functions have to
> >> be handled differently, so I am working on it, and will send it using
> >> a separate patch.
> >>
> >>
> >> >
> >> > Implementation notes for transaction  level locks.
> >> > --------------------------------------------------------
> >> >
> >> > Yet to think on this, but for this, I think we need to propogate the
> calls
> >> > to datanode rather than coordinator. Because datanodes are the ones
> who
> >> > handle transactions. These locks cannot be unlocked. They only unlock
> at end
> >> > of transmission.
> >
> >
>

Hi Pavan,

I am able to run regressions. However there are still issues:

1) A datanode crash
(gdb) bt
#0  0x00007f2a555afa75 in raise () from /lib/libc.so.6
#1  0x00007f2a555b35c0 in abort () from /lib/libc.so.6
#2  0x0000000000857bd6 in ExceptionalCondition (conditionName=0xa08c50
"!(((proc->xid) != ((TransactionId) 0)))", errorType=0xa08c40
"FailedAssertion", fileName=0xa08c14 "procarray.c", lineNumber=440)
    at assert.c:57
#3  0x000000000072fe0a in ProcArrayEndTransaction (proc=0x7f2a55135ac0,
latestXid=23666) at procarray.c:440
#4  0x00000000004b96dd in AbortTransaction () at xact.c:2706
#5  0x00000000004b9e49 in AbortCurrentTransaction () at xact.c:3191
#6  0x0000000000755cff in PostgresMain (argc=2, argv=0xdd9d90,
username=0xdd9c30 "michael") at postgres.c:4010
#7  0x00000000006fc1ea in BackendRun (port=0xe08fd0) at postmaster.c:3763
#8  0x00000000006fb86a in BackendStartup (port=0xe08fd0) at
postmaster.c:3448
#9  0x00000000006f87b6 in ServerLoop () at postmaster.c:1539
#10 0x00000000006f7f57 in PostmasterMain (argc=7, argv=0xdd6ba0) at
postmaster.c:1200
#11 0x0000000000662b39 in main (argc=7, argv=0xdd6ba0) at main.c:199
(gdb) up 3
#3  0x000000000072fe0a in ProcArrayEndTransaction (proc=0x7f2a55135ac0,
latestXid=23666) at procarray.c:440
440            Assert(TransactionIdIsValid(proc->xid));
(gdb) p proc
$1 = (PGPROC *) 0x7f2a55135ac0
(gdb) p *proc
$2 = {links = {prev = 0x0, next = 0x0}, sem = {semId = 26378254, semNum =
0}, waitStatus = 0, lxid = 0, xid = 0, xmin = 0, pid = 18594, backendId =
4, databaseId = 16384, roleId = 10, inCommit = 0 '\000',
  vacuumFlags = 0 '\000', recoveryConflictPending = 0 '\000', isPooler = 0
'\000', lwWaiting = 0 '\000', lwExclusive = 1 '\001', lwWaitLink = 0x0,
waitLock = 0x0, waitProcLock = 0x0, waitLockMode = 0,
  heldLocks = 0, waitLatch = {is_set = 1, is_shared = 1 '\001', owner_pid =
18594}, waitLSN = {xlogid = 0, xrecoff = 0}, syncRepState = 0, syncRepLinks
= {prev = 0x0, next = 0x0}, myProcLocks = {{
      prev = 0x7f2a55135b48, next = 0x7f2a55135b48}, {prev =
0x7f2a55135b58, next = 0x7f2a55135b58}, {prev = 0x7f2a55135b68, next =
0x7f2a55135b68}, {prev = 0x7f2a55135b78, next = 0x7f2a55135b78}, {
      prev = 0x7f2a55135b88, next = 0x7f2a55135b88}, {prev =
0x7f2a55135b98, next = 0x7f2a55135b98}, {prev = 0x7f2a55135ba8, next =
0x7f2a55135ba8}, {prev = 0x7f2a55135bb8, next = 0x7f2a55135bb8}, {
      prev = 0x7f2a55135bc8, next = 0x7f2a55135bc8}, {prev =
0x7f2a55135bd8, next = 0x7f2a55135bd8}, {prev = 0x7f2a55135be8, next =
0x7f2a55135be8}, {prev = 0x7f2a55135bf8, next = 0x7f2a55135bf8}, {
      prev = 0x7f2a55135c08, next = 0x7f2a55135c08}, {prev =
0x7f2a55135c18, next = 0x7f2a55135c18}, {prev = 0x7f2a55135c28, next =
0x7f2a55135c28}, {prev = 0x7f2a55135c38, next = 0x7f2a55135c38}},
  subxids = {overflowed = 0 '\000', nxids = 0, xids = {0 <repeats 64
times>}}}
(gdb) p *proc->xid
Cannot access memory at address 0x0
(gdb) p proc->xid
$3 = 0

It happens for the test case rowtypes, and I extracted the test case:
BEGIN;
CREATE TABLE price (
    id SERIAL PRIMARY KEY,
    active BOOLEAN NOT NULL,
    price NUMERIC
);
CREATE TYPE price_input AS (
    id INTEGER,
    price NUMERIC
);
CREATE TYPE price_key AS (
    id INTEGER
);
CREATE FUNCTION price_key_from_table(price) RETURNS price_key AS $$
    SELECT $1.id
$$ LANGUAGE SQL;
CREATE FUNCTION price_key_from_input(price_input) RETURNS price_key AS $$
    SELECT $1.id
$$ LANGUAGE SQL;
insert into price values (1,false,42), (10,false,100), (11,true,17.99);
UPDATE price
    SET active = true, price = input_prices.price
    FROM unnest(ARRAY[(10, 123.00), (11, 99.99)]::price_input[])
input_prices
    WHERE price_key_from_table(price.*) =
price_key_from_input(input_prices.*);
select * from price;
rollback; --assertion error here

2) A coordinator crash with 5Co/5Dn
postgres=# psql -X -c "create database toto" postgres
postgres-# \q
michael@boheme:~/pgsql $ psql -X -c "create database toto" postgres
CREATE DATABASE
michael@boheme:~/pgsql $ psql toto
psql (9.1beta2)
Type "help" for help.
toto=# create table aa (a int);
server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!> \q

#0  0x000000000065127a in pgxc_node_remote_prepare (prepareGID=0x2076380
"T10056") at execRemote.c:1705
1705            if (connections[i]->state != DN_CONNECTION_STATE_IDLE)
(gdb) bt
#0  0x000000000065127a in pgxc_node_remote_prepare (prepareGID=0x2076380
"T10056") at execRemote.c:1705
#1  0x0000000000657008 in PrePrepare_Remote (prepareGID=0x2076380 "T10056",
localNode=1 '\001', implicit=1 '\001') at execRemote.c:4561
#2  0x00000000004b91dc in PrepareTransaction () at xact.c:2353
#3  0x00000000004b8c54 in CommitTransaction () at xact.c:2005
#4  0x00000000004b9a6a in CommitTransactionCommand () at xact.c:2897
#5  0x0000000000753f8e in finish_xact_command () at postgres.c:2572
#6  0x00000000007518e9 in exec_simple_query (query_string=0x1fcd6c8 "create
table aa (a int);") at postgres.c:1136
#7  0x0000000000755f10 in PostgresMain (argc=2, argv=0x1f11de8,
username=0x1f11c30 "michael") at postgres.c:4155
#8  0x00000000006fc1ea in BackendRun (port=0x1f40e10) at postmaster.c:3763
#9  0x00000000006fb86a in BackendStartup (port=0x1f40e10) at
postmaster.c:3448
#10 0x00000000006f87b6 in ServerLoop () at postmaster.c:1539
#11 0x00000000006f7f57 in PostmasterMain (argc=7, argv=0x1f0eba0) at
postmaster.c:1200
#12 0x0000000000662b39 in main (argc=7, argv=0x1f0eba0) at main.c:199
(gdb) p i
$2 = 4
(gdb) p connections
$3 = (PGXCNodeHandle **) 0x1f2ca40
(gdb) p connections[0]
$4 = (PGXCNodeHandle *) 0x2007638
(gdb) p *connections[0]
$5 = {nodeoid = 16384, sock = 31, transaction_status = 84 'T', state =
DN_CONNECTION_STATE_QUERY, combiner = 0x0, error = 0x0, outBuffer =
0x2019568 "Q", outSize = 16384, outEnd = 0,
  inBuffer = 0x201d5a8 "C", inSize = 16384, inStart = 24, inEnd = 24,
inCursor = 24}
(gdb) p *connections[1]
$6 = {nodeoid = 16385, sock = 41, transaction_status = 84 'T', state =
DN_CONNECTION_STATE_QUERY, combiner = 0x0, error = 0x0, outBuffer =
0x20215e8 "Q", outSize = 16384, outEnd = 0,
  inBuffer = 0x2025628 "C", inSize = 16384, inStart = 24, inEnd = 24,
inCursor = 24}
(gdb) p *connections[2]
$7 = {nodeoid = 16386, sock = 42, transaction_status = 84 'T', state =
DN_CONNECTION_STATE_QUERY, combiner = 0x0, error = 0x0, outBuffer =
0x2029668 "Q", outSize = 16384, outEnd = 0,
  inBuffer = 0x202d6a8 "C", inSize = 16384, inStart = 24, inEnd = 24,
inCursor = 24}
(gdb) p *connections[3]
$8 = {nodeoid = 16387, sock = 43, transaction_status = 84 'T', state =
DN_CONNECTION_STATE_QUERY, combiner = 0x0, error = 0x0, outBuffer =
0x20316e8 "Q", outSize = 16384, outEnd = 0,
  inBuffer = 0x2035728 "C", inSize = 16384, inStart = 24, inEnd = 24,
inCursor = 24}
(gdb) p *connections[4]
Cannot access memory at address 0x100000001

On Mon, Feb 27, 2012 at 8:22 PM, Pavan Deolasee <
pav...@en...> wrote:

> On Mon, Feb 27, 2012 at 12:14 PM, Pavan Deolasee
> <pav...@en...> wrote:
> > On Mon, Feb 27, 2012 at 11:46 AM, Ashutosh Bapat
> > <ash...@en...> wrote:
> >> Ah,
> >> if you see the error even after rebuilding the data-clusters, this
> could be
> >> something to look at.
> >>
> >
> > My observation is that this happens when something goes wrong at the
> > datanode. The problem is since we are seeing this error even without
> > the patch (though intermittently), we need to fix the general issue.
> > The patch might just be manifesting it.
> >
>
> I hope the attached patch would fix the issue for pooler connections
> (to be applied on top of the xact refactoring patch). In
> ExecRemoteUtility we are doing some funny things.. like counting the
> local coordinator and then reducing the count by 1 while running the
> for loop. I am not sure why its necessary and how it was running
> earlier. But I definitely think thats a bad code because we don't
> connect to the local coordinator ever.
>
> I have tested regression with 1 and 1 coordinators and they seem to
> work fine in my environment. I will continue to dig other errors.
>
> Thanks,
> Pavan
>
> --
> Pavan Deolasee
> EnterpriseDB http://www.enterprisedb.com
>



-- 
Michael Paquier
http://michael.otacoo.com

On Tue, Feb 28, 2012 at 2:06 AM, Ashutosh Bapat <
ash...@en...> wrote:

> Michael,
> The write-up looks good to me. But more eyeballing is necessary esp to see
> how it fits in XC's architecture and philosophy.
>
> Triggers are inherently non-immutable, since they are meant to have
> side-effects. Very rarely we will have a trigger doing immutable operation.
> As first cut in supporting triggers, making them (volatile ones, means
> practically all) coordinator only, might be a good option.
>
> This approach has heavy performance implications, since coordinator will
> be involved even if the changes done by the trigger are local to the
> datanode. Sometime down the line, we have to find some efficient mechanism
> to reduce the need of coordinator and do as much as possible at the
> datanode. Oviously any cross-datanode operation will need to be routed
> through the coordinator. But we should try to identify whether a particular
> trigger operation needs any cross-datanode access and only route that
> operation through coordinator. Users should try to design the database and
> triggers such that there is least cross-datanode communication required.
>
Yeah, I understand that this is not the most performant design. But I am
kind of sure that for the time being it is not worth spending time to have
a look at all the corner cases and possible optimizations we may find in
triggers like what we tried to do for window functions and stable
functions. In those cases, we spent time analyzing a lot of things for no
results and we decided to keep everything running on Coordinator for
safety. For the time being and the 1.0 release, I prefer a general, perhaps
not performant implementation than something that might work in some
special cases. We should target optimizations depending on the feedback
that people might have after the 1.0 release. The application guys are the
only ones who might know how to use well cluster-wide triggers with
non-shippable procedures fires.

As I believe this implementation is pretty simple, I hope to gain time for
the numerous stalability issues that we still need to fix and which are
really problematic.


>
> In fact, for now, you can stop any query involving non-immutable trigger
> from directly shipping to datanode. That would avoid extra work needed to
> communicate the rows from datanode to coordinator. But, the decision will
> depend upon how much this extra code is.
>
I believe the extra code to fire triggers on Coordinators is not that huge,
I still need to look at trigger algo first.
-- 
Michael Paquier
http://michael.otacoo.com

Hi,

As per suggestion from David, I had a look on how to setup an IRC channel
on freenode and set up one.

I found a page about how to set an account on freenode. Then what you need
to do is join the channel #pgxc.
http://freenode.net/faq.shtml#nicksetup
You can find me under the name ioltas.

In freenode, there are browser-based clients like this one:
http://webchat.freenode.net/.
This is useful to avoid annoying proxys.

Regards,
**--
Michael Paquier
http://michael.otacoo.com

Hi,

As we got a report from David, we should consider to include *.c file
generated from *.l and *.y file in release tar balls, as done in
vanilla PostgreSQL.    XC has specific handling of sgml docs through
*.sgmlin, to make merge easier from PostgreSQL docs.   Although *.sgml
can be generated from *.sgmlin using makesgml as found in
doc-xc/tools, it may help users if we have *.sgml (for XC, English)
files in tarball.

Any thoughts?
----------
Koichi Suzuki

I am a newbie to freenode :)

But I just found that to setup an account and a channel.
http://blog.freenode.net/2008/04/registering-a-channel-on-freenode/
-- 
Michael Paquier
http://michael.otacoo.com

On Feb 27, 2012, at 4:46 PM, Michael Paquier wrote:

> We only have the mailing lists now. But it may be a good idea to have a freenode dedicated to XC as community is growing.

Yes, it would. I'm on #postgresql and other channels on Freenode all the time, myself.

I just found your post here, though, so it will help with my playing:

  http://michael.otacoo.com/linux-2/setup-and-manipulation-of-a-postgres-xc-cluster-with-node-ddl/

Best,

David