postgres-xc-developers Mailing List for Postgres-XC

>
>> I would personally have a different idea of doing this. We should have the
>> sequence definitions on the data nodes as well, and data nodes should be
>> able to retrieve the nextval of the sequences. Looking at the possibility of
>> having multiple coordinators, we anyway need to synchronise nextval of
>> sequences across coordinators. Same mechanism can be used to synchronise it
>> across data nodes. In such case sending nextval(foo) through query won't be
>> a problem.
>>
> What happens in the case of replicated tables? If datanodes receive a
> nextval value locally tuples will be inconsistent.
> Just for the case of replicated tables, it looks far simpler to let
> Coordinator feed the queries with nextval.
>

Hmm, I didn't think of this case :(

I think, this will be a generic problem, if the queries other than INSERT
also use nextval() on sequences. There is less probability that SELECT will
use it but UPDATEs are other potential candidates.



> --
> Michael Paquier
> http://michael.otacoo.com
>



-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

Hi,

Postgres-XC document was updated in the following URLs:
	http://postgres-xc.sourceforge.net/xc-docs/html/
	http://postgres-xc.sourceforge.net/xc-docs/postgres-xc-A4.pdf
	http://postgres-xc.sourceforge.net/xc-docs/postgres-xc-US.pdf

This time, improved BARRIER description.

---
Koichi Suzuki

On Tue, Jul 19, 2011 at 12:00 PM, Ashutosh Bapat <
ash...@en...> wrote:

>
>
> On Tue, Jul 19, 2011 at 8:00 AM, Michael Paquier <
> mic...@gm...> wrote:
>
>> Hi all,
>>
>> I am currently studying about the ways to be able to feed default values
>> to insert queries like for what is found for example for SERIAL and default
>> values using time and sequence based functions.
>> In XC planner, what is done currently is using a function called
>> deparse_query in ruleutils.c to be able to rebuild a query string from a
>> parse tree.
>>
>> I don't think this is correct as we may need to be able to rewrite a query
>> after getting values corresponding to default expressions. This query
>> rewrite should be done in executor after getting all the necessary values
>> given by expressions.
>>
>> Let's get an example:
>> CREATE SEQUENCE foo;
>> CREATE TABLE aa (a int, b int DEFAULT nextval('foo')) DISTRIBUTE BY
>> REPLICATION; --OK this is not supported yet by XC and returns an error, but
>> let's imagine it is not the case.
>> INSERT INTO aa VALUES (1, DEFAULT);
>>
>> In this case the query sent down to nodes is the one generated by
>> deparse_query at planner level.
>> It results in an error because the query sent down has the form:
>> INSERT INTO aa (a, b) VALUES (1, nextval('foo'::regclass))
>>
>> template1=# INSERT INTO aa VALUES (1, DEFAULT);
>> ERROR:  relation "foo" does not exist
>> template1=# explain verbose INSERT INTO aa VALUES (1, DEFAULT);
>>                                          QUERY
>> PLAN
>>
>> --------------------------------------------------------------------------------------------
>>  Data Node Scan (Primary Node Count [1]) (Node Count [1])
>> (cost=0.00..0.00 rows=0 width=0)
>>    Output: 1, nextval('foo'::regclass)
>>
>> The problem is that foo is a sequence and it is only created at
>> Coordinators.
>> It is also necessary to generate this value on Coordinator to feed
>> Datanodes with the same values.
>>
>
> What happens when the INSERT queries are fired from different coordinators
> in a multi-coordinator environment.
>
GTM always delivers unique sequence values with nextval as sequence value
feed is protected by pthread lock on GTM.
So we are sure about the uniquess of value.

>
>
>>
>> Do you guys have any idea of a mechanism that would permit to rewrite the
>> query at executor level after getting the output of nextval?
>> It may be sufficient to call deparse_query once again but it needs a parse
>> tree and RemoteQuery plan does not have that.
>> deparse_query also looks to be a function that should not be called in
>> executor.
>>
>
> Convert it into parameterised query with default values as parameters.
> During execution pass the evaluated default values are parameter values to
> the remote node.
>
That's a nice idea. It could make the deal when writing the query in
planner.

>
> I would personally have a different idea of doing this. We should have the
> sequence definitions on the data nodes as well, and data nodes should be
> able to retrieve the nextval of the sequences. Looking at the possibility of
> having multiple coordinators, we anyway need to synchronise nextval of
> sequences across coordinators. Same mechanism can be used to synchronise it
> across data nodes. In such case sending nextval(foo) through query won't be
> a problem.
>
What happens in the case of replicated tables? If datanodes receive a
nextval value locally tuples will be inconsistent.
Just for the case of replicated tables, it looks far simpler to let
Coordinator feed the queries with nextval.
-- 
Michael Paquier
http://michael.otacoo.com

On Tue, Jul 19, 2011 at 8:00 AM, Michael Paquier
<mic...@gm...>wrote:

> Hi all,
>
> I am currently studying about the ways to be able to feed default values to
> insert queries like for what is found for example for SERIAL and default
> values using time and sequence based functions.
> In XC planner, what is done currently is using a function called
> deparse_query in ruleutils.c to be able to rebuild a query string from a
> parse tree.
>
> I don't think this is correct as we may need to be able to rewrite a query
> after getting values corresponding to default expressions. This query
> rewrite should be done in executor after getting all the necessary values
> given by expressions.
>
> Let's get an example:
> CREATE SEQUENCE foo;
> CREATE TABLE aa (a int, b int DEFAULT nextval('foo')) DISTRIBUTE BY
> REPLICATION; --OK this is not supported yet by XC and returns an error, but
> let's imagine it is not the case.
> INSERT INTO aa VALUES (1, DEFAULT);
>
> In this case the query sent down to nodes is the one generated by
> deparse_query at planner level.
> It results in an error because the query sent down has the form:
> INSERT INTO aa (a, b) VALUES (1, nextval('foo'::regclass))
>
> template1=# INSERT INTO aa VALUES (1, DEFAULT);
> ERROR:  relation "foo" does not exist
> template1=# explain verbose INSERT INTO aa VALUES (1, DEFAULT);
>                                          QUERY
> PLAN
>
> --------------------------------------------------------------------------------------------
>  Data Node Scan (Primary Node Count [1]) (Node Count [1])  (cost=0.00..0.00
> rows=0 width=0)
>    Output: 1, nextval('foo'::regclass)
>
> The problem is that foo is a sequence and it is only created at
> Coordinators.
> It is also necessary to generate this value on Coordinator to feed
> Datanodes with the same values.
>

What happens when the INSERT queries are fired from different coordinators
in a multi-coordinator environment.


>
> Do you guys have any idea of a mechanism that would permit to rewrite the
> query at executor level after getting the output of nextval?
> It may be sufficient to call deparse_query once again but it needs a parse
> tree and RemoteQuery plan does not have that.
> deparse_query also looks to be a function that should not be called in
> executor.
>

Convert it into parameterised query with default values as parameters.
During execution pass the evaluated default values are parameter values to
the remote node.

I would personally have a different idea of doing this. We should have the
sequence definitions on the data nodes as well, and data nodes should be
able to retrieve the nextval of the sequences. Looking at the possibility of
having multiple coordinators, we anyway need to synchronise nextval of
sequences across coordinators. Same mechanism can be used to synchronise it
across data nodes. In such case sending nextval(foo) through query won't be
a problem.

>
> This is a key point for support of serial tables in XC. Any ideas about how
> to handle that?
> Add a parse tree in RemoteQuery plan?
> Add an inner plan to rewrite query?
> --
> Michael Paquier
> http://michael.otacoo.com
>
>
> ------------------------------------------------------------------------------
> Magic Quadrant for Content-Aware Data Loss Prevention
> Research study explores the data loss prevention market. Includes in-depth
> analysis on the changes within the DLP market, and the criteria used to
> evaluate the strengths and weaknesses of these DLP solutions.
> http://www.accelacomm.com/jaw/sfnl/114/51385063/
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>


-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

Hi all,

I am currently studying about the ways to be able to feed default values to
insert queries like for what is found for example for SERIAL and default
values using time and sequence based functions.
In XC planner, what is done currently is using a function called
deparse_query in ruleutils.c to be able to rebuild a query string from a
parse tree.

I don't think this is correct as we may need to be able to rewrite a query
after getting values corresponding to default expressions. This query
rewrite should be done in executor after getting all the necessary values
given by expressions.

Let's get an example:
CREATE SEQUENCE foo;
CREATE TABLE aa (a int, b int DEFAULT nextval('foo')) DISTRIBUTE BY
REPLICATION; --OK this is not supported yet by XC and returns an error, but
let's imagine it is not the case.
INSERT INTO aa VALUES (1, DEFAULT);

In this case the query sent down to nodes is the one generated by
deparse_query at planner level.
It results in an error because the query sent down has the form:
INSERT INTO aa (a, b) VALUES (1, nextval('foo'::regclass))

template1=# INSERT INTO aa VALUES (1, DEFAULT);
ERROR:  relation "foo" does not exist
template1=# explain verbose INSERT INTO aa VALUES (1, DEFAULT);
                                         QUERY
PLAN
--------------------------------------------------------------------------------------------
 Data Node Scan (Primary Node Count [1]) (Node Count [1])  (cost=0.00..0.00
rows=0 width=0)
   Output: 1, nextval('foo'::regclass)

The problem is that foo is a sequence and it is only created at
Coordinators.
It is also necessary to generate this value on Coordinator to feed Datanodes
with the same values.

Do you guys have any idea of a mechanism that would permit to rewrite the
query at executor level after getting the output of nextval?
It may be sufficient to call deparse_query once again but it needs a parse
tree and RemoteQuery plan does not have that.
deparse_query also looks to be a function that should not be called in
executor.

This is a key point for support of serial tables in XC. Any ideas about how
to handle that?
Add a parse tree in RemoteQuery plan?
Add an inner plan to rewrite query?
-- 
Michael Paquier
http://michael.otacoo.com