postgres-xc-developers Mailing List for Postgres-XC

Hi,
Attached please find the patch that solves cursor visibility problem as
well as the inserting into child by selecting from the parent problem using
the design discussed earlier.
Test cases are added and regressions are fixed accordingly.
Patch is ready for review.

Thanks.

On Tue, Jul 10, 2012 at 6:54 AM, Koichi Suzuki <koi...@gm...>wrote:

> I basically agree on the idea.   I also think Michael needs this kind
> of extension to feedback trigger target rows.    We should design
> NoticeResponse extensible enough for various usage.
>
> Regards;
> ----------
> Koichi Suzuki
>
>
> 2012/7/10 Abbas Butt <abb...@en...>:
> > Here is the update on the issue.
> >
> > It was decided that the changes done by the data nodes in the command id
> > should be communicated back to the coordinator and that the coordinator
> > should choose the largest of all the received values as the next command
> id.
> >
> > It was suggested that we should check that a skipped value of command id
> > should not create a problem for the next operations on the table. I have
> > verified both by studying the code and by actually changing the function
> > CommandCounterIncrement to increment the command id by 3 and running
> > regression. It worked fine so a hole in command id is not a problem.
> >
> > Next it was suggested that we should use the mechanism currently in
> place to
> > send # of tuples affected by a statement to communicate the changed
> command
> > id to the coordinator.
> > Please refer to this link in the documentation
> > http://www.postgresql.org/docs/9.1/static/protocol-message-formats.html
> > Note that  there is no such message format that exists in the current
> over
> > the wire protocol to communicate the # of tuples affected by a statement.
> > The libpq functions that we might suspect do the same are PQntuples and
> > PQcmdTuples. PQntuples simply returns ntups member of PGresult, where as
> > PQcmdTuples extracts the # of tuples affected from the CommandComplete
> 'C'
> > message string. We cannot use these mechanisms for our purpose.
> >
> > I evaluated the use of NoticeResponse 'N' for sending changed command id
> but
> > the message format of NoticeResponse mandates the use of certain fields
> > which will make our sent messages un-necessarily bulky and would consume
> > network bandwidth for no reason.
> >
> > I therefore suggest the we use a new message for communicating XC
> specific
> > information from data node to coordinator. Currently we will use it for
> > command id but we will design the message format flexible enough to
> > accommodate future XC requirements. Whenever the data node increments
> > command id we will send the information to the coordinator and
> > handle_response function in execRemote.c would be changed to accommodate
> new
> > message. Since coordinators will never use the new message therefore the
> > existing clients do not need to bother.
> >
> > Comments or suggestions are welcome.
> >
> > Regards
> >
> >
> > On Wed, Jul 4, 2012 at 8:35 AM, Abbas Butt <abb...@en...>
> > wrote:
> >>
> >> While fixing the regression failures resulting from the changes done by
> >> the patch I was able to fix all except this test case
> >>
> >> set enforce_two_phase_commit = off;
> >>
> >> CREATE TEMP TABLE users (
> >>   id INT PRIMARY KEY,
> >>   name VARCHAR NOT NULL
> >> ) DISTRIBUTE BY REPLICATION;
> >>
> >> INSERT INTO users VALUES (1, 'Jozko');
> >> INSERT INTO users VALUES (2, 'Ferko');
> >> INSERT INTO users VALUES (3, 'Samko');
> >> CREATE TEMP TABLE tasks (
> >>   id INT PRIMARY KEY,
> >>   owner INT REFERENCES users ON UPDATE CASCADE ON DELETE SET NULL,
> >>   worker INT REFERENCES users ON UPDATE CASCADE ON DELETE SET NULL,
> >>   checked_by INT REFERENCES users ON UPDATE CASCADE ON DELETE SET NULL
> >> ) DISTRIBUTE BY REPLICATION;
> >>
> >> INSERT INTO tasks VALUES (1,1,NULL,NULL);
> >> INSERT INTO tasks VALUES (2,2,2,NULL);
> >> INSERT INTO tasks VALUES (3,3,3,3);
> >>
> >> BEGIN;
> >>
> >> UPDATE tasks set id=id WHERE id=2;
> >> SELECT * FROM tasks;
> >>
> >> DELETE FROM users WHERE id = 2;
> >> SELECT * FROM tasks;
> >>
> >> COMMIT;
> >>
> >> The obtained output from the last select statement is
> >>
> >>  id | owner | worker | checked_by
> >> ----+-------+--------+------------
> >>   1 |     1 |        |
> >>   3 |     3 |      3 |          3
> >>   2 |     2 |      2 |
> >> (3 rows)
> >>
> >> where as the expected output is
> >>
> >>  id | owner | worker | checked_by
> >> ----+-------+--------+------------
> >>   1 |     1 |        |
> >>   3 |     3 |      3 |          3
> >>   2 |       |        |
> >> (3 rows)
> >>
> >> Note that the owner and worker have been set to null due to "ON DELETE
> SET
> >> NULL".
> >>
> >> Here is the reason why this does not work properly. Consider the last
> >> transaction
> >>
> >> BEGIN;
> >>
> >> UPDATE tasks set id=id WHERE id=2;
> >> SELECT * FROM tasks;
> >>
> >> DELETE FROM users WHERE id = 2;
> >> SELECT * FROM tasks;
> >>
> >> COMMIT;
> >>
> >> Here are the command id values the coordinator sends to the data node
> >>
> >> 0 for the first update that gets incremented to 1 because this is a DML
> >> and needs to consume a command id
> >> 1 for the first select that remains 1 since it is not required to be
> >> consumed.
> >> 1 for the delete statement that gets incremented to 2 because it is a
> DML
> >> and 2 for the last select.
> >>
> >> Now this is what happens on the data node
> >>
> >> When the data node receives the first update with command id 0, it
> >> increments it once due to the update itself and once due to the update
> run
> >> because of "ON UPDATE CASCADE". Hence the command id at the end of
> update on
> >> data node is 2.
> >> The first select comes to data node with command id 1, which is
> incorrect.
> >> The user's intention is to see data after update and its command id
> should
> >> be 2.
> >> Now delete comes with command id 1, and data node increments it once due
> >> to the delete itself and once due to the update run because of "ON
> DELETE
> >> SET NULL", hence the command id at the end of delete is 3.
> >> Coordinator now sends last select with command id 2, which is again
> >> incorrect since user's intention is to see data after delete and select
> >> should have been sent to data node with command id 3 or 4.
> >>
> >> Every time data node increments command id due to any statements run
> >> implicitly either because of the constraints or triggers, this scheme of
> >> sending command ids to data node from coordinator to solve fetch
> problems
> >> would fail.
> >>
> >> Datanode can have a trigger e.g. inserting rows thrice on every single
> >> insert and would increment command id on every insert. Therefore this
> design
> >> cannot work.
> >>
> >> Either we have to synchronize command ids between datanode and
> coordinator
> >> through GTM
> >> OR
> >> We will have to send the DECLARE CURSOR down to the datanode. In this
> case
> >> however we will not be able to send the cursor query as it is because
> the
> >> query might contain a join on two tables which exist on a disjoint set
> of
> >> data nodes.
> >>
> >> Comments or suggestions are welcome.
> >>
> >>
> >>
> >> On Tue, Jun 19, 2012 at 2:43 PM, Abbas Butt <
> abb...@en...>
> >> wrote:
> >>>
> >>> Thanks for your comments.
> >>>
> >>> On Tue, Jun 19, 2012 at 1:54 PM, Ashutosh Bapat
> >>> <ash...@en...> wrote:
> >>>>
> >>>> Hi Abbas,
> >>>> I have few comments to make
> >>>> 1. With this patch there are two variables for having command Id, that
> >>>> is going to cause confusion and will be a maintenance burden, might
> be error
> >>>> prone. Is it possible to use a single variable instead of two?
> >>>
> >>>
> >>> Are you talking about receivedCommandId and currentCommandId? If yes, I
> >>> would prefer not having a packet received from coordinator overwrite
> the
> >>> currentCommandId at data node, because I am not 100% sure about the
> life
> >>> time of currentCommandId, I might overwrite it before time. It would
> be safe
> >>> to let currentCommandId as is unless we are compelled to get the next
> >>> command ID, and have the received command id take priority at that
> time.
> >>>
> >>>>
> >>>> Right now there is some code which is specific to cursors in your
> patch.
> >>>> If you can plug the coordinator command id somehow into
> currentCommandId,
> >>>> you won't need that code and any other code which needs coordinator
> command
> >>>> ID will be automatically taken care of.
> >>>
> >>>
> >>> That code is required to solve a problem. Consider this case when a
> >>> coordinator received this transaction
> >>>
> >>>
> >>> BEGIN;
> >>>  insert into tt1 values(1);
> >>>  declare c50 cursor for select * from tt1;
> >>>  insert into tt1 values(2);
> >>>  fetch all from c50;
> >>> COMMIT;
> >>>
> >>> While sending select to the data node in response to a fetch we need to
> >>> know what was the command ID of the declare cursor statement and we
> need to
> >>> send that command ID to the data node for this particular fetch. This
> is the
> >>> main idea behind this solution.
> >>>
> >>> The first insert goes to the data node with command id 0, the second
> >>> insert goes with 2. Command ID 1 is consumed by declare cursor. When
> >>> coordinator sees fetch it needs to send select to the data node with
> command
> >>> ID 1 rather than 3.
> >>>
> >>>
> >>>>
> >>>> 2. A non-transaction on coordinator can spawn tranasactions on
> datanode
> >>>> or subtransactions (if there is already a transaction running). Does
> your
> >>>> patch handle that case?
> >>>
> >>>
> >>> No and it does not need to, because that case has no known problems
> that
> >>> we need to solve. I don't think my patch would impact any such case
> but I
> >>> will analyze any failures that I may get in regressions.
> >>>
> >>>>
> >>>> Should we do more thorough research in the transaction management,
> esp.
> >>>> to see the impact of getting same command id for two commands on the
> >>>> datanode?
> >>>
> >>>
> >>> If we issue two commands with the same command ID then we will
> definitely
> >>> have visibility issues according to the rules I have already
> explained. But
> >>> we will not have two commands sent to the data node with same command
> id.
> >>>
> >>>>
> >>>>
> >>>>
> >>>> On Tue, Jun 19, 2012 at 1:56 PM, Abbas Butt
> >>>> <abb...@en...> wrote:
> >>>>>
> >>>>> Hi Ashutosh,
> >>>>> Here are the results with the val column, Thanks.
> >>>>>
> >>>>> test=# drop table mvcc_demo;
> >>>>> DROP TABLE
> >>>>> test=#
> >>>>> test=# create table mvcc_demo (val int);
> >>>>> CREATE TABLE
> >>>>> test=#
> >>>>> test=# TRUNCATE mvcc_demo;
> >>>>> TRUNCATE TABLE
> >>>>> test=#
> >>>>> test=# BEGIN;
> >>>>> BEGIN
> >>>>> test=# DELETE FROM mvcc_demo; -- increment command id to show that
> >>>>> combo id would be different
> >>>>> DELETE 0
> >>>>> test=# DELETE FROM mvcc_demo;
> >>>>> DELETE 0
> >>>>> test=# DELETE FROM mvcc_demo;
> >>>>> DELETE 0
> >>>>> test=# INSERT INTO mvcc_demo VALUES (1);
> >>>>> INSERT 0 1
> >>>>> test=# INSERT INTO mvcc_demo VALUES (2);
> >>>>> INSERT 0 1
> >>>>> test=# INSERT INTO mvcc_demo VALUES (3);
> >>>>> INSERT 0 1
> >>>>> test=# SELECT t_xmin AS xmin,
> >>>>> test-#        t_xmax::text::int8 AS xmax,
> >>>>> test-#        t_field3::text::int8 AS cmin_cmax,
> >>>>> test-#        (t_infomask::integer & X'0020'::integer)::bool AS
> >>>>> is_combocid
> >>>>> test-# FROM heap_page_items(get_raw_page('mvcc_demo', 0))
> >>>>> test-# ORDER BY 2 DESC, 3;
> >>>>>  xmin  | xmax | cmin_cmax | is_combocid
> >>>>> -------+------+-----------+-------------
> >>>>>  80689 |    0 |         3 | f
> >>>>>  80689 |    0 |         4 | f
> >>>>>  80689 |    0 |         5 | f
> >>>>> (3 rows)
> >>>>>
> >>>>> test=#
> >>>>> test=# select xmin,xmax,cmin,cmax,* from mvcc_demo order by val;
> >>>>>  xmin  | xmax | cmin | cmax | val
> >>>>> -------+------+------+------+-----
> >>>>>  80689 |    0 |    3 |    3 |   1
> >>>>>  80689 |    0 |    4 |    4 |   2
> >>>>>  80689 |    0 |    5 |    5 |   3
> >>>>>
> >>>>> (3 rows)
> >>>>>
> >>>>> test=#
> >>>>> test=# DELETE FROM mvcc_demo;
> >>>>> DELETE 3
> >>>>> test=# SELECT t_xmin AS xmin,
> >>>>> test-#        t_xmax::text::int8 AS xmax,
> >>>>> test-#        t_field3::text::int8 AS cmin_cmax,
> >>>>> test-#        (t_infomask::integer & X'0020'::integer)::bool AS
> >>>>> is_combocid
> >>>>> test-# FROM heap_page_items(get_raw_page('mvcc_demo', 0))
> >>>>> test-# ORDER BY 2 DESC, 3;
> >>>>>  xmin  | xmax  | cmin_cmax | is_combocid
> >>>>> -------+-------+-----------+-------------
> >>>>>  80689 | 80689 |         0 | t
> >>>>>  80689 | 80689 |         1 | t
> >>>>>  80689 | 80689 |         2 | t
> >>>>> (3 rows)
> >>>>>
> >>>>> test=#
> >>>>> test=# select xmin,xmax,cmin,cmax,* from mvcc_demo order by val;
> >>>>>  xmin | xmax | cmin | cmax | val
> >>>>> ------+------+------+------+-----
> >>>>> (0 rows)
> >>>>>
> >>>>>
> >>>>> test=#
> >>>>> test=# END;
> >>>>> COMMIT
> >>>>> test=#
> >>>>> test=#
> >>>>> test=# TRUNCATE mvcc_demo;
> >>>>> TRUNCATE TABLE
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> test=# BEGIN;
> >>>>> BEGIN
> >>>>> test=# INSERT INTO mvcc_demo VALUES (1);
> >>>>> INSERT 0 1
> >>>>> test=# INSERT INTO mvcc_demo VALUES (2);
> >>>>> INSERT 0 1
> >>>>> test=# INSERT INTO mvcc_demo VALUES (3);
> >>>>> INSERT 0 1
> >>>>> test=# SELECT t_xmin AS xmin,
> >>>>> test-#        t_xmax::text::int8 AS xmax,
> >>>>> test-#        t_field3::text::int8 AS cmin_cmax,
> >>>>> test-#        (t_infomask::integer & X'0020'::integer)::bool AS
> >>>>> is_combocid
> >>>>> test-# FROM heap_page_items(get_raw_page('mvcc_demo', 0))
> >>>>> test-# ORDER BY 2 DESC, 3;
> >>>>>  xmin  | xmax | cmin_cmax | is_combocid
> >>>>> -------+------+-----------+-------------
> >>>>>  80693 |    0 |         0 | f
> >>>>>  80693 |    0 |         1 | f
> >>>>>  80693 |    0 |         2 | f
> >>>>> (3 rows)
> >>>>>
> >>>>> test=#
> >>>>> test=# select xmin,xmax,cmin,cmax,* from mvcc_demo order by val;
> >>>>>  xmin  | xmax | cmin | cmax | val
> >>>>> -------+------+------+------+-----
> >>>>>  80693 |    0 |    0 |    0 |   1
> >>>>>  80693 |    0 |    1 |    1 |   2
> >>>>>  80693 |    0 |    2 |    2 |   3
> >>>>> (3 rows)
> >>>>>
> >>>>> test=#
> >>>>> test=# UPDATE mvcc_demo SET val = 10;
> >>>>>
> >>>>> UPDATE 3
> >>>>> test=#
> >>>>> test=# SELECT t_xmin AS xmin,
> >>>>> test-#        t_xmax::text::int8 AS xmax,
> >>>>> test-#        t_field3::text::int8 AS cmin_cmax,
> >>>>> test-#        (t_infomask::integer & X'0020'::integer)::bool AS
> >>>>> is_combocid
> >>>>> test-# FROM heap_page_items(get_raw_page('mvcc_demo', 0))
> >>>>> test-# ORDER BY 2 DESC, 3;
> >>>>>  xmin  | xmax  | cmin_cmax | is_combocid
> >>>>> -------+-------+-----------+-------------
> >>>>>  80693 | 80693 |         0 | t
> >>>>>  80693 | 80693 |         1 | t
> >>>>>  80693 | 80693 |         2 | t
> >>>>>  80693 |     0 |         3 | f
> >>>>>  80693 |     0 |         3 | f
> >>>>>  80693 |     0 |         3 | f
> >>>>> (6 rows)
> >>>>>
> >>>>> test=#
> >>>>> test=# select xmin,xmax,cmin,cmax,* from mvcc_demo order by val;
> >>>>>  xmin  | xmax | cmin | cmax | val
> >>>>> -------+------+------+------+-----
> >>>>>  80693 |    0 |    3 |    3 |  10
> >>>>>  80693 |    0 |    3 |    3 |  10
> >>>>>  80693 |    0 |    3 |    3 |  10
> >>>>> (3 rows)
> >>>>>
> >>>>>
> >>>>> test=#
> >>>>> test=# END;
> >>>>> COMMIT
> >>>>> test=#
> >>>>> test=# TRUNCATE mvcc_demo;
> >>>>> TRUNCATE TABLE
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>     -- From one psql issue
> >>>>>     test=# INSERT INTO mvcc_demo VALUES (1);
> >>>>>     INSERT 0 1
> >>>>>     test=# SELECT t_xmin AS xmin,
> >>>>>     test-#        t_xmax::text::int8 AS xmax,
> >>>>>     test-#        t_field3::text::int8 AS cmin_cmax,
> >>>>>     test-#        (t_infomask::integer & X'0020'::integer)::bool AS
> >>>>> is_combocid
> >>>>>     test-# FROM heap_page_items(get_raw_page('mvcc_demo', 0))
> >>>>>     test-# ORDER BY 2 DESC, 3;
> >>>>>      xmin  | xmax | cmin_cmax | is_combocid
> >>>>>     -------+------+-----------+-------------
> >>>>>      80699 |    0 |         0 | f
> >>>>>     (1 row)
> >>>>>
> >>>>>     test=#
> >>>>>     test=# select xmin,xmax,cmin,cmax,* from mvcc_demo order by val;
> >>>>>      xmin  | xmax | cmin | cmax | val
> >>>>>     -------+------+------+------+-----
> >>>>>      80699 |    0 |    0 |    0 |   1
> >>>>>     (1 row)
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> test=# -- From another issue
> >>>>> test=# BEGIN;
> >>>>> BEGIN
> >>>>> test=# INSERT INTO mvcc_demo VALUES (2);
> >>>>> INSERT 0 1
> >>>>> test=# INSERT INTO mvcc_demo VALUES (3);
> >>>>> INSERT 0 1
> >>>>> test=# INSERT INTO mvcc_demo VALUES (4);
> >>>>> INSERT 0 1
> >>>>> test=# SELECT t_xmin AS xmin,
> >>>>> test-#        t_xmax::text::int8 AS xmax,
> >>>>> test-#        t_field3::text::int8 AS cmin_cmax,
> >>>>> test-#        (t_infomask::integer & X'0020'::integer)::bool AS
> >>>>> is_combocid
> >>>>> test-# FROM heap_page_items(get_raw_page('mvcc_demo', 0))
> >>>>> test-# ORDER BY 2 DESC, 3;
> >>>>>  xmin  | xmax | cmin_cmax | is_combocid
> >>>>> -------+------+-----------+-------------
> >>>>>  80699 |    0 |         0 | f
> >>>>>  80700 |    0 |         0 | f
> >>>>>  80700 |    0 |         1 | f
> >>>>>  80700 |    0 |         2 | f
> >>>>> (4 rows)
> >>>>>
> >>>>> test=#
> >>>>> test=# select xmin,xmax,cmin,cmax,* from mvcc_demo order by val;
> >>>>>  xmin  | xmax | cmin | cmax | val
> >>>>> -------+------+------+------+-----
> >>>>>  80699 |    0 |    0 |    0 |   1
> >>>>>  80700 |    0 |    0 |    0 |   2
> >>>>>  80700 |    0 |    1 |    1 |   3
> >>>>>  80700 |    0 |    2 |    2 |   4
> >>>>> (4 rows)
> >>>>>
> >>>>> test=#
> >>>>> test=# UPDATE mvcc_demo SET val = 10;
> >>>>>
> >>>>> UPDATE 4
> >>>>> test=#
> >>>>> test=# SELECT t_xmin AS xmin,
> >>>>> test-#        t_xmax::text::int8 AS xmax,
> >>>>> test-#        t_field3::text::int8 AS cmin_cmax,
> >>>>> test-#        (t_infomask::integer & X'0020'::integer)::bool AS
> >>>>> is_combocid
> >>>>> test-# FROM heap_page_items(get_raw_page('mvcc_demo', 0))
> >>>>> test-# ORDER BY 2 DESC, 3;
> >>>>>  xmin  | xmax  | cmin_cmax | is_combocid
> >>>>> -------+-------+-----------+-------------
> >>>>>  80700 | 80700 |         0 | t
> >>>>>  80700 | 80700 |         1 | t
> >>>>>  80700 | 80700 |         2 | t
> >>>>>  80699 | 80700 |         3 | f
> >>>>>  80700 |     0 |         3 | f
> >>>>>  80700 |     0 |         3 | f
> >>>>>  80700 |     0 |         3 | f
> >>>>>  80700 |     0 |         3 | f
> >>>>> (8 rows)
> >>>>>
> >>>>> test=#
> >>>>> test=# select xmin,xmax,cmin,cmax,* from mvcc_demo order by val;
> >>>>>  xmin  | xmax | cmin | cmax | val
> >>>>> -------+------+------+------+-----
> >>>>>  80700 |    0 |    3 |    3 |  10
> >>>>>  80700 |    0 |    3 |    3 |  10
> >>>>>  80700 |    0 |    3 |    3 |  10
> >>>>>  80700 |    0 |    3 |    3 |  10
> >>>>> (4 rows)
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>     test=# -- Before finishing this, issue these from the first psql
> >>>>>     test=# SELECT t_xmin AS xmin,
> >>>>>     test-#        t_xmax::text::int8 AS xmax,
> >>>>>     test-#        t_field3::text::int8 AS cmin_cmax,
> >>>>>     test-#        (t_infomask::integer & X'0020'::integer)::bool AS
> >>>>> is_combocid
> >>>>>     test-# FROM heap_page_items(get_raw_page('mvcc_demo', 0))
> >>>>>     test-# ORDER BY 2 DESC, 3;
> >>>>>      xmin  | xmax  | cmin_cmax | is_combocid
> >>>>>     -------+-------+-----------+-------------
> >>>>>      80700 | 80700 |         0 | t
> >>>>>      80700 | 80700 |         1 | t
> >>>>>      80700 | 80700 |         2 | t
> >>>>>      80699 | 80700 |         3 | f
> >>>>>      80700 |     0 |         3 | f
> >>>>>      80700 |     0 |         3 | f
> >>>>>      80700 |     0 |         3 | f
> >>>>>      80700 |     0 |         3 | f
> >>>>>     (8 rows)
> >>>>>
> >>>>>     test=#
> >>>>>     test=# select xmin,xmax,cmin,cmax,* from mvcc_demo order by val;
> >>>>>      xmin  | xmax  | cmin | cmax | val
> >>>>>     -------+-------+------+------+-----
> >>>>>      80699 | 80700 |    3 |    3 |   1
> >>>>>     (1 row)
> >>>>>
> >>>>> test=# end;
> >>>>> COMMIT
> >>>>>
> >>>>>
> >>>>> On Tue, Jun 19, 2012 at 10:26 AM, Michael Paquier
> >>>>> <mic...@gm...> wrote:
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> I expect pgxc_node_send_cmd_id to have some impact on performance,
> so
> >>>>>> be sure to send it to remote Datanodes really only if necessary.
> >>>>>> You should put more severe conditions blocking this function cid can
> >>>>>> easily get incremented in Postgres.
> >>>>>>
> >>>>>> Regards,
> >>>>>>
> >>>>>> On Tue, Jun 19, 2012 at 5:31 AM, Abbas Butt
> >>>>>> <abb...@en...> wrote:
> >>>>>>>
> >>>>>>> PFA a WIP patch implementing the design presented earlier.
> >>>>>>> The patch is WIP because it still has and FIXME and it shows some
> >>>>>>> regression failures that need to be fixed, but other than that it
> confirms
> >>>>>>> that the suggested design would work fine. The following test
> cases now work
> >>>>>>> fine
> >>>>>>>
> >>>>>>> drop table tt1;
> >>>>>>> create table tt1(f1 int) distribute by replication;
> >>>>>>>
> >>>>>>>
> >>>>>>> BEGIN;
> >>>>>>>  insert into tt1 values(1);
> >>>>>>>  declare c50 cursor for select * from tt1;
> >>>>>>>  insert into tt1 values(2);
> >>>>>>>  fetch all from c50;
> >>>>>>> COMMIT;
> >>>>>>> truncate table tt1;
> >>>>>>>
> >>>>>>> BEGIN;
> >>>>>>>
> >>>>>>>  declare c50 cursor for select * from tt1;
> >>>>>>>  insert into tt1 values(1);
> >>>>>>>
> >>>>>>>  insert into tt1 values(2);
> >>>>>>>  fetch all from c50;
> >>>>>>> COMMIT;
> >>>>>>> truncate table tt1;
> >>>>>>>
> >>>>>>>
> >>>>>>> BEGIN;
> >>>>>>>  insert into tt1 values(1);
> >>>>>>>  insert into tt1 values(2);
> >>>>>>>
> >>>>>>>  declare c50 cursor for select * from tt1;
> >>>>>>>  insert into tt1 values(3);
> >>>>>>>
> >>>>>>>  fetch all from c50;
> >>>>>>> COMMIT;
> >>>>>>> truncate table tt1;
> >>>>>>>
> >>>>>>>
> >>>>>>> BEGIN;
> >>>>>>>  insert into tt1 values(1);
> >>>>>>>  declare c50 cursor for select * from tt1;
> >>>>>>>  insert into tt1 values(2);
> >>>>>>>  declare c51 cursor for select * from tt1;
> >>>>>>>  insert into tt1 values(3);
> >>>>>>>  fetch all from c50;
> >>>>>>>  fetch all from c51;
> >>>>>>> COMMIT;
> >>>>>>> truncate table tt1;
> >>>>>>>
> >>>>>>>
> >>>>>>> BEGIN;
> >>>>>>>  insert into tt1 values(1);
> >>>>>>>  declare c50 cursor for select * from tt1;
> >>>>>>>  declare c51 cursor for select * from tt1;
> >>>>>>>  insert into tt1 values(2);
> >>>>>>>  insert into tt1 values(3);
> >>>>>>>  fetch all from c50;
> >>>>>>>  fetch all from c51;
> >>>>>>> COMMIT;
> >>>>>>> truncate table tt1;
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, Jun 15, 2012 at 8:07 AM, Abbas Butt
> >>>>>>> <abb...@en...> wrote:
> >>>>>>>>
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> In a multi-statement transaction each statement is given a command
> >>>>>>>> identifier
> >>>>>>>> starting from zero and incrementing for each statement.
> >>>>>>>> These command indentifers are required for extra tracking because
> >>>>>>>> each
> >>>>>>>> statement has its own visibility rules with in the transaction.
> >>>>>>>> For example, a cursor’s contents must remain unchanged even if
> later
> >>>>>>>> statements in the
> >>>>>>>> same transaction modify rows.  Such tracking is implemented using
> >>>>>>>> system command id
> >>>>>>>> columns cmin/cmax, which is internally actually is a single
> column.
> >>>>>>>>
> >>>>>>>> cmin/cmax come into play in case of multi-statement transactions
> >>>>>>>> only,
> >>>>>>>> they are both zero otherwise.
> >>>>>>>>
> >>>>>>>> cmin "The command identifier of the statement within the inserting
> >>>>>>>> transaction."
> >>>>>>>> cmax "The command identifier of the statement within the deleting
> >>>>>>>> transaction."
> >>>>>>>>
> >>>>>>>> Here are the visibility rules (taken from comments of tqual.c)
> >>>>>>>>
> >>>>>>>> (                                        // A heap tuple is valid
> >>>>>>>> "now" iff
> >>>>>>>>   Xmin == my-transaction &&              // inserted by the
> current
> >>>>>>>> transaction
> >>>>>>>>   Cmin < my-command &&                   // before this command,
> and
> >>>>>>>>   (
> >>>>>>>>     Xmax is null ||                      // the row has not been
> >>>>>>>> deleted, or
> >>>>>>>>     (
> >>>>>>>>       Xmax == my-transaction &&          // it was deleted by the
> >>>>>>>> current transaction
> >>>>>>>>       Cmax >= my-command                 // but not before this
> >>>>>>>> command,
> >>>>>>>>     )
> >>>>>>>>   )
> >>>>>>>> )
> >>>>>>>> ||                                       // or
> >>>>>>>> (
> >>>>>>>>   Xmin is committed &&                   // the row was inserted
> by
> >>>>>>>> a committed transaction, and
> >>>>>>>>   (
> >>>>>>>>     Xmax is null ||                      // the row has not been
> >>>>>>>> deleted, or
> >>>>>>>>     (
> >>>>>>>>       Xmax == my-transaction &&          // the row is being
> deleted
> >>>>>>>> by this transaction
> >>>>>>>>       Cmax >= my-command) ||             // but it's not deleted
> >>>>>>>> "yet", or
> >>>>>>>>       (
> >>>>>>>>         Xmax != my-transaction &&        // the row was deleted by
> >>>>>>>> another transaction
> >>>>>>>>         Xmax is not committed            // that has not been
> >>>>>>>> committed
> >>>>>>>>       )
> >>>>>>>>     )
> >>>>>>>>   )
> >>>>>>>> )
> >>>>>>>>
> >>>>>>>> Because cmin and cmax are internally a single system column,
> >>>>>>>> it is therefore not possible to simply record the status of a row
> >>>>>>>> that is created and expired in the same multi-statement
> transaction.
> >>>>>>>> For that reason, a special combo command id is created that
> >>>>>>>> references
> >>>>>>>> a local memory hash that contains the actual cmin and cmax values.
> >>>>>>>> It means that if combo id is being used the number we are seeing
> >>>>>>>> would not be the cmin or cmax it will be an index into a local
> >>>>>>>> array that contains a structure with has the actual cmin and cmax
> >>>>>>>> values.
> >>>>>>>>
> >>>>>>>> The following queries (taken mostly from
> >>>>>>>> http://momjian.us/main/writings/pgsql/mvcc.pdf)
> >>>>>>>> use the contrib module pageinspect, which allows
> >>>>>>>> visibility of internal heap page structures and all stored rows,
> >>>>>>>> including those not visible in the current snapshot.
> >>>>>>>> (Bit 0x0020 is defined as HEAP_COMBOCID.)
> >>>>>>>>
> >>>>>>>> We are exploring 3 examples here:
> >>>>>>>> 1) INSERT & DELETE in a single transaction
> >>>>>>>> 2) INSERT & UPDATE in a single transaction
> >>>>>>>> 3) INSERT from two different transactions & UPDATE from one
> >>>>>>>>
> >>>>>>>> test=# drop table mvcc_demo;
> >>>>>>>> DROP TABLE
> >>>>>>>> test=#
> >>>>>>>> test=# create table mvcc_demo (val int);
> >>>>>>>> CREATE TABLE
> >>>>>>>> test=#
> >>>>>>>> test=# TRUNCATE mvcc_demo;
> >>>>>>>> TRUNCATE TABLE
> >>>>>>>> test=#
> >>>>>>>> test=# BEGIN;
> >>>>>>>> BEGIN
> >>>>>>>> test=# DELETE FROM mvcc_demo; -- increment command id to show that
> >>>>>>>> combo id would be different
> >>>>>>>> DELETE 0
> >>>>>>>> test=# DELETE FROM mvcc_demo;
> >>>>>>>> DELETE 0
> >>>>>>>> test=# DELETE FROM mvcc_demo;
> >>>>>>>> DELETE 0
> >>>>>>>> test=# INSERT INTO mvcc_demo VALUES (1);
> >>>>>>>> INSERT 0 1
> >>>>>>>> test=# INSERT INTO mvcc_demo VALUES (2);
> >>>>>>>> INSERT 0 1
> >>>>>>>> test=# INSERT INTO mvcc_demo VALUES (3);
> >>>>>>>> INSERT 0 1
> >>>>>>>> test=# SELECT t_xmin AS xmin,
> >>>>>>>> test-#        t_xmax::text::int8 AS xmax,
> >>>>>>>> test-#        t_field3::text::int8 AS cmin_cmax,
> >>>>>>>> test-#        (t_infomask::integer & X'0020'::integer)::bool AS
> >>>>>>>> is_combocid
> >>>>>>>> test-# FROM heap_page_items(get_raw_page('mvcc_demo', 0))
> >>>>>>>> test-# ORDER BY 2 DESC, 3;
> >>>>>>>>  xmin  | xmax | cmin_cmax | is_combocid
> >>>>>>>> -------+------+-----------+-------------
> >>>>>>>>  80685 |    0 |         3 | f
> >>>>>>>>  80685 |    0 |         4 | f
> >>>>>>>>  80685 |    0 |         5 | f
> >>>>>>>> (3 rows)
> >>>>>>>>
> >>>>>>>> test=#
> >>>>>>>> test=# DELETE FROM mvcc_demo;
> >>>>>>>> DELETE 3
> >>>>>>>> test=# SELECT t_xmin AS xmin,
> >>>>>>>> test-#        t_xmax::text::int8 AS xmax,
> >>>>>>>> test-#        t_field3::text::int8 AS cmin_cmax,
> >>>>>>>> test-#        (t_infomask::integer & X'0020'::integer)::bool AS
> >>>>>>>> is_combocid
> >>>>>>>> test-# FROM heap_page_items(get_raw_page('mvcc_demo', 0))
> >>>>>>>> test-# ORDER BY 2 DESC, 3;
> >>>>>>>>  xmin  | xmax  | cmin_cmax | is_combocid
> >>>>>>>> -------+-------+-----------+-------------
> >>>>>>>>  80685 | 80685 |         0 | t
> >>>>>>>>  80685 | 80685 |         1 | t
> >>>>>>>>  80685 | 80685 |         2 | t
> >>>>>>>> (3 rows)
> >>>>>>>>
> >>>>>>>> Note that since is_combocid is true the numbers are not cmin/cmax
> >>>>>>>> they are actually
> >>>>>>>> the indexes of the internal array already explained above.
> >>>>>>>> combo id index 0 would contain cmin 3, cmax 6
> >>>>>>>> combo id index 1 would contain cmin 4, cmax 6
> >>>>>>>> combo id index 2 would contain cmin 5, cmax 6
> >>>>>>>>
> >>>>>>>> test=#
> >>>>>>>> test=# END;
> >>>>>>>> COMMIT
> >>>>>>>> test=#
> >>>>>>>> test=#
> >>>>>>>> test=# TRUNCATE mvcc_demo;
> >>>>>>>> TRUNCATE TABLE
> >>>>>>>> test=#
> >>>>>>>> test=#
> >>>>>>>> test=#
> >>>>>>>> test=# BEGIN;
> >>>>>>>> BEGIN
> >>>>>>>> test=# INSERT INTO mvcc_demo VALUES (1);
> >>>>>>>> INSERT 0 1
> >>>>>>>> test=# INSERT INTO mvcc_demo VALUES (2);
> >>>>>>>> INSERT 0 1
> >>>>>>>> test=# INSERT INTO mvcc_demo VALUES (3);
> >>>>>>>> INSERT 0 1
> >>>>>>>> test=# SELECT t_xmin AS xmin,
> >>>>>>>> test-#        t_xmax::text::int8 AS xmax,
> >>>>>>>> test-#        t_field3::text::int8 AS cmin_cmax,
> >>>>>>>> test-#        (t_infomask::integer & X'0020'::integer)::bool AS
> >>>>>>>> is_combocid
> >>>>>>>> test-# FROM heap_page_items(get_raw_page('mvcc_demo', 0))
> >>>>>>>> test-# ORDER BY 2 DESC, 3;
> >>>>>>>>  xmin  | xmax | cmin_cmax | is_combocid
> >>>>>>>> -------+------+-----------+-------------
> >>>>>>>>  80675 |    0 |         0 | f
> >>>>>>>>  80675 |    0 |         1 | f
> >>>>>>>>  80675 |    0 |         2 | f
> >>>>>>>> (3 rows)
> >>>>>>>>
> >>>>>>>> test=#
> >>>>>>>> test=# UPDATE mvcc_demo SET val = val * 10;
> >>>>>>>> UPDATE 3
> >>>>>>>> test=#
> >>>>>>>> test=# SELECT t_xmin AS xmin,
> >>>>>>>> test-#        t_xmax::text::int8 AS xmax,
> >>>>>>>> test-#        t_field3::text::int8 AS cmin_cmax,
> >>>>>>>> test-#        (t_infomask::integer & X'0020'::integer)::bool AS
> >>>>>>>> is_combocid
> >>>>>>>> test-# FROM heap_page_items(get_raw_page('mvcc_demo', 0))
> >>>>>>>> test-# ORDER BY 2 DESC, 3;
> >>>>>>>>  xmin  | xmax  | cmin_cmax | is_combocid
> >>>>>>>> -------+-------+-----------+-------------
> >>>>>>>>  80675 | 80675 |         0 | t
> >>>>>>>>  80675 | 80675 |         1 | t
> >>>>>>>>  80675 | 80675 |         2 | t
> >>>>>>>>  80675 |     0 |         3 | f
> >>>>>>>>  80675 |     0 |         3 | f
> >>>>>>>>  80675 |     0 |         3 | f
> >>>>>>>> (6 rows)
> >>>>>>>>
> >>>>>>>> test=#
> >>>>>>>> test=# END;
> >>>>>>>> COMMIT
> >>>>>>>> test=#
> >>>>>>>> test=#
> >>>>>>>> test=# TRUNCATE mvcc_demo;
> >>>>>>>> TRUNCATE TABLE
> >>>>>>>> test=#
> >>>>>>>>
> >>>>>>>>     -- From one psql issue
> >>>>>>>>     test=# INSERT INTO mvcc_demo VALUES (1);
> >>>>>>>>     INSERT 0 1
> >>>>>>>>     test=# SELECT t_xmin AS xmin,
> >>>>>>>>     test-#        t_xmax::text::int8 AS xmax,
> >>>>>>>>     test-#        t_field3::text::int8 AS cmin_cmax,
> >>>>>>>>     test-#        (t_infomask::integer & X'0020'::integer)::bool
> AS
> >>>>>>>> is_combocid
> >>>>>>>>     test-# FROM heap_page_items(get_raw_page('mvcc_demo', 0))
> >>>>>>>>     test-# ORDER BY 2 DESC, 3;
> >>>>>>>>      xmin  | xmax | cmin_cmax | is_combocid
> >>>>>>>>     -------+------+-----------+-------------
> >>>>>>>>      80677 |    0 |         0 | f
> >>>>>>>>     (1 row)
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> test=# -- From another issue
> >>>>>>>> test=# BEGIN;
> >>>>>>>> BEGIN
> >>>>>>>> test=# INSERT INTO mvcc_demo VALUES (2);
> >>>>>>>> INSERT 0 1
> >>>>>>>> test=# INSERT INTO mvcc_demo VALUES (3);
> >>>>>>>> INSERT 0 1
> >>>>>>>> test=# INSERT INTO mvcc_demo VALUES (4);
> >>>>>>>> INSERT 0 1
> >>>>>>>> test=# SELECT t_xmin AS xmin,
> >>>>>>>> test-#        t_xmax::text::int8 AS xmax,
> >>>>>>>> test-#        t_field3::text::int8 AS cmin_cmax,
> >>>>>>>> test-#        (t_infomask::integer & X'0020'::integer)::bool AS
> >>>>>>>> is_combocid
> >>>>>>>> test-# FROM heap_page_items(get_raw_page('mvcc_demo', 0))
> >>>>>>>> test-# ORDER BY 2 DESC, 3;
> >>>>>>>>  xmin  | xmax | cmin_cmax | is_combocid
> >>>>>>>> -------+------+-----------+-------------
> >>>>>>>>  80677 |    0 |         0 | f
> >>>>>>>>  80678 |    0 |         0 | f
> >>>>>>>>  80678 |    0 |         1 | f
> >>>>>>>>  80678 |    0 |         2 | f
> >>>>>>>> (4 rows)
> >>>>>>>>
> >>>>>>>> test=#
> >>>>>>>> test=# UPDATE mvcc_demo SET val = val * 10;
> >>>>>>>> UPDATE 4
> >>>>>>>> test=# SELECT t_xmin AS xmin,
> >>>>>>>> test-#        t_xmax::text::int8 AS xmax,
> >>>>>>>> test-#        t_field3::text::int8 AS cmin_cmax,
> >>>>>>>> test-#        (t_infomask::integer & X'0020'::integer)::bool AS
> >>>>>>>> is_combocid
> >>>>>>>> test-# FROM heap_page_items(get_raw_page('mvcc_demo', 0))
> >>>>>>>> test-# ORDER BY 2 DESC, 3;
> >>>>>>>>  xmin  | xmax  | cmin_cmax | is_combocid
> >>>>>>>> -------+-------+-----------+-------------
> >>>>>>>>  80678 | 80678 |         0 | t
> >>>>>>>>  80678 | 80678 |         1 | t
> >>>>>>>>  80678 | 80678 |         2 | t
> >>>>>>>>  80677 | 80678 |         3 | f
> >>>>>>>>  80678 |     0 |         3 | f
> >>>>>>>>  80678 |     0 |         3 | f
> >>>>>>>>  80678 |     0 |         3 | f
> >>>>>>>>  80678 |     0 |         3 | f
> >>>>>>>> (8 rows)
> >>>>>>>>
> >>>>>>>> test=#
> >>>>>>>>
> >>>>>>>>     test=# -- Before finishing this, issue these from the first
> psql
> >>>>>>>>     test=# SELECT t_xmin AS xmin,
> >>>>>>>>     test-#        t_xmax::text::int8 AS xmax,
> >>>>>>>>     test-#        t_field3::text::int8 AS cmin_cmax,
> >>>>>>>>     test-#        (t_infomask::integer & X'0020'::integer)::bool
> AS
> >>>>>>>> is_combocid
> >>>>>>>>     test-# FROM heap_page_items(get_raw_page('mvcc_demo', 0))
> >>>>>>>>     test-# ORDER BY 2 DESC, 3;
> >>>>>>>>      xmin  | xmax  | cmin_cmax | is_combocid
> >>>>>>>>     -------+-------+-----------+-------------
> >>>>>>>>      80678 | 80678 |         0 | t
> >>>>>>>>      80678 | 80678 |         1 | t
> >>>>>>>>      80678 | 80678 |         2 | t
> >>>>>>>>      80677 | 80678 |         3 | f
> >>>>>>>>      80678 |     0 |         3 | f
> >>>>>>>>      80678 |     0 |         3 | f
> >>>>>>>>      80678 |     0 |         3 | f
> >>>>>>>>      80678 |     0 |         3 | f
> >>>>>>>>     (8 rows)
> >>>>>>>>
> >>>>>>>> test=# END;
> >>>>>>>> COMMIT
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Now consider the case we are trying to solve
> >>>>>>>>
> >>>>>>>> drop table tt1;
> >>>>>>>> create table tt1(f1 int);
> >>>>>>>>
> >>>>>>>> BEGIN;
> >>>>>>>>  insert into tt1 values(1);
> >>>>>>>>  declare c50 cursor for select * from tt1; -- should show one row
> >>>>>>>> only
> >>>>>>>>  insert into tt1 values(2);
> >>>>>>>>  fetch all from c50;
> >>>>>>>> COMMIT;
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Consider Data node 1 log
> >>>>>>>>
> >>>>>>>> (a) [exec_simple_query][1026][START TRANSACTION ISOLATION LEVEL
> read
> >>>>>>>> committed READ WRITE]
> >>>>>>>> (b) [exec_simple_query][1026][drop table tt1;]
> >>>>>>>> (c) [exec_simple_query][1026][PREPARE TRANSACTION 'T21075']
> >>>>>>>> (d) [exec_simple_query][1026][COMMIT PREPARED 'T21075']
> >>>>>>>> (e) [exec_simple_query][1026][START TRANSACTION ISOLATION LEVEL
> read
> >>>>>>>> committed READ WRITE]
> >>>>>>>> (f) [exec_simple_query][1026][create table tt1(f1 int);]
> >>>>>>>> (g) [exec_simple_query][1026][PREPARE TRANSACTION 'T21077']
> >>>>>>>> (h) [exec_simple_query][1026][COMMIT PREPARED 'T21077']
> >>>>>>>> (i) [exec_simple_query][1026][START TRANSACTION ISOLATION LEVEL
> read
> >>>>>>>> committed READ WRITE]
> >>>>>>>> (j) [exec_simple_query][1026][INSERT INTO tt1 (f1) VALUES (1)]
> >>>>>>>> (k) [exec_simple_query][1026][INSERT INTO tt1 (f1) VALUES (2)]
> >>>>>>>> (l) [PostgresMain][4155][SELECT tt1.f1, tt1.ctid, pgxc_node_str()
> >>>>>>>> FROM tt1]
> >>>>>>>> (m) [exec_simple_query][1026][COMMIT TRANSACTION]
> >>>>>>>>
> >>>>>>>> The cursor currently shows both inserted rows because command id
> at
> >>>>>>>> data node in
> >>>>>>>> step (j) is 0
> >>>>>>>> step (k) is 1 &
> >>>>>>>> step (l) is 2
> >>>>>>>>
> >>>>>>>> Where as we need command ids to be
> >>>>>>>>
> >>>>>>>> step (j) should be 0
> >>>>>>>> step (k) should be 2 &
> >>>>>>>> step (l) should be 1
> >>>>>>>>
> >>>>>>>> This will solve the cursor visibility problem.
> >>>>>>>>
> >>>>>>>> To implement this I suggest we send command IDs to data nodes from
> >>>>>>>> the coordinator
> >>>>>>>> like we send gxid. The only difference will be that we do not need
> >>>>>>>> to take command IDs
> >>>>>>>> from GTM since they are only valid with in the transaction.
> >>>>>>>>
> >>>>>>>> See this example
> >>>>>>>>
> >>>>>>>> test=# select xmin,xmax,cmin,cmax,* from tt1;
> >>>>>>>>  xmin | xmax | cmin | cmax | f1
> >>>>>>>> ------+------+------+------+----
> >>>>>>>> (0 rows)
> >>>>>>>>
> >>>>>>>> test=# begin;
> >>>>>>>> BEGIN
> >>>>>>>> test=#  insert into tt1 values(1);
> >>>>>>>> INSERT 0 1
> >>>>>>>> test=# select xmin,xmax,cmin,cmax,* from tt1;
> >>>>>>>>  xmin  | xmax | cmin | cmax | f1
> >>>>>>>> -------+------+------+------+----
> >>>>>>>>  80615 |    0 |    0 |    0 |  1
> >>>>>>>> (1 row)
> >>>>>>>>
> >>>>>>>> test=#  insert into tt1 values(2);
> >>>>>>>> INSERT 0 1
> >>>>>>>> test=# select xmin,xmax,cmin,cmax,* from tt1;
> >>>>>>>>  xmin  | xmax | cmin | cmax | f1
> >>>>>>>> -------+------+------+------+----
> >>>>>>>>  80615 |    0 |    0 |    0 |  1
> >>>>>>>>  80615 |    0 |    1 |    1 |  2
> >>>>>>>> (2 rows)
> >>>>>>>>
> >>>>>>>> test=#  insert into tt1 values(3);
> >>>>>>>> INSERT 0 1
> >>>>>>>> test=# select xmin,xmax,cmin,cmax,* from tt1;
> >>>>>>>>  xmin  | xmax | cmin | cmax | f1
> >>>>>>>> -------+------+------+------+----
> >>>>>>>>  80615 |    0 |    0 |    0 |  1
> >>>>>>>>  80615 |    0 |    1 |    1 |  2
> >>>>>>>>  80615 |    0 |    2 |    2 |  3
> >>>>>>>> (3 rows)
> >>>>>>>>
> >>>>>>>> test=#  insert into tt1 values(4);
> >>>>>>>> INSERT 0 1
> >>>>>>>> test=# select xmin,xmax,cmin,cmax,* from tt1;
> >>>>>>>>  xmin  | xmax | cmin | cmax | f1
> >>>>>>>> -------+------+------+------+----
> >>>>>>>>  80615 |    0 |    0 |    0 |  1
> >>>>>>>>  80615 |    0 |    1 |    1 |  2
> >>>>>>>>  80615 |    0 |    2 |    2 |  3
> >>>>>>>>  80615 |    0 |    3 |    3 |  4
> >>>>>>>> (4 rows)
> >>>>>>>>
> >>>>>>>> test=# end;
> >>>>>>>> COMMIT
> >>>>>>>> test=#
> >>>>>>>> test=#
> >>>>>>>> test=# select xmin,xmax,cmin,cmax,* from tt1;
> >>>>>>>>  xmin  | xmax | cmin | cmax | f1
> >>>>>>>> -------+------+------+------+----
> >>>>>>>>  80615 |    0 |    0 |    0 |  1
> >>>>>>>>  80615 |    0 |    1 |    1 |  2
> >>>>>>>>  80615 |    0 |    2 |    2 |  3
> >>>>>>>>  80615 |    0 |    3 |    3 |  4
> >>>>>>>> (4 rows)
> >>>>>>>>
> >>>>>>>> test=#  insert into tt1 values(5);
> >>>>>>>> INSERT 0 1
> >>>>>>>> test=# select xmin,xmax,cmin,cmax,* from tt1;
> >>>>>>>>  xmin  | xmax | cmin | cmax | f1
> >>>>>>>> -------+------+------+------+----
> >>>>>>>>  80615 |    0 |    0 |    0 |  1
> >>>>>>>>  80615 |    0 |    1 |    1 |  2
> >>>>>>>>  80615 |    0 |    2 |    2 |  3
> >>>>>>>>  80615 |    0 |    3 |    3 |  4
> >>>>>>>>  80616 |    0 |    0 |    0 |  5
> >>>>>>>> (5 rows)
> >>>>>>>>
> >>>>>>>> test=#  insert into tt1 values(6);
> >>>>>>>> INSERT 0 1
> >>>>>>>> test=#
> >>>>>>>> test=#
> >>>>>>>> test=# select xmin,xmax,cmin,cmax,* from tt1;
> >>>>>>>>  xmin  | xmax | cmin | cmax | f1
> >>>>>>>> -------+------+------+------+----
> >>>>>>>>  80615 |    0 |    0 |    0 |  1
> >>>>>>>>  80615 |    0 |    1 |    1 |  2
> >>>>>>>>  80615 |    0 |    2 |    2 |  3
> >>>>>>>>  80615 |    0 |    3 |    3 |  4
> >>>>>>>>  80616 |    0 |    0 |    0 |  5
> >>>>>>>>  80617 |    0 |    0 |    0 |  6
> >>>>>>>> (6 rows)
> >>>>>>>>
> >>>>>>>> Note that at the end of the multi-statement transaction the
> command
> >>>>>>>> id gets reset to zero.
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Abbas
> >>>>>>>> Architect
> >>>>>>>> EnterpriseDB Corporation
> >>>>>>>> The Enterprise PostgreSQL Company
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> --
> >>>>>>> Abbas
> >>>>>>> Architect
> >>>>>>> EnterpriseDB Corporation
> >>>>>>> The Enterprise PostgreSQL Company
> >>>>>>>
> >>>>>>> Phone: 92-334-5100153
> >>>>>>>
> >>>>>>> Website: www.enterprisedb.com
> >>>>>>> EnterpriseDB Blog: http://blogs.enterprisedb.com/
> >>>>>>> Follow us on Twitter: http://www.twitter.com/enterprisedb
> >>>>>>>
> >>>>>>> This e-mail message (and any attachment) is intended for the use of
> >>>>>>> the individual or entity to whom it is addressed. This message
> >>>>>>> contains information from EnterpriseDB Corporation that may be
> >>>>>>> privileged, confidential, or exempt from disclosure under
> applicable
> >>>>>>> law. If you are not the intended recipient or authorized to receive
> >>>>>>> this for the intended recipient, any use, dissemination,
> >>>>>>> distribution,
> >>>>>>> retention, archiving, or copying of this communication is strictly
> >>>>>>> prohibited. If you have received this e-mail in error, please
> notify
> >>>>>>> the sender immediately by reply e-mail and delete this message.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> ------------------------------------------------------------------------------
> >>>>>>> Live Security Virtual Conference
> >>>>>>> Exclusive live event will cover all the ways today's security and
> >>>>>>> threat landscape has changed and how IT managers can respond.
> >>>>>>> Discussions
> >>>>>>> will include endpoint security, mobile security and the latest in
> >>>>>>> malware
> >>>>>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> >>>>>>> _______________________________________________
> >>>>>>> Postgres-xc-developers mailing list
> >>>>>>> Pos...@li...
> >>>>>>>
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Michael Paquier
> >>>>>> http://michael.otacoo.com
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> --
> >>>>> Abbas
> >>>>> Architect
> >>>>> EnterpriseDB Corporation
> >>>>> The Enterprise PostgreSQL Company
> >>>>>
> >>>>> Phone: 92-334-5100153
> >>>>>
> >>>>> Website: www.enterprisedb.com
> >>>>> EnterpriseDB Blog: http://blogs.enterprisedb.com/
> >>>>> Follow us on Twitter: http://www.twitter.com/enterprisedb
> >>>>>
> >>>>> This e-mail message (and any attachment) is intended for the use of
> >>>>> the individual or entity to whom it is addressed. This message
> >>>>> contains information from EnterpriseDB Corporation that may be
> >>>>> privileged, confidential, or exempt from disclosure under applicable
> >>>>> law. If you are not the intended recipient or authorized to receive
> >>>>> this for the intended recipient, any use, dissemination,
> distribution,
> >>>>> retention, archiving, or copying of this communication is strictly
> >>>>> prohibited. If you have received this e-mail in error, please notify
> >>>>> the sender immediately by reply e-mail and delete this message.
> >>>>>
> >>>>>
> >>>>>
> ------------------------------------------------------------------------------
> >>>>> Live Security Virtual Conference
> >>>>> Exclusive live event will cover all the ways today's security and
> >>>>> threat landscape has changed and how IT managers can respond.
> >>>>> Discussions
> >>>>> will include endpoint security, mobile security and the latest in
> >>>>> malware
> >>>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> >>>>> _______________________________________________
> >>>>> Postgres-xc-developers mailing list
> >>>>> Pos...@li...
> >>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Best Wishes,
> >>>> Ashutosh Bapat
> >>>> EntepriseDB Corporation
> >>>> The Enterprise Postgres Company
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> --
> >>> Abbas
> >>> Architect
> >>> EnterpriseDB Corporation
> >>> The Enterprise PostgreSQL Company
> >>>
> >>> Phone: 92-334-5100153
> >>>
> >>> Website: www.enterprisedb.com
> >>> EnterpriseDB Blog: http://blogs.enterprisedb.com/
> >>> Follow us on Twitter: http://www.twitter.com/enterprisedb
> >>>
> >>> This e-mail message (and any attachment) is intended for the use of
> >>> the individual or entity to whom it is addressed. This message
> >>> contains information from EnterpriseDB Corporation that may be
> >>> privileged, confidential, or exempt from disclosure under applicable
> >>> law. If you are not the intended recipient or authorized to receive
> >>> this for the intended recipient, any use, dissemination, distribution,
> >>> retention, archiving, or copying of this communication is strictly
> >>> prohibited. If you have received this e-mail in error, please notify
> >>> the sender immediately by reply e-mail and delete this message.
> >>
> >>
> >>
> >>
> >> --
> >> --
> >> Abbas
> >> Architect
> >> EnterpriseDB Corporation
> >> The Enterprise PostgreSQL Company
> >>
> >> Phone: 92-334-5100153
> >>
> >> Website: www.enterprisedb.com
> >> EnterpriseDB Blog: http://blogs.enterprisedb.com/
> >> Follow us on Twitter: http://www.twitter.com/enterprisedb
> >>
> >> This e-mail message (and any attachment) is intended for the use of
> >> the individual or entity to whom it is addressed. This message
> >> contains information from EnterpriseDB Corporation that may be
> >> privileged, confidential, or exempt from disclosure under applicable
> >> law. If you are not the intended recipient or authorized to receive
> >> this for the intended recipient, any use, dissemination, distribution,
> >> retention, archiving, or copying of this communication is strictly
> >> prohibited. If you have received this e-mail in error, please notify
> >> the sender immediately by reply e-mail and delete this message.
> >
> >
> >
> >
> > --
> > --
> > Abbas
> > Architect
> > EnterpriseDB Corporation
> > The Enterprise PostgreSQL Company
> >
> > Phone: 92-334-5100153
> >
> > Website: www.enterprisedb.com
> > EnterpriseDB Blog: http://blogs.enterprisedb.com/
> > Follow us on Twitter: http://www.twitter.com/enterprisedb
> >
> > This e-mail message (and any attachment) is intended for the use of
> > the individual or entity to whom it is addressed. This message
> > contains information from EnterpriseDB Corporation that may be
> > privileged, confidential, or exempt from disclosure under applicable
> > law. If you are not the intended recipient or authorized to receive
> > this for the intended recipient, any use, dissemination, distribution,
> > retention, archiving, or copying of this communication is strictly
> > prohibited. If you have received this e-mail in error, please notify
> > the sender immediately by reply e-mail and delete this message.
> >
> >
> ------------------------------------------------------------------------------
> > Live Security Virtual Conference
> > Exclusive live event will cover all the ways today's security and
> > threat landscape has changed and how IT managers can respond. Discussions
> > will include endpoint security, mobile security and the latest in malware
> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> > _______________________________________________
> > Postgres-xc-developers mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >
>



-- 
--
Abbas
Architect
EnterpriseDB Corporation
The Enterprise PostgreSQL Company

Phone: 92-334-5100153

Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb

This e-mail message (and any attachment) is intended for the use of
the individual or entity to whom it is addressed. This message
contains information from EnterpriseDB Corporation that may be
privileged, confidential, or exempt from disclosure under applicable
law. If you are not the intended recipient or authorized to receive
this for the intended recipient, any use, dissemination, distribution,
retention, archiving, or copying of this communication is strictly
prohibited. If you have received this e-mail in error, please notify
the sender immediately by reply e-mail and delete this message.

看得见的朋友a.s_link{display:block; background-color:#cc1717; color:#fff; width:100px; height:28px; line-height:28px; text-align:center; text-decoration:none; }a.s_link:hover { background-color : #B11414; }                   
                  
                                                             　　我住在北京天通苑小区，我在搜旁网认识很多住的非常近的人，我喜欢台球和爬山，希望你在这里也能找到见得着的朋友:)            接受邀请          
                           
                   

如您不希望再次收到我们的邮件，请随时点击这里取消订阅。

>> Probably Nikhil misunderstood intent of CREATE NODE. CREATE NODE has
>> nothing to do with redistribution.
>
> Yes, think so.
>>

Well, in some ways yes. But I was looking at it from the viewpoint of
auto-redistribution of some tables based on the addition of a new
datanode for example.

Regards,
Nikhils

>>
>>>
>>> - For certain redistribution operations, we do NOT always need a
>>> transaction block! Just take the example of a replicated table changed to a
>>> distributed table. You just need to send a DELETE query to remote nodes to
>>> remove only the tuples that do not satisfy a hash condition. This is one of
>>> those things I am working on now.
>>
>>
>> We will need transaction block to make the utility transactionally safe.
>>>
>>>
>>> To my mind, each argument here makes necessary this feature in core. All
>>> combined even strengthen my arguments.
>>>
>>
>> I am not able to answer still serious problems -
>> 1. Checking permissions to allow such a function to be run. Should we
>> allow anyone who can run those DDLs/DMLs run the utility?
>
> Only the owner of the table. But this needs to be controlled internally. We
> may have serious security issues lying under that.
>
>>
>> 2. Cache invalidation - how do we invalidate plan caches after
>> redistributing the data? Plan stores the list of nodes where the queries
>> should be executed. Storing this list of nodes in plans is essential from
>> JOIN reduction.
>
> Yes, and for a couple of other things. You will need to add externally some
> runs of DEALLOCATE ALL, but this will impact all tables and not only the one
> being redistributed. My latest patch only invalidated cache of the table
> involved by redistribution.
>
>>
>>
>> Are there answers to these questions?
>>
>> But this approach has some merits - 1. It reduces the impact on the code.
>
> Yes, but redistribution needs to be a core operation.
>
>>
>> 2. There are few things not so good in this approach, like disk and
>> network usage, which needs to be optimized. By taking this approach, we
>> provide an easy (albeit unoptimised) way of re-distributing the tables and
>> can work on the final version of code changes for ALTER TABLE DISTRIBUTE
>> afterwards, keeping longer slot for the work.
>
> In this case you do not need any code! You could also do a simple CREATE
> TABLE AS to redistribution the table as you wish to a new table, drop the
> old table, and rename the new table with the old name. This could also be
> done with 1.0.
> You should also have a look at my latest patch, it already includes the
> maximum optimizations possible for replicated table redistribution,
> particularly distrib.c. Just by looking at that you will see that a node
> level control is more than necessary.
> --
> Michael Paquier
> http://michael.otacoo.com



-- 
StormDB - http://www.stormdb.com
The Database Cloud

(Moving to dev...)

On Wed, Jul 18, 2012 at 3:04 PM, Ashutosh Bapat <
ash...@en...> wrote:

> Hi Suzuki-san,
> Thanks for this commit. Hope people configuring HA find it useful.
>
> It will also be helpful, to add documentation as to how to configure and
> use this utility. How to specify which nodes it should monitor and if there
> are any options in monitoring. If it allows various modes of notifying
> failure, how to use those, etc.
>
Some docs have been written but Suzuki-san forgot to include them in the
commit.
He is just adding them now.


>
> On Wed, Jul 18, 2012 at 11:27 AM, Koichi Suzuki <
> koi...@us...> wrote:
>
>> Project "Postgres-XC".
>>
>> The branch, master has been updated
>>        via  f6bbc44a1afefd56fb7c7d54d7a921cc648d45c3 (commit)
>>       from  4371aadf993ff4fce458362b0de8d848bbe96e4a (commit)
>>
>>
>> - Log -----------------------------------------------------------------
>>
>> http://postgres-xc.git.sourceforge.net/git/gitweb.cgi?p=postgres-xc/postgres-xc;a=commitdiff;h=f6bbc44a1afefd56fb7c7d54d7a921cc648d45c3
>>
>> commit f6bbc44a1afefd56fb7c7d54d7a921cc648d45c3
>> Author: Koichi Suzuki <koi...@gm...>
>> Date:   Wed Jul 18 14:59:24 2012 +0900
>>
>>     This commit adds pgxc_monitor utility used to check each Postgres-XC
>>     node is up and running.
>>
>> A       contrib/pgxc_monitor/Makefile
>> A       contrib/pgxc_monitor/mcxt.c
>> A       contrib/pgxc_monitor/pgxc_monitor.c
>> M       doc-xc/src/sgml/contrib.sgmlin
>> M       doc-xc/src/sgml/filelist.sgmlin
>>
>> -----------------------------------------------------------------------
>>
>> Summary of changes:
>>  contrib/{pgxc_clean => pgxc_monitor}/Makefile |   11 +-
>>  contrib/pgxc_monitor/mcxt.c                   |   77 +++++++++
>>  contrib/pgxc_monitor/pgxc_monitor.c           |  211
>> +++++++++++++++++++++++++
>>  doc-xc/src/sgml/contrib.sgmlin                |    1 +
>>  doc-xc/src/sgml/filelist.sgmlin               |    3 +
>>  5 files changed, 298 insertions(+), 5 deletions(-)
>>  copy contrib/{pgxc_clean => pgxc_monitor}/Makefile (78%)
>>  create mode 100644 contrib/pgxc_monitor/mcxt.c
>>  create mode 100644 contrib/pgxc_monitor/pgxc_monitor.c
>>
>>
>> hooks/post-receive
>> --
>> Postgres-XC
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Postgres-xc-committers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-committers
>>
>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EntepriseDB Corporation
> The Enterprise Postgres Company
>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Postgres-xc-committers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-committers
>
>


-- 
Michael Paquier
http://michael.otacoo.com