postgres-xc-developers Mailing List for Postgres-XC

On Thu, Mar 8, 2012 at 8:13 PM, Ashutosh Bapat <
ash...@en...> wrote:

> Hi All,
> PFA patch for cleanup of dead-code. There are many functions and structure
> members, which are not used by the new FQS code. Removed them.
>
> Regression does not show any new error.
>
This patch looks good, but it is not realigned with the latest head. Could
you realign it?
Except that, the content looks OK.
-- 
Michael Paquier
http://michael.otacoo.com

On Thu, Mar 8, 2012 at 11:25 PM, Ahsan Hadi <ahs...@en...>wrote:

>
>
> On Thu, Mar 8, 2012 at 8:36 AM, Michael Paquier <mic...@gm...
> > wrote:
>
>> There is also something I forgot to propose.
>> Once we think we are done with beta1.
>> I think we'll also need a beta2 phase because we may need some
>> stabilization work after backporting all the patches of 9.1 stable branch
>> of postgres.
>> This should not be long though.
>
>
> Hmm...that seems like an important task...do we know which patches we are
> missing from 91 stable branch? That might cause some stabilization issues
> hopefully not...
>
Not that huge. It consists in doing a merge with 9.1 stable branch with our
future 1.0 stable branch. In 9.1 stable branch there are only bugs fixes,
and XC code has minimal impact on Postgres' one. So I expect the merge to
take less 5 minutes. The commits to be merged are from 9.1 beta2 to the
latest point of 9.1 stable.

So, just for safety, I want to add a beta2 tag to insure that we do some
stabilization checks before switching to 1.0.



>
>>
>> On Tue, Mar 6, 2012 at 2:21 PM, Koichi Suzuki <koi...@gm...>wrote:
>>
>>> In terms of the planner improvement, this is never-ending effort we
>>> should focus on.   You see Tom Lane is paying much effort to improve
>>> vanilla PostgreSQL planner even now.   (He is trying to do this even
>>> after the code is frozen!)
>>>
>>> For 1.0, it seems to me at reasonable level and we should concentrate
>>> on the stabilization.
>>>
>>> Regarding the report from Michael and related discussions, I'm
>>> thinking to carry over the trigger to later version.    To announce
>>> that 1.0 has sufficient HA feature, I'd like to keep pgxc_clean
>>> effort, which needs a bit of extension to the core, although most of
>>> the work is outside the core.
>>>
>>> Regards;
>>> ----------
>>> Koichi Suzuki
>>>
>>>
>>>
>>> 2012/3/6 Ahsan Hadi <ahs...@en...>:
>>> > I think this makes a lot of sense, we should be gearing towards
>>> code-freeze
>>> > for 1.0 by end of this month. At this point most to of our efforts
>>> should be
>>> > focussed on regression stabilization and fixing important bugs from
>>> the bug
>>> > tracking system.
>>> >
>>> > Amit is already looking at regression failures, Abbas started analysis
>>> of
>>> > the open bugs yesterday, once he is done with his analysis, he will
>>> close
>>> > the bugs already fixed, divide the remaining ones between bugs and
>>> > enhancements and move the obvious ones that we cannot target this
>>> month to
>>> > beyond 1.0. Amit and Abbas can focus on regression fixes and other bug
>>> fixes
>>> > for the remainder of this month.
>>> >
>>> > The planner improvements was an important piece, i assume what was
>>> needed
>>> > for 1.0 is already in place?
>>> >
>>> >
>>> > On Tue, Mar 6, 2012 at 7:47 AM, Michael Paquier <
>>> mic...@gm...>
>>> > wrote:
>>> >>
>>> >> Hi all,
>>> >>
>>> >> As we are entering in the 1.0 release process, I propose that we do a
>>> >> feature freeze at the end of the month.
>>> >> This means that no new features will be added to core after the 31st
>>> of
>>> >> March, but external utilities like pgxc_clean and initgtm (that I
>>> should
>>> >> finish before this date though), do not enter in this scope.
>>> >> So, from this date, all the effort will be focused on stabilization
>>> and
>>> >> bug fix.
>>> >> On the 1st of April, I also propose to mark XC not as 1.0devel but as
>>> >> 1.0beta1.
>>> >> This could be delayed a couple of days of course if a feature we think
>>> >> necessary is late.
>>> >>
>>> >> Any thoughts/comments?
>>> >> --
>>> >> Michael Paquier
>>> >> http://michael.otacoo.com
>>> >>
>>> >>
>>> >>
>>> ------------------------------------------------------------------------------
>>> >> Keep Your Developer Skills Current with LearnDevNow!
>>> >> The most comprehensive online learning library for Microsoft
>>> developers
>>> >> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3,
>>> MVC3,
>>> >> Metro Style Apps, more. Free future releases when you subscribe now!
>>> >> http://p.sf.net/sfu/learndevnow-d2d
>>> >> _______________________________________________
>>> >> Postgres-xc-developers mailing list
>>> >> Pos...@li...
>>> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > Ahsan Hadi
>>> > Snr Director Product Development
>>> > EnterpriseDB Corporation
>>> > The Enterprise Postgres Company
>>> >
>>> > Phone: +92-51-8358874
>>> > Mobile: +92-333-5162114
>>> >
>>> > Website: www.enterprisedb.com
>>> > EnterpriseDB Blog: http://blogs.enterprisedb.com/
>>> > Follow us on Twitter: http://www.twitter.com/enterprisedb
>>> >
>>> > This e-mail message (and any attachment) is intended for the use of the
>>> > individual or entity to whom it is addressed. This message contains
>>> > information from EnterpriseDB Corporation that may be privileged,
>>> > confidential, or exempt from disclosure under applicable law. If you
>>> are not
>>> > the intended recipient or authorized to receive this for the intended
>>> > recipient, any use, dissemination, distribution, retention, archiving,
>>> or
>>> > copying of this communication is strictly prohibited. If you have
>>> received
>>> > this e-mail in error, please notify the sender immediately by reply
>>> e-mail
>>> > and delete this message.
>>> >
>>> >
>>> ------------------------------------------------------------------------------
>>> > Keep Your Developer Skills Current with LearnDevNow!
>>> > The most comprehensive online learning library for Microsoft developers
>>> > is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3,
>>> MVC3,
>>> > Metro Style Apps, more. Free future releases when you subscribe now!
>>> > http://p.sf.net/sfu/learndevnow-d2d
>>> > _______________________________________________
>>> > Postgres-xc-developers mailing list
>>> > Pos...@li...
>>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>> >
>>>
>>
>>
>>
>> --
>> Michael Paquier
>> http://michael.otacoo.com
>>
>
>
>
> --
> Ahsan Hadi
> Snr Director Product Development
> EnterpriseDB Corporation
> The Enterprise Postgres Company
>
> Phone: +92-51-8358874
> Mobile: +92-333-5162114
>
> Website: www.enterprisedb.com
> EnterpriseDB Blog: http://blogs.enterprisedb.com/
> Follow us on Twitter: http://www.twitter.com/enterprisedb
>
> This e-mail message (and any attachment) is intended for the use of the
> individual or entity to whom it is addressed. This message contains
> information from EnterpriseDB Corporation that may be privileged,
> confidential, or exempt from disclosure under applicable law. If you are
> not the intended recipient or authorized to receive this for the intended
> recipient, any use, dissemination, distribution, retention, archiving, or
> copying of this communication is strictly prohibited. If you have received
> this e-mail in error, please notify the sender immediately by reply e-mail
> and delete this message.
>



-- 
Michael Paquier
http://michael.otacoo.com

On Thu, Mar 8, 2012 at 10:38 AM, Michael Paquier
<mic...@gm...>wrote:

> Hi all,
>
> A lot of commits reordering XC error handling and remote planning has
> happened.
> So here are the latest regression test results.
> 38 tests failed for a total of 129. 4~5 tests, perhaps more are failing
> because of diffs not updated after latest commits.
>

thanks...I think it is important to go through all the regression failures
for

1- check that we not introduced any new issues
2- make sure all the regression failures are captured in the bug tracking
system so we can fix it one by one...

I believe Amit is already doing regression analysis, Abbas and Ashutosh
should join soon...

Suzuki-san,
We should go through all the active bugs in the developer meeting next week?



>
> Regards,
> --
> Michael Paquier
> http://michael.otacoo.com
>
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>


-- 
Ahsan Hadi
Snr Director Product Development
EnterpriseDB Corporation
The Enterprise Postgres Company

Phone: +92-51-8358874
Mobile: +92-333-5162114

Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb

This e-mail message (and any attachment) is intended for the use of the
individual or entity to whom it is addressed. This message contains
information from EnterpriseDB Corporation that may be privileged,
confidential, or exempt from disclosure under applicable law. If you are
not the intended recipient or authorized to receive this for the intended
recipient, any use, dissemination, distribution, retention, archiving, or
copying of this communication is strictly prohibited. If you have received
this e-mail in error, please notify the sender immediately by reply e-mail
and delete this message.

On Thu, Mar 8, 2012 at 8:36 AM, Michael Paquier
<mic...@gm...>wrote:

> There is also something I forgot to propose.
> Once we think we are done with beta1.
> I think we'll also need a beta2 phase because we may need some
> stabilization work after backporting all the patches of 9.1 stable branch
> of postgres.
> This should not be long though.


Hmm...that seems like an important task...do we know which patches we are
missing from 91 stable branch? That might cause some stabilization issues
hopefully not...

>
>
> On Tue, Mar 6, 2012 at 2:21 PM, Koichi Suzuki <koi...@gm...>wrote:
>
>> In terms of the planner improvement, this is never-ending effort we
>> should focus on.   You see Tom Lane is paying much effort to improve
>> vanilla PostgreSQL planner even now.   (He is trying to do this even
>> after the code is frozen!)
>>
>> For 1.0, it seems to me at reasonable level and we should concentrate
>> on the stabilization.
>>
>> Regarding the report from Michael and related discussions, I'm
>> thinking to carry over the trigger to later version.    To announce
>> that 1.0 has sufficient HA feature, I'd like to keep pgxc_clean
>> effort, which needs a bit of extension to the core, although most of
>> the work is outside the core.
>>
>> Regards;
>> ----------
>> Koichi Suzuki
>>
>>
>>
>> 2012/3/6 Ahsan Hadi <ahs...@en...>:
>> > I think this makes a lot of sense, we should be gearing towards
>> code-freeze
>> > for 1.0 by end of this month. At this point most to of our efforts
>> should be
>> > focussed on regression stabilization and fixing important bugs from the
>> bug
>> > tracking system.
>> >
>> > Amit is already looking at regression failures, Abbas started analysis
>> of
>> > the open bugs yesterday, once he is done with his analysis, he will
>> close
>> > the bugs already fixed, divide the remaining ones between bugs and
>> > enhancements and move the obvious ones that we cannot target this month
>> to
>> > beyond 1.0. Amit and Abbas can focus on regression fixes and other bug
>> fixes
>> > for the remainder of this month.
>> >
>> > The planner improvements was an important piece, i assume what was
>> needed
>> > for 1.0 is already in place?
>> >
>> >
>> > On Tue, Mar 6, 2012 at 7:47 AM, Michael Paquier <
>> mic...@gm...>
>> > wrote:
>> >>
>> >> Hi all,
>> >>
>> >> As we are entering in the 1.0 release process, I propose that we do a
>> >> feature freeze at the end of the month.
>> >> This means that no new features will be added to core after the 31st of
>> >> March, but external utilities like pgxc_clean and initgtm (that I
>> should
>> >> finish before this date though), do not enter in this scope.
>> >> So, from this date, all the effort will be focused on stabilization and
>> >> bug fix.
>> >> On the 1st of April, I also propose to mark XC not as 1.0devel but as
>> >> 1.0beta1.
>> >> This could be delayed a couple of days of course if a feature we think
>> >> necessary is late.
>> >>
>> >> Any thoughts/comments?
>> >> --
>> >> Michael Paquier
>> >> http://michael.otacoo.com
>> >>
>> >>
>> >>
>> ------------------------------------------------------------------------------
>> >> Keep Your Developer Skills Current with LearnDevNow!
>> >> The most comprehensive online learning library for Microsoft developers
>> >> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3,
>> MVC3,
>> >> Metro Style Apps, more. Free future releases when you subscribe now!
>> >> http://p.sf.net/sfu/learndevnow-d2d
>> >> _______________________________________________
>> >> Postgres-xc-developers mailing list
>> >> Pos...@li...
>> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>> >>
>> >
>> >
>> >
>> > --
>> > Ahsan Hadi
>> > Snr Director Product Development
>> > EnterpriseDB Corporation
>> > The Enterprise Postgres Company
>> >
>> > Phone: +92-51-8358874
>> > Mobile: +92-333-5162114
>> >
>> > Website: www.enterprisedb.com
>> > EnterpriseDB Blog: http://blogs.enterprisedb.com/
>> > Follow us on Twitter: http://www.twitter.com/enterprisedb
>> >
>> > This e-mail message (and any attachment) is intended for the use of the
>> > individual or entity to whom it is addressed. This message contains
>> > information from EnterpriseDB Corporation that may be privileged,
>> > confidential, or exempt from disclosure under applicable law. If you
>> are not
>> > the intended recipient or authorized to receive this for the intended
>> > recipient, any use, dissemination, distribution, retention, archiving,
>> or
>> > copying of this communication is strictly prohibited. If you have
>> received
>> > this e-mail in error, please notify the sender immediately by reply
>> e-mail
>> > and delete this message.
>> >
>> >
>> ------------------------------------------------------------------------------
>> > Keep Your Developer Skills Current with LearnDevNow!
>> > The most comprehensive online learning library for Microsoft developers
>> > is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
>> > Metro Style Apps, more. Free future releases when you subscribe now!
>> > http://p.sf.net/sfu/learndevnow-d2d
>> > _______________________________________________
>> > Postgres-xc-developers mailing list
>> > Pos...@li...
>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>> >
>>
>
>
>
> --
> Michael Paquier
> http://michael.otacoo.com
>



-- 
Ahsan Hadi
Snr Director Product Development
EnterpriseDB Corporation
The Enterprise Postgres Company

Phone: +92-51-8358874
Mobile: +92-333-5162114

Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb

This e-mail message (and any attachment) is intended for the use of the
individual or entity to whom it is addressed. This message contains
information from EnterpriseDB Corporation that may be privileged,
confidential, or exempt from disclosure under applicable law. If you are
not the intended recipient or authorized to receive this for the intended
recipient, any use, dissemination, distribution, retention, archiving, or
copying of this communication is strictly prohibited. If you have received
this e-mail in error, please notify the sender immediately by reply e-mail
and delete this message.

Hi All,
PFA patch for cleanup of dead-code. There are many functions and structure
members, which are not used by the new FQS code. Removed them.

Regression does not show any new error.

-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

While analysing one last issue in plancache.sql, I found out that there are
multiple issues that need to be fixed. Overall, we need to improve the
handling of revalidated cached plans on datanodes.

The latest issue of plancache.sql failure is this:

A prepared statement should remember the search_path set at the time when
it was prepared, even after revalidation. That is, if the prepared plan is
executed after forcing a replan, the EXECUTE <plan> should still use the
old search path, even if it is changed at the time of execute.

set search_path to s1;
prepare p1 as select * from tab1; -- tab1 is in s1
execute p1;
set search_path to public;
alter table tab1 add column newcol; -- This forces replan.
execute p1 ; -- This should still select from s1.tab1 even though s1 is not
in search_path.


Following different issues were identified. All of these need to be fixed
for fixing the plancache revalidation issue, and eventually making the test
pass:

1. When a plan is prepared using PREPARE, the prepared statement name is
assigned to RemoteQuery->statement. When a cached plan is replanned during
RevalidateCachedPlan(), the newly allocated RemoteQuery structure is not
re-populated with stmt_name, so it remains NULL, and hence on the
datanodes, the subsequent EXECUTE statements keep on
re-preparing-and-executing again and again.

Fix:

The SetRemoteStatementName() was being called in StorePreparedStatement(),
but not during RevalidatePlanCache. Actually it should be called each time
a plan is created *and* re-validated. So, I have shifted this function call
from StorePreparedStatement() to StoreCachedPlan() because this is a common
function called by both CreateCachedPlan and RevalidateCachedPlan. Also, a
copy of PreparedStatement->stmt_name is now also stored in
PreparedStatement->plansource.


2. RevalidateCachedPlan() was dropping the data node statements. Once these
are dropped, the next EXECUTE will again re-prepare the statements on
datanodes from scratch. This loses the search_path that was saved in the
earlier plan. These datanode statements should be dropped only when the
user calls DEALLOCATE. i.e. when DropCachedPlan() is called.

Fix : So I have moved the release_datanode_statements() calls from
ReleaseCachedPlan() to DropCachedPlan().


3. SetRemoteStatementName() never uses an existing datanode statement name.
Each time, it is called, it generates a new stmt name that is to be used
for preparing on datanode. This way, we can never re-use an existing
prepared datanode statement after a replan.

Fix: When it finds an existing datanode statement hash entry, it re-uses it.

I have run regression with the attached patch, and did not find any new
diffs. plancache.sql passes now. Let me know of any comments, and any
possible issues/holes in general w.r.t PREPARE handling.

-Amit

Ok so I will commit it.

On 2012/03/08, at 14:04, Pavan Deolasee <pav...@en...> wrote:

> I think its OK to change the error message. We won't find information
> about the a GID if it does not correspond to a prepared transaction,
> same like vanilla PostgreSQL.
> 
> Thanks,
> Pavan
> 
> On Thu, Mar 8, 2012 at 8:49 AM, Koichi Suzuki <koi...@gm...> wrote:
>> Either will be okay.   I'm interested in whether
>> 
>> a) transaction does exist but could not found by internal error,
>> b) transaction does not actually exist.
>> 
>> Difference may not be interesting to users though.
>> ----------
>> Koichi Suzuki
>> 
>> 
>> 
>> 2012年3月8日12:14 Michael Paquier <mic...@gm...>:
>>> 
>>> 
>>> On Thu, Mar 8, 2012 at 11:51 AM, Koichi Suzuki <koi...@gm...> wrote:
>>>> 
>>>> The message looks more specific and better.   I'm not sure if the
>>>> error was caused because there're no prepared transaction actually or
>>>> other error occurred.
>>>> 
>>>> I took a look at this module and found execRemote.c can obtain more
>>>> specific error message from the datanode and it will be nice if
>>>> applications are fed back with it.
>>> 
>>> In this case the 2PC information cannot be recovered from GTM. We have 2
>>> possibilities:
>>> 1) keep the application unaware of the existence of GTM and update the error
>>> message.
>>> 2) make the application aware of GTM existence and keep this error message,
>>> but update regressions.
>>> 
>>> I am a fan of option 1, just thinking of the transparency XC should have
>>> with clients.
>>> 
>>> 
>>>> 
>>>> 
>>>> Thoughts?   Should it be an issue of the future work?
>>>> ----------
>>>> Koichi Suzuki
>>>> 
>>>> 
>>>> 
>>>> 2012年3月8日11:29 Michael Paquier <mic...@gm...>:
>>>>> Hi Pavan,
>>>>> 
>>>>> I found that an error message in execRemote.c is link of incorrect and
>>>>> returns diffs for the regression test prepared_xacts.
>>>>> The patch attached makes the error message more consistent with the rest
>>>>> of
>>>>> postgres.
>>>>> Do you agree if I commit that?
>>>>> 
>>>>> Regards,
>>>>> --
>>>>> Michael Paquier
>>>>> http://michael.otacoo.com
>>>>> 
>>>>> 
>>>>> ------------------------------------------------------------------------------
>>>>> Virtualization & Cloud Management Using Capacity Planning
>>>>> Cloud computing makes use of virtualization - but cloud computing
>>>>> also focuses on allowing computing to be delivered as a service.
>>>>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>>>>> _______________________________________________
>>>>> Postgres-xc-developers mailing list
>>>>> Pos...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>> 
>>> 
>>> 
>>> 
>>> 
>>> --
>>> Michael Paquier
>>> http://michael.otacoo.com
> 
> 
> 
> -- 
> Pavan Deolasee
> EnterpriseDB http://www.enterprisedb.com

I think its OK to change the error message. We won't find information
about the a GID if it does not correspond to a prepared transaction,
same like vanilla PostgreSQL.

Thanks,
Pavan

On Thu, Mar 8, 2012 at 8:49 AM, Koichi Suzuki <koi...@gm...> wrote:
> Either will be okay.   I'm interested in whether
>
> a) transaction does exist but could not found by internal error,
> b) transaction does not actually exist.
>
> Difference may not be interesting to users though.
> ----------
> Koichi Suzuki
>
>
>
> 2012年3月8日12:14 Michael Paquier <mic...@gm...>:
>>
>>
>> On Thu, Mar 8, 2012 at 11:51 AM, Koichi Suzuki <koi...@gm...> wrote:
>>>
>>> The message looks more specific and better.   I'm not sure if the
>>> error was caused because there're no prepared transaction actually or
>>> other error occurred.
>>>
>>> I took a look at this module and found execRemote.c can obtain more
>>> specific error message from the datanode and it will be nice if
>>> applications are fed back with it.
>>
>> In this case the 2PC information cannot be recovered from GTM. We have 2
>> possibilities:
>> 1) keep the application unaware of the existence of GTM and update the error
>> message.
>> 2) make the application aware of GTM existence and keep this error message,
>> but update regressions.
>>
>> I am a fan of option 1, just thinking of the transparency XC should have
>> with clients.
>>
>>
>>>
>>>
>>> Thoughts?   Should it be an issue of the future work?
>>> ----------
>>> Koichi Suzuki
>>>
>>>
>>>
>>> 2012年3月8日11:29 Michael Paquier <mic...@gm...>:
>>> > Hi Pavan,
>>> >
>>> > I found that an error message in execRemote.c is link of incorrect and
>>> > returns diffs for the regression test prepared_xacts.
>>> > The patch attached makes the error message more consistent with the rest
>>> > of
>>> > postgres.
>>> > Do you agree if I commit that?
>>> >
>>> > Regards,
>>> > --
>>> > Michael Paquier
>>> > http://michael.otacoo.com
>>> >
>>> >
>>> > ------------------------------------------------------------------------------
>>> > Virtualization & Cloud Management Using Capacity Planning
>>> > Cloud computing makes use of virtualization - but cloud computing
>>> > also focuses on allowing computing to be delivered as a service.
>>> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
>>> > _______________________________________________
>>> > Postgres-xc-developers mailing list
>>> > Pos...@li...
>>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>> >
>>
>>
>>
>>
>> --
>> Michael Paquier
>> http://michael.otacoo.com



-- 
Pavan Deolasee
EnterpriseDB http://www.enterprisedb.com

On Thu, Mar 8, 2012 at 6:31 AM, Koichi Suzuki <koi...@gm...> wrote:

> I think tables defined in pg_class but not in pgxc_class can be
> regarded as "local".
>

Yes, and those do not have rd_location_info NULL.


> ----------
> Koichi Suzuki
>
>
>
> 2012年3月8日9:19 Michael Paquier <mic...@gm...>:
> >
> >
> > On Wed, Mar 7, 2012 at 6:33 PM, Pavan Deolasee
> > <pav...@en...> wrote:
> >>
> >> On Wed, Mar 7, 2012 at 3:01 PM, Ashutosh Bapat
> >> <ash...@en...> wrote:
> >> > Pavan,
> >> > Instead of making this fix only for catalog tables, you can make it
> >> > generic
> >> > for all local tables. For now catalogs are the only tables local to a
> >> > coordinator, but that's something which may change sometime. Existence
> >> > of
> >> > Relation->rd_locator_info would be an indicator as to whether this
> table
> >> > is
> >> > local or not.
> >> >
> >>
> >> I think we should have a mechanism to identify local/remote tables
> >> whenever we have that situation. But I don't think we need to worry
> >> about that now.
> >
> > Indeed, we might need another solution than the workaround you sent in
> this
> > thread. But perhaps this is enough for the time being.
> > Let's discuss more about that.
> > --
> > Michael Paquier
> > http://michael.otacoo.com
> >
> >
> ------------------------------------------------------------------------------
> > Virtualization & Cloud Management Using Capacity Planning
> > Cloud computing makes use of virtualization - but cloud computing
> > also focuses on allowing computing to be delivered as a service.
> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
> > _______________________________________________
> > Postgres-xc-developers mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >
>
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>



-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

There were some issues with primary and preferred nodes.
Updated patch attached.

On Thu, Mar 8, 2012 at 10:08 AM, Michael Paquier
<mic...@gm...>wrote:

> In case you are interested, I am giving some priority to this patch.
> Here is an updated version realigned with head, with spelling mistakes
> corrected.
> We'll do here more advanced tests based on that.
>
> Regards
>
>
> On Tue, Mar 6, 2012 at 3:05 PM, Andrei Martsinchyk <
> and...@gm...> wrote:
>
>> OK, I will edit the patch and if it will need to be realigned I submit
>> correct version.
>>
>> 6 марта 2012 г. 8:58 пользователь Michael Paquier <
>> mic...@gm...> написал:
>>
>>
>>>
>>> On Tue, Mar 6, 2012 at 2:52 PM, Andrei Martsinchyk <
>>> and...@gm...> wrote:
>>>
>>>>
>>>>
>>>> 6 марта 2012 г. 7:42 пользователь Michael Paquier <
>>>> mic...@gm...> написал:
>>>>
>>>>> Hi,
>>>>>
>>>>> Thanks for the quick update. With your latest patch, I can see the
>>>>> following issues.
>>>>> 1) grammar mistakes at several places:
>>>>> - doc-xc/src/sgml/config.sgmlin, "maxinum" => "maximum"
>>>>> 2) In the docs, instead of referencing to "Maxinum number of datanodes
>>>>> configured in the cluster.", why not using "Maxinum number of datanodes
>>>>> usable in the cluster." The current formulation makes me think that used
>>>>> has to use 16 nodes, but it is not the case. Another formulation may be
>>>>> "Maxinum number of datanodes that can configured in the cluster."
>>>>>
>>>>
>>>> I like the last, but it should be spelled "Maxinum number of datanodes
>>>> that can *be* configured in the cluster."
>>>> May be someone who is a native English speaker can jump in for final
>>>> edition of documentation?
>>>>
>>> Yeah indeed sorry. I'm sure we know some guys, native english-speakers,
>>> who could do that ;)
>>>
>>>
>>>>
>>>>> 3) Same remark as before but this time in the newly-added docs,
>>>>> Coordinator and Datanode should be written with capital letters.
>>>>> Except that the code is clean. Regressions are running smoothly.
>>>>> So I'll move forward and do some performance test on benchmark.
>>>>>
>>>>>
>>>> Would you mind to correct typos etc. immediately as you notice them?
>>>> Taking into account time difference that would be waste of time to repeat
>>>> edit/review rounds because of insignificant changes. Thanks a lot for tests
>>>> and review.
>>>>
>>> I can try to have a look at that, but just I didn't want to change your
>>> code that would have influenced your development environment. However there
>>> will be soon 2 important commits by Pavan and Ashutosh, so perhaps you
>>> should realign your code first after those 2 commits, then I'll have a look
>>> at it.
>>> Thanks,
>>> --
>>> Michael Paquier
>>> http://michael.otacoo.com
>>>
>>
>>
>>
>> --
>> Best regards,
>> Andrei Martsinchyk                   mailto:and...@gm...
>>
>
>
>
> --
> Michael Paquier
> http://michael.otacoo.com
>



-- 
Michael Paquier
http://michael.otacoo.com

Either will be okay.   I'm interested in whether

a) transaction does exist but could not found by internal error,
b) transaction does not actually exist.

Difference may not be interesting to users though.
----------
Koichi Suzuki



2012年3月8日12:14 Michael Paquier <mic...@gm...>:
>
>
> On Thu, Mar 8, 2012 at 11:51 AM, Koichi Suzuki <koi...@gm...> wrote:
>>
>> The message looks more specific and better.   I'm not sure if the
>> error was caused because there're no prepared transaction actually or
>> other error occurred.
>>
>> I took a look at this module and found execRemote.c can obtain more
>> specific error message from the datanode and it will be nice if
>> applications are fed back with it.
>
> In this case the 2PC information cannot be recovered from GTM. We have 2
> possibilities:
> 1) keep the application unaware of the existence of GTM and update the error
> message.
> 2) make the application aware of GTM existence and keep this error message,
> but update regressions.
>
> I am a fan of option 1, just thinking of the transparency XC should have
> with clients.
>
>
>>
>>
>> Thoughts?   Should it be an issue of the future work?
>> ----------
>> Koichi Suzuki
>>
>>
>>
>> 2012年3月8日11:29 Michael Paquier <mic...@gm...>:
>> > Hi Pavan,
>> >
>> > I found that an error message in execRemote.c is link of incorrect and
>> > returns diffs for the regression test prepared_xacts.
>> > The patch attached makes the error message more consistent with the rest
>> > of
>> > postgres.
>> > Do you agree if I commit that?
>> >
>> > Regards,
>> > --
>> > Michael Paquier
>> > http://michael.otacoo.com
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Virtualization & Cloud Management Using Capacity Planning
>> > Cloud computing makes use of virtualization - but cloud computing
>> > also focuses on allowing computing to be delivered as a service.
>> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> > _______________________________________________
>> > Postgres-xc-developers mailing list
>> > Pos...@li...
>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>> >
>
>
>
>
> --
> Michael Paquier
> http://michael.otacoo.com

On Thu, Mar 8, 2012 at 11:51 AM, Koichi Suzuki <koi...@gm...> wrote:

> The message looks more specific and better.   I'm not sure if the
> error was caused because there're no prepared transaction actually or
> other error occurred.
>
> I took a look at this module and found execRemote.c can obtain more
> specific error message from the datanode and it will be nice if
> applications are fed back with it.
>
In this case the 2PC information cannot be recovered from GTM. We have 2
possibilities:
1) keep the application unaware of the existence of GTM and update the
error message.
2) make the application aware of GTM existence and keep this error message,
but update regressions.

I am a fan of option 1, just thinking of the transparency XC should have
with clients.



>
> Thoughts?   Should it be an issue of the future work?
> ----------
> Koichi Suzuki
>
>
>
> 2012年3月8日11:29 Michael Paquier <mic...@gm...>:
> > Hi Pavan,
> >
> > I found that an error message in execRemote.c is link of incorrect and
> > returns diffs for the regression test prepared_xacts.
> > The patch attached makes the error message more consistent with the rest
> of
> > postgres.
> > Do you agree if I commit that?
> >
> > Regards,
> > --
> > Michael Paquier
> > http://michael.otacoo.com
> >
> >
> ------------------------------------------------------------------------------
> > Virtualization & Cloud Management Using Capacity Planning
> > Cloud computing makes use of virtualization - but cloud computing
> > also focuses on allowing computing to be delivered as a service.
> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
> > _______________________________________________
> > Postgres-xc-developers mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >
>



-- 
Michael Paquier
http://michael.otacoo.com

The message looks more specific and better.   I'm not sure if the
error was caused because there're no prepared transaction actually or
other error occurred.

I took a look at this module and found execRemote.c can obtain more
specific error message from the datanode and it will be nice if
applications are fed back with it.

Thoughts?   Should it be an issue of the future work?
----------
Koichi Suzuki



2012年3月8日11:29 Michael Paquier <mic...@gm...>:
> Hi Pavan,
>
> I found that an error message in execRemote.c is link of incorrect and
> returns diffs for the regression test prepared_xacts.
> The patch attached makes the error message more consistent with the rest of
> postgres.
> Do you agree if I commit that?
>
> Regards,
> --
> Michael Paquier
> http://michael.otacoo.com
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

Hi Pavan,

I found that an error message in execRemote.c is link of incorrect and
returns diffs for the regression test prepared_xacts.
The patch attached makes the error message more consistent with the rest of
postgres.
Do you agree if I commit that?

Regards,
-- 
Michael Paquier
http://michael.otacoo.com

Hi Michael,

I have taken a day off today. But hopefully will be able to spend time
looking at the issues, especially the second.

Thanks,
Pavan

On Thu, Mar 8, 2012 at 6:26 AM, Michael Paquier
<mic...@gm...> wrote:
> Hi all,
>
> I found 2 new issues with regression tests.
> 1) test case privileges, there are some queries that make the regression
> test freeze.
> 2) alter_table, I am getting an segenv here. It looks that the combiner data
> is corrupted.
> (gdb) bt
> #0  0x000000000065150d in BufferConnection (conn=0x2ab3590) at
> execRemote.c:945
> #1  0x000000000065290a in pgxc_node_remote_prepare (prepareGID=0x2b31c40
> "T22152") at execRemote.c:1649
> #2  0x00000000006590bf in PrePrepare_Remote (prepareGID=0x2b31c40 "T22152",
> localNode=0 '\000', implicit=0 '\000') at execRemote.c:4587
> #3  0x0000000000658ef0 in PreCommit_Remote (prepareGID=0x2b31c40 "T22152",
> preparedLocalNode=0 '\000') at execRemote.c:4466
> #4  0x00000000004b8d70 in CommitTransaction () at xact.c:2036
> #5  0x00000000004b9b3b in CommitTransactionCommand () at xact.c:2906
> #6  0x0000000000756212 in finish_xact_command () at postgres.c:2572
> #7  0x0000000000753b6d in exec_simple_query (query_string=0x2a0bd18 "delete
> from parent;") at postgres.c:1136
> #8  0x0000000000758194 in PostgresMain (argc=2, argv=0x2a07ee8,
> username=0x2a07c30 "michael") at postgres.c:4155
> #9  0x00000000006fe4b6 in BackendRun (port=0x2a36e10) at postmaster.c:3763
> #10 0x00000000006fdb36 in BackendStartup (port=0x2a36e10) at
> postmaster.c:3448
> #11 0x00000000006faa82 in ServerLoop () at postmaster.c:1539
> #12 0x00000000006fa223 in PostmasterMain (argc=7, argv=0x2a04ba0) at
> postmaster.c:1200
> #13 0x0000000000664c39 in main (argc=7, argv=0x2a04ba0) at main.c:199
> (gdb) p combiner->ss.ss_ScanTupleSlot->tts_mcxt
> Cannot access memory at address 0x7f7f7f7f7f7f7fb7
> (gdb) p combiner->ss.ss_ScanTupleSlot
> $1 = (TupleTableSlot *) 0x7f7f7f7f7f7f7f7f
> (gdb) p combiner
> $2 = (RemoteQueryState *) 0x2b4b020
> (gdb) p *combiner
> $3 = {ss = {ps = {type = 2139062143, plan = 0x7f7f7f7f7f7f7f7f, state =
> 0x7f7f7f7f7f7f7f7f, instrument = 0x7f7f7f7f7f7f7f7f,
>       targetlist = 0x7f7f7f7f7f7f7f7f, qual = 0x7f7f7f7f7f7f7f7f, lefttree =
> 0x7f7f7f7f7f7f7f7f, righttree = 0x7f7f7f7f7f7f7f7f,
>       initPlan = 0x7f7f7f7f7f7f7f7f, subPlan = 0x7f7f7f7f7f7f7f7f, chgParam
> = 0x7f7f7f7f7f7f7f7f, ps_ResultTupleSlot = 0x7f7f7f7f7f7f7f7f,
>       ps_ExprContext = 0x7f7f7f7f7f7f7f7f, ps_ProjInfo = 0x7f7f7f7f7f7f7f7f,
> ps_TupFromTlist = 127 '\177'}, ss_currentRelation = 0x7f7f7f7f7f7f7f7f,
>     ss_currentScanDesc = 0x7f7f7f7f7f7f7f7f, ss_ScanTupleSlot =
> 0x7f7f7f7f7f7f7f7f}, node_count = 2139062143, connections =
> 0x7f7f7f7f7f7f7f7f,
>   conn_count = 2139062143, current_conn = 2139062143, combine_type =
> 2139062143, command_complete_count = 2139062143, request_type = 2139062143,
>   tuple_desc = 0x7f7f7f7f7f7f7f7f, description_count = 2139062143,
> copy_in_count = 2139062143, copy_out_count = 2139062143,
>   errorCode = "\177\177\177\177\177", errorMessage = 0x7f7f7f7f7f7f7f7f
> <Address 0x7f7f7f7f7f7f7f7f out of bounds>,
>   errorDetail = 0x7f7f7f7f7f7f7f7f <Address 0x7f7f7f7f7f7f7f7f out of
> bounds>, query_Done = 127 '\177', currentRow = {
>     msg = 0x7f7f7f7f7f7f7f7f <Address 0x7f7f7f7f7f7f7f7f out of bounds>,
> msglen = 2139062143, msgnode = 2139062143}, rowBuffer = 0x7f7f7f7f7f7f7f7f,
>   tapenodes = 0x7f7f7f7f7f7f7f7f, initAggregates = 127 '\177',
> tuplesortstate = 0x7f7f7f7f7f7f7f7f, eqfunctions = 0x7f7f7f7f7f7f7f7f,
>   tmp_ctx = 0x7f7f7f7f7f7f7f7f, copy_file = 0x7f7f7f7f7f7f7f7f, processed =
> 9187201950435737471,
>   cursor = 0x7f7f7f7f7f7f7f7f <Address 0x7f7f7f7f7f7f7f7f out of bounds>,
> update_cursor = 0x7f7f7f7f7f7f7f7f <Address 0x7f7f7f7f7f7f7f7f out of
> bounds>,
>   cursor_count = 2139062143, cursor_connections = 0x7f7f7f7f7f7f7f7f,
> paramval_data = 0x7f7f7f7f7f7f7f7f <Address 0x7f7f7f7f7f7f7f7f out of
> bounds>,
>   paramval_len = 2139062143, eflags = 2139062143, eof_underlying = 127
> '\177', tuplestorestate = 0x7f7f7f7f7f7f7f7f}
>
> I opened 2 bug tickets for each issue.
> - privileges: 3499130
> - alter_table: 3499133
>
> Regards,
> --
> Michael Paquier
> http://michael.otacoo.com
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>



-- 
Pavan Deolasee
EnterpriseDB http://www.enterprisedb.com

Hi all,

I just went through this bug, which was not there before error handling
reordering.
postgres=# CREATE TEMP TABLE tmp_foo (data text) ON COMMIT DELETE ROWS;
CREATE TABLE
postgres=# create table aa (a int);
ERROR:  cannot PREPARE a transaction that has operated on temporary tables

I believe the is_temp flag of execRemote.c is not correctly reinitialized
on the 1st DDL is done.
It works correctly if a normal temp table is created.

I created a new bug on that => 3499167.
Thx.
-- 
Michael Paquier
http://michael.otacoo.com

In case you are interested, I am giving some priority to this patch.
Here is an updated version realigned with head, with spelling mistakes
corrected.
We'll do here more advanced tests based on that.

Regards

On Tue, Mar 6, 2012 at 3:05 PM, Andrei Martsinchyk <
and...@gm...> wrote:

> OK, I will edit the patch and if it will need to be realigned I submit
> correct version.
>
> 6 марта 2012 г. 8:58 пользователь Michael Paquier <
> mic...@gm...> написал:
>
>
>>
>> On Tue, Mar 6, 2012 at 2:52 PM, Andrei Martsinchyk <
>> and...@gm...> wrote:
>>
>>>
>>>
>>> 6 марта 2012 г. 7:42 пользователь Michael Paquier <
>>> mic...@gm...> написал:
>>>
>>>> Hi,
>>>>
>>>> Thanks for the quick update. With your latest patch, I can see the
>>>> following issues.
>>>> 1) grammar mistakes at several places:
>>>> - doc-xc/src/sgml/config.sgmlin, "maxinum" => "maximum"
>>>> 2) In the docs, instead of referencing to "Maxinum number of datanodes
>>>> configured in the cluster.", why not using "Maxinum number of datanodes
>>>> usable in the cluster." The current formulation makes me think that used
>>>> has to use 16 nodes, but it is not the case. Another formulation may be
>>>> "Maxinum number of datanodes that can configured in the cluster."
>>>>
>>>
>>> I like the last, but it should be spelled "Maxinum number of datanodes
>>> that can *be* configured in the cluster."
>>> May be someone who is a native English speaker can jump in for final
>>> edition of documentation?
>>>
>> Yeah indeed sorry. I'm sure we know some guys, native english-speakers,
>> who could do that ;)
>>
>>
>>>
>>>> 3) Same remark as before but this time in the newly-added docs,
>>>> Coordinator and Datanode should be written with capital letters.
>>>> Except that the code is clean. Regressions are running smoothly.
>>>> So I'll move forward and do some performance test on benchmark.
>>>>
>>>>
>>> Would you mind to correct typos etc. immediately as you notice them?
>>> Taking into account time difference that would be waste of time to repeat
>>> edit/review rounds because of insignificant changes. Thanks a lot for tests
>>> and review.
>>>
>> I can try to have a look at that, but just I didn't want to change your
>> code that would have influenced your development environment. However there
>> will be soon 2 important commits by Pavan and Ashutosh, so perhaps you
>> should realign your code first after those 2 commits, then I'll have a look
>> at it.
>> Thanks,
>> --
>> Michael Paquier
>> http://michael.otacoo.com
>>
>
>
>
> --
> Best regards,
> Andrei Martsinchyk                   mailto:and...@gm...
>



-- 
Michael Paquier
http://michael.otacoo.com

Yes, that's true.   ALTER TABLE D.  should have an option to run at
background and it should not allow to be combined with any other
modification of the table.    It may take long to implement.     For
the people who don't like long time lock may use your technique.
Anyway, redistributing 100M rows will take very long and SELECT INTO
or CREATE TABLE AS will run in single transaction.   Which causes
other problems anyway and ALTER TABLE D. background should allow other
transactions, at least DMLs,  to update the table.

Regards;
----------
Koichi Suzuki



2012年3月8日9:20 Chaos Eternal <cha...@sh...>:
> well,
> I agree that "ALTER TABLE DISTRIBUTION " is cool, but beware that ALTER
> TABLE D. mean a lot of locks on metadata, and if we redistribute a large
> table, in my environment, a table usually contains more than 100M rows, and
> this redistribution process will really take a long time to complete,
> holding a lock so long time is not a good idea.
> on the other way, what ever mechanism you choose to re-distribute a table,
> moving data across the cluster is inevitable, let me detail the procedure:
> if we redistribute a table from n datanode to m datanode, and whatever the
> distribute key is, we need to move (1-1/(n*m)) of the original data out of
> their original node, (the worst case, n and m has no common divider other
> than 1) , and if we keep the table, we will introduce a lot of fractions, so
> it is obvious that keep the original table is not good.
>
> ****
> So if we choose another mechanism , like i suggest, CAS or, IS, the benefits
> would be:
> 1. no long term lock holding
> 2. new tables are created clean
> 3. during the redistribute process, the original table is functional until
> drop/ rename, and drop/rename would be quite short.
>
> and for the views, stored plan, triggers etc, a simple recreate process
> could handle.
>
> BTW: I didn't find evidence that views will hold a tables oid, views are
> only plain texts of their definition, anybody give me more information about
> this?
>
>
> On Wed, Mar 7, 2012 at 5:03 PM, Andrei Martsinchyk
> <and...@gm...> wrote:
>>
>> Redustribution would take time, especially redistribution of a huge table.
>> It is better to restore teble with proper distribution already.
>> That is correct approach to run INSERT SELECT under the cover when
>> redistributing table.
>> It would be user-friendly to provide command like ALTER TABLE DISTRIBUTION
>> that would do all at once, together with all the housekeeping, like proper
>> locking and indexing.
>>
>>
>> 7 марта 2012 г. 11:42 пользователь Koichi Suzuki <koi...@gm...>
>> написал:
>>
>>> I see.
>>>
>>> In table redistribution, it is we have to review what other resource
>>> should be maintained too.   Views, indexes, rules (although XC does
>>> not fully support the rule yet), triggers (when XC supports this),
>>> constraints, and so on.   This is not a simple work but must be cool!
>>> ----------
>>> Koichi Suzuki
>>>
>>>
>>>
>>> 2012/3/7 Ashutosh Bapat <ash...@en...>:
>>> >
>>> >
>>> > On Wed, Mar 7, 2012 at 1:58 PM, Chaos Eternal <cha...@sh...>
>>> > wrote:
>>> >>
>>> >> in most case, views only store tablenames,
>>> >
>>> >
>>> > That's not true, I guess. The views store the parse tree, which
>>> > contains
>>> > relation OIDs and not name. See the test case below
>>> > testdb=# create table tab1 (val int, val2 int);
>>> > selCREATE TABLE
>>> > testdb=# select * from tab1;
>>> >  val | val2
>>> > -----+------
>>> > (0 rows)
>>> >
>>> > testdb=# create view v_tab1 as select * from tab1;
>>> > CREATE VIEW
>>> > testdb=# select * from v_tab1;
>>> >  val | val2
>>> > -----+------
>>> > (0 rows)
>>> >
>>> > testdb=# alter table tab1 rename to tab2;
>>> > ALTER TABLE
>>> > testdb=# select * from v_tab1;
>>> >  val | val2
>>> > -----+------
>>> > (0 rows)
>>> >
>>> > If the view would have retained the tablename it would not have worked
>>> > after
>>> > table tab1 is renamed to tab2. It works because we stick to OID of the
>>> > relation and not the name itself and ALTER TABLE does not change the
>>> > OID.
>>> >
>>> >>
>>> >> while stored plans could be a problem if we use a lot of them.
>>> >>
>>> >> my scenario of using pg-xc is to process a large amount of data, like
>>> >> OLAP, and stored plans are rarely used.
>>> >>
>>> >>
>>> >> 2012/3/7 Ashutosh Bapat <ash...@en...>
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Wed, Mar 7, 2012 at 11:19 AM, Koichi Suzuki <koi...@gm...>
>>> >>> wrote:
>>> >>>>
>>> >>>> Yes, I understand.   Before committing this, I'd like to have a
>>> >>>> review
>>> >>>> of other members.
>>> >>>>
>>> >>>> BTW, a Chinese guy (I don't know his real name) wrote to me that we
>>> >>>> can redistribute rows by using CREATE TABLE AS or SELECT INTO and
>>> >>>> then
>>> >>>> dop the original one and rename the new one.   This can be very
>>> >>>> flexible.   If DBAs are not satisfied with the current node
>>> >>>> configuration, they can dump everything with pg_dump, reconfigure
>>> >>>> the
>>> >>>> nodes, pg_restore and then run CTA or SI.
>>> >>>>
>>> >>>
>>> >>> ALTER TABLE REDISTRIBUTION was an item on my plate for this quarter,
>>> >>> but
>>> >>> FQS took most of the time. Do you think, we should take that up? I am
>>> >>> not
>>> >>> sure, whether that can be completed by end of March, esp. there is
>>> >>> work
>>> >>> involved to send the rows back and forth between datanodes and
>>> >>> coordinator.
>>> >>>
>>> >>> The approach suggested above will change the OID of the relation,
>>> >>> which
>>> >>> invalidates any view definitions, stored plans on the relation. If
>>> >>> that's
>>> >>> not a concern the approach looks good.
>>> >>>
>>> >>>>
>>> >>>> Regards;
>>> >>>> ----------
>>> >>>> Koichi Suzuki
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> 2012/3/7 Andrei Martsinchyk <and...@gm...>:
>>> >>>> > Agree.
>>> >>>> > But now it does not output distribution type and distribution
>>> >>>> > column.
>>> >>>> > The patch fixes this.
>>> >>>> >
>>> >>>> > 7 марта 2012 г. 3:38 пользователь Koichi Suzuki
>>> >>>> > <ko...@in...>
>>> >>>> > написал:
>>> >>>> >
>>> >>>> >> It's nice that pg_dump doesn't dump node group because it is used
>>> >>>> >> to
>>> >>>> >> rebalance tables for different node configuration.     We may
>>> >>>> >> need
>>> >>>> >> some
>>> >>>> >> options on node-group in the future.
>>> >>>> >>
>>> >>>> >> Regards;
>>> >>>> >> ---
>>> >>>> >> Koichi
>>> >>>> >>
>>> >>>> >>  Tue, 6 Mar 2012 21:03:29 +0300
>>> >>>> >> Andrei Martsinchyk <and...@gm...> wrote:
>>> >>>> >>
>>> >>>> >> > Hi,
>>> >>>> >> >
>>> >>>> >> > I have noticed that pg_dump does not output table distribution
>>> >>>> >> > info
>>> >>>> >> > as
>>> >>>> >> > it
>>> >>>> >> > used to do.
>>> >>>> >> > It looks like there was an incorrect merge. Please find fix
>>> >>>> >> > attached.
>>> >>>> >> >
>>> >>>> >> > --
>>> >>>> >> > Best regards,
>>> >>>> >> > Andrei Martsinchyk
>>> >>>> >> > mailto:and...@gm...
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> > --
>>> >>>> > Best regards,
>>> >>>> > Andrei Martsinchyk
>>> >>>> > mailto:and...@gm...
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> > ------------------------------------------------------------------------------
>>> >>>> > Virtualization & Cloud Management Using Capacity Planning
>>> >>>> > Cloud computing makes use of virtualization - but cloud computing
>>> >>>> > also focuses on allowing computing to be delivered as a service.
>>> >>>> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
>>> >>>> > _______________________________________________
>>> >>>> > Postgres-xc-developers mailing list
>>> >>>> > Pos...@li...
>>> >>>> >
>>> >>>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>> >>>> >
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> ------------------------------------------------------------------------------
>>> >>>> Virtualization & Cloud Management Using Capacity Planning
>>> >>>> Cloud computing makes use of virtualization - but cloud computing
>>> >>>> also focuses on allowing computing to be delivered as a service.
>>> >>>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>>> >>>> _______________________________________________
>>> >>>> Postgres-xc-developers mailing list
>>> >>>> Pos...@li...
>>> >>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Best Wishes,
>>> >>> Ashutosh Bapat
>>> >>> EntepriseDB Corporation
>>> >>> The Enterprise Postgres Company
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> ------------------------------------------------------------------------------
>>> >>> Virtualization & Cloud Management Using Capacity Planning
>>> >>> Cloud computing makes use of virtualization - but cloud computing
>>> >>> also focuses on allowing computing to be delivered as a service.
>>> >>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>>> >>> _______________________________________________
>>> >>> Postgres-xc-developers mailing list
>>> >>> Pos...@li...
>>> >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>> >>>
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > Best Wishes,
>>> > Ashutosh Bapat
>>> > EntepriseDB Corporation
>>> > The Enterprise Postgres Company
>>> >
>>
>>
>>
>>
>> --
>> Best regards,
>> Andrei Martsinchyk                   mailto:and...@gm...
>
>

On Thu, Mar 8, 2012 at 10:01 AM, Koichi Suzuki <koi...@gm...> wrote:

> I think tables defined in pg_class but not in pgxc_class can be
> regarded as "local".
>
Yeah, true. Tables classified as not having any distribution data.


> 2012年3月8日9:19 Michael Paquier <mic...@gm...>:
> >
> >
> > On Wed, Mar 7, 2012 at 6:33 PM, Pavan Deolasee
> > <pav...@en...> wrote:
> >>
> >> On Wed, Mar 7, 2012 at 3:01 PM, Ashutosh Bapat
> >> <ash...@en...> wrote:
> >> > Pavan,
> >> > Instead of making this fix only for catalog tables, you can make it
> >> > generic
> >> > for all local tables. For now catalogs are the only tables local to a
> >> > coordinator, but that's something which may change sometime. Existence
> >> > of
> >> > Relation->rd_locator_info would be an indicator as to whether this
> table
> >> > is
> >> > local or not.
> >> >
> >>
> >> I think we should have a mechanism to identify local/remote tables
> >> whenever we have that situation. But I don't think we need to worry
> >> about that now.
> >
> > Indeed, we might need another solution than the workaround you sent in
> this
> > thread. But perhaps this is enough for the time being.
> > Let's discuss more about that.
> > --
> > Michael Paquier
> > http://michael.otacoo.com
> >
> >
> ------------------------------------------------------------------------------
> > Virtualization & Cloud Management Using Capacity Planning
> > Cloud computing makes use of virtualization - but cloud computing
> > also focuses on allowing computing to be delivered as a service.
> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
> > _______________________________________________
> > Postgres-xc-developers mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >
>



-- 
Michael Paquier
http://michael.otacoo.com

I think tables defined in pg_class but not in pgxc_class can be
regarded as "local".
----------
Koichi Suzuki



2012年3月8日9:19 Michael Paquier <mic...@gm...>:
>
>
> On Wed, Mar 7, 2012 at 6:33 PM, Pavan Deolasee
> <pav...@en...> wrote:
>>
>> On Wed, Mar 7, 2012 at 3:01 PM, Ashutosh Bapat
>> <ash...@en...> wrote:
>> > Pavan,
>> > Instead of making this fix only for catalog tables, you can make it
>> > generic
>> > for all local tables. For now catalogs are the only tables local to a
>> > coordinator, but that's something which may change sometime. Existence
>> > of
>> > Relation->rd_locator_info would be an indicator as to whether this table
>> > is
>> > local or not.
>> >
>>
>> I think we should have a mechanism to identify local/remote tables
>> whenever we have that situation. But I don't think we need to worry
>> about that now.
>
> Indeed, we might need another solution than the workaround you sent in this
> thread. But perhaps this is enough for the time being.
> Let's discuss more about that.
> --
> Michael Paquier
> http://michael.otacoo.com
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

Hi all,

I found 2 new issues with regression tests.
1) test case privileges, there are some queries that make the regression
test freeze.
2) alter_table, I am getting an segenv here. It looks that the combiner
data is corrupted.
(gdb) bt
#0  0x000000000065150d in BufferConnection (conn=0x2ab3590) at
execRemote.c:945
#1  0x000000000065290a in pgxc_node_remote_prepare (prepareGID=0x2b31c40
"T22152") at execRemote.c:1649
#2  0x00000000006590bf in PrePrepare_Remote (prepareGID=0x2b31c40 "T22152",
localNode=0 '\000', implicit=0 '\000') at execRemote.c:4587
#3  0x0000000000658ef0 in PreCommit_Remote (prepareGID=0x2b31c40 "T22152",
preparedLocalNode=0 '\000') at execRemote.c:4466
#4  0x00000000004b8d70 in CommitTransaction () at xact.c:2036
#5  0x00000000004b9b3b in CommitTransactionCommand () at xact.c:2906
#6  0x0000000000756212 in finish_xact_command () at postgres.c:2572
#7  0x0000000000753b6d in exec_simple_query (query_string=0x2a0bd18 "delete
from parent;") at postgres.c:1136
#8  0x0000000000758194 in PostgresMain (argc=2, argv=0x2a07ee8,
username=0x2a07c30 "michael") at postgres.c:4155
#9  0x00000000006fe4b6 in BackendRun (port=0x2a36e10) at postmaster.c:3763
#10 0x00000000006fdb36 in BackendStartup (port=0x2a36e10) at
postmaster.c:3448
#11 0x00000000006faa82 in ServerLoop () at postmaster.c:1539
#12 0x00000000006fa223 in PostmasterMain (argc=7, argv=0x2a04ba0) at
postmaster.c:1200
#13 0x0000000000664c39 in main (argc=7, argv=0x2a04ba0) at main.c:199
(gdb) p combiner->ss.ss_ScanTupleSlot->tts_mcxt
Cannot access memory at address 0x7f7f7f7f7f7f7fb7
(gdb) p combiner->ss.ss_ScanTupleSlot
$1 = (TupleTableSlot *) 0x7f7f7f7f7f7f7f7f
(gdb) p combiner
$2 = (RemoteQueryState *) 0x2b4b020
(gdb) p *combiner
$3 = {ss = {ps = {type = 2139062143, plan = 0x7f7f7f7f7f7f7f7f, state =
0x7f7f7f7f7f7f7f7f, instrument = 0x7f7f7f7f7f7f7f7f,
      targetlist = 0x7f7f7f7f7f7f7f7f, qual = 0x7f7f7f7f7f7f7f7f, lefttree
= 0x7f7f7f7f7f7f7f7f, righttree = 0x7f7f7f7f7f7f7f7f,
      initPlan = 0x7f7f7f7f7f7f7f7f, subPlan = 0x7f7f7f7f7f7f7f7f, chgParam
= 0x7f7f7f7f7f7f7f7f, ps_ResultTupleSlot = 0x7f7f7f7f7f7f7f7f,
      ps_ExprContext = 0x7f7f7f7f7f7f7f7f, ps_ProjInfo =
0x7f7f7f7f7f7f7f7f, ps_TupFromTlist = 127 '\177'}, ss_currentRelation =
0x7f7f7f7f7f7f7f7f,
    ss_currentScanDesc = 0x7f7f7f7f7f7f7f7f, ss_ScanTupleSlot =
0x7f7f7f7f7f7f7f7f}, node_count = 2139062143, connections =
0x7f7f7f7f7f7f7f7f,
  conn_count = 2139062143, current_conn = 2139062143, combine_type =
2139062143, command_complete_count = 2139062143, request_type = 2139062143,
  tuple_desc = 0x7f7f7f7f7f7f7f7f, description_count = 2139062143,
copy_in_count = 2139062143, copy_out_count = 2139062143,
  errorCode = "\177\177\177\177\177", errorMessage = 0x7f7f7f7f7f7f7f7f
<Address 0x7f7f7f7f7f7f7f7f out of bounds>,
  errorDetail = 0x7f7f7f7f7f7f7f7f <Address 0x7f7f7f7f7f7f7f7f out of
bounds>, query_Done = 127 '\177', currentRow = {
    msg = 0x7f7f7f7f7f7f7f7f <Address 0x7f7f7f7f7f7f7f7f out of bounds>,
msglen = 2139062143, msgnode = 2139062143}, rowBuffer = 0x7f7f7f7f7f7f7f7f,
  tapenodes = 0x7f7f7f7f7f7f7f7f, initAggregates = 127 '\177',
tuplesortstate = 0x7f7f7f7f7f7f7f7f, eqfunctions = 0x7f7f7f7f7f7f7f7f,
  tmp_ctx = 0x7f7f7f7f7f7f7f7f, copy_file = 0x7f7f7f7f7f7f7f7f, processed =
9187201950435737471,
  cursor = 0x7f7f7f7f7f7f7f7f <Address 0x7f7f7f7f7f7f7f7f out of bounds>,
update_cursor = 0x7f7f7f7f7f7f7f7f <Address 0x7f7f7f7f7f7f7f7f out of
bounds>,
  cursor_count = 2139062143, cursor_connections = 0x7f7f7f7f7f7f7f7f,
paramval_data = 0x7f7f7f7f7f7f7f7f <Address 0x7f7f7f7f7f7f7f7f out of
bounds>,
  paramval_len = 2139062143, eflags = 2139062143, eof_underlying = 127
'\177', tuplestorestate = 0x7f7f7f7f7f7f7f7f}

I opened 2 bug tickets for each issue.
- privileges: 3499130
- alter_table: 3499133

Regards,
-- 
Michael Paquier
http://michael.otacoo.com

well,
I agree that "ALTER TABLE DISTRIBUTION " is cool, but beware that ALTER
TABLE D. mean a lot of locks on metadata, and if we redistribute a large
table, in my environment, a table usually contains more than 100M rows, and
this redistribution process will really take a long time to complete,
holding a lock so long time is not a good idea.
on the other way, what ever mechanism you choose to re-distribute a table,
moving data across the cluster is inevitable, let me detail the procedure:
if we redistribute a table from n datanode to m datanode, and whatever the
distribute key is, we need to move (1-1/(n*m)) of the original data out of
their original node, (the worst case, n and m has no common divider other
than 1) , and if we keep the table, we will introduce a lot of
fractions, so it is obvious that keep the original table is not good.

****
So if we choose another mechanism , like i suggest, CAS or, IS,
the benefits would be:
1. no long term lock holding
2. new tables are created clean
3. during the redistribute process, the original table is functional until
drop/ rename, and drop/rename would be quite short.

and for the views, stored plan, triggers etc, a simple recreate process
could handle.

BTW: I didn't find evidence that views will hold a tables oid, views are
only plain texts of their definition, anybody give me more information
about this?


On Wed, Mar 7, 2012 at 5:03 PM, Andrei Martsinchyk <
and...@gm...> wrote:

> Redustribution would take time, especially redistribution of a huge table.
> It is better to restore teble with proper distribution already.
> That is correct approach to run INSERT SELECT under the cover when
> redistributing table.
> It would be user-friendly to provide command like ALTER TABLE DISTRIBUTION
> that would do all at once, together with all the housekeeping, like proper
> locking and indexing.
>
>
> 7 марта 2012 г. 11:42 пользователь Koichi Suzuki <koi...@gm...>написал:
>
> I see.
>>
>> In table redistribution, it is we have to review what other resource
>> should be maintained too.   Views, indexes, rules (although XC does
>> not fully support the rule yet), triggers (when XC supports this),
>> constraints, and so on.   This is not a simple work but must be cool!
>> ----------
>> Koichi Suzuki
>>
>>
>>
>> 2012/3/7 Ashutosh Bapat <ash...@en...>:
>> >
>> >
>> > On Wed, Mar 7, 2012 at 1:58 PM, Chaos Eternal <cha...@sh...>
>> > wrote:
>> >>
>> >> in most case, views only store tablenames,
>> >
>> >
>> > That's not true, I guess. The views store the parse tree, which contains
>> > relation OIDs and not name. See the test case below
>> > testdb=# create table tab1 (val int, val2 int);
>> > selCREATE TABLE
>> > testdb=# select * from tab1;
>> >  val | val2
>> > -----+------
>> > (0 rows)
>> >
>> > testdb=# create view v_tab1 as select * from tab1;
>> > CREATE VIEW
>> > testdb=# select * from v_tab1;
>> >  val | val2
>> > -----+------
>> > (0 rows)
>> >
>> > testdb=# alter table tab1 rename to tab2;
>> > ALTER TABLE
>> > testdb=# select * from v_tab1;
>> >  val | val2
>> > -----+------
>> > (0 rows)
>> >
>> > If the view would have retained the tablename it would not have worked
>> after
>> > table tab1 is renamed to tab2. It works because we stick to OID of the
>> > relation and not the name itself and ALTER TABLE does not change the
>> OID.
>> >
>> >>
>> >> while stored plans could be a problem if we use a lot of them.
>> >>
>> >> my scenario of using pg-xc is to process a large amount of data, like
>> >> OLAP, and stored plans are rarely used.
>> >>
>> >>
>> >> 2012/3/7 Ashutosh Bapat <ash...@en...>
>> >>>
>> >>>
>> >>>
>> >>> On Wed, Mar 7, 2012 at 11:19 AM, Koichi Suzuki <koi...@gm...>
>> >>> wrote:
>> >>>>
>> >>>> Yes, I understand.   Before committing this, I'd like to have a
>> review
>> >>>> of other members.
>> >>>>
>> >>>> BTW, a Chinese guy (I don't know his real name) wrote to me that we
>> >>>> can redistribute rows by using CREATE TABLE AS or SELECT INTO and
>> then
>> >>>> dop the original one and rename the new one.   This can be very
>> >>>> flexible.   If DBAs are not satisfied with the current node
>> >>>> configuration, they can dump everything with pg_dump, reconfigure the
>> >>>> nodes, pg_restore and then run CTA or SI.
>> >>>>
>> >>>
>> >>> ALTER TABLE REDISTRIBUTION was an item on my plate for this quarter,
>> but
>> >>> FQS took most of the time. Do you think, we should take that up? I am
>> not
>> >>> sure, whether that can be completed by end of March, esp. there is
>> work
>> >>> involved to send the rows back and forth between datanodes and
>> coordinator.
>> >>>
>> >>> The approach suggested above will change the OID of the relation,
>> which
>> >>> invalidates any view definitions, stored plans on the relation. If
>> that's
>> >>> not a concern the approach looks good.
>> >>>
>> >>>>
>> >>>> Regards;
>> >>>> ----------
>> >>>> Koichi Suzuki
>> >>>>
>> >>>>
>> >>>>
>> >>>> 2012/3/7 Andrei Martsinchyk <and...@gm...>:
>> >>>> > Agree.
>> >>>> > But now it does not output distribution type and distribution
>> column.
>> >>>> > The patch fixes this.
>> >>>> >
>> >>>> > 7 марта 2012 г. 3:38 пользователь Koichi Suzuki
>> >>>> > <ko...@in...>
>> >>>> > написал:
>> >>>> >
>> >>>> >> It's nice that pg_dump doesn't dump node group because it is used
>> to
>> >>>> >> rebalance tables for different node configuration.     We may need
>> >>>> >> some
>> >>>> >> options on node-group in the future.
>> >>>> >>
>> >>>> >> Regards;
>> >>>> >> ---
>> >>>> >> Koichi
>> >>>> >>
>> >>>> >>  Tue, 6 Mar 2012 21:03:29 +0300
>> >>>> >> Andrei Martsinchyk <and...@gm...> wrote:
>> >>>> >>
>> >>>> >> > Hi,
>> >>>> >> >
>> >>>> >> > I have noticed that pg_dump does not output table distribution
>> info
>> >>>> >> > as
>> >>>> >> > it
>> >>>> >> > used to do.
>> >>>> >> > It looks like there was an incorrect merge. Please find fix
>> >>>> >> > attached.
>> >>>> >> >
>> >>>> >> > --
>> >>>> >> > Best regards,
>> >>>> >> > Andrei Martsinchyk
>> >>>> >> > mailto:and...@gm...
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > --
>> >>>> > Best regards,
>> >>>> > Andrei Martsinchyk
>> >>>> > mailto:and...@gm...
>> >>>> >
>> >>>> >
>> >>>> >
>> ------------------------------------------------------------------------------
>> >>>> > Virtualization & Cloud Management Using Capacity Planning
>> >>>> > Cloud computing makes use of virtualization - but cloud computing
>> >>>> > also focuses on allowing computing to be delivered as a service.
>> >>>> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> >>>> > _______________________________________________
>> >>>> > Postgres-xc-developers mailing list
>> >>>> > Pos...@li...
>> >>>> >
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>> >>>> >
>> >>>>
>> >>>>
>> >>>>
>> ------------------------------------------------------------------------------
>> >>>> Virtualization & Cloud Management Using Capacity Planning
>> >>>> Cloud computing makes use of virtualization - but cloud computing
>> >>>> also focuses on allowing computing to be delivered as a service.
>> >>>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> >>>> _______________________________________________
>> >>>> Postgres-xc-developers mailing list
>> >>>> Pos...@li...
>> >>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Best Wishes,
>> >>> Ashutosh Bapat
>> >>> EntepriseDB Corporation
>> >>> The Enterprise Postgres Company
>> >>>
>> >>>
>> >>>
>> >>>
>> ------------------------------------------------------------------------------
>> >>> Virtualization & Cloud Management Using Capacity Planning
>> >>> Cloud computing makes use of virtualization - but cloud computing
>> >>> also focuses on allowing computing to be delivered as a service.
>> >>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> >>> _______________________________________________
>> >>> Postgres-xc-developers mailing list
>> >>> Pos...@li...
>> >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>> >>>
>> >>
>> >
>> >
>> >
>> > --
>> > Best Wishes,
>> > Ashutosh Bapat
>> > EntepriseDB Corporation
>> > The Enterprise Postgres Company
>> >
>>
>
>
>
> --
> Best regards,
> Andrei Martsinchyk                   mailto:and...@gm...
>

On Wed, Mar 7, 2012 at 6:33 PM, Pavan Deolasee <
pav...@en...> wrote:

> On Wed, Mar 7, 2012 at 3:01 PM, Ashutosh Bapat
> <ash...@en...> wrote:
> > Pavan,
> > Instead of making this fix only for catalog tables, you can make it
> generic
> > for all local tables. For now catalogs are the only tables local to a
> > coordinator, but that's something which may change sometime. Existence of
> > Relation->rd_locator_info would be an indicator as to whether this table
> is
> > local or not.
> >
>
> I think we should have a mechanism to identify local/remote tables
> whenever we have that situation. But I don't think we need to worry
> about that now.
>
Indeed, we might need another solution than the workaround you sent in this
thread. But perhaps this is enough for the time being.
Let's discuss more about that.
-- 
Michael Paquier
http://michael.otacoo.com

Patch is committed.
Thanks.

On Wed, Mar 7, 2012 at 5:40 PM, Koichi Suzuki <koi...@gm...> wrote:

> Not index redistribution.   It is not practical because CTID will
> change after redistribution.   Rather, index may have to be rebuilt at
> ALTER TABLE redistribution.
> ----------
> Koichi Suzuki
>
>
>
> 2012/3/7 Chaos Eternal <cha...@sh...>:
> > redistribution of indexes?
> > that would be REALLY cool.
> >
> >
> > On Wed, Mar 7, 2012 at 4:32 PM, Koichi Suzuki <koi...@gm...>
> wrote:
> >>
> >> Thanks a lot for the advice.   Also, we should reconstruct indexes
> >> too.   There could be other resources we should consider to support
> >> ALTER TABLE redistribution.
> >>
> >> Best regards;
> >> ----------
> >> Koichi Suzuki
> >>
> >>
> >>
> >> 2012/3/7 Chaos Eternal <cha...@sh...>:
> >> > in most case, views only store tablenames, while stored plans could
> be a
> >> > problem if we use a lot of them.
> >> >
> >> > my scenario of using pg-xc is to process a large amount of data, like
> >> > OLAP,
> >> > and stored plans are rarely used.
> >> >
> >> >
> >> > 2012/3/7 Ashutosh Bapat <ash...@en...>
> >> >>
> >> >>
> >> >>
> >> >> On Wed, Mar 7, 2012 at 11:19 AM, Koichi Suzuki <koi...@gm...
> >
> >> >> wrote:
> >> >>>
> >> >>> Yes, I understand.   Before committing this, I'd like to have a
> review
> >> >>> of other members.
> >> >>>
> >> >>> BTW, a Chinese guy (I don't know his real name) wrote to me that we
> >> >>> can redistribute rows by using CREATE TABLE AS or SELECT INTO and
> then
> >> >>> dop the original one and rename the new one.   This can be very
> >> >>> flexible.   If DBAs are not satisfied with the current node
> >> >>> configuration, they can dump everything with pg_dump, reconfigure
> the
> >> >>> nodes, pg_restore and then run CTA or SI.
> >> >>>
> >> >>
> >> >> ALTER TABLE REDISTRIBUTION was an item on my plate for this quarter,
> >> >> but
> >> >> FQS took most of the time. Do you think, we should take that up? I am
> >> >> not
> >> >> sure, whether that can be completed by end of March, esp. there is
> work
> >> >> involved to send the rows back and forth between datanodes and
> >> >> coordinator.
> >> >>
> >> >> The approach suggested above will change the OID of the relation,
> which
> >> >> invalidates any view definitions, stored plans on the relation. If
> >> >> that's
> >> >> not a concern the approach looks good.
> >> >>
> >> >>>
> >> >>> Regards;
> >> >>> ----------
> >> >>> Koichi Suzuki
> >> >>>
> >> >>>
> >> >>>
> >> >>> 2012/3/7 Andrei Martsinchyk <and...@gm...>:
> >> >>> > Agree.
> >> >>> > But now it does not output distribution type and distribution
> >> >>> > column.
> >> >>> > The patch fixes this.
> >> >>> >
> >> >>> > 7 марта 2012 г. 3:38 пользователь Koichi Suzuki
> >> >>> > <ko...@in...>
> >> >>> > написал:
> >> >>> >
> >> >>> >> It's nice that pg_dump doesn't dump node group because it is used
> >> >>> >> to
> >> >>> >> rebalance tables for different node configuration.     We may
> need
> >> >>> >> some
> >> >>> >> options on node-group in the future.
> >> >>> >>
> >> >>> >> Regards;
> >> >>> >> ---
> >> >>> >> Koichi
> >> >>> >>
> >> >>> >>  Tue, 6 Mar 2012 21:03:29 +0300
> >> >>> >> Andrei Martsinchyk <and...@gm...> wrote:
> >> >>> >>
> >> >>> >> > Hi,
> >> >>> >> >
> >> >>> >> > I have noticed that pg_dump does not output table distribution
> >> >>> >> > info
> >> >>> >> > as
> >> >>> >> > it
> >> >>> >> > used to do.
> >> >>> >> > It looks like there was an incorrect merge. Please find fix
> >> >>> >> > attached.
> >> >>> >> >
> >> >>> >> > --
> >> >>> >> > Best regards,
> >> >>> >> > Andrei Martsinchyk
> >> >>> >> > mailto:and...@gm...
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> > --
> >> >>> > Best regards,
> >> >>> > Andrei Martsinchyk
> >> >>> > mailto:and...@gm...
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> >
> ------------------------------------------------------------------------------
> >> >>> > Virtualization & Cloud Management Using Capacity Planning
> >> >>> > Cloud computing makes use of virtualization - but cloud computing
> >> >>> > also focuses on allowing computing to be delivered as a service.
> >> >>> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
> >> >>> > _______________________________________________
> >> >>> > Postgres-xc-developers mailing list
> >> >>> > Pos...@li...
> >> >>> >
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >> >>> >
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> ------------------------------------------------------------------------------
> >> >>> Virtualization & Cloud Management Using Capacity Planning
> >> >>> Cloud computing makes use of virtualization - but cloud computing
> >> >>> also focuses on allowing computing to be delivered as a service.
> >> >>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> >> >>> _______________________________________________
> >> >>> Postgres-xc-developers mailing list
> >> >>> Pos...@li...
> >> >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Best Wishes,
> >> >> Ashutosh Bapat
> >> >> EntepriseDB Corporation
> >> >> The Enterprise Postgres Company
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> ------------------------------------------------------------------------------
> >> >> Virtualization & Cloud Management Using Capacity Planning
> >> >> Cloud computing makes use of virtualization - but cloud computing
> >> >> also focuses on allowing computing to be delivered as a service.
> >> >> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> >> >> _______________________________________________
> >> >> Postgres-xc-developers mailing list
> >> >> Pos...@li...
> >> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >> >>
> >> >
> >
> >
>
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>



-- 
Michael Paquier
http://michael.otacoo.com