postgres-xc-developers Mailing List for Postgres-XC

There is also something I forgot to propose.
Once we think we are done with beta1.
I think we'll also need a beta2 phase because we may need some
stabilization work after backporting all the patches of 9.1 stable branch
of postgres.
This should not be long though.

On Tue, Mar 6, 2012 at 2:21 PM, Koichi Suzuki <koi...@gm...> wrote:

> In terms of the planner improvement, this is never-ending effort we
> should focus on.   You see Tom Lane is paying much effort to improve
> vanilla PostgreSQL planner even now.   (He is trying to do this even
> after the code is frozen!)
>
> For 1.0, it seems to me at reasonable level and we should concentrate
> on the stabilization.
>
> Regarding the report from Michael and related discussions, I'm
> thinking to carry over the trigger to later version.    To announce
> that 1.0 has sufficient HA feature, I'd like to keep pgxc_clean
> effort, which needs a bit of extension to the core, although most of
> the work is outside the core.
>
> Regards;
> ----------
> Koichi Suzuki
>
>
>
> 2012/3/6 Ahsan Hadi <ahs...@en...>:
> > I think this makes a lot of sense, we should be gearing towards
> code-freeze
> > for 1.0 by end of this month. At this point most to of our efforts
> should be
> > focussed on regression stabilization and fixing important bugs from the
> bug
> > tracking system.
> >
> > Amit is already looking at regression failures, Abbas started analysis of
> > the open bugs yesterday, once he is done with his analysis, he will close
> > the bugs already fixed, divide the remaining ones between bugs and
> > enhancements and move the obvious ones that we cannot target this month
> to
> > beyond 1.0. Amit and Abbas can focus on regression fixes and other bug
> fixes
> > for the remainder of this month.
> >
> > The planner improvements was an important piece, i assume what was needed
> > for 1.0 is already in place?
> >
> >
> > On Tue, Mar 6, 2012 at 7:47 AM, Michael Paquier <
> mic...@gm...>
> > wrote:
> >>
> >> Hi all,
> >>
> >> As we are entering in the 1.0 release process, I propose that we do a
> >> feature freeze at the end of the month.
> >> This means that no new features will be added to core after the 31st of
> >> March, but external utilities like pgxc_clean and initgtm (that I should
> >> finish before this date though), do not enter in this scope.
> >> So, from this date, all the effort will be focused on stabilization and
> >> bug fix.
> >> On the 1st of April, I also propose to mark XC not as 1.0devel but as
> >> 1.0beta1.
> >> This could be delayed a couple of days of course if a feature we think
> >> necessary is late.
> >>
> >> Any thoughts/comments?
> >> --
> >> Michael Paquier
> >> http://michael.otacoo.com
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Keep Your Developer Skills Current with LearnDevNow!
> >> The most comprehensive online learning library for Microsoft developers
> >> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
> >> Metro Style Apps, more. Free future releases when you subscribe now!
> >> http://p.sf.net/sfu/learndevnow-d2d
> >> _______________________________________________
> >> Postgres-xc-developers mailing list
> >> Pos...@li...
> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >>
> >
> >
> >
> > --
> > Ahsan Hadi
> > Snr Director Product Development
> > EnterpriseDB Corporation
> > The Enterprise Postgres Company
> >
> > Phone: +92-51-8358874
> > Mobile: +92-333-5162114
> >
> > Website: www.enterprisedb.com
> > EnterpriseDB Blog: http://blogs.enterprisedb.com/
> > Follow us on Twitter: http://www.twitter.com/enterprisedb
> >
> > This e-mail message (and any attachment) is intended for the use of the
> > individual or entity to whom it is addressed. This message contains
> > information from EnterpriseDB Corporation that may be privileged,
> > confidential, or exempt from disclosure under applicable law. If you are
> not
> > the intended recipient or authorized to receive this for the intended
> > recipient, any use, dissemination, distribution, retention, archiving, or
> > copying of this communication is strictly prohibited. If you have
> received
> > this e-mail in error, please notify the sender immediately by reply
> e-mail
> > and delete this message.
> >
> >
> ------------------------------------------------------------------------------
> > Keep Your Developer Skills Current with LearnDevNow!
> > The most comprehensive online learning library for Microsoft developers
> > is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
> > Metro Style Apps, more. Free future releases when you subscribe now!
> > http://p.sf.net/sfu/learndevnow-d2d
> > _______________________________________________
> > Postgres-xc-developers mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >
>



-- 
Michael Paquier
http://michael.otacoo.com

On Wed, Mar 7, 2012 at 3:01 PM, Ashutosh Bapat
<ash...@en...> wrote:
> Pavan,
> Instead of making this fix only for catalog tables, you can make it generic
> for all local tables. For now catalogs are the only tables local to a
> coordinator, but that's something which may change sometime. Existence of
> Relation->rd_locator_info would be an indicator as to whether this table is
> local or not.
>

I think we should have a mechanism to identify local/remote tables
whenever we have that situation. But I don't think we need to worry
about that now.

> IsSystemRelation is true for relations in toast namespace. Is that expected
> here?
>

Toast tables are always associated with the actual tables. So we don't
need to worry about them.

Thanks,
Pavan

-- 
Pavan Deolasee
EnterpriseDB http://www.enterprisedb.com

Pavan,
Instead of making this fix only for catalog tables, you can make it generic
for all local tables. For now catalogs are the only tables local to a
coordinator, but that's something which may change sometime. Existence of
Relation->rd_locator_info would be an indicator as to whether this table is
local or not.

IsSystemRelation is true for relations in toast namespace. Is that expected
here?

On Wed, Mar 7, 2012 at 2:26 PM, Pavan Deolasee <
pav...@en...> wrote:

> PFA patch which addresses the issue for catalog tables. This is a
> work-around and we should take a better path sometime in future to
> handle remote joins for cleanly.
>
> Thanks,
> Pavan
>
> On Wed, Mar 7, 2012 at 10:00 AM, Ashutosh Bapat
> <ash...@en...> wrote:
> > Pavan,
> > You snatched words from my mouth :). I was drafting a similar proposal
> > yesterday but couldn't complete. I will drop it today.
> >
> >
> > On Wed, Mar 7, 2012 at 9:50 AM, Koichi Suzuki <koi...@gm...>
> wrote:
> >>
> >> +1, Indeed!
> >>
> >> We should be careful to find similar cases.
> >> ----------
> >> Koichi Suzuki
> >>
> >>
> >>
> >> 2012/3/7 Pavan Deolasee <pav...@en...>:
> >> > So after cribbing for it for several days, I ended up investigating
> >> > the super slowness of opr_sanity test. I turned the timing on in psql
> >> > and looked for queries that are taking very long. Its a join on a
> >> > catalog table thats taking a very long time compared to vanilla
> >> > PostgreSQL.
> >> >
> >> > Upon further investigation, I realized that these catalog join queries
> >> > are always using nested-loop join instead of far more efficient hash
> >> > or merge join. The following kludge in relnode.c is causing this:
> >> >
> >> >  100     switch (rte->rtekind)
> >> >  101     {
> >> >  102         case RTE_RELATION:
> >> >  103             /* Table --- retrieve statistics from the system
> >> > catalogs */
> >> >  104             get_relation_info(root, rte->relid, rte->inh, rel);
> >> >  105 #ifdef PGXC
> >> >  106             /*
> >> >  107              * This is a remote table... we have no idea how
> >> > many pages/rows
> >> >  108              * we may get from a scan of this table. However, we
> >> > should set the
> >> >  109              * costs in such a manner that cheapest paths should
> >> > pick up the
> >> >  110              * ones involving these remote rels
> >> >  111              *
> >> >  112              * These allow for maximum query shipping to the
> remote
> >> >  113              * side later during the planning phase
> >> >  114              *
> >> >  115              * This has to be set on a remote Coordinator only
> >> >  116              * as it hugely penalizes performance on backend
> Nodes
> >> >  117              */
> >> >  118             if (IS_PGXC_COORDINATOR && !IsConnFromCoord())
> >> >  119             {
> >> >  120                 rel->pages   = 1;
> >> >  121                 rel->tuples  = 1;
> >> >  122                 rel->rows    = 1;
> >> >  123             }
> >> >  124 #endif
> >> >
> >> >
> >> > We override the relation size estimates at the coordinator. I think we
> >> > did this to enforce a nestloop join on the remote tables at the
> >> > coordinator because we have the ability to reduce only that kind of
> >> > join. But this can have very adverse effect on some other cases,
> >> > especially when the join is not reducible. We would be running a very
> >> > poor plan in those cases. This is demonstrated by the opr_sanity test
> >> > where the query is executed completely at the coordinator, but the
> >> > nest-loop join shows a very bad performance.
> >> >
> >> > The specific case of opr_sanity or catalog tables can be fixed easily
> >> > by using the real estimates for catalog tables. But I think a better
> >> > solution is needed (I and Ashutosh had discussed about this kludge
> >> > before privately). We probably need to hook into the path creation
> >> > logic and also create a path for remote join and pick the best path
> >> > based on the cost estimate. This will be a lot more complicated though
> >> > because we may need to estimate the cost of remote join to accurately
> >> > determine the best plan. But currently we don't have any statistics
> >> > about the remote tables at the coordinator. One way to circumvent the
> >> > problem is to run EXPLAIN ANALYZE first and use its result to
> >> > determine the cost of remote operations, but that can have adverse
> >> > effect especially when query execution time is very small.
> >> >
> >> > I will probably fix the catalog tables case for now. But lets discuss
> >> > a better long term solution.
> >> >
> >> > Thanks,
> >> > Pavan
> >> >
> >> > --
> >> > Pavan Deolasee
> >> > EnterpriseDB http://www.enterprisedb.com
> >> >
> >> >
> >> >
> ------------------------------------------------------------------------------
> >> > Virtualization & Cloud Management Using Capacity Planning
> >> > Cloud computing makes use of virtualization - but cloud computing
> >> > also focuses on allowing computing to be delivered as a service.
> >> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
> >> > _______________________________________________
> >> > Postgres-xc-developers mailing list
> >> > Pos...@li...
> >> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Virtualization & Cloud Management Using Capacity Planning
> >> Cloud computing makes use of virtualization - but cloud computing
> >> also focuses on allowing computing to be delivered as a service.
> >> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> >> _______________________________________________
> >> Postgres-xc-developers mailing list
> >> Pos...@li...
> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >
> >
> >
> >
> > --
> > Best Wishes,
> > Ashutosh Bapat
> > EntepriseDB Corporation
> > The Enterprise Postgres Company
> >
>
>
>
> --
> Pavan Deolasee
> EnterpriseDB http://www.enterprisedb.com
>



-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

redistribution of indexes?
that would be REALLY cool.

On Wed, Mar 7, 2012 at 4:32 PM, Koichi Suzuki <koi...@gm...> wrote:

> Thanks a lot for the advice.   Also, we should reconstruct indexes
> too.   There could be other resources we should consider to support
> ALTER TABLE redistribution.
>
> Best regards;
> ----------
> Koichi Suzuki
>
>
>
> 2012/3/7 Chaos Eternal <cha...@sh...>:
> > in most case, views only store tablenames, while stored plans could be a
> > problem if we use a lot of them.
> >
> > my scenario of using pg-xc is to process a large amount of data, like
> OLAP,
> > and stored plans are rarely used.
> >
> >
> > 2012/3/7 Ashutosh Bapat <ash...@en...>
> >>
> >>
> >>
> >> On Wed, Mar 7, 2012 at 11:19 AM, Koichi Suzuki <koi...@gm...>
> >> wrote:
> >>>
> >>> Yes, I understand.   Before committing this, I'd like to have a review
> >>> of other members.
> >>>
> >>> BTW, a Chinese guy (I don't know his real name) wrote to me that we
> >>> can redistribute rows by using CREATE TABLE AS or SELECT INTO and then
> >>> dop the original one and rename the new one.   This can be very
> >>> flexible.   If DBAs are not satisfied with the current node
> >>> configuration, they can dump everything with pg_dump, reconfigure the
> >>> nodes, pg_restore and then run CTA or SI.
> >>>
> >>
> >> ALTER TABLE REDISTRIBUTION was an item on my plate for this quarter, but
> >> FQS took most of the time. Do you think, we should take that up? I am
> not
> >> sure, whether that can be completed by end of March, esp. there is work
> >> involved to send the rows back and forth between datanodes and
> coordinator.
> >>
> >> The approach suggested above will change the OID of the relation, which
> >> invalidates any view definitions, stored plans on the relation. If
> that's
> >> not a concern the approach looks good.
> >>
> >>>
> >>> Regards;
> >>> ----------
> >>> Koichi Suzuki
> >>>
> >>>
> >>>
> >>> 2012/3/7 Andrei Martsinchyk <and...@gm...>:
> >>> > Agree.
> >>> > But now it does not output distribution type and distribution column.
> >>> > The patch fixes this.
> >>> >
> >>> > 7 марта 2012 г. 3:38 пользователь Koichi Suzuki
> >>> > <ko...@in...>
> >>> > написал:
> >>> >
> >>> >> It's nice that pg_dump doesn't dump node group because it is used to
> >>> >> rebalance tables for different node configuration.     We may need
> >>> >> some
> >>> >> options on node-group in the future.
> >>> >>
> >>> >> Regards;
> >>> >> ---
> >>> >> Koichi
> >>> >>
> >>> >>  Tue, 6 Mar 2012 21:03:29 +0300
> >>> >> Andrei Martsinchyk <and...@gm...> wrote:
> >>> >>
> >>> >> > Hi,
> >>> >> >
> >>> >> > I have noticed that pg_dump does not output table distribution
> info
> >>> >> > as
> >>> >> > it
> >>> >> > used to do.
> >>> >> > It looks like there was an incorrect merge. Please find fix
> >>> >> > attached.
> >>> >> >
> >>> >> > --
> >>> >> > Best regards,
> >>> >> > Andrei Martsinchyk
> >>> >> > mailto:and...@gm...
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Best regards,
> >>> > Andrei Martsinchyk
> >>> > mailto:and...@gm...
> >>> >
> >>> >
> >>> >
> ------------------------------------------------------------------------------
> >>> > Virtualization & Cloud Management Using Capacity Planning
> >>> > Cloud computing makes use of virtualization - but cloud computing
> >>> > also focuses on allowing computing to be delivered as a service.
> >>> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
> >>> > _______________________________________________
> >>> > Postgres-xc-developers mailing list
> >>> > Pos...@li...
> >>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >>> >
> >>>
> >>>
> >>>
> ------------------------------------------------------------------------------
> >>> Virtualization & Cloud Management Using Capacity Planning
> >>> Cloud computing makes use of virtualization - but cloud computing
> >>> also focuses on allowing computing to be delivered as a service.
> >>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> >>> _______________________________________________
> >>> Postgres-xc-developers mailing list
> >>> Pos...@li...
> >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >>
> >>
> >>
> >>
> >> --
> >> Best Wishes,
> >> Ashutosh Bapat
> >> EntepriseDB Corporation
> >> The Enterprise Postgres Company
> >>
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Virtualization & Cloud Management Using Capacity Planning
> >> Cloud computing makes use of virtualization - but cloud computing
> >> also focuses on allowing computing to be delivered as a service.
> >> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> >> _______________________________________________
> >> Postgres-xc-developers mailing list
> >> Pos...@li...
> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >>
> >
>

Redustribution would take time, especially redistribution of a huge table.
It is better to restore teble with proper distribution already.
That is correct approach to run INSERT SELECT under the cover when
redistributing table.
It would be user-friendly to provide command like ALTER TABLE DISTRIBUTION
that would do all at once, together with all the housekeeping, like proper
locking and indexing.


7 марта 2012 г. 11:42 пользователь Koichi Suzuki <koi...@gm...>написал:

> I see.
>
> In table redistribution, it is we have to review what other resource
> should be maintained too.   Views, indexes, rules (although XC does
> not fully support the rule yet), triggers (when XC supports this),
> constraints, and so on.   This is not a simple work but must be cool!
> ----------
> Koichi Suzuki
>
>
>
> 2012/3/7 Ashutosh Bapat <ash...@en...>:
> >
> >
> > On Wed, Mar 7, 2012 at 1:58 PM, Chaos Eternal <cha...@sh...>
> > wrote:
> >>
> >> in most case, views only store tablenames,
> >
> >
> > That's not true, I guess. The views store the parse tree, which contains
> > relation OIDs and not name. See the test case below
> > testdb=# create table tab1 (val int, val2 int);
> > selCREATE TABLE
> > testdb=# select * from tab1;
> >  val | val2
> > -----+------
> > (0 rows)
> >
> > testdb=# create view v_tab1 as select * from tab1;
> > CREATE VIEW
> > testdb=# select * from v_tab1;
> >  val | val2
> > -----+------
> > (0 rows)
> >
> > testdb=# alter table tab1 rename to tab2;
> > ALTER TABLE
> > testdb=# select * from v_tab1;
> >  val | val2
> > -----+------
> > (0 rows)
> >
> > If the view would have retained the tablename it would not have worked
> after
> > table tab1 is renamed to tab2. It works because we stick to OID of the
> > relation and not the name itself and ALTER TABLE does not change the OID.
> >
> >>
> >> while stored plans could be a problem if we use a lot of them.
> >>
> >> my scenario of using pg-xc is to process a large amount of data, like
> >> OLAP, and stored plans are rarely used.
> >>
> >>
> >> 2012/3/7 Ashutosh Bapat <ash...@en...>
> >>>
> >>>
> >>>
> >>> On Wed, Mar 7, 2012 at 11:19 AM, Koichi Suzuki <koi...@gm...>
> >>> wrote:
> >>>>
> >>>> Yes, I understand.   Before committing this, I'd like to have a review
> >>>> of other members.
> >>>>
> >>>> BTW, a Chinese guy (I don't know his real name) wrote to me that we
> >>>> can redistribute rows by using CREATE TABLE AS or SELECT INTO and then
> >>>> dop the original one and rename the new one.   This can be very
> >>>> flexible.   If DBAs are not satisfied with the current node
> >>>> configuration, they can dump everything with pg_dump, reconfigure the
> >>>> nodes, pg_restore and then run CTA or SI.
> >>>>
> >>>
> >>> ALTER TABLE REDISTRIBUTION was an item on my plate for this quarter,
> but
> >>> FQS took most of the time. Do you think, we should take that up? I am
> not
> >>> sure, whether that can be completed by end of March, esp. there is work
> >>> involved to send the rows back and forth between datanodes and
> coordinator.
> >>>
> >>> The approach suggested above will change the OID of the relation, which
> >>> invalidates any view definitions, stored plans on the relation. If
> that's
> >>> not a concern the approach looks good.
> >>>
> >>>>
> >>>> Regards;
> >>>> ----------
> >>>> Koichi Suzuki
> >>>>
> >>>>
> >>>>
> >>>> 2012/3/7 Andrei Martsinchyk <and...@gm...>:
> >>>> > Agree.
> >>>> > But now it does not output distribution type and distribution
> column.
> >>>> > The patch fixes this.
> >>>> >
> >>>> > 7 марта 2012 г. 3:38 пользователь Koichi Suzuki
> >>>> > <ko...@in...>
> >>>> > написал:
> >>>> >
> >>>> >> It's nice that pg_dump doesn't dump node group because it is used
> to
> >>>> >> rebalance tables for different node configuration.     We may need
> >>>> >> some
> >>>> >> options on node-group in the future.
> >>>> >>
> >>>> >> Regards;
> >>>> >> ---
> >>>> >> Koichi
> >>>> >>
> >>>> >>  Tue, 6 Mar 2012 21:03:29 +0300
> >>>> >> Andrei Martsinchyk <and...@gm...> wrote:
> >>>> >>
> >>>> >> > Hi,
> >>>> >> >
> >>>> >> > I have noticed that pg_dump does not output table distribution
> info
> >>>> >> > as
> >>>> >> > it
> >>>> >> > used to do.
> >>>> >> > It looks like there was an incorrect merge. Please find fix
> >>>> >> > attached.
> >>>> >> >
> >>>> >> > --
> >>>> >> > Best regards,
> >>>> >> > Andrei Martsinchyk
> >>>> >> > mailto:and...@gm...
> >>>> >
> >>>> >
> >>>> >
> >>>> >
> >>>> > --
> >>>> > Best regards,
> >>>> > Andrei Martsinchyk
> >>>> > mailto:and...@gm...
> >>>> >
> >>>> >
> >>>> >
> ------------------------------------------------------------------------------
> >>>> > Virtualization & Cloud Management Using Capacity Planning
> >>>> > Cloud computing makes use of virtualization - but cloud computing
> >>>> > also focuses on allowing computing to be delivered as a service.
> >>>> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
> >>>> > _______________________________________________
> >>>> > Postgres-xc-developers mailing list
> >>>> > Pos...@li...
> >>>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >>>> >
> >>>>
> >>>>
> >>>>
> ------------------------------------------------------------------------------
> >>>> Virtualization & Cloud Management Using Capacity Planning
> >>>> Cloud computing makes use of virtualization - but cloud computing
> >>>> also focuses on allowing computing to be delivered as a service.
> >>>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> >>>> _______________________________________________
> >>>> Postgres-xc-developers mailing list
> >>>> Pos...@li...
> >>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> Best Wishes,
> >>> Ashutosh Bapat
> >>> EntepriseDB Corporation
> >>> The Enterprise Postgres Company
> >>>
> >>>
> >>>
> >>>
> ------------------------------------------------------------------------------
> >>> Virtualization & Cloud Management Using Capacity Planning
> >>> Cloud computing makes use of virtualization - but cloud computing
> >>> also focuses on allowing computing to be delivered as a service.
> >>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> >>> _______________________________________________
> >>> Postgres-xc-developers mailing list
> >>> Pos...@li...
> >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >>>
> >>
> >
> >
> >
> > --
> > Best Wishes,
> > Ashutosh Bapat
> > EntepriseDB Corporation
> > The Enterprise Postgres Company
> >
>



-- 
Best regards,
Andrei Martsinchyk                   mailto:and...@gm...

PFA patch which addresses the issue for catalog tables. This is a
work-around and we should take a better path sometime in future to
handle remote joins for cleanly.

Thanks,
Pavan

On Wed, Mar 7, 2012 at 10:00 AM, Ashutosh Bapat
<ash...@en...> wrote:
> Pavan,
> You snatched words from my mouth :). I was drafting a similar proposal
> yesterday but couldn't complete. I will drop it today.
>
>
> On Wed, Mar 7, 2012 at 9:50 AM, Koichi Suzuki <koi...@gm...> wrote:
>>
>> +1, Indeed!
>>
>> We should be careful to find similar cases.
>> ----------
>> Koichi Suzuki
>>
>>
>>
>> 2012/3/7 Pavan Deolasee <pav...@en...>:
>> > So after cribbing for it for several days, I ended up investigating
>> > the super slowness of opr_sanity test. I turned the timing on in psql
>> > and looked for queries that are taking very long. Its a join on a
>> > catalog table thats taking a very long time compared to vanilla
>> > PostgreSQL.
>> >
>> > Upon further investigation, I realized that these catalog join queries
>> > are always using nested-loop join instead of far more efficient hash
>> > or merge join. The following kludge in relnode.c is causing this:
>> >
>> >  100     switch (rte->rtekind)
>> >  101     {
>> >  102         case RTE_RELATION:
>> >  103             /* Table --- retrieve statistics from the system
>> > catalogs */
>> >  104             get_relation_info(root, rte->relid, rte->inh, rel);
>> >  105 #ifdef PGXC
>> >  106             /*
>> >  107              * This is a remote table... we have no idea how
>> > many pages/rows
>> >  108              * we may get from a scan of this table. However, we
>> > should set the
>> >  109              * costs in such a manner that cheapest paths should
>> > pick up the
>> >  110              * ones involving these remote rels
>> >  111              *
>> >  112              * These allow for maximum query shipping to the remote
>> >  113              * side later during the planning phase
>> >  114              *
>> >  115              * This has to be set on a remote Coordinator only
>> >  116              * as it hugely penalizes performance on backend Nodes
>> >  117              */
>> >  118             if (IS_PGXC_COORDINATOR && !IsConnFromCoord())
>> >  119             {
>> >  120                 rel->pages   = 1;
>> >  121                 rel->tuples  = 1;
>> >  122                 rel->rows    = 1;
>> >  123             }
>> >  124 #endif
>> >
>> >
>> > We override the relation size estimates at the coordinator. I think we
>> > did this to enforce a nestloop join on the remote tables at the
>> > coordinator because we have the ability to reduce only that kind of
>> > join. But this can have very adverse effect on some other cases,
>> > especially when the join is not reducible. We would be running a very
>> > poor plan in those cases. This is demonstrated by the opr_sanity test
>> > where the query is executed completely at the coordinator, but the
>> > nest-loop join shows a very bad performance.
>> >
>> > The specific case of opr_sanity or catalog tables can be fixed easily
>> > by using the real estimates for catalog tables. But I think a better
>> > solution is needed (I and Ashutosh had discussed about this kludge
>> > before privately). We probably need to hook into the path creation
>> > logic and also create a path for remote join and pick the best path
>> > based on the cost estimate. This will be a lot more complicated though
>> > because we may need to estimate the cost of remote join to accurately
>> > determine the best plan. But currently we don't have any statistics
>> > about the remote tables at the coordinator. One way to circumvent the
>> > problem is to run EXPLAIN ANALYZE first and use its result to
>> > determine the cost of remote operations, but that can have adverse
>> > effect especially when query execution time is very small.
>> >
>> > I will probably fix the catalog tables case for now. But lets discuss
>> > a better long term solution.
>> >
>> > Thanks,
>> > Pavan
>> >
>> > --
>> > Pavan Deolasee
>> > EnterpriseDB http://www.enterprisedb.com
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Virtualization & Cloud Management Using Capacity Planning
>> > Cloud computing makes use of virtualization - but cloud computing
>> > also focuses on allowing computing to be delivered as a service.
>> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> > _______________________________________________
>> > Postgres-xc-developers mailing list
>> > Pos...@li...
>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>>
>> ------------------------------------------------------------------------------
>> Virtualization & Cloud Management Using Capacity Planning
>> Cloud computing makes use of virtualization - but cloud computing
>> also focuses on allowing computing to be delivered as a service.
>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EntepriseDB Corporation
> The Enterprise Postgres Company
>



-- 
Pavan Deolasee
EnterpriseDB http://www.enterprisedb.com

in most case, views only store tablenames, while stored plans could be a
problem if we use a lot of them.

my scenario of using pg-xc is to process a large amount of data, like OLAP,
and stored plans are rarely used.

2012/3/7 Ashutosh Bapat <ash...@en...>

>
>
> On Wed, Mar 7, 2012 at 11:19 AM, Koichi Suzuki <koi...@gm...>wrote:
>
>> Yes, I understand.   Before committing this, I'd like to have a review
>> of other members.
>>
>> BTW, a Chinese guy (I don't know his real name) wrote to me that we
>> can redistribute rows by using CREATE TABLE AS or SELECT INTO and then
>> dop the original one and rename the new one.   This can be very
>> flexible.   If DBAs are not satisfied with the current node
>> configuration, they can dump everything with pg_dump, reconfigure the
>> nodes, pg_restore and then run CTA or SI.
>>
>>
> ALTER TABLE REDISTRIBUTION was an item on my plate for this quarter, but
> FQS took most of the time. Do you think, we should take that up? I am not
> sure, whether that can be completed by end of March, esp. there is work
> involved to send the rows back and forth between datanodes and coordinator.
>
> The approach suggested above will change the OID of the relation, which
> invalidates any view definitions, stored plans on the relation. If that's
> not a concern the approach looks good.
>
>
>> Regards;
>> ----------
>> Koichi Suzuki
>>
>>
>>
>> 2012/3/7 Andrei Martsinchyk <and...@gm...>:
>> > Agree.
>> > But now it does not output distribution type and distribution column.
>> > The patch fixes this.
>> >
>> > 7 марта 2012 г. 3:38 пользователь Koichi Suzuki <
>> ko...@in...>
>> > написал:
>> >
>> >> It's nice that pg_dump doesn't dump node group because it is used to
>> >> rebalance tables for different node configuration.     We may need some
>> >> options on node-group in the future.
>> >>
>> >> Regards;
>> >> ---
>> >> Koichi
>> >>
>> >>  Tue, 6 Mar 2012 21:03:29 +0300
>> >> Andrei Martsinchyk <and...@gm...> wrote:
>> >>
>> >> > Hi,
>> >> >
>> >> > I have noticed that pg_dump does not output table distribution info
>> as
>> >> > it
>> >> > used to do.
>> >> > It looks like there was an incorrect merge. Please find fix attached.
>> >> >
>> >> > --
>> >> > Best regards,
>> >> > Andrei Martsinchyk                   mailto:
>> and...@gm...
>> >
>> >
>> >
>> >
>> > --
>> > Best regards,
>> > Andrei Martsinchyk                   mailto:
>> and...@gm...
>> >
>> >
>> ------------------------------------------------------------------------------
>> > Virtualization & Cloud Management Using Capacity Planning
>> > Cloud computing makes use of virtualization - but cloud computing
>> > also focuses on allowing computing to be delivered as a service.
>> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> > _______________________________________________
>> > Postgres-xc-developers mailing list
>> > Pos...@li...
>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>> >
>>
>>
>> ------------------------------------------------------------------------------
>> Virtualization & Cloud Management Using Capacity Planning
>> Cloud computing makes use of virtualization - but cloud computing
>> also focuses on allowing computing to be delivered as a service.
>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EntepriseDB Corporation
> The Enterprise Postgres Company
>
>
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>

I see.

In table redistribution, it is we have to review what other resource
should be maintained too.   Views, indexes, rules (although XC does
not fully support the rule yet), triggers (when XC supports this),
constraints, and so on.   This is not a simple work but must be cool!
----------
Koichi Suzuki



2012/3/7 Ashutosh Bapat <ash...@en...>:
>
>
> On Wed, Mar 7, 2012 at 1:58 PM, Chaos Eternal <cha...@sh...>
> wrote:
>>
>> in most case, views only store tablenames,
>
>
> That's not true, I guess. The views store the parse tree, which contains
> relation OIDs and not name. See the test case below
> testdb=# create table tab1 (val int, val2 int);
> selCREATE TABLE
> testdb=# select * from tab1;
>  val | val2
> -----+------
> (0 rows)
>
> testdb=# create view v_tab1 as select * from tab1;
> CREATE VIEW
> testdb=# select * from v_tab1;
>  val | val2
> -----+------
> (0 rows)
>
> testdb=# alter table tab1 rename to tab2;
> ALTER TABLE
> testdb=# select * from v_tab1;
>  val | val2
> -----+------
> (0 rows)
>
> If the view would have retained the tablename it would not have worked after
> table tab1 is renamed to tab2. It works because we stick to OID of the
> relation and not the name itself and ALTER TABLE does not change the OID.
>
>>
>> while stored plans could be a problem if we use a lot of them.
>>
>> my scenario of using pg-xc is to process a large amount of data, like
>> OLAP, and stored plans are rarely used.
>>
>>
>> 2012/3/7 Ashutosh Bapat <ash...@en...>
>>>
>>>
>>>
>>> On Wed, Mar 7, 2012 at 11:19 AM, Koichi Suzuki <koi...@gm...>
>>> wrote:
>>>>
>>>> Yes, I understand.   Before committing this, I'd like to have a review
>>>> of other members.
>>>>
>>>> BTW, a Chinese guy (I don't know his real name) wrote to me that we
>>>> can redistribute rows by using CREATE TABLE AS or SELECT INTO and then
>>>> dop the original one and rename the new one.   This can be very
>>>> flexible.   If DBAs are not satisfied with the current node
>>>> configuration, they can dump everything with pg_dump, reconfigure the
>>>> nodes, pg_restore and then run CTA or SI.
>>>>
>>>
>>> ALTER TABLE REDISTRIBUTION was an item on my plate for this quarter, but
>>> FQS took most of the time. Do you think, we should take that up? I am not
>>> sure, whether that can be completed by end of March, esp. there is work
>>> involved to send the rows back and forth between datanodes and coordinator.
>>>
>>> The approach suggested above will change the OID of the relation, which
>>> invalidates any view definitions, stored plans on the relation. If that's
>>> not a concern the approach looks good.
>>>
>>>>
>>>> Regards;
>>>> ----------
>>>> Koichi Suzuki
>>>>
>>>>
>>>>
>>>> 2012/3/7 Andrei Martsinchyk <and...@gm...>:
>>>> > Agree.
>>>> > But now it does not output distribution type and distribution column.
>>>> > The patch fixes this.
>>>> >
>>>> > 7 марта 2012 г. 3:38 пользователь Koichi Suzuki
>>>> > <ko...@in...>
>>>> > написал:
>>>> >
>>>> >> It's nice that pg_dump doesn't dump node group because it is used to
>>>> >> rebalance tables for different node configuration.     We may need
>>>> >> some
>>>> >> options on node-group in the future.
>>>> >>
>>>> >> Regards;
>>>> >> ---
>>>> >> Koichi
>>>> >>
>>>> >>  Tue, 6 Mar 2012 21:03:29 +0300
>>>> >> Andrei Martsinchyk <and...@gm...> wrote:
>>>> >>
>>>> >> > Hi,
>>>> >> >
>>>> >> > I have noticed that pg_dump does not output table distribution info
>>>> >> > as
>>>> >> > it
>>>> >> > used to do.
>>>> >> > It looks like there was an incorrect merge. Please find fix
>>>> >> > attached.
>>>> >> >
>>>> >> > --
>>>> >> > Best regards,
>>>> >> > Andrei Martsinchyk
>>>> >> > mailto:and...@gm...
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Best regards,
>>>> > Andrei Martsinchyk
>>>> > mailto:and...@gm...
>>>> >
>>>> >
>>>> > ------------------------------------------------------------------------------
>>>> > Virtualization & Cloud Management Using Capacity Planning
>>>> > Cloud computing makes use of virtualization - but cloud computing
>>>> > also focuses on allowing computing to be delivered as a service.
>>>> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
>>>> > _______________________________________________
>>>> > Postgres-xc-developers mailing list
>>>> > Pos...@li...
>>>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>> >
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Virtualization & Cloud Management Using Capacity Planning
>>>> Cloud computing makes use of virtualization - but cloud computing
>>>> also focuses on allowing computing to be delivered as a service.
>>>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>>>> _______________________________________________
>>>> Postgres-xc-developers mailing list
>>>> Pos...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>
>>>
>>>
>>>
>>> --
>>> Best Wishes,
>>> Ashutosh Bapat
>>> EntepriseDB Corporation
>>> The Enterprise Postgres Company
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Virtualization & Cloud Management Using Capacity Planning
>>> Cloud computing makes use of virtualization - but cloud computing
>>> also focuses on allowing computing to be delivered as a service.
>>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>>> _______________________________________________
>>> Postgres-xc-developers mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>
>>
>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EntepriseDB Corporation
> The Enterprise Postgres Company
>

Not index redistribution.   It is not practical because CTID will
change after redistribution.   Rather, index may have to be rebuilt at
ALTER TABLE redistribution.
----------
Koichi Suzuki



2012/3/7 Chaos Eternal <cha...@sh...>:
> redistribution of indexes?
> that would be REALLY cool.
>
>
> On Wed, Mar 7, 2012 at 4:32 PM, Koichi Suzuki <koi...@gm...> wrote:
>>
>> Thanks a lot for the advice.   Also, we should reconstruct indexes
>> too.   There could be other resources we should consider to support
>> ALTER TABLE redistribution.
>>
>> Best regards;
>> ----------
>> Koichi Suzuki
>>
>>
>>
>> 2012/3/7 Chaos Eternal <cha...@sh...>:
>> > in most case, views only store tablenames, while stored plans could be a
>> > problem if we use a lot of them.
>> >
>> > my scenario of using pg-xc is to process a large amount of data, like
>> > OLAP,
>> > and stored plans are rarely used.
>> >
>> >
>> > 2012/3/7 Ashutosh Bapat <ash...@en...>
>> >>
>> >>
>> >>
>> >> On Wed, Mar 7, 2012 at 11:19 AM, Koichi Suzuki <koi...@gm...>
>> >> wrote:
>> >>>
>> >>> Yes, I understand.   Before committing this, I'd like to have a review
>> >>> of other members.
>> >>>
>> >>> BTW, a Chinese guy (I don't know his real name) wrote to me that we
>> >>> can redistribute rows by using CREATE TABLE AS or SELECT INTO and then
>> >>> dop the original one and rename the new one.   This can be very
>> >>> flexible.   If DBAs are not satisfied with the current node
>> >>> configuration, they can dump everything with pg_dump, reconfigure the
>> >>> nodes, pg_restore and then run CTA or SI.
>> >>>
>> >>
>> >> ALTER TABLE REDISTRIBUTION was an item on my plate for this quarter,
>> >> but
>> >> FQS took most of the time. Do you think, we should take that up? I am
>> >> not
>> >> sure, whether that can be completed by end of March, esp. there is work
>> >> involved to send the rows back and forth between datanodes and
>> >> coordinator.
>> >>
>> >> The approach suggested above will change the OID of the relation, which
>> >> invalidates any view definitions, stored plans on the relation. If
>> >> that's
>> >> not a concern the approach looks good.
>> >>
>> >>>
>> >>> Regards;
>> >>> ----------
>> >>> Koichi Suzuki
>> >>>
>> >>>
>> >>>
>> >>> 2012/3/7 Andrei Martsinchyk <and...@gm...>:
>> >>> > Agree.
>> >>> > But now it does not output distribution type and distribution
>> >>> > column.
>> >>> > The patch fixes this.
>> >>> >
>> >>> > 7 марта 2012 г. 3:38 пользователь Koichi Suzuki
>> >>> > <ko...@in...>
>> >>> > написал:
>> >>> >
>> >>> >> It's nice that pg_dump doesn't dump node group because it is used
>> >>> >> to
>> >>> >> rebalance tables for different node configuration.     We may need
>> >>> >> some
>> >>> >> options on node-group in the future.
>> >>> >>
>> >>> >> Regards;
>> >>> >> ---
>> >>> >> Koichi
>> >>> >>
>> >>> >>  Tue, 6 Mar 2012 21:03:29 +0300
>> >>> >> Andrei Martsinchyk <and...@gm...> wrote:
>> >>> >>
>> >>> >> > Hi,
>> >>> >> >
>> >>> >> > I have noticed that pg_dump does not output table distribution
>> >>> >> > info
>> >>> >> > as
>> >>> >> > it
>> >>> >> > used to do.
>> >>> >> > It looks like there was an incorrect merge. Please find fix
>> >>> >> > attached.
>> >>> >> >
>> >>> >> > --
>> >>> >> > Best regards,
>> >>> >> > Andrei Martsinchyk
>> >>> >> > mailto:and...@gm...
>> >>> >
>> >>> >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > Best regards,
>> >>> > Andrei Martsinchyk
>> >>> > mailto:and...@gm...
>> >>> >
>> >>> >
>> >>> >
>> >>> > ------------------------------------------------------------------------------
>> >>> > Virtualization & Cloud Management Using Capacity Planning
>> >>> > Cloud computing makes use of virtualization - but cloud computing
>> >>> > also focuses on allowing computing to be delivered as a service.
>> >>> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> >>> > _______________________________________________
>> >>> > Postgres-xc-developers mailing list
>> >>> > Pos...@li...
>> >>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> ------------------------------------------------------------------------------
>> >>> Virtualization & Cloud Management Using Capacity Planning
>> >>> Cloud computing makes use of virtualization - but cloud computing
>> >>> also focuses on allowing computing to be delivered as a service.
>> >>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> >>> _______________________________________________
>> >>> Postgres-xc-developers mailing list
>> >>> Pos...@li...
>> >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Best Wishes,
>> >> Ashutosh Bapat
>> >> EntepriseDB Corporation
>> >> The Enterprise Postgres Company
>> >>
>> >>
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> Virtualization & Cloud Management Using Capacity Planning
>> >> Cloud computing makes use of virtualization - but cloud computing
>> >> also focuses on allowing computing to be delivered as a service.
>> >> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> >> _______________________________________________
>> >> Postgres-xc-developers mailing list
>> >> Pos...@li...
>> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>> >>
>> >
>
>

On Wed, Mar 7, 2012 at 1:58 PM, Chaos Eternal <cha...@sh...>wrote:

> in most case, views only store tablenames,


That's not true, I guess. The views store the parse tree, which contains
relation OIDs and not name. See the test case below
testdb=# create table tab1 (val int, val2 int);
selCREATE TABLE
testdb=# select * from tab1;
 val | val2
-----+------
(0 rows)

testdb=# create view v_tab1 as select * from tab1;
CREATE VIEW
testdb=# select * from v_tab1;
 val | val2
-----+------
(0 rows)

testdb=# alter table tab1 rename to tab2;
ALTER TABLE
testdb=# select * from v_tab1;
 val | val2
-----+------
(0 rows)

If the view would have retained the tablename it would not have worked
after table tab1 is renamed to tab2. It works because we stick to OID of
the relation and not the name itself and ALTER TABLE does not change the
OID.


> while stored plans could be a problem if we use a lot of them.
>
> my scenario of using pg-xc is to process a large amount of data, like
> OLAP, and stored plans are rarely used.
>
>
> 2012/3/7 Ashutosh Bapat <ash...@en...>
>
>>
>>
>> On Wed, Mar 7, 2012 at 11:19 AM, Koichi Suzuki <koi...@gm...>wrote:
>>
>>> Yes, I understand.   Before committing this, I'd like to have a review
>>> of other members.
>>>
>>> BTW, a Chinese guy (I don't know his real name) wrote to me that we
>>> can redistribute rows by using CREATE TABLE AS or SELECT INTO and then
>>> dop the original one and rename the new one.   This can be very
>>> flexible.   If DBAs are not satisfied with the current node
>>> configuration, they can dump everything with pg_dump, reconfigure the
>>> nodes, pg_restore and then run CTA or SI.
>>>
>>>
>> ALTER TABLE REDISTRIBUTION was an item on my plate for this quarter, but
>> FQS took most of the time. Do you think, we should take that up? I am not
>> sure, whether that can be completed by end of March, esp. there is work
>> involved to send the rows back and forth between datanodes and coordinator.
>>
>> The approach suggested above will change the OID of the relation, which
>> invalidates any view definitions, stored plans on the relation. If that's
>> not a concern the approach looks good.
>>
>>
>>> Regards;
>>> ----------
>>> Koichi Suzuki
>>>
>>>
>>>
>>> 2012/3/7 Andrei Martsinchyk <and...@gm...>:
>>> > Agree.
>>> > But now it does not output distribution type and distribution column.
>>> > The patch fixes this.
>>> >
>>> > 7 марта 2012 г. 3:38 пользователь Koichi Suzuki <
>>> ko...@in...>
>>> > написал:
>>> >
>>> >> It's nice that pg_dump doesn't dump node group because it is used to
>>> >> rebalance tables for different node configuration.     We may need
>>> some
>>> >> options on node-group in the future.
>>> >>
>>> >> Regards;
>>> >> ---
>>> >> Koichi
>>> >>
>>> >>  Tue, 6 Mar 2012 21:03:29 +0300
>>> >> Andrei Martsinchyk <and...@gm...> wrote:
>>> >>
>>> >> > Hi,
>>> >> >
>>> >> > I have noticed that pg_dump does not output table distribution info
>>> as
>>> >> > it
>>> >> > used to do.
>>> >> > It looks like there was an incorrect merge. Please find fix
>>> attached.
>>> >> >
>>> >> > --
>>> >> > Best regards,
>>> >> > Andrei Martsinchyk                   mailto:
>>> and...@gm...
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Best regards,
>>> > Andrei Martsinchyk                   mailto:
>>> and...@gm...
>>> >
>>> >
>>> ------------------------------------------------------------------------------
>>> > Virtualization & Cloud Management Using Capacity Planning
>>> > Cloud computing makes use of virtualization - but cloud computing
>>> > also focuses on allowing computing to be delivered as a service.
>>> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
>>> > _______________________________________________
>>> > Postgres-xc-developers mailing list
>>> > Pos...@li...
>>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>> >
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Virtualization & Cloud Management Using Capacity Planning
>>> Cloud computing makes use of virtualization - but cloud computing
>>> also focuses on allowing computing to be delivered as a service.
>>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>>> _______________________________________________
>>> Postgres-xc-developers mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>
>>
>>
>>
>> --
>> Best Wishes,
>> Ashutosh Bapat
>> EntepriseDB Corporation
>> The Enterprise Postgres Company
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Virtualization & Cloud Management Using Capacity Planning
>> Cloud computing makes use of virtualization - but cloud computing
>> also focuses on allowing computing to be delivered as a service.
>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>>
>


-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

Thanks a lot for the advice.   Also, we should reconstruct indexes
too.   There could be other resources we should consider to support
ALTER TABLE redistribution.

Best regards;
----------
Koichi Suzuki



2012/3/7 Chaos Eternal <cha...@sh...>:
> in most case, views only store tablenames, while stored plans could be a
> problem if we use a lot of them.
>
> my scenario of using pg-xc is to process a large amount of data, like OLAP,
> and stored plans are rarely used.
>
>
> 2012/3/7 Ashutosh Bapat <ash...@en...>
>>
>>
>>
>> On Wed, Mar 7, 2012 at 11:19 AM, Koichi Suzuki <koi...@gm...>
>> wrote:
>>>
>>> Yes, I understand.   Before committing this, I'd like to have a review
>>> of other members.
>>>
>>> BTW, a Chinese guy (I don't know his real name) wrote to me that we
>>> can redistribute rows by using CREATE TABLE AS or SELECT INTO and then
>>> dop the original one and rename the new one.   This can be very
>>> flexible.   If DBAs are not satisfied with the current node
>>> configuration, they can dump everything with pg_dump, reconfigure the
>>> nodes, pg_restore and then run CTA or SI.
>>>
>>
>> ALTER TABLE REDISTRIBUTION was an item on my plate for this quarter, but
>> FQS took most of the time. Do you think, we should take that up? I am not
>> sure, whether that can be completed by end of March, esp. there is work
>> involved to send the rows back and forth between datanodes and coordinator.
>>
>> The approach suggested above will change the OID of the relation, which
>> invalidates any view definitions, stored plans on the relation. If that's
>> not a concern the approach looks good.
>>
>>>
>>> Regards;
>>> ----------
>>> Koichi Suzuki
>>>
>>>
>>>
>>> 2012/3/7 Andrei Martsinchyk <and...@gm...>:
>>> > Agree.
>>> > But now it does not output distribution type and distribution column.
>>> > The patch fixes this.
>>> >
>>> > 7 марта 2012 г. 3:38 пользователь Koichi Suzuki
>>> > <ko...@in...>
>>> > написал:
>>> >
>>> >> It's nice that pg_dump doesn't dump node group because it is used to
>>> >> rebalance tables for different node configuration.     We may need
>>> >> some
>>> >> options on node-group in the future.
>>> >>
>>> >> Regards;
>>> >> ---
>>> >> Koichi
>>> >>
>>> >>  Tue, 6 Mar 2012 21:03:29 +0300
>>> >> Andrei Martsinchyk <and...@gm...> wrote:
>>> >>
>>> >> > Hi,
>>> >> >
>>> >> > I have noticed that pg_dump does not output table distribution info
>>> >> > as
>>> >> > it
>>> >> > used to do.
>>> >> > It looks like there was an incorrect merge. Please find fix
>>> >> > attached.
>>> >> >
>>> >> > --
>>> >> > Best regards,
>>> >> > Andrei Martsinchyk
>>> >> > mailto:and...@gm...
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Best regards,
>>> > Andrei Martsinchyk
>>> > mailto:and...@gm...
>>> >
>>> >
>>> > ------------------------------------------------------------------------------
>>> > Virtualization & Cloud Management Using Capacity Planning
>>> > Cloud computing makes use of virtualization - but cloud computing
>>> > also focuses on allowing computing to be delivered as a service.
>>> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
>>> > _______________________________________________
>>> > Postgres-xc-developers mailing list
>>> > Pos...@li...
>>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>> >
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Virtualization & Cloud Management Using Capacity Planning
>>> Cloud computing makes use of virtualization - but cloud computing
>>> also focuses on allowing computing to be delivered as a service.
>>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>>> _______________________________________________
>>> Postgres-xc-developers mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>>
>>
>>
>> --
>> Best Wishes,
>> Ashutosh Bapat
>> EntepriseDB Corporation
>> The Enterprise Postgres Company
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Virtualization & Cloud Management Using Capacity Planning
>> Cloud computing makes use of virtualization - but cloud computing
>> also focuses on allowing computing to be delivered as a service.
>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>

On Wed, Mar 7, 2012 at 11:19 AM, Koichi Suzuki <koi...@gm...> wrote:

> Yes, I understand.   Before committing this, I'd like to have a review
> of other members.
>
> BTW, a Chinese guy (I don't know his real name) wrote to me that we
> can redistribute rows by using CREATE TABLE AS or SELECT INTO and then
> dop the original one and rename the new one.   This can be very
> flexible.   If DBAs are not satisfied with the current node
> configuration, they can dump everything with pg_dump, reconfigure the
> nodes, pg_restore and then run CTA or SI.
>
>
ALTER TABLE REDISTRIBUTION was an item on my plate for this quarter, but
FQS took most of the time. Do you think, we should take that up? I am not
sure, whether that can be completed by end of March, esp. there is work
involved to send the rows back and forth between datanodes and coordinator.

The approach suggested above will change the OID of the relation, which
invalidates any view definitions, stored plans on the relation. If that's
not a concern the approach looks good.


> Regards;
> ----------
> Koichi Suzuki
>
>
>
> 2012/3/7 Andrei Martsinchyk <and...@gm...>:
> > Agree.
> > But now it does not output distribution type and distribution column.
> > The patch fixes this.
> >
> > 7 марта 2012 г. 3:38 пользователь Koichi Suzuki <
> ko...@in...>
> > написал:
> >
> >> It's nice that pg_dump doesn't dump node group because it is used to
> >> rebalance tables for different node configuration.     We may need some
> >> options on node-group in the future.
> >>
> >> Regards;
> >> ---
> >> Koichi
> >>
> >>  Tue, 6 Mar 2012 21:03:29 +0300
> >> Andrei Martsinchyk <and...@gm...> wrote:
> >>
> >> > Hi,
> >> >
> >> > I have noticed that pg_dump does not output table distribution info as
> >> > it
> >> > used to do.
> >> > It looks like there was an incorrect merge. Please find fix attached.
> >> >
> >> > --
> >> > Best regards,
> >> > Andrei Martsinchyk                   mailto:
> and...@gm...
> >
> >
> >
> >
> > --
> > Best regards,
> > Andrei Martsinchyk                   mailto:and...@gm...
> >
> >
> ------------------------------------------------------------------------------
> > Virtualization & Cloud Management Using Capacity Planning
> > Cloud computing makes use of virtualization - but cloud computing
> > also focuses on allowing computing to be delivered as a service.
> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
> > _______________________________________________
> > Postgres-xc-developers mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >
>
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>



-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

Yes, I understand.   Before committing this, I'd like to have a review
of other members.

BTW, a Chinese guy (I don't know his real name) wrote to me that we
can redistribute rows by using CREATE TABLE AS or SELECT INTO and then
dop the original one and rename the new one.   This can be very
flexible.   If DBAs are not satisfied with the current node
configuration, they can dump everything with pg_dump, reconfigure the
nodes, pg_restore and then run CTA or SI.

Regards;
----------
Koichi Suzuki



2012/3/7 Andrei Martsinchyk <and...@gm...>:
> Agree.
> But now it does not output distribution type and distribution column.
> The patch fixes this.
>
> 7 марта 2012 г. 3:38 пользователь Koichi Suzuki <ko...@in...>
> написал:
>
>> It's nice that pg_dump doesn't dump node group because it is used to
>> rebalance tables for different node configuration.     We may need some
>> options on node-group in the future.
>>
>> Regards;
>> ---
>> Koichi
>>
>>  Tue, 6 Mar 2012 21:03:29 +0300
>> Andrei Martsinchyk <and...@gm...> wrote:
>>
>> > Hi,
>> >
>> > I have noticed that pg_dump does not output table distribution info as
>> > it
>> > used to do.
>> > It looks like there was an incorrect merge. Please find fix attached.
>> >
>> > --
>> > Best regards,
>> > Andrei Martsinchyk                   mailto:and...@gm...
>
>
>
>
> --
> Best regards,
> Andrei Martsinchyk                   mailto:and...@gm...
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

Agree.
But now it does not output distribution type and distribution column.
The patch fixes this.

7 марта 2012 г. 3:38 пользователь Koichi Suzuki
<ko...@in...>написал:

> It's nice that pg_dump doesn't dump node group because it is used to
> rebalance tables for different node configuration.     We may need some
> options on node-group in the future.
>
> Regards;
> ---
> Koichi
>
>  Tue, 6 Mar 2012 21:03:29 +0300
> Andrei Martsinchyk <and...@gm...> wrote:
>
> > Hi,
> >
> > I have noticed that pg_dump does not output table distribution info as it
> > used to do.
> > It looks like there was an incorrect merge. Please find fix attached.
> >
> > --
> > Best regards,
> > Andrei Martsinchyk                   mailto:and...@gm...
>



-- 
Best regards,
Andrei Martsinchyk                   mailto:and...@gm...

Pavan,
You snatched words from my mouth :). I was drafting a similar proposal
yesterday but couldn't complete. I will drop it today.

On Wed, Mar 7, 2012 at 9:50 AM, Koichi Suzuki <koi...@gm...> wrote:

> +1, Indeed!
>
> We should be careful to find similar cases.
> ----------
> Koichi Suzuki
>
>
>
> 2012/3/7 Pavan Deolasee <pav...@en...>:
> > So after cribbing for it for several days, I ended up investigating
> > the super slowness of opr_sanity test. I turned the timing on in psql
> > and looked for queries that are taking very long. Its a join on a
> > catalog table thats taking a very long time compared to vanilla
> > PostgreSQL.
> >
> > Upon further investigation, I realized that these catalog join queries
> > are always using nested-loop join instead of far more efficient hash
> > or merge join. The following kludge in relnode.c is causing this:
> >
> >  100     switch (rte->rtekind)
> >  101     {
> >  102         case RTE_RELATION:
> >  103             /* Table --- retrieve statistics from the system
> catalogs */
> >  104             get_relation_info(root, rte->relid, rte->inh, rel);
> >  105 #ifdef PGXC
> >  106             /*
> >  107              * This is a remote table... we have no idea how
> > many pages/rows
> >  108              * we may get from a scan of this table. However, we
> > should set the
> >  109              * costs in such a manner that cheapest paths should
> > pick up the
> >  110              * ones involving these remote rels
> >  111              *
> >  112              * These allow for maximum query shipping to the remote
> >  113              * side later during the planning phase
> >  114              *
> >  115              * This has to be set on a remote Coordinator only
> >  116              * as it hugely penalizes performance on backend Nodes
> >  117              */
> >  118             if (IS_PGXC_COORDINATOR && !IsConnFromCoord())
> >  119             {
> >  120                 rel->pages   = 1;
> >  121                 rel->tuples  = 1;
> >  122                 rel->rows    = 1;
> >  123             }
> >  124 #endif
> >
> >
> > We override the relation size estimates at the coordinator. I think we
> > did this to enforce a nestloop join on the remote tables at the
> > coordinator because we have the ability to reduce only that kind of
> > join. But this can have very adverse effect on some other cases,
> > especially when the join is not reducible. We would be running a very
> > poor plan in those cases. This is demonstrated by the opr_sanity test
> > where the query is executed completely at the coordinator, but the
> > nest-loop join shows a very bad performance.
> >
> > The specific case of opr_sanity or catalog tables can be fixed easily
> > by using the real estimates for catalog tables. But I think a better
> > solution is needed (I and Ashutosh had discussed about this kludge
> > before privately). We probably need to hook into the path creation
> > logic and also create a path for remote join and pick the best path
> > based on the cost estimate. This will be a lot more complicated though
> > because we may need to estimate the cost of remote join to accurately
> > determine the best plan. But currently we don't have any statistics
> > about the remote tables at the coordinator. One way to circumvent the
> > problem is to run EXPLAIN ANALYZE first and use its result to
> > determine the cost of remote operations, but that can have adverse
> > effect especially when query execution time is very small.
> >
> > I will probably fix the catalog tables case for now. But lets discuss
> > a better long term solution.
> >
> > Thanks,
> > Pavan
> >
> > --
> > Pavan Deolasee
> > EnterpriseDB http://www.enterprisedb.com
> >
> >
> ------------------------------------------------------------------------------
> > Virtualization & Cloud Management Using Capacity Planning
> > Cloud computing makes use of virtualization - but cloud computing
> > also focuses on allowing computing to be delivered as a service.
> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
> > _______________________________________________
> > Postgres-xc-developers mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>



-- 
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company

+1, Indeed!

We should be careful to find similar cases.
----------
Koichi Suzuki



2012/3/7 Pavan Deolasee <pav...@en...>:
> So after cribbing for it for several days, I ended up investigating
> the super slowness of opr_sanity test. I turned the timing on in psql
> and looked for queries that are taking very long. Its a join on a
> catalog table thats taking a very long time compared to vanilla
> PostgreSQL.
>
> Upon further investigation, I realized that these catalog join queries
> are always using nested-loop join instead of far more efficient hash
> or merge join. The following kludge in relnode.c is causing this:
>
>  100     switch (rte->rtekind)
>  101     {
>  102         case RTE_RELATION:
>  103             /* Table --- retrieve statistics from the system catalogs */
>  104             get_relation_info(root, rte->relid, rte->inh, rel);
>  105 #ifdef PGXC
>  106             /*
>  107              * This is a remote table... we have no idea how
> many pages/rows
>  108              * we may get from a scan of this table. However, we
> should set the
>  109              * costs in such a manner that cheapest paths should
> pick up the
>  110              * ones involving these remote rels
>  111              *
>  112              * These allow for maximum query shipping to the remote
>  113              * side later during the planning phase
>  114              *
>  115              * This has to be set on a remote Coordinator only
>  116              * as it hugely penalizes performance on backend Nodes
>  117              */
>  118             if (IS_PGXC_COORDINATOR && !IsConnFromCoord())
>  119             {
>  120                 rel->pages   = 1;
>  121                 rel->tuples  = 1;
>  122                 rel->rows    = 1;
>  123             }
>  124 #endif
>
>
> We override the relation size estimates at the coordinator. I think we
> did this to enforce a nestloop join on the remote tables at the
> coordinator because we have the ability to reduce only that kind of
> join. But this can have very adverse effect on some other cases,
> especially when the join is not reducible. We would be running a very
> poor plan in those cases. This is demonstrated by the opr_sanity test
> where the query is executed completely at the coordinator, but the
> nest-loop join shows a very bad performance.
>
> The specific case of opr_sanity or catalog tables can be fixed easily
> by using the real estimates for catalog tables. But I think a better
> solution is needed (I and Ashutosh had discussed about this kludge
> before privately). We probably need to hook into the path creation
> logic and also create a path for remote join and pick the best path
> based on the cost estimate. This will be a lot more complicated though
> because we may need to estimate the cost of remote join to accurately
> determine the best plan. But currently we don't have any statistics
> about the remote tables at the coordinator. One way to circumvent the
> problem is to run EXPLAIN ANALYZE first and use its result to
> determine the cost of remote operations, but that can have adverse
> effect especially when query execution time is very small.
>
> I will probably fix the catalog tables case for now. But lets discuss
> a better long term solution.
>
> Thanks,
> Pavan
>
> --
> Pavan Deolasee
> EnterpriseDB http://www.enterprisedb.com
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers

So after cribbing for it for several days, I ended up investigating
the super slowness of opr_sanity test. I turned the timing on in psql
and looked for queries that are taking very long. Its a join on a
catalog table thats taking a very long time compared to vanilla
PostgreSQL.

Upon further investigation, I realized that these catalog join queries
are always using nested-loop join instead of far more efficient hash
or merge join. The following kludge in relnode.c is causing this:

 100     switch (rte->rtekind)
  101     {
  102         case RTE_RELATION:
  103             /* Table --- retrieve statistics from the system catalogs */
  104             get_relation_info(root, rte->relid, rte->inh, rel);
  105 #ifdef PGXC
  106             /*
  107              * This is a remote table... we have no idea how
many pages/rows
  108              * we may get from a scan of this table. However, we
should set the
  109              * costs in such a manner that cheapest paths should
pick up the
  110              * ones involving these remote rels
  111              *
  112              * These allow for maximum query shipping to the remote
  113              * side later during the planning phase
  114              *
  115              * This has to be set on a remote Coordinator only
  116              * as it hugely penalizes performance on backend Nodes
  117              */
  118             if (IS_PGXC_COORDINATOR && !IsConnFromCoord())
  119             {
  120                 rel->pages   = 1;
  121                 rel->tuples  = 1;
  122                 rel->rows    = 1;
  123             }
  124 #endif


We override the relation size estimates at the coordinator. I think we
did this to enforce a nestloop join on the remote tables at the
coordinator because we have the ability to reduce only that kind of
join. But this can have very adverse effect on some other cases,
especially when the join is not reducible. We would be running a very
poor plan in those cases. This is demonstrated by the opr_sanity test
where the query is executed completely at the coordinator, but the
nest-loop join shows a very bad performance.

The specific case of opr_sanity or catalog tables can be fixed easily
by using the real estimates for catalog tables. But I think a better
solution is needed (I and Ashutosh had discussed about this kludge
before privately). We probably need to hook into the path creation
logic and also create a path for remote join and pick the best path
based on the cost estimate. This will be a lot more complicated though
because we may need to estimate the cost of remote join to accurately
determine the best plan. But currently we don't have any statistics
about the remote tables at the coordinator. One way to circumvent the
problem is to run EXPLAIN ANALYZE first and use its result to
determine the cost of remote operations, but that can have adverse
effect especially when query execution time is very small.

I will probably fix the catalog tables case for now. But lets discuss
a better long term solution.

Thanks,
Pavan

-- 
Pavan Deolasee
EnterpriseDB http://www.enterprisedb.com

It's nice that pg_dump doesn't dump node group because it is used to rebalance tables for different node configuration.     We may need some options on node-group in the future.

Regards;
---
Koichi

 Tue, 6 Mar 2012 21:03:29 +0300
Andrei Martsinchyk <and...@gm...> wrote:

> Hi,
> 
> I have noticed that pg_dump does not output table distribution info as it
> used to do.
> It looks like there was an incorrect merge. Please find fix attached.
> 
> -- 
> Best regards,
> Andrei Martsinchyk                   mailto:and...@gm...