postgres-xc-developers Mailing List for Postgres-XC (Page 139)

Hi All,

Here is a first set of assertion failure fix. With this, I am now able to
connect to the PGXC fine. But haven't tried anything more than that yet.
Still I thought its important to get these fixes in so that others can look
for other issues.

Please review it carefully since it involves a change about setting
RecentGlobalXmin/RecentXmin and we had few issues around this in the past.
But I don't see why it should not be set for coordinator. In fact,
RecentGlobalXmin must be set to a valid value before we do any operation.

Let me know comments/objections.

Thanks,
Pavan

-- 
Pavan Deolasee
EnterpriseDB     http://www.enterprisedb.com

2010/5/13 Michael Paquier <mic...@gm...>: > Sorry to interrupt
you,
> but this issue is a postgres-XC one > so is it really OK to continue this
conversation on this mailing list? > I understand and agree. > I think it
should be moved to the postgres-xc developer mailing instead. > Thanks. >
Moving the thread to XC mailing list. Thanks, Pavan -- Pavan Deolasee
EnterpriseDB http://www.enterprisedb.com
-- 
Pavan Deolasee
EnterpriseDB     http://www.enterprisedb.com

Hi,

I'm beginning to write a reference manual for the new feature.   I
need the following info for this.

1. COPY TO/ COPY FROM statement definition,
2. pg_aggregate extension (columns, types and descriptions)
3. Any other changes to the catalog.
4. Supported aggregate function list.

User function release is scheduled at the end of June so it will not
be included yet.

For the information of catalog, you may send me .h files.

Also, I think we can add "aggregate function" algorithm to the
release.   Would like to have a comment on this.

Regards;
----------
Koichi Suzuki

Hi Mason-san;

Thanks a lot for the work.  I'd like to summarize what we can and what
we cannot with PG-XC and pgadmin.
----------
Koichi Suzuki



2010/5/13 Mason Sharp <mas...@en...>:
> Pavan Deolasee wrote:
>>
>>
>> On Wed, May 12, 2010 at 3:19 AM, Mason Sharp
>> <mas...@en... <mailto:mas...@en...>>
>> wrote:
>>
>>     I started the task to recognize more "Postgres-XC safe" queries.
>>
>>     I initially focused on native pg_catalog based views like
>>     pg_settings. This is because it used by pgadmin. With the attached
>>     patch, pgadmin no longer complains about such queries because they
>>     are now supported.
>>
>>     The PostgreSQL query rewriter appears to convert the view into a
>>     subquery. I modified the XC planner to check for pg_catalog table
>>     usage in the FROM clause (and subqueries thereof).
>>
>>
>> FWIW here are couple of things that I noticed:
>>
>> +   foreach(item, rtable)
>> +   {
>> +       RangeTblEntry *rte = (RangeTblEntry *) lfirst(item);
>> +
>> +       if (rte->rtekind == RTE_RELATION)
>> +       {
>> +           if (rte->relid > FirstNormalObjectId)
>> +               return false;
>> +       } else if (rte->rtekind == RTE_SUBQUERY)
>> +           return contains_only_pg_catalog (rte->subquery->rtable);
>> +   }
>>
>> This code will break out of the loop if the subquery contains only
>> pg_catalog tables. But other rtable entries may contain reference to
>> non-system tables.
>>
>> The other thing I am worried about is whether its safe to just rely on
>> FirstNormalObjectId to decide if a relation belongs to pg_catalog or
>> not. A user can create a table in pg_catalog (though I think it
>> requires changes to GUC) and that may get an OID larger than the
>> FirstNormalObjectId or we may add a new default schema and create
>> tables in the schema as part of initdb (we do that routinely in EDBAS
>> for example). Should we just explicitly lookup the namespace of the
>> relation instead ?
>>
> Thanks, I made those changes and committed.
>
> Mason
>
>> Thanks,
>> Pavan
>>
>> --
>> Pavan Deolasee
>> EnterpriseDB     http://www.enterprisedb.com
>
>
> --
> Mason Sharp
> EnterpriseDB Corporation
> The Enterprise Postgres Company
>
>
> This e-mail message (and any attachment) is intended for the use of
> the individual or entity to whom it is addressed. This message
> contains information from EnterpriseDB Corporation that may be
> privileged, confidential, or exempt from disclosure under applicable
> law. If you are not the intended recipient or authorized to receive
> this for the intended recipient, any use, dissemination, distribution,
> retention, archiving, or copying of this communication is strictly
> prohibited. If you have received this e-mail in error, please notify
> the sender immediately by reply e-mail and delete this message.
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

Pavan Deolasee wrote:
>
>
> On Wed, May 12, 2010 at 3:19 AM, Mason Sharp 
> <mas...@en... <mailto:mas...@en...>> 
> wrote:
>
>     I started the task to recognize more "Postgres-XC safe" queries.
>
>     I initially focused on native pg_catalog based views like
>     pg_settings. This is because it used by pgadmin. With the attached
>     patch, pgadmin no longer complains about such queries because they
>     are now supported.
>
>     The PostgreSQL query rewriter appears to convert the view into a
>     subquery. I modified the XC planner to check for pg_catalog table
>     usage in the FROM clause (and subqueries thereof).
>
>
> FWIW here are couple of things that I noticed:
>
> +   foreach(item, rtable) 
> +   {
> +       RangeTblEntry *rte = (RangeTblEntry *) lfirst(item);
> +
> +       if (rte->rtekind == RTE_RELATION)
> +       {
> +           if (rte->relid > FirstNormalObjectId)
> +               return false;
> +       } else if (rte->rtekind == RTE_SUBQUERY)
> +           return contains_only_pg_catalog (rte->subquery->rtable);
> +   }
>
> This code will break out of the loop if the subquery contains only 
> pg_catalog tables. But other rtable entries may contain reference to 
> non-system tables.
>
> The other thing I am worried about is whether its safe to just rely on 
> FirstNormalObjectId to decide if a relation belongs to pg_catalog or 
> not. A user can create a table in pg_catalog (though I think it 
> requires changes to GUC) and that may get an OID larger than the 
> FirstNormalObjectId or we may add a new default schema and create 
> tables in the schema as part of initdb (we do that routinely in EDBAS 
> for example). Should we just explicitly lookup the namespace of the 
> relation instead ?
>
Thanks, I made those changes and committed.

Mason

> Thanks,
> Pavan
>  
> -- 
> Pavan Deolasee
> EnterpriseDB     http://www.enterprisedb.com


-- 
Mason Sharp
EnterpriseDB Corporation
The Enterprise Postgres Company


This e-mail message (and any attachment) is intended for the use of
the individual or entity to whom it is addressed. This message
contains information from EnterpriseDB Corporation that may be
privileged, confidential, or exempt from disclosure under applicable
law. If you are not the intended recipient or authorized to receive
this for the intended recipient, any use, dissemination, distribution,
retention, archiving, or copying of this communication is strictly
prohibited. If you have received this e-mail in error, please notify
the sender immediately by reply e-mail and delete this message.

On Wed, May 12, 2010 at 3:19 AM, Mason Sharp
<mas...@en...>wrote:

> I started the task to recognize more "Postgres-XC safe" queries.
>
> I initially focused on native pg_catalog based views like pg_settings. This
> is because it used by pgadmin. With the attached patch, pgadmin no longer
> complains about such queries because they are now supported.
>
> The PostgreSQL query rewriter appears to convert the view into a subquery.
> I modified the XC planner to check for pg_catalog table usage in the FROM
> clause (and subqueries thereof).
>
>
FWIW here are couple of things that I noticed:

+   foreach(item, rtable)
+   {
+       RangeTblEntry *rte = (RangeTblEntry *) lfirst(item);
+
+       if (rte->rtekind == RTE_RELATION)
+       {
+           if (rte->relid > FirstNormalObjectId)
+               return false;
+       } else if (rte->rtekind == RTE_SUBQUERY)
+           return contains_only_pg_catalog (rte->subquery->rtable);
+   }

This code will break out of the loop if the subquery contains only
pg_catalog tables. But other rtable entries may contain reference to
non-system tables.

The other thing I am worried about is whether its safe to just rely on
FirstNormalObjectId to decide if a relation belongs to pg_catalog or not. A
user can create a table in pg_catalog (though I think it requires changes to
GUC) and that may get an OID larger than the FirstNormalObjectId or we may
add a new default schema and create tables in the schema as part of initdb
(we do that routinely in EDBAS for example). Should we just explicitly
lookup the namespace of the relation instead ?

Thanks,
Pavan

-- 
Pavan Deolasee
EnterpriseDB     http://www.enterprisedb.com

I started the task to recognize more "Postgres-XC safe" queries.

I initially focused on native pg_catalog based views like pg_settings. 
This is because it used by pgadmin. With the attached patch, pgadmin no 
longer complains about such queries because they are now supported.

The PostgreSQL query rewriter appears to convert the view into a 
subquery. I modified the XC planner to check for pg_catalog table usage 
in the FROM clause (and subqueries thereof).

In addition, pgadmin was sending "SET client_encoding to 'UNICODE'". It 
was being swallowed and proxied by Postgres-XC, but no response was sent 
back because it did not recognize the 'S' message response. I have now 
added support for that, too.

Note that we have planned a whole separate task planned for proper SET 
handling. These commands will now be processed, but the user should not 
assume that these will remain set, due to Postgres-XC pooler. Again, we 
will address this in the future.

With this patch pgadmin runs much more smoothly when connected to 
Postgres-XC. Users can view schema information and issue SQL 
statements.  There is still an error window that pops up when clicking 
on a table because it issues a "SELECT count(*) FROM table", which is 
not yet supported in Postgres-XC. This problem will disappear soon 
however, as a patch is almost ready for basic aggregate support.

Thanks,

Mason


-- 
Mason Sharp
EnterpriseDB Corporation
The Enterprise Postgres Company


This e-mail message (and any attachment) is intended for the use of
the individual or entity to whom it is addressed. This message
contains information from EnterpriseDB Corporation that may be
privileged, confidential, or exempt from disclosure under applicable
law. If you are not the intended recipient or authorized to receive
this for the intended recipient, any use, dissemination, distribution,
retention, archiving, or copying of this communication is strictly
prohibited. If you have received this e-mail in error, please notify
the sender immediately by reply e-mail and delete this message.

Koichi Suzuki wrote:
> Hello, Koichi Suzuki,
>
> we downloaded pgxc_v0.9.tar.gz file from
> https://sourceforge.net/projects/postgres-xc/ and tried to install
> Postgres-XC on our database test server:
>
> $ uname -a
> Linux dbtest 2.6.26-2-amd64 #1 SMP Thu Nov 5 02:23:12 UTC 2009 x86_64 GNU/Linux
> $ gcc --version
> gcc (Debian 4.3.2-1.1) 4.3.2
> make command ran into the same errors like posted from Devrim Gunduz
> on http://archives.postgresql.org/pgsql-cluster-hackers/2010-03/msg00057.php.
>
> Is there a new version, where these bugs are fixed?
>
>   

If you grab the latest code from git, you should get past this issue, 
but Devrim reported a new one after that.  We have not looked into it 
yet however. We will try and do it soon.

Regards,

Mason



> Best regards,
>
> Rainer Kirsten
> Database Administrator
>
> University of Potsdam
> Zentrale Einrichtung fuer Informationsverarbeitung
> und Kommunikation (ZEIK)
> Am Neuen Palais 10
> 14469 Potsdam
> GERMANY
> Fon ++49-(0)331-977-1392
> Fax ++49-(0)331-977-1750
> Eml ki...@un...
>   


-- 
Mason Sharp
EnterpriseDB Corporation
The Enterprise Postgres Company


This e-mail message (and any attachment) is intended for the use of
the individual or entity to whom it is addressed. This message
contains information from EnterpriseDB Corporation that may be
privileged, confidential, or exempt from disclosure under applicable
law. If you are not the intended recipient or authorized to receive
this for the intended recipient, any use, dissemination, distribution,
retention, archiving, or copying of this communication is strictly
prohibited. If you have received this e-mail in error, please notify
the sender immediately by reply e-mail and delete this message.

On Fri, 2010-04-23 at 16:42 +0900, Koichi Suzuki wrote:
>   Could anybody follow Rainer's problem?

On 32-bit machines, applying this patch will work:

http://svn.pgrpms.org/browser/rpm/redhat/8.4/postgres-xc/F-12/postgres-xc-09-build.patch

1st part of the patch is Mason's patch in git. 2nd part is my patch, but
as I wrote before, it may be dirty, so comments are welcome.

...and yes, I built initial RPM packages. They are under QA process, and
I expect to announce first set of packages in the next 2 weeks.

Regards,
-- 
Devrim GÜNDÜZ
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
PostgreSQL RPM Repository: http://yum.pgrpms.org
Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org  Twitter: http://twitter.com/devrimgunduz

this is a test
----------
Koichi Suzuki

This is a test.
----------
Koichi Suzuki

Hi,

Here's another error in the build.   I've not followed Devrim's
problem in detail.   Could anybody follow Rainer's problem?
----------
Koichi Suzuki




---------- Forwarded message ----------
From: Rainer Kirsten <ki...@un...>
Date: 2010/4/23
Subject: Postgres-XC 0.9 - Errors while installing
To: koi...@gm...
Cc: Marko Jaskiewicz <ja...@un...>


Hello, Koichi Suzuki,

we downloaded pgxc_v0.9.tar.gz file from
https://sourceforge.net/projects/postgres-xc/ and tried to install
Postgres-XC on our database test server:

$ uname -a
Linux dbtest 2.6.26-2-amd64 #1 SMP Thu Nov 5 02:23:12 UTC 2009 x86_64 GNU/Linux
$ gcc --version
gcc (Debian 4.3.2-1.1) 4.3.2
make command ran into the same errors like posted from Devrim Gunduz
on http://archives.postgresql.org/pgsql-cluster-hackers/2010-03/msg00057.php.

Is there a new version, where these bugs are fixed?

Best regards,

Rainer Kirsten
Database Administrator

University of Potsdam
Zentrale Einrichtung fuer Informationsverarbeitung
und Kommunikation (ZEIK)
Am Neuen Palais 10
14469 Potsdam
GERMANY
Fon ++49-(0)331-977-1392
Fax ++49-(0)331-977-1750
Eml ki...@un...

On Wed, 2010-04-21 at 09:19 +0900, Mason Sharp wrote:
> At this point, the regression tests are  probably not very useful
> because of the amount of SQL that simply is not yet supported. Still,
> we should make sure that these can be launched. 

Thanks for the info. I disabled regression checks in spec file.

Regards,
-- 
Devrim GÜNDÜZ
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
PostgreSQL RPM Repository: http://yum.pgrpms.org
Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org  Twitter: http://twitter.com/devrimgunduz

Hi Devrim,

Thanks for pointing this out.  At this point, the regression tests are 
probably not very useful because of the amount of SQL that simply is not 
yet supported. Still, we should make sure that these can be launched.

I added this to the SF bug tracker.

Thanks,

Mason




Devrim GÜNDÜZ wrote:
> I'm seeing this in postmaster.log :
>
> postgres: PG-XC: must start as either a Coordinator (-C) or Data Node
> (-X)
>
> Regards,
>   
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> ------------------------------------------------------------------------
>
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>   


-- 
Mason Sharp
EnterpriseDB Corporation
The Enterprise Postgres Company


This e-mail message (and any attachment) is intended for the use of
the individual or entity to whom it is addressed. This message
contains information from EnterpriseDB Corporation that may be
privileged, confidential, or exempt from disclosure under applicable
law. If you are not the intended recipient or authorized to receive
this for the intended recipient, any use, dissemination, distribution,
retention, archiving, or copying of this communication is strictly
prohibited. If you have received this e-mail in error, please notify
the sender immediately by reply e-mail and delete this message.

I'm seeing this in postmaster.log :

postgres: PG-XC: must start as either a Coordinator (-C) or Data Node
(-X)

Regards,
-- 
Devrim GÜNDÜZ
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
PostgreSQL RPM Repository: http://yum.pgrpms.org
Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org  Twitter: http://twitter.com/devrimgunduz

On Tue, 2010-04-20 at 16:31 +0300, Devrim GÜNDÜZ wrote:
> /usr/bin/ld: fe-auth.o: relocation R_X86_64_32 against `.rodata' can
> not be used when making a shared object; recompile with -fPIC
> fe-auth.o: could not read symbols: Bad value
> collect2: ld returned 1 exit status 

Of course, this is x86_64. It does not happen in x86.
-- 
Devrim GÜNDÜZ
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
PostgreSQL RPM Repository: http://yum.pgrpms.org
Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org  Twitter: http://twitter.com/devrimgunduz

Hi again,

I applied 48fb54b318a3efd41fae3d97c5baf5c35190d9e2 and attached patch to
0.9 tarball (attached patch may be a dirty solution, feel free to fix
it. I'm not a C guy ). and I'm getting this error:

gcc -DPGXC -I/usr/include/et -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing -fwrapv  -fpic -shared -Wl,-soname,libpq.so.5 -Wl,--version-script=exports.list  fe-auth.o fe-connect.o fe-exec.o fe-misc.o fe-print.o fe-lobj.o fe-protocol2.o fe-protocol3.o pqexpbuffer.o pqsignal.o fe-secure.o libpq-events.o md5.o ip.o wchar.o encnames.o noblock.o pgstrcasecmp.o thread.o strlcpy.o  -L../../../src/port -L/usr/lib64 -lssl -lcrypto -lkrb5 -lcom_err -lgssapi_krb5 -lcrypt -lldap -o libpq.so.5.2
/usr/bin/ld: fe-auth.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
fe-auth.o: could not read symbols: Bad value
collect2: ld returned 1 exit status
make[4]: *** [libpq.so.5.2] Error 1
make[4]: Leaving directory `/home/devrim/belgeler/postgresqlrpms.org/svn/rpm/redhat/8.4/postgres-xc/F-12/pgxc_v0.9/src/interfaces/libpq'
make[3]: *** [../../src/interfaces/libpq-recursive] Error 2

What should we do in here?

Regards,
-- 
Devrim GÜNDÜZ
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
PostgreSQL RPM Repository: http://yum.pgrpms.org
Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org  Twitter: http://twitter.com/devrimgunduz