postgres-xc-developers Mailing List for Postgres-XC

Hi,

because struct pgxc_node_handle is not a kind of Node, we could not use
function list_member, list_delete.

Patch attached.

On Thu, Jul 24, 2014 at 9:38 PM, 鈴木 幸市 <ko...@in...> wrote:

>  Background is to make configuration as simple as possible and to make
> promotion simpler.    XC primitive nodes support different port for master
> and slave, as in vanilla PG.
>
>  When promoting slave with the same port number, VIP will help to connect
> to the node without change in the target port/host.   VIP cannot do this if
> port number is different.   I understand this helps only for coordinator.
> I think it’s also better to make coordinator/datanode configuration as same
> as possible.
>

I had assumed this to be the reason. It makes sense to do as the default. I
think it would be a nice option to be able to configure differently for
development and debugging.

I added this to Postgres-XL as a ticket with a low priority, since I would
like to see it at least get in there eventually, to optionally override.

https://sourceforge.net/p/postgres-xl/tickets/25/



>
>  Regards;
> ---
> Koichi Suzuki
>
>  2014/07/24 21:08、Pavan Deolasee <pav...@gm...> のメール：
>
>  I noticed that we can't specify port number for datanode slave nodes
> that are different from the master nodes. Is it that way for a good reason?
> I'm asking before if I need to test this feature on a single node, I would
> like to run master and slave on the same machine. But since they use the
> same port number, its not possible to do that. Should we consider
> supporting different ports for datanode (and coordinator slaves) just like
> we do for GTM slave?
>
>  Thanks,
> Pavan
>
>  --
> Pavan Deolasee
> http://www.linkedin.com/in/pavandeolasee
>
> ------------------------------------------------------------------------------
> Want fast and easy access to all the code in your enterprise? Index and
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest code
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/sfu/bds_______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>
>
>
> ------------------------------------------------------------------------------
> Want fast and easy access to all the code in your enterprise? Index and
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest code
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/sfu/bds
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>


-- 
Mason Sharp

TransLattice - http://www.translattice.com
Distributed and Clustered Database Solutions

Background is to make configuration as simple as possible and to make promotion simpler.    XC primitive nodes support different port for master and slave, as in vanilla PG.

When promoting slave with the same port number, VIP will help to connect to the node without change in the target port/host.   VIP cannot do this if port number is different.   I understand this helps only for coordinator.   I think it’s also better to make coordinator/datanode configuration as same as possible.

Regards;
---
Koichi Suzuki

2014/07/24 21:08、Pavan Deolasee <pav...@gm...<mailto:pav...@gm...>> のメール：

I noticed that we can't specify port number for datanode slave nodes that are different from the master nodes. Is it that way for a good reason? I'm asking before if I need to test this feature on a single node, I would like to run master and slave on the same machine. But since they use the same port number, its not possible to do that. Should we consider supporting different ports for datanode (and coordinator slaves) just like we do for GTM slave?

Thanks,
Pavan

--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee
------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds_______________________________________________
Postgres-xc-developers mailing list
Pos...@li...
https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers

Thanks Pavan;

The patch has been committed to all the relevant branches.
---
Koichi Suzuki

2014/07/24 20:15、Pavan Deolasee <pav...@gm...<mailto:pav...@gm...>> のメール：

On Thu, Jul 24, 2014 at 11:25 AM, Koichi Suzuki <koi...@gm...<mailto:koi...@gm...>> wrote:

Thanks a lot for pointing out.   Could you submit a patch for this?

Sure. Attached.

Thanks,
Pavan

--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee
<xc_use_pclose.patch>------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds_______________________________________________
Postgres-xc-developers mailing list
Pos...@li...
https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers

I noticed that we can't specify port number for datanode slave nodes that
are different from the master nodes. Is it that way for a good reason? I'm
asking before if I need to test this feature on a single node, I would like
to run master and slave on the same machine. But since they use the same
port number, its not possible to do that. Should we consider supporting
different ports for datanode (and coordinator slaves) just like we do for
GTM slave?

Thanks,
Pavan

-- 
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee

Thanks a lot for pointing out.   Could you submit a patch for this?
2014/07/24 13:09 "Pavan Deolasee" <pav...@gm...>:

> Hello All,
>
> AFAICS that pgxc_ctl uses popen() at many places to execute a command and
> sometimes to also parse back the output. I've been chasing a bug that would
> cause subsequent reads on the FILE * returned by popen() to return EOF.
> After some investigations, it seems that the correct way to close the file
> descriptor returned by popen() is to use pclose() which fclose() does not
> reset the pipe state properly whereas pclose() does. The current XC code
> uses fclose() instead which probably works fine on some platforms such as
> Linux, but fails on other such as Mac OSX.
>
> Should we consider replacing fclose() with pclose() for pipes opened by
> popen()?
>
> Thanks,
> Pavan
>
> --
> Pavan Deolasee
> http://www.linkedin.com/in/pavandeolasee
>
>
> ------------------------------------------------------------------------------
> Want fast and easy access to all the code in your enterprise? Index and
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest code
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/sfu/bds
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
>

Hello All,

AFAICS that pgxc_ctl uses popen() at many places to execute a command and
sometimes to also parse back the output. I've been chasing a bug that would
cause subsequent reads on the FILE * returned by popen() to return EOF.
After some investigations, it seems that the correct way to close the file
descriptor returned by popen() is to use pclose() which fclose() does not
reset the pipe state properly whereas pclose() does. The current XC code
uses fclose() instead which probably works fine on some platforms such as
Linux, but fails on other such as Mac OSX.

Should we consider replacing fclose() with pclose() for pipes opened by
popen()?

Thanks,
Pavan

-- 
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee

My apologies. It was meant for XL list.

Thanks,
Pavan


On Fri, Jul 18, 2014 at 8:32 PM, Koichi Suzuki <koi...@gm...>
wrote:

> The patch looks specific to XL and I don' t understand the directive
> #ifdef XCP in bash script.   Could I have more explanation on this?
> ---
> Koichi Suzuki
>
>
> 2014-07-18 15:04 GMT+09:00 Pavan Deolasee <pav...@gm...>:
> > Hello,
> >
> > AFAICS contrib/pgxc_ctl/pgxc_ctl_bash.c is auto generated from dependent
> > files such as pgxc_ctl_bash_2. We have one version of pgxc_ctl_bash.c
> > checked in the repository which contains logic to read the datanode
> pooler
> > ports information correctly. But if the file gets overwritten during
> build
> > process, we lose those changes. How about applying the attached patch?
> >
> > Thanks,
> > Pavan
> >
> > --
> > Pavan Deolasee
> > http://www.linkedin.com/in/pavandeolasee
> >
> >
> ------------------------------------------------------------------------------
> > Want fast and easy access to all the code in your enterprise? Index and
> > search up to 200,000 lines of code with a free copy of Black Duck
> > Code Sight - the same software that powers the world's largest code
> > search on Ohloh, the Black Duck Open Hub! Try it now.
> > http://p.sf.net/sfu/bds
> > _______________________________________________
> > Postgres-xc-developers mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >
>



-- 
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee

The patch looks specific to XL and I don' t understand the directive
#ifdef XCP in bash script.   Could I have more explanation on this?
---
Koichi Suzuki


2014-07-18 15:04 GMT+09:00 Pavan Deolasee <pav...@gm...>:
> Hello,
>
> AFAICS contrib/pgxc_ctl/pgxc_ctl_bash.c is auto generated from dependent
> files such as pgxc_ctl_bash_2. We have one version of pgxc_ctl_bash.c
> checked in the repository which contains logic to read the datanode pooler
> ports information correctly. But if the file gets overwritten during build
> process, we lose those changes. How about applying the attached patch?
>
> Thanks,
> Pavan
>
> --
> Pavan Deolasee
> http://www.linkedin.com/in/pavandeolasee
>
> ------------------------------------------------------------------------------
> Want fast and easy access to all the code in your enterprise? Index and
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest code
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/sfu/bds
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

On Fri, Jul 18, 2014 at 3:26 PM, Pavan Deolasee
<pav...@gm...> wrote:
> I think its a bad idea to use killall to terminate existing processes. I
> noticed this for GTM, but the  same may apply to other kinds of processes
> too. In my case, I was setting up GTM master and standby on the same
> machine. pgxc_ctl was failing to start up GTM correctly. Upon investigation,
> I figured out that while initialising/starting GTM slave, it first kills any
> existing GTM processes using "killall gtm". This kills the GTM master as
> well.
>
> There are several issues with using killall
>
> 1. It may kill unrelated processes e.g. I'd a vim session on gtm_cmd.c and
> it got killed too
That's actually kind of funny, just laughed lonely in front of my screen.

> 2. It may kill processes belonging to another instance, if user is running
> multiple database clusters on the same machine
This one is less funny :(

> 3. killall -9 It doesn't do a clean shutdown which may cause issues
> I wonder why can't we just rely on -m immediate? If the process fails to
> respond to that, either we have a bug to fix or worst administrator can
> manually kill the processes he wants. Even if we have to use kill command,
> can we not read PID from the .pid file and send signal to that particular
> process only?
Clearly. Partially written WAL record is an example of a bad thing
that could happen for Postgres. For GTM there are for sure similar
race conditions with a standby. The same applies to Postgres as well.
The usual, recommended, way is to use pg_ctl stop -m immediate in the
worst case. Isn't XC bundled with gtm_ctl that could be used instead
of killall? That's the same as sending a SIGQUIT after scanning the
GTM PID though..

Regards,
-- 
Michael

Hello,

I think its a bad idea to use killall to terminate existing processes. I
noticed this for GTM, but the  same may apply to other kinds of processes
too. In my case, I was setting up GTM master and standby on the same
machine. pgxc_ctl was failing to start up GTM correctly. Upon
investigation, I figured out that while initialising/starting GTM slave, it
first kills any existing GTM processes using "killall gtm". This kills the
GTM master as well.

There are several issues with using killall

1. It may kill unrelated processes e.g. I'd a vim session on gtm_cmd.c and
it got killed too
2. It may kill processes belonging to another instance, if user is running
multiple database clusters on the same machine
3. killall -9 It doesn't do a clean shutdown which may cause issues

I wonder why can't we just rely on -m immediate? If the process fails to
respond to that, either we have a bug to fix or worst administrator can
manually kill the processes he wants. Even if we have to use kill command,
can we not read PID from the .pid file and send signal to that particular
process only?

Thanks,
Pavan

-- 
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee

I applied the fix to 1.1, 1.2 and master.  1.0 does not accept this fix.

Thank you ;
---
Koichi Suzuki


2014-07-02 11:24 GMT+09:00 Koichi Suzuki <koi...@gm...>:
> Thank you Mark for the fix.
>
> It looks reasonable and should go to all the versions.   I'll commit it soon.
> ---
> Koichi Suzuki
>
>
> 2014-07-01 23:05 GMT+09:00 Mark, Stefan <Ste...@se...>:
>> Hallo,
>>
>> there's a problem in postgres-xc v1.1 (and maybe also in other versions) that data that contains "\" and column separator is not replicated correctly to newly added data nodes. After some investigation we figured out that "\" and column separator are not quoted. The appended patch fixes this problem.
>>
>> Regards
>> Stefan
>>
>> --
>> Dipl. Inf.
>> Stefan Mark
>> Consultant
>> Public Sector
>> secunet Security Networks AG
>>
>>
>> Phone: +49-201-5454-3583, Fax: +49-201-5454-1323
>> E-Mail: ste...@se...
>> Ammonstraße 64, 01067 Dresden, Germany
>> www.secunet.com
>>
>> ______________________________________________________________________
>>
>> Registered at: Kronprinzenstraße 30, 45128 Essen, Deutschland
>> Amtsgericht Essen HRB 13615
>> Management Board: Dr Rainer Baumgart (CEO), Willem Bulthuis, Thomas Pleines
>> Chairman of Supervisory Board: Dr Peter Zattler
>> ______________________________________________________________________
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Open source business process management suite built on Java and Eclipse
>> Turn processes into business applications with Bonita BPM Community Edition
>> Quickly connect people, data, and systems into organized workflows
>> Winner of BOSSIE, CODIE, OW2 and Gartner awards
>> http://p.sf.net/sfu/Bonitasoft
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>

Thank you Mark for the fix.

It looks reasonable and should go to all the versions.   I'll commit it soon.
---
Koichi Suzuki


2014-07-01 23:05 GMT+09:00 Mark, Stefan <Ste...@se...>:
> Hallo,
>
> there's a problem in postgres-xc v1.1 (and maybe also in other versions) that data that contains "\" and column separator is not replicated correctly to newly added data nodes. After some investigation we figured out that "\" and column separator are not quoted. The appended patch fixes this problem.
>
> Regards
> Stefan
>
> --
> Dipl. Inf.
> Stefan Mark
> Consultant
> Public Sector
> secunet Security Networks AG
>
>
> Phone: +49-201-5454-3583, Fax: +49-201-5454-1323
> E-Mail: ste...@se...
> Ammonstraße 64, 01067 Dresden, Germany
> www.secunet.com
>
> ______________________________________________________________________
>
> Registered at: Kronprinzenstraße 30, 45128 Essen, Deutschland
> Amtsgericht Essen HRB 13615
> Management Board: Dr Rainer Baumgart (CEO), Willem Bulthuis, Thomas Pleines
> Chairman of Supervisory Board: Dr Peter Zattler
> ______________________________________________________________________
>
>
>
> ------------------------------------------------------------------------------
> Open source business process management suite built on Java and Eclipse
> Turn processes into business applications with Bonita BPM Community Edition
> Quickly connect people, data, and systems into organized workflows
> Winner of BOSSIE, CODIE, OW2 and Gartner awards
> http://p.sf.net/sfu/Bonitasoft
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>

Hallo,

there's a problem in postgres-xc v1.1 (and maybe also in other versions) that data that contains "\" and column separator is not replicated correctly to newly added data nodes. After some investigation we figured out that "\" and column separator are not quoted. The appended patch fixes this problem.

Regards
Stefan

-- 
Dipl. Inf.
Stefan Mark
Consultant
Public Sector
secunet Security Networks AG 


Phone: +49-201-5454-3583, Fax: +49-201-5454-1323
E-Mail: ste...@se...
Ammonstraße 64, 01067 Dresden, Germany
www.secunet.com

______________________________________________________________________

Registered at: Kronprinzenstraße 30, 45128 Essen, Deutschland 
Amtsgericht Essen HRB 13615
Management Board: Dr Rainer Baumgart (CEO), Willem Bulthuis, Thomas Pleines 
Chairman of Supervisory Board: Dr Peter Zattler
______________________________________________________________________