Menu
▾
▴
postgres-xc-developers — Postgres-XC hackers and developers
You can subscribe to this list here.
| 2010 |
Jan
|
Feb
|
Mar
|
Apr
(10) |
May
(17) |
Jun
(3) |
Jul
|
Aug
|
Sep
(8) |
Oct
(18) |
Nov
(51) |
Dec
(74) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2011 |
Jan
(47) |
Feb
(44) |
Mar
(44) |
Apr
(102) |
May
(35) |
Jun
(25) |
Jul
(56) |
Aug
(69) |
Sep
(32) |
Oct
(37) |
Nov
(31) |
Dec
(16) |
| 2012 |
Jan
(34) |
Feb
(127) |
Mar
(218) |
Apr
(252) |
May
(80) |
Jun
(137) |
Jul
(205) |
Aug
(159) |
Sep
(35) |
Oct
(50) |
Nov
(82) |
Dec
(52) |
| 2013 |
Jan
(107) |
Feb
(159) |
Mar
(118) |
Apr
(163) |
May
(151) |
Jun
(89) |
Jul
(106) |
Aug
(177) |
Sep
(49) |
Oct
(63) |
Nov
(46) |
Dec
(7) |
| 2014 |
Jan
(65) |
Feb
(128) |
Mar
(40) |
Apr
(11) |
May
(4) |
Jun
(8) |
Jul
(16) |
Aug
(11) |
Sep
(4) |
Oct
(1) |
Nov
(5) |
Dec
(16) |
| 2015 |
Jan
(5) |
Feb
|
Mar
(2) |
Apr
(5) |
May
(4) |
Jun
(12) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(4) |
| 2019 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(2) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| S | M | T | W | T | F | S |
|---|---|---|---|---|---|---|
|
|
|
|
|
|
1
|
2
|
|
3
|
4
|
5
(1) |
6
(1) |
7
(2) |
8
(3) |
9
|
|
10
|
11
(2) |
12
(6) |
13
|
14
(4) |
15
(6) |
16
(1) |
|
17
|
18
|
19
(5) |
20
(2) |
21
(3) |
22
(1) |
23
(4) |
|
24
(1) |
25
(5) |
26
(5) |
27
(2) |
28
|
29
(1) |
30
|
|
31
(1) |
|
|
|
|
|
|
|
From: Ashutosh B. <ash...@en...> - 2011-07-31 05:17:44
|
Hi Mason, Sorry for delay in replying. I was busy yesterday whole day. On Fri, Jul 29, 2011 at 6:36 PM, Michael Paquier <mic...@gm...>wrote: > > Great. Is this a general solution for multi-step queries, too? >> > No. I don't think so. Some more general solution for multi-step queries is > in study. > Mason did you mean multi-step or multi-statement queries. Let me bring up the difference here, Multi-step query means, the query involving multiple nodes in the plans - so even a query like "Select * from tab1 limit 5" is a multi-step query, since it has Limit->SeqScan plan. In the context of XC, a query can get reduced to a single RemoteQuery, thus a mutli-step query in XC means a query which can not be reduced to a single RemoteQuery. So, multi-statement queries are subset of multi-step queries. From this angle, PREPARE/EXECUTE support for multi-step queries is in place. So, answer to your question is Yes. For multi-statement queries, the answer is yes and no. There is some work going on about multi-statement queries, and till that finishes, we won't be able to confirm the support for PREPARE/EXECUTE OR for that matter any other clauses and variations of queries, which can involve multi-statement queries. But at the current state, the PREPARE/EXECUTE support for multi-statement queries is complete. We may not need much changes in this support unless multi-statement queries add their own parameters in the prepared statements. > >> Prepared statements are also related to stored function support. Is >> there any impact on support for that? >> > Nope. The only impact would be that there are less bugs than before in that > area :) > Let me give examples here create or replace function bound_param_func(a int) returns int language plpgsql as $$ declare b int; begin *select val2 into b from tab1 where val = a limit 1;* return b; end; $$; Earlier before 90 this used convert the SELECT query into a parameterised query with value of a passed as parameter. But starting 90 the local variables are not converted into parameters but they are handled by backend through language specific hooks. So, these kind of stored functions we don't need PREPARED query support. create or replace function bound_param_func(a int) returns int language plpgsql as $$ declare b int; begin execute '*select val2 from tab1 where val = $1 limit 1*' into b using a; return b; end; $$; The dynamic queries with parameters which have parameter values bound to them using "USING" clause, are still treated as PREPARED queries with bound parameters. Few examples I tried out are working as expected, so probably we have not problems in that area. But still I am looking at those, to see the things are kosher. Does that answer your question? If you know any other usages of PREPAREd statements and parameterised PREPARE statement, please let me know. I am looking for those. > > -- > Michael Paquier > http://michael.otacoo.com > > > ------------------------------------------------------------------------------ > Got Input? Slashdot Needs You. > Take our quick survey online. Come on, we don't ask for help often. > Plus, you'll get a chance to win $100 to spend on ThinkGeek. > http://p.sf.net/sfu/slashdot-survey > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > -- Best Wishes, Ashutosh Bapat EntepriseDB Corporation The Enterprise Postgres Company |
|
From: Michael P. <mic...@gm...> - 2011-07-29 13:06:45
|
> Great. Is this a general solution for multi-step queries, too? > No. I don't think so. Some more general solution for multi-step queries is in study. > > Prepared statements are also related to stored function support. Is > there any impact on support for that? > Nope. The only impact would be that there are less bugs than before in that area :) -- Michael Paquier http://michael.otacoo.com |
|
From: Michael P. <mic...@gm...> - 2011-07-27 06:24:30
|
Hi all, Read access of the official git repository of postgresql is getting very slow this days. I mean this one => http://git.postgresql.org/git/postgresql.git/ If you are developing with PostgreSQL as it is the case for Postgres-XC, you should perhaps feed your local repository with the following URL: https://github.com/postgres/postgres.git This is the mirror repository of PostgreSQL located in GitHub which is far faster for read access. Just to remind to everybody, you can delete a remote link with this command in GIT: git remote rm postgres Then you can add the new URL link: git remote add postgres https://github.com/postgres/postgres.git In case you are not sure which is the remote link to delete, check with "git config -l", the name of each remote and each URL will be printed. Then fetch the new remote for the latest updates. git fetch postgres. Regards, -- Michael Paquier http://michael.otacoo.com |
|
From: Michael P. <mic...@gm...> - 2011-07-27 04:30:57
|
Hi all, Here is the final version of the patch. Regressions are fully updated and code is code refactoring is made a maximum. Comments are welcome. -- Michael Paquier http://michael.otacoo.com |
|
From: Michael P. <mic...@gm...> - 2011-07-26 08:01:13
|
Hi all, Please find attached a patch that allows support for temporary sequences. When creating a temporary sequence, everything is made locally (ALTER, CREATE, DROP). nextval, lastval and currval also are supported with this patch. There is still a bit of code refactoring I want to do before committing this patch, but comments are welcome. I will try next to have a look at the effort necessary to support temporary views and tables. There are a lot of regression tests using temporary objects so there may be hidden issues. This patch passes regressions and is updated in accordance. Regards, -- Michael Paquier http://michael.otacoo.com |
|
From: Pavan D. <pav...@gm...> - 2011-07-26 02:31:42
|
I was suggesting doing that only for GTM and GTM-proxy, not for the postgresql.conf itself. I am not sure if we really need to pull in guc.c code in gtm/gtm-proxy given that the number of parameters to parse are really small and may not need various other things that postgresql.conf needs. But then I haven't looked at guc.c today to judge the additional code. Thanks, Pavan On Mon, Jul 25, 2011 at 10:15 PM, Koichi Suzuki <koi...@gm...> wrote: > Understood. I'd like not to change current configuration drastically. > > Thanks. > ---------- > Koichi Suzuki > > > > 2011/7/26 Michael Paquier <mic...@gm...>: >> I am not sure it is the thing to do. >> I would believe this would make future merges with PostgreSQL even harder >> than it should be, and adding sections in GUC configuration would make it >> harder for users to understand XC configuration logic even if they are used >> to PostgreSQL. We keep our configuration (guc APIs) as much consistent as >> possible with postgres. >> Also, we've used until now configuration files without any sections and >> nobody really complained about it. >> >> So I don't really believe it is worth the effort. >> >> -- >> Michael Paquier >> http://michael.otacoo.com >> >> ------------------------------------------------------------------------------ >> Magic Quadrant for Content-Aware Data Loss Prevention >> Research study explores the data loss prevention market. Includes in-depth >> analysis on the changes within the DLP market, and the criteria used to >> evaluate the strengths and weaknesses of these DLP solutions. >> http://www.accelacomm.com/jaw/sfnl/114/51385063/ >> _______________________________________________ >> Postgres-xc-developers mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> >> > > ------------------------------------------------------------------------------ > Magic Quadrant for Content-Aware Data Loss Prevention > Research study explores the data loss prevention market. Includes in-depth > analysis on the changes within the DLP market, and the criteria used to > evaluate the strengths and weaknesses of these DLP solutions. > http://www.accelacomm.com/jaw/sfnl/114/51385063/ > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com |
|
From: Koichi S. <koi...@gm...> - 2011-07-26 02:15:52
|
Understood. I'd like not to change current configuration drastically. Thanks. ---------- Koichi Suzuki 2011/7/26 Michael Paquier <mic...@gm...>: > I am not sure it is the thing to do. > I would believe this would make future merges with PostgreSQL even harder > than it should be, and adding sections in GUC configuration would make it > harder for users to understand XC configuration logic even if they are used > to PostgreSQL. We keep our configuration (guc APIs) as much consistent as > possible with postgres. > Also, we've used until now configuration files without any sections and > nobody really complained about it. > > So I don't really believe it is worth the effort. > > -- > Michael Paquier > http://michael.otacoo.com > > ------------------------------------------------------------------------------ > Magic Quadrant for Content-Aware Data Loss Prevention > Research study explores the data loss prevention market. Includes in-depth > analysis on the changes within the DLP market, and the criteria used to > evaluate the strengths and weaknesses of these DLP solutions. > http://www.accelacomm.com/jaw/sfnl/114/51385063/ > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > |
|
From: Michael P. <mic...@gm...> - 2011-07-26 02:11:55
|
I am not sure it is the thing to do. I would believe this would make future merges with PostgreSQL even harder than it should be, and adding sections in GUC configuration would make it harder for users to understand XC configuration logic even if they are used to PostgreSQL. We keep our configuration (guc APIs) as much consistent as possible with postgres. Also, we've used until now configuration files without any sections and nobody really complained about it. So I don't really believe it is worth the effort. -- Michael Paquier http://michael.otacoo.com |
|
From: Koichi S. <ko...@in...> - 2011-07-26 01:59:52
|
So configuration file, at all, will be like: [postgresql] # Standalone PostgreSQL configuration ..... ..... [coordinator] # All the configuration parameters for coordinator. If postgres starts as a coordinator, it will see this section, not "postgresql" section. ...... ...... [datanode] # All the configruration parameters for datanode. ...... ...... [gtm] # for GTM and GTM-Standby ...... ...... [gtm_proxy] $ for GTM Proxy ..... ..... It needs some extension to guc-file.l and guc-c. Current guc.c deeply depends upon postgresql context and need substatial change anyway. Do you agree to introduce [coordinator] and [datanode] section (if it looks fine) as a part of conf file extention? --- Koichi On Mon, 25 Jul 2011 12:22:42 +0900 Michael Paquier <mic...@gm...> wrote: > On Mon, Jul 25, 2011 at 11:58 AM, Koichi Suzuki <ko...@in...>wrote: > > > Hi, > > > > I think introducing sections like [gtm] may make configuration files easier > > to review. I have a couple of issues on this though. > > > That's true, but regarding the portability of Postgres-XC for PostgreSQL > itself, is it worth to create something more complicated than necessary in > guc by adding sections? > Let's imagine that in the future GTM becomes a contribution module or is > partially integrated in PostgreSQL, it may be better to keep the GUC > interface of GTM as much consistent as possible with the existing one in > PostgreSQL, no? > -- > Michael Paquier > http://michael.otacoo.com |
|
From: Michael P. <mic...@gm...> - 2011-07-25 03:22:50
|
On Mon, Jul 25, 2011 at 11:58 AM, Koichi Suzuki <ko...@in...>wrote: > Hi, > > I think introducing sections like [gtm] may make configuration files easier > to review. I have a couple of issues on this though. > That's true, but regarding the portability of Postgres-XC for PostgreSQL itself, is it worth to create something more complicated than necessary in guc by adding sections? Let's imagine that in the future GTM becomes a contribution module or is partially integrated in PostgreSQL, it may be better to keep the GUC interface of GTM as much consistent as possible with the existing one in PostgreSQL, no? -- Michael Paquier http://michael.otacoo.com |
|
From: Koichi S. <ko...@in...> - 2011-07-25 02:54:37
|
Hi, I think introducing sections like [gtm] may make configuration files easier to review. I have a couple of issues on this though. 1) Users can append any configuration lines to postgresql.conf to disable any preceding options. This may be a bit difficult because users have to find the last like of the section. We can allow to appear same section any time to override but this will not help simple configuration file review. 2) I'm considering to reuse guc-files.l and guc.c which does not support the section now. I think user comment can serve as section header too. To minimize the code to write, and to make any *.conf file as compatible as possible, I'd like to inherit the current postgresql.conf file format unless there's other background of sections. Like to have any ideas. --- Koichi On Sun, 24 Jul 2011 18:11:34 -0400 Pavan Deolasee <pav...@en...> wrote: > I think its a good idea to have a common conf file for gtm/proxy. We > can possibly have a section inside the conf file to identify gtm-only > or proxy-only parameters. > > Something along these lines: > > hostname= > port= > > [gtm] > start_xid= > > [gtm_proxy] > conn_string > num_threads > > (The conf parameter names are not well thought of above) > > Thanks, > Pavan > > On Sat, Jul 23, 2011 at 10:35 AM, Michael Paquier > <mic...@gm...> wrote: > > GTM parameters: > > hostname, string, GTM server hostname/IP to listen. > > port, string, GTM server port number to listen. > > nodenum, integer, node number for GTM server. > > start_xid, integer, start value of GXID feed > > log_filename, name of log file in GTM data directory > > standby, boolean Start as a GTM standby server. > > standby_conn_string, in the same model as what is done in recovery.conf for > > standby nodes. > > log_min_messages, minimum level of logs that are written. > > > > Proxy options: > > hostname, string, GTM server hostname/IP to listen. > > port, string, GTM server port number to listen. > > nodenum, integer, node number for GTM server. > > log_filename, name of log file in GTM data directory > > gtm_conn_string, connection string to use when contacting GTM (This string > > is completed with nodeID when registering) > > worker_thread_num, integer, number of worker threads > > log_min_messages, minimum level of logs that are written. > > > > Common options are: > > hostname, port, node number, log file name, log minimum messages. > > > > Proxy uses exclusively: > > 1) Connection string to GTM > > 2) worker thread number > > > > GTM uses: > > 1) Standby mode boolean > > 2) start XID > > 3) Connection string for a standby to connect to GTM > > > > Both have a lot in common. We may even use the same option name for > > connection string in the case of Proxy connecting to GTM and GTM-Standby > > connecting to GTM master. > > More than half of the options are the same. Making also configuration file > > having the same name for GTM and proxy may be also easier to understand for > > the users. > > -- > > Michael Paquier > > http://michael.otacoo.com > > > > ------------------------------------------------------------------------------ > > Storage Efficiency Calculator > > This modeling tool is based on patent-pending intellectual property that > > has been used successfully in hundreds of IBM storage optimization engage- > > ments, worldwide. Store less, Store more with what you own, Move data to > > the right place. Try It Now! > > http://www.accelacomm.com/jaw/sfnl/114/51427378/ > > _______________________________________________ > > Postgres-xc-developers mailing list > > Pos...@li... > > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > > > > > > > -- > Pavan Deolasee > EnterpriseDB http://www.enterprisedb.com > > ------------------------------------------------------------------------------ > Magic Quadrant for Content-Aware Data Loss Prevention > Research study explores the data loss prevention market. Includes in-depth > analysis on the changes within the DLP market, and the criteria used to > evaluate the strengths and weaknesses of these DLP solutions. > http://www.accelacomm.com/jaw/sfnl/114/51385063/ > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > |
|
From: Michael P. <mic...@gm...> - 2011-07-25 01:49:44
|
Monday morning is definitely harder than the other moments of the week... On Mon, Jul 25, 2011 at 10:45 AM, Michael Paquier <mic...@gm... > wrote: > It is also necessary to add a little bit access control in nextval and > currval to avoid trying to look for a sequence value in GTM for a temporary > sequence, but nothing heavy, really. > A little bit OF access control. > Temporary tables will be create on local Coordinator and remote Datanodes > if necessary. > Temporary tables will be createD on local Coordinator and remote Datanodes if necessary. > There is already a cleanup mechanism with "RESET ALL" at the end of a > transaction or of a session for session parameters inside pooler There is > already a cleanup mechanism with "RESET ALL" at the end of a transaction or > of a session for session parameters inside pooler so it may be reuse it (and > extend it a bit) for temporary tables. (and extend it a bit) for temporary > tables. > ... so it may be worth reusing it ... My apologies. Michael |
|
From: Michael P. <mic...@gm...> - 2011-07-25 01:45:12
|
Hi, Here are some thoughts about the implementation of temporary tables and sequences. Those objects basically live for the duration of a database session. The case of a temporary sequence is particularly easy. As such a sequence will be used to feed values to Datanodes only, it is not even necessary to create it neither on other Coordinators nor on GTM. This avoids to have to do a cleanup on GTM when cutting a database session. It is also necessary to add a little bit access control in nextval and currval to avoid trying to look for a sequence value in GTM for a temporary sequence, but nothing heavy, really. Temporary tables will be create on local Coordinator and remote Datanodes if necessary. As now XC is using 2PC to create tables, I was thinking about adding the temporary table DDLs in pooler the same way as SET commands. There is already a cleanup mechanism with "RESET ALL" at the end of a transaction or of a session for session parameters inside pooler so it may be reuse it (and extend it a bit) for temporary tables. By using the same mechanism as session parameters, temporary tables will be created to backend nodes only when connections are established to them. This implementation described in this email may not require a huge effort (less than 1 week). The hardest part would be to update all the regression tests that use temporary tables (there are a lot!) and those tests may make appear some bugs we haven't found yet in XC. Any comments? -- Michael Paquier http://michael.otacoo.com |
|
From: Michael P. <mic...@gm...> - 2011-07-25 00:35:40
|
Thanks for this report, it is an important diff. I committed that as commit f0f8a4b0. -- Michael Paquier http://michael.otacoo.com |
|
From: Pavan D. <pav...@en...> - 2011-07-24 22:12:03
|
I think its a good idea to have a common conf file for gtm/proxy. We can possibly have a section inside the conf file to identify gtm-only or proxy-only parameters. Something along these lines: hostname= port= [gtm] start_xid= [gtm_proxy] conn_string num_threads (The conf parameter names are not well thought of above) Thanks, Pavan On Sat, Jul 23, 2011 at 10:35 AM, Michael Paquier <mic...@gm...> wrote: > GTM parameters: > hostname, string, GTM server hostname/IP to listen. > port, string, GTM server port number to listen. > nodenum, integer, node number for GTM server. > start_xid, integer, start value of GXID feed > log_filename, name of log file in GTM data directory > standby, boolean Start as a GTM standby server. > standby_conn_string, in the same model as what is done in recovery.conf for > standby nodes. > log_min_messages, minimum level of logs that are written. > > Proxy options: > hostname, string, GTM server hostname/IP to listen. > port, string, GTM server port number to listen. > nodenum, integer, node number for GTM server. > log_filename, name of log file in GTM data directory > gtm_conn_string, connection string to use when contacting GTM (This string > is completed with nodeID when registering) > worker_thread_num, integer, number of worker threads > log_min_messages, minimum level of logs that are written. > > Common options are: > hostname, port, node number, log file name, log minimum messages. > > Proxy uses exclusively: > 1) Connection string to GTM > 2) worker thread number > > GTM uses: > 1) Standby mode boolean > 2) start XID > 3) Connection string for a standby to connect to GTM > > Both have a lot in common. We may even use the same option name for > connection string in the case of Proxy connecting to GTM and GTM-Standby > connecting to GTM master. > More than half of the options are the same. Making also configuration file > having the same name for GTM and proxy may be also easier to understand for > the users. > -- > Michael Paquier > http://michael.otacoo.com > > ------------------------------------------------------------------------------ > Storage Efficiency Calculator > This modeling tool is based on patent-pending intellectual property that > has been used successfully in hundreds of IBM storage optimization engage- > ments, worldwide. Store less, Store more with what you own, Move data to > the right place. Try It Now! > http://www.accelacomm.com/jaw/sfnl/114/51427378/ > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com |
|
From: Michael P. <mic...@gm...> - 2011-07-23 14:35:46
|
GTM parameters: hostname, string, GTM server hostname/IP to listen. port, string, GTM server port number to listen. nodenum, integer, node number for GTM server. start_xid, integer, start value of GXID feed log_filename, name of log file in GTM data directory standby, boolean Start as a GTM standby server. standby_conn_string, in the same model as what is done in recovery.conf for standby nodes. log_min_messages, minimum level of logs that are written. Proxy options: hostname, string, GTM server hostname/IP to listen. port, string, GTM server port number to listen. nodenum, integer, node number for GTM server. log_filename, name of log file in GTM data directory gtm_conn_string, connection string to use when contacting GTM (This string is completed with nodeID when registering) worker_thread_num, integer, number of worker threads log_min_messages, minimum level of logs that are written. Common options are: hostname, port, node number, log file name, log minimum messages. Proxy uses exclusively: 1) Connection string to GTM 2) worker thread number GTM uses: 1) Standby mode boolean 2) start XID 3) Connection string for a standby to connect to GTM Both have a lot in common. We may even use the same option name for connection string in the case of Proxy connecting to GTM and GTM-Standby connecting to GTM master. More than half of the options are the same. Making also configuration file having the same name for GTM and proxy may be also easier to understand for the users. -- Michael Paquier http://michael.otacoo.com |
|
From: Koichi S. <koi...@gm...> - 2011-07-23 12:41:01
|
I don't care about the name, but I'd like to see what configuration parameters GTM and GTM-Proxy should need. If they're very different, there will be no reason to have the same configuration file name. I believe introduction of GUC code should independent from HA support and should be treated as separate patches. This is why I'd like to have this before further effort of HA related improvements. Regards; ---------- Koichi Suzuki 2011/7/23 Michael Paquier <mic...@gm...>: > > > On Thu, Jul 21, 2011 at 3:52 PM, Koichi Suzuki <ko...@in...> > wrote: >> >> I totally agree with having gtm.conf and gtm_proxy.conf to reduce the >> number of runt-time options, especially those passed with -o option from >> gtm_ctl. > > About the configuration file, perhaps GTM and GTM-proxy could use a > configuration file with the same name. > I would imagine that if the guc APIs of postgresql are brought in GTM it > will be included in gtm/common so as a base it looks more consistent to use > the same file name: gtm.conf. >> >> I need to bring code for GUC from postgres. Could be done as a part of >> additional HA feature development. > > If you want to include that in the HA effort, it could be nice to make > separate patches for the configuration file and GTM-Standby additional > features. > It will make commit comprehension simpler. > -- > Michael Paquier > http://michael.otacoo.com > > ------------------------------------------------------------------------------ > Storage Efficiency Calculator > This modeling tool is based on patent-pending intellectual property that > has been used successfully in hundreds of IBM storage optimization engage- > ments, worldwide. Store less, Store more with what you own, Move data to > the right place. Try It Now! > http://www.accelacomm.com/jaw/sfnl/114/51427378/ > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > |
|
From: Michael P. <mic...@gm...> - 2011-07-23 11:57:34
|
On Thu, Jul 21, 2011 at 3:52 PM, Koichi Suzuki <ko...@in...>wrote: > I totally agree with having gtm.conf and gtm_proxy.conf to reduce the > number of runt-time options, especially those passed with -o option from > gtm_ctl. > About the configuration file, perhaps GTM and GTM-proxy could use a configuration file with the same name. I would imagine that if the guc APIs of postgresql are brought in GTM it will be included in gtm/common so as a base it looks more consistent to use the same file name: gtm.conf. > > I need to bring code for GUC from postgres. Could be done as a part of > additional HA feature development. > If you want to include that in the HA effort, it could be nice to make separate patches for the configuration file and GTM-Standby additional features. It will make commit comprehension simpler. -- Michael Paquier http://michael.otacoo.com |
|
From: Michael P. <mic...@gm...> - 2011-07-23 05:22:34
|
Thanks. I will try to have a look at your patch as soon as possible. Regards, -- Michael Paquier http://michael.otacoo.com |
|
From: Koichi S. <ko...@in...> - 2011-07-21 06:48:45
|
I totally agree with having gtm.conf and gtm_proxy.conf to reduce the number of runt-time options, especially those passed with -o option from gtm_ctl. I need to bring code for GUC from postgres. Could be done as a part of additional HA feature development. --- Koichi On Thu, 21 Jul 2011 15:43:08 +0900 Michael Paquier <mic...@gm...> wrote: > Hi, > > In the same model as postgresql.conf, why not creating a configuration file > only for GTM/Proxy? > This advantage of that could be the reduction of call parameters for GTM and > proxy > Here is the list of parameters that could be used: > hostname, string, GTM server hostname/IP to listen. > port, string, GTM server port number to listen. > nodenum, integer, node number for GTM server. > start_xid, integer, start value of GXID feed > log_filename, name of log file in GTM data directory > > Standby options > standby, boolean Start as a GTM standby server. > standby_conn_string, in the same model as what is done in recovery.conf for > standby nodes. > > Proxy options: > gtm_conn_string, connection string to use when contacting GTM (This string > is completed with nodeID when registering) > worker_thread_num, integer, number of worker threads > > Other options: > log_min_messages, minimum level of logs that are written. > > The main purpose of this initiative is to have: > 1) an easier configuration interface for GTM and proxy > 2) have a control on the logs GTM write, now GTM writes a large quantity of > logs and log file can reach easily several GB after 1 hour of run. > > For the implementation, it could be possible to use the guc interface but I > am not that much familiar with that. > Any thoughts? Complementary ideas? > -- > Michael Paquier > http://michael.otacoo.com |
|
From: Michael P. <mic...@gm...> - 2011-07-21 06:43:18
|
Hi, In the same model as postgresql.conf, why not creating a configuration file only for GTM/Proxy? This advantage of that could be the reduction of call parameters for GTM and proxy Here is the list of parameters that could be used: hostname, string, GTM server hostname/IP to listen. port, string, GTM server port number to listen. nodenum, integer, node number for GTM server. start_xid, integer, start value of GXID feed log_filename, name of log file in GTM data directory Standby options standby, boolean Start as a GTM standby server. standby_conn_string, in the same model as what is done in recovery.conf for standby nodes. Proxy options: gtm_conn_string, connection string to use when contacting GTM (This string is completed with nodeID when registering) worker_thread_num, integer, number of worker threads Other options: log_min_messages, minimum level of logs that are written. The main purpose of this initiative is to have: 1) an easier configuration interface for GTM and proxy 2) have a control on the logs GTM write, now GTM writes a large quantity of logs and log file can reach easily several GB after 1 hour of run. For the implementation, it could be possible to use the guc interface but I am not that much familiar with that. Any thoughts? Complementary ideas? -- Michael Paquier http://michael.otacoo.com |
|
From: Michael P. <mic...@gm...> - 2011-07-21 06:19:09
|
Hi,
Just a comment about that.
The commit message is not correct but the link refers to the correct commit
regarding reactivation of preferred_data_nodes.
This is because I first committed the fix in 0.9.5 stable branch and then in
master.
Sorry about that, it is a bug in the hooks used in remote repository.
Michael
---------- Forwarded message ----------
From: Michael Paquier <mic...@us...>
Date: Thu, Jul 21, 2011 at 3:10 PM
Subject: [Postgres-xc-committers] Postgres-XC branch, REL0_9_5_STABLE,
updated. XC0_9_5_PG9_0-3-gc2d6ca9
To: pos...@li...
Project "Postgres-XC".
The branch, REL0_9_5_STABLE has been updated
via c2d6ca92c3ee26eddbddb79d0c7fbab9f960c884 (commit)
from 9c9346e33d899a9aaebaa1e1a6d5f79bb1d8d49d (commit)
- Log -----------------------------------------------------------------
http://postgres-xc.git.sourceforge.net/git/gitweb.cgi?p=postgres-xc/postgres-xc;a=commit;h=c2d6ca92c3ee26eddbddb79d0c7fbab9f960c884
commit 0b8135db11723d586033bf45de94705003ef2bf6
Author: Ashutosh Bapat <ash...@en...>
Date: Mon Jul 18 17:42:25 2011 +0530
If the havingQuals in query contain aggregates, the aggregates and the
VARs not
included in the expression trees rooted in those aggregates are included
in the
targetlist to be pushed to the data node. The aggregates are finalised at
the
coordinator and havingQual is evaluated.
The same technique is used to push aggregates and VARs involved in the
expressions in the targetlist to the data nodes.
With this patch, we apply the grouping optimizations to the queries
containing
having clause.
M src/backend/optimizer/plan/createplan.c
M src/backend/pgxc/plan/planner.c
M src/backend/pgxc/pool/postgresql_fdw.c
M src/include/pgxc/postgresql_fdw.h
M src/test/regress/expected/xc_groupby.out
M src/test/regress/expected/xc_having.out
-----------------------------------------------------------------------
Summary of changes:
src/backend/pgxc/locator/locator.c | 54
+++++++++++++++++++++++++++++++++--
1 files changed, 50 insertions(+), 4 deletions(-)
hooks/post-receive
--
Postgres-XC
------------------------------------------------------------------------------
5 Ways to Improve & Secure Unified Communications
Unified Communications promises greater efficiencies for business. UC can
improve internal communications as well as offer faster, more efficient ways
to interact with customers and streamline customer service. Learn more!
http://www.accelacomm.com/jaw/sfnl/114/51426253/
_______________________________________________
Postgres-xc-committers mailing list
Pos...@li...
https://lists.sourceforge.net/lists/listinfo/postgres-xc-committers
|
|
From: Ashutosh B. <ash...@en...> - 2011-07-20 09:17:27
|
On Wed, Jul 20, 2011 at 5:56 AM, Michael Paquier
<mic...@gm...>wrote:
> Even if the method of parameters is a good approach, I am still wondering
> if it makes the implementation easier or not.
> Is there a basic mechanism in Postgres that permits to transform an
> expression function into a parameter?
>
> I am also wondering about cases where you may have functions calling other
> functions.
> date_trunc(now()), statement_timestamp - transaction_timestamp, etc.
>
> Maybe we may need some new basic mechanism to be able to support queries
> like:
> create sequence foo;
> create table footab (a int);
> INSERT INTO footab values (nextval('foo'));
>
> This could be a new node type that could be used inside RemoteQuery plans
> to deal with non-immutable functions feed.
> And the remote query plan may be able to rebuild the query in a correct way
> before pushing it down to nodes.
> But I am still new to those mechanisms so perhaps my words are not adapted.
> Any help is welcome of course.
>
Not sure, but take a look at ExecEvalParamExec and in general ParamExec
usage.
>
> On Tue, Jul 19, 2011 at 4:02 PM, Ashutosh Bapat <
> ash...@en...> wrote:
>
>>
>>
>>>
>>>> Hmm, I didn't think of this case :(
>>>>
>>>> I think, this will be a generic problem, if the queries other than
>>>> INSERT also use nextval() on sequences. There is less probability that
>>>> SELECT will use it but UPDATEs are other potential candidates.
>>>>
>>> Yes, you got a point here.
> I am just wondering about what kind of mechanisms I could borrow or extend
> in Postgres to support such things.
> Those basics are necessary for the feed of non-immutable functions, and its
> design should be done with care.
> --
> Michael Paquier
> http://michael.otacoo.com
>
--
Best Wishes,
Ashutosh Bapat
EntepriseDB Corporation
The Enterprise Postgres Company
|
|
From: Michael P. <mic...@gm...> - 2011-07-20 00:26:29
|
Even if the method of parameters is a good approach, I am still wondering if
it makes the implementation easier or not.
Is there a basic mechanism in Postgres that permits to transform an
expression function into a parameter?
I am also wondering about cases where you may have functions calling other
functions.
date_trunc(now()), statement_timestamp - transaction_timestamp, etc.
Maybe we may need some new basic mechanism to be able to support queries
like:
create sequence foo;
create table footab (a int);
INSERT INTO footab values (nextval('foo'));
This could be a new node type that could be used inside RemoteQuery plans to
deal with non-immutable functions feed.
And the remote query plan may be able to rebuild the query in a correct way
before pushing it down to nodes.
But I am still new to those mechanisms so perhaps my words are not adapted.
Any help is welcome of course.
On Tue, Jul 19, 2011 at 4:02 PM, Ashutosh Bapat <
ash...@en...> wrote:
>
>
>>
>>> Hmm, I didn't think of this case :(
>>>
>>> I think, this will be a generic problem, if the queries other than INSERT
>>> also use nextval() on sequences. There is less probability that SELECT will
>>> use it but UPDATEs are other potential candidates.
>>>
>> Yes, you got a point here.
I am just wondering about what kind of mechanisms I could borrow or extend
in Postgres to support such things.
Those basics are necessary for the feed of non-immutable functions, and its
design should be done with care.
--
Michael Paquier
http://michael.otacoo.com
|
|
From: Ashutosh B. <ash...@en...> - 2011-07-19 07:02:57
|
> >> I would personally have a different idea of doing this. We should have the >> sequence definitions on the data nodes as well, and data nodes should be >> able to retrieve the nextval of the sequences. Looking at the possibility of >> having multiple coordinators, we anyway need to synchronise nextval of >> sequences across coordinators. Same mechanism can be used to synchronise it >> across data nodes. In such case sending nextval(foo) through query won't be >> a problem. >> > What happens in the case of replicated tables? If datanodes receive a > nextval value locally tuples will be inconsistent. > Just for the case of replicated tables, it looks far simpler to let > Coordinator feed the queries with nextval. > Hmm, I didn't think of this case :( I think, this will be a generic problem, if the queries other than INSERT also use nextval() on sequences. There is less probability that SELECT will use it but UPDATEs are other potential candidates. > -- > Michael Paquier > http://michael.otacoo.com > -- Best Wishes, Ashutosh Bapat EntepriseDB Corporation The Enterprise Postgres Company |
1 message has been excluded from this view by a project administrator.
