pgsql: Implement "fastupdate" support for GIN indexes, in which we try

Log Message:
-----------
Implement "fastupdate" support for GIN indexes, in which we try to accumulate
multiple index entries in a holding area before adding them to the main index
structure. This helps because bulk insert is (usually) significantly faster
than retail insert for GIN.

This patch also removes GIN support for amgettuple-style index scans. The
API defined for amgettuple is difficult to support with fastupdate, and
the previously committed partial-match feature didn't really work with
it either. We might eventually figure a way to put back amgettuple
support, but it won't happen for 8.4.

catversion bumped because of change in GIN's pg_am entry, and because
the format of GIN indexes changed on-disk (there's a metapage now,
and possibly a pending list).

Teodor Sigaev

Modified Files:
--------------
pgsql/doc/src/sgml:
gin.sgml (r2.16 -> r2.17)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/doc/src/sgml/gin.sgml?r1=2.16&r2=2.17)
indexam.sgml (r2.29 -> r2.30)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/doc/src/sgml/indexam.sgml?r1=2.29&r2=2.30)
textsearch.sgml (r1.47 -> r1.48)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/doc/src/sgml/textsearch.sgml?r1=1.47&r2=1.48)
pgsql/doc/src/sgml/ref:
create_index.sgml (r1.70 -> r1.71)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/doc/src/sgml/ref/create_index.sgml?r1=1.70&r2=1.71)
vacuum.sgml (r1.54 -> r1.55)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/doc/src/sgml/ref/vacuum.sgml?r1=1.54&r2=1.55)
pgsql/src/backend/access/common:
reloptions.c (r1.23 -> r1.24)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/common/reloptions.c?r1=1.23&r2=1.24)
pgsql/src/backend/access/gin:
Makefile (r1.3 -> r1.4)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/Makefile?r1=1.3&r2=1.4)
ginbulk.c (r1.14 -> r1.15)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/ginbulk.c?r1=1.14&r2=1.15)
gindatapage.c (r1.13 -> r1.14)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/gindatapage.c?r1=1.13&r2=1.14)
ginget.c (r1.22 -> r1.23)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/ginget.c?r1=1.22&r2=1.23)
gininsert.c (r1.18 -> r1.19)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/gininsert.c?r1=1.18&r2=1.19)
ginutil.c (r1.20 -> r1.21)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/ginutil.c?r1=1.20&r2=1.21)
ginvacuum.c (r1.27 -> r1.28)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/ginvacuum.c?r1=1.27&r2=1.28)
ginxlog.c (r1.17 -> r1.18)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/ginxlog.c?r1=1.17&r2=1.18)
pgsql/src/backend/access/gist:
gistvacuum.c (r1.42 -> r1.43)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gist/gistvacuum.c?r1=1.42&r2=1.43)
pgsql/src/backend/access/hash:
hash.c (r1.108 -> r1.109)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/hash/hash.c?r1=1.108&r2=1.109)
pgsql/src/backend/access/index:
indexam.c (r1.112 -> r1.113)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/index/indexam.c?r1=1.112&r2=1.113)
pgsql/src/backend/access/nbtree:
nbtree.c (r1.167 -> r1.168)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/nbtree/nbtree.c?r1=1.167&r2=1.168)
pgsql/src/backend/catalog:
index.c (r1.312 -> r1.313)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/catalog/index.c?r1=1.312&r2=1.313)
pgsql/src/backend/commands:
analyze.c (r1.133 -> r1.134)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/commands/analyze.c?r1=1.133&r2=1.134)
vacuum.c (r1.385 -> r1.386)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/commands/vacuum.c?r1=1.385&r2=1.386)
vacuumlazy.c (r1.118 -> r1.119)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/commands/vacuumlazy.c?r1=1.118&r2=1.119)
pgsql/src/backend/nodes:
tidbitmap.c (r1.17 -> r1.18)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/nodes/tidbitmap.c?r1=1.17&r2=1.18)
pgsql/src/include/access:
genam.h (r1.75 -> r1.76)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/access/genam.h?r1=1.75&r2=1.76)
gin.h (r1.28 -> r1.29)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/access/gin.h?r1=1.28&r2=1.29)
pgsql/src/include/catalog:
catversion.h (r1.524 -> r1.525)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/catalog/catversion.h?r1=1.524&r2=1.525)
pg_am.h (r1.61 -> r1.62)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/catalog/pg_am.h?r1=1.61&r2=1.62)
pg_proc.h (r1.537 -> r1.538)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/catalog/pg_proc.h?r1=1.537&r2=1.538)
pgsql/src/include/nodes:
tidbitmap.h (r1.9 -> r1.10)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/nodes/tidbitmap.h?r1=1.9&r2=1.10)

Added Files:
-----------
pgsql/src/backend/access/gin:
ginfast.c (r1.1)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/gin/ginfast.c?rev=1.1&content-type=text/x-cvsweb-markup)

2009/3/24 Tom Lane <tgl(at)postgresql(dot)org>:
> Implement "fastupdate" support for GIN indexes, in which we try to accumulate
> multiple index entries in a holding area before adding them to the main index
> structure.  This helps because bulk insert is (usually) significantly faster
> than retail insert for GIN.
>
> This patch also removes GIN support for amgettuple-style index scans.  The
> API defined for amgettuple is difficult to support with fastupdate, and
> the previously committed partial-match feature didn't really work with
> it either.  We might eventually figure a way to put back amgettuple
> support, but it won't happen for 8.4.
>
> catversion bumped because of change in GIN's pg_am entry, and because
> the format of GIN indexes changed on-disk (there's a metapage now,
> and possibly a pending list).

Will this break pg_migrator?

...Robert

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> 2009/3/24 Tom Lane <tgl(at)postgresql(dot)org>:
>> catversion bumped because of change in GIN's pg_am entry, and because
>> the format of GIN indexes changed on-disk (there's a metapage now,
>> and possibly a pending list).

> Will this break pg_migrator?

No worse than it's already broken by hash indexes. I would imagine
the strategy will just be to force a REINDEX on any non-btree indexes.

regards, tom lane

Tom Lane wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> > 2009/3/24 Tom Lane <tgl(at)postgresql(dot)org>:
> >> catversion bumped because of change in GIN's pg_am entry, and because
> >> the format of GIN indexes changed on-disk (there's a metapage now,
> >> and possibly a pending list).
>
> > Will this break pg_migrator?
>
> No worse than it's already broken by hash indexes. I would imagine
> the strategy will just be to force a REINDEX on any non-btree indexes.

Agreed.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +