summaryrefslogtreecommitdiff
path: root/src/bin/pg_verifybackup/pg_verifybackup.c
AgeCommit message (Collapse)Author
6 hoursStandardize LSN formatting by zero paddingÁlvaro Herrera
This commit standardizes the output format for LSNs to ensure consistent representation across various tools and messages. Previously, LSNs were inconsistently printed as `%X/%X` in some contexts, while others used zero-padding. This often led to confusion when comparing. To address this, the LSN format is now uniformly set to `%X/%08X`, ensuring the lower 32-bit part is always zero-padded to eight hexadecimal digits. Author: Japin Li <[email protected]> Reviewed-by: Masahiko Sawada <[email protected]> Reviewed-by: Álvaro Herrera <[email protected]> Discussion: https://postgr.es/m/ME0P300MB0445CA53CA0E4B8C1879AF84B641A@ME0P300MB0445.AUSP300.PROD.OUTLOOK.COM
2025-04-29pg_verifybackup: Message style improvementsPeter Eisentraut
2025-04-27Don't use double-quotes in #include's of system headers, redux.Tom Lane
This cleans up some loose ends left by commit e8ca9ed1d. I hadn't looked closely enough at these places before, but now I have. The use of double-quoted #includes for Perl headers in plperl_system.h seems to be simply a mistake introduced in 6c944bf3c and faithfully copied forward since then. (I had thought possibly it was required by some weird Windows build setup, but there's no evidence of that in our history.) The occurrences in SectionMemoryManager.h and SectionMemoryManager.cpp evidently stem from those files' origin as LLVM code. It's understandable that LLVM would treat their own files as needing double-quoted #includes; but they're still system headers to us. I also applied the same check to *.c files, and found a few other random incorrect usages in both directions. Our ECPG headers and test files routinely use angle brackets to refer to ECPG headers. I left those usages alone, since it seems reasonable for an ECPG user to regard those headers as system headers.
2025-04-07Use XLOG_CONTROL_FILE macro consistently for control file name.Fujii Masao
The XLOG_CONTROL_FILE macro (defined in access/xlog_internal.h) represents the control file name. While some parts of the codebase already use this macro, others previously hardcoded the file name as a string. This commit replaces those hardcoded strings with the macro, ensuring consistent usage throughout the code. This makes future maintenance easier and improves searchability, for example when grepping for control file usage. Author: Anton A. Melnikov <[email protected]> Reviewed-by: Daniel Gustafsson <[email protected]> Reviewed-by: Kyotaro Horiguchi <[email protected]> Reviewed-by: Masao Fujii <[email protected]> Discussion: https://postgr.es/m/[email protected]
2025-03-29Use PRI?64 instead of "ll?" in format strings (continued).Peter Eisentraut
Continuation of work started in commit 15a79c73, after initial trial. Author: Thomas Munro <[email protected]> Discussion: https://postgr.es/m/b936d2fb-590d-49c3-a615-92c3a88c6c19%40eisentraut.org
2025-03-13pg_noreturn to replace pg_attribute_noreturn()Peter Eisentraut
We want to support a "noreturn" decoration on more compilers besides just GCC-compatible ones, but for that we need to move the decoration in front of the function declaration instead of either behind it or wherever, which is the current style afforded by GCC-style attributes. Also rename the macro to "pg_noreturn" to be similar to the C11 standard "noreturn". pg_noreturn is now supported on all compilers that support C11 (using _Noreturn), as well as GCC-compatible ones (using __attribute__, as before), as well as MSVC (using __declspec). (When PostgreSQL requires C11, the latter two variants can be dropped.) Now, all supported compilers effectively support pg_noreturn, so the extra code for !HAVE_PG_ATTRIBUTE_NORETURN can be dropped. This also fixes a possible problem if third-party code includes stdnoreturn.h, because then the current definition of #define pg_attribute_noreturn() __attribute__((noreturn)) would cause an error. Note that the C standard does not support a noreturn attribute on function pointer types. So we have to drop these here. There are only two instances at this time, so it's not a big loss. In one case, we can make up for it by adding the pg_noreturn to a wrapper function and adding a pg_unreachable(), in the other case, the latter was already done before. Reviewed-by: Dagfinn Ilmari Mannsåker <[email protected]> Reviewed-by: Andres Freund <[email protected]> Discussion: https://www.postgresql.org/message-id/flat/pxr5b3z7jmkpenssra5zroxi7qzzp6eswuggokw64axmdixpnk@zbwxuq7gbbcw
2025-01-01Update copyright for 2025Bruce Momjian
Backpatch-through: 13
2024-10-02File size in a backup manifest should use uint64, not size_t.Robert Haas
size_t is the size of an object in memory, not the size of a file on disk. Thanks to Tom Lane for noting the error. Discussion: http://postgr.es/m/[email protected]
2024-10-01Fix some pg_verifybackup issues reported by Coverity.Robert Haas
Commit 8dfd3129027969fdd2d9d294220c867d2efd84aa introduced a few problems. verify_tar_file() forgot to free a buffer; the leak can't add up to anything material, but might as well fix it. precheck_tar_backup_file() intended to return after reporting an error but didn't actually do so. member_copy_control_data() could try to copy zero bytes (and maybe Coverity thinks it can even be trying to copy a negative number of bytes). Per discussion with Tom Lane. Discussion: http://postgr.es/m/[email protected]
2024-09-27Reindent pg_verifybackup.c.Robert Haas
2024-09-27pg_verifybackup: Verify tar-format backups.Robert Haas
This also works for compressed tar-format backups. However, -n must be used, because we use pg_waldump to verify WAL, and it doesn't yet know how to verify WAL that is stored inside of a tarfile. Amul Sul, reviewed by Sravan Kumar and by me, and revised by me.
2024-08-16pg_verifybackup: Move some declarations to new pg_verifybackup.hRobert Haas
This is in preparation for adding a second source file to this directory. Amul Sul, reviewed by Sravan Kumar and revised a bit by me. Discussion: http://postgr.es/m/CAAJ_b95mcGjkfAf1qduOR97CokW8-_i-dWLm3v6x1w2-OW9M+A@mail.gmail.com
2024-08-16pg_verifybackup: Move skip_checksums into verifier_context.Robert Haas
This is in preparation for adding a second source file to this directory. It will need access to this value. Also, fewer global variables is usually a good thing. Amul Sul, reviewed by Sravan Kumar and revised a bit by me. Discussion: http://postgr.es/m/CAAJ_b95mcGjkfAf1qduOR97CokW8-_i-dWLm3v6x1w2-OW9M+A@mail.gmail.com
2024-06-21parse_manifest: Use const char *Peter Eisentraut
This adapts the manifest parsing code to take advantage of the const-ified jsonapi. Reviewed-by: Andrew Dunstan <[email protected]> Discussion: https://www.postgresql.org/message-id/flat/f732b014-f614-4600-a437-dba5a2c3738b%40eisentraut.org
2024-04-12Don't allocate large buffer on the stack in pg_verifybackupAndrew Dunstan
Per complaint from Andres Freund. Follow his suggestion to allocate the buffer once in the calling routine instead. Also make a tiny indentation improvement. Discussion: https://postgr.es/m/[email protected]
2024-04-12Fix some memory leaks associated with parsing json and manifestsAndrew Dunstan
Coverity complained about not freeing some memory associated with incrementally parsing backup manifests. To fix that, provide and use a new shutdown function for the JsonManifestParseIncrementalState object, in line with a suggestion from Tom Lane. While analysing the problem, I noticed a buglet in freeing memory for incremental json lexers. To fix that remove a bogus condition on freeing the memory allocated for them.
2024-04-06Convert uses of hash_string_pointer to fasthash equivalentJohn Naylor
Remove duplicate hash_string_pointer() function definitions by creating a new inline function hash_string() for this purpose. This has the added advantage of avoiding strlen() calls when doing hash lookup. It's not clear how many of these are perfomance-sensitive enough to benefit from that, but the simplification is worth it on its own. Reviewed by Jeff Davis Discussion: https://postgr.es/m/CANWCAZbg_XeSeY0a_PqWmWqeRATvzTzUNYRLeT%2Bbzs%2BYQdC92g%40mail.gmail.com
2024-04-04Use incremental parsing of backup manifests.Andrew Dunstan
This changes the three callers to json_parse_manifest() to use json_parse_manifest_incremental_chunk() if appropriate. In the case of the backend caller, since we don't know the size of the manifest in advance we always call the incremental parser. Author: Andrew Dunstan Reviewed-By: Jacob Champion Discussion: https://postgr.es/m/[email protected]
2024-03-13Add the system identifier to backup manifests.Robert Haas
Before this patch, if you took a full backup on server A and then tried to use the backup manifest to take an incremental backup on server B, it wouldn't know that the manifest was from a different server and so the incremental backup operation could potentially complete without error. When you later tried to run pg_combinebackup, you'd find out that your incremental backup was and always had been invalid. That's poor timing, because nobody likes finding out about backup problems only at restore time. With this patch, you'll get an error when trying to take the (invalid) incremental backup, which seems a lot nicer. Amul Sul, revised by me. Review by Michael Paquier. Discussion: http://postgr.es/m/CA+TgmoYLZzbSAMM3cAjV4Y+iCRZn-bR9H2+Mdz7NdaJFU1Zb5w@mail.gmail.com
2024-03-04pg_verifybackup: Refactor parse_manifest_file.Robert Haas
Return a pointer to the manifest_data instead of individual pointers to relevant data stored within the manifest_data object. The previous approach scales poorly if we add more things to the backup manifest, as has been proposed. Amul Sul, reviewed by Sravan Velagandula, Michael Paquier, and me. Discussion: http://postgr.es/m/CAAJ_b95=1LONf99-M_ep588fL_WgLJfdnb7XG4GWE7JDD22E4w@mail.gmail.com
2024-01-04Update copyright for 2024Bruce Momjian
Reported-by: Michael Paquier Discussion: https://postgr.es/m/[email protected] Backpatch-through: 12
2023-12-19Move src/bin/pg_verifybackup/parse_manifest.c into src/common.Robert Haas
This makes it possible for the code to be easily reused by other client-side tools, and/or by the server. Patch by me. Review of this patch in particular by at least Peter Eisentraut; reviewers for the patch series in general include Dilip Kumar, Andres Fruend, David Steele, Álvaro Herrera, and Jakub Wartak. Discussion: http://postgr.es/m/CA+TgmoZ6UGZVnSy5iak6s6+AXu_DewXovDjhLs3-su6nmU_x_g@mail.gmail.com
2023-12-05Rename pg_verifybackup's JsonManifestParseContext callback functions.Robert Haas
The old names were too generic, and would have applied to any binary that made use of JsonManifestParseContext. Rename to make the names specific to pg_verifybackup, since there are plans afoot to reuse this infrastructure. Per suggestion from Álvaro Herrra. Discussion: http://postgr.es/m/[email protected]
2023-12-05Rename JsonManifestParseContext callbacks.Robert Haas
There is no real reason to just run multiple words together when we can instead punctuate them with marks intended for that purpose. Per suggestion from Álvaro Herrera. Discussion: http://postgr.es/m/[email protected]
2023-02-06Add support for progress reporting to pg_verifybackupMichael Paquier
This adds a new option to pg_verifybackup called -P/--progress, showing every second some information about the progress of the checksum verification based on the data of a backup manifest. Similarly to what is done for pg_rewind and pg_basebackup, the information printed in the progress report consists of the current amount of data computed and the total amount of data that will be computed. Note that files found with an incorrect size do not have their checksum verified, hence their size is not appended to the total amount of data estimated during the first scan of the manifest data (such incorrect sizes could be overly high, for one, falsifying the progress report). Author: Masahiko Sawada Discussion: https://postgr.es/m/CAD21AoC5+JOgMd4o3z_oxw0f8JDSsCYY7zSbhe-O9x7f33rw_A@mail.gmail.com
2023-01-02Update copyright for 2023Bruce Momjian
Backpatch-through: 11
2022-09-20Harmonize more parameter names in bulk.Peter Geoghegan
Make sure that function declarations use names that exactly match the corresponding names from function definitions in optimizer, parser, utility, libpq, and "commands" code, as well as in remaining library code. Do the same for all code related to frontend programs (with the exception of pg_dump/pg_dumpall related code). Like other recent commits that cleaned up function parameter names, this commit was written with help from clang-tidy. Later commits will handle ecpg and pg_dump/pg_dumpall. Author: Peter Geoghegan <[email protected]> Reviewed-By: David Rowley <[email protected]> Discussion: https://postgr.es/m/CAH2-WznJt9CMM9KJTMjJh_zbL5hD9oX44qdJ4aqZtjFi-zA3Tg@mail.gmail.com
2022-08-29Clean up inconsistent use of fflush().Tom Lane
More than twenty years ago (79fcde48b), we hacked the postmaster to avoid a core-dump on systems that didn't support fflush(NULL). We've mostly, though not completely, hewed to that rule ever since. But such systems are surely gone in the wild, so in the spirit of cleaning out no-longer-needed portability hacks let's get rid of multiple per-file fflush() calls in favor of using fflush(NULL). Also, we were fairly inconsistent about whether to fflush() before popen() and system() calls. While we've received no bug reports about that, it seems likely that at least some of these call sites are at risk of odd behavior, such as error messages appearing in an unexpected order. Rather than expend a lot of brain cells figuring out which places are at hazard, let's just establish a uniform coding rule that we should fflush(NULL) before these calls. A no-op fflush() is surely of trivial cost compared to launching a sub-process via a shell; while if it's not a no-op then we likely need it. Discussion: https://postgr.es/m/[email protected]
2022-04-13Remove extraneous blank lines before block-closing bracesAlvaro Herrera
These are useless and distracting. We wouldn't have written the code with them to begin with, so there's no reason to keep them. Author: Justin Pryzby <[email protected]> Discussion: https://postgr.es/m/[email protected] Discussion: https://postgr.es/m/attachment/133167/0016-Extraneous-blank-lines.patch
2022-04-08Improve frontend error logging style.Tom Lane
Get rid of the separate "FATAL" log level, as it was applied so inconsistently as to be meaningless. This mostly involves s/pg_log_fatal/pg_log_error/g. Create a macro pg_fatal() to handle the common use-case of pg_log_error() immediately followed by exit(1). Various modules had already invented either this or equivalent macros; standardize on pg_fatal() and apply it where possible. Invent the ability to add "detail" and "hint" messages to a frontend message, much as we have long had in the backend. Except where rewording was needed to convert existing coding to detail/hint style, I have (mostly) resisted the temptation to change existing message wording. Patch by me. Design and patch reviewed at various stages by Robert Haas, Kyotaro Horiguchi, Peter Eisentraut and Daniel Gustafsson. Discussion: https://postgr.es/m/[email protected]
2022-01-08Update copyright for 2022Bruce Momjian
Backpatch-through: 10
2021-07-29Add missing exit() in pg_verifybackup when failing to find pg_waldumpMichael Paquier
pg_verifybackup needs by default pg_waldump to check after a range of WAL segments required for a backup, except if --no-parse-wal is specified. The code checked for the presence of the binary pg_waldump in an installation and reported an error, but it forgot to properly exit(). This could lead to confusing errors reported. Reviewed-by: Robert Haas, Fabien Coelho Discussion: https://postgr.es/m/[email protected] Backpatch-through: 13
2021-02-23Simplify printing of LSNsPeter Eisentraut
Add a macro LSN_FORMAT_ARGS for use in printf-style printing of LSNs. Convert all applicable code to use it. Reviewed-by: Ashutosh Bapat <[email protected]> Reviewed-by: Kyotaro Horiguchi <[email protected]> Reviewed-by: Michael Paquier <[email protected]> Discussion: https://www.postgresql.org/message-id/flat/CAExHW5ub5NaTELZ3hJUCE6amuvqAtsSxc7O+uK7y4t9Rrk23cw@mail.gmail.com
2021-01-02Update copyright for 2021Bruce Momjian
Backpatch-through: 9.5
2020-12-07Fix fd leak in pg_verifybackupMichael Paquier
An error code path newly-introduced by 87ae969 forgot to close a file descriptor when verifying a file's checksum. Per report from Coverity, via Tom Lane.
2020-12-02Move SHA2 routines to a new generic API layer for crypto hashesMichael Paquier
Two new routines to allocate a hash context and to free it are created, as these become necessary for the goal behind this refactoring: switch the all cryptohash implementations for OpenSSL to use EVP (for FIPS and also because upstream does not recommend the use of low-level cryptohash functions for 20 years). Note that OpenSSL hides the internals of cryptohash contexts since 1.1.0, so it is necessary to leave the allocation to OpenSSL itself, explaining the need for those two new routines. This part is going to require more work to properly track hash contexts with resource owners, but this not introduced here. Still, this refactoring makes the move possible. This reduces the number of routines for all SHA2 implementations from twelve (SHA{224,256,386,512} with init, update and final calls) to five (create, free, init, update and final calls) by incorporating the hash type directly into the hash context data. The new cryptohash routines are moved to a new file, called cryptohash.c for the fallback implementations, with SHA2 specifics becoming a part internal to src/common/. OpenSSL specifics are part of cryptohash_openssl.c. This infrastructure is usable for more hash types, like MD5 or HMAC. Any code paths using the internal SHA2 routines are adapted to report correctly errors, which are most of the changes of this commit. The zones mostly impacted are checksum manifests, libpq and SCRAM. Note that e21cbb4 was a first attempt to switch SHA2 to EVP, but it lacked the refactoring needed for libpq, as done here. This patch has been tested on Linux and Windows, with and without OpenSSL, and down to 1.0.1, the oldest version supported on HEAD. Author: Michael Paquier Reviewed-by: Daniel Gustafsson Discussion: https://postgr.es/m/[email protected]
2020-09-24Standardize the printf format for st_sizePeter Eisentraut
Existing code used various inconsistent ways to printf struct stat's st_size member. The type of that is off_t, which is in most cases a signed 64-bit integer, so use the long long int format for it.
2020-09-14Message fixes and style improvementsPeter Eisentraut
2020-05-14Initial pgindent and pgperltidy run for v13.Tom Lane
Includes some manual cleanup of places that pgindent messed up, most of which weren't per project style anyway. Notably, it seems some people didn't absorb the style rules of commit c9d297751, because there were a bunch of new occurrences of function calls with a newline just after the left paren, all with faulty expectations about how the rest of the call would get indented.
2020-05-08Unify find_other_exec() error messagesPeter Eisentraut
There were a few different ways to line-wrap the error messages. Make them all the same, and use placeholders for the actual program names, to save translation work.
2020-05-02Add NLS to pg_verifybackupPeter Eisentraut
2020-04-23Fix option related issues in pg_verifybackup.Fujii Masao
This commit does: - get rid of the garbage code for unused --print-parse-wal option. - add help message for --quiet option into usage(). - fix typo of option name in help message. Author: Fujii Masao Reviewed-by: Robert Haas Discussion: https://postgr.es/m/[email protected]
2020-04-12Rename pg_validatebackup to pg_verifybackup.Robert Haas
Also, use "verify" rather than "validate" to refer to the process being undertaken here. Per discussion, that is a more appropriate term. Discussion: https://www.postgresql.org/message-id/[email protected] Discussion: http://postgr.es/m/CA+TgmobLgMh6p8FmLbj_rv9Uhd7tPrLnAyLgGd2SoSj=qD-bVg@mail.gmail.com