Skip to content

db: fix copy compaction logic for tables with blobs #4780

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

RaduBerinde
Copy link
Member

db: set up the blob value fetcher for internal scans

We also add a safeguard to the code to fall back on the error fetcher
when it is not set, and we update the error fetcher to panic in
invariant builds.

db: fix copy compaction logic for tables with blobs

The condition checking for blob references was incorrect; it caused
move compactions that shouldn't be allowed. We fix the logic and
refactor the code so it's more readable.

We also add a safeguard to the code to fall back on the error fetcher
when it is not set, and we update the error fetcher to panic in
invariant builds.
The condition checking for blob references was incorrect; it caused
move compactions that shouldn't be allowed. We fix the logic and
refactor the code so it's more readable.
@RaduBerinde RaduBerinde requested review from jbowens and annrpom May 28, 2025 19:27
@RaduBerinde RaduBerinde requested a review from a team as a code owner May 28, 2025 19:27
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Collaborator

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 5 of 5 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @annrpom)


compaction.go line 481 at r2 (raw file):

	// Shared and external tables can always be moved. We can also move a local
	// table unless we need the result to be on shared storage.
	if isRemote || !preferSharedStorage {

if isRemote=true and preferSharedStorage=false, wouldn't we want to rewrite the table in an ordinary compaction? maybe this isn't possible unless you previously had shared storage enabled and then disable it?

Copy link
Member Author

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR!

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @annrpom and @jbowens)


compaction.go line 481 at r2 (raw file):

Previously, jbowens (Jackson Owens) wrote…

if isRemote=true and preferSharedStorage=false, wouldn't we want to rewrite the table in an ordinary compaction? maybe this isn't possible unless you previously had shared storage enabled and then disable it?

It is possible to get an ingestion of a shared table that goes to L0 (at least in the metamorphic test - probably not with CRDB as we'd always excise). I think since the file is shared already, there's no problem keeping it shared until it goes further down in the LSM.

@RaduBerinde RaduBerinde merged commit 71b1568 into cockroachdb:master May 29, 2025
6 checks passed
@RaduBerinde RaduBerinde deleted the internal-scan-lazy-value-crash branch May 29, 2025 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants