db

package
v0.0.0-...-463cb6c Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 20, 2026 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func BuildArrayIndexOfDBResult

func BuildArrayIndexOfDBResult[R any, K comparable](dbi Interface, keyFunc func(R) K, query string, args ...any) (result map[K][]R, err error)

BuildArrayIndexOfDBResult executes an SQL query and returns a map (index) of the result. The key should not be unique among the whole result set

func BuildIndexOfDBResult

func BuildIndexOfDBResult[R any, K comparable](dbi Interface, keyFunc func(R) K, query string, args ...any) (result map[K]R, err error)

BuildIndexOfDBResult executes an SQL query and returns a map (index) of the result. The key should be unique among the whole result set.

func BuildSimpleWhereClause

func BuildSimpleWhereClause(fields map[string]any, parameterOffset int) (queryFragment string, queryArgs []any)

BuildSimpleWhereClause constructs a WHERE clause of the form "field1 = val1 AND field2 = val2 AND field3 IN (val3, val4)".

If parameterOffset is not 0, start counting placeholders ("$1", "$2", etc.) after that offset.

func Configuration

func Configuration() easypg.Configuration

Configuration returns the easypg.Configuration object that func Init() needs to initialize the DB connection.

func ExpandEnumPlaceholders

func ExpandEnumPlaceholders(query string) string

ExpandEnumPlaceholders takes an SQL query literal from the source code and replaces placeholders {{like.This}} with SQL string literals 'like-this'. The placeholder must refer to an enum variant one of the following types:

  • liquid.CommitmentStatus

Canonical usage looks like this:

var query = sqlext.SimplifyWhitespace(db.ExpandEnumPlaceholders(`
	...
`))

func Init

func Init() (*sql.DB, error)

Init initializes the connection to the database.

func InitORM

func InitORM(dbConn *sql.DB) *gorp.DbMap

InitORM wraps a database connection into a gorp.DbMap instance.

func RunOLAPQueries

func RunOLAPQueries(dbm *gorp.DbMap, action func(tx *gorp.Transaction) error) error

RunOLAPQueries executes a DB transaction with increased `work_mem` setting. As the name implies, this is useful for OLAP queries that perform expensive joins and aggregations in a way that benefits from having more RAM available than the default.

This should only be used sparingly; each process is only allowed to run two such queries at the same time to limit the total memory usage on the DB server.

Types

type AZResource

type AZResource struct {
	ID               AZResourceID           `db:"id"`
	ResourceID       ResourceID             `db:"resource_id"`
	AvailabilityZone limes.AvailabilityZone `db:"az"`
	// a unique identifier for this record in the form "servicetype/resourcename"; mostly intended for manual lookup
	Path string `db:"path"`

	RawCapacity uint64         `db:"raw_capacity"`
	Usage       Option[uint64] `db:"usage"`
	// ” for az=total
	SubcapacitiesJSON string `db:"subcapacities"`

	// LastNonzeroRawCapacity is None initially, and gets filled whenever capacity scrape sees a non-zero capacity value.
	// We use this as a signal for ACPQ to distinguish new AZs in buildup that should be ignored for the purposes of base quota overcommit,
	// from existing AZs with faulty capacity recording that should block base quota overcommit.
	// None for az=total
	LastNonzeroRawCapacity Option[uint64] `db:"last_nonzero_raw_capacity"`
}

AZResource contains a record from the `az_resources` table.

type AZResourceID

type AZResourceID int64

AZResourceID is an ID into the az_resources table. This typedef is used to distinguish these IDs from IDs of other tables or raw int64 values.

type CommitmentReason

type CommitmentReason string

CommitmentReason is an enum. It appears in type CommitmentWorkflowContext.

const (
	CommitmentReasonCreate  CommitmentReason = "create"
	CommitmentReasonSplit   CommitmentReason = "split"
	CommitmentReasonConvert CommitmentReason = "convert"
	CommitmentReasonMerge   CommitmentReason = "merge"
	CommitmentReasonRenew   CommitmentReason = "renew"
	CommitmentReasonConsume CommitmentReason = "consume"
	// this reason is only intended for use in audit events
	// TODO: audit events should not refer to this enum (maybe structure around liquid.CommitmentChangeRequest instead?)
	CommitmentReasonConfirm CommitmentReason = "confirm"
)

type CommitmentWorkflowContext

type CommitmentWorkflowContext struct {
	Reason                 CommitmentReason        `json:"reason"`
	RelatedCommitmentIDs   []ProjectCommitmentID   `json:"related_ids,omitempty"` // TODO: remove when v1 API is removed (v2 API uses only UUIDs to refer to commitments)
	RelatedCommitmentUUIDs []liquid.CommitmentUUID `json:"related_uuids,omitempty"`
}

CommitmentWorkflowContext is the type definition for the JSON payload in the CreationContextJSON and SupersedeContextJSON fields of type ProjectCommitment.

type Domain

type Domain struct {
	ID   DomainID `db:"id"`
	Name string   `db:"name"`
	UUID string   `db:"uuid"`
}

Domain contains a record from the `domains` table.

type DomainID

type DomainID int64

DomainID is an ID into the domains table. This typedef is used to distinguish these IDs from IDs of other tables or raw int64 values.

type Interface

type Interface interface {
	// from database/sql
	sqlext.Executor

	// from github.com/go-gorp/gorp
	Insert(args ...any) error
	Update(args ...any) (int64, error)
	Delete(args ...any) (int64, error)
	Select(i any, query string, args ...any) ([]any, error)
}

Interface provides the common methods that both SQL connections and transactions implement.

type MailNotification

type MailNotification struct {
	ID                int64     `db:"id"`
	ProjectID         ProjectID `db:"project_id"`
	Subject           string    `db:"subject"`
	Body              string    `db:"body"`
	NextSubmissionAt  time.Time `db:"next_submission_at"`
	FailedSubmissions int64     `db:"failed_submissions"`
}

MailNotification contains a record from the `project_mail_notifications` table.

type Project

type Project struct {
	ID         ProjectID          `db:"id"`
	DomainID   DomainID           `db:"domain_id"`
	Name       string             `db:"name"`
	UUID       liquid.ProjectUUID `db:"uuid"`
	ParentUUID string             `db:"parent_uuid"`
}

Project contains a record from the `projects` table.

type ProjectAZResource

type ProjectAZResource struct {
	ID           ProjectAZResourceID `db:"id"`
	ProjectID    ProjectID           `db:"project_id"`
	AZResourceID AZResourceID        `db:"az_resource_id"`
	// None if hasQuota=false OR (az=total AND topology=az-separated) OR az=unknown
	Quota Option[uint64] `db:"quota"`
	// None if hasQuota=false OR (az=total AND topology=az-separated) OR (az!=total AND topology!=az-separated) OR az=unknown
	BackendQuota  Option[int64]  `db:"backend_quota"`
	Usage         uint64         `db:"usage"`
	PhysicalUsage Option[uint64] `db:"physical_usage"`
	// ” for az=total
	SubresourcesJSON string `db:"subresources"`
	// ” for az=total
	HistoricalUsageJSON string `db:"historical_usage"`
}

ProjectAZResource contains a record from the `project_az_resources` table.

type ProjectAZResourceID

type ProjectAZResourceID int64

ProjectAZResourceID is an ID into the project_az_resources table. This typedef is used to distinguish these IDs from IDs of other tables or raw int64 values.

type ProjectCommitment

type ProjectCommitment struct {
	ID           ProjectCommitmentID               `db:"id"`
	UUID         liquid.CommitmentUUID             `db:"uuid"`
	ProjectID    ProjectID                         `db:"project_id"`
	AZResourceID AZResourceID                      `db:"az_resource_id"`
	Amount       uint64                            `db:"amount"`
	Duration     limesresources.CommitmentDuration `db:"duration"`
	CreatedAt    time.Time                         `db:"created_at"`
	CreatorUUID  string                            `db:"creator_uuid"` // format: "username@userdomainname"
	CreatorName  string                            `db:"creator_name"`
	ConfirmBy    Option[time.Time]                 `db:"confirm_by"`
	ConfirmedAt  Option[time.Time]                 `db:"confirmed_at"`
	ExpiresAt    time.Time                         `db:"expires_at"`

	// Commitments can be superseded due to splits, conversions or merges.
	// The context columns contain information about the reason and related commitments
	SupersededAt         Option[time.Time]       `db:"superseded_at"`
	CreationContextJSON  json.RawMessage         `db:"creation_context_json"`
	SupersedeContextJSON Option[json.RawMessage] `db:"supersede_context_json"`
	RenewContextJSON     Option[json.RawMessage] `db:"renew_context_json"`

	// For a commitment to be transferred between projects, it must first be
	// marked for transfer in the source project. Then a new commitment can be
	// created in the target project to supersede the transferable commitment.
	//
	// While a commitment is marked for transfer, it does not count towards quota
	// calculation, but it still blocks capacity and still counts towards billing.
	TransferStatus limesresources.CommitmentTransferStatus `db:"transfer_status"`
	TransferToken  Option[string]                          `db:"transfer_token"`
	// publicly transferred commitments are ordered by the time of their posting
	TransferStartedAt Option[time.Time] `db:"transfer_started_at"`

	// To a certain extent, this column is technically redundant, since the
	// status can often be derived from the values of other fields. For example,
	// a commitment is in status "superseded" iff `SupersededAt.IsSome()`.
	//
	// However, having this field simplifies lots of queries significantly
	// because we do not need to carry a NOW() argument into the query,
	// and complex conditions like `WHERE superseded_at IS NULL AND expires_at > $now AND confirmed_at IS NULL AND confirm_by < $now`
	// become simple readable conditions like `WHERE status IN ('pending', 'guaranteed')`.
	//
	// This field is updated by the CapacityScrapeJob.
	Status liquid.CommitmentStatus `db:"status"`

	// During commitment planning, a user can specify
	// if a mail should be sent after the commitments confirmation.
	NotifyOnConfirm bool `db:"notify_on_confirm"`

	// If commitments are about to expire, they get added into the mail queue.
	// This attribute helps to identify commitments that are already queued.
	NotifiedForExpiration bool `db:"notified_for_expiration"`
}

ProjectCommitment contains a record from the `project_commitments` table.

type ProjectCommitmentID

type ProjectCommitmentID int64

ProjectCommitmentID is an ID into the project_commitments table. This typedef is used to distinguish these IDs from IDs of other tables or raw int64 values.

type ProjectID

type ProjectID int64

ProjectID is an ID into the projects table. This typedef is used to distinguish these IDs from IDs of other tables or raw int64 values.

type ProjectRate

type ProjectRate struct {
	ID            ProjectRateID             `db:"id"`
	ProjectID     ProjectID                 `db:"project_id"`
	RateID        RateID                    `db:"rate_id"`
	Limit         Option[uint64]            `db:"rate_limit"`      // None for rates that don't have a limit (just a usage)
	Window        Option[limesrates.Window] `db:"window_ns"`       // None for rates that don't have a limit (just a usage)
	UsageAsBigint string                    `db:"usage_as_bigint"` // empty for rates that don't have a usage (just a limit)

}

ProjectRate contains a record from the `project_rates` table.

type ProjectRateID

type ProjectRateID int64

ProjectRateID is an ID into the project_rates table. This typedef is used to distinguish these IDs from IDs of other tables or raw int64 values.

type ProjectResource

type ProjectResource struct {
	ID                       ProjectResourceID `db:"id"`
	ProjectID                ProjectID         `db:"project_id"`
	ResourceID               ResourceID        `db:"resource_id"`
	Forbidden                bool              `db:"forbidden"`
	ForbidAutogrowth         bool              `db:"forbid_autogrowth"`
	MaxQuotaFromOutsideAdmin Option[uint64]    `db:"max_quota_from_outside_admin"`
	OverrideQuotaFromConfig  Option[uint64]    `db:"override_quota_from_config"`
}

ProjectResource contains a record from the `project_resources` table. Quota values are NULL for resources that do not track quota.

type ProjectResourceID

type ProjectResourceID int64

ProjectResourceID is an ID into the project_resources table. This typedef is used to distinguish these IDs from IDs of other tables or raw int64 values.

type ProjectService

type ProjectService struct {
	ID                    ProjectServiceID  `db:"id"`
	ProjectID             ProjectID         `db:"project_id"`
	ServiceID             ServiceID         `db:"service_id"`
	ScrapedAt             Option[time.Time] `db:"scraped_at"` // None if never scraped so far
	CheckedAt             Option[time.Time] `db:"checked_at"`
	NextScrapeAt          time.Time         `db:"next_scrape_at"`
	Stale                 bool              `db:"stale"`
	ScrapeDurationSecs    float64           `db:"scrape_duration_secs"`
	ScrapeErrorMessage    string            `db:"scrape_error_message"`
	SerializedScrapeState string            `db:"serialized_scrape_state"`
	SerializedMetrics     string            `db:"serialized_metrics"`
	QuotaDesyncedAt       Option[time.Time] `db:"quota_desynced_at"` // None if all quota = backend quota
	QuotaSyncDurationSecs float64           `db:"quota_sync_duration_secs"`
}

ProjectService contains a record from the `project_services` table.

type ProjectServiceID

type ProjectServiceID int64

ProjectServiceID is an ID into the project_services table. This typedef is used to distinguish these IDs from IDs of other tables or raw int64 values.

type Rate

type Rate struct {
	ID        RateID          `db:"id"`
	ServiceID ServiceID       `db:"service_id"`
	Name      liquid.RateName `db:"name"`
	// following fields get filled from liquid.ServiceInfo
	LiquidVersion int64           `db:"liquid_version"`
	Unit          liquid.Unit     `db:"unit"`
	Topology      liquid.Topology `db:"topology"`
	HasUsage      bool            `db:"has_usage"`
}

Rate contains a record from the `rates` table.

type RateID

type RateID int64

RateID is an ID into the rates table. This typedef is used to distinguish these IDs from IDs of other tables or raw int64 values.

type Resource

type Resource struct {
	ID        ResourceID          `db:"id"`
	ServiceID ServiceID           `db:"service_id"`
	Name      liquid.ResourceName `db:"name"`
	// a unique identifier for this record in the form "servicetype/resourcename"; mostly intended for manual lookup
	Path string `db:"path"`

	// following fields get filled from liquid.ServiceInfo
	LiquidVersion       int64           `db:"liquid_version"`
	Unit                liquid.Unit     `db:"unit"`
	Topology            liquid.Topology `db:"topology"`
	HasCapacity         bool            `db:"has_capacity"`
	NeedsResourceDemand bool            `db:"needs_resource_demand"`
	HasQuota            bool            `db:"has_quota"`
	AttributesJSON      string          `db:"attributes_json"`
	HandlesCommitments  bool            `db:"handles_commitments"`
}

Resource contains a record from the `resources` table.

type ResourceID

type ResourceID int64

ResourceID is an ID into the resources table. This typedef is used to distinguish these IDs from IDs of other tables or raw int64 values.

type Service

type Service struct {
	ID                 ServiceID         `db:"id"`
	Type               ServiceType       `db:"type"`
	ScrapedAt          Option[time.Time] `db:"scraped_at"` // None if never scraped so far
	ScrapeDurationSecs float64           `db:"scrape_duration_secs"`
	SerializedMetrics  string            `db:"serialized_metrics"`
	NextScrapeAt       time.Time         `db:"next_scrape_at"`
	ScrapeErrorMessage string            `db:"scrape_error_message"`
	// following fields get filled from liquid.ServiceInfo
	LiquidVersion                   int64  `db:"liquid_version"`
	CapacityMetricFamiliesJSON      string `db:"capacity_metric_families_json"`
	UsageMetricFamiliesJSON         string `db:"usage_metric_families_json"`
	UsageReportNeedsProjectMetadata bool   `db:"usage_report_needs_project_metadata"`
	QuotaUpdateNeedsProjectMetadata bool   `db:"quota_update_needs_project_metadata"`
}

Service contains a record from the `services` table.

type ServiceID

type ServiceID int64

ServiceID is an ID into the services table. This typedef is used to distinguish these IDs from IDs of other tables or raw int64 values.

type ServiceType

type ServiceType string

ServiceType identifies a backend service that can have resources or rates.

This type is used for the service type columns that appear in the DB. It is legally distinct from `limes.ServiceType` to ensure that the ResourceBehavior.IdentityInV1API mapping is applied when converting between API-level and DB-level identifiers.

type SetUpdate

type SetUpdate[R any, K comparable] struct {
	// All relevant records that currently exist in the DB.
	ExistingRecords []R
	// All keys for which we want to have a record in the DB.
	WantedKeys []K

	// KeyForRecord reads the key out of an existing record.
	// This does not need to be the primary key.
	// Whatever unique identifier the caller has available is fine.
	KeyForRecord func(R) K

	// Callback for creating a new record for a missing key.
	//
	// After this, the Update callback will also be called on the new record.
	// This avoids code duplication between the create and update callbacks.
	Create func(K) (R, error)
	// Callback for updating an existing record.
	Update func(*R) error
}

SetUpdate describes an operation where we have an existing set of records (type R), and a set of records that we want to have, as identified by some key (type K). Records that we want to keep are updated, missing records are created, and existing records that we do not want to have are deleted.

TODO This is not yet used in most of the places that could benefit from it.

func (SetUpdate[R, K]) Execute

func (u SetUpdate[R, K]) Execute(tx *gorp.Transaction) ([]R, error)

Execute executes this SetUpdate. Returns the set of records that exist in the DB after this update.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL