[DocDB] Duplicate Prometheus Metrics

[yusong-yan]

Jira Link: DB-14463

Description

We observed many server metrics with the same name rpcs_in_queue_yb_master_MasterService

rpcs_in_queue_yb_master_MasterService{exported_instance="dev-n3",metric_type="server",metric_id="yb.master"} 0 1733761038868
rpcs_in_queue_yb_master_MasterService{exported_instance="dev-n3",metric_type="server",metric_id="yb.master"} 0 1733761038868
rpcs_in_queue_yb_master_MasterService{exported_instance="dev-n3",metric_type="server",metric_id="yb.master"} 0 1733761038868
rpcs_in_queue_yb_master_MasterService{exported_instance="dev-n3",metric_type="server",metric_id="yb.master"} 0 1733761038868
rpcs_in_queue_yb_master_MasterService{exported_instance="dev-n3",metric_type="server",metric_id="yb.master"} 0 1733761038868
rpcs_in_queue_yb_master_MasterService{exported_instance="dev-n3",metric_type="server",metric_id="yb.master"} 0 1733761038868
rpcs_in_queue_yb_master_MasterService{exported_instance="dev-n3",metric_type="server",metric_id="yb.master"} 0 1733761038868
rpcs_in_queue_yb_master_MasterService{exported_instance="dev-n3",metric_type="server",metric_id="yb.master"} 0 1733761038868
rpcs_in_queue_yb_master_MasterService{exported_instance="dev-n3",metric_type="server",metric_id="yb.master"} 0 1733761038868

Issue Type

kind/bug

Warning: Please confirm that this issue does not contain any sensitive information

• I confirm this issue does not contain any sensitive information.

[yusong-yan]

A rpcs_in_queue_+<service_name()> metric is created within each of those services’ ServicePool. The reason are seeing these “duplicated metrics” is because, those services all have the same service_name yb.master.MasterService

std::unique_ptr<rpc::ServiceIf> MakeMasterAdminService(Master* master);
std::unique_ptr<rpc::ServiceIf> MakeMasterBackupService(Master* master);
std::unique_ptr<rpc::ServiceIf> MakeMasterClientService(Master* master);
std::unique_ptr<rpc::ServiceIf> MakeMasterClusterService(Master* master);
std::unique_ptr<rpc::ServiceIf> MakeMasterDclService(Master* master);
std::unique_ptr<rpc::ServiceIf> MakeMasterDdlService(Master* master);
std::unique_ptr<rpc::ServiceIf> MakeMasterEncryptionService(Master* master);
std::unique_ptr<rpc::ServiceIf> MakeMasterHeartbeatService(Master* master);
std::unique_ptr<rpc::ServiceIf> MakeMasterReplicationService(Master* master);