[couchbase] Add Integration Package with Cluster Data Stream

[bhagyaraj-crest]

• Enhancement

What does this PR do?

• Generated the skeleton of Couchbase integration package.
• Added 1 data stream ( cluster )
• Added data collection logic.
• Added the ingest pipelines.
• Mapped fields according to the ECS schema and added Fields metadata in the appropriate yml files.
• Added system test cases.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• If I'm introducing a new feature, I have modified the Kibana version constraint in my package's manifest.yml file to point to the latest Elastic stack release (e.g. ^7.13.0).

How to test this PR locally

• Clone integrations repo.
• Install elastic-package locally.
• Start elastic stack using elastic-package.
• Move to integrations/packages/couchbase directory.
• Run the following command to run tests.

elastic-package test

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-08-12T06:06:26.284+0000

• Duration: 28 min 4 sec

Test stats 🧪

Test	Results
Failed	0
Passed	12
Skipped	0
Total	12

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[yug-rajani]

/test

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (2/2)	💚
Files	100.0% (2/2)	💚 2.901
Classes	100.0% (2/2)	💚 2.901
Methods	100.0% (14/14)	💚 10.905
Lines	100.0% (203/203)	💚 8.461
Conditionals	100.0% (0/0)	💚

[yug-rajani]

LGTM!

[yug-rajani]

Hey @ManojS-shetty, this PR is kept in the draft stage as the PR for the first data stream (#3666) is waiting for review from the team. Otherwise, from the development perspective, this PR is completed. Feel free to review the same and provide your feedback.

[ManojS-shetty]

@bhagyaraj-crest I do not see any kibana dashboards here.Because of that i am not seeing any assets in integration page.

[bhagyaraj-crest]

@bhagyaraj-crest I do not see any kibana dashboards here.Because of that i am not seeing any assets in integration page.

The reason for not shipping the Kibana dashboards here is because there is a combined OOTB dashboard for bucket, cluster and nodes which we can either ship along with the last PR (nodes) or include as a part of a separate follow-up PR. Please let us know whatever suits the best for you. We have had the same discussion with the team earlier so that we can keep the PRs small, concise and easier to review, and agreed upon it. Please let me know if you have any other concerns with the PR.

[ManojS-shetty]

@bhagyaraj-crest Is there any specific requirement for the Operating system to run the system Test? i am running on the Mac OS docker desktop setup and the system test are not working for me as the pulling and bringing up couchbase container is failing on my laptop. Will this couchbase docker image supports Mac OS?

[yug-rajani]

@bhagyaraj-crest Is there any specific requirement for the Operating system to run the system Test? i am running on the Mac OS docker desktop setup and the system test are not working for me as the pulling and bringing up couchbase container is failing on my laptop. Will this couchbase docker image supports Mac OS?

Hey @ManojS-shetty, we try running the tests locally on our CentOS 7 machines and the tests are working fine. I see the tags mentioned in the Couchbase Docker image (Linux, ARM 64, x86-64), but I believe the image should be platform independent. I think @ritalwar was also facing some similar issues which were specific to Mac before.

Could you please share more details on the issue that you are facing?

[ManojS-shetty]

@bhagyaraj-crest Is there any specific requirement for the Operating system to run the system Test? i am running on the Mac OS docker desktop setup and the system test are not working for me as the pulling and bringing up couchbase container is failing on my laptop. Will this couchbase docker image supports Mac OS?

Hey @ManojS-shetty, we try running the tests locally on our CentOS 7 machines and the tests are working fine. I see the tags mentioned in the Couchbase Docker image (Linux, ARM 64, x86-64), but I believe the image should be platform independent. I think @ritalwar was also facing some similar issues which were specific to Mac before.

Could you please share more details on the issue that you are facing?

Hi @bhagyaraj-crest

I am seeing the below error while running the command elastic-package test system -v its failing while bringing up the couchbase container. As far i can see green in CI i do not have any problem with system test.

[yug-rajani]

@bhagyaraj-crest Is there any specific requirement for the Operating system to run the system Test? i am running on the Mac OS docker desktop setup and the system test are not working for me as the pulling and bringing up couchbase container is failing on my laptop. Will this couchbase docker image supports Mac OS?

Hey @ManojS-shetty, we try running the tests locally on our CentOS 7 machines and the tests are working fine. I see the tags mentioned in the Couchbase Docker image (Linux, ARM 64, x86-64), but I believe the image should be platform independent. I think @ritalwar was also facing some similar issues which were specific to Mac before.
Could you please share more details on the issue that you are facing?

Hi @bhagyaraj-crest

I am seeing the below error while running the command elastic-package test system -v its failing while bringing up the couchbase container. As far i can see green in CI i do not have any problem with system test.

Hey @ManojS-shetty, I tried googling this issue quickly and found a similar discussion on one of the Stack Overflow questions. The issue seems to be specific to Mac.
https://stackoverflow.com/questions/64221861/an-error-failed-to-solve-with-frontend-dockerfile-v0

This was one of the proposed solutions out there, you might want to try it out.

export DOCKER_BUILDKIT=0
export COMPOSE_DOCKER_CLI_BUILD=0

However, as you mentioned, let's get this PR out as the CI is green so that we can go ahead with the next set of data streams. Please feel free to comment if you have any other concerns with the changes made as a part of this PR.

[kush-elastic]

Thanks for the approval, @ManojS-shetty! Just FYI, the current PR will also have changes similar to the follow up PR for bucket data stream as the changes are in the main manifest file which is common for all the integrations.
Hence, let's keep this PR approved at this point of time and we can ship it once the above mentioned PR is out.