Skip to content

ssi_all: use triple-brace templating in complex templates #11314

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 3, 2024

Conversation

efd6
Copy link
Contributor

@efd6 efd6 commented Oct 3, 2024

Proposed commit message

This is a second pass at replacing double brace template marks with triple brace
marks. It addresses more complex uses that the sed rewrite did not find by using
PCRE via perl.

[git-generate]
for f in $(
	(
		for p in $(
			yq 'select(.owner.github == "elastic/security-service-integrations")|.name' packages/**/manifest.yml \
			| grep -v -- '---'
		); do
			rg -l -g '*.yml' "[^{]\{\{[^{][ .a-zA-Z0-9_]*[^}]}}[^}]" packages/$p
		done
	)|grep -v "cloudflare"|grep "elasticsearch/ingest_pipeline"|sort|uniq
); do
	perl -pi -e 's/(?<!\{)(\{\{[^{][ .a-zA-Z0-9_]*[^}]}})(?!\})/{$1}/g' $f
done
for p in $(git diff --name-only HEAD~1|cut -d/ -f1,2|sort|uniq); do
	(
		cd $p
		elastic-package test pipeline -g
		elastic-package changelog add \
			--description "Use triple-brace Mustache templating when referencing variables in ingest pipelines." \
			--type bugfix \
			--next patch \
			--link https://github.com/elastic/integrations/pull/
	)>/dev/null 2>&1
done

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@efd6 efd6 added Integration:All Applies to all integrations [Integration not found in source] bugfix Pull request that fixes a bug issue Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Oct 3, 2024
@efd6 efd6 self-assigned this Oct 3, 2024
This is a second pass at replacing double brace template marks with triple brace
marks. It addresses more complex uses that the sed rewrite did not find by using
PCRE via perl.

[git-generate]
for f in $(
	(
		for p in $(
			yq 'select(.owner.github == "elastic/security-service-integrations")|.name' packages/**/manifest.yml \
			| grep -v -- '---'
		); do
			rg -l -g '*.yml' "[^{]\{\{[^{][ .a-zA-Z0-9_]*[^}]}}[^}]" packages/$p
		done
	)|grep -v "cloudflare"|grep "elasticsearch/ingest_pipeline"|sort|uniq
); do
	perl -pi -e 's/(?<!\{)(\{\{[^{][ .a-zA-Z0-9_]*[^}]}})(?!\})/{$1}/g' $f
done
for p in $(git diff --name-only HEAD~1|cut -d/ -f1,2|sort|uniq); do
	(
		cd $p
		elastic-package test pipeline -g
		elastic-package changelog add \
			--description "Use triple-brace Mustache templating when referencing variables in ingest pipelines." \
			--type bugfix \
			--next patch \
			--link elastic#11314
	)>/dev/null 2>&1
done
@efd6 efd6 force-pushed the 7641-ssi_all_complex branch from 410ae7b to c43a37b Compare October 3, 2024 02:33
@elastic-vault-github-plugin-prod

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@elasticmachine
Copy link

💚 Build Succeeded

History

  • 💔 Build #16737 failed 410ae7b97b799b72aee5bcd3303c94153af03f82

cc @efd6

Copy link

Quality Gate failed Quality Gate failed

Failed conditions
51.7% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

@andrewkroh andrewkroh removed the Integration:All Applies to all integrations [Integration not found in source] label Oct 3, 2024
@efd6 efd6 marked this pull request as ready for review October 3, 2024 03:06
@efd6 efd6 requested a review from a team as a code owner October 3, 2024 03:06
@elasticmachine
Copy link

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@andrewkroh andrewkroh added Integration:slack Slack Logs (Community supported) Integration:ti_abusech AbuseCH Integration:atlassian_jira Atlassian Jira (Community supported) Integration:bitdefender BitDefender (Community supported) Integration:gigamon Gigamon (Partner supported) Integration:mimecast Mimecast (Partner supported) Integration:crowdstrike CrowdStrike Integration:symantec_endpoint Symantec Endpoint Protection Integration:akamai Akamai (Community supported) Integration:sysdig Sysdig Integration:falco Falco Integration:ti_maltiverse Maltiverse (Partner supported) Integration:github GitHub Integration:zoom Zoom Integration:cisa_kevs CISA Known Exploited Vulnerabilities (Community supported) labels Oct 3, 2024
@elastic-vault-github-plugin-prod

Package atlassian_jira - 1.27.2 containing this change is available at https://epr.elastic.co/search?package=atlassian_jira

@elastic-vault-github-plugin-prod

Package aws_bedrock - 0.10.1 containing this change is available at https://epr.elastic.co/search?package=aws_bedrock

@elastic-vault-github-plugin-prod

Package bitdefender - 2.1.3 containing this change is available at https://epr.elastic.co/search?package=bitdefender

@elastic-vault-github-plugin-prod

Package cisa_kevs - 1.3.1 containing this change is available at https://epr.elastic.co/search?package=cisa_kevs

@elastic-vault-github-plugin-prod

Package cisco_duo - 2.0.1 containing this change is available at https://epr.elastic.co/search?package=cisco_duo

@elastic-vault-github-plugin-prod

Package cisco_umbrella - 1.26.2 containing this change is available at https://epr.elastic.co/search?package=cisco_umbrella

@elastic-vault-github-plugin-prod

Package crowdstrike - 1.42.2 containing this change is available at https://epr.elastic.co/search?package=crowdstrike

@elastic-vault-github-plugin-prod

Package falco - 1.0.2 containing this change is available at https://epr.elastic.co/search?package=falco

@elastic-vault-github-plugin-prod

Package forcepoint_web - 1.9.1 containing this change is available at https://epr.elastic.co/search?package=forcepoint_web

@elastic-vault-github-plugin-prod

Package gigamon - 1.0.1 containing this change is available at https://epr.elastic.co/search?package=gigamon

@elastic-vault-github-plugin-prod

Package github - 1.29.3 containing this change is available at https://epr.elastic.co/search?package=github

@elastic-vault-github-plugin-prod

Package keycloak - 1.23.2 containing this change is available at https://epr.elastic.co/search?package=keycloak

@elastic-vault-github-plugin-prod

Package mimecast - 1.27.2 containing this change is available at https://epr.elastic.co/search?package=mimecast

@elastic-vault-github-plugin-prod

Package netskope - 1.20.2 containing this change is available at https://epr.elastic.co/search?package=netskope

@elastic-vault-github-plugin-prod

Package okta - 2.12.2 containing this change is available at https://epr.elastic.co/search?package=okta

@elastic-vault-github-plugin-prod

Package opencanary - 0.1.3 containing this change is available at https://epr.elastic.co/search?package=opencanary

@elastic-vault-github-plugin-prod

Package qualys_vmdr - 5.2.1 containing this change is available at https://epr.elastic.co/search?package=qualys_vmdr

@elastic-vault-github-plugin-prod

Package slack - 1.21.2 containing this change is available at https://epr.elastic.co/search?package=slack

@elastic-vault-github-plugin-prod

Package symantec_endpoint - 2.16.1 containing this change is available at https://epr.elastic.co/search?package=symantec_endpoint

@elastic-vault-github-plugin-prod

Package sysdig - 0.1.1 containing this change is available at https://epr.elastic.co/search?package=sysdig

@elastic-vault-github-plugin-prod

Package thycotic_ss - 1.8.1 containing this change is available at https://epr.elastic.co/search?package=thycotic_ss

@elastic-vault-github-plugin-prod

Package ti_abusech - 2.3.5 containing this change is available at https://epr.elastic.co/search?package=ti_abusech

@elastic-vault-github-plugin-prod

Package ti_custom - 0.1.3 containing this change is available at https://epr.elastic.co/search?package=ti_custom

@elastic-vault-github-plugin-prod

Package ti_cybersixgill - 1.30.5 containing this change is available at https://epr.elastic.co/search?package=ti_cybersixgill

@elastic-vault-github-plugin-prod

Package ti_maltiverse - 1.2.5 containing this change is available at https://epr.elastic.co/search?package=ti_maltiverse

@elastic-vault-github-plugin-prod

Package ti_misp - 1.35.5 containing this change is available at https://epr.elastic.co/search?package=ti_misp

@elastic-vault-github-plugin-prod

Package zoom - 1.20.1 containing this change is available at https://epr.elastic.co/search?package=zoom

harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 4, 2025
)

This is a second pass at replacing double brace template marks with triple brace
marks. It addresses more complex uses that the sed rewrite did not find by using
PCRE via perl.

[git-generate]
for f in $(
	(
		for p in $(
			yq 'select(.owner.github == "elastic/security-service-integrations")|.name' packages/**/manifest.yml \
			| grep -v -- '---'
		); do
			rg -l -g '*.yml' "[^{]\{\{[^{][ .a-zA-Z0-9_]*[^}]}}[^}]" packages/$p
		done
	)|grep -v "cloudflare"|grep "elasticsearch/ingest_pipeline"|sort|uniq
); do
	perl -pi -e 's/(?<!\{)(\{\{[^{][ .a-zA-Z0-9_]*[^}]}})(?!\})/{$1}/g' $f
done
for p in $(git diff --name-only HEAD~1|cut -d/ -f1,2|sort|uniq); do
	(
		cd $p
		elastic-package test pipeline -g
		elastic-package changelog add \
			--description "Use triple-brace Mustache templating when referencing variables in ingest pipelines." \
			--type bugfix \
			--next patch \
			--link elastic#11314
	)>/dev/null 2>&1
done
harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 5, 2025
)

This is a second pass at replacing double brace template marks with triple brace
marks. It addresses more complex uses that the sed rewrite did not find by using
PCRE via perl.

[git-generate]
for f in $(
	(
		for p in $(
			yq 'select(.owner.github == "elastic/security-service-integrations")|.name' packages/**/manifest.yml \
			| grep -v -- '---'
		); do
			rg -l -g '*.yml' "[^{]\{\{[^{][ .a-zA-Z0-9_]*[^}]}}[^}]" packages/$p
		done
	)|grep -v "cloudflare"|grep "elasticsearch/ingest_pipeline"|sort|uniq
); do
	perl -pi -e 's/(?<!\{)(\{\{[^{][ .a-zA-Z0-9_]*[^}]}})(?!\})/{$1}/g' $f
done
for p in $(git diff --name-only HEAD~1|cut -d/ -f1,2|sort|uniq); do
	(
		cd $p
		elastic-package test pipeline -g
		elastic-package changelog add \
			--description "Use triple-brace Mustache templating when referencing variables in ingest pipelines." \
			--type bugfix \
			--next patch \
			--link elastic#11314
	)>/dev/null 2>&1
done
@efd6 efd6 deleted the 7641-ssi_all_complex branch February 5, 2025 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix Pull request that fixes a bug issue Integration:akamai Akamai (Community supported) Integration:atlassian_bitbucket Atlassian Bitbucket (Community supported) Integration:atlassian_jira Atlassian Jira (Community supported) Integration:aws_bedrock Amazon Bedrock Integration:bitdefender BitDefender (Community supported) Integration:cisa_kevs CISA Known Exploited Vulnerabilities (Community supported) Integration:cisco_duo Cisco Duo Integration:cisco_umbrella Cisco Umbrella Integration:crowdstrike CrowdStrike Integration:falco Falco Integration:forcepoint_web Forcepoint Web Security (Community supported) Integration:gigamon Gigamon (Partner supported) Integration:github GitHub Integration:keycloak Keycloak (Community supported) Integration:mimecast Mimecast (Partner supported) Integration:netskope Netskope Integration:okta Okta Integration:opencanary OpenCanary (Community supported) Integration:qualys_vmdr Qualys VMDR Integration:slack Slack Logs (Community supported) Integration:symantec_endpoint Symantec Endpoint Protection Integration:sysdig Sysdig Integration:thycotic_ss Thycotic Secret Server (Community supported) Integration:ti_abusech AbuseCH Integration:ti_custom Custom Threat Intelligence Integration:ti_cybersixgill Cybersixgill Integration:ti_maltiverse Maltiverse (Partner supported) Integration:ti_misp MISP Integration:zoom Zoom Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants