Skip to content

Conversation

@legoguy1000
Copy link
Contributor

What does this PR do?

adds teh fingerprint processor to deduplicate events for Cloudflare LogPull when using multiple agents.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@legoguy1000 legoguy1000 requested a review from a team as a code owner April 25, 2022 15:43
@elasticmachine
Copy link

elasticmachine commented Apr 25, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-04-26T21:57:38.651+0000

  • Duration: 15 min 40 sec

Test stats 🧪

Test Results
Failed 0
Passed 96
Skipped 0
Total 96

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@efd6
Copy link
Contributor

efd6 commented Apr 25, 2022

/test

@efd6
Copy link
Contributor

efd6 commented Apr 25, 2022

Please run elastic-package build.

@legoguy1000
Copy link
Contributor Author

Please run elastic-package build.

I did multiple times and it hasn't made any changes. I normally use the latest Elastic Package build from main branch and even changed to the latest tag, v0.47.0 and still no changes from whats already in the branch.

[git-generate]
go install github.com/elastic/elastic-package
cd packages/cloudflare
elastic-package build
@andrewkroh andrewkroh added enhancement New feature or request Team:Security-External Integrations Integration:cloudflare Cloudflare (Community supported) labels Apr 26, 2022
@elasticmachine
Copy link

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@andrewkroh
Copy link
Member

I did multiple times and it hasn't made any changes. I normally use the latest Elastic Package build from main branch and even changed to the latest tag, v0.47.0 and still no changes from whats already in the branch.

Double check the version you have installed with elastic-package version. From this branch it had changes for me. I used:

gh pr checkout 3187
go install github.com/elastic/elastic-package
cd packages/cloudflare
elastic-package build

@andrewkroh
Copy link
Member

/test

@legoguy1000
Copy link
Contributor Author

@andrewkroh Ya installing elastic package from the go install command instead of building from source appeared to do it.

@andrewkroh
Copy link
Member

/test

@elasticmachine
Copy link

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (2/2) 💚
Files 100.0% (3/3) 💚 2.918
Classes 100.0% (3/3) 💚 2.918
Methods 100.0% (36/36) 💚 12.106
Lines 95.17% (670/704) 👍 6.492
Conditionals 100.0% (0/0) 💚

@CyberTaoFlow
Copy link

How exactly does fingerprint de-duplicate events when using a data-stream? My experience has historically (before data-streams) been that the fingerprint becomes the document id which you then either upsert or update any existing document with that ID.

However with data-streams that is not going to work as the actions are limited to create when targeting the data-stream alias right?

From the elasticsearch datastream documentation...

If needed, you can update or delete documents by submitting requests directly to the document’s backing index.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Integration:cloudflare Cloudflare (Community supported)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants