Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
initial commit
  • Loading branch information
emnp committed Oct 18, 2022
commit 7b7eca3bac3e718e35d1b4116fbfda1092e9c90e
3 changes: 3 additions & 0 deletions packages/trendmicro/_dev/build/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
dependencies:
ecs:
reference: [email protected]
16 changes: 16 additions & 0 deletions packages/trendmicro/_dev/build/docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Trendmicro Integration

This is an integration for parsing trendmicro events. It can accept
data over syslog or read it from a file.

## Compatibility

The trendmicro deepsecurity log has been developed against Deep Security 12 Long-Term Support but is expected to work with other versions.

## Logs

# Deep Security Logs

Deep Security logs collect the trendmicro deep security logs.

{{fields "deep_security"}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{{fields "deep_security"}}

What is the goal here ? {{fields "deep_security"}} is not parsable here and will cause errors, please remove it and add relevant info. If you want to add the fields .. then please add them manually or use json tags .

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ShourieG , yes actually I would like to add the fields and I think elastic-package build command will render this file packages/trendmicro/_dev/build/docs/README.md to this file packages/trendmicro/docs/README.md. The fields {{fields "deep_security"}} are parsed in this file packages/trendmicro/docs/README.md.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fields are correct, but you need to use the system tests to generate a test sample, we can help with this @emnp @ShourieG

8 changes: 8 additions & 0 deletions packages/trendmicro/_dev/deploy/docker/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
version: '2.3'
services:
trendmicro-log-logfile:
image: alpine
volumes:
- ./sample_logs:/sample_logs:ro
- ${SERVICE_LOGS_DIR}:/var/log
command: /bin/sh -c "cp /sample_logs/* /var/log/"
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
CEF:0|Trend Micro|Deep Security Agent|10.2.229|4000000|Eicar_test_file|6|cn1=1 cn1Label=Host ID dvchost=hostname cn2=205 cn2Label=Quarantine File Size cs6=ContainerImageName | ContainerName | ContainerID cs6Label=Container filePath=C:\\Users\\trend\\Desktop\\eicar.exe act=Delete msg=Realtime TrendMicroDsMalwareTarget=N/A TrendMicroDsMalwareTargetType=N/A TrendMicroDsFileMD5=44D88612FEA8A8F36DE82E1278ABB02F TrendMicroDsFileSHA1=3395856CE81F2B7382DEE72602F798B642F14140 TrendMicroDsFileSHA256=275A021BBFB6489E54D471899F7DB9D1663FC695EC2FE2A2C4538AABF651FD0F TrendMicroDsDetectionConfidence=95 TrendMicroDsRelevantDetectionNames=Ransom_CERBER.BZC;Ransom_CERBER.C;Ransom_CRYPNISCA.SM
CEF:0|Trend Micro|Deep Security Agent|10.2.229|5000000|WebReputation|5|cn1=1 cn1Label=Host ID dvchost=hostname request=example.com msg=Blocked By Admin
CEF:0|Trend Micro|Deep Security Agent|10.2.229|6001200|AppControl detectOnly|6|cn1=202 cn1Label=Host ID dvc=192.168.33.128 TrendMicroDsTenant=Primary TrendMicroDsTenantId=0 fileHash=80D4AC182F97D2AB48EE4310AC51DA5974167C596D133D64A83107B9069745E0 suser=root suid=0 act=detectOnly filePath=/home/user1/Desktop/Directory1//heartbeatSync.sh fsize=20 aggregationType=0 repeatCount=1 cs1=notWhitelisted cs1Label=actionReason cs2=0CC9713BA896193A527213D9C94892D41797EB7C cs2Label=sha1 cs3=7EA8EF10BEB2E9876D4D7F7E5A46CF8D cs3Label=md5
6 changes: 6 additions & 0 deletions packages/trendmicro/changelog.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# newer versions go on top
- version: "0.0.1"
changes:
- description: Initial draft of the package
type: enhancement
link: https://github.com/elastic/integrations/pull/1 # FIXME Replace with the real PR link
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#dynamic_fields:
# event.ingested: ".*"
fields:
tags:
- preserve_original_event
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
CEF:0|Trend Micro|Deep Security Manager|10.2.229|600|User Signed In|3|src=10.52.116.160 suser=admin target=admin msg=User signed in from 2001:db8::5
CEF:0|Trend Micro|Deep Security Agent|10.2.229|4000000|Eicar_test_file|6|cn1=1 cn1Label=Host ID dvchost=hostname cn2=205 cn2Label=Quarantine File Size cs6=ContainerImageName | ContainerName | ContainerID cs6Label=Container filePath=C:\\Users\\trend\\Desktop\\eicar.exe act=Delete msg=Realtime TrendMicroDsMalwareTarget=N/A TrendMicroDsMalwareTargetType=N/TrendMicroDsFileMD5=44D88612FEA8A8F36DE82E1278ABB02F TrendMicroDsFileSHA1=3395856CE81F2B7382DEE72602F798B642F14140 TrendMicroDsFileSHA256=275A021BBFB6489E54D471899F7DB9D1663FC695EC2FE2A2C4538AABF651FD0F TrendMicroDsDetectionConfidence=95 TrendMicroDsRelevantDetectionNames=Ransom_CERBER.BZC;Ransom_CERBER.C;Ransom_CRYPNISCA.SM
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
{
"expected": [
{
"ecs": {
"version": "8.4.0"
},
"error": {
"message": "For input string: \\\"\\\""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This parsing seems to have errors?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got this when I test the pipeline but it worked well when I did system test and also when I tried to send the log file with elastic agent, it worked well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@emnp the issue seems to be in signature_id which is being set to event.code, but the value of event.code is never populated in the pipeline. This is making signature_id as empty string and causing pipeline test failures

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved by changing pipeline tests log structure as it should be the step after cef_decode processor

},
"event": {
"ingested": "2022-10-18T09:04:35.311147327Z"
},
"tags": [
"preserve_original_event"
],
"trendmicro": {
"event": {
"message": "CEF:0|Trend Micro|Deep Security Manager|10.2.229|600|User Signed In|3|src=10.52.116.160 suser=admin target=admin msg=User signed in from 2001:db8::5",
"signature_id": ""
}
}
},
{
"ecs": {
"version": "8.4.0"
},
"error": {
"message": "For input string: \\\"\\\""
},
"event": {
"ingested": "2022-10-18T09:04:35.311175329Z"
},
"tags": [
"preserve_original_event"
],
"trendmicro": {
"event": {
"message": "CEF:0|Trend Micro|Deep Security Agent|10.2.229|4000000|Eicar_test_file|6|cn1=1 cn1Label=Host ID dvchost=hostname cn2=205 cn2Label=Quarantine File Size cs6=ContainerImageName | ContainerName | ContainerID cs6Label=Container filePath=C:\\\\Users\\\\trend\\\\Desktop\\\\eicar.exe act=Delete msg=Realtime TrendMicroDsMalwareTarget=N/A TrendMicroDsMalwareTargetType=N/TrendMicroDsFileMD5=44D88612FEA8A8F36DE82E1278ABB02F TrendMicroDsFileSHA1=3395856CE81F2B7382DEE72602F798B642F14140 TrendMicroDsFileSHA256=275A021BBFB6489E54D471899F7DB9D1663FC695EC2FE2A2C4538AABF651FD0F TrendMicroDsDetectionConfidence=95 TrendMicroDsRelevantDetectionNames=Ransom_CERBER.BZC;Ransom_CERBER.C;Ransom_CRYPNISCA.SM",
"signature_id": ""
}
}
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
vars: ~
service: trendmicro-log-logfile
input: logfile
data_stream:
vars:
paths:
- "{{SERVICE_LOGS_DIR}}/trendmicro.log"
decode_trendmicro_timezone: UTC
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
paths:
{{#each paths as |path i|}}
- {{path}}
{{/each}}
exclude_files: [".gz$"]
tags:
{{#if preserve_original_event}}
- preserve_original_event
{{/if}}
{{#each tags as |tag i|}}
- {{tag}}
{{/each}}
{{#contains "forwarded" tags}}
publisher_pipeline.disable_host: true
{{/contains}}
processors:
- rename:
fields:
- {from: "message", to: "event.original"}
- decode_cef:
field: event.original
{{#if decode_trendmicro_timezone}}
timezone: {{ decode_trendmicro_timezone }}
{{/if}}
{{#if processors}}
{{processors}}
{{/if}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
host: "{{syslog_host}}:{{syslog_port}}"
tags:
{{#if preserve_original_event}}
- preserve_original_event
{{/if}}
{{#each tags as |tag i|}}
- {{tag}}
{{/each}}
{{#contains "forwarded" tags}}
publisher_pipeline.disable_host: true
{{/contains}}
processors:
- rename:
fields:
- {from: "message", to: "event.original"}
- decode_cef:
field: event.original
{{#if decode_cef_timezone}}
timezone: {{ decode_cef_timezone }}
{{/if}}
{{#if processors}}
{{processors}}
{{/if}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
description: Pipeline for processing Application Control event logs
processors:
- set:
field: trendmicro.event.category
value: "app-control"
- script:
lang: painless
params:
extensions:
# Rename field
- name: deviceCustomNumber1
to: host.id
- name: deviceCustomString1
to: trendmicro.event.action_reason
- name: deviceCustomString3
to: file.hash.md5
- name: deviceCustomString2
to: file.hash.sha1
- name: fileHash
to: file.hash.sha256
- name: deviceAddress
to: host.ip
- name: deviceHostName
to: host.hostname
- name: TrendMicroDsTags
to: trendmicro.event.tags
- name: TrendMicroDsTenantId
to: trendmicro.event.tenant_id
- name: TrendMicroDsTenant
to: trendmicro.event.tenant_name
- name: aggregationType
to: trendmicro.event.aggregation_type
convert:
"0": The event is not aggregated
"1": The event is aggregated based on file name,path,and event type
"2": The event is aggregated based on event type
- name: repeatCount
to: trendmicro.event.count
#- name: event.severity
# to: log.syslog.severity.name
# convert:
# '0': emergency
# '1': alert
# '2': critical
# '3': error
# '4': warning
# '5': notice
# '6': informational
# '7': debug

source: |
def actions = new ArrayList();
def exts = ctx.cef?.extensions;
if (exts == null) return;
for (entry in params.extensions) {
def value = exts[entry.name];
if (value == null ||
(entry.convert != null &&
(value=entry.convert[value.toLowerCase()]) == null))
continue;
if (entry.to != null) {
actions.add([
"value": value,
"to": entry.to
]);
continue;
}
def label = exts[entry.name + "Label"];
if (label == null) continue;
def dest = entry.labels[label.toLowerCase()];
if (dest == null) continue;
actions.add([
"value": value,
"to": dest
]);
}
ctx["_tmp_copy"] = actions;
- foreach:
field: _tmp_copy
processor:
set:
field: "{{_ingest._value.to}}"
value: "{{_ingest._value.value}}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add ignore_missing to avoid failure


- remove:
field:
- _tmp_copy

on_failure:
- set:
field: error.message
value: "{{ _ingest.on_failure_message }}"
Loading