-
Notifications
You must be signed in to change notification settings - Fork 25.2k
Handle streaming request body in audit log #127798
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle streaming request body in audit log #127798
Conversation
The audit event for a successfully-authenticated REST request occurs when we start to process the request. For APIs that accept a streaming request body this means we have received the request headers, but not its body, at the time of the audit event. Today such requests will fail with a `ClassCastException`. This change fixes the handling of streaming requests in the audit log to now report that the request body was not available when writing the audit entry.
Pinging @elastic/es-security (Team:Security) |
Hi @DaveCTurner, I've created a changelog YAML for you. |
@@ -27,6 +27,9 @@ public class AuditUtil { | |||
|
|||
public static String restRequestContent(RestRequest request) { | |||
if (request.hasContent()) { | |||
if (request.isStreamedContent()) { | |||
return "Request body had not been received at the time of the audit event"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we want to eventually support this use case? If not, can we provide more details here? Something like
Audit logging with request body is not supported when the request is streamed. To disable request streaming, set [rest.incremental_bulk] to [false].
What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes I think eventually we'll need this for streaming requests (because eventually we will be handling all requests as streaming). It's not reasonable to log an arbitrarily-large body in a single audit event tho, instead we will need to record each chunk in the audit log as they arrive.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also note that today because of how we try and log the whole body in a single message we end up truncating it anyway after a few kiB even with rest.incremental_bulk: true
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for explaining. The fix here is certainly a lot better than throwing ClassCastException. But we may still want to create an issue to say a future fix is pending?
we end up truncating it anyway after a few kiB
IIRC, we don't truncate audit logs. At least payloads of a few hundred KB are fully logged.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I started a design doc and opened ES-11760
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IIRC, we don't truncate audit logs
You're right, TIL. And yet we seem to truncate other logs messages emitted by Log4J. I wonder why (but not hard enough to go digging further).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
The audit event for a successfully-authenticated REST request occurs when we start to process the request. For APIs that accept a streaming request body this means we have received the request headers, but not its body, at the time of the audit event. Today such requests will fail with a `ClassCastException` if the `emit_request_body` flag is set. This change fixes the handling of streaming requests in the audit log to now report that the request body was not available when writing the audit entry.
The audit event for a successfully-authenticated REST request occurs when we start to process the request. For APIs that accept a streaming request body this means we have received the request headers, but not its body, at the time of the audit event. Today such requests will fail with a `ClassCastException` if the `emit_request_body` flag is set. This change fixes the handling of streaming requests in the audit log to now report that the request body was not available when writing the audit entry.
The audit event for a successfully-authenticated REST request occurs when we start to process the request. For APIs that accept a streaming request body this means we have received the request headers, but not its body, at the time of the audit event. Today such requests will fail with a `ClassCastException` if the `emit_request_body` flag is set. This change fixes the handling of streaming requests in the audit log to now report that the request body was not available when writing the audit entry.
💔 Backport failed
You can use sqren/backport to manually backport by running |
The audit event for a successfully-authenticated REST request occurs when we start to process the request. For APIs that accept a streaming request body this means we have received the request headers, but not its body, at the time of the audit event. Today such requests will fail with a `ClassCastException` if the `emit_request_body` flag is set. This change fixes the handling of streaming requests in the audit log to now report that the request body was not available when writing the audit entry. Backport of elastic#127798 to `8.17`
8.17 backport is #127843 |
The audit event for a successfully-authenticated REST request occurs when we start to process the request. For APIs that accept a streaming request body this means we have received the request headers, but not its body, at the time of the audit event. Today such requests will fail with a `ClassCastException` if the `emit_request_body` flag is set. This change fixes the handling of streaming requests in the audit log to now report that the request body was not available when writing the audit entry.
* Handle streaming request body in audit log The audit event for a successfully-authenticated REST request occurs when we start to process the request. For APIs that accept a streaming request body this means we have received the request headers, but not its body, at the time of the audit event. Today such requests will fail with a `ClassCastException` if the `emit_request_body` flag is set. This change fixes the handling of streaming requests in the audit log to now report that the request body was not available when writing the audit entry. Backport of #127798 to `8.17` * Enable incremental bulks in AuditIT
* Handle streaming request body in audit log (#127798) The audit event for a successfully-authenticated REST request occurs when we start to process the request. For APIs that accept a streaming request body this means we have received the request headers, but not its body, at the time of the audit event. Today such requests will fail with a `ClassCastException` if the `emit_request_body` flag is set. This change fixes the handling of streaming requests in the audit log to now report that the request body was not available when writing the audit entry. * Enable incremental bulks in AuditIT
* Handle streaming request body in audit log (#127798) The audit event for a successfully-authenticated REST request occurs when we start to process the request. For APIs that accept a streaming request body this means we have received the request headers, but not its body, at the time of the audit event. Today such requests will fail with a `ClassCastException` if the `emit_request_body` flag is set. This change fixes the handling of streaming requests in the audit log to now report that the request body was not available when writing the audit entry. * Enable incremental bulks in AuditIT
The audit event for a successfully-authenticated REST request occurs when we start to process the request. For APIs that accept a streaming request body this means we have received the request headers, but not its body, at the time of the audit event. Today such requests will fail with a `ClassCastException` if the `emit_request_body` flag is set. This change fixes the handling of streaming requests in the audit log to now report that the request body was not available when writing the audit entry.
The audit event for a successfully-authenticated REST request occurs when we start to process the request. For APIs that accept a streaming request body this means we have received the request headers, but not its body, at the time of the audit event. Today such requests will fail with a `ClassCastException` if the `emit_request_body` flag is set. This change fixes the handling of streaming requests in the audit log to now report that the request body was not available when writing the audit entry.
The audit event for a successfully-authenticated REST request occurs when we start to process the request. For APIs that accept a streaming request body this means we have received the request headers, but not its body, at the time of the audit event. Today such requests will fail with a `ClassCastException` if the `emit_request_body` flag is set. This change fixes the handling of streaming requests in the audit log to now report that the request body was not available when writing the audit entry.
The audit event for a successfully-authenticated REST request occurs
when we start to process the request. For APIs that accept a streaming
request body this means we have received the request headers, but not
its body, at the time of the audit event. Today such requests will fail
with a
ClassCastException
if theemit_request_body
flag is set. Thischange fixes the handling of streaming requests in the audit log to now
report that the request body was not available when writing the audit
entry.