Data pipeline in MS Fabric fails to read objects in Amazon S3 bucket with TLS error

[shingot]

I'm using Microsoft Fabric and setting up a data pipeline to fetch data from Amazon S3 bucket. Connection test is successful but pipeline job fails with the following error.

ErrorCode=S3OperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=S3 operation failed for: A WebException with status TrustFailure was thrown..,Source=Microsoft.DataTransfer.ClientLibrary.MultipartBinaryConnector,''Type=Amazon.Runtime.AmazonServiceException,Message=A WebException with status TrustFailure was thrown.,Source=AWSSDK.Core,''Type=System.Net.WebException,Message=The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.,Source=System,''Type=System.Security.Authentication.AuthenticationException,Message=The remote certificate is invalid according to the validation procedure.,Source=System,'

The connectiom is configured with an account key of an IAM user identity which has permissions to read objects. When I use aws cli, I can read data in the S3 bucket but Fabric cannot.

Does anyone know how can I fix this?

[shingot]

I managed to find a workaround by keepng service URL blank in connection setting. I initially set it to the URL for S3 bucket but it does not seem mandatory so I removed it. Instead, we can set the S3 URL in pipeline job.

[rafaelromero]

Thanks you @shingot , the work-around aproach worked.

[RH205]

Hi,

Same problem here..how did you do this exactly? Because I cannot leave it empty in the connection settings as it results in an error. 

[MLNS]

I just recently experience same error from Azure ADF connecting to AWS S3 bucket.  it use to work just fine but the error started on August 09

[GraceGu]

Would you report a support ticket?

[shingot]

@GraceGu No I didn't as I found a workaround. Please see my comment on ‎08-22-2023. It's confusing so I hope Microsoft Fabric team could improve this.

[zehmenezes]

the workaround you said: i'm leaving service URL empty but doesn't know where to set in "pipeline job". Can you help me? Thanks a lot.

[GraceGu]

@shingot Thanks for the feeback. Great that you have work-around. I will follow up on how we can improve.