S3Client with UploadPartCommand is not working after updating to the latest SDK version from 3.583.0

[ssafayet]

Checkboxes for prior research

• I've gone through Developer Guide and API reference
• I've checked AWS Forums and StackOverflow.
• I've searched for previous similar issues and didn't find any solution.

Describe the bug

I have updated my SDK js library to 3.738.0 from 3.583.0 and now it is throwing an error after sending it via S3Client using UploadPartCommand.

Params look like this:

new UploadPartCommand({
    Bucket: <BUCKET>
    Key: <KEY>,
    UploadId: <UPLOAD_ID>
    PartNumber: 3,    
    ContentLength: 20992280,
    Body: <FileStream>
})

Error:

An error was encountered in a non-retryable streaming request.
S3ServiceException [InvalidChunkSizeError]: Only the last chunk is allowed to have a size less than 8192 bytes

But this is not the last chunk. It is the number 3 chunk of 11 chunks. This has also failed for subsequent chunks.

Regression Issue

• Select this option if this issue appears to be a regression.

SDK version number

@aws-sdk/[email protected]

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

20.13.1

Reproduction Steps

Try to create a createMultipartUpload using s3 client and then try to upload FileStream using UploadPartCommand.

Observed Behavior

This error should occur:

An error was encountered in a non-retryable streaming request.
S3ServiceException [InvalidChunkSizeError]: Only the last chunk is allowed to have a size less than 8192 bytes

Expected Behavior

FileStream should be uploaded correctly.

Possible Solution

No response

Additional Information/Context

No response

[aBurmeseDev]

Hi @ssafayet - thanks for reaching out.

I'm not able to reproduce this error with reported version on my end. Could you share your code snippet that shows entire Multipart Upload which includes CreateMultipartUploadCommand, UploadPartCommand, CompleteMultipartUploadCommand, AbortMultipartUploadCommand?

Any code changes since your last working version?

[abustany]

We also get the same error since upgrading from 3.713.0 to 3.738.0. In our case, we use a regular PutObjectCommand where the body is a nodejs readable stream (NodeJS version 20.18.1 on linux x64). Could this issue be the JS cousin of aws/aws-sdk-cpp#3132 ?

[ssafayet]

@abustany Yes, same here. Only for Readable Stream. If I convert the readable stream to buffer, the error goes away.

@aBurmeseDev, I will share a code snippet to reproduce this.

[ssafayet]

@aBurmeseDev Hi, Here is my REPO with the reproducible code. Kindly follow the readme file to reproduce the issue. Let me know if you need any help. Just to mention that if you change back the s3 client version to 3.583.0 in that repo it starts working again.

[macourteau]

I'm experiencing the same issue. If it's any help, I've narrowed it down to working with 3.726.0 and failing with 3.729.0.

[ssafayet]

@macourteau That's right. Confirming that this is also the case on my side. It works till 3.726.1 then it breaks from 3.729.0

[aBurmeseDev]

Thanks everyone for reporting, I was able to reproduce using @ssafayet repro although the error occurs intermittently with different file sizes. I'll bring this up internally for further investigation to understand these inconsistencies and determine the root cause.