Support for input and output filter for batch transform in sagemaker Local Mode

[lorenzwalthert]

Describe the feature you'd like

Support for input and output filter for batch transform in sagemaker local mode. Currently, these filters only work when executed in AWS infra, but not in local mode (I can provide a reproducible example if needed). Checking the documentation, I could also not find any hint that the feature is not supported in the local mode.

How would this feature be used? Please describe.

Enable frictionless development in local mode, in particular for pipelines with multiple steps that involve a batch transform job.

Describe alternatives you've considered

• Add more processing steps before and after the transform job to do the filtering / merging.
• Selective pipeline execution while relaying on a previous transform job that was ran on AWS.

Additional context

The issue was raised before in aws-samples/amazon-sagemaker-local-mode#22 and the OP was directed to this repo, but to my knowledge, did not open an issue here.

cc: @eitansela.

[ShwetaSingh801]

Hi @lorenzwalthert,

Thanks for using SageMaker and taking the time to suggest ways to improve SageMaker Python SDK. We have added your feature request it to our backlog of feature requests and may consider putting it into future SDK versions. I will go ahead and close the issue now, please let me know if you have any more feedback. Let me know if you have any other questions.

Best,
Shweta