Transfer File From Aws S3 To Sftp Using Boto 3

I am a beginner in using Boto3 and I would like to transfer a file from an S3 bucket to am SFTP server directly. My final goal is to write a Python script for AWS Glue. I have foun

Solution 1:

"transfer ... directly" can mean number of different things.

Let's assume that you want to transfer the file via the local machine (where the Python code runs), without actually storing a temporary copy of the file to the local file system.

---

For SFTP upload, you can use Paramiko library.

Assuming you already have your Paramiko SFTPClient (sftp) and Boto 3 client (s3) instances ready (what is covered in the article you have linked in your question), you can simply "glue" them together using file-like objects:

with sftp.open('/sftp/path/filename', 'wb', 32768) as f:
    s3.download_fileobj('mybucket', 'mykey', f)

For the purpose of the 32768 argument, see Writing to a file on SFTP server opened using pysftp "open" method is slow.