S3Express: Amazon S3 Command Line Utility

Amazon S3 command line utility for Windows. Copy, query, backup multiple files to S3 using the Windows command line.

S3Express FAQ and Knowledge Base

Main Page > Browse Categories > How-To > How to upload only changed or new files since the last upload to S3 (-onlydiff)

How to upload only changed or new files since the last upload to S3 (-onlydiff)

The following is an example of the put command that can be used to upload only changed or new files since the last upload to an Amazon S3 bucket:

put c:\myfolder\ my_bucket_name -s -onlydiff

where:

c:\myfolder\ : the local folder to upload.
my_bucket_name : the S3 bucket name.
-s : include subfolders too (= recursive).
-onlydiff : only upload files that are different compared to the matching files that are already in the S3 bucket. Different files are files that have the same path, and the same name, but a different MD5 value. Different files are also files that are not yet uploaded to S3, i.e. new files. So using the '-onlydiff' flag will upload files that are not yet on S3 plus all the files whose content has changed compared to the files already on S3.

The -onlydiff flag can be used to perform incremental uploads / backups to Amazon S3 from the command line. S3Express will firstly compare the local files vs the corresponding remote files and will then only upload files whose MD5 value is different or files that do not already have a corresponding matching file on S3 (that is, new files).

Other flags that could be used instead of -onlydiff are :

-onlynewer : only upload files that are newer compared to the matching files that are already on S3. Newer files are files that have the same path and the same name but a newer modified time. Newer files are also files that are not yet uploaded to S3. So using the '-onlynewer' flag uploads files that are not yet on S3 plus all the files whose timestamp is newer compared to files already on S3.
The difference between -onlynewer and -onlydiff is that -onlydiff uses the MD5 value to compare files, e.g. files are different if the MD5 value is different, while -onlynewer uses the file timestamp, e.g. a file is different if the timestamp is different.

-onlydifferent : only upload files that are new, that is not yet on S3. Using -onlynew only uploads files that are not yet on S3.

-onlyexisting : only upload files that are already existing on S3. Using -onlyexisting only uploads files that already have a corresponding matching file with same name and path on S3.