python - Retrieve data from previous version of file in Amazon S3 Version

Question

I have a strange use case where within production tables, values are overwritten entirely rather than creating a new record for each edit made to the record in the table. IT has decided not to change the architecture to allow us to audit the table over time.

Our workaround is to have versions of the table saved in an AWS S3 bucket. I am wondering if it is possible to retrieve data from a previous version and combine that same data with the current version? Thereby creating our own ability to audit changes in the table over time.

A	Version Number
A1	3
A2	2
A3	1

Answer 1

Assuming you just want to merge two versions of the object and create a new one( I trimmed the versioned_objects response):

    In [2]: versioned_objects = s3.list_object_versions('myversionedbucket')
    In [4]: versioned_objects
    Out[4]:
    {'ResponseMetadata': {,,
    'Versions': [{',
    'Key': 'test.json',
    'VersionId': 'Bx1atH2cYGErrIsd_A2ApmMPFdIFo4c0',
    'IsLatest': True,
    },
    {
    'Key': 'test.json',
    'VersionId': 'BNE6HyQ38lWhb.iLIMt6sMJE1RcaG1j2',
    'IsLatest': False,
    },
    {,

    'Key': 'test.json',
    'VersionId': 'bYu_OzyXdRLLtYLaHqgnf.srvKpZmI3F',
    'Name': 'myversionedbucket',
      }
    }


    In [17]: current_version = json.loads(s3.get_object(Bucket='myversionedbucket', Key='test.json', VersionId='Bx1atH2cYGErrIsd_A2ApmMPFdIFo4c0')['Body'].read())

    In [18]: older_version = json.loads(s3.get_object(Bucket='myversionedbucket', Key='test.json', VersionId='BNE6HyQ38lWhb.iLIMt6sMJE1RcaG1j2')['Body'].read())

    In [19]: current_version
    Out[19]: {'name': 'markus', 'lastname': 'schneider'}

    In [20]: older_version
    Out[20]: {'name': 'markus'}


    In [31]: merged = {**current_version, **older_version}

    In [32]: merged
    Out[32]: {'name': 'markus', 'lastname': 'schneider'}