3.1.19

How to Stream s3 Object Line by Line

This answer from stackoverflow with 0 vote is actually the best answer in my opinion.

https://stackoverflow.com/questions/7624900/how-can-i-use-boto-to-stream-a-file-out-of-amazon-s3-to-rackspace-cloudfiles

Botocore's StreamingBody has an iter_lines() method:
So:
import boto3
s3r = boto3.resource('s3')
iterator = s3r.Object(bucket, key).get()['Body'].iter_lines()

for line in iterator:
    print(line)

No comments: