Jump to content

[TOPIC: topicViewTemplate]
[GLOBAL: userSmallPhoto]
Photo

Secure File Transfers with Amazon S3
Started by Develephant Sep 01 2017 09:38 AM

- - - - -
12 replies to this topic
amazon files upload download cloud secure storage
[TOPIC CONTROLS]
[/TOPIC CONTROLS]
[modOptionsDropdown]
[/modOptionsDropdown]
[reputationFilter]
[TOPIC: post.html]
#1

Develephant

[GLOBAL: userInfoPane.html]
Develephant
  • Corona Geek

  • 1,440 posts
  • Corona SDK

Hi,
 
I've just released a new plugin that allows developers of Corona apps and games to securly transfer files using the Amazon Simple Storage Service (Amazon S3).
 
The S3 Lite plugin is available in the Corona Marketplace.
 
To learn more about the plugin api, visit the documentation site.
 
-dev

  • roaminggamer and sporkfin like this

[TOPIC: post.html]
#2

nmichaud

[GLOBAL: userInfoPane.html]
nmichaud
  • Contributor

  • 492 posts
  • Corona SDK

Hi Chris,

 

Thanks for this wonderful plugin. I just bought it and will start integrating it in a few projects.

 

Best,
Nick


  • Develephant likes this

[TOPIC: post.html]
#3

sporkfin

[GLOBAL: userInfoPane.html]
sporkfin
  • Contributor

  • 393 posts
  • Corona SDK

Keep those products rolling out and count me in!

 

Thanks Chris!  

-Jonathan


  • Develephant likes this

[TOPIC: post.html]
#4

John_M

[GLOBAL: userInfoPane.html]
John_M
  • Enthusiast

  • 89 posts
  • Corona SDK

Hi Thanks for the great S3 Lite plugin!

 

I'm getting this error (the output from this line of code:  print(evt.error, evt.message, evt.status))

 

               InfoFailed   Could not obtain bucket meta    400

 

This is the code i"m using. My bucket has AmazonS3FullAccess. permission for now

 

local function onPutObject( evt )
              if evt.error then
                    print("error:")
                    print(evt.error, evt.message, evt.status)
              else
                    if evt.progress then
                        print("progress:"..evt.progress)
                    else
                        print("object upload complete")
                    end
              end
        end
        local params9={
            
            storage=s3.STANDARD
        }
        
        
        print("doing s3:putObject now")
        s3:putObject(
          system.TemporaryDirectory,
          "photoToUpload.jpg",
          "MYBUCKETNAMEWASHERE",
          "photoToUpload.jpg",
          onPutObject,
            params9
        )
 

 

What does that error mean?

 

Thanks in advance for your help!



[TOPIC: post.html]
#5

Develephant

[GLOBAL: userInfoPane.html]
Develephant
  • Corona Geek

  • 1,440 posts
  • Corona SDK

Hi,

 

Sorry for the troubles. Not sure exactly what that error is offhand. I'm looking into it. Some questions:

 

  • Does the bucket already exist on S3?
  • Are you able to perform other operations, like downloading?
  • Are you using the correct region?
  • What platform are you on?

 

-dev



[TOPIC: post.html]
#6

John_M

[GLOBAL: userInfoPane.html]
John_M
  • Enthusiast

  • 89 posts
  • Corona SDK

Hi thanks for the fast reply. I'm on Android, yep the bucket exists already in S3. Good idea, I'll try other operations to see if those work. I don't actually recall specifying the region in the 'constructor' call for the s3 object. I'll check as soon at I get back on my pc.

I guess this error message is generated within the plugin code? I couldn't find any mention of that text in any Google searches so I'm guessing that message is not from s3?

Thanks again

[TOPIC: post.html]
#7

John_M

[GLOBAL: userInfoPane.html]
John_M
  • Enthusiast

  • 89 posts
  • Corona SDK

You're right, I see this in your documentation
s3:new({
  key = "aws-key-1234",
  secret = "aws-secret-abcd",
  region = s3.EU_WEST_1
})


I might have forgotten to change the region when I cut/pasted that in. I definitely changed the key and secret but I don't remember setting the region. Sorry about that, I'll try again with the correct region and let you know how that goes!

[TOPIC: post.html]
#8

Develephant

[GLOBAL: userInfoPane.html]
Develephant
  • Corona Geek

  • 1,440 posts
  • Corona SDK

Hi,

 

Let me know. All the errors returned are coming from Amazon, but I too had very little luck finding that one on the net. A putObject request first checks the bucket looking at the meta data via a call to Amazons API to check if the file exists so that is probably where that error is coming from, but I need to dig into the code tomorrow if the region doesn't work out for you.

 

-dev



[TOPIC: post.html]
#9

Develephant

[GLOBAL: userInfoPane.html]
Develephant
  • Corona Geek

  • 1,440 posts
  • Corona SDK

Hi,

 

I actually take that back, I may be massaging some of the errors to make them more human readable. The 400 would indicate a resource not being found that's why my suspicion is on the region. The plugin defaults to US East, so if that's not where your bucket is, then that is probably the issue.

 

-dev



[TOPIC: post.html]
#10

John_M

[GLOBAL: userInfoPane.html]
John_M
  • Enthusiast

  • 89 posts
  • Corona SDK

Thanks, it worked!!!  You guys are awesome!  Thanks for solving it so fast, your first response had exactly what it was as one of the bullets... the region.  Thank you again.

 

I checked my code, I had actually left the region out, so I only had the key and the secret as keys passed in to the constructor. The plugin docs showed region as not required so I hadn't realized I needed it. (I'm new to S3, I didn't realize buckets wouldn't be found without a region specified too to the S3 side. I see you mention if we don't explicitly state one the plugin defaults to US East).  Thanks again!



[TOPIC: post.html]
#11

John_M

[GLOBAL: userInfoPane.html]
John_M
  • Enthusiast

  • 89 posts
  • Corona SDK

One last thought... Are you really familiar with S3?  It took maybe a full minute or two for the uploaded image to actually show up in the AWS S3 console. I guess that's because of the S3 replication since it takes a bit of time till it propagates to all of their servers. But if my app itself uploads the image and then GETs it right after to show what has been uploaded, would the app see the image right away? (kind of like when you upload an image to Instagram, it shows in the app within a second or 2)


I see this in the S3 docs:

S3 provides read-after-write consistency for PUTS of new objects in your S3 bucket in all regions with one caveat. The caveat is that if you make a HEAD or GET request to the key name (to find if the object exists) before creating the object, Amazon S3 provides eventual consistency for read-after-write.

Amazon S3 offers eventual consistency for overwrite PUTS and DELETES in all regions

 

I guess this is really a question for the S3 forum, but I was just wondering what your experience has been with it via the plugin

 

If you're not sure no problem, I'll post elsewhere. I guess if I read the above literally regarding the read-after-write consistency I should be ok. The caveat regarding eventual consistency just had me a little worried.

 

Anyway thanks again for the awesome plugin. I was hunting around a while for an easier way to upload images (which I definitely found with this plugin).  I hit a roadblock with PUTs to my hosting server using network.upload() (my server host doesn't allow PUTs), and POSTs had lots of complications due to multi-part form data, so your plugin is turning out to be an excellent alternative!. I'll mention that in the forum posts regarding image uploads to help spread the word for you



[TOPIC: post.html]
#12

Develephant

[GLOBAL: userInfoPane.html]
Develephant
  • Corona Geek

  • 1,440 posts
  • Corona SDK

Hi,

 

I'm glad to hear everything worked out. And thanks for using the plugin.

 

So, I assume by your post that you are using the S3 replication feature? The best way to answer your question, would be to do a test, and try to download the image right after you upload. In the demo project, the test image seems to return right away, but that is not using replication.

 

If the replication is a problem, I may be able to provide an override flag for checking the bucket. That functionality is in there to help ensure a developer doesn't overwrite an image by accident.

 

-dev



[TOPIC: post.html]
#13

John_M

[GLOBAL: userInfoPane.html]
John_M
  • Enthusiast

  • 89 posts
  • Corona SDK

Hi I didn't actually enable replication explicitly, I'm just using their "standard" storage classification which seems to be their default.  Their docs just mention they use replication behind-the-scenes within one region to provide Amazon's amazing high durability and availability.  I think their other storage classifications use redundancy/replication too but some to a lesser degree. I'll do a test within the app itself as you recommend.  Most likely it will work out great.

 

PS I mentioned your plugin in a few of the other posts where people ran into problems with multipart form posts for image uploads since I was having the same problem and found your plugin much easier to use. Hopefully it will help those others out too!

 

Thanks again for being so responsive


  • Develephant likes this


[topic_controls]
[/topic_controls]

Also tagged with one or more of these keywords: amazon, files, upload, download, cloud, secure, storage