Uploading Files

We all know the evils of uploading unknown files to the site root. But what about to the directory above the root?

I’m investigating ways to allow people to upload files into what will essentially be a sandbox where they will get scanned or whatever before going live. I found a handy script that allows for uploading to a secure Dropbox folder. I like it. But I’m also curious about using my own server.

Doable? Or just too risky without a lot of serious hoop-jumping?

Todd
http://xiiro.com


dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

If you know who is uploading (make sure they have to log in first) then there’s really not much of an issue. Yes, sandbox the files somewhere outside of the Web root, so there’s no way to do this:

  1. Upload file containing script.
  2. Link to that script in a browser, which causes it to execute.
  3. Profit!

But most importantly, when you’re allowing these files to upload, stash their metadata in a database along with the user ID of who uploaded it. That way you know who to blame later.

What upload handler system are you using? In Rails, both of the really popular ones (Paperclip and Carrierwave) have input scanners that look inside the file to see if it really is what its extension says it is. You supply a whitelist of file types you will accept, and anything else just throws an error and goes in the /dev/null bit bucket.

Walter

On Nov 8, 2014, at 4:50 PM, Todd email@hidden wrote:

We all know the evils of uploading unknown files to the site root. But what about to the directory above the root?

I’m investigating ways to allow people to upload files into what will essentially be a sandbox where they will get scanned or whatever before going live. I found a handy script that allows for uploading to a secure Dropbox folder. I like it. But I’m also curious about using my own server.

Doable? Or just too risky without a lot of serious hoop-jumping?

Todd
http://xiiro.com


dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options


dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

For the sake of clarity perhaps I should offer some background and broad stroke what I envision happening. If there’s a better way I’m open to suggestions.

When I started outlining this community project one of the many expected technical hurdles I knowingly stepped into was how to allow anyone to publish content, inc. downloadable tutorial example files, without adding them to the back-end as a (very) restricted user. That said, there will be trusted users (some from this list) who will have user-access so these issues do not apply to them. In any case there are several ways I could approach it, and yes, a member area is one way.

One part of the solution I’ve researched and intend (hope) to implement which I find so very über cool (Walter, you especially may appreciate this aspect) is using the submitted form content to dynamically create a back-end resource (i.e. webpage), select the proper template based on the content, and fill-in the appropriate variables of the page. Now I have an unpublished tutorial/article/review ready to go with minimal effort by the contributor and without having to build a member area. I only wish any media could be placed inline and sent that way, all nice and tidy.

The problem (in addition to the security issue) is when the contributor wants to add a file for download. If the main content gets sent to the back-end for processing but file attachments need to get sandboxed (Dropbox, my server etc.) how does that work without imposing a 2-step upload process: one for content, one for files. Not ideal. On top of which I need a way to match uploaded files with posts. Perhaps there’s a way to “tag” them so the site admin knows what goes with what.

Now, back to the sandbox, I assume there are tools that will watch a directory and scan them for malicious code without having to manually do it?

Todd

If you know who is uploading (make sure they have to log in first) then there’s really not much of an issue. Yes, sandbox the files somewhere outside of the Web root, so there’s no way to do this:

  1. Upload file containing script.
  2. Link to that script in a browser, which causes it to execute.
  3. Profit!

But most importantly, when you’re allowing these files to upload, stash their metadata in a database along with the user ID of who uploaded it. That way you know who to blame later.

What upload handler system are you using? In Rails, both of the really popular ones (Paperclip and Carrierwave) have input scanners that look inside the file to see if it really is what its extension says it is. You supply a whitelist of file types you will accept, and anything else just throws an error and goes in the /dev/null bit bucket.

Walter

On Nov 8, 2014, at 4:50 PM, Todd email@hidden wrote:

We all know the evils of uploading unknown files to the site root. But what about to the directory above the root?

I’m investigating ways to allow people to upload files into what will essentially be a sandbox where they will get scanned or whatever before going live. I found a handy script that allows for uploading to a secure Dropbox folder. I like it. But I’m also curious about using my own server.

Doable? Or just too risky without a lot of serious hoop-jumping?

Todd
http://xiiro.com


dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options


dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options


dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

I’ve build these sorts of divided forms before using S3 for storage. There’s a way to do this there, where your form first submits in your application to save the metadata, which includes some “nonce” value. Then you submit a different form to S3 through a keyhole iframe, and trigger an overlay with an animated GIF to signal that the file is uploading. The response from that form includes your nonce, so you know which form completed, and you then break out of the iframe and close the uploading animation. (The second request is asynchronous – you don’t know how long it will take to upload the files.) I have some code I can dig out for you on this, but not until late tonight.

Walter


dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

Thanks, Walter. There’s no rush but I would certainly be interested in seeing it at some point. I’m intentionally putting off building these forms because I think it will be my biggest headache, technically and UX-wise.

Todd

I’ve build these sorts of divided forms before using S3 for storage. There’s a way to do this there, where your form first submits in your application to save the metadata, which includes some “nonce” value. Then you submit a different form to S3 through a keyhole iframe, and trigger an overlay with an animated GIF to signal that the file is uploading. The response from that form includes your nonce, so you know which form completed, and you then break out of the iframe and close the uploading animation. (The second request is asynchronous – you don’t know how long it will take to upload the files.) I have some code I can dig out for you on this, but not until late tonight.


dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

Are you using S3 as a sandbox or for some other non-security purpose?

Todd
http://xiiro.com http://xiiro.com/

I’ve build these sorts of divided forms before using S3 for storage. There’s a way to do this there, where your form first submits in your application to save the metadata, which includes some “nonce” value. Then you submit a different form to S3 through a keyhole iframe, and trigger an overlay with an animated GIF to signal that the file is uploading.


dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

In the site I built, I was using it because it is nearly infinite in size (4GB per upload) and these were movies that people were uploading.

Walter

On Nov 10, 2014, at 6:16 PM, Todd email@hidden wrote:

Are you using S3 as a sandbox or for some other non-security purpose?

Todd
http://xiiro.com http://xiiro.com/

I’ve build these sorts of divided forms before using S3 for storage. There’s a way to do this there, where your form first submits in your application to save the metadata, which includes some “nonce” value. Then you submit a different form to S3 through a keyhole iframe, and trigger an overlay with an animated GIF to signal that the file is uploading.


dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options


dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

Oh, nice. I can see where that would be useful for movies.

Todd
http://xiiro.com http://xiiro.com/

In the site I built, I was using it because it is nearly infinite in size (4GB per upload) and these were movies that people were uploading.


dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options