1.18.2008

Fixing attachment_fu on Windows

Like many others, I've encountered issue when developing Rails applications using attachment_fu on Windows. After doing some research, I've come up with the following solution to the problem.

The problem has two parts :
  1. Size is not included in the list error message,
  2. Timeout error when uploading to S3.

Fixing "Size is not included in the list" error message

Some people have reported that there is a timing issue, when trying to get the file size, with Tempfile on Windows. It seems that the size of the file is not properly reported by Windows after writing data to it. Proposed solutions for this problem include :
  1. Sleeping in a loop as long as the file size is 0,
  2. Reading back the entire file in memory.

I think I found a better and less patchy solution for this issue: forcing the OS to flush the file to disk before reading it's size.
Here is the code to do it :
require 'tempfile'
class Tempfile
def size
if @tmpfile
@tmpfile.fsync # added this line
@tmpfile.flush
@tmpfile.stat.size
else
0
end
end
end

Doing a flush is not enough... flush will flush the Ruby buffer but the file may not be immediately written to the disk by the OS. Doing the fsync ensure that the file is written to disk by the OS before continuing. After that, Windows will properly report the actual file size.

Fixing the Timeout error when uploading to S3

This issue is related to opening files for reading on Windows. On Windows, you have to open the file in binary mode. So patching attachment_fu is simple :
require 'technoweenie/attachment_fu/backends/s3_backend'
module Technoweenie
module AttachmentFu
module Backends
module S3Backend
protected
def save_to_storage
if save_attachment?
S3Object.store(
full_filename,
(temp_path ? File.open(temp_path, "rb") : temp_data), # added , "rb"
bucket_name,
:content_type => content_type,
:access => attachment_options[:s3_access]
)
end

@old_filename = nil
true
end
end
end
end
end

I've also included a fix from someone else (which was not enough in itself to solve my S3 upload problem):
module Technoweenie
module AttachmentFu
# Gets the data from the latest temp file. This will read the file into memory.
def temp_data
if save_attachment?
f = File.new( temp_path )
f.binmode
return f.read
else
return nil
end
end
end
end

Wrapping it up

So I put all this code in lib/attachment_fu_patch.rb and required it in environment.rb.

Problem fixed!

Note, I did not test it on other OSes, but these fixes should not have any adverse effects.

20 comments:

Anonymous said...

This is a great fix to an issue I have been using sleep for.. Thanks!

Anonymous said...

You should submit this as a patch :)

Anonymous said...

patch is not working.

use

sleep(3) #sleep(5)

instead.

Emmanuel Pirsch said...

To Anonymous,

Maybe you can explain what did not work for you with this fix. What is your setup?

This fix has been working for me (and others).

The sleep(x) fix, will not always wor as it depends on the file size being downloaded. For really large size, you will have to sleep longer as Windows will take more time to empty its cache.

Anonymous said...

As I know this issue is only on Windows. Because my deploy env is linux and my development machine is Windows... how can I patch my Rails code to only affect development env?. Currently I have the patch on environment.rb file. I did the patch on development.rb but server crashes.
Thanks a lot.

Emmanuel Pirsch said...

you can put the require statement in :
if RUBY_PLATFORM =~ /mswin32/
# put the require here.
end

Chris Bloom said...

I'm having a similar problem using the April 2, 2007 version of attachment_fu on Rails 1.2.3 - does this patch require any adjustment for that environment? Also, where does the 'tempfile' library (as in `require 'tempfile'`) come from?

Chris Bloom said...

Got it working - turns out require won't work if you type the file name wrong :)

Anonymous said...

Wonderful. Thanks for saving me debugging time!!!

Anonymous said...

The temp_data method overwrites code need another module :

module Technoweenie
module AttachmentFu
module InstanceMethods
def temp_data

instead of

module Technoweenie
module AttachmentFu
def temp_data

Thanks a lot for this awesome fix

Roger Pack said...

how do I recreate this problem, to submit a test case to ruby core?
Thanks!
-=R

Unknown said...

Very nice, thanks for Tempfile fix! It's now working for me!

Anonymous said...

I did a bit of reading around the internet for "Size is not included in the list", saw a few different proposals, and your solution gets my vote. Well done!

Anonymous said...

You're great. It works after all.

Thank You.

Anonymous said...
This comment has been removed by a blog administrator.
Nelson Enzo said...

for the super-newbs like myself, the syntax to add to environments.rb is:

....
Rails::Initializer.run do |config|

require 'attachment_fu_patch'

.....

sweet tutorial!

Unknown said...

Great blog post. It was a lifesaver.

Omega Sharp said...

Great patch. It's such a generic fix that it should be included in the official release.

Anonymous said...
This comment has been removed by a blog administrator.
Anonymous said...
This comment has been removed by a blog administrator.

AdSense Links