Brian M. Clapper's Profile

I built a Rails-based client portal for a client, and I initially used Roo to permit them to upload the spreadsheets full of data that the portal needed to parse. I later switched to RubyXL. I ran into severe constraints with both packages, on large spreadsheets (where "large" means more than 50,000 rows of data, in my case). Both packages performed poorly with large XLSX files, which stands to reason, given how those files are encoded. I had memory use issues and CPU spikes, severe at times. These issues were especially challenging on the production system (a Linode VPS).

My ultimate solution was to use the Gnumeric spreadsheet tool "ssconvert" to convert the XLSX files to CSV format; from there, the standard Ruby CSV module worked fine. The same Resque job that initially used Roo to parse the spreadsheets just runs "ssconvert" and produces CSVs in a temp directory, before continuing its work. This solution isn't always suitable, of course; if you need to preserve the formulae in the spreadsheets, CSV won't work. In my case, I just needed the data in the columns.

Roo and RubyXL are great for some purposes. In fact, I'd have preferred to use them, rather than my hack solution. But my wishes were thwarted by the sheer size of the spreadsheets I had to import. If you run into a problem like that, using "ssconvert" to convert the Excel spreadsheets to CSV might help.

I'll second the rails_config comment. I've been using it (in projects for two separate clients), and it does pretty much exactly what I want. It allows for default settings, environment-specific overrides, and local (i.e., not checked-in) overrides. As with the final approach in Ryan's screencast, it makes the config settings available in a global.

There are certainly edge cases where it might not work well, but for my uses (passwords, email configuration, API keys, and the like), it's cleaned things up considerably.

Also of possible use: I recently had to add a monthpicker to a client's Rails project, allowing the user to select a month and a year, but no day. There's a useful add-on for this functionality here: https://github.com/thebrowser/jquery.ui.monthpicker

I've actually run into a similar problem. Rubber blows up toward the end, with:

 ** [out :: production.doctrfindr.com] fail
 ** [out :: production.doctrfindr.com] 
 ** [out :: production.doctrfindr.com] ]
 ** [out :: production.doctrfindr.com] 
    command finished in 394ms
failed: "/bin/bash -l -c 'sudo -p '\\''sudo password: '\\''  bash -l -c '\\''service haproxy start'\\'''" on production.doctrfindr.com

So, now, I'm off to join the mailing list. ;-)

@robbied72, for the first year, it'd be much cheaper, since you can most likely get away with the Amazon free tier, which gives you a micro instance free for a year.

I was wrong, above. Even with pg_search, I run into problems with the ActiveRecord-generated queries. e.g.:

https://github.com/Casecommons/pg_search/issues/14

From trolling the web, it looks like the problem in ActiveRecord won't actually be fixed until Rails 4.0.

BTW, the problem I noted above does not afflict me if I use pg_search (so, thanks for the alternative, Ryan).

You may run into problems if you combine Texticle with eager loading (as I have done). Depending on your particular queries, you could end up with an error like this:

PG::Error: ERROR:  column "rank0.46200151534283984" does not exist
LINE 1: SELECT  DISTINCT "medical_people".id, "rank0.462001515342839...

The gory details (sans a simple solution) are here https://github.com/tenderlove/texticle/issues/71, here https://github.com/Casecommons/pg_search/issues/14, and here https://github.com/rails/rails/issues/950

I wanted a more compact log file, with additional information (timestamp and log level, for instance), and more control over colorization. Since none of the existing logging plugins did quite what I wanted, I wrote one. It's here:

https://github.com/bmc/grizzled-rails-logger

It works with the existing ActiveSupport::BufferedLogger, instead of replacing it, so it should still work with new features like tagged logging.

Comments and criticisms are welcome. And, of course, feel free to adapt, hack, or otherwise steal.