#388 Multitenancy with Scopes pro
Oct 20, 2012 | 12 minutes | Active Record, Authorization
A multi-tenant application can be difficult to implement because the data for each tenant must be completely separate. Here I show how to do this using subdomains and default scopes in Active Record.
- Download:
- source code
- mp4
- m4v
- webm
- ogv
Why not putting
in a Concern and include that module in models?
Don't you think it would be great to talk about testing in Pro episods?
Thanks
+1 for testing... if not talk about it, include some example tests in the episode source code!
+1 for tests
+1 for TESTS!!! I found this to be quite hard to test. The complexity grows when you have an authentication system on top of the subdomains.
I recently created capybara and rspec tests for an entire app specifically to verify that multi tenancy does not leak data.
Basically I got it down to:
* Setting up a tenant that the tests use. For me a subdomain which is created in a before filter.
* Making sure there is fixture/factory data belonging to at least two tenants
Then you just check to make sure that models don't find data for the "other" domain, views don't display that data and so on. It is not the most fun tests but it does feel a lot safer knowing that I am pretty sure I don't leak data.
I also went down the anal-retentive path of switching tenants mid-test and verifying that I only saw the correct data there too.
We have been using the one-db-per-tenant concept for a few years but since moving to MongoDB this is starting to create ridiculous overhead. I was a bit paranoid about switching to a scoped tenancy but it has, so far been solid. (Mongo has pretty large overhead for databases. Out DB is >100GB on disk but only a few GBs exported and zipped.)
Hi Martin,
I have multitenancy setup as per this railscast and have just started to learn Minitest Spec. Would you mind sharing some of your testing code to show how you implemented multitenacny testing?
Thanks.
+1 for tests!
Here's what I ended up doing to test my models. Other thoughts and ideas are welcome (like a shared example to use on all account scoped models based on this gist)
Interesting but I don't like this approach, works well for simple applications, but it's very open to attacks, I think a better approach is using PostgreSQL schemas, and changing the schema with the subdomain, this way you truly create a separate schema (DB) with it's own data, of course when you should create a new schema for each new subdomain, and you should run the migrations in all schemas.
In this blog (http://railscraft.tumblr.com/post/21403448184/multi-tenanting-ruby-on-rails-applications-on-heroku) the author recommends rows based multitenancy over schema based and gives the following pros and cons:
Summary of ROW-BASED Pros & Cons
Pros:
simpler implementation
ability to use existing rails migration tools
use of Postgres Views/Rules & Rails 3 scope to enforce
faster pg_dump, typical use case optimization in DB parser/planner for SELECT QUERIES
no need to jerry-rig IDs
no monkey-patching of Rails A/R (maybe just the connection)
Cons:
might break down at 20 to 50M records in a single table?
difficulty in partitioning?
I don't think you can argue that something that switched DBs is less prone to error and attack than something that is query-based... I prefer it, personally, and have used it for many years.
See my comment above about testing where i mention why I changed... TLDR: The app grew and many DBs became an overhead we could not afford.
I should also note that it was not a huge thing for me to add a default scope (in a module btw) and including that in all models. Then it was just a matter of setting the right tenant association on existing data before migrating it all to a single DB. The testing was probably what I spent the longest on.
Well I would like to mention that episode 389 is about using schemas with PostgreSQL, so far I have read you are using mongodb and I would like to know what you were using before.
Before mongo we were on MySQL
PostgreSQL schemas are a great thing, but I completely agree with Martin -- whether you have multiple databases, or a single database with schema namespaces, you're only solving one part of the problem -- the application still needs to be aware of multi-tenancy.
For example, the Admin back-end of an app I just finished usually needed to see information across tenants. Further, we wanted admins to be able to log in to our tenants' sites with their normal username (that had a role whose permissions allowed this).
And you're creating a larger problem managing the system, etc., and again, while PostgreSQL is (IMHO) by far the best RDBMS out there, why tie yourself to a specific database feature unless you really have to?
The bottom line is that building a multi-tenant app is an up-front design decision that needs to be baked in at many levels. It's harder to write, test, and so on, but that's just because it's a hard problem. Not impossible, just harder. This is most definitely not something I would suggest most apps farm out to a gem -- it's never that simple.
Tom
I've been working on a multi-tenant app for about four years now. I didn't know it was called that and have seen so little written about it, I made it all up from scratch. And it's been ridiculously complex. Especially with a myriad of roles per tenant, and several tenant types. Although mine works better without subdomains, I think I can apply the default scope to my current methodology for enhanced security/protection against developer forgetfulness.
Thanks Ryan!
Does anyone have any idea to how to make this work with reserved subdomains? I want to render a StaticPages controller if no subdomain is present, or if it's a reserved one.
Originally I scoped everything through a
current_account
method as Ryan briefly mentioned in this episode. I only asked for thecurrent_account
if the subdomain was not reserved, or if there was no subdomain.ruby
def current_account
if request.subdomain.present? && !Account.reserved_subdomain?(request.subdomain)
@account ||= Account.find_by_subdomain!(request.subdomain)
end
end
This meant
www
or no subdomain can be routed to a controller of my choice. Using Ryna's approach, however, prevents the view from rendering when no subdomain is specified, or when it's reserved.Any ideas how to get around this limitation? This is what I get, but the page is completely blank:
Started GET "/" for 127.0.0.1 at 2012-10-22 21:47:06 -0200
Processing by StaticPagesController#home as HTML
Completed 200 OK in 0ms (ActiveRecord: 0.0ms)
I figured it out. Stupidly, my yield was inside of my if condition!
You can also make use of routes.rb to perform something similar to this and most sub domain setup work.
In the above example Domain is a class with a method named matches? in it. More about making use of this trigger (which is very early in the request process) to setup the tenant stuff is found in the rails guides starting here:
http://guides.rubyonrails.org/routing.html#request-based-constraints
We trigger all subdomain/tenant setup from this kind of route constraint.
Hi Martin,
I've followed your routing tip above and it seems to mostly work for me however "Routes to use if no tenant subdomain exists" is tripping up on the 'around_filter :scope_current_tenant' used in the application controller. If I comment that out, the routing works for no subdomain, but then subdomain stop working. Any ideas how to get around this?
Thanks
Ryan
I have this issue as well and it's the only thing keeping me from deploying my app right now. I can do a skip_around_filter and the homepage (static pages) will work and so will the subdomain but this is not a good solution on controllers that have actions that either need to be scoped or unscoped. For example, the registrations controller needs to have the new/create actions unscoped so people can register for a new account and then have the edit/update action scoped so only the right user can update their account.
It seems like this railscast should have covered routing. I followed his railscast for subdomains but for some reason, doing a match doesn't do anything in rails 4.2. I think it's still supposed to be working but it has no affect on anything if I use constraints as I've found in the rails casts and on stackoverflow.
I guess I can do the skip filter now and try to be more specific in the registration controller.
I find it better to put the global scope directly into the controller:
That way everything outside it will work the same way as before.
Don't forget that if you are going to use something like this in a secure environment (SSL) then you should get a wildcard certificate. Otherwise your clients will get certificate errors.
Agree! And one other point we learned the hard way: we had used subdomains for environments for testing and staging (e.g. tenant1.staging.example.com) and didn't realized until too late that wildcard domains only give you one level of subdomain. A better strategy is to do something like tenant1-staging.example.com.
Once again your provide timely info. Thanks!
I have a project with two sister schools. The sites are almost identical except for some small cosmetic changes, and they have different dynamic content. The users of each site should be able to login to the other site (but of course privileged accounts should only be privileged in the home site). I can see how to implement this based on your Railscast.
My question is can this be done with two domains instead of sub-domains? I would like to have siteA.com and siteB.com instead of siteA.domain.com and siteB.domain.com. Is this possible? How would I start? Or should I just maintain two sites and have some sort of joint login system (and how would I do that?)
Bump
Same question: Can this approach be used for deploying a rails app configured for multiple root domains (e.g. domain1.com and domain2.com)?
If so, what changes would be required to make it work? I'm thinking about things like routing.
Thanks
If you want a multi-domain solution, I think you'd do that configuration in your front-end web-server/proxy, configuring nginx/apache to handle multiple vhosts.
It's not really all that different than what Ryan shows in this episode, or perhaps not at all -- there's something in the request that identifies which tenant is current (the
current_tenant
method, for example), and there's some sort of table that contains details about the tenant.You can decide how you want to set up your web server -- you might set up a virtual host for each domain, or in the case of a small number, a single vhost that handles all domains (e.g. using Apache's ServerAlias directive). While it's also possible to let all requests through on the default host (sometimes called the "catch all host") and handle the ones you know, I prefer having a more declarative setup.
I also recommend having some sort of immutable unique text key (all lowercase, single word) that you use to identify the tenant in the database. Once you do that, you can use it as a kind of namespace that allows your app to pick up tenant-specific CSS selectors, locations for assets like logos, and so on.
Yes, you can have one app so people only have to log in once to either system. You can set current_school just like you set current_user with something like the following if you save siteA.com and siteB.com as the domain for each school.
Then you could use @current_school for scoping as mentioned in the episode.
Did anyone experience a problem with using Thread.current store in a default scope? When i change the ID stored in Thread.current i can see that it is overwritten but the default scope query stays intact - like it was cached with the previous ID. Can i get rid of that cache from default scope? The alternative is to use a scope with a parameter, but then it doesn't make sense to use Thread.current for storing current tenant ID simply because it can be stored i current_user.* (whatever).
I am also struggling with how to administrate a multi tenant app, and have not found much info about this elsewhere.
I'm not sure, but maybe creating an admin user:
would suffice?
Not sure how to do this though.. searching...
Remington,
have you found a solution?
I am just creating an admin user in create_schema method of tenant. Since admin table is there in every schema, you end up with schema (or tenant) specif admin(s). This is not too bad if you want to distribute the admin work to a team of support persons and distribute the load based on accounts/regions.
However, if you prefer to have one admin table for all schemas, then leave it in public and drop from tenant schema when you create a new tenant.
Can anyone explain why we have to do this so that it's threadsafe?
How about this?
In the earlier part of the episode, Ryan used a
cattr_accessor
which is a Rails shortcut for creating class methods.(I think this is right, but anyone who knows better, please correct!) The reason to consider thread safety is that unlike instance variables, class methods are considered "static". Depending on how you have Rails deployed in a production environment this is not inherently thread-safe -- Ryan did a recent RailsCast on thread safety that explains more, but the short answer is whether or not something will be thread-safe depends on an external configuration that might change with a new version, or work differently in a few months, etc.
For me, at least, that's reason enough to add this little bit of safety.
I don't have a solid answer on the use of the instance variable, but I think it boils down to this: is
@tenant_id
that you reference at one moment necessarily the one you reference at another, or is it one on another thread? Again, the fact that this is a class method, not an instance method is what makes the difference.(And if my explanation is wrong, I do hope someone can correct me so we all understand :-)
Now I may have missed something obvious, but the one area that seems to be missing is securing the Tenant model.
You only want a none authenticated person to sign up and create a new tenant
You want an existing tenant to only edit, update, and destroy, and only their own account
And then you want the admin to do anything, but that is a special case for many things
So I guess this is all going to be custom code within the controller (ie force the id in edit, update, and destroy. Only allow new and create if current_id does not exist)
thoughts?
It is the user that create/read/update/delete the tenant, not the other way around, so he can have multiple tenants. And in theory, a user can exist without any tenant if you desire so. You just need to implement a role based authorization system in your app.
App wide super-admin can be implemented as a kind of super user that has access to all tenants, but it's better to have it's own class so you can also assign different roles to it.
Hello, anybody could tell me a good way to bypass the tenant scope? I have tried calling unscoped and perhaps I'm not doing it right but it seems not to do anything. All the queries are made with
where tenant_id=...
.Thanks!
After adding a tenant_id and default_scope, where is the best place to add this to be used by Rspec tests (where all database queries are currently failing due to the tenant_id being null)? Can I put this in a single place to be applied this across multiple tests?
Thank you
Hi,
How do I redirect www subdomain to another webapp?
Thanks!
Hi,
after setting value in around filter, the values are nil during using them in scopes ?
actually they are not nil they lose values
Wonderful episode!
There are some tangential security issues with this, but they are outside the scope (no pun intended) of multitenancy. The most obvious one is any user can create a resource in another scope by simply POSTing a tenant_id other than his own. To prevent this hole, simply make sure the controller does not accept a tenant_id parameter.
Regardless, this is awesome stuff, well thought out, and extremely helpful. Thanks Ryan!
How does this compare with a gem like https://github.com/dsaronin/milia? Is there a multi-tenancy gem that works with Mongoid?
I have this working on my local iMac. When I upload to Heroku, the app gets this log when starting up "/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.12/lib/active_record/relation/finder_methods.rb:310:in `find_with_ids': Couldn't find Tenant without an ID (ActiveRecord::RecordNotFound)"
Do I need to do anything special to run this on Heroku?
You have probably created a Tenant on your local machine, but have you created a Tenant via the Heroku console (or some other method)? When you push to Heroku, it does not push your local data.
Hi,
I created 2 admin, based on the user role it will be navigated.
I want to create multiple domains. How to make the admin can create domain and populate the views accordingly.
Multitenancy using instance variables.
https://gist.github.com/ashraf-majdee/6811863
If someone is interested, I wrote a blog post on how to implement multi tenancy with Devise and default scope but without subdomains. This is useful when you don't want or can't use subdomains or for example when you only enable subdomains as a premium feature, meaning that your application needs to have multi tenancy both with and without subdomains.
http://vitobotta.com/rails-multi-tenancy-devise-default-scope/
I had to change around_filter :scope_current_tenant to prepend_around_filter :scope_current_tenant to make sure the tenant was always loaded before my authorization before filter.
Back to this article... has anyone used this technique with Puma?
If anyone is using this with Devise gem and want to have unique emails per scope, here is what I added to the user model file.
This disables the Devise validation for email (only) so you can implement your own. ie Use the validation Ryan shows in this video.
Testing
If you follow this cast and are testing with RSpec here is a tip.
In order for the scopes to work I added this to the before block
I have a macro setup called sign_up_tenant that simply fills in the form and submits, but you can just create the Tenant using FactoryGirl or something similar.
The important bit is the
Tenant.current_id = @tenant.id
Anyone get this working with sorcery?
This isn't working for me without subdomains. It seems current_account remains on the first account created no matter what. So basically when account2 logs in, it can see all the records for account 1 and when account2 creates a record, the record gets assigned the account_id of account1.
I actually got this to work just now with Sorcery.
This is what the code has to be to work:
def current_account
current_user.account
end
helper_method :current_account
def scope_current_account
Account.current_id = current_account
yield
ensure
Account.current_id = nil
end
I believe adding the ability to change the scope for the user model was so an account admin could add new users with the correct account id. Currently this is the part that threw me with Sorcery since it doesn't allow this (at least not using this method). I could make it so when the admins create a new user, the account_id gets added via something like @user.account_id = @account.id.