GitHub User: kindkid
I do something similar. You may find the constantinople gem useful.
Put all your non-secret config settings (eg - hostname), as well as "dummy" secrets into a config/app.yml.default file that you store in source control.
In production, use chef templates to generate config/app.yml. Devs can also maintain their own local over-rides there as well.
If you prefer, instead of keeping config/app.yml.default and overriding with gitignore'd config/app.yml... you can keep config/app.yml in source control, and override with a gitignore'd config/app.yml.override