If you don't like DEPC, why not try DMPC!
To make your RNASE-free containers, I'd DEPC treat them...maybe using a 5% solution swilled around the inside. If the bottles have had media in them before, you could use 1M HCl acid wash/soak overnight followed by multiple rinses in milliQ water, then DEPC treat it, then fill it direct from the milliQ (no pipes). If there is RNase in there (easy to tell, just mix some water with a good RNA sample and incubate for an hour at 37, then run on a gel against the original source of RNA), then change the terminal filter. If the water ain't 18.2 megaohm, then it won't be as pure.
If you do the above and still have RNase activity, change over the filters in the milliQ (expensive, but at least you'll be getting out what you are supposed to!)
Lastly, a grad student in our lab found a huge fungus growing in a water bottle which supposedly had been cleaned...so acid washing the bottle to removed adsorbed nutrients and films of the same is not pananoic overkill...and you can put your trust in the bottle for a long while. Once the acid wash is done, be sure to check the pH of the water in the bottle, and rinse until it is the same as what is coming out of the milliQ (around pH 5.5 , pure water)
PS. I run a molecular biology website called http://molbiol.net - please make use of it