Dynamic Robots.txt in ASP MVC

October 17, 2011

I recently had the need to switch a robots.txt file dynamically, depending on whether a site was deployed to live or beta servers. The solution was very simple, ditch the real file and replace it with a controller action. This is a brief note on how I did this. I don’t pretend this is an ideal solution, it was something I knocked together before I went out for dinner.

The solution I went for was in three parts: a web.config transform, routing and the controller itself.

The web config transform for my live site looks like this:

The route is equally simple:

And finally the actual controller:

And there we have it, my robots.txt file now is customised on a per deployment basis.

Discussion, links, and tweets

comments powered by Disqus

I'm a developer at Purplebricks and I sporadicaly tweet here. These posts in no way represent the views of my employeer.