As an analogy, there was a time when breaking the four minute mile barrier as a runner was considered impossible. Now it's quite common. Yet it's just as difficult to be the fastest runner.

My advice would be to become familiar with a number of web frameworks _and_ understand how to build your own. This will take some time, but the benefit will be your understanding of when to use frameworks and when not to. One question to consider is what sort of projects are you planning to build? If it's a one-off for a client, would a framework get most of the job done? If it's a product you'll be hosting or distributing, will you be living with the code base for a long time and therefore benefit from the investment of your time in development?

Try out something like RoR or Django.

I had the desire to write my own framework for many years, and completed a few half-baked frameworks, but it is definitely worthwhile considering one of the major frameworks. There is a lot of time, thought and effort put into them that you can leverage!

The improvements in game technology have been more visibly stunning but there is plenty going in in the web space. Think about what Virtualization technology is doing vis-a-vis deployment effort. Imagine someone running a "server-less" company even a few years ago. Now lots of companies are doing this with Amazon.

[0] Obviously, we can make richer interfaces today, but that often means more work, so I wouldn't say it makes development easier.

I have created my own toolkit that sits over the top of the asp.net 2.0 and that fits my needs and gets used for all my projects.

I think in a few more years time your be able to use tools like Access to deploy web apps(I know you can do it to a point now).

On the other hand, it does look like good client-side JavaScript frameworks provide a competitive advantage, if used judiciously. I haven't investigated which is best yet.

That seems like a pretty silly set of expectations for a framework. By virtue of making it easier to do things, you may eliminate a lot of bad things, but the goal really is to make it easier to do things. This might seem like splitting hairs, but you split the hair here, and I disagree with the conclusion.

It's a good point. Web development does feel like it's far more of a chore than it should need to be.

It's still very difficult to have a development methodology that allows you to develop mega-pages (so as to offer a lot of functionality on one page and thereby keep the request/response loop down) without the codebase turning into spaghetti. I thought I'd done it at my last job, I've since spoken to the maintenance programmer, he's unimpressed, and that says 'failure' to me :). The combination I used for that was tapestry and cayenne and I still think it's the best grouping although Tapestry 5 will be a strong improvement (don't get into it yet though - the tutorial on the site isn't finished). Also, you're stuck with java if you do this.

> early days CGI programming with Perl?

There are huge advantages to using a templating system. If you want to understand why, have a think about how you'd implement an algorithm in CGI that allowed you to edit the first and last name for a list of people in a single page, and to have the resulting data factored into objects automatically before you start writing your handling code. Good templating is not new, but it is new to the mainstream. WebObjects was doing all this and more in the late nineties. But the API was so quirky, the end-user tools so poor and the system so expensive in its early years that few people got to benefit from it. Expect to see a bit of a renaissance from WebObjects - Apple have some top-notch people working on it and have really opened it up. (Still -I'm not sure how or if they'll be able to overcome the verbose and quirky API though.)

Coming up with a framework that can do mega-pages well is a problem I've revisited many times, and I'm not there yet. My latest attempt is at datamagi.org. This approach attempts to make the web server a dumb client that asks a 'logicbase' for the user's current position in the system, and then renders it to them. Basically - you define a state machine that is the application, and then the interface wraps around it. In this way you can develop dynamic webapps without touching a line of HTML. In time I'd like to come up with a description language that describes how data should be rendered, and then have this dumb-client webserver then lay out a webpage matching that view. I don't even have transactions properly worked out yet. But I do think this is the right approach to the problem. Once we have this it will be near-trivial to painlessly develop better user interfaces than the web browser.

The approach I'm taking on a web project I'm working on right now (that I need to work - hence not using my immature framework) is to use python and cherrypy, and do all the processing manually. It's cumbersome, but I find it quite rapid to refactor, and once I understand where the repetition points are I can work out my own framework.

Django looks cool. You could check out rails as well. I've avoided it because the ORM in that platform is primitive, and because I prefer python for lots of little reasons. I can't remember why I avoided Django - might have been a maturity thing. I'm very distrustful of heavy frameworks in general because I've been stung a few times by situations where I want to do something and can't work it out (or find that it's not possible because I haven't followed some disgusting 'convention' wired into the platform), whereas I find that if I develop a framework myself I don't get into that position. I had this with tapestry 3 - had to go to ridiculous lengths to get what in .ASP would have been called a 'page'-scoped variable.

For you - developing a framework would have the advantage that you'd understand the problems that frameworks solve. So when you go looking for a framework later on you'll know what you're looking for and why.

I love web development. :)