Bad, vile and meaningless: The hardness of web applications from Alan's clob

Webapp -- the surprisingly hard technology

It would be faintly comical if it wasn't happening to me. It is so painstakingly hard to figure out the way to build decent webapps. I don't just mean stuff like, how many tiers or whether to make it distributed or chunk it into frontend and backend and so on.

I mean, you are tasked to write applications that do the right thing despite the random torture caused by users with their reloads, back/forwards, submits, url modifications by hand (!) and leaving site open for the weekend and attempting to resume on monday... It is gradually dawning upon me that the details you have to handle well to produce an all-around good browsing experience are hard.

The golden rule of a webapp is: never lose any user-submitted information.

Today I shall offer you glimpses to how I came to violate that rule, mostly unwittingly, and introduced serious troubles for myself otherwise.

Session is usually a browser-global resouce

Session is a classic way to go wrong in a webapp. Session is only good for storing information that will not change during the session. Think of it as write-only-RAM. Or in database terms, that you're, in the main, only allowed to add new data but not modify or erase previous data. If you do something else, then back/forward/reload button presses quite conceiveably cause you unforeseen trouble. The HTTP request that you get will not match the state of the session.

For instance, assuming that you make the mistake I was habitually making which is to put data user is currently working on in the session. Imagine a multi-phase web questionnaire. Rather than writing all the previous answers as form hiddens, I'd put them in the session. That's fine, but users have a tendency to open multiple such questionnaires simultaneously. And now you're in trouble, because your session data will be a browser-global resource but the questionnaire is a window-local resource. Also, in many cases it's not feasible to write data you wish to keep secret on the form, so you probably need session somehow to make sure someone won't tamper with it.

So what you eventually get in your database is some kind of sludge of the two intended reports. You lost information, game over, seppuku time.

One method to solve this dilemma is to keep the data in session, but add an extra "instance" or "window_id" or such variable that you generate for each request and track session data by such instances. So if you have two browser window opens with distinct "instance" number, you can still separate the server-side session instances, too.

You could also try running them in separate sessions, which wouldn't be dumb at all. But that means that session can't live in cookies! Cookies are browser-global, and the only way to make them window-local is some hacks to make them target unique URLs. With that complexity, you could just as well place session on the URL as one extra parameter. So, I advice not to use cookies for session for serious webapps. Or if you do use cookies, you must also carry an unique part in the URL, like the webbanks do.

Oh, and one more security detail: if you are not using https protocol, it's too easy to hijack user session. You have to have https protocol going on first, and then start your sessions.

Serialization

Additionally, if you have a session, that session will almost by necessity serialize your webapp's requests. If you don't serialize your application's requests, you could easily end up allowing double submits performing the same job at the same time, updating the session at the same way, and this rare incident will easily cause trouble at the database backend.

For instance, suppose you are receiving form data and putting it on the database, and your clueless users will go and press double-submit because they've learnt that you must doubleclick things on computers. Now, if you allow your site to have two requests executing in parallel, you'll do the same operations, perhaps crash the latter one if you end up violating uniqueness constraints, or perhaps you insert two rows and have to later explain to your customer why there's duplicate entries here and there.

So, to prepare against that eventuality, you realize that you basically have to somehow serialize the requests that would hit the same resources.

That's bad news. If the session is handled by application, and Apache doesn't know anything about it, then a user submitting a ton of requests to the application will tie a number of httpd processes and your application server backends while all of them race for the session lock.

So, I hope that by now you shall think twice about using a session. It has important and unforeseen consequences that affect the scalability and robustness of your application that may not show up in testing (for instance, if your testers aren't using multiple windows at the site simultaneously).

Reflection

It's said that to every problem there's usually an elegant and simple solution. Each day I become more and more conscious of the results of simple design decisions that I used to make everyday.

Most of my experience comes from unlearning things like object-oriented programming. OO is great if you have the right interfaces. But to find the right interfaces is extremely hard. Because of the difficulty of figuring out the right interfaces that do not obstruct or hinder the essential properties of the system (such as elegant database design), I mostly find that the artificial castles of OO structures make it harder to express and make changes to what is essentially a simple idea. In other words, a little bit more chaos and relaxation of extra structure is a mixture that works much better for me.

A lttile time ago Ton Hospel pointed me to perlmonks.org which contained several versions of solvers to the glotski (klotski?) games. Like me, Ton had based his solution on image of map which was virtually identical in mine, and checked the ending condition of the game with regex (his regex was XX[^#]##. Mine was usually of form aa[^-]--). His solution was not generic solver but was rather tailored to solving that one problem, though, but it was nice to notice that he had solved it essentially the same way.

My unique bit was using xor operation to move the pieces. I did the classic xor cursor trick: Use xor to remove the piece, move the xor mask, then xor with the moved mask again, and the piece reappears. However, I also saw several fancily OO-ish approaches that were invariably extremely long and most likely also extremely slow.

(A lengthy rant about OO)

What exactly was OO supposed to buy us, after all? The maintainability and isolation benefits are only relevant when the components attain sufficient sizes so that they can be packaged and shipped. OO is about controlling complexity. But most talk about using OO seems to me like micromanaging complexity.

You see, the interface, I find, is more tasking as a concept as the details of implementation of a particular routine, so I for one would much rather see the full routine rather than think about the interface whose implementation executing the behaviour I pretend not to know. Hell, I don't know. Perhaps different people are better suited at thinking on different levels of complexity -- for me, the OO concepts are very hard indeed because the interfaces and objects are, by their nature, complex and somewhat ill defined. For instance, think about an object with methods "foo" and "bar", and to execute some particular behaviour, you must call both of them. But in which order? Is it possible to know whether you should call "foo" or "bar" first?

It isn't, because in the above you have no working knowledge about what the hell foo and bar mean. However, in the following case, it might be obvious:

Foo foo = new Foo();
foo.setXyz('this');
foo.setBar('that');
foo.doSomething(withSomethingElse);

Here we have an unconfigured instance created, which must first be configured and only then used. I chose to write this using Java syntax, because this style of programming is typical in Java.

We may be able to intuitively accept that the accessor methods to configure the instances must be called before doing the work the object is capable of doing. But what if they weren't necessary? At best, you just wasted extra CPU cycles doing what would be trivial to see that wasn't necessary–if only you were allowed to view the implementation of the doSomething() method!

We usually agree to ignore performance-related arguments in OO code. It's agreeable because OO is supposed to be so nice. But I think it proves my point about the senselessness of micromanaging your own code. Here's a better way to do it: Make interfaces when someone else is supposed to work with the code, or where it's conceiveable that someone soon will. An interface is a point of reusability, and it comes with a cost. Only make interfaces at the components that you can reasonably expect to be reusable. But do keep artificial barriers down otherwise. OO is a great way to contain the mess of a real-world problem, but it's often also unnecessarily strict and thus micromanages the "mess".

Consider also the tired, old mess of the One True Accessor access syntax:

foo.bar # get attribute value directly
foo.getBar() # get attribute via specific accessor
foo.get_attr('bar') # get attribute value via a generic method call

Now, should we expose this property only through an accessor or should we not? Different programming languages differ on this point. The fact that this question can even be asked points to me about another fundamental fallacy. To be frank, I hate accessor methods. I do all in my power to make my objects immutable. After the point of creation, they are set. This means that my constructor methods take plenty of arguments. Now, it's great in good languages like Perl or Python that allow you to name your arguments:

Foo->new( # Perl
    this  => $that,
    other => $foo
);
Foo(this=that, other=foo) # Python

However, if there is a sensible attribute that you need to get, I make the point of always making it a method call except in Python which allows me to wrap accessors access into function calls when necessary. Python rules the scene, in other words: get the best performance of the simple assignment and still retain flexibility to wildly alter the internal details of the class.

Eat that, suckers.

Summary of my anti-OO rant

I claim that OO is a great for external interfaces. It's a horrible internal interface, because it tasks my brain more than the straightforward piece of code doing the equivalent with would. In other words, I claim that interfaces are comparatively harder to handle than the raw code that will be run behind the interfaces. Up to some point. There's a breakeven where an interface is necessary for progress, but it's a lot higher than most people who habitually microsplit every program into OO nonsense realize.

In general, there's a disturbing tendency in Java-land to shuffle complexity around without solving it. The guys make interfaces to hide a few lines of dumb code behind them. Instead of writing Java code to do a few statements they invoke XML parser and some elaborate XML description of the action. (I hear this is because XML is agile compared to Java. Maybe. I hate XML too.)

And a note of joy

Programming, however, is essentially about discovering facts and unlearning the falsehoods and discovering the truths of what makes a great computer program. Remember the joy and excitement of getting some hack working? That's the joy inside that must act our guiding beacon. Strive for the light; strive for the joy. For me, OO and other abstractions hold no joy. I'm an engineer. I like to see how things work. I love tinkering on that lower level even if it means that I must task my brain with more details than are strictly speaking necessary.

Down with the unnecessary complexity! Down with abstractions! Let the love for raw machines conquer the separationist agendas of OO folks.