Hibernation, suspend-to-disk, whatever you want to call it. I'm talking about ACPI S4: Saving the contents of the main memory to secondary storage and powering the machine completely down. Upon the next restart the previous memory contents are restored.

This is a great feature since one can continue working as if nothing happened. At least in theory.

There are several issues. What if there is not enough space on the hard disk? What about devices and their drivers?

Hardware problems concerning ACPI S4 (and others) are very common on all traditional operating systems[1]. I believe this is partly due to the fact that this feature has been mostly hacked on top of the rest of the system.

The system itself still uses explicit memory management which can lead to a problem that I've encountered often. When the system is commanded to hibernate midst in (memory) intense work there might not be enough space on secondary storage[2]. However, if you instruct the application(s) manually to store their state, close them and then start them again to load the previously state, all their internal non-essential state[3] has been dropped. Oftentimes hibernation is now possible.

Now, image if you have to do this for a dozen applications, each a different interface and underlying mechanism. Pretty lame, isn't it? The advantage of hibernation, both speed and convenience, have neatly been destroyed.

So, what would happen if we put this kind of functionality into the operating system? Memory management actually is already in there. The problem is that applications are free to claim memory they don't need -- with good intentions. I'm not going to research a "proper" conservative solution, since it is inconsistent (see next paragraph) and most likely hard to implement -- and probably impossible to enforce.

Again, a proper solution is simple but contrary to traditional systems. If the whole secondary storage is only a swap space for all the objects in memory, applications can claim as much memory as they want -- until the whole storage is used up[4]. Also, if the changed contents of the main memory are periodically written back, you get hibernation for free. Even better, you get "hibernation" always, even in case of sudden power failures.

This is known as transparent persistence. And I want it.


  1. Except maybe for a certain vendor that sells both its own hardware and OS.
  2. At least on GNU/Linux. Honestly, I don't get it. Having a swap partition of four times the size of the main memory and using not even close to half of it should be sufficient. Maybe my data is inaccurate, but still it looks buggy to me.
  3. e.g. application-level caches
  4. Quotas are orthogonal to this issue and needed in traditional systems, too.
Posted 2011-04-18 23:15 Tags: persistence

Linux and Laptops

Linux has a feature called laptop_mode. According to /usr/src/linux/Documentation/laptops/laptop-mode.txt, laptop_mode "is used to minimize the time that the hard disk needs to be spun up, to conserve battery power on laptops. It has been reported to cause significant power savings". However, it has also been reported that GNU/Linux causes harddrives to fail far too early[1].

The issue I'm referring to in this post is a bug[2] in the Ubuntu distribution of GNU/Linux. In short, Ubuntu did a

hdparm -B 1 /dev/sd*

which causes all block devices to use the most aggressive form of power saving possible. As a result, the head gets parked and the platters will spin down after a short idle period. Having the disk spin down early is good. Having it spin up and down again frequently is bad. And that's where laptop_mode comes into play.

Laptop_mode allows to automatically delay writes to the disk and to flush all data to be written once the disk has to be accessed. Sounds clever? No, it's a hack. It's an exemplary semi-solution since the disk is not only accessed by writing but also by reading[3].

Linux, like all other traditional systems, has the inconvenient bugfeature that nearly everything has to access the disk[4], sometimes described as the disk-ram-dichotomy[5]. While information that is read from the disk can be cached so that subsequent accesses won't trigger another drive spinup, caches have to be filled before being useful. GNU/Linux' semi-solution provides no means to accomplish this, leaving users with drives that frequently have to spin up. Oh, how should I know you want to access those files in your home dir you always use? And why should I automatically cache them when there is memory left and CPU not in active usage? This is a deep-rooted conceptual issue of traditional systems.

Judging from my own experience, laptop_mode does not even seem to delay writes properly. I got frequent spinups even when not working on the machine, and I'm using a quite minimal install[6] (traditional GUIs are so bad that even the Unix Shell is better). Well, it might just've been another bug or a hole in this kludgy concept.

But there is another point to mention. Let's assume there is an operating system that features proper transparent persistence[7]. Wouldn't it be nice to have the disk spin up in a user-defined interval like fifteen minutes (or in case of low battery) to write all changed data and to instantly stay shut afterwards until the end of the next interval? In other words, let's assume an operating system supporting aggressive power management. Wouldn't it be nice if the hardware supported such aggressive power management, too?

Some people think the above mentioned bug was not an operating system issue[8]. Blaming the hardware vendors for supporting aggressive power management when the OS misuses it is, however, a sign of blatant ignorance.

And not being able to temporarily disable the disk during normal system usage[9] is a sign of a bad operating system[10].


Footnotes

  1. http://paul.luon.net/journal/hacking/BrokenHDDs.html
  2. http://hardware.slashdot.org/article.pl?sid=07/10/30/1742258
  3. Additionaly, harddrives may get probed (e.g. SMART), which triggers spinups always. Now the combination of a SMART deamon and hdparm -B 1...
  4. The disk is commonly accessed rather directly (through filesystems) always when persistence is desired. All data is usually available stored only on the disk.
  5. http://c2.com/cgi-bin/wiki?ProgrammingWithoutRamDiskDichotomy
  6. This includes not using a SMART deamon or similar.
  7. Read: High-level transparent persistence and economic resource management, and that includes intelligent pre-caching data when possible and probably beneficial.
  8. https://bugs.launchpad.net/ubuntu/+source/acpi-support/+bug/59695
  9. We assume there is enough RAM, which usually is the case.
  10. Of course it can be partially hacked onto GNU/Linux, it's basically mount -t tmpfs tmpfs $DIR with moving data around before and afterwards. The resulting workflow is, however, not straight-forward and the mechanism very error prone if one does not glue all those loose ends together that might cause data loss. A sophisticated solution can only be part of a sophisticated OS.
Posted 2010-03-30 11:57 Tags: persistence

Squeak is an insult to Smalltalk

Squeak is, actually, the platform I used for getting in (practical) touch with Smalltalk. It was a very influencing and important experience. However, Squeak is not only stuck in the 1980s, it is also broken.

Smalltalk has always been a kind of operating system. Squeak is no exception and carries on the spirit of (pre) 1980: It is intended as a single-user system and has no security concept (object capability model would be easy to implement, though). AFAIK, program windows were originally invented in Smalltalk. To this day, Squeak adheres to this obsolete WIMP (windows, icons, menus, pulldowns) concept which limits direct interaction with objects on an otherwise rather live and reflexive system. Additionally, the GUI is hard-coded: you can't run Squeak properly without using its GUI. The "headless" parameter renders it anti-interactive (unusable, in other words). Not only is the user interface inflexible, its distinguishes between "novice mode" and "expert mode", which is anti-pedagogic. The reason why there supposedly must be an expert mode is that you can perform several actions that could destroy your Squeak image.

That brings us to the next issue: Squeak is irreversibly destructive. AFAIK, Objects in Squeak are not versioned, you can't simply restore a previous state, neither individual nor global, after you've screwed up. Screwing up is actually fairly easy. Sometimes the update of (what I believe seems to be) the basic image from within Squeak is enough to render it unusable. Even if you just want to uninstall a package you installed using Squeaks native package manager, you won't find an "uninstall" button or menu entry -- because there is no possibility to properly uninstall formerly installed software. (No, I'm not kidding.)

Well, since it is easy to irreversibly damage an image, at least Squeak doesn't automatically write it to disk (i.e. transparent persistence). Oh, wait, that's not a feature, that's a hack. I want transparent persistence!

What would be even better is error-free persistence. Squeak somehow managed to lose my original source code and seems to display silently reverse-compiled bytecode. Parameters are named tn where n is the parameter number and all comments are lost. Except for class comments -- these are referenced by RemoteStrings that point beyond end of file, neatly throwing an exception each time I try to access a class in the system browser. Considering these severe bugs, I suppose the next time the virtual machine segfaults, which it does once in a while, I should be happy -- at least I have only lost few data instead of corrupting it all.

Ultimately, even though a proper smalltalkish system would be far more sophisticated, powerful and efficient than a unix shell, I enjoy working on my command line more than trying to click my way through Squeak, having to tediously move, drag and drop windows and objects all over the place.

It's not even fast enough for my neurons.

Posted 2010-03-01 01:01 Tags: persistence

You might want to check out the archive of posts tagged "persistence".