trenchant.org

by adam mathes
archive · subscribe

The Death Of The Filesystem

When Andy Baio recently released a plethora of “fake UI” screengrabs from movies, I remarked that I love files and the filesystem. They’re great!

And I was referring not just to the content in the vintage fake interfaces, but that this was just a dump of image files. How refreshing!

With a directory of image files, you can wget the whole lot, display them with any number of programs, write your own little interface, whatever. I dragged a directory of some of them over to eio and it worked just as I thought it would.

Because the beauty of files and the filesystem is that it’s the closest thing we have to a universal and stable interface.

(When I talk about “the filesystem”, I don’t mean any specific technology or interface, but the basic abstraction of a user-manipulable hierarchical set of directories and files, where files are distinct and coherent content objects.)

With the filesystem we have a set of basic verbs (create, copy, move, rename, delete) that work universally across all types of file objects. We have visual interfaces to these tools shipped by default in all the major desktop operating systems. In 2013, they mostly work.

We have tools to backup files for safekeeping. We can easily deploy servers to transmit files over the web. We have version control systems to manage sets of files. We have increasingly sophisticated synchronization tools for files. We can modify these files in whatever programs we have today or can dream up later.

All of these things work, and have worked fairly well over years. If your software depends on manipulating a set of files and directories, it can (more or less) work on any set of files and directories for a long time without too much work.

But the downsides of the filesystem are eclipsing those advantages. Filesystems:

  • have serious usability problems
  • are increasingly irrelevant to mobile and web users
  • don’t foster collaborative work

Usability

Despite decades of near ubiquity in computing the entire concept of a filesystem has been a documented source of frustration for users for years.

A major breakthrough in usability for desktop software in the late 90’s and early 2000’s was media-specific applications that hid the filesystem like Apple’s iLife. iTunes wasn’t an “MP3 player” like most music software at the time, it was a tool to manage music. It presented “songs” and “albums” and “artists” and handled complex operations like transcoding a CD into a set of MP3s and storing them in a directory and transferring them to an iPod with a single click, all without exposing the underlying files that had to be created and organized.

iPhoto similarly addressed difficulties of digital photo management by providing an interface specifically crafted for images, and tools to handle their import into the filesystem seamlessly.

These applications still had the files neatly arranged in directories that users could access and edit if they wanted to.

But the conclusion to this trend is modern mobile operating systems like iOS that tightly couple applications and the data they manipulate. With iOS, the filesystem is no longer accessible directly to users. Nor is it even directly accessible to applications – sandboxing keeps applications limited to managing their own data except in some particular, managed exceptions (photos, contacts, calendar entires, and a few others.)

OS X is also moving in this direction, new applications in the App Store by default generally no longer have unfettered access to your whole computer, but only a small “sandbox” unless specific additional permissions are given.

Irrelevance

Managing a set of files and editing them in various applications was for a time a primary model of users interacting with computers.

Today, it’s a model that is increasingly irrelevant in the context of mobile devices and web sites.

Mobile devices generally tightly couple applications and data, to the point where (usually) when you are using an application you have no idea how the underlying data is stored or organized, nor could you ever get to it outside the context of that application. You may not even know if it’s on your device or deeply buried on a datacenter in another country.

The expected behavior of web applications is to completely manage data on the server side, without users having to worry about it all. Users have no idea how their Tweets or Facebook photos are stored in gigantic datacenters.

Google has released a whole computer whose whole existence is predicated on the belief you don’t need local files at all.

Collaboration

Despite decades worth of networking tools and solutions, a normal user’s files on a PC are an independent silo – a single location, not accessible elsewhere, not accessible to other users.

If two people want to edit text together, the easiest way to do that is probably Google Docs or a similar web service, not something that directly involves network sharing files on a desktop computer. Which wouldn’t really handle collaboration, versioning, users, and other things very well either. (You may say “Sharepoint” here but I rarely see people actually using that, and if they are they are complaining about it.)

As programmers, we have elaborate complex tools to handle the versioning of text files for large projects, but these tools are outside the accessibility of less technical teams.

The Decline Of Universal Interfaces

The universal and ubiquitous interface for power users that the filesystem provides hasn’t gone away, and probably never will entirely for programmers and advanced users. And to the extent that it has gone away and become less prominent it’s probably making most users’ lives easier.

Progress requires new abstractions. The filesystem may not be a useful one anymore for most users.

But when something like ekko, a project that I wrote to mirror content from web, has totally broken due to web service API changes in just a few months, it reminds me we are definitely losing something in this new post-filesystem world.

ekko already has to jump through a ton of hoops to mirror data from web services to local files using a different API for each service, and those APIs break over time. (In most cases, it involved adding complex authentication schemes designed for other web services to interact with the data, that are nearly impossible to use in the context of a single user’s locally running program compared to just entering password.)

We no longer have a universal interface with slow, backwards-compatible changes shepherded by OS vendors slow releases; we have a million incompatible changing APIs over HTTP or other protocols with disparate wire formats.

This makes certain kinds of applications, scripts, and behaviors very hard.

Additive Integrations

Interactions across programs becomes harder with each new service and format – want to make something that saves photos? Well, in the old days that app would read formats like JPEG and write JPEG to the file system and that was probably it. Now you have to think about how you might save to Flickr which will authenticate and operate completely differently than saving to Instagram. Or Tumblr. Or Google Photos or Apple Photostreams.

In some imaginary world, we’d just POST a JPEG to a WebDav server that the user could change as new services came along. And the metadata would just be read from ExIf and stored in that photo directly.

We had something of a universal API with RSS. (I know you probably have a lot of great reasons why HTTP itself or REST is the universal API and it sort of is but hear me out.)

RSS despite its flaws and limitations, allowed a ton of different applications to flourish by creating a baseline lowest common denominator serialization of web content objects explicitly earmarked for “syndication” that was easily fetched. Now we see large companies (Facebook, Twitter, Google) abandoning that format in various ways, in favor of their own application specific APIs with terms of service, authentication procedures, and regular breakage.

You can “do more” with each individual API than you could with RSS, but you “do less” because each new service needs new plumbing.

As a result, when we see the modern take of an “RSS newsreader” reinvented these days it’s more likely to depend on a proprietary service like Twitter, like Flipboard did at launch.

What To Do

I don’t have a magical new abstraction for the network-enabled post-PC world that replaces the filesystem, and the current trends of locking down filesystems and controlling access to web services with site-specific APIs is unlikely to change.

Dropbox is probably the closest thing we have to something that is trying to preserve what’s good about the filesystem but move it forward. The downside is you’re completely dependent on a third-party company to make the whole thing work. (And, more technically, you generally give an app access to your whole filesystem or one sandbox folder, which is often too dangerous or too limiting.)

Dropbox isn’t like the protocols and systems that make the internet work, it’s a company with a good product. But still, a single company that can change their rules and pricing and can probably enable government access to your data.

Apple’s iCloud effort has not quite yet met the expectations of developers, or users, and by design is quite restrictive.

I’m not familiar with Microsoft’s offerings in this area.

I don’t think there’s a great open source or free equivalent to Dropbox, though I need to look into owncloud.

Things like WebDav haven’t worked for this.

Is it conceivable to create something like Dropbox that offers a “universal interface” that is open and offers more value to users and developers?

Is there something that can leapfrog that that hasn’t even been thought of?

In The Meantime

I try to support software that respects the filesystem and less proprietary formats. Calca seems pretty spectacular, but when I saw it uses plain text I was even more excited and bought it immediately.

I think there’s something interesting in this endless i/o prototype I started in 2010, to enable “file blogging.” The python script style is probably the wrong approach – probably better to write a web application that better serves directories of files and then have Dropbox or BitTorrent Sync or Rsync or whatever to handle sync’ing the files around, more like a modern update of Stacey and indexhibit.

· · ·

If you enjoyed this post, please join my mailing list