[McGranaghan]: You gotta sit down every week for 30 years and make sure the software doesn't break, and that's sort of a necessary but not sufficient condition.
[Wiggins]: And the implication there is just maintenance, right? And that maybe ties into some of our discussions about software supported by subscriptions versus one-off payments, which is really is an ongoing effort, important ongoing maintenance effort, rather than a one and done.
McGranaghan & Wiggins, Metamuse Episode 49, Software Longevity, 26:05, https://open.spotify.com/episode/1DCNy94TBt3aoGbM8PNlRv?si=-2-SDcZXSUaI6iFWTO7eDg
One thing we've alluded to is idea of narrow-defined APIs, because your software has to run in something, and the broader, the more complex, the more ill-specified that environment is, the harder it's going to be to do this job of keeping the software running.
And this would include things like taking on few dependencies, minimizing weird binary dependencies that are hard to compile, limiting the APIs that you access, things like that.
McGranaghan & Wiggins, Metamuse Episode 49, Software Longevity, 26:39, https://open.spotify.com/episode/1DCNy94TBt3aoGbM8PNlRv?si=-2-SDcZXSUaI6iFWTO7eDg
[McGranaghan]: Another thing that I'm very big on is really focusing on data, often, especially in this world of rolling forward over multiple generations, what you really want to keep as the data.
This is more true for productivity-like stuff than games for example, but data is very important. And, if you want to roll forward data, you either need to adopt an existing open data format or you need to do your own, which can be done, but it's a lot of work. I think that programs or program lineages, if you call that, embrace this idea of really focusing on the data so that that can always be ejected and rolled forward into the next generation of software.
[...]
One of my favorite examples is SQLite. If you use SQLite to write even your semi-custom data format, if you write it into SQLite database, you're absolutely gonna be able to read that in 30 years, even if you do nothing yourself. Whereas, if you roll your own format on disk, there's a very good chance that it will at some point become totally lost.
[Wiggins]: Yeah, SQLite is a favorite example […] of well-made simple software that does one thing and does it well. They have a great page though about their long-term support where they list off some techniques, like, testing and making their database files cross-platform and disaster planning and not embracing hot new technologies with too much gusto.
McGranaghan & Wiggins, Metamuse Episode 49, Software Longevity, 27:05, https://open.spotify.com/episode/1DCNy94TBt3aoGbM8PNlRv?si=7WHCqDsQRaK-HaTeS4HQVA
[Wiggins] […] one of the things that points to that to me feels like maybe the most fundamental is to just think about longevity in the first place and just care about it in the first place. Part of the reason I thought this topic would be an interesting one.I think, again, tech industry skews (?) young, I think also had the same gap in my thinking as a younger person, which was I just didn't have the longer experience to be able to see what it is like for software that I've created that's in production for three years, five years, ten years, and longer and what happens over time. It's a part of what I like with the SQLite long-term support page, they basically lead with a statement that our goal is to support this until the year 2050 and so, will they achieve that? hard to know… But just by making that statement, calling it out as a value for themselves and thinking about it actively, maybe this comes back to the long-now clock idea, which is, they may or may not succeed but certainly one good way to not succeed is to not think about it in the first place. And so this makes it a first-class concern.
Wiggins & McGranaghan, Metamuse Episode 49, Software Longevity, 30:14, https://open.spotify.com/episode/1DCNy94TBt3aoGbM8PNlRv?si=UT2qfBJAQWWjr5QsUGkunw
[McGranaghan]: […] that circles back to this idea of data. I see people typically design and implement programs […] we tend to focus on the interface and it behavior because that's what users are demanding because that's what the short-term success of our business depends on.
But the reality is that interface, including its implementation, is going to almost certainly fully churn at least once potentially several times over the course of software. But what's not going away is the data. And so, one thing that I think is helpful for longevity of software is really focusing on the underlying data model. Not only in the sense of how it's stored, which is mostly what we've been talking about in the podcast so far in terms of text files or SQLite, but also what is the data model itself. Like if you were to draw out the SQL tables, or equivalent, draw the boxes and put labels on them, it sounds really basic but it's a step that I feel like people often skip over and then it leads to all kinds of longevity headaches down the road.
Future-proofing I think is a word we sometimes use in general for talking about helping things be longer-lived, but certainly I think that's most notable in the data model, precisely for the reason you said: it's just much less changeable and harder to change and heavier weight to change.
McGranaghan & Wiggins, Metamuse Episode 49, Software Longevity, 31:20, https://open.spotify.com/episode/1DCNy94TBt3aoGbM8PNlRv?si=2S81RLe9Q4mA36etUZN7QA
[Wiggins] The version thing is also making me think of, I think it’s the image file format, maybe tiff is the one I'm thinking of, but this approach has been used in a lot of different places, and I feel like you're doing a version of this in the local-first sync infrastructure we’re working on right now.
And so you can add new chunks to a stream of data or to a file that older versions of the software can load, recognize they don't know this chunk and they can skip past it and just try to interpret the rest of it the best they can. Maybe this is something the web browsers do pretty well which is, you load a newer page in an older web browser, if there's some not supported features it doesn't completely break, it just does its best to render it. […] It's not that it just gives up the moment it sees something it doesn't understand.
[McGranaghan]: And this gets into the peculiar challenges of longevity with distributed system software. We've been talking about longevity in terms of supporting the past, but with distributed systems you gotta support the future because the future arrives unevenly. So that's the sort of challenge you deal with in your working on distributed systems like the data synchronization layer from Muse.
[Wiggins]: As usual Ink & Switch has some good research on this subject with the data lenses in the Cambria project. One of the things there is about data migrations, but it's assuming you need to translate both ways. You need to translate back in time the older versions and forward in time to newer versions, but in fact, it's actually more complex than that, because there may be just a branching tree of systems that all understand that the data in slightly different ways.
Wiggins & McGranaghan, Metamuse Episode 49, Software Longevity, 34:55, https://open.spotify.com/episode/1DCNy94TBt3aoGbM8PNlRv?si=A2dedJMzR4KhKIgM8COa2g
[Wiggins]: […] For archival purposes I have put a lot of effort in the past to preserving things I've created, whether it's a essay, or back when I used to make music, […] usually I find the kind of flattened artifact, read-only, is really kind of what I want in the long term because I do go back to reference those things or look at them for inspiration or just for take a walk down memory lane, but I'm very rarely want to edit, even if I could, and so in that sense just it's actually superior to flatten out to an image or a video or something.
[McGranaghan]: Yeah, it's a good point, in many ways, it's a benefit, not a downside, especially when you have this habit of building up an archive that's all on these handful of formats. Everything is basically text, PNG, PDF, maybe video, MP3, you can browse it all together and you can do it very quickly, basically, in Finder or the equivalent, whereas you can imagine, if for every picture you want to check out you had to open up Adobe Photoshop or whatever, you’d wait 2 minutes, so it's nice to have that very lightweight, archival copy.
Wiggins & McGranaghan, Metamuse Episode 49, Software Longevity, 38:21, https://open.spotify.com/episode/1DCNy94TBt3aoGbM8PNlRv?si=ttBllaQQTES8dUyfX6kGMA
[Wiggins]: And the other small thing, it's sort of obvious in some ways, it's kind of the Lindy Effect of file formats. So, Lindy effect here being sort of the idea that something that’s been around a long time probably will be around a long time.
So, I'm always a big fan of those flat file formats, PNG, JPEG, PDF, plain text, because they've proven themselves to be durable over the long term and they don't always do everything you need but where you can, it's really nice to just go down to that simple common format that is understood by many applications both now and in the past and potentially will be in the future.
Wiggins & McGranaghan, Metamuse Episode 49, Software Longevity, 36:41, https://open.spotify.com/episode/1DCNy94TBt3aoGbM8PNlRv?si=ozNX-91xRhKag6u_febOKw
In the fullness of time, the files you create are more important than the tools you use to create them. Apps are ephemeral, but your files have a chance to last.
The ancient temples of Egypt contain hieroglyphs that were chiseled in stone thousands of years ago. The ideas hieroglyphs convey are more important than the type of chisel that was used to carve them.
The world is filled with ideas from generations past, transmitted through many mediums, from clay tablets to manuscripts, paintings, sculptures, and tapestries. These artifacts are objects that you can touch, hold, own, store, preserve, and look at. To read something written on paper all you need is eyeballs.
Today, we are creating innumerable digital artifacts, but most of these artifacts are out of our control. They are stored on servers, in databases, gated behind an internet connection, and login to a cloud service. Even the files on your hard drive use proprietary formats that make them incompatible with older systems and other tools.
[…]
If you want your writing to still be readable on a computer from the 2060s or 2160s, it’s important that your notes can be read on a computer from the 1960s.
You should want the files you create to be durable, not only for posterity, but also for your future self. You never know when you might want to go back to something you created years or decades ago. Don’t lock your data into a format you can’t retrieve.
These days I write using an app I help make called Obsidian, but it’s a delusion to think it will last forever. The app will eventually become obsolete. It’s the plain text files I create that are designed to last. Who knows if anyone will want to read them besides me, but future me is enough of an audience to make it worthwhile.
Steph Ango, File over App, https://stephango.com/file-over-app