[Wiggins]: If we come to software as a special class of information, and it is information but it's highly dynamic, and I think that points to some of the particular challenges with it in […] software itself and its ability to be long-lived.
And I think some of this is cultural, the tech industry is dynamic. It thrives on change, that's sort of the nature of it and many things that are good about software and the internet and the tech world come from that willingness to embrace the new and almost an endless seeking of novelty.
But the downside is, you get something that is not just an upgrade treadmill but it just seems like it can be very much the case that the lifetime of product or of data or any particular piece of software can be measured in years, and only a small number of years, which is comparatively just a very, very short time span. […]
But I feel there's more to it than just cultural, the tech world doesn't do the buy-it-for-life / built-to-last thing. I feel like there's something inherent to the dynamic-information nature of software, that makes it hard for it to be long-lived.
[McGranaghan]: Yes, I think that's true. There's a lot of things that need to go right for software to work, and you can see it, in fact, as something that's strictly harder than preserving simple textual information for the following reason:
You have at least the source code, then you need to preserve information about the dependencies. You potentially have the compiled source and then you have the whole runtime around it, very broadly defined as the computer and the APIs, maybe even the data. And this is just a very broad, multi-dimensional, and many cases, ill-specified state space: if you don't get your thing, exactly right in that state space, it doesn't work at all. Which, by the way, is something that's a little bit unique to digital systems versus classic analog systems. If your wrench is Rusty, you know, doesn't work as well, but it's still basically works as a wrench. Whereas, if you have one bit off in your software it just crashes - you're out of luck. It's very high-dimensional and it's very sensitive to errors in preserving the environment.
Wiggins and McGranaghan, Metamuse Episode 49, Software Longevity, 10:52, https://open.spotify.com/episode/1DCNy94TBt3aoGbM8PNlRv?si=u0GQfL1US5ujSnD0yl1Jsw
[Wiggins]: First I have to ask what it means to you to talk about something being long-lived or having longevity, whether its software or a plane or something else.
[McGranaghan]: I think it's the software being able to serve its stated purpose, which sounds very straightforward, but has several important constituent parts:
- First of all, you actually need to software, you know, sometimes we just lose it.
- It needs to run and needs to run correctly and
- it needs to have access to the relevant data.
- It needs to have access to relevant pieces of the outside world in terms of APIs and
- it needs the appropriate substrate to run on an interact with.
- And it's probably other pieces that we could come up with.
McGranaghan and Wiggins, Metamuse Episode 49, Software Longevity, 04:20, https://open.spotify.com/episode/1DCNy94TBt3aoGbM8PNlRv?si=ca3gfTLARQ2EExDeBeQ5yA
[Wiggins]: Even thinking of a binary artifact, you've got a compiled executable, you can download it, you don't need to the dependencies, you don't need the compiler chain and I think there's the feeling that “shouldn’t that work forever?”. But of course as you said it sits within this context of the system: if it's accessing files on your file system, it's calling out to APIs on the network, it's accessing APIs in the operating system. Even something like it just makes assumptions about what kind of hardware exists, the keyboard and mouse has being standard input devices that exist and then you go onto a touch device and those aren't there. So over time, those fundamental assumptions and apis and the system changes around it and I think that's gotten dramatically more so with the internet, where you might call out to even something as simple as a static website, that has a Google Maps embed, well, now a big portion of that website is dependent on this exact integration to the service and that service is still online. So I think that the one-off binary build, shouldn’t-it-work-forever does make some sense in, for example, games, which don't have as many hooks (let's say just like a single player game, not some complicated cooperative thing) as it tends to have very simple inputs and outputs, displays things on the screen, maybe it needs to save a high-score file to the disc, as opposed to productivity software where all those little integration points: you drag and drop and how do you open a thing in the browser and all the different apis that are used to be a good citizen on the system, and those change all the time - they should with an operating system that's growing and improving and evolving - but then that means the apps have to keep up with that. And if they don't, they fall out of date. And eventually either become somewhat irrelevant or more commonly just stop working.
[McGranaghan]: Yeah, I think this idea of the complexity of the surface area and the APIs is very important. You mentioned games: I think we actually had it much easier back in the good old days of games where we had keyboard input, mouse input, you draw to a pixel buffer, and that's it. The issue with games now, in addition to stuff around networking and multiplayer is that the graphics pipeline is incredibly complex, and unless you end up emulating earlier versions of graphics pipeline there's little chance this stuff rolls forward, whereas you could plausibly roll forward writing into a pixel bus offering. You kind of go the other way - you can emulate a pixel buffer with our advanced graphics APIs today, but there’s no chance of going the other direction. I think this generalizes to API complexity broadly and it's one of the reasons why it's so hard for software to persist
Wiggins and McGranaghan, Metamuse Episode 49, Software Longevity, 13:12, https://open.spotify.com/episode/1DCNy94TBt3aoGbM8PNlRv?si=ST_DRl-YQ3W4nxTMYYOD3w
[Wiggins] The version thing is also making me think of, I think it’s the image file format, maybe tiff is the one I'm thinking of, but this approach has been used in a lot of different places, and I feel like you're doing a version of this in the local-first sync infrastructure we’re working on right now.
And so you can add new chunks to a stream of data or to a file that older versions of the software can load, recognize they don't know this chunk and they can skip past it and just try to interpret the rest of it the best they can. Maybe this is something the web browsers do pretty well which is, you load a newer page in an older web browser, if there's some not supported features it doesn't completely break, it just does its best to render it. […] It's not that it just gives up the moment it sees something it doesn't understand.
[McGranaghan]: And this gets into the peculiar challenges of longevity with distributed system software. We've been talking about longevity in terms of supporting the past, but with distributed systems you gotta support the future because the future arrives unevenly. So that's the sort of challenge you deal with in your working on distributed systems like the data synchronization layer from Muse.
[Wiggins]: As usual Ink & Switch has some good research on this subject with the data lenses in the Cambria project. One of the things there is about data migrations, but it's assuming you need to translate both ways. You need to translate back in time the older versions and forward in time to newer versions, but in fact, it's actually more complex than that, because there may be just a branching tree of systems that all understand that the data in slightly different ways.
Wiggins & McGranaghan, Metamuse Episode 49, Software Longevity, 34:55, https://open.spotify.com/episode/1DCNy94TBt3aoGbM8PNlRv?si=A2dedJMzR4KhKIgM8COa2g