Disappearing data and need for censorship-resistant networks

We have technologies to make publicly funded data more durable

Publicly funded data hosted on websites operated by the United States federal government are disappearing.

Noticeable changes to the amount of data available first started in mid-January, following the change in administration.

According to a Wired story:

"Decades worth of taxpayer-funded reports and analysis gone in an instant," says the employee, who requested anonymity because they are not authorized to speak to the press. "We have no idea what is happening behind the scenes or what will be back, when, and in what form."

Other reports find that entire datasets are no longer available. The removal of data raises issues on multiple levels, including:

who owns data funded by tax-payer dollars
the risks of closed or centralized systems
data censorship to align with political objectives

Archiving and cataloging massive amounts of federal data is not easy. That's why data.gov was created in 2009. The goal, at first, was to make government data more accessible to scientists, researchers, and accountability groups.

In 2019, legislation further compelled government agencies funded by public money to adopt data collection and reporting standards under the Open Government Data Act.

Still, collecting and organizing massive amounts of public data is complex. Even in a world where the data isn't under threat of censorship or erasure, it's a massive project to keep links to datasets current (or prevent link rot) and to standardize government data in a way that makes it accessible to the general public.

So, a naive interpretation of what is happening is that a change in administration is disrupting websites and links to datasets, making it appear as if the data is disappearing.

Or, as 404 Media reported:

"Because data.gov is an aggregator that doesn't always host the data itself, this doesn't always mean that the data itself has been deleted, that it doesn't exist elsewhere on federal government websites, or that it won't be re-hosted elsewhere. Further research will be necessary to determine what has happened to any given dataset, or to see if it turns up elsewhere on a government website."

A more cynical interpretation of the disappearing data situation is that the changes attempt to make research realities bend to political ideas.

Again, according to the 404 Media story about this:

"Disproportionately, the datasets that are no longer accessible through the portal come from the Department of Energy, the National Oceanic and Atmospheric Administration, the Department of the Interior, NASA, and the Environmental Protection Agency. But determining what is actually gone and what has simply moved or is backed up elsewhere by the government is a manual task, and it's too early to say for sure what is gone and what may have been renamed or updated with a newer version."

Why is this an Open Money issue?

Open Money is a framework designed to help understand the shift happening in financial technologies.

The goal is to help understand how the pivot toward open systems based on cryptography and distributed ledgers (like blockchain) will make money more accessible and efficient and make room for new kinds of economic activities.

Open Money is an iteration or more specific offshoot of earlier open-source movements designed to make information more accessible and resilient to change or corruption. In was in this same spirit that data.gov was first created.

Opening data (or opening money systems) is the first step in creating more access. The next phase — making the systems durable and censorship-resistant — will require using technologies that make it impossible for data to disappear.

We looked into this issue in mid-2024 when we discussed the Arweave blockchain and the idea of the permaweb. Even though that post is a little bit more than six months old, it feels like it was written a lifetime ago — when the idea that the federal government would start disappearing publicly funded data still felt like some kind of far-off, worst-case scenario.

Since the integrity and utility of open data are essential facets of the evolution of the Open Money story, we'll keep an eye on the government data story.

In the meantime, organizations like the Harvard Law School Library Innovation Lab are working to preserve government data before it disappears. If you know of a dataset that should be part of those preservation efforts, you can contact the project at lil@law.harvard.edu.

Progress on the Open Money project

This week, we wrapped up section two of the Open Money project. The main theme was a high-level look at what makes money and how money is really just an elaborate data system that's been developed over thousands of years.

The middle of the section is about the parts of the current money system that work well (it's generally stable and well-regulated). This part balances against the parts that don't work well, mainly because it's hard to create economies at either end of the spectrum.

Micro, or hyper-localized economies, don't work with the current money system, and global-scale systems are also a challenge. It's like we are stuck with a medium-sized money system. But Open Money could change that — making use cases at the edges more viable by making them more valuable.

The section ends with a post about why people haven't been able to create some universal money system — at least not yet.

From here, we will dive into section three by taking a closer look at some of the concepts that make it possible to build Open Money systems. As always, let me know what you think.

Why is this an Open Money issue?

Progress on the Open Money project

Recent posts in the Open Money project