Public data ought to be public. It isn’t if you cannot download and manipulate it with common computer programs.
North Carolina voter rolls were a very stark example of how the Secretary of State can take something one would think is public and make it impenetrable.
The voter integrity team in North Carolina tried to download registration rolls and they were just a mess. Their computer programs, common ones like Excel, could not easily deal with the North Carolina voter registration data.
The Fractal team downloaded North Carolina data and immediately found the problem. The data had been hijacked internally to make it pretty much unreadable by common computer programs.
The Secretary of State’s data team was either incompetent or sinister or possibly both. The data was filled with control characters mixed in with the text that made the data unreadable.
There were partial quote marks.
What’s that?
Often, “quote marks” like the ones just typed in are used as delimiters. Everyone does it in Dataland. But not North Carolina.
They used one of the quote marks and left out the other.
This was not something done once, it was persistent.
So a computer could not tell where a field ended and another began.
Fortunately, the Fractal team ingests this kind of data and could make sense of it. So now, North Carolina data can be downloaded, made useful and citizens can get their tax money’s worth.
The lesson here, however, is that the government can be sinister or incompetent or both and citizens need a technology to keep them honest.
Welcome to Project Omega.