How Technologies Move From Empowering to Imprisoning
One of the big worries we have had about digital identity systems is that they will be embraced by the private sector for tracking. Last year we warned that once this infrastructure was created, the private sector is likely to leverage it to the max:
Seemingly overnight, you find demands to share your digital ID coming from every direction, online and off. Want to enter a convenience store? Scan your ID. Want to browse a clothing store, buy a cup of coffee, park your car? Scan your ID. When you try to watch a video online, log on to social media, or look at a news site, you get a pop-up demanding that you “click here to submit your driver’s license.” Now that the infrastructure exists, you find yourself asked to submit your license at every turn. And submit you will, because you will have little choice.
In July I made this point to a reporter for PBS, and in his story he included from Ian Grossman, the Executive Director of AAMVA, the little-known trade association for state DMVs that has been playing an outsized role in pushing digital IDs on the American public. “I think overall the consumer will drive that demand,” he told PBS, “and they will decide where they want to accept to be identified, versus where they’re going to look for another alternative where they want to stay anonymous.”
It’s a very comforting thought that we as consumers will have the power to stop omnipresent ID demands by, apparently, simply voting with our pocketbooks. If only we could have any faith that will be true.
I recently came across a that brilliantly outlines why we may not be so in control of our destinies. It was by an urban planning expert, of the University of North Carolina, on the dangers that autonomous vehicles (AVs) may present to individual choice and autonomy in transportation. He lays out a framework and evolution that he applies to AVs but much of which could also describe what happens with other new technologies we are forced to adopt even though they may be bad for our privacy interests or otherwise disempowering.
The dynamics he describes may also apply not only to digital driver’s licenses but also to past, current, and future convenience-promising but privacy-invading technologies. Credit cards were once an additional option — now it’s possible to live a fully modern life without one and there’s a war against cash underway. Face recognition is being pushed on consumers as a “convenient” replacement for tickets at sports stadiums and elsewhere. Many people find that exploring the possibilities of AI in their work is interesting in exciting — but the technology threatens to become mandatory in many fields.
“I’ve been thinking about the subtle, often indetectable shifts that turn an optional tool into a systemic requirement,” LaJeunesse writes. “When do infrastructure priorities change? When does non-AV mobility become framed as ‘legacy’? At what point does adaptive behavior get mistaken for real preference?” LaJeunesse told me his framework was inspired by the philosopher and social critic and the scientific literature on nonlinear phase shifts in ecology and elsewhere.
I could go through how each of LaJeunesse’s points applies to various technologies, but I think everyone will recognize these dynamics, so in the interest of brevity, I’ll just quote and paraphrase some of the major points from his slides.
FROM TOOL TO SYSTEM
First, LaJeunesse talks about how a technology, even when it is still optional, begins to structure behavior. He identifies three dynamics through which this occurs:
1. Baseline participation inflation
-
- Access to jobs, housing, or services increasingly assumes access to a technology
- Existing ways of doing business begin to be described as “legacy” or “traditional”
- Alternatives are framed as anomalies rather than norms
2. Planning dependency
-
- Infrastructure planning begins prioritizing users of the new technology
- Planning assumes penetration of the new technology
- Human variability and unpredictability are treated as “noise”
3. Preference laundering
-
- Surveys show “acceptance” despite lack of meaningful alternatives
- Use is framed as a revealed preference rather than constrained adoption (i.e. something forced upon us)
- Willingness-to-pay is used to judge success instead of human capability or autonomy measures
FROM EMPOWERING TO DISEMPOWERING
Eventually, LaJeunesse writes, a new technology, once an empowering option, becomes a disempowering requirement as people are boxed out of choosing not to use it.
1. Opt-out penalties
-
- Alternatives are restricted, unfairly priced, or stigmatized
- Costs of doing business rise for non-users
- Those who don’t use the new technology are portrayed as irrational, nostalgic, or obstructive
- Equity is framed as access to the new technology rather than freedom from dependence on it
2. Safety discourse
-
- Alternatives are framed as unsafe
- The safety discourse shifts from harm reduction to “system confidence.” (In other words, moving from “how do we protect people from the technology” to “how do we make sure the technology is trustworthy,” shifting the framing from protecting people to protecting the system.)
- Safety failures are reframed as “edge cases” rather than systemic failures
3. Democratic hollowing
-
- Governance of the new technology occurs via MOUs, pilots, or proprietary standards
- Decisions are framed as “too technical” for public contestation
- Local veto power erodes in favor of state or corporate control
This is a great encapsulation of how we can be forced to use technologies even when they invade our privacy and disempower us. What are the motivations and incentives that push these dynamics? LaJeunesse doesn’t address that in his piece, but I would say they include some combination of:
- Corporate profit incentives (including incentives to monetize data).
- Bureaucratic goals of efficiency and rationalization, which prioritize those ends over humanistic values.
- A particular conception of what “the future” looks like, and a conviction that it is inevitable. In particular, a conception that centers around rationalizing human life in ways that reflect the dominant metaphor of our age: computers.
But none of us should accept these technologies as inevitable, and should fight for richer, more humane visions of the future.
From Tool to System
First, LaJeunesse talks about how a technology, even when it is still optional, begins to structure behavior. He identifies three dynamics through which this occurs:
1. Baseline participation inflation
- Access to jobs, housing, or services increasingly assumes access to a technology
- Existing ways of doing business begin to be described as “legacy” or “traditional”
- Alternatives are framed as anomalies rather than norms
2. Planning dependency
- Infrastructure planning begins prioritizing users of the new technology
- Planning assumes penetration of the new technology
- Human variability and unpredictability are treated as “noise”
3. Preference laundering
- Surveys show “acceptance” despite lack of meaningful alternatives
- Use is framed as a revealed preference rather than constrained adoption (i.e. something forced upon us)
- Willingness-to-pay is used to judge success instead of human capability or autonomy measures
From Empowering to Disempowering
Eventually, LaJeunesse writes, a new technology, once an empowering option, becomes a disempowering requirement as people are boxed out of choosing not to use it.
Opt-out penalties
- Alternatives are restricted, unfairly priced, or stigmatized
- Costs of doing business rise for non-users
- Those who don’t use the new technology are portrayed as irrational, nostalgic, or obstructive
- Equity is framed as access to the new technology rather than freedom from dependence on it
Safety discourse
- Alternatives are framed as unsafe
- The safety discourse shifts from harm reduction to “system confidence.” (In other words, moving from “how do we protect people from the technology” to “how do we make sure the technology is trustworthy,” thereby shifting the framing from protecting people to protecting the system.)
- Safety failures are reframed as “edge cases” rather than systemic failures
Hollowing-out of democratic processes
- Governance of the new technology occurs via MOUs, pilots, or proprietary standards
- Decisions are framed as “too technical” for public contestation
- Local veto power erodes in favor of state or corporate control
Overall, this is a brilliant encapsulation of how technologies, including those that invade privacy and disempower us, can get forced upon us. What are the motivations and incentives that push these dynamics? LaJeunesse doesn’t address that in his piece, but I would say they include some combination of:
- Corporate profit incentives (including incentives to monetize data).
- Bureaucratic goals of efficiency and rationalization, which prioritize those ends over humanistic values.
- A particular conception of what “the future” looks like, and a conviction that it is inevitable. In particular, a conception that centers around rationalizing human life in ways that reflect the dominant metaphor of our age: computers.
But none of us should accept these technologies as inevitable, and should fight for richer, more humane visions of the future.