What decentralization requires

My thoughts for decentralization are to follow the model of podcasts and revisit and update what Feedburner once was.

Multiple providers for data content hosting (you pay for this service and own all content) - RSS standards for publication of short form posts (ala Twitter), long form posts (facebook / blog posts), video content (YouTube). Software & App platforms are developed to pull in content, just like Podcast apps do. Then people have to get creative about how to monetize content, but that's something that could be provided by the apps serving up the content.

If your data hosting provider decided they no longer wish to host your content, you upload your content to a host who does.

Yes, I agree

I say look at how usenet worked (and still works to some extent).  I'm an old fart and we used usenet (uucp) for years on 4800 bps leased line modems to move a lot of message traffic.  Today, a $40 USD Raspberry pi could act as a node.  Also, bit torrent worked pretty well for distributed file sharing.  Dont re-invent the wheel unless you really need to.

Hello. @zxq9 and I have been working on writing a forum post that somewhat addresses those questions. The short answer is those are very high-level requirements, and in order to meaningfully address them, work needs to be done to meet many low-level requirements that are not currently met.

Title:
    On the topic of the Parler debacle, and introducing the Orange Pill project

Body:

I am seeing a lot of posts on this forum asking bad questions, and even more
people replying with bad answers.  Creating decentralized replacements for
social networks and video hosting sites does NOT amount to simply writing
browser extensions, WordPress plugins, or web apps.

The basic technology required to create a truly decentralized internet does not
currently exist.

As an analogy: before you write a web app, you need HTTP to exist. Before HTTP,
you need TCP. The decentralized web is at the stage where it doesn't have TCP
yet.

Let me now pivot to an anecdote, and I will circle back and explain its
relevance.

Parler has been canceled. It's all over the news, but if you aren't familiar
with the story, here are two articles:

1.  https://archive.vn/33i7a 
2.  https://archive.vn/4jaBd 

Additionally, there are unconfirmed rumors that

1. Parler suffered a database breach that has led to leakage of private user
   data such as cell phone numbers and scans of photo IDs

    https://archive.vn/AFSa8 

2. Parler failed to scrub metadata from files uploaded to the website (such as
   geotagging in photos), resulting in leakage of identifying information such
   as GPS coordinates where photos were taken.

    https://archive.vn/7i8kO 

I haven't seen any concrete evidence of the latter two claims, just reports.

According to those articles, Parler is built on WordPress. WordPress is
notoriously insecure. Supposing that it is true that Parler is built on
WordPress, leakage of user data shouldn't surprise anyone. For instance, the
Panama Papers leak was in part made possible by poorly-written WordPress
plugins (  https://archive.vn/APr5Y  ).  Moreover, Parler chose to build their
house on enemy turf (i.e. AWS), with predictable results.

Bottom line: the reason Parler is in the situation they are in is because they
made every bad design decision one could possibly make.

Parler teaches us an important point: we cannot skip over getting the low-level
technical details right. Duct taping existing garbage technology together is
not an adequate vision for a decentralized internet.

With that in mind, we introduce our new project, the Orange Pill Storage
System (opss or OPSS).

The following is a copy of the current draft of the OPSS manifesto. The OPSS
manifesto should answer obvious questions like ``Why are you creating a new
thing?'', ``Why should I care?'', ``What can I do to help?'', and ``Why don't
you just use $ExistingThing?''.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
THE ORANGE PILL STORAGE SYSTEM (opss) MANIFESTO [draft]
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

by
    Dr. Ajay Kumar PHD, The Founder <[email protected]>
    Craig Everett <zxq9@zxq9.com>

Version 0.1 [draft]
January 11, 2021


This document will be updated. The latest version can be found here:
 https://gitlab.com/DoctorAjayKumar/opss/-/blob/master/opss_manifesto 


        The bottom line: there is no off-the-shelf system with all of the
        necessary properties. There is no getting around the fact that getting
        the infrastructure *right* and properly implemented is a necessary
        precondition to application development. Look no further than the
        Parler debacle.

    -- Craig Everett

-------------------------------------------------------------------------------
ABSTRACT
-------------------------------------------------------------------------------

opss is a decentralized data storage system. opss is the first low-level step
required in solving the cancelation problem. The cancelation problem is the
ability of fiduciaries in centralized networks to delete ("cancel") nodes in
the network.  In order to build distributed networks, one needs software and
protocols for storing, and distributing data in a decentralized manner. That is
what opss aims to be.

-------------------------------------------------------------------------------
INTRODUCTION (by Craig Everett)
-------------------------------------------------------------------------------

I have wanted to write a data system to solve the social media censorship
problem and have been discussing one with Dr. Kumar for quite a while. I'll
explain in very rough terms how such a thing can work without getting too in
the weeds.

First off: Separation of concerns.

1. Distributed data is about infrastructure: retrieving the correct bits,
   verifying they are the expected bits, and hopefully doing that in a timely
   manner.

2. Social media features are an application issue and an orthogonal concern.

You could certainly write a distributed social media app that conflates
everything together, but that would be a massive waste of effort.

The same infrastructure that will support the creation of a Twitter-like
alternative will also support the creation of a YouTube-like alternative.

There is a tradeoff between anonymity and latency.  In the context of social
media, latency is the more valuable feature to favor. This does not mean ``real
names'' are necessarily known. ``Real names'' is an application-level issue.
Rather, just as in any other peer-to-peer system, canonical network origin is
known. CANONICAL NETWORK ORIGIN in essence means the IP address of a node
providing data.

In the case of a very popular resource being located somewhere in a
bandwidth-constrained network (residential networks, for example), pure peer
systems suffer from resource contention.  Systems such as BitTorrent get around
this by chunking data and distributing them broadly. A robust data
infrastructure must combine approaches to achieve both of robust, distributed
data provision in addition to having very high responsiveness on the level
required by streaming, chat, and similar services.

The way this manifests in software will feel familiar to anyone who has
used a P2P application before:

- The users' systems are the physical infrastructure the system runs on
- The nodes inform each other of other available connections to form a data
  distribution mesh
- A canonical name registry origin is defined, with the contents of that
  registry being write-only to enable distribution and caching of it (so
  that outages do not damage the network -- this is as close to a
  single-point-of-failure the system can have)

There are a million little details to tweak on the way to making that a highly
responsive system. But that's the basics of it.

There is the detail of variable network transmission (direct TCP for those who
know how to forward ports, UDP hole punching for those that don't, UPnP where
it is available, proxying schemes for passing data around blocked intermediate
routes and so on -- every approach must be employed at once), and this is where
a lot of the actual work needs to be put in.

The bottom line: there is no off-the-shelf system with all of the necessary
properties. There is no getting around the fact that getting the infrastructure
*right* and properly implemented is a necessary precondition to application
development. Look no further than the Parler debacle.

-------------------------------------------------------------------------------
DEFINITIONS
-------------------------------------------------------------------------------

We use the term "software" to refer to both software and protocols.

We define classes of software called Based and OrangePilled.

Roughly speaking, ``Based'' refers to technical details of how
addressing/routing is implemented, and ``OrangePilled'' refers to the technical
details of its trust model.

Based software is software that has all of the following properties:

    - canonical origin for the data
    - coordinated torrenting from cache
    - distributed data cache
    - redundant connection methods between nodes
    - redundant routing
    - streaming capability

OrangePilled software is software that has all of the following properties:

    - ability to both push and pull
    - access control
    - no trust network or trust network contains no cycles
    - no trust network or trust network contains no permanently trusted nodes

-------------------------------------------------------------------------------
OVERVIEW OF EXISTING SOFTWARE
-------------------------------------------------------------------------------

Our claim is that no existing software is both Based and OrangePilled,
and most software is neither.

By definition, any centralized software contains a trust network with a
permanently trusted node. Therefore, no centralized software is OrangePilled.

Existing decentralized software all fails at least one criterion. As examples,
we examine

- BitTorrent
- Freenet
- I2P
- IPFS
- OrbitDB
- Tor

TABLE OF FEATURE DEFICITS

    FEATURE                                     | HAS DEFICIT
    --------------------------------------------+--------------------------------------------
    ability to both push and pull               | all
    access control                              | BitTorrent, Freenet, IPFS, OrbitDB
    canonical origin for the data               | BitTorrent, Freenet, IPFS, OrbitDB
    coordinated torrenting from cache           | all
    distributed data cache                      | I2P, Tor
    no cycles of trust                          | none
    no permanently trusted nodes                | none
    redundant connection methods between nodes  | all
    redundant routing                           | BitTorrent
    streaming capability                        | all

-------------------------------------------------------------------------------
ABOUT THE AUTHORS
-------------------------------------------------------------------------------

Dr. Ajay Kumar PHD is a mathematician.

Craig Everett is a distributed systems engineer.
</zxq9@zxq9.com></[email protected]>

@doctorajaykumar

This is an interesting proposition; where can I learn more?

I agree in that the concern here needs to be focused at lower-level domains. That said, I find myself asking: should we be focused on developing and hardening a new infrastructure, or is it perhaps more worthwhile to find a means to enable the layman to interface with one of these existing - albeit flawed as you've noted - protocols? I defer to a previous post mentioning the old adage about reinventing the wheel.

I think there exist lower layer systems sufficient for something "Orange Pill"; what we need to be looking at here, then, is encapsulation, transport, and provenance. How does one donate computing power without suspending privacy or protections of IP provenance? How do we enable the layperson to contribute to such an architecture in a way that is at least moderately accessible? There's protocols like Lokinet but I worry that their blockchain integration - while certainly an apt way to gamify and incentivise their participants - is off-putting to many of the large amount of persons needed to make something like this functional. Too, I imagine for someone who isn't a software developer or like-kind, configuring a service node isn't exactly fun either.

Perhaps it's by way of my own proclivities that I'm so subscribed to the idea of creating some sort of API for a federated system, but I truly think this is what's lacking on the web right now. Put simply, the foundation is available but it's a pain in the ass for the average internet user to participate in. That needs to change, now more than ever.

...

As an aside, I looked into these claims of metadata scraping from Parler. While unsophisticated, the data dumps did happen. An individual apparently found an unprotected API that rather liberally served user metadata contingent on whatever query params you threw at it. It's very disappointing to see someone leveraging their ability to write code to blatantly virtue signal and expressly violate others' privacy. This entire debacle is exceedingly ridiculous and the internet is in a sad state because of it.

I would emphasize the following principles:

The right to make a mistake

People should be able to remove their own posts.

The right to start all over again

People should be able to abandon their old identity and create a new one.

The right for an active life

People should be able to have multiple identities separating different areas of their life.