Sanctuary (V0)

Anyone reading this might have their own idea of what the constrained web is. Sanctuary is meant to avoid these problems in the first place by design.

Sanctuary is a global document space organized as a tree structure. The root level is universally mutable - anyone can add content. But every document below the home block carries its own mutation rules that govern who can modify it or add children.

Each piece of content is simultaneously:

1. The content itself
2. The rules for who can change it

You cannot create content on Sanctuary without simultaneously defining who has power over that content.

Slang is the DSL for Sanctuary; it resembles Unison or Nix. It defines documents and their governance and permission logic. Slang is tailored to building and organizing documents on Sanctuary and allowing user interaction (mutation) through actions. It is not meant for general-purpose computation, like the modern web.

To read and write Sanctuary, you need a user interface called a slang client. Slang clients are dumb; their only job is to manage the presentation of the document structure and provide a way to commit actions.

The client is a frontend to the slang interpreter, which is where most of the magic happens. The interpreter is the process which manages your personal copy of the Sanctuary network. This copy is stored in a database on your computer, and it updates the database when it receives valid actions from you or from the network.

The average user does not need to learn how to program actions, and their experience using Sanctuary client should feel familiar. Slang includes presentation hints so documents can be organized like Reddit, Wikipedia, or other online spaces. actions which allow interaction (like commenting) are clickable and show a generic interface for adding text.

The Sanctuary document structure is stored in a database which is replicated across the different users of Sanctuary. Each node only holds their own view into the network with the communities and documents they have chosen to participate in. Peers are found using the BitTorrent DHT. Sanctuary itself is an extension of the BitTorrent protocol, and documents are actual torrents. Sanctuary nodes are valid torrent clients and may download regular torrents.

Each node in the Sanctuary network is a Slang interpreter - quietly gossiping with peers, validating and sharing changes, and keeping its database up to date.

Documents can optionally specify one or more 'seed nodes' to improve availability. Seed nodes do not have any special permissions or trust, but are simply meant to be nodes that are reliably online.

The slang we'll talk about here is called "interactive slang", and it's what you can read and write in a client. You can also write it in a text editor and send it to the interpreter manually. It is an abstraction that allows you to view and modify your Sanctuary database. It has cryptographic details replaced with a more human-readable shorthand.

The underlying object and network model "bleed" into interactive slang in some places. We'll briefly cover them here before we dive into writing useful code.

#{ // begin new block
    // variable assignments
    my_int = 4;
    my_empty_block = #{};
    "Some text";     // text content
    #{
	"this is a child block";
    };
};

Before we get into the the details of the language, let's open up a Slang client so we can write and test code.

When you first open your client, you should see something like this:

// home block
#{
    // edit or add slang here

    actions = #{
	✍ = <everyone> -> edit(this, $new);	
    };
}

The client may render it differently, and the actions variable at the bottom may be hidden, but either way, you are looking at the same thing. This is the home block, a conceptual root where all blocks in Sanctuary live. You can edit slang code inside this block, and when you join other communities they will also live here as top-level blocks.

To edit code in this block, you should be able to click the "✍" button at the bottom of the block - but many clients will also let you just click anywhere in the block to start editing.

One final note before we start adding code: your ability to edit this block is granted by the actions section, specifically the "✍" variable - this defines an action, which as you may have guessed, grants all users the power to edit the home block. This is worth thinking about for a second. Sanctuary is yours to edit. Your right to edit is part of the protocol. And an actions section like this is behind every single mutation on the whole network.

Naming the action with the Unicode grapheme like "✍" is typical; clients will expose actions as buttons that the user can click on at the bottom of blocks.

Alright, let's write some code.

#{
    title = "My first block";
    slang_version = 0.1.1;
    me = @whatever;
};

For the last variable, your client should let you autocomplete between keys or generate a new key.

When you started editing, your client may have changed the color of your home block, indicating you're in the process of committing an action. Look for an "evaluate" or "run" button and click it (you may get a second dialog that shows the changes and asks you to confirm). Your client should indicate that the code evaluated successfully, and you should see the new block as part of your home block, with your title clearly indicated.

If your slang code wasn't valid, your editor will likely do two things:

1. Display an "action rejected" message, with a reason for the rejection
2. Drop you back into the editor where the error was detected

Actions can be rejected for all sorts of reasons - some simply expire after a certain amount of time, or fail to reach consensus, or become invalid for some other reason. It's the job of the interpreter to constantly evaluate and validate changes - nothing invalid is allowed to be stored in your database.

Normally the rejection would be silent, but in this case the client recognizes that you created the action, so you might want to keep editing it until it becomes valid.

Click inside the new block you created and try to edit it.

You should discover that your block is immutable 🤯. The reason is that the block didn't contain any actions allowing you to mutate it. Once you add a block to Sanctuary, it's a living thing, and it can only evolve according to the rules you set when you created it. This means there is a lot of pressure to design your blocks well! You can study other code in the community, or even fork or import "community approved" code in your blocks. Over time, a consensus around effective design patterns will emerge, and some code will even become part of the Slang standard library.

For now, we'll create something new. If you want to remove the block you created, your client should have an "X" button, or it may allow you to select and delete the text. Just know that this doesn't delete the block, it just removes your view into it (we'll talk about views later).

Let's create something more useful. Any type of textual communication platform you've used online can be built in Slang.

For this example, we'll create a simple messageboard. People can post topics on the board with a title, and then users can respond to that topic. We'll make the comments "threaded", like Slashdot or Reddit, and you, as the admin, will retain power to remove or edit topics and comments.

Let's say the topic of your messageboard is the video game Skyrim. And for this example, let's say your name is Antonio.

Let's fill out a homepage that shows an overview of the forum, and we'll give ourselves edit permission:

#{
    title = "Skyrim Community Adventures";
    slang_version = 0.1.1;
    admins = <@antonio>;	// your client will let you select a key for "antonio"
    users = <>;

    "Welcome to my cozy little Skyrim forum! Create a new topic for your
next playthrough and share it with the community! Be sure to read the
FAQ and look through our other playthroughs!";

    posts = #{
	title = "Posts";
	// Display the title of each of the following topics
	presentation = .title;

	// Hardcoded first FAQ post. No comments section and editable by admins
	#{
	    title = "FAQ";
	    """
Q: Do you allow mods?
A: Yes, mods are fine, but please mention them in your playthrough summary!
...
""";      
	};

	// ... more posts will go here

    }

    actions = #{
	// Admins can edit this document (remove posts, change the
	// title or welcome text, edit the "Posts" block, add admins
	// or users etc.)
	admin_edit = admins -> edit(this, $new);
};

After evaluating this, you should have the new forum in your home block. The posts variable contains the first child block with the future list of posts, starting with the FAQ. But there's still no way for users to create posts. Let's add an action for that in the posts block.

Side note: You may be wondering how we can edit the posts block, if it doesn't have any actions. The answer is, we can't! That block is forever immutable. When our clients let us edit immutable blocks like this, we're really creating a brand new block and replacing the old one with it in the parent block. If admins need to remove posts, they'll also use this same power. We're doing things this way because it's simple - but in real life, it'd be common to add a more complex (perhaps collaborative) moderation system to the posts block, and limit the power of the admins in the parent block. You don't need to think too hard about this right now - the community will generate useful templates and best practices. But it's worth knowing that this forum we're building is fatally flawed, and likely wouldn't be accepted by many communities.

For each post, we want to enable comments, so we'll need to define a "comment" function to put in the actions block of each post and each comment.

We run into a recursive problem. To respond to comments, you need a new "comment" action on each comment; the comment function includes itself.

One of the easiest and safest solves here is to put the comment function by itself somewhere - like the posts block- then we can just link to it using the # block notation discussed earlier. Type the pound character, and then let your client help you select the main forum, then the dot notation to select the .posts.comment variable

// Defining a recursive comment function, which we will enable for posts
comment = fn($comment_text) {
    // block which will contain the comment
    child = #{
	actions = #{
	    // action calls this same function, referencing the root block
	    comment = <admins.., users..> -> #skyrim_forum.posts.comment;
	};
    };
    insert_text(child[0], $comment_text);
    append(this, child);
};

// in the posts block
actions = #{
    // creates a new post under the FAQ 
    create_post = <admins.., users..> -> fn($title, $post_text) {
	post = #{
	    title = "";
	    text = "";
	    actions = #{
		// enable comments on the post
		comment = <admins.., users..> -> #skyrim_forum.posts.comment
		};
	    // child comment blocks will display threaded
	    organisation = .threaded;
	};
	edit(post.title, $title);
	edit(post.text, $post_text);
	insert_after(this.nth_child(1), // after FAQ, which is the first child block in the forum
		     post);
    };
};

The home block is a conceptual root defined by the language standard. It does not have a hash and cannot be linked to. Every block on Sanctuary lives in the home block. Individual nodes/users only see their personal piece of the network in the home block. The home block has one action that enables the builtin edit function for everyone. Clients may choose what to name this action.

// home block
#{

    actions = #{
	✍ = <everyone> -> fn () { edit(this, $new)};
    };
}

Whether a block is written out with curly braces or as a hash link does not matter; the interpretation and storage is identical. Therefore, choose the most convenient/sensible method when writing slang. For instance, when referencing a variable in another existing block, it makes sense to reference the hash, rather than rewriting the block. When designing the child of a block, you may write it directly in its parent, or work on it separately in the home block and then add the hash in the intended parent.

Blocks can only have 3 types of statements: variable declarations, child blocks, and strings. All statements are semicolon-terminated.

Variable declarations use the equals sign. The value may be any primitive type, a string or block, a function or action, or even an expression. Expressions are not evaluated in variable declarations, but they may be reused in action blocks or functions. Variables can be references with dot syntax, like objects in other programming languages.

Child blocks and strings can be added without variable names and interleaved throughout the block to achieve the desired document structure. The difference between code and "content" is fluid and left to users. The entire block is visible to every user. Presentation and organization hints may collapse parts of the document like metadata or actions, but clients may ignore them.

Actions are the only way to mutate the document tree in Sanctuary. Every single block in slang is the result of executing an action (with the exception of the home block which is not a real block). The only way to add slang code, (including actions), is by executing actions.

We'll start with the most basic example from the home block:

// home block
#{
    // edit or add slang here

    actions = #{
	✍ = <everyone> -> edit(this, $new);
    };
}

<entity set> -> function definition

These primitives allow you to reconstruct the interaction patterns of most successful discourse platforms: Reddit, Discourse, Wikipedia, Stack Overflow, blogs, etc.

Different clients can interpret these hints differently - mobile clients might present threaded conversations in a more compact format, for example.

Every object (a URL, GPS coordinate, text string) can be hashed to create a convergence point. Multiple communities can build different structures on the same hash. For instance, a news article URL might have:

• An unmoderated comment section
• A heavily moderated version using one community's trust network
• A version with specific topic-focus rules
• A technical discussion version

I also want clients to include maps and the ability to chat with people on the map. Users should be able to connect with neighbors or others in their geographic area without requiring a specific service or account. (This is fairly easy by hashing What3words, for example).

I know how these work, but still not 100% sure how much these features will bleed into the language and how explicit they will be.

Remember, this is still the Slang reference, to learn more about the network, go down to The Sanctuary Network.

Community libraries consist of top-level blocks on the network itself, pinned to specific Slang versions. Different parts of the network can run different Slang versions simultaneously, within an acceptable version window. Communities upgrade on their own schedules.

Standard library functions and primitives (bundled with the protocol and interpreter) should be kept as small as possible to start with.

Mostly not a concern of the language, but the interpreter and will be described below. Regular torrents can be included in any document, probably with a different syntax. So you can embed pictures, videos, PDFs, directories, whatever. But the interpreter will not download them automatically.

Communities, documents, and entities can have explicit lifecycle and expiration mechanisms.

Objects are stored in a content-addressable DB, like Git, Restic, IPFS, Nix, and so on. Sanctuary "blocks" and actions are both stored together as objects.

Actions are a linear, backward-linked chain of signed mutation objects (similar to git commits, without branching). Each block maintains its own independent action chain. Actions affecting the same block must be totally ordered.

To verify a document, you replay the root block's action chain. For each child block referenced in a root action, recursively replay its action chain to get its current state.

Child blocks’ latest hashes are not stored in the parent; parents reference the child hash at the time of the action. Any block can act as a root or a child in different contexts. Blocks can be shared across multiple roots or parents.

There is no concept of a canonical document root - documents can be freely interlinked and composed. But often there's a root suggested by the actions and views.

• Actions may create or delete child blocks (by hash) or mutate existing content.
• Separate actions table contains mutable references to the latest action for each block (key is the original block hash, value is the hash of the latest action)
• To reconstruct the DB from scratch, you need a copy of the DB (from a swarm), and a pointer to the latest action on the root block. You can then traverse up the action chain and validate the db while updating your 'view' (ref) to the root block as it mutates.
• Except for "orphaning" blocks (which can include whole communities that the user wants to clean), the object database never cleans old blocks. They're part of the history of the documents.

• "views" are like git refs - simply a pointer to a block that you've indicated you want visible in your home block.
• In the purest sense, Slang code is totally composable and flexible. Any block can contain any block, even creating circular situations - it's similar to roam research's links. Views indicate where you want your 'root' to be for a document structure - if a child block includes that root, it will just render as a link upwards. (The circular links are generally discouraged, but they are perfectly valid)
• So while slang can contain natural pointers to views, views are really a client-side decision and choice.

Sanctuary will be built directly on top of BitTorrent infrastructure, with every Sanctuary client bundling a BitTorrent client.

Between snapshots, mutations accumulate and are shared via a slightly extended BitTorrent protocol in swarms.

This is the only place where considerations about the performance and distribution of Sanctuary on the network 'leak' into slang syntax itself. A swarm is basically a hint to peers that they can organize around a specific block. A common use case is an independent community. Peers organize around the original root block, possibly several seed nodes, and coordinate on snapshots. This is a great stabilizing force, improves availability, makes parallel history attacks harder, and reduces CPU and network usage.

Swarms are orthagonal to the view and display of blocks in the client, but they do tend to correlate with views in many cases.

If a swarm contains another swarm, it's up to the client which to participate in, or whether to participate in both. Swarms don't change the database in any way.

To improve overall performance of the system, swarms should be used as much as possible!

Without swarms, every object on the live sanctuary document is served up as a torrent. Reconstructing a new view can be slow (or impossible) because you have to build up the database bit-by-bit

I still need to decide how swarms should be managed - manually, by humans, automatically, by clients?

Number one, primary flaw is that this is all too complicated to make intuitive UX around, too hard to program, and this is obvious just from reading the slang tutorial.

The rest are technical flaws that were explicitly left in to make building V1 possible, but you could fix or ameliorate them.

A lot of these flaws are shared with blockchains.

• timestamps help, even though not anchored/verified
• good block design helps
• stronger community helps
• swarms help a lot
• auto expiring entities helps
• connections to other robust blocks can help
• diverse founding entities
• could 'witnesses' from elsewhere in the network help?
• easy to detect, easy to determine and blacklist culprits. UI can display ⚠️ conflicting histories with a warning and details. Community can act if they're under attack

Parallel history attacks require the threshold to be met for consensus from the founding admins, the attack is not external but comes from within. So while it can sometimes happen, it's not something that you can launch on a mass scale against the network or anything

This is more of an issue with slow communities, because it's easy to forge an earlier timestamp if there aren't new changes

• Can be extremely powerful, too, but need to figure out best practices
• not needed for most tasks

Both privacy issues can be bandaided with Tor or a VPN.

Something like IPFS' multiformats may be considered in the future. For now, we will update algorithms directly in the protocol specification.

Availability increases with more viewers, but there's probably a lowish cap on actions per second.

Local search will always be possible, but regular centralized crawlers will probably be a necessity, so we'll build a simple API to serve results to clients and offer basic levels of search for free.

The API design can be quite simple. HTTP endpoints that return JSON.

Will think about this more later.

I don't want to go into my own diagnosis - each decentralized p2p project makes this mistake IMO, but it only serves to filter people. The internet has sucked for a long time now, and everyone has their own take on what went wrong. But I believe many people share the same desire for freedom and openness.