Skip to content

data-mesher

Set up data-mesher

System

Experimental

This service is experimental and will change in the future.

This service will set up data-mesher, a file synchronization daemon that uses a gossip protocol to replicate files across a cluster of nodes.

Architecture

Data-mesher v2 uses a file-based approach where:

  • Files are defined with a list of authorized public keys (ED25519)
  • Only files listed in the configuration can be uploaded or synced
  • Files must be signed by one of the configured public keys

Roles

  • admin: A node that can sign files and act as a bootstrap node
  • peer: A node that participates in the cluster and can receive files

Usage

inventory.instances = {
  data-mesher = {
    module = {
      name = "data-mesher";
      input = "clan-core";
    };
    roles.admin.machines.server0.settings = {
      network.interface = "tailscale0";
      bootstrapNodes = [
        "192.168.1.1:7946"
        "192.168.1.2:7946"
      ];
      # Define which files can be synced and their authorized signers
      files = {
        "config:app" = [
          "azwT+VhTxA+BF73Hwq0uqdXHG8XvHU2BknoVXgmEjww="  # admin key
        ];
        "shared:data" = [
          "azwT+VhTxA+BF73Hwq0uqdXHG8XvHU2BknoVXgmEjww="  # admin key
          "Mdtz9s2DEyEk0DL8ZzW7WqwAehoQ97PFHVbJJdskkGo="  # peer key
        ];
      };
    };
    roles.peer.machines.server1.settings = {
      network.interface = "tailscale0";
      bootstrapNodes = [
        "192.168.1.1:7946"
        "192.168.1.2:7946"
      ];
      # Peers need the same file definitions to participate in syncing
      files = {
        "config:app" = [
          "azwT+VhTxA+BF73Hwq0uqdXHG8XvHU2BknoVXgmEjww="
        ];
        "shared:data" = [
          "azwT+VhTxA+BF73Hwq0uqdXHG8XvHU2BknoVXgmEjww="
          "Mdtz9s2DEyEk0DL8ZzW7WqwAehoQ97PFHVbJJdskkGo="
        ];
      };
    };
  };
}

Configuration Options

Common Settings (all roles)

  • network.interface: The network interface for cluster communication
  • network.port: Port for cluster communication (default: 7946)
  • bootstrapNodes: List of bootstrap nodes to connect to when joining
  • files: Map of file names to lists of authorized ED25519 public keys

Uploading Files

Once the cluster is running, you can upload files using the CLI:

# Create a file and upload it
echo "my content" > /tmp/myfile
data-mesher file update /tmp/myfile --url http://localhost:7331 --key-path /path/to/signing.key

Files will automatically sync to all nodes in the cluster that have the same file definition in their configuration.

Roles

The data-mesher service has the following roles:

  • admin
  • default
  • peer
  • signer

The admin role has no configuration

Options for the default role

bootstrapNodes

A list of bootstrap nodes that act as an initial gateway when joining the cluster.

Type: null or (list of string)

Example 
[
  "192.168.1.1:7946"
  "192.168.1.2:7946"
]

Declared in: clanServices/data-mesher/default.nix

files

A mapping of file names to lists of base64-encoded ED25519 public keys. Only files listed here can be uploaded or imported from other nodes, and they must be signed by one of the configured public keys.

Type: attribute set of list of string

Default:

{ }
Example 
{
  "config:app" = [
    "ZasdhiAVJTa5b2qG8ynWvdHqALUxC6Eg8pdn6RVXuQE="
    "1ru2QQ1eWV7yDlyfTTDEml3xTiacASYn0KprzknN8Pc="
  ];
  "dns:sol" = [
    "P6AE0lukf9/qmVglYrGPNYo5ZnpFrnqLeAzlCZF0lTk="
  ];
}

Declared in: clanServices/data-mesher/default.nix

logLevel

Log level

Type: one of "fatal", "error", "warn", "info", "debug"

Default:

"info"

Declared in: clanServices/data-mesher/default.nix

network.interface

The interface over which cluster communication should be performed. All the ip addresses associate with this interface will be part of our host claim, including both ipv4 and ipv6.

This should be set to an internal/VPN interface.

Type: string

Example 
"tailscale0"

Declared in: clanServices/data-mesher/default.nix

network.port

Port to listen on for cluster communication.

Type: 16 bit unsigned integer; between 0 and 65535 (both inclusive)

Default:

7946

Declared in: clanServices/data-mesher/default.nix

The peer role has no configuration

The signer role has no configuration