Skip to content

Blog

Dev Report: Introducing the NixOS to JSON Schema Converter

Overview

We’ve developed a new library designed to extract interfaces from NixOS modules and convert them into JSON schemas, paving the way for effortless GUI generation. This blog post outlines the motivations behind this development, demonstrates the capabilities of the library, and guides you through leveraging it to create GUIs seamlessly.

Motivation

In recent months, our team has been exploring various graphical user interfaces (GUIs) to streamline NixOS machine configuration. While our opinionated Clan modules simplify NixOS configurations, there's a need to configure these modules from diverse frontends, such as:

  • Command-line interfaces (CLIs)
  • Web-based UIs
  • Desktop applications
  • Mobile applications
  • Large Language Models (LLMs)

Given this need, a universal format like JSON is a natural choice. It is already possible as of now, to import json based NixOS configurations, as illustrated below:

configuration.json:

{ "networking": { "hostName": "my-machine" } }

This configuration can be then imported inside a classic NixOS config:

{config, lib, pkgs, ...}: {
  imports = [
    (lib.importJSON ./configuration.json)
  ];
}

This straightforward approach allows us to build a frontend that generates JSON, enabling the configuration of NixOS machines. But, two critical questions arise:

  1. How does the frontend learn about existing configuration options?
  2. How can it verify user input without running Nix?

Introducing JSON schema, a widely supported standard that defines interfaces in JSON and validates input against them.

Example schema for networking.hostName:

{
  "type": "object",
  "properties": {
    "networking": {
      "type": "object",
      "properties": {
        "hostName": {
          "type": "string",
          "pattern": "^$|^[a-z0-9]([a-z0-9_-]{0,61}[a-z0-9])?$"
        }
      }
    }
  }
}

Client-Side Input Validation

Validating input against JSON schemas is both efficient and well-supported across numerous programming languages. Using JSON schema validators, you can accurately check configurations like our configuration.json.

Validation example:

$ nix-shell -p check-jsonschema
$ jsonschema -o pretty ./schema.json -i ./configuration.json
===[SUCCESS]===(./configuration.json)===

In case of invalid input, schema validators provide explicit error messages:

$ echo '{ "networking": { "hostName": "my/machine" } }' > configuration.json
$ jsonschema -o pretty ./schema.json -i ./configuration.json
===[ValidationError]===(./configuration.json)===

'my/machine' does not match '^$|^[a-z0-9]([a-z0-9_-]{0,61}[a-z0-9])?$'

Failed validating 'pattern' in schema['properties']['networking']['properties']['hostName']:
    {'pattern': '^$|^[a-z0-9]([a-z0-9_-]{0,61}[a-z0-9])?$',
     'type': 'string'}

On instance['networking']['hostName']:
    'my/machine'

Automatic GUI Generation

Certain libraries facilitate straightforward GUI generation from JSON schemas. For instance, the react-jsonschema-form playground auto-generates a form for any given schema.

NixOS Module to JSON Schema Converter

To enable the development of responsive frontends, our library allows the extraction of interfaces from NixOS modules to JSON schemas. Open-sourced for community collaboration, this library supports building sophisticated user interfaces for NixOS.

Here’s a preview of our library's functions exposed through the clan-core flake:

  • lib.jsonschema.parseModule - Generates a schema for a NixOS module.
  • lib.jsonschema.parseOption - Generates a schema for a single NixOS option.
  • lib.jsonschema.parseOptions - Generates a schema from an attrset of NixOS options.

Example: module.nix:

{lib, config, pkgs, ...}: {
  # a simple service with two options
  options.services.example-web-service = {
    enable = lib.mkEnableOption "Example web service";
    port = lib.mkOption {
      type = lib.types.int;
      description = "Port used to serve the content";
    };
  };
}

Converted, using the parseModule function:

$ cd clan-core
$ nix eval --json --impure --expr \
  '(import ./lib/jsonschema {}).parseModule ./module.nix' | jq | head
{
  "properties": {
    "services": {
      "properties": {
        "example-web-service": {
          "properties": {
            "enable": {
              "default": false,
              "description": "Whether to enable Example web service.",
              "examples": [
...

This utility can also generate interfaces for existing NixOS modules or options.

GUI for NGINX in Under a Minute

Creating a prototype GUI for the NGINX module using our library and react-jsonschema-form playground can be done quickly:

  1. Export all NGINX options into a JSON schema using a Nix expression:
# export.nix
let
  pkgs = import <nixpkgs> {};
  clan-core = builtins.getFlake "git+https://git.clan.lol/clan/clan-core";
  options = (pkgs.nixos {}).options.services.nginx;
in
  clan-core.lib.jsonschema.parseOption options
  1. Write the schema into a file:

    $ nix eval --json -f ./export.nix | jq > nginx.json
    

  2. Open the react-jsonschema-form playground, select Blank and paste the nginx.json contents.

This provides a quick look at a potential GUI (screenshot is cropped).

Image title

Limitations

Laziness

JSON schema mandates the declaration of all required fields upfront, which might be configured implicitly or remain unused. For instance, services.nginx.virtualHosts.<name>.sslCertificate must be specified even if SSL isn’t enabled.

Limited Types

Certain NixOS module types, like types.functionTo and types.package, do not map straightforwardly to JSON. For full compatibility, adjustments to NixOS modules might be necessary, such as substituting listOf package with listOf str.

Parsing NixOS Modules

Currently, our converter relies on the options attribute of evaluated NixOS modules, extracting information from the type.name attribute, which is suboptimal. Enhanced introspection capabilities within the NixOS module system would be beneficial.

Future Prospects

We hope these experiments inspire the community, encourage contributions and further development in this space. Share your ideas and contributions through our issue tracker or matrix channel!

New documentation site and weekly new meetup

Last week, we added a new documentation hub for clan at docs.clan.lol. We are still working on improving the installation procedures, so stay tuned. We now have weekly office hours where people are invited to hangout and ask questions. They are every Wednesday 15:30 UTC (17:30 CEST) in our jitsi. Otherwise drop by in our matrix channel.

Introducing Clan: Full-Stack Computing Redefined

In a digital age where users are guided increasingly toward submission and dependence, Clan reclaims computing and networking from the ground up.

Clan enables users to build any system from a git repository, automate secret handling, and join devices in a secure darknet. This control extends beyond applications to communication protocols and the operating system itself, putting you fully in charge of your own digital environment.

Why We're Building Clan

Our mission is simple: to restore fun, freedom, and functionality to computing as an open source project. We believe in building tools that empower users, foster innovation, and challenge the limitations imposed by outdated paradigms. Clan, in its essence, is an open source endeavor; it's our contribution to a future where technology serves humanity, not the other way around.

How Clan Changes the Game

Clan embodies a new philosophy in system, application, and network design. It enables seamless, secure communication across devices, simplifies software distribution and updates, and offers both public and private network configurations. Here are some of the ways it accomplishes this:

  • Nix as a Foundation: Imagine a safety net for your computer's operating system, one that lets you make changes or updates without the fear of causing a crash or losing data. Nix simplifies the complexities of system design, ensuring that updates are safe and systems are more reliable.

  • Simplified System Deployment: Building and managing a computer system, from the operating system to the software you use, often feels like putting together a complex puzzle. With Clan, the puzzle pieces are replaced by a set of building blocks. Leveraging the power of Nix and Clan's innovative toolkit, anyone from tech-savvy administrators to everyday users can create and maintain what we call "full-stack systems" (everything your computer needs to run smoothly).

  • A Leap in Connectivity: Imagine if you could create private, secure pathways between your devices, bypassing the noisy and often insecure internet. Clan makes this possible through something called "overlay networks." These networks are like private tunnels, allowing your devices to talk to each other securely and directly. With Clan's built-in overlay networks and automatically configured services, connecting your devices becomes seamless, secure, and hassle-free.

  • Security Through Separation: Clan employs sandboxing and virtual machines, a technology that runs code in isolated environments - so even if you explore new Clans, your system remains protected from potential threats.

  • Reliable: With Clan, your data and services are preserved for the long haul. We focus on self-hosted backups and integration with the Fediverse, a network of interconnected, independent online communities, so your digital life remains uninterrupted and under your control.

A Glimpse at Clan's Features

  • Social Scaling: Choose between creating a private sanctuary for your closest contacts, a dynamic space for a self-contained community, or embracing the open web with public Clans anyone can join.
  • Seamless VM Integration: Applications running in virtual machines can appear and behave as if they're part of your main operating system — a blend of power and simplicity.
  • Robust Backup Management: Keep your data safe forever - never worry about cloud services disappearing in 10 years.
  • Intuitive Secret Management: Clan simplifies digital security by automating the creation and management of encryption keys and passwords for your services.
  • Remote Install: Set up and manage Clan systems anywhere in the world with just a QR scan or SSH access, making remote installations as easy as snapping a photo or sharing a link.

Who Stands to Benefit?

Clan is for anyone and everyone who believes in the power of open source technology to connect, empower, and protect. From system administrators to less tech-savvy individuals, small business owners to privacy-conscious users, Clan offers something for everyone — a way to reclaim control and redefine how we interact with technology.

Join the Revolution

Ready to control your digital world? Clan is more than a tool—it's a movement. Secure your data, manage your systems easily, or connect with others how you like. Start with Clan for a better digital future.

Connect with us on our Matrix channel at clan.lol or through our IRC bridges (coming soon).

Want to see the code? Check it out on our Gitea or on GitHub.

Or follow our RSS feed!

Join us and be part of changing technology for the better, together.