programming/projects/shelter

shelter is my design for a potential operating system and runtime which foregoes the entire legacy of today’s systems
2023-11-20
- compatibility is not a design goal for shelter, the idea is that we build the entire universe from scratch
  2023-11-20
  - therefore shelter is not compatible with UNIX, POSIX, or Windows
    2023-11-20
  - it should be possible to build compatibility layers, but they probably won’t be part of the project
    2023-11-20
- the design goal is to build a secure operating system you can trust
  2023-11-20
  - gone will be the days where you download an executable from the Internet and have no idea what harm it can do to your system
    2023-11-20
  - security is to be achieved while keeping the system fundamentally simple. the less code you have to inspect, the better
    2023-11-20
- NOTE: at this point shelter is nothing more than an incomplete design. OS development is something I wanna get into but haven’t enough time to research everything as of now, therefore I’m jotting down my ideas here
  2023-11-20
  - but if all goes well you’ll be able to run it one day
    2023-11-20
design
2023-11-20
- execution environment
  2023-11-20
  - the execution environment of shelter is one big JIT compiler. code is portable between CPU architectures and exchanged via a compact intermediate representation (IR), which is then compiled to machine code when installed into the system
    2023-11-20
    compilers which emit this IR must not perform aggressive inlining. this is important for the OS’s function database to work correctly and be able to deduplicate functions aggresively. instead, inlining is done by the JIT upon compilation (or maybe even based on profiling)
    2023-11-20
  - note that although there is a JIT, there is no garbage collection. memory is allocated and freed manually by running programs
    2023-11-20
  - the environment supports algebraic effects, which are used to annotate functions which may perform I/O, access the network, and perform mischief using them.
    2023-11-20
- executable code
  2023-11-20
  - this is probably the most exciting part of shelter: how executable code is not stored within .exe and .dll files, but rather in a database managed by the OS
    2023-11-20
    the database is basically just this: rust struct CodeDatabase { types: HashMap<Hash<Type>, Type>, functions: HashMap<Hash<Function>, Function>, }
    2023-11-20
    Hash<T> is the hash of the value T.
    2023-11-20
    Hash<T> is large enough to practically prevent any and all collisions (is 256 bits with a cryptographically secure hash function enough?)
    2023-11-20
    functions in the database are fully anonymous; function names can be given via separate debug info that can be attached to a running program to provide stack traces
    2023-11-20
    this debug info correlates a function’s properties with source code and is completely optional
    2023-11-20
    functions from within the database can be aliased in the filesystem. you can create a file which executes a function annotated as a valid entrypoint
    2023-11-20
    function metadata includes reflection data - argument/return types, generics (so that monomorphization is performed on-demand by the JIT to save disk space), and annotations (so that the OS can know eg. which functions are valid program entry points)
    2023-11-20
    this reflection data can be queried by anyone in userspace
    2023-11-20
    it can be used eg. to implement a shell, which executes named functions, such as ls here: @os.entrypoint fun ls( caps: (working_directory: shell.WorkingDirectory(:read)), args: (compact: shell.Flag(short: "l")), ): Result(()) :: os.stdio.Write + os.Filesystem = caps.working_directory.path | fs.walk fun (entry) = { if args.compact.is_set then { print("\{entry | fs.dirent.path? | path.filename}") } else { print("\{entry.kind}\t\{entry | fs.dirent.path? | path.filename}") } }
    2023-11-20
- capability based security
  2023-11-20
  - functions can only do what they say they do, and access what they say they access
    2023-11-20
    the first of these is achieved through an effect system within the language runtime
    2023-11-20
    a function can only write to stdout if it declares it performs the os.stdio.Write effect
    2023-11-20
    the second of these is achieved through explicitly passing capabilities as function arguments
    2023-11-20
    you do not have access to a directory if you’re not explicitly given a fs.Directory value
    2023-11-20
    moreso, if you need to read or write directory values, the directory needs to explicitly be locked (using an rwlock) to help prevent TOCTOU race conditions
    2023-11-20
    not to say such race conditions will be completely impossible, but they will be much harder to run into on accident
    2023-11-20
    it’s also impossible to fabricate capabilities because low-level memory access can only be performed explicitly through byte slices, and only types whose definition is public can be cast into byte slices
    2023-11-20
- package management
  2023-11-20
  - because functions are identifiable by their hash, it’s easy to implement a decentralized function registry
    2023-11-20
    the OS can store a list of mirrors and request functions from them as needed
    2023-11-20
    if one mirror doesn’t have a function, the system can request it from another mirror
    2023-11-20
    if no mirrors have a function, tell the user
    2023-11-20
    functions downloaded from the Internet can be validated by checking that the hash of the received function matches that of the requested function
    2023-11-20
    the bytecode’s structure should be validated at this point as well
    2023-11-20

programming/projects/shelter

design