I’ve never felt the urge to make a PL until recently. I’ve been quite happy with a combination of Rust and Julia for most things, but after learning more about BEAM languages, LEAN4, Zig’s comptime, and some newer languages implementing algebraic effects, I think I at least have a compelling set of features I would like to see in a new language. All of these features are inspired by actual problems I have programming today.

I want to make a language that achieves the following (non-exhaustive):

  • significantly faster to compile than Rust
  • at least has better performance than Python
  • processes can be hot-reloaded like on the BEAM
  • most concurrency is implemented via actors and message passing
  • built-in pub/sub buses for broadcast-style communication between actors
  • runtime is highly observable and introspective, providing things like tracing, profiling, and debugging out of the box
  • built-in API versioning semantics with automatic SemVer violation detection and backward compatible deployment strategies
  • can be extended by implementing actors in Rust and communicating via message passing
  • multiple memory management options, including GC and arenas
  • opt-in linear types to enable forced consumption of resources
  • something like Jane Street’s Ocaml “modes” for simpler borrow checking without lifetime variables
  • generators / coroutines
  • Zig’s comptime that mostly replaces macros
  • algebraic data types and pattern matching
  • more structural than nominal typing; some kind of reflection (via comptime) that makes it easy to do custom data layouts like structure-of-arrays
  • built-in support for multi-dimensional arrays, like Julia, plus first-class support for database-like tables
  • standard library or runtime for distributed systems primitives, like mesh topology, consensus protocols, replication, object storage and caching, etc

I think with this feature set, we would have a pretty awesome language for working in data-driven systems, which seems to be increasingly common today.

One thing I can’t decide yet, mostly due to ignorance, is whether it’s worth it to implement algebraic effects or monads. I’m pretty convinced that effects, if done well, would be strictly better than monads, but I’m not sure how feasible it is to incorporate effects into a type system without requiring a lot of syntactical overhead. I’m hoping most effects can be inferred.

I’m also nervous that if I add too many static analysis features, compile times will suffer. It’s really important to me that compile times are productive.

Anyway, I’m just curious if anyone thinks this would be worth implementing. I know it’s totally unbaked, so it’s hard to say, but maybe it’s already possible to spot issues with the idea, or suggest improvements. Or maybe you already know of a language that solves all of these problems.

  • tatterdemalionOP
    link
    fedilink
    arrow-up
    2
    ·
    9 months ago

    Not sure if your questions are rhetorical or not. I’m not at the point of being able to answer them in detail, but I appreciate your direction.

    What’s your intended underlying formalism?

    I don’t know. I’m not well-studied in language theory.

    What do your modules look like? Can they be compiled separately? How expensive is linking? You’ll need to figure this out if you want to compile faster than Rust or C++.

    No spec yet, but there would certainly be some independent compilation step into an IR which would need to be linked with other modules later. No clue how expensive linking would be. I haven’t implemented any of this yet, it’s just an idea. But I appreciate the tip that linking will have a large performance impact.

    Thank you for pointing me to some relevant languages I had not heard of.

    • Corbin
      link
      fedilink
      English
      arrow-up
      2
      ·
      9 months ago

      My questions are serious and you should eventually be able to answer them, but right now it’s okay to think of them as guides and you don’t have to justify yourself to me. I think what you’re doing is fun; but if you want anybody to use your language, then you should be prepared for the language checklist.

      Your formalism will matter a lot for performance. You should at least know about abstract machines, which are how C, C++, Java, and Rust achieve impressive runtime performance; they all are defined such that their compilers can optimize based on abstract behaviors and those abstractions are relatively lightweight to map to physical hardware. There are also approaches based on wrapping – taming – the lowest-level parts of the hardware to add various safety properties; after wrapping, various components can be composed directly with respect to those properties.

      • tatterdemalionOP
        link
        fedilink
        arrow-up
        1
        ·
        8 months ago

        and those abstractions are relatively lightweight to map to physical hardware

        Or in the case of Java, a virtual machine. A VM would be my preferred approach to keep things simple at the outset.