November 9, 2024

Voltair: A New Kind of Programming Language

Voltair is a new programming language that is powerful, safe, and fun.

In essence, Voltair seeks to combine the best of Go, V, Python, and Janet (a Lisp), and adds features seen nowhere else -- like macro methods, seamless contracts/type constraints, math that actually works correctly (thanks to Douglas Crockford's DEC64; no more 1.1 + 2.2 != 3.3), and a distributed, end-to-end encrypted actor model.

Here is a quick example:

struct User
  id        int            {:min 1}
  username  str            {:min_len 2}
  birthdate ?time.datetime {:constraints [ $(v == nil or v <= time.now() - time.years(18)) ]}


fn main()
  user := User(:id 1, :username "alan_kay", :birthdate time.datetime(:year 1940, :month 5, :day 17)) !
  print("Hello, ${user.username}")

The features utilized here that are explained in more detail below:

  1. Struct types

  2. Struct field metadata that includes constraints (e.g., {:min_len 2})

  3. Sumtypes! ?time.datetime is short for sumtype<nil time.datetime>, representing a value that is either nil or of type time.datetime

    • In general the special syntax seen above, namely ?T where T a type, is short for sumtype<nil T>

  4. More generally: variable constraints (see below)

  5. Lambdas with type inference (i.e., $(...))

    • For example, in a case like {:constraints [ $(v == nil or v <= time.now() - time.years(18)) ]} above (where v stands for "value"), the type system knows that the constraint functions are of type fn<T>(v T) bool, which in the above case concretizes to fn(v ?time.datetime) bool
    • Thus, $(...) is shorthand for (fn(v ?time.datetime) bool return ...)(user.birthdate) inside of a for-loop that iterates over each constraint and passes in user.birthdate
    • That is, $(v == nil or v <= time.now() - time.years(18)) is shorthand for (fn(v ?time.datetime) bool return v == nil or v <= time.now() - time.years(18))(user.birthdate). Much cleaner!

  6. Type instantiation via type functions (e.g., User(...))

  7. Named function parameters (e.g., User(:id ..., :username ..., :birthdate ...))

  8. Automatic error handling via trailing ! after a function call that returns an error (e.g., user := User(...) !); see below for more

  9. String interpolation (e.g., "Hello, ${user.username}")

Voltair is Powerful

Macros

By combining macros with flexible but "static" typing, Voltair can be very flexible and very safe; see below.

Voltair macros look like normal Voltair code:

# `$#`, below, purposely looks kind of like a comment because it doesn't run at
# runtime, just like comments don't run at runtime (or at all)

macro def_strs(vars ...symbol)
  $# for var in vars
    $var := "$var"

def_strs(a b c)

# The above macro-expands at compile-time to:

a := "a"
b := "b"
c := "c"

# Thefore, at the end of the call to the `def_strs(...)` macro:

print(a == "a")  # true
print(b == "b")  # true
print(c == "c")  # true

Compile-time type generation using macros:

$# for n in 1 thru 8
  type int${n*8} [$n]byte

This macro-expands to:

type int8 [1]byte
type int16 [2]byte
type int24 [3]byte
type int32 [4]byte
type int40 [5]byte
type int48 [6]byte
type int56 [7]byte
type int64 [8]byte

Macros can also be used to ensure that certain computations are done at compile-time, not at runtime, although the compiler will do its best to compute as much up front so as to prevent runtime slow-downs:

macro calc_powers(base int, max_power int) []int
  @pow_arr_ := []int() {:cap $max_power + 1}
  @current_ := 1

  $# for _ in 0 thru $max_power
    @pow_arr_ += current_
    @current_ *= $base

  emit pow_arr_


fn main()
  powers_of_two := calc_powers(:base 2, :max_power 10)

  print("2 to the 10th power is ${powers_of_two[10]})

This macro-expands to something like:

fn main()
  powers_of_two := do
    @pow_arr_a := []int() {:cap 10 + 1}
    @current_b := 1

    $# for _ in 0 thru 10
      @pow_arr_a += current_b
      @current_b *= 2

    emit pow_arr_a

  print("2 to the 10th power is ${powers_of_two[10]})

The $# before for ensures that the loop is executed at compile-time.

Here is the fully expanded code that gets directly compiled after all macro expansions and calculations have occurred:

fn main()
  powers_of_two := do
    emit []int(1 2 4 8 16 32 64 128 256 512 1024)

  print("2 to the 10th power is ${powers_of_two[10]})
Macro methods

This code:

doubled := [1 2 3].map(v * 2)

...macro-expands to:

doubled := [v * 2 from [1 2 3]]

...which finally macro-expands to:

# `i` stands for "index", `v` stands for "value"
doubled := [v * 2 for i v in [1 2 3]]

Under the hood, this code is as efficient as the array.map(...) macro method:

macro (arr []T) map(use_v_or_i expr) []T
  # This is a mutable variable whose ultimate name begins with `new_arr_` and
  # ends with a randomly-generated suffix guaranteed to prevent this variable
  # name from colliding with an existing variable in the context where
  # this macro method is called; this is true for any variable defined
  # in a macro whose name ends with `_`
  @new_arr_ := []T() {:cap arr.len}

  for i v in arr
    # `$use_v_or_i` expands to `v * 2` in the example above
    @new_arr_ += $use_v_or_i

  emit new_arr_

Voltair macros are purposely unhygienic so as to make the above possible, where v from the calling context can be referenced inside the macro.

Unwanted conflicts between the macro's "local" variables with the calling context's variables is done via the _ suffix trick described above.

Another macro method, arr.partition():

db := pq.connect("...") !

struct User                {:db db}
  id        int            {:min 1}
  username  str            {:min_len 2}
  birthdate ?time.datetime {:constraints [ $(v == nil or v <= time.now() - time.years(18)) ]}

fn main()
  all_users := db.User!()
  with_birthdates nil_birthdates := all_users.partition(v.birthdate != nil)

Another macro method, arr.count(...), as a more efficient way to do arr.filter(...).len:

how_many_evens_under_100 := to(1 100).count(v mod 2 == 0)

Voltair is Safe

Type constraints and other metadata
# The following creates mutable variables `x`, `y`, and `z`, makes
# each of them of type `num` with a default value of `-1`, and uses a
# constraint to ensure they cannot ever be set equal to 0:

def @x @y @z num {:default -1, :constraints [ $(v != 0) ]}

x = 2 !
x = 1 !
x = 8 - 5 - 3 !  # Will throw error
Voltair has dependent types

The number of outputs depends on the number of inputs. Here we also see the use of a macro method and generics:

macro (arr []T) group(exprs ...expr) [exprs.len][]T
  @groups_ := [exprs.len][]T()

  for v in arr
    for i exp_ in exprs
      if $exp_
        @groups_[i] += v

  emit groups_

# Note that this requires being able to destructure a fixed-size
# array, which is a good idea anyway:

small big := to(1 1000).group(v < 100, v >= 100)
a2m n2z vowels := to("a" "z").group(v <= "m", v > "m", v in "aeiou")
Accessing array elements is always safe
fn main()
  arr := []str()

  # Trying to get the first element of an empty array will not blow up.
  #
  # This will not panic or crash, but gracefully handle the error and, since
  # we are in `main`, `eprint(err)` then `exit(1)`; in any other
  # function, an attempted out-of-bounds array access would result in
  # `return errors.out_of_bounds`
  result := arr[0] !
  print(result)

Note that arr![0] is shorthand for arr[0] !, which again is shorthand for arr[0] ! return errors.out_of_bounds, which in turn is shorthand for something like:

n := int(0)

result := if n < 0 or n > arr.len - 1
    return errors.out_of_bounds
  else
    arr[n]


print(result)

Negative indices are OK as long as they are hardcoded:

last := arr[-1] !
Voltair does not have unsafe nil or null

In Voltair, nil is its own type; there is no such thing as a nil map, array, channel, pointer, or any other type. If we want to indicate the lack of a value, we can create a ?T rather than a T, where T is a type.

For example:

fn find(arr []T, wanted T) ?int
  for i v in arr
    if v == wanted
      return i

  return nil


fn main()
  arr := ["zero" "one" "two" "three" "four"]
  wanted := "four"
  print("four is at position ${find(arr wanted)}")         # "4"
  print("ten is at position ${find(arr, :wanted "ten")}")  # "nil"

  # Another example:

  hello := ?str("I am essentially a nilable string that can be safely used as a string only after ensuring I'm not nil")

  if hello == nil
    print("nil")
  else
    print("`hello` as a str: $hello")

Voltair is Fun

Voltair provides super high expressivity; it feels like a scripting language while providing type safety

The arr.apply(some_func) method "applies" in the Lisp sense; that is, arr.apply(some_func) means some_func(arr). This is syntactic sugar to enable convenient method chaining as well as to improve the readability of using functions that look like operators:

fn main()
  nums := [1 2 3 4 5]

  # Note that `.map(...)` is a macro method; see below for details
  nums_str := nums.map(v.str()).join(", ")

  print("The sum of $nums_str is ${nums.apply(+)}")
  print("The product of $nums_str is ${nums.apply(*)}")
  print("10 factorial is ${factorial(10)}")

  total := thru(2 10).apply(*)
  print("10 factorial, calculated more simply, is (thankfully) still $total")


fn factorial(n int) int
  if n <= 1
    return 1
  return thru(2 n).apply(*)
No cruft!

Every single character that Voltair makes you type is necessary while maintaining readability and clarity regarding what exactly is happening (regarding mutations, allocations, etc):

fn main()
  # The `:` in `:=` means we are allocating, and the `=` means we are assigning.
  #
  # No commas are required between values in an array:

  nums := [1 2 3]
  print(nums)
Seamless database integration

All imports and global constants are defined in globals.volt and can be used from any other .volt file in the same directory:

#
# globals.volt
#

postgres_connect := "..." {:priv true}
db := pq.connect(postgres_connect) !

go fn()
  # `10m` is a duration literal representing 10 minutes
  for _ in time.every(10m)
    db.ping() !
      # Manual error handling (in the case where `db.ping` returns an error)
      eprint("Error pinging Postgres: $err")
      continue

    log.debug("DB pinged successfully")


#
# main.volt
#

struct User              {:db db}  # Enables database integration
  id       int           {:required true, :constraints [ $(v > 0) ]}
  username str           {:required true, :constraints [ $(v.len > 0) ]}
  created  time.datetime {:default time.now}

# Struct default values can either be a `T` or a `fn() T`. Use the
# latter to  delay when the default value is set until object
# instantiation (rather than, if possible (e.g., `{:default 5}`),
# compile-time)


fn main()
  # `db.MyType()` acts as if it returns a `![]MyType` -- that is, an
  # `error` or a `[]MyType` for any struct type that opts into database
  # integration by by doing `{:db db}` above

  recent_user_ids := [user.id for user in db.User!() if user.created >= time.now() - 5h]

  # Note that `f!()` is shorthand for `f() !`, but is much more
  # easily chainable.
  #
  # Alternate syntax, using `.filter(...)` method:
  prev_users := db.User!().filter(v.created < time.now() - 5h)

Mutable and Immutable Versions of Methods

In Python we have to do this:

nums = [5, 4, 3, 2, 1]
ordered = sorted(nums)  # Immutable sort
nums.sort()             # Mutable sort

But in Voltair we don't need 2 names for the same concept. Instead we can create 2 different methods with the same name -- one for sorting immutable arrays (and thus returning a value), and one for sorting mutable ones (and thus not).

@nums := [5 4 3 2 1]    # `@` makes `nums` mutable
ordered := nums.sort()  # Immutable sort; no `@` needed here because `nums` is not mutated on this line
@nums.sort()            # Mutable sort; `@` is needed here because `nums` _is_ mutated

We can of course do the same when it comes to reversing an array:

@nums := [5 4 3 2 1]
ordered := nums.reverse()
@nums.reverse()

How? Here's the implementation:

# Mutable `arr`; notice the `@` prefix telling us that `arr` is mutable
fn (@arr []T) reverse()
  for i in 0 to arr.len/2
    swap(@arr[i], @arr[arr.len - i - 1])


macro swap(@a @b expr)
  @temp_ := $a
  @$a = $b
  @$b = temp_


# Immutable `arr`
fn (arr []T) reverse() []T
  @new_arr := []T() {:cap arr.len}

  for i _ in arr
    @new_arr += arr[arr.len - i - 1]

  return new_arr

Goroutines and Channels

Voltair features Go-style channels, only more powerful, because Voltair also gives us bidirectional typed channels. That is, you can define not just a chan<T> but a chan<T U> to support request/response patterns, where you want to send a value of one type over a channel, and receive a response that is a value of another type:

# Define a channel where we want to send a `tup<str str>` and receive back a `num`
word_counter := chan<tup<str str>, num>() {:cap tup(1 0)}

# Remember, `tup("..." "...")` is shorthand for `tup<str str>("..." "...")`
word_counter.req <- tup("Hello there! Count the words."  " ") !

go fn()
  # Note that `<-ch` is the same as `ch.read()`
  # Tuple destructuring:
  outer_str substr := <-word_counter.req !
    eprint("Error reading request from channel `word_counter`: $err")
    return

  # This trailing `!` is also for handling write errs (e.g., chan could be closed)
  word_counter.resp <- outer_str.count(substr) + 1 !

print("Our string contains approximately ${word_counter.resp.read!()} words")

Tuples and Destructuring

Voltair has generic tuples with type inference:

# Tuples
fn get_tup() tup<num bool str>
  return tup(1 true "three")

# Note that
#
#   tup(1 true "three")
#
# is shorthand for
#
#   tup<num bool str>(1 true "three")

# Valid
t := get_tup()

# Destructuring a tuple; the variable names can be anything
a b c := get_tup()

# Destructuring a struct; the variable names must match the `user` struct's field names
id username email := user

Facts About Voltair

  • All mutations are explicit; variables that are about to be modified must be prefixed by @

  • Key axioms guiding its design:

    • Asking, "What should I be able to do?"

  • Optionals via sumtypes: ?T is short for sumtype<nil T> where T is a type (e.g., num, str, bool, etc)

  • Results via sumtypes: !T is short for sumtype<error T> where T is a type, but with one very important difference -- Voltair by default requires that errors be handled immediately, not passed around

    • If you want the !T instead of returning the error immediately, you can do err_or_return_value := f() ! err

  • _args is a tuple that is automatically created which contains the current function or method's arguments

    • This is very similar to JavaScript's arguments variable

  • _fn_name is a string that is automatically created which equals the current function or method's name

  • num is the primary numerical type, and it is implemented as a DEC64 as smartly designed by Douglas Crockford

    • Voltair originally used Dec128 (IEEE 754-2008) but this was later rejected for being too complex, needlessly slow, needlessly large, and for treating positive and negative infinity as valid numbers, which one almost never wants

  • int should be treated as if it is defined as such, thus making it compatible with num: type int num {:constraints [ $(v mod 1 == 0) ]}

  • from arr is exactly equivalent to for i v in arr

  • The methods arr.keys() and arr.values() are not needed because we can simply do arr.map(k) and arr.map(v), respectively

  • In Lisp-speak, macro bodies are quoted by default, and unquoted using the $ prefix. The analogy here is that a macro body is almost like a (type-checked) multi-line string, and so we use string interpolation-like syntax to unquote the symbols or expression passed in.

  • struct MyType ... is currently shorthand for type MyType struct<...>, but may be removed in the future

  • Voltair has generic, anonymous struct literals, e.g., {:name "Alice", :age 35}, which is shorthand for struct<name str, age num>(:name "Alice", :age 35)

    • Create an empty struct like this: struct(). {} may also be supported in the future

  • All variables are public by default. To make them private, use defpriv to define them or, more generally, set {:priv true}

  • Named enums

    • E.g., type FD int {:min 0, :named {:stdin 0, :stdout 1, :stderr 2}}
    • This enables us to do write_bytes_to_fd(.stdin "Hello, world.bytes()); see below for full code example

  • emit ... causes a macro or block to evaluate to the given expression, kind of like a return value

Examples:

big := if n > 100
    emit "Yes"
  else
    emit "No"

But that example is so simple that we might as well just do

big := if n > 100
    "Yes"
  else
    "No"

More complex example that puts emit to better use:

body status_code := http.get_all(url) !
  if err.str() == "Some error we don't care about"
    emit tup([]byte() 200)

  return err

See the macro methods section above for an example of using emit inside a macro.

  • Voltair also supports one-line if expressions: big := if n > 100 then "Yes" else "No"

    • Leaving out the else clause will result in a ?T rather than a T, so big := if n > 100 then "Yes" would result in type(big) == ?str, also known as sumtype<nil str>

  • The compiler will optimize for n in min to max into a traditional for-loop (i.e., for @n := min; n < max; @n += 1)

Non-features

Voltair, proudly, does not have these features:

  • Per-source file imports

    • Instead, in globals.volt we define which libraries should show up globally, in every .volt source file in the same folder

  • nil values that can be passed around and blow shit up when used

    • Unlike in Go, the default value of arrays (slices), channels, and pointers is not nil but rather a useful, allocated, empty value of the desired type

  • Variable shadowing

  • Boilerplate everywhere

  • Parentheses everywhere, as in all other Lisp-like languages

  • Commas everywhere, as in non-Lisps (Go, C, C++, Python, JavaScript, Java, etc)

  • Print functions other than print, prin (to exclude trailing newline), eprint (for printing to stderr), and eprin

    • No println, printf, sprintf, etc

  • Ways of defining a variable other than type for types, fn for functions, and def for everything else

    • No var, let, const, etc
    • Putting an @ in front of the var makes it mutable (e.g., def @nums [1 2 3] or @nums := [1 2 3])

  • Static-feeling code

  • Pretending types don't exist

  • Unhelpful helper functions that make the language gigantic and harder to learn

    • See: much of Clojure

  • Unnecessary types

    • In Janet, I believe that mutable arrays and immutable arrays are different types with different implementations; same with maps and other built-in types. In Voltair's case, mutability is attached to the variable, not the type.

  • Operators

    • Voltair has "op'able functions" like +, -, *, and /, but no operators; put another way, all Voltair operators are functions, which eliminates the need for having 2 names for the same concept and functionality; + is a real function that can be passed around, for example, like in Lisps, but not in the weaker scripting languages we are used to

  • Explicit pointers

    • If for performance reasons it is important for certain types or values to not be pointer types under the hood, you can do {:no_ptr true}

Future Features

  1. SkyCastles, which is what Voltair calls a not-common-enough pattern in Go code.

    • In essence, a SkyCastle provides a service to the other parts of your program
    • It's not just a passive, inert data structure, but a service that receives values and even functions for mutating the data structures that are private to said service/SkyCastle
    • This is usually implemented as an infinite loop in a function or method that's been launched in a goroutine listens on 1 or more channels for values that are then processed. By making SkyCastle a built-in data type, the boilerplate code can be eliminated and this pattern made more popular, often supplanting the passing around of a mutable data structure
    • With SkyCastles in place, combined with secure distributed actors (see below), scaling systems should become much simpler, since the major parts of the system will be able to refer to each other via local objects, with local "router" goroutines directing requests to SkyCastles that happen to be running in the same process or machine, or routing them to other servers as need be (via Relay)
    • Using this machinery, I imagine a world in which "microservices" self-monitor and, when scaling or reconfiguration is necessary, can move themselves between physical servers on the fly, or "call for back up" and spin up new instances of themselves, all without any downtime, and without the code talking to these microservices changing a single line

  2. Secure distributed actors (thanks to the Relay protocol, which comes from the Relay Anonymous Messenger I've been creating since mid-2023)

    • Will leverage bidirectional channels that represent remote services, just like bidirectional channels represent local services

  3. Svelte-style reactivity using the $$ prefix (e.g., $$followers := db.User!().filter(me.id in v.follows))

    • The frontend should be able to look like it is directly representing the backend database and subscribe to updates coming from the DB, receiving them via WebSocket, just like Hyperfiddle brilliantly does (see below)

  4. Various features copy/paste/modify/transplanted from a few additional sources

    • Vale, which is safer than Rust yet dramatically simpler; generational references can replace Voltair's locks on global mutable variables, for example
    • Hyperfiddle's Electric Clojure, which abstracts away the network calls between the frontend and backend! It also lets programmers write simple functions that get compiled to some frontend code and some backend code

  5. Generative testing, a la clojure/test.generative

Possible Future Features

  1. Create convenient syntax for creating lazy sequences?

    • Perhaps by allowing array metadata to include {:lazy true}

  2. Create an implementation of Voltair meant for very low-level systems in an attempt to replace/displace C.

C code example from a book on Xv6 Unix:

char buf[512];
int n;

for(;;){
  n = read(0, buf, sizeof buf);
  if(n == 0)
    break;

  if(n < 0){
    fprintf(2, "read error\n");
    exit(1);
  }

  if(write(1, buf, n) != n){
    fprintf(2, "write error\n");
    exit(1);
  }
}

The equivalent solution in Voltair:

# `FD` == File Descriptor
type FD int {:min 0, :named {:stdin 0, :stdout 1, :stderr 2}}

fn main()
  cat() !

fn cat() !
  @buf := []byte() {:len 512, :fixed_len true}

  for
    n := read(.stdin @buf buf.len) !
    if n == 0
      break

    # `.stdout` is short for `FD.stdout` and `FD[:stdout]`.  This works because
    # I'm imagining that `write()`'s first argument is of type `FD`
    write(.stdout buf n) !

Implementation Status

The Voltair compiler is currently being prototyped and is in the very early stages. Please steal these language features to help ensure that the future of software is much more reliable than it is today!

In particular, I consider very few programming languages to be designed well. The following languages largely are, and thus they are worth salvaging, and thus I hope they steal as many good ideas from Voltair as possible:

  • V
  • Go
  • Janet
  • Vale
  • Electric Clojure
  • Elixir
  • Amber
  • Nim
  • Kotlin
  • TypeScript