Replace built-ins by general overloaded functions

[EvanKirshenbaum]

It's time to replace another kludge.

Currently, DML has the notion of a CallableType, which has a Signature, and which is implemented by a CallableValue (which also has a Signature). This works fine for macros, and also things like MOTION, DELTA, and TWIDDLE, implemented by instances of subclasses of CallableType. Type.as_func_type, which allows conversion to a supertype that's a subclass of CallableType also means that it works for things like DIR. This works pretty well for functions that have a single signature.

But there are a number of common operations that we need to be overloaded. For example, transfer in currently has the following signatures:

well * liquid -> well
liquid -> (well -> well)          // sort of.  see below
well * liquid * reagent -> well   // The reagent is the result
liquid * reagent -> (well -> well)
well * volume -> well
volume -> (well -> well)
extraction point * liquid -> extraction point
liquid -> (extraction point -> extraction point)

The forms that return callables are created by currying the ones above them. Note that there are two forms that both take a LIQUID and return a callable. In fact, the way things are currently implemented, if you say transfer in(2 uL), what you'll get is a function that returns an ep -> ep, but there's a special kludge that allows both

w : transfer in(2 uL)
ep : transfer in(2 uL)

to work.

The way all this works, is that there's a special (Python) global dictionary, BuiltIns that maps names to Func objects, which have the ability to register multiple functions and find the tightest match for overloads.

The downside to this is that when transfer in is seen, the name expression compiler has a special case where it looks in BuiltIns and returns the Func with type BUILT_IN. And the function call, injection, and is expression compilers have special cases to handle seeing them.

That's the problem. Solution to follow.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Jul 09, 2023 at 10:13 AM PDT.

[EvanKirshenbaum]

My thought is to have a FunctionType that represents a (possibly) overloaded function that can handle multiple Signatures. CallableType would be a subtype that only handles a single Signature. Type.as_func_type would now return a FunctionType.

One FunctionType would be convertible to another if the first has some signature whose parameters are all at least as wide and whose return is at least as narrow of every signature of the second. The notion is that anything you could do with the second, you can also do with the first. It's okay if the first can also do other things that you wouldn't try to do with the second.

There would also be a FunctionValue, above CallableValue, that can return an Optional[CallableValue] given a Signature. For CallableValue. Finally, there would be an OverloadedFunction that held a mapping from Signature to CallableValue and would find the narrowest (perhaps with some conversion, or maybe the conversion is done in FunctionValue after asking itself for the Signature and CallableValue.)

With this in place, transfer in(2 uL) could simply return an OverloadedFunction that handled both well -> well and ep -> ep, and the Signature for that CallableType would be liquid -> (well -> well | ep -> ep).

To further simplify things, we can use the SpecialVars dictionary, which is only used in name lookup and assignment (and declaration, to disallow shadowing, but we might want to relax that), to just treat these names as part of the namespace. If we do that, we can get rid of the BUILT_IN type and the BuiltIns table (and the Functions table can go back to just handling internal dispatch for things like addition and relations). We'd probably want a BuiltInFunction subclass of both OverloadedFunction and SpecialVariable that allows new definitions to be added. (Note that currently SpecialVariable.var_type is Final, so we will have to turn it into a property, because BuiltInFunction's type will change with each addition.

One of the nice things about this is that with currying, if you wind up with a Signature you already handle, you can just turn its callable into one that returns and overloaded function with a case for each elided argument.

On the other hand, BuiltInFunction should at least warn if an alternative that doesn't return a functional type can completely handle the args of an existing alternative (or if it does return a functional type, returns one that can handle all args the existing return value can). This would mean that it would completely shadow the other.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Jul 09, 2023 at 11:03 AM PDT.

[EvanKirshenbaum]

This would also make it straightforward to support both default values for function arguments (#160) and the ability to specify an argument as injectable (i.e., what I'm doing now with currying for built-ins). We would just compile the function as we currently do, but then munge it in several ways to provide wrappers for the other signatures and turn it into an OverloadedFunction.

If I ever get the time to add statement-level function definitions (also #160), having multiple function declarations with the same name declared in the same scope could similarly result in a single overloaded function bound to that name.

I'll have to think about what it would mean for one function to shadow another. The simplest thing to do would be to have the inner one completely shadow the other. It might be less surprising to have the inner one shadow any external definition that could take its arguments, e.g., in

function foo(float) -> string { ... }
function foo(string) -> int { ... }
function bar() {
   function foo(int) -> int { ...}
}

the inner foo would shadow the outer foo(float) but not the foo(string), so foo(5) would call the inner one, foo("hi") would call the outer one, and foo(2.4) would be a compile-time error.

Another possibility would be to say that all three definitions are there, but the inner one handles int parameters, so foo(5) calls the inner one, while foo(2.5) calls the outer one. This is probably too confusing, as it would mean that in

float f = 5;
foo(f)

it would be the outer one that was called.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Jul 09, 2023 at 11:21 AM PDT.