Reuh/anselme
1
0
Fork 0
mirror of https://github.com/Reuh/anselme.git synced 2025-10-27 16:49:31 +00:00
Code Releases Activity

Anselme v2.0.0-alpha rewrite

Browse source 
Woke up and felt like changing a couple things. It's actually been worked on for a while, little at a time...

The goal was to make the language and implementation much simpler. Well I don't know if it really ended up being simpler but it sure is more robust.

Main changes:
* proper first class functions and closures supports! proper scoping rules! no more namespace shenanigans!
* everything is an expression, no more statements! make the implementation both simpler and more complex, but it's much more consistent now! the syntax has massively changed as a result though.
* much more organized and easy to modify codebase: one file for each AST node, no more random fields or behavior set by some random node exceptionally, everything should now follow the same API defined in ast.abstract.Node

Every foundational feature should be implemented right now. The vast majority of things that were possible in v2 are possible now; some things aren't, but that's usually because v2 is a bit more sane.
The main missing things before a proper release are tests and documentation. There's a few other things that might be implemented later, see the ideas.md file.
This commit is contained in:
Étienne Fildadut 2023-12-17 17:15:16 +01:00
parent 2ff494d108
commit fe351b5ca4
484 changed files with 7099 additions and 18084 deletions
Download patch file Download diff file Expand all files Collapse all files

1133
LANGUAGE.md
View file

File diff suppressed because it is too large Load diff

5
LICENSE
View file

@ -1,5 +0,0 @@
Copyright 2019-2022 Étienne "Reuh" Fildadut
Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

76
README.md
View file

@ -1,76 +0,0 @@
Anselme
=======
The overengineered dialog scripting system in pure Lua.
Whatever is on the master branch should work fine. **Documentation and language are still WIP and will change. I am using this in a project and modify it as my needs change.** Breaking changes are documented in commit messages.
Purpose
-------
Once upon a time, I wanted to do a game with a branching story. I could store the current state in a bunch of variables and just write everything like the rest of my game's code. But no, that would be *too simple*. I briefly looked at [ink](https://github.com/inkle/ink), which looked nice but lacked some features I felt like I needed. Mainly, I wanted something more language independant and less linear. Also, I wasn't a fan of the syntax. And I'm a weirdo who make their game in Lua *and* likes making their own scripting languages (by the way, if you like Lua but not my weird idiosyncratic language, there's actually some other options ([Erogodic](https://github.com/oniietzschan/erogodic) looks nice)).
So, here we go. Let's make a new scripting language.
Anselme ended up with some features that are actually quite useful compared to the alternatives:
* allows for concurently running scripts (a conversation bores you? why not start another at the same time!)
* allows for script interuption with gracious fallback (so you can *finally* make that NPC shut up mid-sentence)
* a mostly consistent and easy to read syntax based around lines and whitespace
* easily extensible (at least from Lua ;))
And most stuff you'd expect from such a language:
* easy text writing, can integrate expressions into text, can assign tags to (part of) lines
* choices that lead to differents paths
* variables, functions, arbitrary expressions (not Lua, it's its own thing)
* can pause the interpreter when needed
* can save and restore state
And things that are halfway there but *should* be there eventually (i.e., TODO):
* language independant; scripts should (hopefully) be easily localizable into any language (it's possible, but doesn't provide any batteries for this right now).
Defaults variables use emoji and then it's expected to alias them; works but not the most satisfying solution.
* a good documentation (need to work on consistent naming of Anselme concepts, a step by step tutorial)
Things that Anselme is not:
* a game engine. It's very specific to dialogs and text, so unless you make a text game you will need to do a lot of other stuff.
* a language based on Lua. It's imperative and arrays start at 1 but there's not much else in common.
* a high-performance language. No, really, I didn't even try to make anything fast, so don't use Anselme to compute primes.
Example
-------
Sometimes we need some simplicity:
```
HELLO SIR, HOW ARE YOU TODAY
> why are you yelling
I LIKE TO
> Well that's stupid.
I DO NOT LIKE YOU SIR.
> I AM FINE AND YOU
I AM FINE THANK YOU
LOVELY WEATHER WE'RE HAVING, AREN'T WE?
> Sure is!
YEAH. YEAH.
> I've seen better.
NOT NICE.
WELL, GOOD BYE.
```
Othertimes we don't:
TODO: stupidly complex script
See [TUTORIAL.md](TUTORIAL.md) for a short introduction (not yet done).
Reference
------------------
See [LANGUAGE.md](LANGUAGE.md) for a reference of the language.
See [anselme.md](anselme.md) for the Lua API's documentation.

77
TUTORIAL.md
View file

@ -1,77 +0,0 @@
Anselme short tutorial
======================
This document is a work-in-progress and currently mostly useless.
Level 1: basics
---------------
Basics of Anselme. Should be enough to make a "choose-your-own-adventure" type script using the test game runner.
### Text
Writing simple text.
### Choices
Defining multiple choices.
### Basic functions
Defining functions without arguments and switching to them.
### Variables and conditions
Variable & constant definition, simple conditions and expressions.
Level 2: intermediate
---------------------
More advanced features that you would likely need if you intend to integrate Anselme into your game.
### Checkpoints
Purpose and how they work.
### Tags
Tag lines, subtexts.
### Events and adding Anselme to your game
Events buffer, basic Lua API.
### Other line types
Loops, comments.
Level 3: advanced
-----------------
If you want to make full use of Anselme's features for your game, or just want to flex about a language nobody's heard of in your CV. This part will assume previous programming knowledge.
### Advanced functions
Arguments, scopes, return values.
### Advanced expressions
Variable types, operators, built-in functions. This part is going to be long...
#### General rules of an expression
#### Operators
#### Main types
#### Sequences and maps
#### Objects
#### References
#### Function dispatch
### Translation
Aliases, what can be translated, what is saved.

838
anselme.lua
View file

@ -1,794 +1,80 @@
--- anselme main module
--- The main module.
--- Anselme Lua API reference
--
-- We actively support LuaJIT and Lua 5.4. Lua 5.1, 5.2 and 5.3 *should* work but I don't always test against them.
--
-- This documentation is generated from the main module file `anselme.lua` using `ldoc --ext md anselme.lua`.
--
-- Example usage:
-- Naming conventions:
-- * Classes
-- * everything_else
-- * (note: "classes" that are not meat to be instancied and are just here to benefit from inheritance fall into everything_else, e.g. parsing classes)
--- Usage:
-- ```lua
-- local anselme = require("anselme") -- load main module
-- local anselme = require("anselme")
--
-- local vm = anselme() -- create new VM
-- vm:loadgame("game") -- load some scripts, etc.
-- local interpreter = vm:rungame() -- create a new interpreter using what was loaded with :loadgame
-- -- create a new state
-- local state = anselme.new()
-- state:load_stdlib()
--
-- -- simple function to convert text event data into a string
-- -- in your game you may want to handle tags, here we ignore them for simplicity
-- local function format_text(text)
-- local r = ""
-- for _, l in ipairs(t) do
-- r = r .. l.text
-- end
-- return r
-- end
-- -- read an anselme script file
-- local f = assert(io.open("script.ans"))
-- local script = anselme.parse(f:read("*a"), "script.ans")
-- f:close()
--
-- -- event loop
-- repeat
-- local event, data = interpreter:step() -- progress script until next event
-- if event == "text" then
-- print(format_text(d))
-- elseif event == "choice" then
-- for j, choice in ipairs(d) do
-- print(j.."> "..format_text(choice))
-- -- load the script in a new branch
-- local run_state = state:branch()
-- run_state:run(script)
--
-- -- run the script
-- while run_state:active() do
-- local e, data = run_state:step()
-- if e == "text" then
-- for _, l in ipairs(data) do
-- print(l:format(run_state))
-- end
-- interpreter:choose(io.read())
-- elseif event == "error" then
-- elseif e == "choice" then
-- for i, l in ipairs(data) do
-- print(("%s> %s"):format(i, l:format(run_state)))
-- end
-- local choice = tonumber(io.read("*l"))
-- data:choose(choice)
-- elseif e == "return" then
-- run_state:merge()
-- elseif e == "error" then
-- error(data)
-- end
-- until t == "return" or t == "error"
-- end
-- ```
--
-- Calling the Anselme main module will create a return a new [VM](#vms).
--
-- The main module also contain a few fields:
--
-- @type anselme
local anselme = {
--- Anselme version information table.
--
-- Contains version informations as number (higher means more recent) of Anselme divied in a few categories:
--
-- * `save`, which is incremented at each update which may break save compatibility
-- * `language`, which is incremented at each update which may break script file compatibility
-- * `api`, which is incremented at each update which may break Lua API compatibility
versions = {
save = 2,
language = 25,
api = 6
},
--- General version number.
--
-- It is incremented at each update.
version = 26,
--- Currently running [interpreter](#interpreters).
-- `nil` if no interpreter running.
running = nil
}
package.loaded[...] = anselme
-- load libs
local anselme_root = (...):gsub("anselme$", "")
local preparse = require(anselme_root.."parser.preparser")
local postparse = require(anselme_root.."parser.postparser")
local expression = require(anselme_root.."parser.expression")
local eval = require(anselme_root.."interpreter.expression")
local injections = require(anselme_root.."parser.common").injections
local run_line = require(anselme_root.."interpreter.interpreter").run_line
local run = require(anselme_root.."interpreter.interpreter").run
local to_lua = require(anselme_root.."interpreter.common").to_lua
local merge_state = require(anselme_root.."interpreter.common").merge_state
local stdfuncs = require(anselme_root.."stdlib.functions")
local bootscript = require(anselme_root.."stdlib.bootscript")
local copy = require(anselme_root.."common").copy
local should_be_persisted = require(anselme_root.."interpreter.common").should_be_persisted
local check_persistable = require(anselme_root.."interpreter.common").check_persistable
local parser = require("parser")
local State = require("state.State")
require("ast.abstract.Node"):_i_hate_cycles()
-- wrappers for love.filesystem / luafilesystem
local function list_directory(path)
local t = {}
if love then
t = love.filesystem.getDirectoryItems(path)
else
local lfs = require("lfs")
for item in lfs.dir(path) do
table.insert(t, item)
end
end
return t
end
local function is_directory(path)
if love then
return not not love.filesystem.getInfo(path, "directory")
else
local lfs = require("lfs")
return lfs.attributes(path, "mode") == "directory"
end
end
local function is_file(path)
if love then
return not not love.filesystem.getInfo(path, "file")
else
local lfs = require("lfs")
return lfs.attributes(path, "mode") == "file"
end
end
--- Interpreters
--
-- An interpreter is in charge of running Anselme code and is spawned from a [VM](#vms).
-- Several interpreters from the same VM can run at the same time.
--
-- Typically, you would have a interpreter for each script that need at the same time, for example one for every NPC
-- that is currently talking.
--
-- Each interpreter can only run one script at a time, and will run it sequentially.
-- You can advance in the script by calling the `:step` method, which will run the script until an event is sent (for example some text needs to be displayed),
-- which will pause the whole interpreter until `:step` is called again.
--
-- @type interpreter
local interpreter_methods = {
--- interpreter state
-- for internal use, you shouldn't touch this
-- @local
state = nil,
--- [VM](#vms) this interpreter belongs to.
vm = nil,
--- String, type of the event that stopped the interpreter (`nil` if interpreter is still running).
end_event = nil,
--- Run the interpreter until the next event.
-- Returns event type (string), data (any).
--
-- Will merge changed variables on successful script end.
--
-- If event is `"return"` or `"error"`, the interpreter can not be stepped further and should be discarded.
--
-- Default event types and their associated data:
-- * `text`: text to display, data is a list of text elements, each with a `text` field, containing the text contents, and a `tags` field, containing the tags associated with this text
-- * `choice`: choices to choose from, data is a list of choices Each of these choice is a list of text elements like for the `text` event
-- * `return`: when the script ends, data is the returned value (`nil` if nothing returned)
-- * `error`: when there is an error, data is the error message.
--
-- See [LANGUAGE.md](LANGUAGE.md) for more details on events.
step = function(self)
-- check status
if self.end_event then
return "error", ("interpreter can't be restarted after receiving a %s event"):format(self.end_event)
end
if coroutine.status(self.state.interpreter.coroutine) ~= "suspended" then
return "error", ("can't step interpreter because it has already finished or is already running (coroutine status: %s)"):format(coroutine.status(self.state.interpreter.coroutine))
end
-- handle interrupt
if self.state.interpreter.interrupt then
local expr = self.state.interpreter.interrupt
if expr == true then
return "return", "" -- nothing to do after interrupt
else
local line = self.state.interpreter.running_line
local namespace = self:current_namespace()
-- replace state with interrupted state
local exp, err = expression(expr, self.state.interpreter.global_state, namespace or "", "interpreter:interrupt")
if not exp then return "error", ("%s; during interrupt %q at %s"):format(err, expr, line and line.source or "unknown") end
local r, e = self.vm:run(exp)
if not r then return "error", e end
self.state = r.state
end
end
-- run
local previous = anselme.running
anselme.running = self
local success, event, data = coroutine.resume(self.state.interpreter.coroutine)
anselme.running = previous
if not success then event, data = "error", event end
if event == "return" then merge_state(self.state) end
if event == "return" or event == "error" then self.end_event = event end
return event, data
end,
--- Select a choice.
-- `i` is the index (number) of the choice in the choice list (from the choice event's data).
--
-- The choice will be selected on the next interpreter step.
--
-- Returns this interpreter.
choose = function(self, i)
self.state.interpreter.choice_selected = tonumber(i)
return self
end,
--- Interrupt (abort the currently running script) the interpreter on the next step, executing an expression (string, if specified) in the current namespace instead.
--
-- Returns this interpreter.
interrupt = function(self, expr)
self.state.interpreter.interrupt = expr or true
return self
end,
--- Returns the namespace (string) the last ran line belongs to.
current_namespace = function(self)
local line = self.state.interpreter.running_line
local namespace = ""
if line then
local cur_line = line
namespace = cur_line.namespace
while not namespace do
local block = cur_line.parent_block
if not block.parent_line then break end -- reached root
cur_line = block.parent_line
namespace = cur_line.namespace
end
end
return namespace
end,
--- Run an expression (string) or block, optionally in a specific namespace (string, will use root namespace if not specified).
-- This may trigger events and must be called from within the interpreter coroutine (i.e. from a function called from a running script).
--
-- No automatic merge if this change the interpreter state, merge is done once we reach end of script in a call to `:step` as usual.
--
-- Returns the returned value (nil if nothing returned).
run = function(self, expr, namespace)
-- check status
if coroutine.status(self.state.interpreter.coroutine) ~= "running" then
error("run must be called from whithin the interpreter coroutine")
end
-- parse
local err
if type(expr) ~= "table" then expr, err = expression(tostring(expr), self.state.interpreter.global_state, namespace or "", "interpreter:run") end
if not expr then coroutine.yield("error", err) end
-- run
local r, e
if expr.type == "block" then
r, e = run(self.state, expr)
else
r, e = eval(self.state, expr)
end
if not r then coroutine.yield("error", e) end
if self.state.interpreter.current_event then -- flush final events
local rf, re = run_line(self.state, { type = "flush events" })
if re then coroutine.yield("error", re) end
if rf then r = rf end
end
return to_lua(r, self.state)
end,
--- Evaluate an expression (string) or block, optionally in a specific namespace (string, will use root namespace if not specified).
-- The expression can't yield events.
-- Can be called from outside the interpreter coroutine. Will create a new coroutine that operate on this interpreter state.
--
-- No automatic merge if this change the interpreter state, merge is done once we reach end of script in a call to `:step` as usual.
--
-- Returns the returned value in case of success (nil if nothing returned).
--
-- Returns nil, error message in case of error.
eval = function(self, expr, namespace)
if self.end_event then
return "error", ("interpreter can't be restarted after receiving a %s event"):format(self.end_event)
end
-- parse
local err
if type(expr) ~= "table" then expr, err = expression(tostring(expr), self.state.interpreter.global_state, namespace or "", "interpreter:eval") end
if not expr then return nil, err end
-- run
local co = coroutine.create(function()
local r, e
if expr.type == "block" then
r, e = run(self.state, expr)
else
r, e = eval(self.state, expr)
end
if not r then return "error", e end
return "return", r
end)
local previous = anselme.running
anselme.running = self
local success, event, data = coroutine.resume(co)
anselme.running = previous
if not success then
return nil, event
elseif event == "error" then
self.end_event = "error"
return nil, data
elseif event ~= "return" then
return nil, ("evaluated expression generated an %q event; at %s"):format(event, self.state.interpreter.running_line.source)
else
return to_lua(data, self.state)
end
end,
}
interpreter_methods.__index = interpreter_methods
--- VMs
--
-- A VM stores the state required to run Anselme scripts. Each VM is completely independant from each other.
--
-- @type vm
local vm_mt = {
--- anselme state
-- for internal use, you shouldn't touch this
-- @local
state = nil,
--- loaded game state
-- for internal use, you shouldn't touch this
-- @local
game = nil,
--- Wrapper for loading a whole set of scripts (a "game").
-- Should be preferred to other loading functions if possible as this sets all the common options on its own.
--
-- Requires LÖVE or LuaFileSystem.
--
-- Will load from the directory given by `path` (string), in order:
-- * `config.ans`, which will be executed in the "config" namespace and may contains various optional configuration options:
-- * `anselme version`: number, version of the anselme language this game was made for
-- * `game version`: any, version information of the game. Can be used to perform eventual migration of save with an old version in the main file.
-- Always included in saved variables.
-- * `language`: string, built-in language file to load
-- * `inject directory`: string, directory that may contain "function start.ans", "checkpoint end.ans", etc. which content will be used to setup
-- the custom code injection methods (see vm:setinjection)
-- * `global directory`: string, path of global script directory. Every script file and subdirectory in the path will be loaded in the global namespace.
-- * `start expression`: string, expression that will be ran when starting the game
-- * files in the global directory, if defined in config.ans
-- * every other file in the path and subdirectories, using their path as namespace (i.e., contents of path/world1/john.ans will be defined in a function world1.john)
--
-- Returns this VM in case of success.
--
-- Returns nil, error message in case of error.
loadgame = function(self, path)
if self.game then error("game already loaded") end
-- load config
if is_file(path.."/config.ans") then
local s, e = self:loadfile(path.."/config.ans", "config")
if not s then return s, e end
s, e = self:eval("config")
if e then return s, e end
end
-- get config
self.game = {
anselme_version = self:eval("config.anselme version"),
game_version = self:eval("config.game version"),
language = self:eval("config.language"),
inject_directory = self:eval("config.inject directory"),
global_directory = self:eval("config.global directory"),
start_expression = self:eval("config.start expression")
}
-- check language version
if self.game.anselme_version and self.game.anselme_version ~= anselme.versions.language then
return nil, ("trying to load game made for Anselme language %s, but currently using version %s"):format(self.game.anselme_version, anselme.versions.language)
end
-- load language
if self.game.language then
local s, e = self:loadlanguage(self.game.language)
if not s then return s, e end
end
-- load injections
if self.game.inject_directory then
for inject, ninject in pairs(injections) do
local f = io.open(path.."/"..self.game.inject_directory.."/"..inject..".ans", "r")
if f then
self.state.inject[ninject] = f:read("*a")
f:close()
end
end
end
-- load global scripts
for _, item in ipairs(list_directory(path.."/"..self.game.global_directory)) do
if item:match("[^%.]") then
local p = path.."/"..self.game.global_directory.."/"..item
local s, e
if is_directory(p) then
s, e = self:loaddirectory(p)
elseif item:match("%.ans$") then
s, e = self:loadfile(p)
end
if not s then return s, e end
end
end
-- load other files
for _, item in ipairs(list_directory(path)) do
if item:match("[^%.]") and
item ~= "config.ans" and
item ~= self.game.global_directory and
item ~= self.game.inject_directory
then
local p = path.."/"..item
local s, e
if is_directory(p) then
s, e = self:loaddirectory(p, item)
elseif item:match("%.ans$") then
s, e = self:loadfile(p, item:gsub("%.ans$", ""))
end
if not s then return s, e end
end
end
return self
end,
--- Return a interpreter which runs the game start expression (if given).
--
-- Returns interpreter in case of success.
--
-- Returns nil, error message in case of error.
rungame = function(self)
if not self.game then error("no game loaded") end
if self.game.start_expression then
return self:run(self.game.start_expression)
else
return self:run("()")
end
end,
--- Load code from a string.
-- Similar to Lua's code loading functions.
--
-- Compared to their Lua equivalents, these also take an optional `name` argument (default="") that set the namespace to load the code in. Will define a new function is specified; otherwise, code will be parsed but not executable from an expression (as it is not named).
--
-- Returns parsed block in case of success.
--
-- Returns nil, error message in case of error.
loadstring = function(self, str, name, source)
local s, e = preparse(self.state, str, name or "", source)
if not s then return s, e end
return s
end,
--- Load code from a file.
-- See `vm:loadstring`.
loadfile = function(self, path, name)
local content
if love then
local e
content, e = love.filesystem.read(path)
if not content then return content, e end
else
local f, e = io.open(path, "r")
if not f then return f, e end
content = f:read("*a")
f:close()
end
local s, err = self:loadstring(content, name, path)
if not s then return s, err end
return s
end,
-- Load every file in a directory, using filename (without .ans extension) as its namespace.
--
-- Requires LÖVE or LuaFileSystem.
--
-- Returns this VM in case of success.
--
-- Returns nil, error message in case of error.
loaddirectory = function(self, path, name)
if not name then name = "" end
name = name == "" and "" or name.."."
for _, item in ipairs(list_directory(path)) do
if item:match("[^%.]") then
local p = path.."/"..item
local s, e
if is_directory(p) then
s, e = self:loaddirectory(p, name..item)
elseif item:match("%.ans$") then
s, e = self:loadfile(p, name..item:gsub("%.ans$", ""))
end
if not s then return s, e end
end
end
return self
end,
--- Set aliases for built-in variables 👁️, 🔖 and 🏁 that will be defined on every new checkpoint and function.
-- This does not affect variables that were defined before this function was called.
-- Set to nil for no alias.
--
-- Returns this VM.
setaliases = function(self, seen, checkpoint, reached)
self.state.builtin_aliases["👁️"] = seen
self.state.builtin_aliases["🔖"] = checkpoint
self.state.builtin_aliases["🏁"] = reached
return self
end,
--- Set some code that will be injected at specific places in all code loaded after this is called.
-- Can typically be used to define variables for every function like 👁️, setting some value on every function resume, etc.
--
-- Possible inject types:
-- * `"function start"`: injected at the start of every non-scoped function
-- * `"function end"`: injected at the end of every non-scoped function
-- * `"function return"`: injected at the end of each return's children that is contained in a non-scoped function
-- * `"checkpoint start"`: injected at the start of every checkpoint
-- * `"checkpoint end"`: injected at the end of every checkpoint
-- * `"class start"`: injected at the start of every class
-- * `"class end"`: injected at the end of every class
-- * `"scoped function start"`: injected at the start of every scoped function
-- * `"scoped function end"`: injected at the end of every scoped function
-- * `"scoped function return"`: injected at the end of each return's children that is contained in a scoped function
--
-- Set `code` to nil to disable the inject.
--
-- Returns this VM.
setinjection = function(self, inject, code)
assert(injections[inject], ("unknown injection type %q"):format(inject))
self.state.inject[injections[inject]] = code
return self
end,
--- Load and execute a built-in language file.
--
-- The language file may optionally contain the special variables:
-- * alias 👁️: string, default alias for 👁️
-- * alias 🏁: string, default alias for 🏁
-- * alias 🔖: string, default alias for 🔖
--
-- Returns this VM in case of success.
--
-- Returns nil, error message in case of error.
loadlanguage = function(self, lang)
local namespace = "anselme.languages."..lang
-- execute language file
local code = require(anselme_root.."stdlib.languages."..lang)
local s, e = self:loadstring(code, namespace, lang)
if not s then return s, e end
s, e = self:eval(namespace)
if e then return s, e end
-- set aliases for built-in variables
local seen_alias = self:eval(namespace..".alias 👁️")
local checkpoint_alias = self:eval(namespace..".alias 🔖")
local reached_alias = self:eval(namespace..".alias 🏁")
self:setaliases(seen_alias, checkpoint_alias, reached_alias)
return self
end,
--- Define functions from Lua.
--
-- * `signature`: string, full signature of the function
-- * `fn`: function (Lua function or table, see examples in `stdlib/functions.lua`)
--
-- Alternatively, can also take a table as a sole argument to load several functions: { ["signature"] = fn, ... }
--
-- Returns this VM.
loadfunction = function(self, signature, fn)
if type(signature) == "table" then
for k, v in pairs(signature) do
local s, e = self:loadfunction(k, v)
if not s then return nil, e end
end
else
if type(fn) == "function" then fn = { value = fn } end
self.state.link_next_function_definition_to_lua_function = fn
local s, e = self:loadstring(":$"..signature, "", "lua")
if not s then return nil, e end
assert(self.state.link_next_function_definition_to_lua_function == nil, "unexpected error while defining lua function")
return self
end
return self
end,
--- Save/load script state
--
-- Only saves persistent variables' full names and values.
-- Make sure to not change persistent variables names, class name, class attribute names, checkpoint names and functions names between a
-- save and a load (alias can of course be changed), as Anselme will not be able to match them to the old names stored in the save file.
--
-- If a variable is stored in the save file but is not marked as persistent in the current scripts (e.g. if you updated the Anselme scripts to
-- remove the persistence), it will not be loaded.
--
-- Loading should be done after loading all the game scripts (otherwise you will get "variable already defined" errors).
--
-- Returns this VM.
load = function(self, data)
assert(anselme.versions.save == data.anselme.versions.save, ("trying to load data from an incompatible version of Anselme; save was done using save version %s but current version is %s"):format(data.anselme.versions.save, anselme.versions.save))
for k, v in pairs(data.variables) do
if self.state.variable_metadata[k] then
if self.state.variable_metadata[k].persistent then
self.state.variables[k] = v
end
else
self.state.variables[k] = v -- non-existent variable: keep it in case there was a mistake, it's not going to affect anything anyway
end
end
return self
end,
--- Save script state.
-- See `vm:load`.
--
-- Returns save data in case of success.
--
-- Returns nil, error message in case of error.
save = function(self)
local vars = {}
for k, v in pairs(self.state.variables) do
if should_be_persisted(self.state, k, v) then
local s, e = check_persistable(v)
if not s then return nil, ("%s; while saving variable %s"):format(e, k) end
vars[k] = v
end
end
return {
anselme = {
versions = anselme.versions,
version = anselme.version
},
variables = vars
}
end,
--- Global version string. Follow semver.
version = "2.0.0-alpha",
--- Perform parsing that needs to be done after loading code.
-- This is automatically ran before starting an interpreter, but you may want to execute it before if you want to check for parsing error manually.
--
-- Returns self in case of success.
--
-- Returns nil, error message in case of error.
postload = function(self)
if #self.state.queued_lines > 0 then
local r, e = postparse(self.state)
if not r then return nil, e end
end
return self
end,
--- Table containing per-category version numbers. Incremented by one for any change that may break compatibility.
versions = {
--- Version number for languages and standard library changes.
language = 27,
--- Version number for save/AST format changes.
save = 4,
--- Version number for Lua API changes.
api = 8
},
--- Enable feature flags.
-- Available flags:
-- * `"strip trailing spaces"`: remove trailing spaces from choice and text events (enabled by default)
-- * `"strip duplicate spaces"`: remove duplicated spaces between text elements from choice and text events (enabled by default)
--- Parse a `code` string and return the generated AST.
--
-- Returns this VM.
enable = function(self, ...)
for _, flag in ipairs{...} do
self.state.feature_flags[flag] = true
end
return self
-- `source` is an optional string; it will be used as the code source name in error messages.
--
-- Usage:
-- ```lua
-- local ast = anselme.parse("1 + 2", "test")
-- ast:eval()
-- ```
parse = function(code, source)
return parser(code, source)
end,
--- Disable features flags.
-- Returns this VM.
disable = function(self, ...)
for _, flag in ipairs{...} do
self.state.feature_flags[flag] = nil
end
return self
--- Return a new [State](#state).
new = function()
return State:new()
end,
--- Run code.
-- Will merge state after successful execution
--
-- * `expr`: expression to evaluate (string or parsed expression), or a block to run
-- * `namespace`(default=""): namespace to evaluate the expression in
-- * `tags`(default={}): defaults tags when evaluating the expression (Lua value)
--
-- Return interpreter in case of success.
--
-- Returns nil, error message in case of error.
run = function(self, expr, namespace, tags)
local s, e = self:postload()
if not s then return s, e end
--
local err
if type(expr) ~= "table" then expr, err = expression(tostring(expr), self.state, namespace or "", "vm:run") end
if not expr then return expr, err end
-- interpreter state
local interpreter
interpreter = {
state = {
inject = self.state.inject,
feature_flags = self.state.feature_flags,
builtin_aliases = self.state.builtin_aliases,
aliases = setmetatable({}, { __index = self.state.aliases }),
functions = self.state.functions, -- no need for a cache as we can't define or modify any function from the interpreter for now
variable_metadata = self.state.variable_metadata, -- no cache as metadata are expected to be constant
variables = setmetatable({}, {
__index = function(variables, k)
local cache = getmetatable(variables).cache
if cache[k] == nil then
cache[k] = copy(self.state.variables[k], getmetatable(variables).copy_cache)
end
return cache[k]
end,
-- variables that keep current state and should be cleared at each checkpoint
cache = {}, -- cache of previously read values (copies), to get repeatable reads & handle mutable types without changing global state
modified_tables = {}, -- list of modified tables (copies) that should be merged with global state on next checkpoint
copy_cache = {}, -- table of [original table] = copied table. Automatically filled by copy().
-- keep track of scoped variables in scoped functions [fn line] = {{scoped variables}, next scope, ...}
-- (scoped variables aren't merged on checkpoint, shouldn't be cleared at checkpoints)
-- (only stores scoped variables that have been reassigned at some point (i.e. every accessed one since they start as undefined))
scoped = {}
}),
interpreter = {
-- constant
global_state = self.state,
coroutine = coroutine.create(function() return "return", interpreter:run(expr, namespace) end),
-- status
running_line = nil,
-- choice event
choice_selected = nil,
-- skip next choices until next event change (to skip currently running choice block when resuming from a checkpoint)
skip_choices_until_flush = nil,
-- active event buffer stack
event_buffer_stack = {},
-- current event waiting to be sent
current_event = nil,
-- interrupt
interrupt = nil,
-- tag stack
tags = {},
-- default tags for everything in this interpreter (Lua values)
base_lua_tags = tags,
},
},
vm = self
}
return setmetatable(interpreter, interpreter_methods)
end,
--- Evaluate code.
-- Behave like `:run`, except the expression can not emit events and will return the result of the expression directly.
-- Merge state after sucessful execution automatically like `:run`.
--
-- * `expr`: expression to evaluate (string or parsed expression), or a block to evaluate
-- * `namespace`(default=""): namespace to evaluate the expression in
-- * `tags`(default={}): defaults tags when evaluating the expression (Lua value)
--
-- Return value in case of success (nil if nothing returned).
--
-- Returns nil, error message in case of error.
eval = function(self, expr, namespace, tags)
local interpreter, err = self:run("()", namespace, tags)
if not interpreter then return interpreter, err end
local r, e = interpreter:eval(expr, namespace)
if e then return r, e end
assert(interpreter:step() == "return", "evaluated expression can not emit events") -- trigger merge / end-of-script things
return r
end
}
vm_mt.__index = vm_mt
-- return anselme module
return setmetatable(anselme, {
__call = function()
-- global state
local state = {
inject = {
-- function_start = "code block...", ...
},
feature_flags = {
["strip trailing spaces"] = true,
["strip duplicate spaces"] = true
},
builtin_aliases = {
-- ["👁️"] = "seen",
-- ["🔖"] = "checkpoint",
-- ["🏁"] = "reached"
},
aliases = {
-- ["bonjour.salutation"] = "hello.greeting", ...
},
functions = {
-- ["script.fn"] = {
-- {
-- function or checkpoint table
-- }, ...
-- }, ...
},
variable_metadata = {
-- foo = { constant = true, persistent = true, constraint = constraint, ... }, ...
},
variables = {
-- foo = {
-- type = "number",
-- value = 42
-- }, ...
},
queued_lines = {
-- { line = line, namespace = "foo" }, ...
},
link_next_function_definition_to_lua_function = nil -- temporarly set to tell the preparser to link a anselme function definition with a lua function
}
local vm = setmetatable({ state = state }, vm_mt)
-- bootscript
local boot = assert(vm:loadstring(bootscript, "", "boot script"))
local _, e = vm:eval(boot)
if e then error(e) end
-- lua-defined functions
assert(vm:loadfunction(stdfuncs.lua))
-- anselme-defined functions
local ansfunc = assert(vm:loadstring(stdfuncs.anselme, "", "built-in functions"))
_, e = vm:eval(ansfunc)
if e then return error(e) end
return vm
end
})

322
anselme.md
View file

@ -1,322 +0,0 @@
## Anselme Lua API reference
We actively support LuaJIT and Lua 5.4. Lua 5.1, 5.2 and 5.3 *should* work but I don't always test against them.
This documentation is generated from the main module file `anselme.lua` using `ldoc --ext md anselme.lua`.
Example usage:
```lua
local anselme = require("anselme") -- load main module
local vm = anselme() -- create new VM
vm:loadgame("game") -- load some scripts, etc.
local interpreter = vm:rungame() -- create a new interpreter using what was loaded with :loadgame
-- simple function to convert text event data into a string
-- in your game you may want to handle tags, here we ignore them for simplicity
local function format_text(text)
local r = ""
for _, l in ipairs(t) do
r = r .. l.text
end
return r
end
-- event loop
repeat
local event, data = interpreter:step() -- progress script until next event
if event == "text" then
print(format_text(d))
elseif event == "choice" then
for j, choice in ipairs(d) do
print(j.."> "..format_text(choice))
end
interpreter:choose(io.read())
elseif event == "error" then
error(data)
end
until t == "return" or t == "error"
```
Calling the Anselme main module will create a return a new [VM](#vms).
The main module also contain a few fields:
### anselme.versions
Anselme version information table.
Contains version informations as number (higher means more recent) of Anselme divied in a few categories:
* `save`, which is incremented at each update which may break save compatibility
* `language`, which is incremented at each update which may break script file compatibility
* `api`, which is incremented at each update which may break Lua API compatibility
### anselme.version
General version number.
It is incremented at each update.
### anselme.running
Currently running [interpreter](#interpreters).
`nil` if no interpreter running.
## Interpreters
An interpreter is in charge of running Anselme code and is spawned from a [VM](#vms).
Several interpreters from the same VM can run at the same time.
Typically, you would have a interpreter for each script that need at the same time, for example one for every NPC
that is currently talking.
Each interpreter can only run one script at a time, and will run it sequentially.
You can advance in the script by calling the `:step` method, which will run the script until an event is sent (for example some text needs to be displayed),
which will pause the whole interpreter until `:step` is called again.
### interpreter.vm
[VM](#vms) this interpreter belongs to.
### interpreter.end_event
String, type of the event that stopped the interpreter (`nil` if interpreter is still running).
### interpreter:step ()
Run the interpreter until the next event.
Returns event type (string), data (any).
Will merge changed variables on successful script end.
If event is `"return"` or `"error"`, the interpreter can not be stepped further and should be discarded.
Default event types and their associated data:
* `text`: text to display, data is a list of text elements, each with a `text` field, containing the text contents, and a `tags` field, containing the tags associated with this text
* `choice`: choices to choose from, data is a list of choices Each of these choice is a list of text elements like for the `text` event
* `return`: when the script ends, data is the returned value (`nil` if nothing returned)
* `error`: when there is an error, data is the error message.
See [LANGUAGE.md](LANGUAGE.md) for more details on events.
### interpreter:choose (i)
Select a choice.
`i` is the index (number) of the choice in the choice list (from the choice event's data).
The choice will be selected on the next interpreter step.
Returns this interpreter.
### interpreter:interrupt (expr)
Interrupt (abort the currently running script) the interpreter on the next step, executing an expression (string, if specified) in the current namespace instead.
Returns this interpreter.
### interpreter:current_namespace ()
Returns the namespace (string) the last ran line belongs to.
### interpreter:run (expr, namespace)
Run an expression (string) or block, optionally in a specific namespace (string, will use root namespace if not specified).
This may trigger events and must be called from within the interpreter coroutine (i.e. from a function called from a running script).
No automatic merge if this change the interpreter state, merge is done once we reach end of script in a call to `:step` as usual.
Returns the returned value (nil if nothing returned).
### interpreter:eval (expr, namespace)
Evaluate an expression (string) or block, optionally in a specific namespace (string, will use root namespace if not specified).
The expression can't yield events.
Can be called from outside the interpreter coroutine. Will create a new coroutine that operate on this interpreter state.
No automatic merge if this change the interpreter state, merge is done once we reach end of script in a call to `:step` as usual.
Returns the returned value in case of success (nil if nothing returned).
Returns nil, error message in case of error.
## VMs
A VM stores the state required to run Anselme scripts. Each VM is completely independant from each other.
### vm:loadgame (path)
Wrapper for loading a whole set of scripts (a "game").
Should be preferred to other loading functions if possible as this sets all the common options on its own.
Requires LÖVE or LuaFileSystem.
Will load from the directory given by `path` (string), in order:
* `config.ans`, which will be executed in the "config" namespace and may contains various optional configuration options:
* `anselme version`: number, version of the anselme language this game was made for
* `game version`: any, version information of the game. Can be used to perform eventual migration of save with an old version in the main file.
Always included in saved variables.
* `language`: string, built-in language file to load
* `inject directory`: string, directory that may contain "function start.ans", "checkpoint end.ans", etc. which content will be used to setup
the custom code injection methods (see vm:setinjection)
* `global directory`: string, path of global script directory. Every script file and subdirectory in the path will be loaded in the global namespace.
* `start expression`: string, expression that will be ran when starting the game
* files in the global directory, if defined in config.ans
* every other file in the path and subdirectories, using their path as namespace (i.e., contents of path/world1/john.ans will be defined in a function world1.john)
Returns this VM in case of success.
Returns nil, error message in case of error.
### vm:rungame ()
Return a interpreter which runs the game start expression (if given).
Returns interpreter in case of success.
Returns nil, error message in case of error.
### vm:loadstring (str, name, source)
Load code from a string.
Similar to Lua's code loading functions.
Compared to their Lua equivalents, these also take an optional `name` argument (default="") that set the namespace to load the code in. Will define a new function is specified; otherwise, code will be parsed but not executable from an expression (as it is not named).
Returns parsed block in case of success.
Returns nil, error message in case of error.
### vm:loadfile (path, name)
Load code from a file.
See `vm:loadstring`.
### vm:setaliases (seen, checkpoint, reached)
Set aliases for built-in variables 👁️, 🔖 and 🏁 that will be defined on every new checkpoint and function.
This does not affect variables that were defined before this function was called.
Set to nil for no alias.
Returns this VM.
### vm:setinjection (inject, code)
Set some code that will be injected at specific places in all code loaded after this is called.
Can typically be used to define variables for every function like 👁️, setting some value on every function resume, etc.
Possible inject types:
* `"function start"`: injected at the start of every non-scoped function
* `"function end"`: injected at the end of every non-scoped function
* `"function return"`: injected at the end of each return's children that is contained in a non-scoped function
* `"checkpoint start"`: injected at the start of every checkpoint
* `"checkpoint end"`: injected at the end of every checkpoint
* `"class start"`: injected at the start of every class
* `"class end"`: injected at the end of every class
* `"scoped function start"`: injected at the start of every scoped function
* `"scoped function end"`: injected at the end of every scoped function
* `"scoped function return"`: injected at the end of each return's children that is contained in a scoped function
Set `code` to nil to disable the inject.
Returns this VM.
### vm:loadlanguage (lang)
Load and execute a built-in language file.
The language file may optionally contain the special variables:
* alias 👁️: string, default alias for 👁️
* alias 🏁: string, default alias for 🏁
* alias 🔖: string, default alias for 🔖
Returns this VM in case of success.
Returns nil, error message in case of error.
### vm:loadfunction (signature, fn)
Define functions from Lua.
* `signature`: string, full signature of the function
* `fn`: function (Lua function or table, see examples in `stdlib/functions.lua`)
Returns this VM.
### vm:load (data)
Save/load script state
Only saves persistent variables' full names and values.
Make sure to not change persistent variables names, class name, class attribute names, checkpoint names and functions names between a
save and a load (alias can of course be changed), as Anselme will not be able to match them to the old names stored in the save file.
If a variable is stored in the save file but is not marked as persistent in the current scripts (e.g. if you updated the Anselme scripts to
remove the persistence), it will not be loaded.
Loading should be done after loading all the game scripts (otherwise you will get "variable already defined" errors).
Returns this VM.
### vm:save ()
Save script state.
See `vm:load`.
Returns save data in case of success.
Returns nil, error message in case of error.
### vm:postload ()
Perform parsing that needs to be done after loading code.
This is automatically ran before starting an interpreter, but you may want to execute it before if you want to check for parsing error manually.
Returns self in case of success.
Returns nil, error message in case of error.
### vm:enable (...)
Enable feature flags.
Available flags:
* `"strip trailing spaces"`: remove trailing spaces from choice and text events (enabled by default)
* `"strip duplicate spaces"`: remove duplicated spaces between text elements from choice and text events (enabled by default)
Returns this VM.
### vm:disable (...)
Disable features flags.
Returns this VM.
### vm:run (expr, namespace, tags)
Run code.
Will merge state after successful execution
* `expr`: expression to evaluate (string or parsed expression), or a block to run
* `namespace`(default=""): namespace to evaluate the expression in
* `tags`(default={}): defaults tags when evaluating the expression (Lua value)
Return interpreter in case of success.
Returns nil, error message in case of error.
### vm:eval (expr, namespace, tags)
Evaluate code.
Behave like `:run`, except the expression can not emit events and will return the result of the expression directly.
Merge state after sucessful execution automatically like `:run`.
* `expr`: expression to evaluate (string or parsed expression), or a block to evaluate
* `namespace`(default=""): namespace to evaluate the expression in
* `tags`(default={}): defaults tags when evaluating the expression (Lua value)
Return value in case of success (nil if nothing returned).
Returns nil, error message in case of error.

207
ast/ArgumentTuple.lua Normal file
View file

@ -0,0 +1,207 @@
local ast = require("ast")
local Identifier, Number
local operator_priority = require("common").operator_priority
local ArgumentTuple
ArgumentTuple = ast.abstract.Node {
type = "argument tuple",
list = nil, -- list of expr
named = nil, -- { [string] = expr, ... }
assignment = nil, -- expr
arity = 0,
init = function(self, ...)
self.list = { ... }
self.named = {}
self.arity = #self.list
end,
insert_positional = function(self, position, val) -- only for construction
local l = {}
for k, v in pairs(self.list) do
if k >= position then l[k+1] = v
else l[k] = v end
end
l[position] = val
self.list = l
self.arity = self.arity + 1
end,
set_positional = function(self, position, val) -- only for construction
assert(not self.list[position])
self.list[position] = val
self.arity = self.arity + 1
end,
set_named = function(self, identifier, val) -- only for construction
local name = identifier.name
assert(not self.named[name])
self.named[name] = val
self.arity = self.arity + 1
end,
set_assignment = function(self, val) -- only for construction
assert(not self.assignment)
self.assignment = val
self.arity = self.arity + 1
self.format_priority = operator_priority["_=_"]
end,
_format = function(self, state, priority, ...)
local l = {}
for _, e in pairs(self.list) do
table.insert(l, e:format(state, operator_priority["_,_"], ...))
end
for n, e in pairs(self.named) do
table.insert(l, n.."="..e:format_right(state, operator_priority["_=_"], ...))
end
local s = ("(%s)"):format(table.concat(l, ", "))
if self.assignment then
s = s .. (" = %s"):format(self.assignment:format_right(state, operator_priority["_=_"], ...))
end
return s
end,
traverse = function(self, fn, ...)
for _, e in pairs(self.list) do
fn(e, ...)
end
for _, e in pairs(self.named) do
fn(e, ...)
end
if self.assignment then
fn(self.assignment, ...)
end
end,
-- need to redefine hash to include a table.sort as pairs() in :traverse is non-deterministic
-- as well as doesn't account for named arguments names
_hash = function(self)
local t = {}
for _, e in pairs(self.list) do
table.insert(t, e:hash())
end
for n, e in pairs(self.named) do
table.insert(t, ("%s=%s"):format(n, e:hash()))
end
if self.assignment then
table.insert(t, self.assignment:hash())
end
table.sort(t)
return ("%s<%s>"):format(self.type, table.concat(t, ";"))
end,
_eval = function(self, state)
local r = ArgumentTuple:new()
for i, e in pairs(self.list) do
r:set_positional(i, e:eval(state))
end
for n, e in pairs(self.named) do
r:set_named(Identifier:new(n), e:eval(state))
end
if self.assignment then
r:set_assignment(self.assignment:eval(state))
end
return r
end,
with_first_argument = function(self, first)
local r = ArgumentTuple:new()
r:set_positional(1, first)
for i, e in pairs(self.list) do
r:set_positional(i+1, e)
end
for n, e in pairs(self.named) do
r:set_named(Identifier:new(n), e)
end
if self.assignment then
r:set_assignment(self.assignment)
end
return r
end,
-- return specificity (>=0), secondary specificity (>=0)
-- return false, failure message
match_parameter_tuple = function(self, state, params)
-- basic arity checks
if self.arity > params.max_arity or self.arity < params.min_arity then
if params.min_arity == params.max_arity then
return false, ("expected %s arguments, received %s"):format(params.min_arity, self.arity)
else
return false, ("expected between %s and %s arguments, received %s"):format(params.min_arity, params.max_arity, self.arity)
end
end
if params.assignment and not self.assignment then
return false, "expected an assignment argument"
end
-- search for parameter -> argument match
local specificity = 0
local used_list = {}
local used_named = {}
local used_assignment = false
for i, param in ipairs(params.list) do
-- search in args
local arg
if self.list[i] then
used_list[i] = true
arg = self.list[i]
elseif self.named[param.identifier.name] then
used_named[param.identifier.name] = true
arg = self.named[param.identifier.name]
elseif i == params.max_arity and params.assignment and self.assignment then
used_assignment = true
arg = self.assignment
elseif param.default then
arg = param.default
end
-- not found
if not arg then return false, ("missing parameter %s"):format(param.identifier:format(state)) end
-- type check
if param.type_check then
local r = param.type_check:call(state, ArgumentTuple:new(arg))
if not r:truthy() then return false, ("type check failure for parameter %s in function %s"):format(param.identifier:format(state), params:format(state)) end
if Number:is(r) then
specificity = specificity + r.number
else
specificity = specificity + 1
end
end
end
-- check for unused arguments
for i in pairs(self.list) do
if not used_list[i] then
return false, ("%sth positional argument is unused"):format(i)
end
end
for n in pairs(self.named) do
if not used_named[n] then
return false, ("named argument %s is unused"):format(n)
end
end
if self.assignment and not used_assignment then
return false, "assignment argument is unused"
end
-- everything is A-ok
return specificity, params.eval_depth
end,
-- assume :match_parameter_tuple was already called and returned true
bind_parameter_tuple = function(self, state, params)
for i, arg in ipairs(params.list) do
if self.list[i] then
state.scope:define(arg.identifier:to_symbol(), self.list[i])
elseif self.named[arg.identifier.name] then
state.scope:define(arg.identifier:to_symbol(), self.named[arg.identifier.name])
elseif i == params.max_arity and params.assignment then
state.scope:define(arg.identifier:to_symbol(), self.assignment)
elseif arg.default then
state.scope:define(arg.identifier:to_symbol(), arg.default:eval(state))
else
error(("no argument matching parameter %q"):format(arg.identifier.name))
end
end
end
}
package.loaded[...] = ArgumentTuple
Identifier, Number = ast.Identifier, ast.Number
return ArgumentTuple

37
ast/Assignment.lua Normal file
View file

@ -0,0 +1,37 @@
local ast = require("ast")
local Nil
local operator_priority = require("common").operator_priority
local Assignment = ast.abstract.Node {
type = "assignment",
identifier = nil,
expression = nil,
format_priority = operator_priority["_=_"],
init = function(self, identifier, expression)
self.identifier = identifier
self.expression = expression
end,
_format = function(self, ...)
return self.identifier:format(...).." = "..self.expression:format_right(...)
end,
traverse = function(self, fn, ...)
fn(self.identifier, ...)
fn(self.expression, ...)
end,
_eval = function(self, state)
local val = self.expression:eval(state)
state.scope:set(self.identifier, val)
return Nil:new()
end,
}
package.loaded[...] = Assignment
Nil = ast.Nil
return Assignment

49
ast/AttachBlock.lua Normal file
View file

@ -0,0 +1,49 @@
local ast = require("ast")
local Identifier, Quote
local attached_block_identifier
local AttachBlock = ast.abstract.Node {
type = "attach block",
expression = nil,
block = nil,
init = function(self, expression, block)
self.expression = expression
self.block = block
self.format_priority = self.expression.format_priority
end,
_format = function(self, state, priority, indentation, ...)
return self.expression:format(state, priority, indentation, ...).."\n\t"..self.block:format(state, priority, indentation + 1, ...)
end,
traverse = function(self, fn, ...)
fn(self.expression, ...)
fn(self.block, ...)
end,
_eval = function(self, state)
state.scope:push_partial(attached_block_identifier)
state.scope:define(attached_block_identifier:to_symbol(), Quote:new(self.block)) -- _ is always wrapped in a Call when it appears
local exp = self.expression:eval(state)
state.scope:pop()
return exp
end,
_prepare = function(self, state)
state.scope:push_partial(attached_block_identifier)
state.scope:define(attached_block_identifier:to_symbol(), Quote:new(self.block))
self.expression:prepare(state)
state.scope:pop()
end
}
package.loaded[...] = AttachBlock
Identifier, Quote = ast.Identifier, ast.Quote
attached_block_identifier = Identifier:new("_")
return AttachBlock

80
ast/Block.lua Normal file
View file

@ -0,0 +1,80 @@
local ast = require("ast")
local Nil, Return, AutoCall, ArgumentTuple, Flush
local Block = ast.abstract.Node {
type = "block",
expressions = {},
init = function(self)
self.expressions = {}
end,
add = function(self, expression) -- only for construction
table.insert(self.expressions, expression)
end,
_format = function(self, state, prio, ...)
local l = {}
for _, e in ipairs(self.expressions) do
if Flush:is(e) then
table.insert(l, (e:format(state, 0, ...):gsub("\n$", "")))
else
table.insert(l, e:format(state, 0, ...))
end
end
return table.concat(l, "\n")
end,
traverse = function(self, fn, ...)
for _, e in ipairs(self.expressions) do
fn(e, ...)
end
end,
_eval = function(self, state)
local r
state.scope:push()
if self:resuming(state) then
local resuming = self:get_resume_data(state)
local resumed = false
for _, e in ipairs(self.expressions) do
if e == resuming then resumed = true end
if resumed then
r = e:eval(state)
if AutoCall:issub(r) then
r = r:call(state, ArgumentTuple:new())
end
if Return:is(r) then
break -- pass on to parent block until we reach a function boundary
end
end
end
else
for _, e in ipairs(self.expressions) do
self:set_resume_data(state, e)
r = e:eval(state)
if AutoCall:issub(r) then
r = r:call(state, ArgumentTuple:new())
end
if Return:is(r) then
break -- pass on to parent block until we reach a function boundary
end
end
end
state.scope:pop()
return r or Nil:new()
end,
_prepare = function(self, state)
state.scope:push()
for _, e in ipairs(self.expressions) do
e:prepare(state)
end
state.scope:pop()
end
}
package.loaded[...] = Block
Nil, Return, AutoCall, ArgumentTuple, Flush = ast.Nil, ast.Return, ast.abstract.AutoCall, ast.ArgumentTuple, ast.Flush
return Block

28
ast/Boolean.lua Normal file
View file

@ -0,0 +1,28 @@
local ast = require("ast")
return ast.abstract.Node {
type = "boolean",
_evaluated = true, -- no evaluation needed
value = nil,
init = function(self, val)
self.value = val
end,
_hash = function(self)
return ("boolean<%q>"):format(self.value)
end,
_format = function(self)
return tostring(self.value)
end,
to_lua = function(self, state)
return self.value
end,
truthy = function(self)
return self.value
end
}

59
ast/Branched.lua Normal file
View file

@ -0,0 +1,59 @@
-- branched: associate to each branch a different value
-- used to handle mutability. probably the only mutable node you'll ever need! it's literally perfect!
-- note: all values here are expected to be already evaluated
local ast = require("ast")
local Branched = ast.abstract.Runtime {
type = "branched",
mutable = true,
value = nil, -- { [branch name] = value, ... }
init = function(self, state, value)
self.value = {}
self:set(state, value)
end,
in_branch = function(self, state)
return not not self.value[state.branch_id]
end,
get = function(self, state)
return self.value[state.branch_id] or self.value[state.source_branch_id]
end,
set = function(self, state, value)
self.value[state.branch_id] = value
end,
_merge = function(self, state, cache)
local val = self.value[state.branch_id]
if val then
self.value[state.source_branch_id] = val
self.value[state.branch_id] = nil
val:merge(state, cache)
end
end,
_format = function(self, state, ...)
if state then
return self:get(state):format(state, ...)
else
local t = {}
for b, v in pairs(self.value) do
table.insert(t, ("%s→%s"):format(b, v))
end
return "<"..table.concat(t, ", ")..">"
end
end,
traverse = function(self, fn, ...)
for _, v in pairs(self.value) do
fn(v, ...)
end
end,
_eval = function(self, state)
return self:get(state)
end
}
return Branched

86
ast/Call.lua Normal file
View file

@ -0,0 +1,86 @@
local ast = require("ast")
local Identifier
local regular_operators = require("common").regular_operators
local operator_priority = require("common").operator_priority
local function reverse(t, fmt)
for _, v in ipairs(t) do t[fmt:format(v[1])] = v[2] end
return t
end
local infix = reverse(regular_operators.infixes, "_%s_")
local prefix = reverse(regular_operators.prefixes, "%s_")
local suffix = reverse(regular_operators.suffixes, "_%s")
local Call
Call = ast.abstract.Node {
type = "call",
func = nil,
arguments = nil, -- ArgumentTuple
format_priority = infix["_!"], -- often overwritten in :init
init = function(self, func, arguments)
self.func = func
self.arguments = arguments
-- get priority: operators
if Identifier:is(self.func) then
local name, arity = self.func.name, self.arguments.arity
if infix[name] and arity == 2 then
self.format_priority = infix[name]
elseif prefix[name] and arity == 1 then
self.format_priority = prefix[name]
elseif suffix[name] and arity == 1 then
self.format_priority = suffix[name]
end
end
if self.arguments.assignment then
self.format_priority = operator_priority["_=_"]
end
end,
_format = function(self, ...)
if self.arguments.arity == 0 then
if Identifier:is(self.func) and self.func.name == "_" then
return "_" -- the _ identifier is automatically re-wrapped in a Call when it appears
end
local func = self.func:format(...)
return func.."!"
else
if Identifier:is(self.func) then
local name, arity = self.func.name, self.arguments.arity
if infix[name] and arity == 2 then
local left = self.arguments.list[1]:format(...)
local right = self.arguments.list[2]:format_right(...)
return ("%s %s %s"):format(left, name:match("^_(.*)_$"), right)
elseif prefix[name] and arity == 1 then
local right = self.arguments.list[1]:format_right(...)
return ("%s%s"):format(name:match("^(.*)_$"), right)
elseif suffix[name] and arity == 1 then
local left = self.arguments.list[1]:format(...)
return ("%s%s"):format(left, name:match("^_(.*)$"))
end
end
return self.func:format(...)..self.arguments:format(...) -- no need for format_right, we already handle the assignment priority here
end
end,
traverse = function(self, fn, ...)
fn(self.func, ...)
fn(self.arguments, ...)
end,
_eval = function(self, state)
local func = self.func:eval(state)
local arguments = self.arguments:eval(state)
return func:call(state, arguments)
end
}
package.loaded[...] = Call
Identifier = ast.Identifier
return Call

52
ast/Choice.lua Normal file
View file

@ -0,0 +1,52 @@
local ast = require("ast")
local ArgumentTuple
local operator_priority = require("common").operator_priority
local Choice
Choice = ast.abstract.Runtime {
type = "choice",
text = nil,
func = nil,
format_priority = operator_priority["_|>_"],
init = function(self, text, func)
self.text = text
self.func = func
end,
traverse = function(self, fn, ...)
fn(self.text, ...)
fn(self.func, ...)
end,
_format = function(self, ...)
return ("%s |> %s"):format(self.text:format(...), self.func:format_right(...))
end,
build_event_data = function(self, state, event_buffer)
local l = {
_selected = nil,
choose = function(self, choice)
self._selected = choice
end
}
for _, c in event_buffer:iter(state) do
table.insert(l, c.text)
end
return l
end,
post_flush_callback = function(self, state, event_buffer, data)
local choice = data._selected
assert(choice, "no choice made")
assert(choice > 0 and choice <= event_buffer:len(state), "choice out of bounds")
event_buffer:get(state, choice).func:call(state, ArgumentTuple:new())
end
}
package.loaded[...] = Choice
ArgumentTuple = ast.ArgumentTuple
return Choice

58
ast/Closure.lua Normal file
View file

@ -0,0 +1,58 @@
-- note: functions only appear in non-evaluated nodes! once evaluated, they always become closures
local ast = require("ast")
local Overloadable, Runtime = ast.abstract.Overloadable, ast.abstract.Runtime
local Definition
local Closure
Closure = Runtime(Overloadable) {
type = "closure",
func = nil, -- Function
scope = nil, -- Environment
exported_scope = nil, -- Environment
init = function(self, func, state)
self.func = func
self.scope = state.scope:capture()
-- layer a new export layer on top of captured/current scope
state.scope:push_export()
self.exported_scope = state.scope:capture()
-- pre-define exports
for sym, exp in pairs(self.func.exports) do
Definition:new(sym, exp):eval(state)
end
state.scope:pop()
end,
_format = function(self, ...)
return self.func:format(...)
end,
traverse = function(self, fn, ...)
fn(self.func, ...)
fn(self.scope, ...)
fn(self.exported_scope, ...)
end,
compatible_with_arguments = function(self, state, args)
return args:match_parameter_tuple(state, self.func.parameters)
end,
format_parameters = function(self, state)
return self.func.parameters:format(state)
end,
call_compatible = function(self, state, args)
state.scope:push(self.exported_scope)
local exp = self.func:call_compatible(state, args)
state.scope:pop()
return exp
end,
}
package.loaded[...] = Closure
Definition = ast.Definition
return Closure

60
ast/Definition.lua Normal file
View file

@ -0,0 +1,60 @@
local ast = require("ast")
local Nil, Overloadable
local operator_priority = require("common").operator_priority
local Definition = ast.abstract.Node {
type = "definition",
symbol = nil,
expression = nil,
format_priority = operator_priority["_=_"],
init = function(self, symbol, expression)
self.symbol = symbol
self.expression = expression
end,
_format = function(self, ...)
return self.symbol:format(...).." = "..self.expression:format_right(...)
end,
traverse = function(self, fn, ...)
fn(self.symbol, ...)
fn(self.expression, ...)
end,
_eval = function(self, state)
if self.symbol.exported and state.scope:defined_in_current(self.symbol) then
return Nil:new() -- export vars: can reuse existing defining
end
local symbol = self.symbol:eval(state)
local val = self.expression:eval(state)
if Overloadable:issub(val) then
state.scope:define_overloadable(symbol, val)
else
state.scope:define(symbol, val)
end
return Nil:new()
end,
_prepare = function(self, state)
local symbol, val = self.symbol, self.expression
symbol:prepare(state)
val:prepare(state)
if Overloadable:issub(val) then
state.scope:define_overloadable(symbol, val)
else
state.scope:define(symbol, val)
end
end
}
package.loaded[...] = Definition
Nil, Overloadable = ast.Nil, ast.abstract.Overloadable
return Definition

214
ast/Environment.lua Normal file
View file

@ -0,0 +1,214 @@
local ast = require("ast")
local operator_priority = require("common").operator_priority
local Branched, ArgumentTuple, Overload, Overloadable, Table
local VariableMetadata = ast.abstract.Runtime {
type = "variable metadata",
symbol = nil,
branched = nil,
format_priority = operator_priority["_=_"],
init = function(self, state, symbol, value)
self.symbol = symbol
self.branched = Branched:new(state, value)
end,
get = function(self, state)
return self.branched:get(state)
end,
set = function(self, state, value)
assert(not self.symbol.constant, ("trying to change the value of constant %s"):format(self.symbol.string))
if self.symbol.type_check then
local r = self.symbol.type_check:call(state, ArgumentTuple:new(value))
if not r:truthy() then error(("type check failure for %s; %s does not satisfy %s"):format(self.symbol.string, value, self.symbol.type_check)) end
end
self.branched:set(state, value)
end,
_format = function(self, ...)
return ("%s=%s"):format(self.symbol:format(...), self.branched:format(...))
end,
traverse = function(self, fn, ...)
fn(self.symbol, ...)
fn(self.branched, ...)
end,
_merge = function(self, state, cache)
if not self.symbol.confined_to_branch then
self.branched:merge(state, cache)
end
end
}
local Environment = ast.abstract.Runtime {
type = "environment",
parent = nil, -- environment or nil
variables = nil, -- Table of { {identifier} = variable metadata, ... }
partial = nil, -- { [name string] = true, ... }
export = nil, -- bool
init = function(self, state, parent, partial_names, is_export)
self.variables = Table:new(state)
self.parent = parent
self.partial = partial_names
self.export = is_export
end,
traverse = function(self, fn, ...)
if self.parent then
fn(self.parent, ...)
end
fn(self.variables, ...)
end,
_format = function(self, state)
return "<environment>"
end,
-- define new variable in the environment
define = function(self, state, symbol, exp)
local name = symbol.string
if self:defined_in_current(state, symbol) then
error(name.." is already defined in the current scope")
end
if (self.partial and not self.partial[name])
or (self.export ~= symbol.exported) then
return self.parent:define(state, symbol, exp)
end
self.variables:set(state, symbol:to_identifier(), VariableMetadata:new(state, symbol, exp))
end,
-- define or redefine new overloadable variable in current environment, inheriting existing overload variants from (parent) scopes
define_overloadable = function(self, state, symbol, exp)
assert(Overloadable:issub(exp), "trying to add an non-overloadable value to an overload")
local identifier = symbol:to_identifier()
-- add overload variants already defined in current or parent scope
if self:defined(state, identifier) then
local val = self:get(state, identifier)
if Overload:is(val) then
exp = Overload:new(exp)
for _, v in ipairs(val.list) do
exp:insert(v)
end
elseif Overloadable:issub(val) then
exp = Overload:new(exp, val)
elseif self:defined_in_current(state, symbol) then
error(("can't add an overload variant to non-overloadable variable %s defined in the same scope"):format(identifier))
end
end
-- update/define in current scope
if self:defined_in_current(state, symbol) then
self:set(state, identifier, exp)
else
self:define(state, symbol, exp)
end
end,
-- returns bool if variable defined in current or parent environment
defined = function(self, state, identifier)
if self.variables:has(state, identifier) then
return true
elseif self.parent then
return self.parent:defined(state, identifier)
end
return false
end,
-- returns bool if variable defined in current environment layer
-- (note: by current layer, we mean the closest one where the variable is able to exist - if it is exported, the closest export layer, etc.)
defined_in_current = function(self, state, symbol)
local name = symbol.string
if self.variables:has(state, symbol:to_identifier()) then
return true
elseif (self.partial and not self.partial[name])
or (self.export ~= symbol.exported) then
return self.parent:defined_in_current(state, symbol)
end
return false
end,
-- return bool if variable is defined in the current environment only - won't search in parent event for exported & partial names
defined_in_current_strict = function(self, state, identifier)
return self.variables:has(state, identifier)
end,
-- get variable in current or parent scope, with metadata
_get_variable = function(self, state, identifier)
if self:defined(state, identifier) then
if self.variables:has(state, identifier) then
return self.variables:get(state, identifier)
elseif self.parent then
return self.parent:_get_variable(state, identifier)
end
else
error(("identifier %q is undefined in branch %s"):format(identifier.name, state.branch_id), 0)
end
end,
-- get variable value in current or parent environment
get = function(self, state, identifier)
return self:_get_variable(state, identifier):get(state)
end,
-- set variable value in current or parent environment
set = function(self, state, identifier, val)
return self:_get_variable(state, identifier):set(state, val)
end,
-- returns a list {[symbol]=val,...} of all persistent variables in the current strict layer
list_persistent = function(self, state)
assert(self.export, "not an export scope layer")
local r = {}
for _, vm in self.variables:iter(state) do
if vm.symbol.persistent then
r[vm.symbol] = vm:get(state)
end
end
return r
end,
-- returns a list {[symbol]=val,...} of all exported variables in the current strict layer
list_exported = function(self, state)
assert(self.export, "not an export scope layer")
local r = {}
for _, vm in self.variables:iter(state) do
if vm.symbol.exported then
r[vm.symbol] = vm:get(state)
end
end
return r
end,
-- return the depth of the environmenet, i.e. the number of parents
depth = function(self)
local d = 0
local e = self
while e.parent do
e = e.parent
d = d + 1
end
return d
end,
_debug_state = function(self, state, filter, t, level)
level = level or 0
t = t or {}
local indentation = string.rep(" ", level)
table.insert(t, ("%s> %s %s scope"):format(indentation, self.export and "exported" or "", self.partial and "partial" or ""))
for name, var in self.variables:iter(state) do
if name.name:match(filter) then
table.insert(t, ("%s| %s = %s"):format(indentation, name, var:get(state)))
end
end
if self.parent then
self.parent:_debug_state(state, filter, t, level+1)
end
return t
end,
}
package.loaded[...] = Environment
Branched, ArgumentTuple, Overload, Overloadable, Table = ast.Branched, ast.ArgumentTuple, ast.Overload, ast.abstract.Overloadable, ast.Table
return Environment

28
ast/Flush.lua Normal file
View file

@ -0,0 +1,28 @@
local ast = require("ast")
local Nil
local event_manager = require("state.event_manager")
local Flush = ast.abstract.Node {
type = "flush",
init = function(self) end,
_hash = function(self)
return "flush"
end,
_format = function(self)
return "\n"
end,
_eval = function(self, state)
event_manager:flush(state)
return Nil:new()
end
}
package.loaded[...] = Flush
Nil = ast.Nil
return Flush

78
ast/Function.lua Normal file
View file

@ -0,0 +1,78 @@
-- note: functions only appear in non-evaluated nodes! once evaluated, they always become closures
local ast = require("ast")
local Overloadable = ast.abstract.Overloadable
local Closure, ReturnBoundary
local operator_priority = require("common").operator_priority
local Function
Function = Overloadable {
type = "function",
parameters = nil, -- ParameterTuple
expression = nil,
format_priority = operator_priority["$_"],
exports = nil, -- { [sym] = exp, ... }, exctracted from expression during :prepare
init = function(self, parameters, expression, exports)
self.parameters = parameters
self.expression = ReturnBoundary:new(expression)
self.exports = exports or {}
end,
_format = function(self, ...)
if self.parameters.assignment then
return "$"..self.parameters:format(...).."; "..self.expression:format_right(...)
else
return "$"..self.parameters:format(...).." "..self.expression:format_right(...)
end
end,
traverse = function(self, fn, ...)
fn(self.parameters, ...)
fn(self.expression, ...)
end,
compatible_with_arguments = function(self, state, args)
return args:match_parameter_tuple(state, self.parameters)
end,
format_parameters = function(self, state)
return self.parameters:format(state)
end,
call_compatible = function(self, state, args)
state.scope:push()
args:bind_parameter_tuple(state, self.parameters)
local exp = self.expression:eval_resumable(state)
state.scope:pop()
-- reminder: don't do any additionnal processing here as that won't be executed when resuming self.expression
-- instead wrap it in some additional node, like our friend ReturnBoundary
return exp
end,
_eval = function(self, state)
return Closure:new(Function:new(self.parameters:eval(state), self.expression, self.exports), state)
end,
_prepare = function(self, state)
state.scope:push_export() -- recreate scope context that will be created by closure
state.scope:push()
self.parameters:prepare(state)
self.expression:prepare(state)
state.scope:pop()
self.exports = state.scope:capture():list_exported(state)
state.scope:pop()
end,
}
package.loaded[...] = Function
Closure, ReturnBoundary = ast.Closure, ast.ReturnBoundary
return Function

45
ast/FunctionParameter.lua Normal file
View file

@ -0,0 +1,45 @@
local ast = require("ast")
local operator_priority = require("common").operator_priority
local FunctionParameter
FunctionParameter = ast.abstract.Node {
type = "function parameter",
identifier = nil,
default = nil, -- can be nil
type_check = nil, -- can be nil
init = function(self, identifier, default, type_check)
self.identifier = identifier
self.default = default
self.type_check = type_check
if default then
self.format_priority = operator_priority["_=_"]
elseif type_check then -- type_check has higher prio than assignment in any case
self.format_priority = operator_priority["_::_"]
end
end,
_format = function(self, state, prio, ...)
local s = self.identifier:format(state, prio, ...)
if self.type_check then
s = s .. "::" .. self.type_check:format_right(state, operator_priority["_::_"], ...)
end
if self.default then
s = s .. "=" .. self.default:format_right(state, operator_priority["_=_"], ...)
end
return s
end,
traverse = function(self, fn, ...)
fn(self.identifier, ...)
if self.default then fn(self.default, ...) end
if self.type_check then fn(self.type_check, ...) end
end,
_eval = function(self, state)
return FunctionParameter:new(self.identifier, self.default, self.type_check and self.type_check:eval(state))
end
}
return FunctionParameter

44
ast/Identifier.lua Normal file
View file

@ -0,0 +1,44 @@
local ast = require("ast")
local Symbol, String
local Identifier
Identifier = ast.abstract.Node {
type = "identifier",
name = nil,
init = function(self, name)
self.name = name
end,
_hash = function(self)
return ("identifier<%q>"):format(self.name)
end,
_format = function(self)
return self.name
end,
_eval = function(self, state)
return state.scope:get(self)
end,
to_string = function(self)
return String:new(self.name)
end,
to_symbol = function(self, modifiers)
return Symbol:new(self.name, modifiers)
end,
_prepare = function(self, state)
if state.scope:defined(self) then
state.scope:get(self):prepare(state)
end
end
}
package.loaded[...] = Identifier
Symbol, String = ast.Symbol, ast.String
return Identifier

81
ast/List.lua Normal file
View file

@ -0,0 +1,81 @@
local ast = require("ast")
local Branched, Tuple
local operator_priority = require("common").operator_priority
local List
List = ast.abstract.Runtime {
type = "list",
format_priority = operator_priority["*_"],
-- note: yeah technically this isn't mutable, only .branched is
-- note: this a Branched of Tuple, and we *will* forcefully mutate the tuples, so make sure to not disseminate any reference to them outside the List
-- unless you want rumors about mutable tuples to spread
branched = nil,
init = function(self, state, from_tuple)
from_tuple = from_tuple or Tuple:new()
self.branched = Branched:new(state, from_tuple:copy())
end,
_format = function(self, ...)
return "*"..self.branched:format_right(...)
end,
traverse = function(self, fn, ...)
fn(self.branched, ...)
end,
-- List is always created from an evaluated Tuple, so no need to _eval here
-- create copy of the list in branch if not here
-- do this before any mutation
-- return the tuple for the current branch
_prepare_branch = function(self, state)
if not self.branched:in_branch(state) then
self.branched:set(state, self.branched:get(state):copy())
end
return self.branched:get(state)
end,
len = function(self, state)
return #self.branched:get(state).list
end,
iter = function(self, state)
return ipairs(self.branched:get(state).list)
end,
get = function(self, state, index)
local list = self.branched:get(state)
if index < 0 then index = #list.list + 1 + index end
if index > #list.list or index == 0 then error("list index out of bounds") end
return list.list[index]
end,
set = function(self, state, index, val)
local list = self:_prepare_branch(state)
if index < 0 then index = #list.list + 1 + index end
if index > #list.list or index == 0 then error("list index out of bounds") end
list.list[index] = val
end,
insert = function(self, state, val)
local l = self:_prepare_branch(state)
table.insert(l.list, val)
end,
remove = function(self, state)
local l = self:_prepare_branch(state)
table.remove(l.list)
end,
to_tuple = function(self, state)
return self.branched:get(state):copy()
end,
to_lua = function(self, state)
return self.branched:get(state):to_lua(state)
end,
}
package.loaded[...] = List
Branched, Tuple = ast.Branched, ast.Tuple
return List

63
ast/LuaFunction.lua Normal file
View file

@ -0,0 +1,63 @@
local ast = require("ast")
local Overloadable = ast.abstract.Overloadable
local operator_priority = require("common").operator_priority
local LuaFunction
LuaFunction = ast.abstract.Runtime(Overloadable) {
type = "lua function",
parameters = nil, -- ParameterTuple
func = nil, -- lua function
format_priority = operator_priority["$_"],
init = function(self, parameters, func)
self.parameters = parameters
self.func = func
end,
traverse = function(self, fn, ...)
fn(self.parameters, ...)
end,
_format = function(self, ...)
if self.parameters.assignment then
return "$"..self.parameters:format(...).."; <lua function>"
else
return "$"..self.parameters:format(...).." <lua function>"
end
end,
compatible_with_arguments = function(self, state, args)
return args:match_parameter_tuple(state, self.parameters)
end,
format_parameters = function(self, state)
return self.parameters:format(state)
end,
call_compatible = function(self, state, args)
local lua_args = { state }
state.scope:push()
args:bind_parameter_tuple(state, self.parameters)
for _, param in ipairs(self.parameters.list) do
table.insert(lua_args, state.scope:get(param.identifier))
end
state.scope:pop()
local r = self.func(table.unpack(lua_args))
assert(r, "lua function returned no value")
return r
end,
_eval = function(self, state)
return LuaFunction:new(self.parameters:eval(state), self.func)
end,
to_lua = function(self, state)
return self.func
end,
}
return LuaFunction

20
ast/Nil.lua Normal file
View file

@ -0,0 +1,20 @@
local ast = require("ast")
return ast.abstract.Node {
type = "nil",
_evaluated = true, -- no evaluation needed
init = function(self) end,
_hash = function(self)
return "nil"
end,
_format = function(self)
return "()"
end,
to_lua = function(self, state) return nil end,
truthy = function(self) return false end
}

25
ast/Number.lua Normal file
View file

@ -0,0 +1,25 @@
local ast = require("ast")
local Number
Number = ast.abstract.Node {
type = "number",
_evaluated = true, -- no evaluation needed
number = nil,
init = function(self, number)
self.number = number
end,
_hash = function(self)
return ("number<%s>"):format(self.number)
end,
_format = function(self)
return tostring(self.number)
end,
to_lua = function(self, state) return self.number end,
}
return Number

62
ast/Overload.lua Normal file
View file

@ -0,0 +1,62 @@
local ast = require("ast")
local Overload
Overload = ast.abstract.Node {
type = "overload",
_evaluated = true,
list = nil,
init = function(self, ...)
self.list = { ... }
end,
insert = function(self, val) -- only for construction
table.insert(self.list, val)
end,
_format = function(self, ...)
local s = "overload<"
for i, e in ipairs(self.list) do
s = s .. e:format(...)
if i < #self.list then s = s .. ", " end
end
return s..">"
end,
traverse = function(self, fn, ...)
for _, e in ipairs(self.list) do
fn(e, ...)
end
end,
call = function(self, state, args)
local failure = {} -- list of failure messages (kept until we find the first success)
local success, success_specificity, success_secondary_specificity = nil, -1, -1
-- some might think that iterating a list for every function call is a terrible idea, but that list has a fixed number of elements, so big O notation says suck it up
for _, fn in ipairs(self.list) do
local specificity, secondary_specificity = fn:compatible_with_arguments(state, args)
if specificity then
if specificity > success_specificity then
success, success_specificity, success_secondary_specificity = fn, specificity, secondary_specificity
elseif specificity == success_specificity then
if secondary_specificity > success_secondary_specificity then
success, success_specificity, success_secondary_specificity = fn, specificity, secondary_specificity
elseif secondary_specificity == success_secondary_specificity then
error(("more than one function match %s, matching functions were at least (specificity %s.%s):\n\t• %s\n\t• %s"):format(args:format(state), specificity, secondary_specificity, fn:format_parameters(state), success:format_parameters(state)), 0)
end
end
-- no need to add error message for less specific function since we already should have at least one success
elseif not success then
table.insert(failure, fn:format_parameters(state) .. ": " .. secondary_specificity)
end
end
if success then
return success:call_compatible(state, args)
else
-- error
error(("no function match %s, possible functions were:\n\t• %s"):format(args:format(state), table.concat(failure, "\n\t• ")), 0)
end
end
}
return Overload

25
ast/Pair.lua Normal file
View file

@ -0,0 +1,25 @@
local ast = require("ast")
local operator_priority = require("common").operator_priority
return ast.abstract.Runtime {
type = "pair",
name = nil,
value = nil,
format_priority = operator_priority["_:_"],
init = function(self, name, value)
self.name = name
self.value = value
end,
traverse = function(self, fn, ...)
fn(self.name, ...)
fn(self.value, ...)
end,
_format = function(self, ...)
return ("%s:%s"):format(self.name:format(...), self.value:format(...))
end,
}

67
ast/ParameterTuple.lua Normal file
View file

@ -0,0 +1,67 @@
local ast = require("ast")
local operator_priority = require("common").operator_priority
local ParameterTuple
ParameterTuple = ast.abstract.Node {
type = "parameter tuple",
assignment = false,
list = nil,
min_arity = 0,
max_arity = 0,
eval_depth = 0, -- scope deth where this parametertuple was evaluated, used as secondary specificity
init = function(self, ...)
self.list = {...}
end,
insert = function(self, val) -- only for construction
assert(not self.assignment, "can't add new parameters after assignment parameter was added")
table.insert(self.list, val)
self.max_arity = self.max_arity + 1
if not val.default then
self.min_arity = self.min_arity + 1
end
end,
insert_assignment = function(self, val) -- only for construction
self:insert(val)
self.assignment = true
self.format_priority = operator_priority["_=_"]
end,
_format = function(self, state, prio, ...)
local l = {}
for i, e in ipairs(self.list) do
if i < self.max_arity or not self.assignment then
table.insert(l, e:format(state, operator_priority["_,_"], ...))
end
end
local s = ("(%s)"):format(table.concat(l, ", "))
if self.assignment then
s = s .. (" = %s"):format(self.list[#self.list]:format_right(state, operator_priority["_=_"], ...))
end
return s
end,
traverse = function(self, fn, ...)
for _, e in ipairs(self.list) do
fn(e, ...)
end
end,
_eval = function(self, state)
local r = ParameterTuple:new()
for i, param in ipairs(self.list) do
if i < self.max_arity or not self.assignment then
r:insert(param:eval(state))
else
r:insert_assignment(param:eval(state))
end
end
r.eval_depth = state.scope:depth()
return r
end
}
return ParameterTuple

34
ast/Quote.lua Normal file
View file

@ -0,0 +1,34 @@
-- prevent an expression from being immediately evaluated, and instead only evaluate it when the node is explicitely called
-- it can be used to evaluate the expression on demand, as if the quote call AST was simply replaced by the unevaluated associated expression AST (like a macro)
-- keep in mind that this thus bypass any scoping rule, closure, etc.
--
-- used for infix operators where the evaluation of the right term depends of the left one (lazy boolean operators, conditionals, etc.)
local ast = require("ast")
local Quote
Quote = ast.abstract.Node {
type = "quote",
expression = nil,
init = function(self, expression)
self.expression = expression
self.format_priority = expression.format_priority
end,
_format = function(self, ...)
return self.expression:format(...) -- Quote is generated transparently by operators
end,
traverse = function(self, fn, ...)
fn(self.expression, ...)
end,
call = function(self, state, args)
assert(args.arity == 0, "Quote! does not accept arguments")
return self.expression:eval(state)
end
}
return Quote

60
ast/Resumable.lua Normal file
View file

@ -0,0 +1,60 @@
local ast = require("ast")
local Table
local resumable_manager
local Resumable
Resumable = ast.abstract.Runtime {
type = "resumable",
resuming = false,
expression = nil,
scope = nil,
data = nil,
init = function(self, state, expression, scope, data)
self.expression = expression
self.scope = scope
self.data = data or Table:new(state)
end,
_format = function(self)
return "<resumable>"
end,
traverse = function(self, fn, ...)
fn(self.expression, ...)
fn(self.data, ...)
fn(self.scope, ...)
end,
-- returns a copy with the data copied
capture = function(self, state)
return Resumable:new(state, self.expression, self.scope, self.data:copy(state))
end,
-- resume from this resumable
call = function(self, state, args)
assert(args.arity == 0, "Resumable! does not accept arguments")
state.scope:push(self.scope)
local resuming = self:capture(state)
resuming.resuming = true
resumable_manager:push(state, resuming)
local r = self.expression:eval(state)
resumable_manager:pop(state)
state.scope:pop()
return r
end,
}
package.loaded[...] = Resumable
Table = ast.Table
resumable_manager = require("state.resumable_manager")
return Resumable

44
ast/ResumeParentFunction.lua Normal file
View file

@ -0,0 +1,44 @@
-- intended to be wrapped in a Function, so that when resuming from the function, will keep resuming to where the function was called from
-- used in Choices to resume back from where the event was flushed
-- note: when resuming, the return value will be discarded, instead returning what the parent function will return
local ast = require("ast")
local ArgumentTuple
local resumable_manager
local ResumeParentFunction = ast.abstract.Node {
type = "resume parent function",
expression = nil,
init = function(self, expression)
self.expression = expression
self.format_priority = expression.format_priority
end,
_format = function(self, ...)
return self.expression:format(...)
end,
traverse = function(self, fn, ...)
fn(self.expression, ...)
end,
_eval = function(self, state)
if resumable_manager:resuming(state, self) then
self.expression:eval(state)
return resumable_manager:get_data(state, self):call(state, ArgumentTuple:new())
else
resumable_manager:set_data(state, self, resumable_manager:capture(state, 1))
return self.expression:eval(state)
end
end
}
package.loaded[...] = ResumeParentFunction
ArgumentTuple = ast.ArgumentTuple
resumable_manager = require("state.resumable_manager")
return ResumeParentFunction

33
ast/Return.lua Normal file
View file

@ -0,0 +1,33 @@
local ast = require("ast")
local operator_priority = require("common").operator_priority
local Return
Return = ast.abstract.Node {
type = "return",
expression = nil,
format_priority = operator_priority["@_"],
init = function(self, expression)
self.expression = expression
end,
_format = function(self, ...)
return ("@%s"):format(self.expression:format_right(...))
end,
traverse = function(self, fn, ...)
fn(self.expression, ...)
end,
_eval = function(self, state)
return Return:new(self.expression:eval(state))
end,
to_lua = function(self, state)
return self.expression:to_lua(state)
end
}
return Return

37
ast/ReturnBoundary.lua Normal file
View file

@ -0,0 +1,37 @@
-- used stop propagating Return when leaving functions
local ast = require("ast")
local Return
local ReturnBoundary = ast.abstract.Node {
type = "return boundary",
expression = nil,
init = function(self, expression)
self.expression = expression
self.format_priority = self.expression.format_priority
end,
_format = function(self, ...)
return self.expression:format(...)
end,
traverse = function(self, fn, ...)
fn(self.expression, ...)
end,
_eval = function(self, state)
local exp = self.expression:eval(state)
if Return:is(exp) then
return exp.expression
else
return exp
end
end
}
package.loaded[...] = ReturnBoundary
Return = ast.Return
return ReturnBoundary

34
ast/String.lua Normal file
View file

@ -0,0 +1,34 @@
local ast = require("ast")
local Identifier
local String = ast.abstract.Node {
type = "string",
_evaluated = true, -- no evaluation needed
string = nil,
init = function(self, str)
self.string = str
end,
_hash = function(self)
return ("string<%q>"):format(self.string)
end,
_format = function(self)
return ("%q"):format(self.string)
end,
to_lua = function(self, state)
return self.string
end,
to_identifier = function(self)
return Identifier:new(self.string)
end
}
package.loaded[...] = String
Identifier = ast.Identifier
return String

53
ast/StringInterpolation.lua Normal file
View file

@ -0,0 +1,53 @@
local ast = require("ast")
local String
local StringInterpolation = ast.abstract.Node {
type = "string interpolation",
list = nil,
init = function(self, ...)
self.list = {...}
end,
insert = function(self, val) -- only for construction
table.insert(self.list, val)
end,
traverse = function(self, fn, ...)
for _, e in ipairs(self.list) do
fn(e, ...)
end
end,
_format = function(self, ...)
local l = {}
for _, e in ipairs(self.list) do
if String:is(e) then
local t = e.string:gsub("\\", "\\\\"):gsub("\n", "\\n"):gsub("\t", "\\t"):gsub("\"", "\\\"")
table.insert(l, t)
else
table.insert(l, ("{%s}"):format(e:format(...)))
end
end
return ("\"%s\""):format(table.concat(l))
end,
_eval = function(self, state)
local t = {}
for _, e in ipairs(self.list) do
local r = e:eval(state)
if String:is(r) then
r = r.string
else
r = r:format(state)
end
table.insert(t, r)
end
return String:new(table.concat(t))
end
}
package.loaded[...] = StringInterpolation
String = ast.String
return StringInterpolation

121
ast/Struct.lua Normal file
View file

@ -0,0 +1,121 @@
local ast = require("ast")
local Pair, Number, Nil
local operator_priority = require("common").operator_priority
local Struct
local TupleToStruct = ast.abstract.Node {
type = "tuple to struct",
tuple = nil,
init = function(self, tuple)
self.tuple = tuple
end,
traverse = function(self, fn, ...)
fn(self.tuple, ...)
end,
_format = function(self, ...)
return self.tuple:format(...):gsub("^%[", "{"):gsub("%]$", "}")
end,
_eval = function(self, state)
local t = Struct:new()
for i, e in ipairs(self.tuple.list) do
if Pair:is(e) then
t:set(e.name, e.value)
else
t:set(Number:new(i), e)
end
end
return t
end
}
Struct = ast.abstract.Runtime {
type = "struct",
table = nil,
init = function(self)
self.table = {}
end,
set = function(self, key, value) -- only for construction
self.table[key:hash()] = { key, value }
end,
include = function(self, other) -- only for construction
for _, e in pairs(other.table) do
self:set(e[1], e[2])
end
end,
copy = function(self)
local s = Struct:new()
for _, e in pairs(self.table) do
s:set(e[1], e[2])
end
return s
end,
-- build from (non-evaluated) tuple
-- results needs to be evaluated
from_tuple = function(self, tuple)
return TupleToStruct:new(tuple)
end,
_format = function(self, state, prio, ...)
local l = {}
for _, e in pairs(self.table) do
-- _:_ has higher priority than _,_
table.insert(l, e[1]:format(state, operator_priority["_:_"], ...)..":"..e[2]:format_right(state, operator_priority["_:_"], ...))
end
return ("{%s}"):format(table.concat(l, ", "))
end,
traverse = function(self, fn, ...)
for _, e in pairs(self.table) do
fn(e[1], ...)
fn(e[2], ...)
end
end,
-- need to redefine hash to include a table.sort as pairs() in :traverse is non-deterministic
_hash = function(self)
local t = {}
for _, e in pairs(self.table) do
table.insert(t, ("%s;%s"):format(e[1]:hash(), e[2]:hash()))
end
table.sort(t)
return ("%s<%s>"):format(self.type, table.concat(t, ";"))
end,
-- regarding eval: Struct is built from TupleToStruct function call which already eval, so every Struct should be fully evaluated
to_lua = function(self, state)
local l = {}
for _, e in ipairs(self.table) do
l[e[1]:to_lua(state)] = e[2]:to_lua(state)
end
return l
end,
get = function(self, key)
local hash = key:hash()
if self.table[hash] then
return self.table[hash][2]
else
return Nil:new()
end
end,
has = function(self, key)
local hash = key:hash()
return not not self.table[hash]
end
}
package.loaded[...] = Struct
Pair, Number, Nil = ast.Pair, ast.Number, ast.Nil
return Struct

78
ast/Symbol.lua Normal file
View file

@ -0,0 +1,78 @@
local ast = require("ast")
local Identifier, String
local operator_priority = require("common").operator_priority
local Symbol
Symbol = ast.abstract.Node {
type = "symbol",
string = nil,
constant = nil, -- bool
type_check = nil, -- exp
exported = nil, -- bool
persistent = nil, -- bool, imply exported
confined_to_branch = nil, -- bool
init = function(self, str, modifiers)
modifiers = modifiers or {}
self.string = str
self.constant = modifiers.constant
self.persistent = modifiers.persistent
self.type_check = modifiers.type_check
self.confined_to_branch = modifiers.confined_to_branch
self.exported = modifiers.exported or modifiers.persistent
if self.type_check then
self.format_priority = operator_priority["_::_"]
end
end,
_eval = function(self, state)
return Symbol:new(self.string, {
constant = self.constant,
persistent = self.persistent,
type_check = self.type_check and self.type_check:eval(state),
confined_to_branch = self.confined_to_branch,
exported = self.exported
})
end,
_hash = function(self)
return ("symbol<%q>"):format(self.string)
end,
_format = function(self, state, prio, ...)
local s = ":"
if self.constant then
s = s .. ":"
end
if self.persistent then
s = s .. "&"
end
if self.exported then
s = s .. "@"
end
s = s .. self.string
if self.type_check then
s = s .. "::" .. self.type_check:format_right(state, operator_priority["_::_"], ...)
end
return s
end,
to_lua = function(self, state)
return self.string
end,
to_identifier = function(self)
return Identifier:new(self.string)
end,
to_string = function(self)
return String:new(self.string)
end
}
package.loaded[...] = Symbol
Identifier, String = ast.Identifier, ast.String
return Symbol

85
ast/Table.lua Normal file
View file

@ -0,0 +1,85 @@
local ast = require("ast")
local Branched, Struct, Nil = ast.Branched, ast.Struct, ast.Nil
local operator_priority = require("common").operator_priority
local Table
Table = ast.abstract.Runtime {
type = "table",
format_priority = operator_priority["*_"],
-- note: technically this isn't mutable, only .branched is
-- note: this a Branched of Struct, and we *will* forcefully mutate the tuples, so make sure to not disseminate any reference to them outside the Table
-- unless you want rumors about mutable structs to spread
branched = nil,
init = function(self, state, from_struct)
from_struct = from_struct or Struct:new()
self.branched = Branched:new(state, from_struct:copy())
end,
_format = function(self, ...)
return "*"..self.branched:format_right(...)
end,
traverse = function(self, fn, ...)
fn(self.branched, ...)
end,
-- Table is always created from an evaluated Struct, so no need to _eval here
-- create copy of the table in branch if not here
-- do this before any mutation
-- return the struct for the current branch
_prepare_branch = function(self, state)
if not self.branched:in_branch(state) then
self.branched:set(state, self.branched:get(state):copy())
end
return self.branched:get(state)
end,
get = function(self, state, key)
local s = self.branched:get(state)
return s:get(key)
end,
set = function(self, state, key, val)
local s = self:_prepare_branch(state)
local hash = key:hash()
if Nil:is(val) then
s.table[hash] = nil
else
s.table[hash] = { key, val }
end
end,
has = function(self, state, key)
local s = self.branched:get(state)
return s:has(key)
end,
iter = function(self, state)
local t, h = self.branched:get(state).table, nil
return function()
local e
h, e = next(t, h)
if h == nil then return nil
else return e[1], e[2]
end
end
end,
to_struct = function(self, state)
return self.branched:get(state):copy()
end,
to_lua = function(self, state)
return self.branched:get(state):to_lua(state)
end,
copy = function(self, state)
return Table:new(state, self:to_struct(state))
end
}
package.loaded[...] = Table
Branched, Struct, Nil = ast.Branched, ast.Struct, ast.Nil
return Table

36
ast/Text.lua Normal file
View file

@ -0,0 +1,36 @@
local ast = require("ast")
local AutoCall, Event, Runtime = ast.abstract.AutoCall, ast.abstract.Event, ast.abstract.Runtime
return Runtime(AutoCall, Event) {
type = "text",
list = nil, -- { { String, tag Table }, ... }
init = function(self)
self.list = {}
end,
insert = function(self, str, tags) -- only for construction
table.insert(self.list, { str, tags })
end,
traverse = function(self, fn, ...)
for _, e in ipairs(self.list) do
fn(e[1], ...)
fn(e[2], ...)
end
end,
_format = function(self, ...)
local t = {}
for _, e in ipairs(self.list) do
table.insert(t, ("%s%s"):format(e[2]:format(...), e[1]:format(...)))
end
return ("| %s |"):format(table.concat(t, " "))
end,
-- Text comes from TextInterpolation which already evals the contents
to_event_data = function(self)
return self
end
}

59
ast/TextInterpolation.lua Normal file
View file

@ -0,0 +1,59 @@
local ast = require("ast")
local Text, String
local tag_manager = require("state.tag_manager")
local TextInterpolation = ast.abstract.Node {
type = "text interpolation",
list = nil,
init = function(self, ...)
self.list = {...}
end,
insert = function(self, val) -- only for construction
table.insert(self.list, val)
end,
traverse = function(self, fn, ...)
for _, e in ipairs(self.list) do
fn(e, ...)
end
end,
_format = function(self, ...)
local l = {}
for _, e in ipairs(self.list) do
if String:is(e) then
local t = e.string:gsub("\\", "\\\\"):gsub("\n", "\\n"):gsub("\t", "\\t"):gsub("\"", "\\\"")
table.insert(l, t)
else
table.insert(l, ("{%s}"):format(e:format(...)))
end
end
return ("| %s|"):format(table.concat(l))
end,
_eval = function(self, state)
local t = Text:new()
local tags = tag_manager:get(state)
for _, e in ipairs(self.list) do
local r = e:eval(state)
if String:is(r) then
t:insert(r, tags)
elseif Text:is(r) then
for _, v in ipairs(r.list) do
t:insert(v[1], v[2])
end
else
t:insert(String:new(r:format(state)), tags)
end
end
return t
end,
}
package.loaded[...] = TextInterpolation
Text, String = ast.Text, ast.String
return TextInterpolation

66
ast/Tuple.lua Normal file
View file

@ -0,0 +1,66 @@
local ast = require("ast")
local operator_priority = require("common").operator_priority
local Tuple
Tuple = ast.abstract.Node {
type = "tuple",
explicit = true, -- false for implicitely created tuples, e.g. 1,2,3 without the brackets []
list = nil,
init = function(self, ...)
self.list = { ... }
end,
insert = function(self, val) -- only for construction
table.insert(self.list, val)
end,
_format = function(self, state, prio, ...)
local l = {}
for _, e in ipairs(self.list) do
table.insert(l, e:format(state, operator_priority["_,_"], ...))
end
return ("[%s]"):format(table.concat(l, ", "))
end,
traverse = function(self, fn, ...)
for _, e in ipairs(self.list) do
fn(e, ...)
end
end,
_eval = function(self, state)
local t = Tuple:new()
for _, e in ipairs(self.list) do
t:insert(e:eval(state))
end
if not self.explicit then
t.explicit = false
end
return t
end,
copy = function(self)
local t = Tuple:new()
for _, e in ipairs(self.list) do
t:insert(e)
end
return t
end,
to_lua = function(self, state)
local l = {}
for _, e in ipairs(self.list) do
table.insert(l, e:to_lua(state))
end
return l
end,
get = function(self, index)
if index < 0 then index = #self.list + 1 + index end
if index > #self.list or index == 0 then error("tuple index out of bounds") end
return self.list[index]
end
}
return Tuple

24
ast/Typed.lua Normal file
View file

@ -0,0 +1,24 @@
local ast = require("ast")
local operator_priority = require("common").operator_priority
return ast.abstract.Runtime {
type = "typed",
expression = nil,
type_expression = nil,
init = function(self, type, expression)
self.type_expression = type
self.expression = expression
end,
_format = function(self, state, prio, ...)
return ("type(%s, %s)"):format(self.type_expression:format(state, operator_priority["_,_"], ...), self.expression:format_right(state, operator_priority["_,_"], ...))
end,
traverse = function(self, fn, ...)
fn(self.type_expression, ...)
fn(self.expression, ...)
end
}

8
ast/abstract/AutoCall.lua Normal file
View file

@ -0,0 +1,8 @@
-- called automatically when returned by one of the expression in a block
local ast = require("ast")
return ast.abstract.Node {
type = "auto call",
init = false
}

22
ast/abstract/Event.lua Normal file
View file

@ -0,0 +1,22 @@
-- for nodes that can be written to the event buffer
local ast = require("ast")
return ast.abstract.Node {
type = "event",
init = false,
-- returns value that will be yielded by the whole event buffer data on flush
-- by default a list of what is returned by :to_event_data for each event of the buffer
build_event_data = function(self, state, event_buffer)
local l = {}
for _, event in event_buffer:iter(state) do
table.insert(l, event:to_event_data(state))
end
return l
end,
to_event_data = function(self, state) error("unimplemented") end,
-- post_flush_callback(self, state, event_buffer, event_data)
post_flush_callback = false
}

282
ast/abstract/Node.lua Normal file
View file

@ -0,0 +1,282 @@
local class = require("class")
local fmt = require("common").fmt
local binser = require("lib.binser")
-- NODES SHOULD BE IMMUTABLE AFTER CREATION IF POSSIBLE!
-- i don't think i actually rely on this behavior for anything but it makes me feel better about life in general
-- (well, unless node.mutable == true, in which case go ahead and break my little heart)
-- UPDATE: i actually assumed nodes to be immutable by default in a lot of places now, thank you past me, it did indeed make me feel better about life in general
-- reminder: when requiring AST nodes somewhere, try to do it at the end of the file. and if you need to require something in this file, do it in the :_i_hate_cycles method.
-- i've had enough headaches with cyclics references and nodes required several times...
local uuid = require("common").uuid
local State, Runtime
local resumable_manager
local custom_call_identifier
local context_max_length = 50
local function cutoff_text(str)
if str:match("\n") or utf8.len(str) > context_max_length then
local cut_pos = math.min((str:match("()\n") or math.huge)-1, (utf8.offset(str, context_max_length, 1) or math.huge)-1)
str = str:sub(1, cut_pos) .. "…"
end
return str
end
local function format_error(state, node, message)
local ctx = cutoff_text(node:format(state)) -- get some context code around error
return fmt("%{red}%s%{reset}\n\t↳ from %{underline}%s%{reset} in %s: %{dim}%s", message, node.source, node.type, ctx)
end
-- traverse helpers
local traverse
traverse = {
set_source = function(self, source)
self:set_source(source)
end,
prepare = function(self, state)
self:prepare(state)
end,
merge = function(self, state, cache)
self:merge(state, cache)
end,
hash = function(self, t)
table.insert(t, self:hash())
end
}
local Node
Node = class {
type = "node",
source = "?",
mutable = false,
-- abstract class
-- must be redefined
init = false,
-- set the source of this node and its children (unless a source is already set)
-- to be preferably used during construction only
set_source = function(self, source)
local str_source = tostring(source)
if self.source == "?" then
self.source = str_source
self:traverse(traverse.set_source, str_source)
end
return self
end,
-- call function callback with args ... on the children Nodes of this Node
-- by default, assumes no children Nodes
-- you will want to redefine this for nodes with children nodes
-- (note: when calling, remember that cycles are common place in the AST, so stay safe use a cache)
traverse = function(self, callback, ...) end,
-- returns new AST
-- whatever this function returned is assumed to be already evaluated
-- the actual evaluation is done in _eval
eval = function(self, state)
if self._evaluated then return self end
local s, r = pcall(self._eval, self, state)
if s then
r._evaluated = true
return r
else
error(format_error(state, self, r), 0)
end
end,
_evaluated = false, -- if true, node is assumed to be already evaluated and :eval will be the identity function
-- evaluate this node and return the result
-- by default assume the node can't be evaluated further and return itself; redefine for everything else, probably
-- THIS SHOULD NOT MUTATE THE CURRENT NODE; create and return a new Node instead! (even if node is mutable)
_eval = function(self, state)
return self
end,
-- prepare the AST after parsing and before evaluation
-- this behave like a cached :traverse through the AST, except this keeps track of the scope stack
-- i.e. when :prepare is called on a node, it should be in a similar scope stack context as will be when it will be evaluated
-- used to predefine exported variables and other compile-time variable handling
-- note: the state here is a temporary state only used during the prepare step
-- the actual preparation is done in _prepare
-- (this can mutate the node as needed and is automatically called after each parse)
prepare = function(self, state)
assert(not Runtime:issub(self), ("can't prepare a %s node that should only exist at runtime"):format(self.type))
state = state or State:new()
if self._prepared then return end
local s, r = pcall(self._prepare, self, state)
if s then
self._prepared = true
else
error(format_error(state, self, r), 0)
end
end,
_prepared = false, -- indicate that the node was prepared and :prepare should nop
-- prepare this node. can mutate the node (considered to be part of construction).
_prepare = function(self, state)
self:traverse(traverse.prepare, state)
end,
-- same as eval, but make the evaluated expression as a resume boundary
-- i.e. if a checkpoint is defined somewhere in this eval, it will start back from this node eval when resuming
eval_resumable = function(self, state)
return resumable_manager:eval(state, self)
end,
-- set the current resume data for this node
-- (relevant inside :eval)
set_resume_data = function(self, state, data)
resumable_manager:set_data(state, self, data)
end,
-- get the current resume data for this node
get_resume_data = function(self, state)
return resumable_manager:get_data(state, self)
end,
-- returns true if the current node is in a resuming state
-- (relevant inside :eval)
resuming = function(self, state)
return resumable_manager:resuming(state, self)
end,
-- return result AST
-- arg is a ArgumentTuple node (already evaluated)
-- redefine if relevant
call = function(self, state, arg)
if state.scope:defined(custom_call_identifier) then
local custom_call = custom_call_identifier:eval(state)
return custom_call:call(state, arg:with_first_argument(self))
else
error("trying to call a "..self.type..": "..self:format(state))
end
end,
-- merge any changes back into the main branch
-- cache is a table indicating nodes when the merge has already been triggered { [node] = true, ... }
-- (just give an empty table on the initial call)
-- redefine :_merge if needed, not this
merge = function(self, state, cache)
if not cache[self] then
cache[self] = true
self:_merge(state, cache)
self:traverse(traverse.merge, state, cache)
end
end,
_merge = function(self, state, cache) end,
-- return string that uniquely represent this node
-- the actual hash is computed in :_hash, don't redefine :hash directly
-- note: if the node is mutable, this will return a UUID instead of calling :_hash
hash = function(self)
if not self._hash_cache then
if self.mutable then
self._hash_cache = uuid()
else
self._hash_cache = self:_hash()
end
end
return self._hash_cache
end,
_hash_cache = nil, -- cached hash
-- return string that uniquely represent this node
-- by default, build a "node type<children node hash;...>" representation using :traverse
-- you may want to redefine this for base types and other nodes with discriminating info that's not in children nodes.
-- also beware if :traverse uses pairs() or any other non-deterministic function, it'd be nice if this was properly bijective...
-- (no need to redefine for mutable nodes, since an uuid is used instead)
_hash = function(self)
local t = {}
self:traverse(traverse.hash, t)
return ("%s<%s>"):format(self.type, table.concat(t, ";"))
end,
-- return a pretty string representation of the node.
-- for non-runtime nodes (what was generated by a parse without any evaluation), this should return valid Anselme code that is functionnally equivalent to the parsed code. note that it currently does not preserve comment.
-- redefine _format, not this - note that _format is a mandary method for all nodes.
-- state is optional and should only be relevant for runtime nodes; if specified, only show what is relevant for the current branch.
-- indentation_level and parent_priority are optional value that respectively keep track in nester :format calls of the indentation level (number) and parent operator priority (number); if the node has a strictly lower priority than the parent node, parentheses will be added
-- also remember that execution is done left-to-right, so in case of priority equality, all is fine if the term appear left of the operator, but parentheses will need to be added if the term is right of the operator - so make sure to call :format_right for such cases
-- (:format is not cached as even immutable nodes may contain mutable children)
format = function(self, state, parent_priority, indentation_level)
indentation_level = indentation_level or 0
parent_priority = parent_priority or 0
local s = self:_format(state, self.format_priority, indentation_level)
if self.format_priority < parent_priority then
s = ("(%s)"):format(s)
end
local indentation = ("\t"):rep(indentation_level)
s = s:gsub("\n", "\n"..indentation)
return s
end,
-- same as :format, but should be called only for nodes right of the current operator
format_right = function(self, state, parent_priority, indentation_level)
indentation_level = indentation_level or 0
parent_priority = parent_priority or 0
local s = self:_format(state, self.format_priority, indentation_level)
if self.format_priority <= parent_priority then
s = ("(%s)"):format(s)
end
local indentation = (" "):rep(indentation_level)
s = indentation..s:gsub("\n", "\n"..indentation)
return s
end,
-- redefine this to provide a custom :format. returns a string.
_format = function(self, state, self_priority, identation)
error("format not implemented for "..self.type)
end,
-- priority of the node that will be used in :format to add eventually needed parentheses.
-- should not be modified after object construction!
format_priority = math.huge, -- by default, assumes primary node, i.e. never wrap in parentheses
-- return Lua value
-- this should probably be only called on a Node that is already evaluated
-- redefine if you want, probably only for nodes that are already evaluated
to_lua = function(self, state)
error("cannot convert "..self.type.." to a Lua value")
end,
-- returns truthiness of node
-- redefine for false stuff
truthy = function(self)
return true
end,
-- register the node for serialization on creation
__created = function(self)
if self.init then -- only call on non-abstract node
binser.register(self, self.type)
end
end,
__tostring = function(self) return self:format() end,
-- Node is required by every other AST node, some of which exist in cyclic require loops.
-- Delaying the requires in each node after it is defined is enough to fix it, but not for abstract Nodes, since because we are subclassing each node from
-- them, we need them to be available BEFORE the Node is defined. But Node require several other modules, which themselves require some other AST...
-- The worst thing with this kind of require loop combined with our existing cycle band-aids is that Lua won't error, it will just execute the first node to subclass from Node twice. Which is annoying since now we have several, technically distinct classes representing the same node frolicking around.
-- Thus, any require here that may require other Nodes shall be done here. This method is called in anselme.lua after everything else is required.
_i_hate_cycles = function(self)
local ast = require("ast")
custom_call_identifier = ast.Identifier:new("_!")
Runtime = ast.abstract.Runtime
State = require("state.State")
resumable_manager = require("state.resumable_manager")
end,
_debug_traverse = function(self, level)
level = level or 0
local t = {}
self:traverse(function(v) table.insert(t, v:_debug_ast(level+1)) end)
return ("%s%s:\n%s"):format((" "):rep(level), self.type, table.concat(t, "\n"))
end,
}
return Node

27
ast/abstract/Overloadable.lua Normal file
View file

@ -0,0 +1,27 @@
local ast = require("ast")
return ast.abstract.Node {
type = "overloadable",
init = false,
-- return specificity (number>=0), secondary specificity (number >=0)
-- return false, failure message (string)
compatible_with_arguments = function(self, state, args)
error("not implemented for "..self.type)
end,
-- same as :call, but assumes :compatible_with_arguments was checked before the call
call_compatible = function(self, state, args)
error("not implemented for "..self.type)
end,
-- return string
format_parameters = function(self, state)
return self:format(state)
end,
-- default for :call
call = function(self, state, args)
assert(self:compatible_with_arguments(state, args))
return self:call_compatible(state, args)
end
}

12
ast/abstract/Runtime.lua Normal file
View file

@ -0,0 +1,12 @@
-- indicate a Runtime node: it should not exist in the AST generated by the parser but only as a result of an evaluation or call
-- is assumed to be already evaluated and prepared (will actually error on prepare)
local ast = require("ast")
return ast.abstract.Node {
type = "runtime",
init = false,
_evaluated = true,
_prepared = true
}

13
ast/init.lua Normal file
View file

@ -0,0 +1,13 @@
return setmetatable({
abstract = setmetatable({}, {
__index = function(self, key)
self[key] = require("ast.abstract."..key)
return self[key]
end
})
}, {
__index = function(self, key)
self[key] = require("ast."..key)
return self[key]
end
})

169
class.lua Normal file
View file

@ -0,0 +1,169 @@
--- classtoi v2: finding a sweet spot between classtoi-light and classtoi-heavy
-- aka getlost v2
--
-- usage:
--
-- local class = require("class")
-- local Vehicle = class {
-- type = "vehicle", -- class name, optional
--
-- stability_threshold = 3, -- class variable, also availabe in instances
-- wheel_count = nil, -- doesn't do anything, but i like to keep track of variables that will need to be defined later in a subclass or a constructor
--
-- init = false, -- abstract class, can't be instanciated
--
-- is_stable = function(self) -- method, available both in class and instances
-- return self.wheel_count > self.stability_threshold
-- end
-- }
--
-- local Car = Vehicle { -- subclassing by calling the parent class; multiple inheritance possible by either chaining calls or passing several tables as arguments
-- type = "car",
-- wheel_count = 4,
-- color = nil,
-- init = function(self, color) -- constructor
-- self.color = color
-- end
-- }
-- local car = Car:new("red") -- instancing
-- print(car:is_stable(), car.color) -- true, "red"
--
-- the default class returned by require("class") contains a few other default methods that will be inherited by all subclasses
-- see line 99 and further for details & documentation
--
-- design philosophy:
-- do not add feature until we need it
-- what we want to be fast: instance creation, class & instance method call & property acces
-- do not care: class creation
--
-- and if you're wondering, no i'm not using either classtoi-heavy nor classtoi-light in any current project anymore.
--# helper functions #--
-- tostring that ignore __tostring methamethod
local function rawtostring(v)
local mt = getmetatable(v)
setmetatable(v, nil)
local str = tostring(v)
setmetatable(v, mt)
return str
end
-- deep table copy, preserve metatable
local function copy(t, cache)
if cache == nil then cache = {} end
if cache[t] then return cache[t] end
local r = {}
cache[t] = r
for k, v in pairs(t) do
r[k] = type(v) == "table" and copy(v, cache) or v
end
return setmetatable(r, getmetatable(t))
end
-- add val to set
local function add_to_set(set, val)
if not set[val] then
table.insert(set, val)
set[val] = true
end
end
--# class creation logic #--
local new_class, class_mt
new_class = function(...)
local class = {}
local include = {...}
for i=1, #include do
local parent = include[i]
parent = parent.__included ~= nil and parent:__included(class) or parent
for k, v in pairs(parent) do
class[k] = v
end
end
class.__index = class
setmetatable(class, class_mt)
return class.__created ~= nil and class:__created() or class
end
class_mt = {
__call = new_class,
__tostring = function(self)
local name = self.type and ("class %q"):format(self.type) or "class"
return rawtostring(self):gsub("^table", name)
end
}
class_mt.__index = class_mt
--# base class and its contents #--
-- feel free to redefine these as needed in your own classes; all of these are also optional and can be deleted.
return new_class {
--- instanciate. arguments are passed to the (eventual) constructor :init.
-- behavior undefined when called on an object.
-- set to false to make class non-instanciable (will give unhelpful error on instanciation attempt).
-- obj = class:new(...)
new = function(self, ...)
local obj = setmetatable({}, self)
return obj.init ~= nil and obj:init(...) or obj
end,
--- constructor. arguments are passed from :new. if :init returns a value, it will be returned by :new instead of the self object.
-- set to false to make class abstract (will give unhelpful error on instanciation attempt), redefine in subclass to make non-abstract again.
-- init = function(self, ...) content... end
init = nil,
--- check if the object is an instance of this class.
-- class:is(obj)
-- obj:is(class)
is = function(self, other) -- class:is(obj)
if getmetatable(self) == class_mt then
return getmetatable(other) == self
else
return other:is(self)
end
end,
--- check if the object is an instance of this class or of a class that inherited this class.
-- parentclass:issub(obj)
-- parentclass:issub(class)
-- obj:issub(parentclass)
issub = function(self, other)
if getmetatable(self) == class_mt then
return other.__parents and other.__parents[self] or self:is(other)
else
return other:issub(self)
end
end,
--- check if self is a class
-- class:isclass()
isclass = function(self)
return getmetatable(self) == class_mt
end,
--- called when included in a new class. if it returns a value, it will be used as the included table instead of the self table.
-- default function tracks parent classes and is needed for :issub to work, and returns a deep copy of the included table.
__included = function(self, into)
-- add to parents
if not into.__parents then
into.__parents = {}
end
local __parents = self.__parents
if __parents then
for i=1, #__parents do
add_to_set(into.__parents, __parents[i])
end
end
add_to_set(into.__parents, self)
-- create copied table
local copied = copy(self)
copied.__parents = nil -- prevent __parents being overwritten
return copied
end,
-- automatically created by __included and needed for :issub to work
-- list and set of classes that are parents of this class: { parent_a, [parent_a] = true, parent_b, [parent_b] = true, ... }
__parents = nil,
--- called on the class when it is created. if it returns a value, it will be returned as the new class instead of the self class.
__created = nil,
--- pretty printing. type is used as the name of the class.
type = "object",
__tostring = function(self)
return rawtostring(self):gsub("^table", self.type)
end
}

91
common.lua
View file

@ -1,91 +0,0 @@
local common
--- replace values recursively in table t according to to_replace ([old table] = new table)
-- already_replaced is a temporary table to avoid infinite loop & duplicate processing, no need to give it
local function replace_in_table(t, to_replace, already_replaced)
already_replaced = already_replaced or {}
already_replaced[t] = true
for k, v in pairs(t) do
if to_replace[v] then
t[k] = to_replace[v]
elseif type(v) == "table" and not already_replaced[v] then
replace_in_table(v, to_replace, already_replaced)
end
end
end
common = {
--- recursively copy a table (key & values), handle cyclic references, no metatable
-- cache is table with copied tables [original table] = copied value, will create temporary table if argument is omitted
copy = function(t, cache)
if type(t) == "table" then
cache = cache or {}
if cache[t] then
return cache[t]
else
local c = {}
cache[t] = c
for k, v in pairs(t) do
c[common.copy(k, cache)] = common.copy(v, cache)
end
return c
end
else
return t
end
end,
--- given a table t from which some copy was issued, the copy cache, and a list of tables from the copied version,
-- put theses copied tables in t in place of their original values, preserving references to non-modified values
replace_with_copied_values = function(t, cache, copied_to_replace)
-- reverse copy cache
local ehcac = {}
for k, v in pairs(cache) do ehcac[v] = k end
-- build table of [original table] = replacement copied table
local to_replace = {}
for _, v in ipairs(copied_to_replace) do
local original = ehcac[v]
if original then -- table doesn't have an original value if it's a new table...
to_replace[original] = v
end
end
-- fix references to not-modified tables in modified values
local not_modified = {}
for original, modified in pairs(cache) do
if not to_replace[original] then
not_modified[modified] = original
end
end
for _, m in ipairs(copied_to_replace) do
replace_in_table(m, not_modified)
end
-- replace in t
replace_in_table(t, to_replace)
end,
--- given a table t issued from some copy, the copy cache, and a list of tables from the copied version,
-- put the original tables that are not in the list in t in place of their copied values
fix_not_modified_references = function(t, cache, copied_to_replace)
-- reverse copy cache
local ehcac = {}
for k, v in pairs(cache) do ehcac[v] = k end
-- build table of [original table] = replacement copied table
local to_replace = {}
for _, v in ipairs(copied_to_replace) do
local original = ehcac[v]
if original then -- table doesn't have an original value if it's a new table...
to_replace[original] = v
end
end
-- fix references to not-modified tables in t
local not_modified = {}
for original, modified in pairs(cache) do
if not to_replace[original] then
not_modified[modified] = original
end
end
replace_in_table(t, not_modified)
end
}
package.loaded[...] = common
return common

70
common/init.lua Normal file
View file

@ -0,0 +1,70 @@
local escape_cache = {}
local ansicolors = require("lib.ansicolors")
local common = {
-- escape text to be used as an exact pattern
escape = function(str)
if not escape_cache[str] then
escape_cache[str] = str:gsub("[^%w]", "%%%1")
end
return escape_cache[str]
end,
--- transform an identifier into a clean version (trim each part)
trim = function(str)
return str:match("^%s*(.-)%s*$")
end,
fmt = function(str, ...)
return ansicolors(str):format(...)
end,
uuid = function()
return ("xxxxxxxx-xxxx-4xxx-Nxxx-xxxxxxxxxxxx") -- version 4
:gsub("N", math.random(0x8, 0xb)) -- variant 1
:gsub("x", function() return ("%x"):format(math.random(0x0, 0xf)) end) -- random hexadecimal digit
end,
-- list of operators and their priority that are handled through regular function calls & can be overloaded/etc. by the user
regular_operators = {
prefixes = {
{ "~", 3.5 }, -- just below _~_ so else-if (~ condition ~ expression) parses as (~ (condition ~ expression))
{ "!", 11 },
{ "-", 11 },
{ "*", 11 },
},
suffixes = {
{ ";", 1 },
{ "!", 12 }
},
infixes = {
{ ";", 1 },
{ "#", 2 },
{ "~", 4 }, { "~?", 4 },
{ "|>", 5 }, { "&", 5 }, { "|", 5 },
{ "==", 7 }, { "!=", 7 }, { ">=", 7 }, { "<=", 7 }, { "<", 7 }, { ">", 7 },
{ "+", 8 }, { "-", 8 },
{ "//", 9 }, { "/", 9 }, { "*", 9 }, { "%", 9 },
{ "^", 10 },
{ "::", 11 },
{ ".", 14 },
{ ":", 5 }
}
},
-- list of all operators and their priority
operator_priority = {
[";_"] = 1,
["$_"] = 1,
["@_"] = 2,
["_,_"] = 2,
["_=_"] = 3,
["_!_"] = 12,
["_()"] = 13
-- generated at run-time for regular operators
}
}
local function store_priority(t, fmt)
for _, v in ipairs(t) do common.operator_priority[fmt:format(v[1])] = v[2] end
end
store_priority(common.regular_operators.infixes, "_%s_")
store_priority(common.regular_operators.prefixes, "%s_")
store_priority(common.regular_operators.suffixes, "_%s")
return common

26
common/to_anselme.lua Normal file
View file

@ -0,0 +1,26 @@
local ast = require("ast")
local Number, Struct, String, Nil, Boolean
local function to_anselme(val)
if type(val) == "number" then
return Number:new(val)
elseif type(val) == "table" then
local s = Struct:new()
for k, v in pairs(val) do
s:set(to_anselme(k), to_anselme(v))
end
return s
elseif type(val) == "string" then
return String:new(val)
elseif type(val) == "nil" then
return Nil:new()
elseif type(val) == "boolean" then
return Boolean:new(val)
else
error("can't convert "..type(val).." to an Anselme value")
end
end
Number, Struct, String, Nil, Boolean = ast.Number, ast.Struct, ast.String, ast.Nil, ast.Boolean
return to_anselme

263
doc/api.md Normal file
View file

@ -0,0 +1,263 @@
This document describes how to use the main Anselme modules. This is generated automatically from the source files.
Note that this file only describes the `anselme` and `state.State` modules, which are only a selection of what I consider to be the "public API" of Anselme that I will try to keep stable.
If you need more advanced control on Anselme, feel free to look into the other source files to find more; the most useful functions should all be reasonably commented.
# anselme
The main module.
Usage:
```lua
local anselme = require("anselme")
-- create a new state
local state = anselme.new()
state:load_stdlib()
-- read an anselme script file
local f = assert(io.open("script.ans"))
local script = anselme.parse(f:read("*a"), "script.ans")
f:close()
-- load the script in a new branch
local run_state = state:branch()
run_state:run(script)
-- run the script
while run_state:active() do
local e, data = run_state:step()
if e == "text" then
for _, l in ipairs(data) do
print(l:format(run_state))
end
elseif e == "choice" then
for i, l in ipairs(data) do
print(("%s> %s"):format(i, l:format(run_state)))
end
local choice = tonumber(io.read("*l"))
data:choose(choice)
elseif e == "return" then
run_state:merge()
elseif e == "error" then
error(data)
end
end
```
### .version
Global version string. Follow semver.
_defined at line 52 of [anselme.lua](../anselme.lua):_ `version = "2.0.0-alpha",`
### .versions
Table containing per-category version numbers. Incremented by one for any change that may break compatibility.
_defined at line 55 of [anselme.lua](../anselme.lua):_ `versions = {`
#### .language
Version number for languages and standard library changes.
_defined at line 57 of [anselme.lua](../anselme.lua):_ `language = 27,`
#### .save
Version number for save/AST format changes.
_defined at line 59 of [anselme.lua](../anselme.lua):_ `save = 4,`
#### .api
Version number for Lua API changes.
_defined at line 61 of [anselme.lua](../anselme.lua):_ `api = 8`
### .parse (code, source)
Parse a `code` string and return the generated AST.
`source` is an optional string; it will be used as the code source name in error messages.
Usage:
```lua
local ast = anselme.parse("1 + 2", "test")
ast:eval()
```
_defined at line 73 of [anselme.lua](../anselme.lua):_ `parse = function(code, source)`
### .new ()
Return a new [State](#state).
_defined at line 77 of [anselme.lua](../anselme.lua):_ `new = function()`
---
_file generated at 2023-12-21T20:56:31Z_
# State
Contains all state relative to an Anselme interpreter. Each State is fully independant from each other.
Each State can run a single script at a time, and variable changes are isolated between each State (see [branching](#branching-and-merging)).
### :load_stdlib ()
Load standard library.
You will probably want to call this on every State right after creation.
_defined at line 40 of [state/State.lua](../state/State.lua):_ `load_stdlib = function(self)`
## Branching and merging
### .branch_id
Name of the branch associated to this State.
_defined at line 47 of [state/State.lua](../state/State.lua):_ `branch_id = "main",`
### .source_branch_id
Name of the branch this State was branched from.
_defined at line 49 of [state/State.lua](../state/State.lua):_ `source_branch_id = "main",`
### :branch ()
Return a new branch of this State.
Branches act as indepent copies of this State where any change will not be reflected in the source State until it is merged back into the source branch.
Note: probably makes the most sense to create branches from the main State only.
_defined at line 55 of [state/State.lua](../state/State.lua):_ `branch = function(self)`
### :merge ()
Merge everything that was changed in this branch back into the main State branch.
Recommendation: only merge if you know that the state of the variables is consistent, for example at the end of the script, checkpoints, ...
If your script errored or was interrupted at an unknown point in the script, you might be in the middle of a calculation and variables won't contain
values you want to merge.
_defined at line 64 of [state/State.lua](../state/State.lua):_ `merge = function(self)`
## Variable definition
### :define (name, value, func, raw_mode)
Define a value in the global scope, converting it from Lua to Anselme if needed.
* for lua functions: `define("name", "(x, y, z=5)", function(x, y, z) ... end)`, where arguments and return values of the function are automatically converted between anselme and lua values
* for other lua values: `define("name", value)`
* for anselme AST: `define("name", value)`
`name` can be prefixed with symbol modifiers, for example ":name" for a constant variable.
If `raw_mode` is true, no anselme-to/from-lua conversion will be performed in the function.
The function will receive the state followed by AST nodes as arguments, and is expected to return an AST node.
_defined at line 82 of [state/State.lua](../state/State.lua):_ `define = function(self, name, value, func, raw_mode)`
### :define_local (name, value, func, raw_mode)
Same as `:define`, but define the expression in the current scope.
_defined at line 88 of [state/State.lua](../state/State.lua):_ `define_local = function(self, name, value, func, raw_mode)`
For anything more advanced, you can directly access the current scope stack stored in `state.scope`.
See [state/ScopeStack.lua](../state/ScopeStack.lua) for details; the documentation is not as polished as this file but you should still be able to find your way around.
## Saving and loading persistent variables
### :save ()
Return a serialized (string) representation of all global persistent variables in this State.
This can be loaded back later using `:load`.
_defined at line 100 of [state/State.lua](../state/State.lua):_ `save = function(self)`
### :load (save)
Load a string generated by `:save`.
Variables that do not exist currently in the global scope will be defined, those that do will be overwritten with the loaded data.
_defined at line 107 of [state/State.lua](../state/State.lua):_ `load = function(self, save)`
## Current script state
### :active ()
Indicate if a script is currently loaded in this branch.
_defined at line 127 of [state/State.lua](../state/State.lua):_ `active = function(self)`
### :state ()
Returns `"running`" if a script is currently loaded and running (i.e. this was called from the script).
Returns `"active"` if a script is loaded but not currently running (i.e. the script has not started or is waiting on an event).
Returns `"inactive"` if no script is loaded.
_defined at line 135 of [state/State.lua](../state/State.lua):_ `state = function(self)`
### :run (code, source)
Load a script in this branch. It will become the active script.
`code` is the code string or AST to run, `source` is the source name string to show in errors (optional).
Note that this will only load the script; execution will only start by using the `:step` method. Will error if a script is already active in this State.
_defined at line 147 of [state/State.lua](../state/State.lua):_ `run = function(self, code, source)`
### :step ()
When a script is active, will resume running it until the next event.
Will error if no script is active.
Returns `event type string, event data`.
_defined at line 160 of [state/State.lua](../state/State.lua):_ `step = function(self)`
### :interrupt (code, source)
Stops the currently active script.
Will error if no script is active.
If `code` is given, the script will not be disabled but instead will be immediately replaced with this new script.
The new script will then be started on the next `:step` and will preserve the current scope. This can be used to trigger an exit function or similar in the active script.
_defined at line 178 of [state/State.lua](../state/State.lua):_ `interrupt = function(self, code, source)`
### :eval (code, source)
Evaluate an expression in the global scope.
This can be called from outside a running script, but an error will be triggered the expression raise any event other than return.
* returns AST in case of success. Run `:to_lua(state)` on it to convert to a Lua value.
* returns `nil, error message` in case of error.
_defined at line 199 of [state/State.lua](../state/State.lua):_ `eval = function(self, code, source)`
### :eval_local (code, source)
Same as `:eval`, but evaluate the expression in the current scope.
_defined at line 206 of [state/State.lua](../state/State.lua):_ `eval_local = function(self, code, source)`
If you want to perform more advanced manipulation of the resulting AST nodes, look at the `ast` modules.
In particular, every Node inherits the methods from [ast.abstract.Node](../ast/abstract/Node.lua).
Otherwise, each Node has its own module file defined in the [ast/](../ast) directory.
---
_file generated at 2023-12-21T20:56:31Z_

12
doc/api.md.template Normal file
View file

@ -0,0 +1,12 @@
This document describes how to use the main Anselme modules. This is generated automatically from the source files.
Note that this file only describes the `anselme` and `state.State` modules, which are only a selection of what I consider to be the "public API" of Anselme that I will try to keep stable.
If you need more advanced control on Anselme, feel free to look into the other source files to find more; the most useful functions should all be reasonably commented.
# anselme
{{anselme.lua}}
# State
{{state/State.lua}}

86
doc/gendocs.lua Normal file
View file

@ -0,0 +1,86 @@
-- LDoc doesn't like me so I don't like LDoc.
-- Behold! A documentation generator that doesn't try to be smart!
-- Call this from the root anselme directory: `lua doc/gendocs.lua`
local files = {
"doc/api.md"
}
local source_link_prefix = "../"
local base_header_level = 2
local title_extractors = {
-- methods
{ "(.-)%s*=%s*function%s*%(%s*self%s*%)", ":%1 ()" },
{ "(.-)%s*=%s*function%s*%(%s*self%s*%,%s*(.-)%)", ":%1 (%2)" },
-- functions
{ "(.-)%s*=%s*function%s*%((.-)%)", ".%1 (%2)" },
-- fields
{ "(.-)%s*=", ".%1" },
}
local function extract_block_title(line)
local title = line
for _, ext in ipairs(title_extractors) do
if line:match(ext[1]) then
title = line:gsub(("^%s.-$"):format(ext[1]), ext[2])
break
end
end
return title
end
local function process(content)
return content:gsub("{{(.-)}}", function(lua_file)
local f = io.open(lua_file, "r")
local c = f:read("*a")
f:close()
local output = {}
local comment_block
local line_no = 1
for line in c:gmatch("[^\n]*") do
if line:match("^%s*%-%-%-") then
comment_block = {}
table.insert(comment_block, (line:match("^%s*%-%-%-%s?(.-)$")))
elseif comment_block then
if line:match("^%s*%-%-") then
table.insert(comment_block, (line:match("^%s*%-%-%s?(.-)$")))
else
if line:match("[^%s]") then
local ident, code = line:match("^(%s*)(.-)$")
table.insert(comment_block, 1, ("%s %s\n"):format(
("#"):rep(base_header_level+utf8.len(ident)),
extract_block_title(code)
))
table.insert(comment_block, ("\n_defined at line %s of [%s](%s):_ `%s`"):format(line_no, lua_file, source_link_prefix..lua_file, code))
end
table.insert(comment_block, "")
table.insert(output, table.concat(comment_block, "\n"))
comment_block = nil
end
end
line_no = line_no + 1
end
table.insert(output, ("\n---\n_file generated at %s_"):format(os.date("!%Y-%m-%dT%H:%M:%SZ")))
return table.concat(output, "\n")
end)
end
local function generate_file(input, output)
local f = io.open(input, "r")
local content = f:read("*a")
f:close()
local out = process(content, output)
f = io.open(output, "w")
f:write(out)
f:close()
end
for _, path in ipairs(files) do
generate_file(path..".template", path)
end

1
doc/language.md Normal file
View file

@ -0,0 +1 @@
TODO

1
doc/tutorial.md Normal file
View file

@ -0,0 +1 @@
TODO

127
ideas.md Normal file
View file

@ -0,0 +1,127 @@
Various ideas and things that may or may not be done. It's like GitHub issues, but I don't have to leave my text editor or connect to the scary Internet.
Loosely ordered by willingness to implement.
---
Documentation:
* language reference
* tutorial
---
Write tests. Kinda mandatory actually, while I've tried to improve and do it much better than Anselme v1 there's still plenty interweaved moving parts here. Not sure how much better I can do with the same design requirements tbh. See anselme v1 tests to get a base library of tests.
---
Make requires relative. Currently Anselme expect its directory to be properly somewhere in package.path.
Also improve compatibility with Lua 5.3 and LuaJIT (I don't think we should support anything other than 5.4, 5.3 and LuaJIT).
---
Translation. TODO Design
Translation model:
- for text, choices: text+line+file as id, translation (either text or function)
- for strings, assets, ...: ? translatable string ?
- for variable names: ?
- for stdlib: ?
---
Persistence "issue": Storing a closure stores it whole environment, which includes all the stdlib. Technically it works, but that's a lot of useless information. Would need to track which variable is used (should be doable in prepare) and prune the closure.
Or register all functions as ressources in binser - that makes kinda sense, they're immutable, and their signature should be unique. Would need to track which functions are safe to skip / can be reloaded from somewhere on load.
---
Redesign the Node hierarchy to avoid cycles.
---
Standard library.
* Text manipulation would make sense, but that would require a full UTF-8/Unicode support library like https://github.com/starwing/luautf8.
* Something to load other files. Maybe not load it by default to let the calling game sandbox Anselme.
* Implement the useful functions from Anselme v1.
* Checkpoint management.
* Overloadable :format for custom types.
---
Server API.
To be able to use Anselme in another language, it would be nice to be able to access it over some form of IPC.
No need to bother with networking I think. Just do some stdin/stdout handling, maybe use something like JSON-RPC: https://www.jsonrpc.org/specification (reminder: will need to add some metadata to specify content length, not aware of any streaming json lib in pure Lua - here's a rxi seal of quality library btw: https://github.com/rxi/json.lua). Or just make our own protocol around JSON.
Issue: how to represent Anselme values? they will probably contain cycles, needs to access their methods, etc.
Probably wise to look into how other do it. LSP: https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/
---
Syntax modifications:
* on the subject of assignments:
- multiple assignments:
:a, :b = 5, 6
a, b = list!($(l) l[3], l[6])
Easy by interpreting the left operand as a List.
- regular operator assignments:
Could interpret the left operand as a string when it is an identifier, like how _._ works.
Would feel good to have less nodes. But because we can doesn't mean we should. Also Assignment is reused in a few other places.
---
Reduce the number of AST node types ; try to merge similar node and make simpler individuals nodes if possible by composing them.
Won't help with performance but make me feel better, and easier to extend. Anselme should be more minimal is possible.
---
Static analysis tools.
To draw a graph of branches, keep track of used variables and prune the unused ones from the Environments, pre-filter Overloads, etc.
---
Multiline expressions.
* add the ability to escape newlines
Issue: need a way to correctly track line numbers, the current parser assumes one expression = one source
* allow some expressions to run over several lines (the ones that expect a closing token, like paren/list/structs)
Issue: the line and expression parsing is completely separate
---
Performance:
* the most terribly great choice is the overload with parameter filtering.
Assuming the filter functions are pure seems reasonable, so caching could be done.
Could also hardcode some shortcut paths for the simple type equality check case.
Or track function/expression purity and cache/precompute the results. Not sure how to do that with multiple dispatch though.
(note for future reference: once a function is first evaluated into a closure, its parameters are fixed, including the type check overloads)
* the recursive AST interpreter is also pretty meh, could do a bytecode VM.
This one seems like a lot more work.
Could also compile to Lua and let LuaJIT deal with it. Or WASM, that sounds trendy.
Then again, performance has never been a goal of Anselme.
---
Macros.
Could be implemented by creating functions to build AST nodes from Anselme that can also take quotes as arguments.
That should be easy, but I don't remember why I wanted macros in the first place, so until I want them again, shrug.
---
High concept ideas / stuff that sounds cool but maybe not worth it.
* Instead of using a bunch of sigils as operators, accept fancy unicode caracters.
Easy to parse, but harder to write.
Could implement a formatter/linter/whatever this is called these days and have Anselme recompile the AST into a nice, properly Unicodified output.
Issue: the parser may be performing some transformations on the AST that would make the output an uncanny valley copy of the original. Also we need to preserve comments.
* Files are so 2000; instead put everything in a big single file and use a custom editor to edit it.
Imagine selecting an identifier, and then it zooms in and show the AST associated to it. Nested indefinitely. Feels very futuristic, so probably worth it.
* Frankly the event buffer system still feel pretty janky, but I don't have any better idea for now.

1
init.lua
View file

@ -1 +0,0 @@
return require((...)..".anselme")

699
interpreter/common.lua
View file

@ -1,699 +0,0 @@
local atypes, ltypes
local eval, run_block
local replace_with_copied_values, fix_not_modified_references
local common
local identifier_pattern
local copy
local function random_identifier()
local r = ""
for _=1, 16 do -- that's like 10^31 possibilities, ought to be enough for anyone
r = r .. string.char(math.random(32, 126))
end
return r
end
common = {
--- merge interpreter state with global state
merge_state = function(state)
local mt = getmetatable(state.variables)
-- store current scoped variables before merging them
for fn in pairs(mt.scoped) do
common.scope:store_last_scope(state, fn)
end
-- merge alias state
local global = state.interpreter.global_state
for alias, fqm in pairs(state.aliases) do
global.aliases[alias] = fqm
state.aliases[alias] = nil
end
-- merge modified mutable varables
local copy_cache, modified_tables = mt.copy_cache, mt.modified_tables
replace_with_copied_values(global.variables, copy_cache, modified_tables)
mt.copy_cache = {}
mt.modified_tables = {}
mt.cache = {}
-- merge modified re-assigned variables
for var, value in pairs(state.variables) do
if var:match("^"..identifier_pattern.."$") then -- skip scoped variables
global.variables[var] = value
state.variables[var] = nil
end
end
-- scoping: since merging means we will re-copy every variable from global state again, we need to simulate this
-- behaviour for scoped variables (to have consistent references for mutables values in particular), including
-- scopes that aren't currently active
fix_not_modified_references(mt.scoped, copy_cache, modified_tables) -- replace not modified values in scope with original before re-copying to keep consistent references
for _, scopes in pairs(mt.scoped) do
for _, scope in ipairs(scopes) do
for var, value in pairs(scope) do
-- pretend the value for this scope is the global value so the cache system perform the new copy from it
local old_var = global.variables[var]
global.variables[var] = value
state.variables[var] = nil
scope[var] = state.variables[var]
mt.cache[var] = nil
global.variables[var] = old_var
end
end
end
-- restore last scopes
for fn in pairs(mt.scoped) do
common.scope:set_last_scope(state, fn)
end
end,
--- checks if the value is compatible with the variable's (eventual) constraint
-- returns depth, or math.huge if no constraint
-- returns nil, err
check_constraint = function(state, fqm, val)
local constraint = state.variable_metadata[fqm].constraint
if constraint then
if not constraint.value then
local v, e = eval(state, constraint.pending)
if not v then
return nil, ("%s; while evaluating constraint for variable %q"):format(e, fqm)
end
constraint.value = v
end
local depth = common.is_of_type(val, constraint.value)
if not depth then
return nil, ("constraint check failed")
end
return depth
end
return math.huge
end,
--- checks if the variable is mutable
-- returns true
-- returns nil, mutation illegal message
check_mutable = function(state, fqm)
if state.variable_metadata[fqm].constant then
return nil, ("can't change the value of a constant %q"):format(fqm)
end
return true
end,
--- mark a value as constant, recursively affecting all the potentially mutable subvalues
mark_constant = function(v)
return assert(common.traverse(v, function(v)
if v.hash_id then v.hash_id = nil end -- no longer need to compare by id
end, "mark_constant"))
end,
-- traverse v and all the subvalues it contains
-- callback(v) is called on every value after traversing its subvalues
-- if pertype_callback is given, will then call the associated callback(v) in the type table for each value
-- both those callbacks can either returns nil (success) or nil, err (error)
-- returns true
-- return nil, error
traverse = function(v, callback, pertype_callback)
if atypes[v.type] and atypes[v.type].traverse then
local r, e = atypes[v.type].traverse(v.value, callback, pertype_callback)
if not r then return nil, e end
r, e = callback(v)
if e then return nil, e end
if pertype_callback and atypes[v.type][pertype_callback] then
r, e = atypes[v.type][pertype_callback](v)
if e then return nil, e end
end
return true
else
error(("don't know how to traverse type %s"):format(v.type))
end
end,
--- checks if the value can be persisted
-- returns true
-- returns nil, persist illegal message
check_persistable = function(v)
return common.traverse(v, function(v)
if v.nonpersistent then
return nil, ("can't put a non persistable %s into a persistent variable"):format(v.type)
end
end)
end,
--- returns a variable's value, evaluating a pending expression if neccessary
-- if you're sure the variable has already been evaluated, use state.variables[fqm] directly
-- return var
-- return nil, err
get_variable = function(state, fqm)
local var = state.variables[fqm]
if var.type == "pending definition" then
-- evaluate
local v, e = eval(state, var.value.expression)
if not v then
return nil, ("%s; while evaluating default value for variable %q defined at %s"):format(e, fqm, var.value.source)
end
-- make constant if variable is constant
if state.variable_metadata[fqm].constant then
v = copy(v)
common.mark_constant(v)
end
-- set variable
local s, err = common.set_variable(state, fqm, v, state.variable_metadata[fqm].constant)
if not s then return nil, err end
return v
else
return var
end
end,
--- set the value of a variable
-- returns true
-- returns nil, err
set_variable = function(state, name, val, bypass_constant_check)
if val.type ~= "pending definition" then
-- check constant
if not bypass_constant_check then
local s, e = common.check_mutable(state, name)
if not s then
return nil, ("%s; while assigning value to variable %q"):format(e, name)
end
end
-- check persistence
if state.variable_metadata[name].persistent then
local s, e = common.check_persistable(val)
if not s then
return nil, ("%s; while assigning value to variable %q"):format(e, name)
end
end
-- check constraint
local s, e = common.check_constraint(state, name, val)
if not s then
return nil, ("%s; while assigning value to variable %q"):format(e, name)
end
end
state.variables[name] = val
return true
end,
--- handle scoped function
scope = {
init_scope = function(self, state, fn)
local scoped = getmetatable(state.variables).scoped
if not fn.scoped then error("trying to initialize the scope stack for a non-scoped function") end
if not scoped[fn] then scoped[fn] = {} end
-- check scoped variables
for _, name in ipairs(fn.scoped) do
-- put fresh variable from global state in scope
local val = state.interpreter.global_state.variables[name]
if val.type ~= "undefined argument" and val.type ~= "pending definition" then -- only possibilities for scoped variable, and they're immutable
error("invalid scoped variable")
end
end
end,
--- push a new scope for this function
push = function(self, state, fn)
local scoped = getmetatable(state.variables).scoped
self:init_scope(state, fn)
-- preserve current values in last scope
self:store_last_scope(state, fn)
-- add scope
local fn_scope = {}
table.insert(scoped[fn], fn_scope)
self:set_last_scope(state, fn)
end,
--- pop the last scope for this function
pop = function(self, state, fn)
local scoped = getmetatable(state.variables).scoped
if not scoped[fn] then error("trying to pop a scope without any pushed scope") end
-- remove current scope
table.remove(scoped[fn])
-- restore last scope
self:set_last_scope(state, fn)
-- if the stack is empty,
-- we could remove mt.scoped[fn] I guess, but I don't think there's going to be a million different functions in a single game so should be ok
-- (anselme's performance is already bad enough, let's not create tables at each function call...)
end,
--- store the current values of the scoped variables in the last scope of this function
store_last_scope = function(self, state, fn)
local scopes = getmetatable(state.variables).scoped[fn]
local last_scope = scopes[#scopes]
if last_scope then
for _, name in pairs(fn.scoped) do
local val = rawget(state.variables, name)
if val then
last_scope[name] = val
end
end
end
end,
--- set scopped variables to previous scope
set_last_scope = function(self, state, fn)
local scopes = getmetatable(state.variables).scoped[fn]
for _, name in ipairs(fn.scoped) do
state.variables[name] = nil
end
local last_scope = scopes[#scopes]
if last_scope then
for name, val in pairs(last_scope) do
state.variables[name] = val
end
end
end
},
--- mark a table as modified, so it will be merged on the next checkpoint if it appears somewhere in a value
mark_as_modified = function(state, v)
local modified = getmetatable(state.variables).modified_tables
table.insert(modified, v)
end,
--- returns true if a variable should be persisted on save
-- will exclude: variable that have not been evaluated yet and non-persistent variable
-- this will by consequence excludes variable in scoped variables (can be neither persistent not evaluated into global state), constants (can not be persistent), internal anselme variables (not marked persistent), etc.
-- You may want to check afterwards with check_persistable to check if the value can actually be persisted.
should_be_persisted = function(state, name, value)
return value.type ~= "pending definition" and state.variable_metadata[name].persistent
end,
--- check truthyness of an anselme value
truthy = function(val)
if val.type == "number" then
return val.value ~= 0
elseif val.type == "nil" then
return false
else
return true
end
end,
--- compare two anselme values for equality.
-- for immutable values or constants: compare by value
-- for mutable values: compare by reference
compare = function(a, b)
if a.type ~= b.type or a.constant ~= b.constant then
return false
end
if a.type == "pair" or a.type == "annotated" then
return common.compare(a.value[1], b.value[1]) and common.compare(a.value[2], b.value[2])
elseif a.type == "function reference" then
if #a.value ~= #b.value then
return false
end
for _, aname in ipairs(a.value) do
local found = false
for _, bname in ipairs(b.value) do
if aname == bname then
found = true
break
end
end
if not found then
return false
end
end
return true
-- mutable types: need to be constant
elseif a.constant and a.type == "list" then
if #a.value ~= #b.value then
return false
end
for i, v in ipairs(a.value) do
if not common.compare(v, b.value[i]) then
return false
end
end
return true
elseif a.constant and a.type == "map" then
return common.hash(a) == common.hash(b)
elseif a.constant and a.type == "object" then
if a.value.class ~= b.value.class then
return false
end
-- check every attribute redefined in a and b
-- NOTE: comparaison will fail if an attribute has been redefined in only one of the object, even if it was set to the same value as the original class attribute
local compared = {}
for name, v in pairs(a.value.attributes) do
compared[name] = true
if not b.value.attributes[name] or not common.compare(v, b.value.attributes[name]) then
return false
end
end
for name, v in pairs(b.value.attributes) do
if not compared[name] then
if not a.value.attributes[name] or not common.compare(v, a.value.attributes[name]) then
return false
end
end
end
return true
-- the rest
else
return a.value == b.value
end
end,
--- format a anselme value to something printable
-- does not call custom {}() functions, only built-in ones, so it should not be able to fail
-- str: if success
-- nil, err: if error
format = function(val)
if atypes[val.type] and atypes[val.type].format then
return atypes[val.type].format(val.value)
else
return nil, ("no formatter for type %q"):format(val.type)
end
end,
--- compute a hash for a value.
-- A hash is a Lua string such as, given two values, they are considered equal by Anselme if and only if their hash are considered equal by Lua.
-- Will generate random identifiers for mutable values (equality test by reference) in order for the identifier to stay the same accross checkpoints and
-- other potential variable copies.
-- str: if success
-- nil, err: if error
hash = function(val)
if atypes[val.type] and atypes[val.type].hash then
if atypes[val.type].mutable and not val.constant then
if not val.hash_id then val.hash_id = random_identifier() end
return ("mut(%s)"):format(val.hash_id)
else
return atypes[val.type].hash(val.value)
end
else
return nil, ("no hasher for type %q"):format(val.type)
end
end,
--- recompute all the hases in a map.
-- str: if success
-- nil, err: if error
update_hashes = function(map)
for k, v in pairs(map.value) do
local hash, e = common.hash(v[1])
if not hash then return nil, e end
map[k] = nil
map[hash] = v
end
end,
--- convert anselme value to lua
-- lua value: if success (may be nil!)
-- nil, err: if error
to_lua = function(val, state)
if atypes[val.type] and atypes[val.type].to_lua then
return atypes[val.type].to_lua(val.value, state)
else
return nil, ("no Lua exporter for type %q"):format(val.type)
end
end,
--- convert lua value to anselme
-- anselme value: if success
-- nil, err: if error
from_lua = function(val)
if ltypes[type(val)] and ltypes[type(val)].to_anselme then
return ltypes[type(val)].to_anselme(val)
else
return nil, ("no Lua importer for type %q"):format(type(val))
end
end,
--- evaluate a text AST into a single Lua string
-- string: if success
-- nil, err: if error
eval_text = function(state, text)
local l = {}
local s, e = common.eval_text_callback(state, text, function(str) table.insert(l, str) end)
if not s then return nil, e end
return table.concat(l)
end,
--- same as eval_text, but instead of building a Lua string, call callback for every evaluated part of the text
-- callback returns nil, err in case of error
-- true: if success
-- nil, err: if error
eval_text_callback = function(state, text, callback)
for _, item in ipairs(text) do
if type(item) == "string" then
callback(item)
else
local v, e = eval(state, item)
if not v then return v, e end
v, e = common.format(v)
if not v then return v, e end
if v ~= "" then
local r, err = callback(v)
if err then return r, err end
end
end
end
return true
end,
--- check if an anselme value is of a certain type or annotation
-- specificity(number): if var is of type type. lower is more specific
-- false: if not
is_of_type = function(var, type)
local depth = 1
-- var has a custom annotation
if var.type == "annotated" then
-- special case: if we just want to see if a value is annotated
if type.type == "string" and type.value == "annotated" then
return depth
end
-- check annotation
local var_type = var.value[2]
while true do
if common.compare(var_type, type) then -- same type
return depth
elseif var_type.type == "annotated" then -- compare parent type
depth = depth + 1
var_type = var_type.value[2]
else -- no parent, fall back on base type
depth = depth + 1
var = var.value[1]
break
end
end
end
-- var has a base type
return type.type == "string" and type.value == var.type and depth
end,
-- return a pretty printable type value for var
pretty_type = function(var)
if var.type == "annotated" then
return common.format(var.value[2])
else
return var.type
end
end,
--- tag management
tags = {
--- push new tags on top of the stack, from Anselme values. val is expected to be a map.
push = function(self, state, val)
local new = { type = "map", value = {} }
-- copy
local last = self:current(state)
for k, v in pairs(last.value) do new.value[k] = v end
-- append new values
for k, v in pairs(val.value) do new.value[k] = v end
-- add
table.insert(state.interpreter.tags, new)
end,
--- same but do not merge with last stack item
push_no_merge = function(self, state, val)
table.insert(state.interpreter.tags, val)
end,
-- pop tag table on top of the stack
pop = function(self, state)
table.remove(state.interpreter.tags)
end,
--- return current lua tags table
current = function(self, state)
return state.interpreter.tags[#state.interpreter.tags] or { type = "map", value = {} }
end,
--- returns length of tags stack
len = function(self, state)
return #state.interpreter.tags
end,
--- pop item until we reached desired stack length
-- so in case there's a possibility to mess up the stack somehow, it will restore the stack to a good state
trim = function(self, state, len)
while #state.interpreter.tags > len do
self:pop(state)
end
end
},
--- event buffer management
-- i.e. only for text and choice events
events = {
--- add a new element to the last event in the current buffer
-- will create new event if needed
append = function(self, state, type, data)
local buffer = self:current_buffer(state)
local last = buffer[#buffer]
if not last or last.type ~= type then
last = { type = type, value = {} }
table.insert(buffer, last)
end
table.insert(last.value, data)
end,
--- new events will be collected in this event buffer (any table) until the next pop
-- this is handled by a stack so nesting is allowed
push_buffer = function(self, state, buffer)
table.insert(state.interpreter.event_buffer_stack, buffer)
end,
--- stop capturing events of a certain type.
-- must be called after a push_buffer
pop_buffer = function(self, state)
table.remove(state.interpreter.event_buffer_stack)
end,
--- returns the current buffer value
current_buffer = function(self, state)
return state.interpreter.event_buffer_stack[#state.interpreter.event_buffer_stack]
end,
-- flush event buffer if it's neccessary to push an event of the given type
-- returns true in case of success
-- returns nil, err in case of error
make_space_for = function(self, state, type)
if #state.interpreter.event_buffer_stack == 0 and state.interpreter.current_event and state.interpreter.current_event.type ~= type then
return self:manual_flush(state)
end
return true
end,
--- write all the data in a buffer into the current buffer, or to the game is no buffer is currently set
write_buffer = function(self, state, buffer)
for _, event in ipairs(buffer) do
if #state.interpreter.event_buffer_stack == 0 then
if event.type == "flush" then
local r, e = self:manual_flush(state)
if not r then return r, e end
elseif state.interpreter.current_event then
if state.interpreter.current_event.type == event.type then
for _, v in ipairs(event.value) do
table.insert(state.interpreter.current_event.value, v)
end
else
local r, e = self:manual_flush(state)
if not r then return r, e end
state.interpreter.current_event = event
end
else
state.interpreter.current_event = event
end
else
local current_buffer = self:current_buffer(state)
table.insert(current_buffer, event)
end
end
return true
end,
--- same as manual_flush but add the flush to the current buffer if one is set instead of directly to the game
flush = function(self, state)
if #state.interpreter.event_buffer_stack == 0 then
return self:manual_flush(state)
else
local current_buffer = self:current_buffer(state)
table.insert(current_buffer, { type = "flush" })
return true
end
end,
--- flush events and send them to the game if possible
-- returns true in case of success
-- returns nil, err in case of error
manual_flush = function(self, state)
while state.interpreter.current_event do
local event = state.interpreter.current_event
state.interpreter.current_event = nil
state.interpreter.skip_choices_until_flush = nil
local type = event.type
local buffer
local choices
-- copy & process text buffer
if type == "text" then
buffer = common.post_process_text(state, event.value)
-- copy & process choice buffer
elseif type == "choice" then
-- copy & process choice text content into buffer, and needed private state into choices for each choice
buffer = {}
choices = {}
for _, c in ipairs(event.value) do
table.insert(buffer, common.post_process_text(state, c))
table.insert(choices, c._state)
end
-- discard empty choices
for i=#buffer, 1, -1 do
if #buffer[i] == 0 then
table.remove(buffer, i)
table.remove(choices, i)
end
end
-- nervermind
if #choices == 0 then
return true
end
end
-- yield event
coroutine.yield(type, buffer)
-- run choice
if type == "choice" then
local sel = state.interpreter.choice_selected
state.interpreter.choice_selected = nil
if not sel or sel < 1 or sel > #choices then
return nil, "invalid choice"
else
local choice = choices[sel]
-- execute in expected tag & event capture state
local capture_state = state.interpreter.event_capture_stack
state.interpreter.event_capture_stack = {}
common.tags:push_no_merge(state, choice.tags)
local _, e = run_block(state, choice.block)
common.tags:pop(state)
state.interpreter.event_capture_stack = capture_state
if e then return nil, e end
-- we discard return value from choice block as the execution is delayed until an event flush
-- and we don't want to stop the execution of another function unexpectedly
end
end
end
return true
end
},
--- copy some text & process it to be suited to be sent to Lua in an event
post_process_text = function(state, text)
local r = {}
-- copy into r & convert tags to lua
for _, t in ipairs(text) do
local tags = common.to_lua(t.tags, state)
if state.interpreter.base_lua_tags then
for k, v in pairs(state.interpreter.base_lua_tags) do
if tags[k] == nil then tags[k] = v end
end
end
table.insert(r, {
text = t.text,
tags = tags
})
end
-- remove trailing spaces
if state.feature_flags["strip trailing spaces"] then
local final = r[#r]
if final then
final.text = final.text:match("^(.-) *$")
if final.text == "" then
table.remove(r)
end
end
end
-- remove duplicate spaces
if state.feature_flags["strip duplicate spaces"] then
for i=1, #r-1 do
local a, b = r[i], r[i+1]
local na = #a.text:match(" *$")
local nb = #b.text:match("^ *")
if na > 0 and nb > 0 then -- remove duplicated spaces from second element first
b.text = b.text:match("^ *(.-)$")
end
if na > 1 then
a.text = a.text:match("^(.- ) *$")
end
end
end
return r
end
}
package.loaded[...] = common
local types = require((...):gsub("interpreter%.common$", "stdlib.types"))
atypes, ltypes = types.anselme, types.lua
eval = require((...):gsub("common$", "expression"))
run_block = require((...):gsub("common$", "interpreter")).run_block
local acommon = require((...):gsub("interpreter%.common$", "common"))
replace_with_copied_values, fix_not_modified_references = acommon.replace_with_copied_values, acommon.fix_not_modified_references
identifier_pattern = require((...):gsub("interpreter%.common$", "parser.common")).identifier_pattern
copy = require((...):gsub("interpreter%.common$", "common")).copy
return common

586
interpreter/expression.lua
View file

@ -1,586 +0,0 @@
local expression
local to_lua, from_lua, eval_text, truthy, format, pretty_type, get_variable, tags, eval_text_callback, events, flatten_list, set_variable, scope, check_constraint, hash
local run
local unpack = table.unpack or unpack
--- evaluate an expression
-- returns evaluated value (table) if success
-- returns nil, error if error
local function eval(state, exp)
-- nil
if exp.type == "nil" then
return {
type = "nil",
value = nil
}
-- number
elseif exp.type == "number" then
return {
type = "number",
value = exp.value
}
-- string
elseif exp.type == "string" then
local t, e = eval_text(state, exp.text)
if not t then return nil, e end
return {
type = "string",
value = t
}
-- text buffer
elseif exp.type == "text buffer" then
-- eval text expression
local v, e = eval(state, exp.text)
if not v then return v, e end
local l = v.type == "list" and v.value or { v }
-- write resulting buffers (plural if loop in text expression) into a single result buffer
local buffer = {}
for _, item in ipairs(l) do
if item.type == "event buffer" then
for _, event in ipairs(item.value) do
if event.type ~= "text" and event.type ~= "flush" then
return nil, ("event %q can't be captured in a text buffer"):format(event.type)
end
table.insert(buffer, event)
end
end
end
return {
type = "event buffer",
value = buffer
}
-- parentheses
elseif exp.type == "parentheses" then
return eval(state, exp.expression)
-- list defined in brackets
elseif exp.type == "list brackets" then
if exp.expression then
local v, e = eval(state, exp.expression)
if not v then return nil, e end
if exp.expression.type == "list" then
return v
-- contained a single element, wrap in list manually
else
return {
type = "list",
value = { v }
}
end
else
return {
type = "list",
value = {}
}
end
-- map defined in brackets
elseif exp.type == "map brackets" then
-- get constructing list
local list, e = eval(state, { type = "list brackets", expression = exp.expression })
if not list then return nil, e end
-- make map
local map = {}
for i, v in ipairs(list.value) do
local key, value
if v.type == "pair" then
key = v.value[1]
value = v.value[2]
else
key = { type = "number", value = i }
value = v
end
local h, err = hash(key)
if not h then return nil, err end
map[h] = { key, value }
end
return {
type = "map",
value = map
}
-- list defined using , operator
elseif exp.type == "list" then
local flat = flatten_list(exp)
local l = {}
for _, ast in ipairs(flat) do
local v, e = eval(state, ast)
if not v then return nil, e end
table.insert(l, v)
end
return {
type = "list",
value = l
}
-- assignment
elseif exp.type == ":=" then
if exp.left.type == "variable" then
local name = exp.left.name
local val, vale = eval(state, exp.right)
if not val then return nil, vale end
local s, e = set_variable(state, name, val)
if not s then return nil, e end
return val
else
return nil, ("don't know how to perform assignment on %s expression"):format(exp.left.type)
end
-- lazy boolean operators
elseif exp.type == "&" then
local left, lefte = eval(state, exp.left)
if not left then return nil, lefte end
if truthy(left) then
local right, righte = eval(state, exp.right)
if not right then return nil, righte end
if truthy(right) then
return {
type = "number",
value = 1
}
end
end
return {
type = "number",
value = 0
}
elseif exp.type == "|" then
local left, lefte = eval(state, exp.left)
if not left then return nil, lefte end
if truthy(left) then
return {
type = "number",
value = 1
}
end
local right, righte = eval(state, exp.right)
if not right then return nil, righte end
return {
type = "number",
value = truthy(right) and 1 or 0
}
-- conditional
elseif exp.type == "~" then
local right, righte = eval(state, exp.right)
if not right then return nil, righte end
if truthy(right) then
local left, lefte = eval(state, exp.left)
if not left then return nil, lefte end
return left
end
return {
type = "nil",
value = nil
}
-- while loop
elseif exp.type == "~?" then
local right, righte = eval(state, exp.right)
if not right then return nil, righte end
local l = {}
while truthy(right) do
local left, lefte = eval(state, exp.left)
if not left then return nil, lefte end
table.insert(l, left)
-- next iteration
right, righte = eval(state, exp.right)
if not right then return nil, righte end
end
return {
type = "list",
value = l
}
-- tag
elseif exp.type == "#" then
local right, righte = eval(state, { type = "map brackets", expression = exp.right })
if not right then return nil, righte end
tags:push(state, right)
local left, lefte = eval(state, exp.left)
tags:pop(state)
if not left then return nil, lefte end
return left
-- variable
elseif exp.type == "variable" then
return get_variable(state, exp.name)
-- references
elseif exp.type == "function reference" then
return {
type = "function reference",
value = exp.names
}
elseif exp.type == "variable reference" then
-- check if variable is already a reference
local v, e = eval(state, exp.expression)
if not v then return nil, e end
if v.type == "function reference" or v.type == "variable reference" then
return v
else
return { type = "variable reference", value = exp.name }
end
elseif exp.type == "implicit call if reference" then
local v, e = eval(state, exp.expression)
if not v then return nil, e end
if v.type == "function reference" or v.type == "variable reference" then
exp.variant.argument.expression.value = v
return eval(state, exp.variant)
else
return v
end
-- function
elseif exp.type == "function call" then
-- eval args: map brackets
local args = {}
local last_contiguous_positional = 0
if exp.argument then
local arg, arge = eval(state, exp.argument)
if not arg then return nil, arge end
-- map into args table
for _, v in pairs(arg.value) do
if v[1].type == "string" or v[1].type == "number" then
args[v[1].value] = v[2]
else
return nil, ("unexpected key of type %s in argument map; keys must be string or number"):format(v[1].type)
end
end
-- get length of contiguous positional arguments (#args may not be always be equal depending on implementation...)
for i, _ in ipairs(args) do
last_contiguous_positional = i
end
end
-- function reference: call the referenced function
local variants = exp.variants
local paren_call = exp.paren_call
if args[1] and args[1].type == "function reference" and (exp.called_name == "()" or exp.called_name == "_!") then
-- remove func ref as first arg
local refv = args[1].value
table.remove(args, 1)
-- set paren_call for _!
if exp.called_name == "_!" then
paren_call = false
end
-- get variants of the referenced function
variants = {}
for _, ffqm in ipairs(refv) do
for _, variant in ipairs(state.functions[ffqm]) do
table.insert(variants, variant)
end
end
end
-- eval assignment arg
local assignment
if exp.assignment then
local arge
assignment, arge = eval(state, exp.assignment)
if not assignment then return nil, arge end
end
-- try to select a function
local tried_function_error_messages = {}
local selected_variant = { depths = { assignment = nil }, variant = nil, args_to_set = nil }
for _, fn in ipairs(variants) do
if fn.type ~= "function" then
return nil, ("unknown function type %q"):format(fn.type)
-- functions
else
if not fn.assignment or exp.assignment then
local ok = true
-- get and set args
local variant_args = {}
local used_args = {}
local depths = { assignment = nil }
for j, param in ipairs(fn.params) do
local val
-- named
if param.alias and args[param.alias] then
val = args[param.alias]
used_args[param.alias] = true
elseif args[param.name] then
val = args[param.name]
used_args[param.name] = true
-- vararg
elseif param.vararg then
val = { type = "list", value = {} }
for k=j, last_contiguous_positional do
table.insert(val.value, args[k])
used_args[k] = true
end
-- positional
elseif args[j] then
val = args[j]
used_args[j] = true
end
if val then
-- check type constraint
local depth, err = check_constraint(state, param.full_name, val)
if not depth then
ok = false
local v = state.variable_metadata[param.full_name].constraint.value
table.insert(tried_function_error_messages, ("%s: argument %s is not of expected type %s"):format(fn.pretty_signature, param.name, format(v) or v))
break
end
depths[j] = depth
-- set
variant_args[param.full_name] = val
-- default: evaluate once function is selected
-- there's no need to type check because the type constraint is already the default value's type, because of syntax
elseif param.default then
variant_args[param.full_name] = { type = "pending definition", value = { expression = param.default, source = fn.source } }
else
ok = false
table.insert(tried_function_error_messages, ("%s: missing mandatory argument %q in function %q call"):format(fn.pretty_signature, param.name, fn.name))
break
end
end
-- check for unused arguments
if ok then
for key, arg in pairs(args) do
if not used_args[key] then
ok = false
if arg.type == "pair" and arg.value[1].type == "string" then
table.insert(tried_function_error_messages, ("%s: unexpected %s argument"):format(fn.pretty_signature, arg.value[1].value))
else
table.insert(tried_function_error_messages, ("%s: unexpected argument in position %s"):format(fn.pretty_signature, i))
end
break
end
end
end
-- assignment arg
if ok and exp.assignment then
-- check type constraint
local param = fn.assignment
local depth, err = check_constraint(state, param.full_name, assignment)
if not depth then
ok = false
local v = state.variable_metadata[param.full_name].constraint.value
table.insert(tried_function_error_messages, ("%s: argument %s is not of expected type %s"):format(fn.pretty_signature, param.name, format(v) or v))
end
depths.assignment = depth
-- set
variant_args[param.full_name] = assignment
end
if ok then
if not selected_variant.variant then
selected_variant.depths = depths
selected_variant.variant = fn
selected_variant.args_to_set = variant_args
else
-- check specificity order
local lower
for j, d in ipairs(depths) do
local current_depth = selected_variant.depths[j] or math.huge -- not every arg may be set on every variant (varargs)
if d < current_depth then -- stricly lower, i.e. more specific function
lower = true
break
elseif d > current_depth then -- stricly greater, i.e. less specific function
lower = false
break
end
end
if lower == nil and exp.assignment then -- use assignment if still ambigous
local current_depth = selected_variant.depths.assignment
if depths.assignment < current_depth then -- stricly lower, i.e. more specific function
lower = true
elseif depths.assignment > current_depth then -- stricly greater, i.e. less specific function
lower = false
end
end
if lower then
selected_variant.depths = depths
selected_variant.variant = fn
selected_variant.args_to_set = variant_args
elseif lower == nil then -- equal, ambigous dispatch
return nil, ("function call %q is ambigous; may be at least either:\n\t%s\n\t%s"):format(exp.called_name, fn.pretty_signature, selected_variant.variant.pretty_signature)
end
end
end
end
end
end
-- function successfully selected: run
if selected_variant.variant then
local fn = selected_variant.variant
if fn.type ~= "function" then
return nil, ("unknown function type %q"):format(fn.type)
-- checkpoint: no args and can resume execution
elseif fn.subtype == "checkpoint" then
-- set current checkpoint
local s, e = set_variable(state, fn.parent_resumable.namespace.."🔖", {
type = "function reference",
value = { fn.name }
})
if not s then return nil, e end
-- run checkpoint content, eventually resuming
local r, e = run(state, fn.child, not paren_call)
if not r then return nil, e end
return r
-- other functions
else
local ret
-- push scope
-- NOTE: if error happens between here and scope:pop, will leave the stack a mess
-- should not be an issue since an interpreter is supposed to be discarded after an error, but should change this if we ever
-- add some excepetion handling in anselme at some point
if fn.scoped then
scope:push(state, fn)
end
-- set arguments
for name, val in pairs(selected_variant.args_to_set) do
local s, e = set_variable(state, name, val)
if not s then return nil, e end
end
-- get function vars
local checkpoint, checkpointe
if fn.resumable then
checkpoint, checkpointe = get_variable(state, fn.namespace.."🔖")
if not checkpoint then return nil, checkpointe end
end
-- execute lua functions
-- I guess we could technically skip getting & updating the seen and checkpoints vars since they can't be used from Anselme
-- but it's also kinda fun to known how many time a function was ran
if fn.lua_function then
local lua_fn = fn.lua_function
-- get args
local final_args = {}
for j, param in ipairs(fn.params) do
local v, e = get_variable(state, param.full_name)
if not v then return nil, e end
final_args[j] = v
end
if fn.assignment then
local v, e = get_variable(state, fn.assignment.full_name)
if not v then return nil, e end
final_args[#final_args+1] = v
end
-- execute function
-- raw mode: pass raw anselme values to the Lua function; support return nil, err in case of error
if lua_fn.mode == "raw" then
local r, e = lua_fn.value(unpack(final_args))
if r then
ret = r
else
return nil, ("%s; in Lua function %q"):format(e or "raw function returned nil and no error message", exp.called_name)
end
-- unannotated raw mode: same as raw, but strips custom annotations from the arguments
elseif lua_fn.mode == "unannotated raw" then
-- extract value from custom types
for i, arg in ipairs(final_args) do
if arg.type == "annotated" then
final_args[i] = arg.value[1]
end
end
local r, e = lua_fn.value(unpack(final_args))
if r then
ret = r
else
return nil, ("%s; in Lua function %q"):format(e or "unannotated raw function returned nil and no error message", exp.called_name)
end
-- normal mode: convert args to Lua and convert back Lua value to Anselme
elseif lua_fn.mode == nil then
local l_lua = {}
for _, v in ipairs(final_args) do
local lv, e = to_lua(v, state)
if e then return nil, e end
table.insert(l_lua, lv)
end
local r, e
if _VERSION == "Lua 5.1" and not jit then -- PUC Lua 5.1 doesn't allow yield from a pcall
r, e = true, lua_fn.value(unpack(l_lua))
else
r, e = pcall(lua_fn.value, unpack(l_lua)) -- pcall to produce a more informative error message (instead of full coroutine crash)
end
if r then
ret = from_lua(e)
else
return nil, ("%s; in Lua function %q"):format(e, exp.called_name)
end
else
return nil, ("unknown Lua function mode %q"):format(lua_fn.mode)
end
-- execute anselme functions
else
local e
-- eval function from start
if paren_call or not fn.resumable or checkpoint.type == "nil" then
ret, e = run(state, fn.child)
-- resume at last checkpoint
else
local expr, err = expression(checkpoint.value[1], state, fn.namespace, "resume from checkpoint")
if not expr then return nil, err end
ret, e = eval(state, expr)
end
if not ret then return nil, e end
end
-- for classes: build resulting object
if fn.subtype == "class" and ret and ret.type == "nil" then
ret = {
type = "annotated",
value = {
{
type = "object",
value = {
class = fn.name,
attributes = {}
}
},
{
type = "function reference",
value = { fn.name }
}
}
}
end
-- pop scope
if fn.scoped then
scope:pop(state, fn)
end
-- return value
if not ret then return nil, ("function %q didn't return a value"):format(exp.called_name) end
return ret
end
end
-- no matching function found
local args_txt = {}
for key, arg in pairs(args) do
local s = ""
if type(key) == "string" or (type(key) == "number" and key > last_contiguous_positional) then
s = s .. ("%s="):format(key)
end
s = s .. pretty_type(arg)
table.insert(args_txt, s)
end
local called_name = ("%s(%s)"):format(exp.called_name, table.concat(args_txt, ", "))
if assignment then
called_name = called_name .. " := " .. pretty_type(assignment)
end
return nil, ("no compatible function found for call to %s; potential candidates were:\n\t%s"):format(called_name, table.concat(tried_function_error_messages, "\n\t"))
-- event buffer (internal type, only issued from a text or choice line)
elseif exp.type == "text" then
local l = {}
events:push_buffer(state, l)
local current_tags = tags:current(state)
local v, e = eval_text_callback(state, exp.text, function(text)
events:append(state, "text", { text = text, tags = current_tags })
end)
events:pop_buffer(state)
if not v then return nil, e end
return {
type = "event buffer",
value = l
}
elseif exp.type == "nonpersistent" then
local v, e = eval(state, exp.expression)
if not v then return nil, e end
v.nonpersistent = true
return v
-- pass the value along (internal type, used for variable reference implicit calls)
elseif exp.type == "value passthrough" then
return exp.value
else
return nil, ("unknown expression %q"):format(tostring(exp.type))
end
end
package.loaded[...] = eval
run = require((...):gsub("expression$", "interpreter")).run
expression = require((...):gsub("interpreter%.expression$", "parser.expression"))
flatten_list = require((...):gsub("interpreter%.expression$", "parser.common")).flatten_list
local common = require((...):gsub("expression$", "common"))
to_lua, from_lua, eval_text, truthy, format, pretty_type, get_variable, tags, eval_text_callback, events, set_variable, scope, check_constraint, hash = common.to_lua, common.from_lua, common.eval_text, common.truthy, common.format, common.pretty_type, common.get_variable, common.tags, common.eval_text_callback, common.events, common.set_variable, common.scope, common.check_constraint, common.hash
return eval

234
interpreter/interpreter.lua
View file

@ -1,234 +0,0 @@
local eval
local truthy, merge_state, escape, get_variable, tags, events, set_variable
local run_line, run_block
-- returns var in case of success and there is a return
-- return nil in case of success and there is no return
-- return nil, err in case of error
run_line = function(state, line)
-- store line
state.interpreter.running_line = line
-- line types
if line.type == "condition" then
line.parent_block.last_condition_success = nil
local v, e = eval(state, line.expression)
if not v then return nil, ("%s; at %s"):format(e, line.source) end
if truthy(v) then
line.parent_block.last_condition_success = true
v, e = run_block(state, line.child)
if e then return nil, e end
if v then return v end
end
elseif line.type == "else-condition" then
if not line.parent_block.last_condition_success then
local v, e = eval(state, line.expression)
if not v then return nil, ("%s; at %s"):format(e, line.source) end
if truthy(v) then
line.parent_block.last_condition_success = true
v, e = run_block(state, line.child)
if e then return nil, e end
if v then return v end
end
end
elseif line.type == "while" then
line.parent_block.last_condition_success = nil
local v, e = eval(state, line.expression)
if not v then return nil, ("%s; at %s"):format(e, line.source) end
while truthy(v) do
line.parent_block.last_condition_success = true
v, e = run_block(state, line.child)
if e then return nil, e end
if v then return v end
-- next iteration
v, e = eval(state, line.expression)
if not v then return nil, ("%s; at %s"):format(e, line.source) end
end
elseif line.type == "choice" then
local v, e = events:make_space_for(state, "choice")
if not v then return nil, ("%s; in automatic event flush at %s"):format(e, line.source) end
v, e = eval(state, line.text)
if not v then return nil, ("%s; at %s"):format(e, line.source) end
local l = v.type == "list" and v.value or { v }
-- convert text events to choices
for _, item in ipairs(l) do
if item.type == "event buffer" then
local current_tags = tags:current(state)
local choice_block_state = { tags = current_tags, block = line.child }
local final_buffer = {}
for _, event in ipairs(item.value) do
if event.type == "text" then
-- create new choice block if needed
local last_choice_block = final_buffer[#final_buffer]
if not last_choice_block or last_choice_block.type ~= "choice" then
last_choice_block = { type = "choice", value = {} }
table.insert(final_buffer, last_choice_block)
end
-- create new choice item in choice block if needed
local last_choice = last_choice_block.value[#last_choice_block.value]
if not last_choice then
last_choice = { _state = choice_block_state }
table.insert(last_choice_block.value, last_choice)
end
-- add text to last choice item
for _, txt in ipairs(event.value) do
table.insert(last_choice, txt)
end
else
table.insert(final_buffer, event)
end
end
local iv, ie = events:write_buffer(state, final_buffer)
if not iv then return nil, ("%s; at %s"):format(ie, line.source) end
end
end
elseif line.type == "tag" then
local v, e = eval(state, line.expression)
if not v then return nil, ("%s; at %s"):format(e, line.source) end
tags:push(state, v)
v, e = run_block(state, line.child)
tags:pop(state)
if e then return nil, e end
if v then return v end
elseif line.type == "return" then
local v, e = eval(state, line.expression)
if not v then return nil, ("%s; at %s"):format(e, line.source) end
local cv, ce = run_block(state, line.child)
if ce then return nil, ce end
if cv then return cv end
return v
elseif line.type == "text" then
local v, e = events:make_space_for(state, "text") -- do this before any evaluation start
if not v then return nil, ("%s; in automatic event flush at %s"):format(e, line.source) end
v, e = eval(state, line.text)
if not v then return nil, ("%s; at %s"):format(e, line.source) end
local l = v.type == "list" and v.value or { v }
for _, item in ipairs(l) do
if item.type == "event buffer" then
local iv, ie = events:write_buffer(state, item.value)
if not iv then return nil, ("%s; at %s"):format(ie, line.source) end
end
end
elseif line.type == "flush events" then
local v, e = events:flush(state)
if not v then return nil, ("%s; in event flush at %s"):format(e, line.source) end
elseif line.type == "function" and line.subtype == "checkpoint" then
local reached, reachede = get_variable(state, line.namespace.."🏁")
if not reached then return nil, reachede end
local s, e = set_variable(state, line.namespace.."🏁", {
type = "number",
value = reached.value + 1
})
if not s then return nil, e end
s, e = set_variable(state, line.parent_resumable.namespace.."🔖", {
type = "function reference",
value = { line.name }
})
if not s then return nil, e end
merge_state(state)
else
return nil, ("unknown line type %q; at %s"):format(line.type, line.source)
end
end
-- returns var in case of success and there is a return
-- return nil in case of success and there is no return
-- return nil, err in case of error
run_block = function(state, block, resume_from_there, i, j)
i = i or 1
local max = math.min(#block, j or math.huge)
while i <= max do
local line = block[i]
local skip = false
-- skip current choice block if enabled
if state.interpreter.skip_choices_until_flush and line.type == "choice" then
skip = true
end
-- run line
if not skip then
local v, e = run_line(state, line)
if e then return nil, e end
if v then return v end
end
i = i + 1
end
-- if we reach the end of a checkpoint block (we are resuming execution from a checkpoint), merge state
if block.parent_line and block.parent_line.type == "function" and block.parent_line.subtype == "checkpoint" then
merge_state(state)
end
-- go up hierarchy if asked to resume
-- will stop at resumable function boundary
-- if parent is a choice, will ignore choices that belong to the same block (like the whole block was executed naturally from a higher parent)
-- if parent if a condition, will mark it as a success (skipping following else-conditions) (for the same reasons as for choices)
-- if parent pushed a tag, will pop it (tags from parents are added to the stack in run())
if resume_from_there and block.parent_line and not block.parent_line.resumable then
local parent_line = block.parent_line
if parent_line.type == "choice" then
state.interpreter.skip_choices_until_flush = true
elseif parent_line.type == "condition" or parent_line.type == "else-condition" then
parent_line.parent_block.last_condition_success = true
end
if parent_line.type == "tag" then
tags:pop(state)
end
local v, e = run_block(state, parent_line.parent_block, resume_from_there, parent_line.parent_position+1)
if e then return nil, e end
if v then return v end
end
end
-- returns var in case of success
-- return nil, err in case of error
local function run(state, block, resume_from_there, i, j)
-- restore tags from parents when resuming
local tags_len = tags:len(state)
if resume_from_there then
local tags_to_add = {}
-- go up in hierarchy in ascending order until resumable function boundary
local parent_line = block.parent_line
while parent_line and not parent_line.resumable do
if parent_line.type == "tag" then
local v, e = eval(state, parent_line.expression)
if not v then return nil, ("%s; at %s"):format(e, parent_line.source) end
table.insert(tags_to_add, v)
end
parent_line = parent_line.parent_block.parent_line
end
-- re-add tag in desceding order
for k=#tags_to_add, 1, -1 do
tags:push(state, tags_to_add[k])
end
end
-- run
local v, e = run_block(state, block, resume_from_there, i, j)
-- return to previous tag state
-- when resuming is done, tag stack pop when exiting the tag block
-- stray elements may be left on the stack if there is a return before we go up all the tag blocks, so we trim them
if resume_from_there then
tags:trim(state, tags_len)
end
-- return
if e then return nil, e end
if v then
return v
else
-- default no return value
return {
type = "nil",
value = nil
}
end
end
local interpreter = {
run = run,
run_block = run_block,
run_line = run_line
}
package.loaded[...] = interpreter
eval = require((...):gsub("interpreter$", "expression"))
local common = require((...):gsub("interpreter$", "common"))
truthy, merge_state, tags, get_variable, events, set_variable = common.truthy, common.merge_state, common.tags, common.get_variable, common.events, common.set_variable
escape = require((...):gsub("interpreter%.interpreter$", "parser.common")).escape
return interpreter

100
lib/ansicolors.lua Normal file
View file

@ -0,0 +1,100 @@
-- ansicolors.lua v1.0.2 (2012-08)
-- Copyright (c) 2009 Rob Hoelz <rob@hoelzro.net>
-- Copyright (c) 2011 Enrique García Cota <enrique.garcia.cota@gmail.com>
--
-- Permission is hereby granted, free of charge, to any person obtaining a copy
-- of this software and associated documentation files (the "Software"), to deal
-- in the Software without restriction, including without limitation the rights
-- to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-- copies of the Software, and to permit persons to whom the Software is
-- furnished to do so, subject to the following conditions:
--
-- The above copyright notice and this permission notice shall be included in
-- all copies or substantial portions of the Software.
--
-- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-- IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-- AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-- OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-- THE SOFTWARE.
-- support detection
local function isWindows()
return type(package) == 'table' and type(package.config) == 'string' and package.config:sub(1,1) == '\\'
end
local supported = not isWindows()
if isWindows() then supported = os.getenv("ANSICON") end
local keys = {
-- reset
reset = 0,
-- misc
bright = 1,
dim = 2,
underline = 4,
blink = 5,
reverse = 7,
hidden = 8,
-- foreground colors
black = 30,
red = 31,
green = 32,
yellow = 33,
blue = 34,
magenta = 35,
cyan = 36,
white = 37,
-- background colors
blackbg = 40,
redbg = 41,
greenbg = 42,
yellowbg = 43,
bluebg = 44,
magentabg = 45,
cyanbg = 46,
whitebg = 47
}
local escapeString = string.char(27) .. '[%dm'
local function escapeNumber(number)
return escapeString:format(number)
end
local function escapeKeys(str)
if not supported then return "" end
local buffer = {}
local number
for word in str:gmatch("%w+") do
number = keys[word]
assert(number, "Unknown key: " .. word)
table.insert(buffer, escapeNumber(number) )
end
return table.concat(buffer)
end
local function replaceCodes(str)
str = string.gsub(str,"(%%{(.-)})", function(_, str) return escapeKeys(str) end )
return str
end
-- public
local function ansicolors( str )
str = tostring(str or '')
return replaceCodes('%{reset}' .. str .. '%{reset}')
end
return setmetatable({noReset = replaceCodes}, {__call = function (_, str) return ansicolors (str) end})

753
lib/binser.lua Normal file
View file

@ -0,0 +1,753 @@
-- binser.lua
--[[
Copyright (c) 2016-2019 Calvin Rose
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
]]
local assert = assert
local error = error
local select = select
local pairs = pairs
local getmetatable = getmetatable
local setmetatable = setmetatable
local type = type
local loadstring = loadstring or load
local concat = table.concat
local char = string.char
local byte = string.byte
local format = string.format
local sub = string.sub
local dump = string.dump
local floor = math.floor
local frexp = math.frexp
local unpack = unpack or table.unpack
local huge = math.huge
-- Lua 5.3 frexp polyfill
-- From https://github.com/excessive/cpml/blob/master/modules/utils.lua
if not frexp then
local log, abs, floor = math.log, math.abs, math.floor
local log2 = log(2)
frexp = function(x)
if x == 0 then return 0, 0 end
local e = floor(log(abs(x)) / log2 + 1)
return x / 2 ^ e, e
end
end
local function pack(...)
return {...}, select("#", ...)
end
local function not_array_index(x, len)
return type(x) ~= "number" or x < 1 or x > len or x ~= floor(x)
end
local function type_check(x, tp, name)
assert(type(x) == tp,
format("Expected parameter %q to be of type %q.", name, tp))
end
local bigIntSupport = false
local isInteger
if math.type then -- Detect Lua 5.3
local mtype = math.type
bigIntSupport = loadstring[[
local char = string.char
return function(n)
local nn = n < 0 and -(n + 1) or n
local b1 = nn // 0x100000000000000
local b2 = nn // 0x1000000000000 % 0x100
local b3 = nn // 0x10000000000 % 0x100
local b4 = nn // 0x100000000 % 0x100
local b5 = nn // 0x1000000 % 0x100
local b6 = nn // 0x10000 % 0x100
local b7 = nn // 0x100 % 0x100
local b8 = nn % 0x100
if n < 0 then
b1, b2, b3, b4 = 0xFF - b1, 0xFF - b2, 0xFF - b3, 0xFF - b4
b5, b6, b7, b8 = 0xFF - b5, 0xFF - b6, 0xFF - b7, 0xFF - b8
end
return char(212, b1, b2, b3, b4, b5, b6, b7, b8)
end]]()
isInteger = function(x)
return mtype(x) == 'integer'
end
else
isInteger = function(x)
return floor(x) == x
end
end
-- Copyright (C) 2012-2015 Francois Perrad.
-- number serialization code modified from https://github.com/fperrad/lua-MessagePack
-- Encode a number as a big-endian ieee-754 double, big-endian signed 64 bit integer, or a small integer
local function number_to_str(n)
if isInteger(n) then -- int
if n <= 100 and n >= -27 then -- 1 byte, 7 bits of data
return char(n + 27)
elseif n <= 8191 and n >= -8192 then -- 2 bytes, 14 bits of data
n = n + 8192
return char(128 + (floor(n / 0x100) % 0x100), n % 0x100)
elseif bigIntSupport then
return bigIntSupport(n)
end
end
local sign = 0
if n < 0.0 then
sign = 0x80
n = -n
end
local m, e = frexp(n) -- mantissa, exponent
if m ~= m then
return char(203, 0xFF, 0xF8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00)
elseif m == huge then
if sign == 0 then
return char(203, 0x7F, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00)
else
return char(203, 0xFF, 0xF0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00)
end
elseif m == 0.0 and e == 0 then
return char(0xCB, sign, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00)
end
e = e + 0x3FE
if e < 1 then -- denormalized numbers
m = m * 2 ^ (52 + e)
e = 0
else
m = (m * 2 - 1) * 2 ^ 52
end
return char(203,
sign + floor(e / 0x10),
(e % 0x10) * 0x10 + floor(m / 0x1000000000000),
floor(m / 0x10000000000) % 0x100,
floor(m / 0x100000000) % 0x100,
floor(m / 0x1000000) % 0x100,
floor(m / 0x10000) % 0x100,
floor(m / 0x100) % 0x100,
m % 0x100)
end
-- Copyright (C) 2012-2015 Francois Perrad.
-- number deserialization code also modified from https://github.com/fperrad/lua-MessagePack
local function number_from_str(str, index)
local b = byte(str, index)
if not b then error("Expected more bytes of input.") end
if b < 128 then
return b - 27, index + 1
elseif b < 192 then
local b2 = byte(str, index + 1)
if not b2 then error("Expected more bytes of input.") end
return b2 + 0x100 * (b - 128) - 8192, index + 2
end
local b1, b2, b3, b4, b5, b6, b7, b8 = byte(str, index + 1, index + 8)
if (not b1) or (not b2) or (not b3) or (not b4) or
(not b5) or (not b6) or (not b7) or (not b8) then
error("Expected more bytes of input.")
end
if b == 212 then
local flip = b1 >= 128
if flip then -- negative
b1, b2, b3, b4 = 0xFF - b1, 0xFF - b2, 0xFF - b3, 0xFF - b4
b5, b6, b7, b8 = 0xFF - b5, 0xFF - b6, 0xFF - b7, 0xFF - b8
end
local n = ((((((b1 * 0x100 + b2) * 0x100 + b3) * 0x100 + b4) *
0x100 + b5) * 0x100 + b6) * 0x100 + b7) * 0x100 + b8
if flip then
return (-n) - 1, index + 9
else
return n, index + 9
end
end
if b ~= 203 then
error("Expected number")
end
local sign = b1 > 0x7F and -1 or 1
local e = (b1 % 0x80) * 0x10 + floor(b2 / 0x10)
local m = ((((((b2 % 0x10) * 0x100 + b3) * 0x100 + b4) * 0x100 + b5) * 0x100 + b6) * 0x100 + b7) * 0x100 + b8
local n
if e == 0 then
if m == 0 then
n = sign * 0.0
else
n = sign * (m / 2 ^ 52) * 2 ^ -1022
end
elseif e == 0x7FF then
if m == 0 then
n = sign * huge
else
n = 0
end
else
n = sign * (1.0 + m / 2 ^ 52) * 2 ^ (e - 0x3FF)
end
return n, index + 9
end
local function newbinser()
-- unique table key for getting next value
local NEXT = {}
local CTORSTACK = {}
-- NIL = 202
-- FLOAT = 203
-- TRUE = 204
-- FALSE = 205
-- STRING = 206
-- TABLE = 207
-- REFERENCE = 208
-- CONSTRUCTOR = 209
-- FUNCTION = 210
-- RESOURCE = 211
-- INT64 = 212
-- TABLE WITH META = 213
local mts = {}
local ids = {}
local serializers = {}
local deserializers = {}
local resources = {}
local resources_by_name = {}
local types = {}
types["nil"] = function(x, visited, accum)
accum[#accum + 1] = "\202"
end
function types.number(x, visited, accum)
accum[#accum + 1] = number_to_str(x)
end
function types.boolean(x, visited, accum)
accum[#accum + 1] = x and "\204" or "\205"
end
function types.string(x, visited, accum)
local alen = #accum
if visited[x] then
accum[alen + 1] = "\208"
accum[alen + 2] = number_to_str(visited[x])
else
visited[x] = visited[NEXT]
visited[NEXT] = visited[NEXT] + 1
accum[alen + 1] = "\206"
accum[alen + 2] = number_to_str(#x)
accum[alen + 3] = x
end
end
local function check_custom_type(x, visited, accum)
local res = resources[x]
if res then
accum[#accum + 1] = "\211"
types[type(res)](res, visited, accum)
return true
end
local mt = getmetatable(x)
local id = mt and ids[mt]
if id then
local constructing = visited[CTORSTACK]
if constructing[x] then
error("Infinite loop in constructor.")
end
constructing[x] = true
accum[#accum + 1] = "\209"
types[type(id)](id, visited, accum)
local args, len = pack(serializers[id](x))
accum[#accum + 1] = number_to_str(len)
for i = 1, len do
local arg = args[i]
types[type(arg)](arg, visited, accum)
end
visited[x] = visited[NEXT]
visited[NEXT] = visited[NEXT] + 1
-- We finished constructing
constructing[x] = nil
return true
end
end
function types.userdata(x, visited, accum)
if visited[x] then
accum[#accum + 1] = "\208"
accum[#accum + 1] = number_to_str(visited[x])
else
if check_custom_type(x, visited, accum) then return end
error("Cannot serialize this userdata.")
end
end
function types.table(x, visited, accum)
if visited[x] then
accum[#accum + 1] = "\208"
accum[#accum + 1] = number_to_str(visited[x])
else
if check_custom_type(x, visited, accum) then return end
visited[x] = visited[NEXT]
visited[NEXT] = visited[NEXT] + 1
local xlen = #x
local mt = getmetatable(x)
if mt then
accum[#accum + 1] = "\213"
types.table(mt, visited, accum)
else
accum[#accum + 1] = "\207"
end
accum[#accum + 1] = number_to_str(xlen)
for i = 1, xlen do
local v = x[i]
types[type(v)](v, visited, accum)
end
local key_count = 0
for k in pairs(x) do
if not_array_index(k, xlen) then
key_count = key_count + 1
end
end
accum[#accum + 1] = number_to_str(key_count)
for k, v in pairs(x) do
if not_array_index(k, xlen) then
types[type(k)](k, visited, accum)
types[type(v)](v, visited, accum)
end
end
end
end
types["function"] = function(x, visited, accum)
if visited[x] then
accum[#accum + 1] = "\208"
accum[#accum + 1] = number_to_str(visited[x])
else
if check_custom_type(x, visited, accum) then return end
visited[x] = visited[NEXT]
visited[NEXT] = visited[NEXT] + 1
local str = dump(x)
accum[#accum + 1] = "\210"
accum[#accum + 1] = number_to_str(#str)
accum[#accum + 1] = str
end
end
types.cdata = function(x, visited, accum)
if visited[x] then
accum[#accum + 1] = "\208"
accum[#accum + 1] = number_to_str(visited[x])
else
if check_custom_type(x, visited, #accum) then return end
error("Cannot serialize this cdata.")
end
end
types.thread = function() error("Cannot serialize threads.") end
local function deserialize_value(str, index, visited)
local t = byte(str, index)
if not t then return nil, index end
if t < 128 then
return t - 27, index + 1
elseif t < 192 then
local b2 = byte(str, index + 1)
if not b2 then error("Expected more bytes of input.") end
return b2 + 0x100 * (t - 128) - 8192, index + 2
elseif t == 202 then
return nil, index + 1
elseif t == 203 or t == 212 then
return number_from_str(str, index)
elseif t == 204 then
return true, index + 1
elseif t == 205 then
return false, index + 1
elseif t == 206 then
local length, dataindex = number_from_str(str, index + 1)
local nextindex = dataindex + length
if not (length >= 0) then error("Bad string length") end
if #str < nextindex - 1 then error("Expected more bytes of string") end
local substr = sub(str, dataindex, nextindex - 1)
visited[#visited + 1] = substr
return substr, nextindex
elseif t == 207 or t == 213 then
local mt, count, nextindex
local ret = {}
visited[#visited + 1] = ret
nextindex = index + 1
if t == 213 then
mt, nextindex = deserialize_value(str, nextindex, visited)
if type(mt) ~= "table" then error("Expected table metatable") end
end
count, nextindex = number_from_str(str, nextindex)
for i = 1, count do
local oldindex = nextindex
ret[i], nextindex = deserialize_value(str, nextindex, visited)
if nextindex == oldindex then error("Expected more bytes of input.") end
end
count, nextindex = number_from_str(str, nextindex)
for i = 1, count do
local k, v
local oldindex = nextindex
k, nextindex = deserialize_value(str, nextindex, visited)
if nextindex == oldindex then error("Expected more bytes of input.") end
oldindex = nextindex
v, nextindex = deserialize_value(str, nextindex, visited)
if nextindex == oldindex then error("Expected more bytes of input.") end
if k == nil then error("Can't have nil table keys") end
ret[k] = v
end
if mt then setmetatable(ret, mt) end
return ret, nextindex
elseif t == 208 then
local ref, nextindex = number_from_str(str, index + 1)
return visited[ref], nextindex
elseif t == 209 then
local count
local name, nextindex = deserialize_value(str, index + 1, visited)
count, nextindex = number_from_str(str, nextindex)
local args = {}
for i = 1, count do
local oldindex = nextindex
args[i], nextindex = deserialize_value(str, nextindex, visited)
if nextindex == oldindex then error("Expected more bytes of input.") end
end
if not name or not deserializers[name] then
error(("Cannot deserialize class '%s'"):format(tostring(name)))
end
local ret = deserializers[name](unpack(args))
visited[#visited + 1] = ret
return ret, nextindex
elseif t == 210 then
local length, dataindex = number_from_str(str, index + 1)
local nextindex = dataindex + length
if not (length >= 0) then error("Bad string length") end
if #str < nextindex - 1 then error("Expected more bytes of string") end
local ret = loadstring(sub(str, dataindex, nextindex - 1))
visited[#visited + 1] = ret
return ret, nextindex
elseif t == 211 then
local resname, nextindex = deserialize_value(str, index + 1, visited)
if resname == nil then error("Got nil resource name") end
local res = resources_by_name[resname]
if res == nil then
error(("No resources found for name '%s'"):format(tostring(resname)))
end
return res, nextindex
else
error("Could not deserialize type byte " .. t .. ".")
end
end
local function serialize(...)
local visited = {[NEXT] = 1, [CTORSTACK] = {}}
local accum = {}
for i = 1, select("#", ...) do
local x = select(i, ...)
types[type(x)](x, visited, accum)
end
return concat(accum)
end
local function make_file_writer(file)
return setmetatable({}, {
__newindex = function(_, _, v)
file:write(v)
end
})
end
local function serialize_to_file(path, mode, ...)
local file, err = io.open(path, mode)
assert(file, err)
local visited = {[NEXT] = 1, [CTORSTACK] = {}}
local accum = make_file_writer(file)
for i = 1, select("#", ...) do
local x = select(i, ...)
types[type(x)](x, visited, accum)
end
-- flush the writer
file:flush()
file:close()
end
local function writeFile(path, ...)
return serialize_to_file(path, "wb", ...)
end
local function appendFile(path, ...)
return serialize_to_file(path, "ab", ...)
end
local function deserialize(str, index)
assert(type(str) == "string", "Expected string to deserialize.")
local vals = {}
index = index or 1
local visited = {}
local len = 0
local val
while true do
local nextindex
val, nextindex = deserialize_value(str, index, visited)
if nextindex > index then
len = len + 1
vals[len] = val
index = nextindex
else
break
end
end
return vals, len
end
local function deserializeN(str, n, index)
assert(type(str) == "string", "Expected string to deserialize.")
n = n or 1
assert(type(n) == "number", "Expected a number for parameter n.")
assert(n > 0 and floor(n) == n, "N must be a poitive integer.")
local vals = {}
index = index or 1
local visited = {}
local len = 0
local val
while len < n do
local nextindex
val, nextindex = deserialize_value(str, index, visited)
if nextindex > index then
len = len + 1
vals[len] = val
index = nextindex
else
break
end
end
vals[len + 1] = index
return unpack(vals, 1, n + 1)
end
local function readFile(path)
local file, err = io.open(path, "rb")
assert(file, err)
local str = file:read("*all")
file:close()
return deserialize(str)
end
-- Resources
local function registerResource(resource, name)
type_check(name, "string", "name")
assert(not resources[resource],
"Resource already registered.")
assert(not resources_by_name[name],
format("Resource %q already exists.", name))
resources_by_name[name] = resource
resources[resource] = name
return resource
end
local function unregisterResource(name)
type_check(name, "string", "name")
assert(resources_by_name[name], format("Resource %q does not exist.", name))
local resource = resources_by_name[name]
resources_by_name[name] = nil
resources[resource] = nil
return resource
end
-- Templating
local function normalize_template(template)
local ret = {}
for i = 1, #template do
ret[i] = template[i]
end
local non_array_part = {}
-- The non-array part of the template (nested templates) have to be deterministic, so they are sorted.
-- This means that inherently non deterministicly sortable keys (tables, functions) should NOT be used
-- in templates. Looking for way around this.
for k in pairs(template) do
if not_array_index(k, #template) then
non_array_part[#non_array_part + 1] = k
end
end
table.sort(non_array_part)
for i = 1, #non_array_part do
local name = non_array_part[i]
ret[#ret + 1] = {name, normalize_template(template[name])}
end
return ret
end
local function templatepart_serialize(part, argaccum, x, len)
local extras = {}
local extracount = 0
for k, v in pairs(x) do
extras[k] = v
extracount = extracount + 1
end
for i = 1, #part do
local name
if type(part[i]) == "table" then
name = part[i][1]
len = templatepart_serialize(part[i][2], argaccum, x[name], len)
else
name = part[i]
len = len + 1
argaccum[len] = x[part[i]]
end
if extras[name] ~= nil then
extracount = extracount - 1
extras[name] = nil
end
end
if extracount > 0 then
argaccum[len + 1] = extras
else
argaccum[len + 1] = nil
end
return len + 1
end
local function templatepart_deserialize(ret, part, values, vindex)
for i = 1, #part do
local name = part[i]
if type(name) == "table" then
local newret = {}
ret[name[1]] = newret
vindex = templatepart_deserialize(newret, name[2], values, vindex)
else
ret[name] = values[vindex]
vindex = vindex + 1
end
end
local extras = values[vindex]
if extras then
for k, v in pairs(extras) do
ret[k] = v
end
end
return vindex + 1
end
local function template_serializer_and_deserializer(metatable, template)
return function(x)
local argaccum = {}
local len = templatepart_serialize(template, argaccum, x, 0)
return unpack(argaccum, 1, len)
end, function(...)
local ret = {}
local args = {...}
templatepart_deserialize(ret, template, args, 1)
return setmetatable(ret, metatable)
end
end
-- Used to serialize classes withh custom serializers and deserializers.
-- If no _serialize or _deserialize (or no _template) value is found in the
-- metatable, then the metatable is registered as a resources.
local function register(metatable, name, serialize, deserialize)
if type(metatable) == "table" then
name = name or metatable.name
serialize = serialize or metatable._serialize
deserialize = deserialize or metatable._deserialize
if (not serialize) or (not deserialize) then
if metatable._template then
-- Register as template
local t = normalize_template(metatable._template)
serialize, deserialize = template_serializer_and_deserializer(metatable, t)
else
-- Register the metatable as a resource. This is semantically
-- similar and more flexible (handles cycles).
registerResource(metatable, name)
return
end
end
elseif type(metatable) == "string" then
name = name or metatable
end
type_check(name, "string", "name")
type_check(serialize, "function", "serialize")
type_check(deserialize, "function", "deserialize")
assert((not ids[metatable]) and (not resources[metatable]),
"Metatable already registered.")
assert((not mts[name]) and (not resources_by_name[name]),
("Name %q already registered."):format(name))
mts[name] = metatable
ids[metatable] = name
serializers[name] = serialize
deserializers[name] = deserialize
return metatable
end
local function unregister(item)
local name, metatable
if type(item) == "string" then -- assume name
name, metatable = item, mts[item]
else -- assume metatable
name, metatable = ids[item], item
end
type_check(name, "string", "name")
mts[name] = nil
if (metatable) then
resources[metatable] = nil
ids[metatable] = nil
end
serializers[name] = nil
deserializers[name] = nil
resources_by_name[name] = nil;
return metatable
end
local function registerClass(class, name)
name = name or class.name
if class.__instanceDict then -- middleclass
register(class.__instanceDict, name)
else -- assume 30log or similar library
register(class, name)
end
return class
end
return {
VERSION = "0.0-8",
-- aliases
s = serialize,
d = deserialize,
dn = deserializeN,
r = readFile,
w = writeFile,
a = appendFile,
serialize = serialize,
deserialize = deserialize,
deserializeN = deserializeN,
readFile = readFile,
writeFile = writeFile,
appendFile = appendFile,
register = register,
unregister = unregister,
registerResource = registerResource,
unregisterResource = unregisterResource,
registerClass = registerClass,
newbinser = newbinser
}
end
return newbinser()

109
notes.txt
View file

@ -1,109 +0,0 @@
# Symbol selection
Anselme favor symbols over keywords, as it make translation easier.
We prefer to use symbols available on a standard US keyboard as it often is the lowest common denominator.
As we want to be able to write identifiers with little restriction, we try to only use symbols which are unlikely to appear naturally in a name.
Considering Anselme is aimed to people with a light programming introduction, are safe to use for syntax purposes:
* Diacritics (should be safe when used on their own): ~`^
* Usual mathematical symbols (should be safe to use): +-=<>/
* Unusual punctuation / main use is already programming (should be safe to use): []*{}|\_
* Usual punctuation used to separate parts of a sentence (should be safe to use): !?.,;:()
* Signs (could be used in a name, but also common programming symbols): @&$#%
* Usual punctuation (could be used in a name): '"
In the end, we decided to reserve all of those except '.
Using other unicode symbols may be also alright, but there also should be a way to only use these symbols.
Reserved symbols that are still not used in expressions: `\_?@$
Reserved symbols that are still not used as a line type: `^+-=</[]*{}|\_!?.,;)"&%$
# Code Q&A
* What does "fqm" means?
It means "fully qualified matriname", which is the same as a fully qualified name, but considers the hierarchy to be mostly mother-daugher based.
It has nothing to do with the fact I'm inept at writing acronyms and realized I wrote it wrong after using it for a whole year.
* Why are built-in anselme scripts stored in Lua files?
I believe it was to avoid reimplementing the require() file search algorithm myself.
* What's a "variant"?
One of the different forms of a same function with a given fqm. No idea why I chose "variant".
* Why emojis?
They're kinda language independent I guess. I have no idea.
* Why?
I still have no idea.
# Other
Broad goals and ideas that may never be implemented. Mostly used as personal post-it notes.
TODO: support parametric types in contraints: list(number)
should then also allow polymorphic constraint like
$ fn(l::list('a), x::'a)
(push a constraint context when checking compatibility)
or more generic constraints? allow custom functions to check constraints, etc (performance?)
TODO: type system is not super nice to use.
UPDATE: tried to implement a static type system, ain't worth it. Would require major refactoring to go full static with good type inference. The language is supposed to allow to not have to worry about low level stuff, so no deal if the type inference isn't good. Thank you multiple dispatch for making everything complicated (i still love you though)... Well, if I ever reimplement Anselme, let's thank the Julia devs for doing the hard work: https://docs.julialang.org/en/v1/devdocs/inference/
TODO: some sensible way to capture text event in string litterals (string interpolation/subtexts)? -> meh, this means we can capture choice events, and choice events contain their code block, and we don't want to store code blocks in the save file (as code can be updated/translated/whatever)
ignoring choice events, we might be able to use subtexts in string litterals; using the same code for text lines and text litterals? we would lose tags...
-> no. would break ability to switch between two translations of the script as the save would contain the text events from the previous language.
TODO: simplify language, it is much too complicated. Less line types? (var def, func, checkpoint, tag). Rewrite some ad hoc syntax using the expression system?
TODO: fn/checkpoint/tag: maybe consider them a regular func call that takes children as arg; can keep compatibility using $/§ as shortcut for the actual call.
would allow more flexibility esp. for tags...
a func def would be:
:a = $
stuff
but then args?
:a = $(args)
stuff
how are children passed on? overloading? -> means a code block would be passed as a value, how to avoid it ending up in the save file?
if not possible, at least make checkpoint or fn defined using the other or some superset... -> checkpoints defined using fn
OK for tag though: pushtag/poptag fn:
# color:red
a
translate to something like
~ tag.push(color:red)
a
~ tag.pop()
TODO: make language simple enough to be able to reimplement it in, say, nim. Especially the AST interpreter (we could precompile a lot of stuff...)
TODO: test reacheability of script paths + visualization of different branches the script can take. For one of those overarching story visualization thingy.
TODO: redisign the checkpoint system to work better when used with parallel scripts (if both change the same variable, will be overwritten); not sure how to do that, would need some complicated conflict resolution code or something like CRDT...
TODO: redisign a static type checking system
If we want to go full gradual typing, it would help to:
* add type anotation+type check to functions return ($ f()::number) -> there's a lot of function calls, so probably checked at compiling only
* enforce some restrictions on type (assume they're constant/sublanguage, not full expressions)
* make some tuple/list distinction (homogenous/heterogenous types) as right now index operations are a type roulette. Or type annotate lists using some parametric type.
Advantages:
* can be used for better static variant selection; if everything is type annotated, selection could be restricted to a single function
Disadvantages:
* idk if it's worth the trouble
* could do something like `$ ()(l::list(?), i::number)::?`, but then can't return nil on not found...
TODO: write a translation guide/simplify translation process
TODO: make injection nicer. Some decorator-like syntax? to select specific functions to inject to
TODO: allow multiple aliases for a single identifier?
TODO: closures. Especially for when returning a subfunction from a scoped variable.

38
parser/Source.lua Normal file
View file

@ -0,0 +1,38 @@
local class = require("class")
local Source
Source = class {
name = "?",
line = -1,
position = -1,
init = function(self, name, line, position)
self.name = name
self.line = line
self.position = position
end,
increment = function(self, n, ...)
self.position = self.position + n
end,
count = function(self, capture, ...)
self:increment(utf8.len(capture))
return capture, ...
end,
consume = function(self, capture, ...)
self:increment(utf8.len(capture))
return ...
end,
clone = function(self)
return Source:new(self.name, self.line, self.position)
end,
set = function(self, other)
self.name, self.line, self.position = other.name, other.line, other.position
end,
__tostring = function(self)
return ("%s:%s:%s"):format(self.name, self.line, self.position)
end
}
return Source

65
parser/code_to_tree.lua Normal file
View file

@ -0,0 +1,65 @@
--- transform raw code string into a nested tree of lines
local Source = require("parser.Source")
local function indented_to_tree(indented)
local tree = {}
local current_parent = tree
local current_level = 0
local last_line_empty = nil
for _, l in ipairs(indented) do
-- indentation of empty line is determined using the next line
-- (consecutive empty lines are merged into one)
if l.content == "" then
last_line_empty = l
else
-- raise indentation
if l.level > current_level then
if #current_parent == 0 then -- can't add children to nil
error(("invalid indentation; at %s"):format(l.source))
end
current_parent = current_parent[#current_parent]
current_level = l.level
-- lower indentation
elseif l.level < current_level then
current_parent = tree
current_level = 0
while current_level < l.level do -- find correct level starting back from the root
current_parent = current_parent[#current_parent]
current_level = current_parent[1].level
end
if current_level ~= l.level then
error(("invalid indentation; at %s"):format(l.source))
end
end
-- add line
if last_line_empty then
last_line_empty.level = current_level
table.insert(current_parent, last_line_empty)
last_line_empty = nil
end
table.insert(current_parent, l)
end
end
return tree
end
local function code_to_indented(code, source_name)
local indented = {}
local i = 1
for line in (code.."\n"):gmatch("(.-)\n") do
local indent, rem = line:match("^(%s*)(.-)$")
local indent_len = utf8.len(indent)
table.insert(indented, { level = indent_len, content = rem, source = Source:new(source_name, i, 1+indent_len) })
i = i + 1
end
return indented
end
return function(code, source_name)
return indented_to_tree(code_to_indented(code, source_name or "?"))
end

342
parser/common.lua
View file

@ -1,342 +0,0 @@
local expression
local escapeCache = {}
local common
--- rewrite name to use defined aliases (under namespace only)
-- namespace should not contain aliases
-- returns the final fqm
local replace_aliases = function(aliases, namespace, name)
local name_list = common.split(name)
local prefix = namespace
for i=1, #name_list, 1 do -- search alias for each part of the fqm
local n = ("%s%s%s"):format(prefix, prefix == "" and "" or ".", name_list[i])
if aliases[n] then
prefix = aliases[n]
else
prefix = n
end
end
return prefix
end
local disallowed_set = ("~`^+-=<>/[]*{}|\\_!?,;:()\"@&$#%"):gsub("[^%w]", "%%%1")
common = {
--- valid identifier pattern
identifier_pattern = "%s*[^0-9%s"..disallowed_set.."][^"..disallowed_set.."]*",
-- names allowed for a function that aren't valid identifiers, mainly for overloading operators
special_functions_names = {
-- operators not included here and why:
-- * assignment operators (:=, +=, -=, //=, /=, *=, %=, ^=): handled with its own syntax (function assignment)
-- * list operator (,): is used when calling every functions, sounds like more trouble than it's worth
-- * |, &, ~? and ~ operators: are lazy and don't behave like regular functions
-- * # operator: need to set tag state _before_ evaluating the left arg
-- prefix unop
"-_", "!_",
"&_",
-- binop
"_;_",
"_=_", "_:_",
"_!=_", "_==_", "_>=_", "_<=_", "_<_", "_>_",
"_+_", "_-_",
"_*_", "_//_", "_/_", "_%_",
"_::_",
"_^_",
"_!_",
"_._",
-- suffix unop
"_;",
"_!",
-- special
"()",
"{}"
},
-- escapement code and their value in strings
-- only includes the "special" escape codes, as the generic \. -> . is handled by default in parse_text
-- I don't think there's a point in supporting form feed, carriage return, and other printer and terminal related codes
string_escapes = {
["\\\\"] = "\\",
["\\n"] = "\n",
["\\t"] = "\t",
},
-- list of possible injections and their associated name in vm.state.inject
injections = {
["function start"] = "function_start", ["function end"] = "function_end", ["function return"] = "function_return",
["scoped function start"] = "scoped_function_start", ["scoped function end"] = "scoped_function_end", ["scoped function return"] = "scoped_function_return",
["checkpoint start"] = "checkpoint_start", ["checkpoint end"] = "checkpoint_end",
["class start"] = "class_start", ["class end"] = "class_end"
},
--- escape a string to be used as an exact match pattern
escape = function(str)
if not escapeCache[str] then
escapeCache[str] = str:gsub("[^%w]", "%%%1")
end
return escapeCache[str]
end,
--- trim a string by removing whitespace at the start and end
trim = function(str)
return str:match("^%s*(.-)%s*$")
end,
--- split a string separated by .
split = function(str)
local address = {}
for name in (str.."."):gmatch("(.-)%.") do
table.insert(address, name)
end
return address
end,
--- find a variable/function in a list, going up through the namespace hierarchy
-- will apply aliases
-- returns value, fqm in case of success
-- returns nil, err in case of error
find = function(aliases, list, namespace, name)
if namespace ~= "" then
local ns = common.split(namespace:gsub("%.$", ""))
for i=#ns, 1, -1 do
local current_namespace = table.concat(ns, ".", 1, i)
local fqm = replace_aliases(aliases, current_namespace, name)
if list[fqm] then
return list[fqm], fqm
end
end
end
-- root namespace
name = replace_aliases(aliases, "", name)
if list[name] then
return list[name], name
end
return nil, ("can't find %q in namespace %s"):format(name, namespace)
end,
--- same as find, but return a list of every encoutered possibility
-- returns a list of fqm
find_all = function(aliases, list, namespace, name)
local l = {}
if namespace ~= "" then
local ns = common.split(namespace:gsub("%.$", ""))
for i=#ns, 1, -1 do
local current_namespace = table.concat(ns, ".", 1, i)
local fqm = replace_aliases(aliases, current_namespace, name)
if list[fqm] then
table.insert(l, fqm)
end
end
end
-- root namespace
name = replace_aliases(aliases, "", name)
if list[name] then
table.insert(l, name)
end
return l
end,
--- transform an identifier into a clean version (trim each part)
format_identifier = function(identifier)
local r = identifier:gsub("[^%.]+", function(str)
return common.trim(str)
end)
return r
end,
--- flatten a nested list expression into a list of expressions
flatten_list = function(list, t)
t = t or {}
if list.type == "list" then
table.insert(t, 1, list.right)
common.flatten_list(list.left, t)
else
table.insert(t, 1, list)
end
return t
end,
-- parse interpolated expressions in a text
-- type sets the type of the returned expression (text is in text field)
-- allow_subtext (bool) to enable or not [subtext] support
-- if allow_binops is given, if one of the caracters of allow_binops appear unescaped in the text, it will interpreter a binary operator expression
-- * returns an expression with given type (string by default) and as a value a list of strings and expressions (text elements)
-- * if allow_binops is given, also returns remaining string (if the right expression stop before the end of the text)
-- * nil, err: in case of error
parse_text = function(text, state, namespace, type, allow_binops, allow_subtext, in_subtext)
local l = {}
local text_exp = { type = type, text = l }
local delimiters = ""
if allow_binops then
delimiters = allow_binops
end
if allow_subtext then
delimiters = delimiters .. "%["
end
if in_subtext then
delimiters = delimiters .. "%]"
end
while text:match(("[^{%s]+"):format(delimiters)) do
local t, r = text:match(("^([^{%s]*)(.-)$"):format(delimiters))
-- text
if t ~= "" then
-- handle \{ and binop escape: skip to next { until it's not escaped
while t:match("\\$") and r:match(("^[{%s]"):format(delimiters)) do
local t2, r2 = r:match(("^([{%s][^{%s]*)(.-)$"):format(delimiters, delimiters))
t = t .. t2 -- don't need to remove \ as it will be stripped with other escapes codes 3 lines later
r = r2
end
-- replace escape codes
local escaped = t:gsub("\\.", function(escape)
return common.string_escapes[escape] or escape:match("^\\(.)$")
end)
table.insert(l, escaped)
end
-- expr
if r:match("^{") then
local exp, rem = expression(r:gsub("^{", ""), state, namespace, "interpolated expression")
if not exp then return nil, rem end
if not rem:match("^%s*}") then return nil, ("expected closing } at end of expression before %q"):format(rem) end
-- wrap in format() call
local variant, err = common.find_function(state, namespace, "{}", { type = "parentheses", expression = exp }, true)
if not variant then return variant, err end
-- add to text
table.insert(l, variant)
text = rem:match("^%s*}(.*)$")
-- start subtext
elseif allow_subtext and r:match("^%[") then
local exp, rem = common.parse_text(r:gsub("^%[", ""), state, namespace, "text", "#~", allow_subtext, true)
if not exp then return nil, rem end
if not rem:match("^%]") then return nil, ("expected closing ] at end of subtext before %q"):format(rem) end
-- add to text
table.insert(l, exp)
text = rem:match("^%](.*)$")
-- end subtext
elseif in_subtext and r:match("^%]") then
if allow_binops then
return text_exp, r
else
return text_exp
end
-- binop expression at the end of the text
elseif allow_binops and r:match(("^[%s]"):format(allow_binops)) then
local exp, rem = expression(r, state, namespace, "text binop suffix", nil, text_exp)
if not exp then return nil, rem end
return exp, rem
elseif r == "" then
break
else
error(("unexpected %q at end of text or string"):format(r))
end
end
if allow_binops then
return text_exp, ""
else
return text_exp
end
end,
-- find a list of compatible function variants from a fully qualified name
-- this functions does not guarantee that the returned variants are fully compatible with the given arguments and only performs a pre-selection without the ones which definitely aren't
-- * list of compatible variants: if success
-- * nil, err: if error
find_function_variant_from_fqm = function(fqm, state, arg)
local err = ("compatible function %q variant not found"):format(fqm)
local func = state.functions[fqm]
local args = arg and common.flatten_list(arg) or {}
local variants = {}
for _, variant in ipairs(func) do
local ok = true
-- arity check
-- note: because named args can't be predicted in advance (pairs need to be evaluated), this arity check isn't enough to guarantee a compatible arity
-- (e.g., if there's 3 required args but only provide 3 optional arg in a call, will pass)
local min, max = variant.arity[1], variant.arity[2]
if #args < min or #args > max then
if min == max then
err = ("function %q expected %s arguments but received %s"):format(fqm, min, #args)
else
err = ("function %q expected between %s and %s arguments but received %s"):format(fqm, min, max, #args)
end
ok = false
end
-- done
if ok then
table.insert(variants, variant)
end
end
if #variants > 0 then
return variants
else
return nil, err
end
end,
--- same as find_function_variant_from_fqm, but will search every function from the current namespace and up using find
-- returns directly a function expression in case of success
-- return nil, err otherwise
find_function = function(state, namespace, name, arg, paren_call, implicit_call)
local l = common.find_all(state.aliases, state.functions, namespace, name)
return common.find_function_from_list(state, namespace, name, l, arg, paren_call, implicit_call)
end,
--- same as find_function, but take a list of already found ffqm instead of searching
find_function_from_list = function(state, namespace, name, names, arg, paren_call, implicit_call)
local variants = {}
local err = ("compatible function %q variant not found"):format(name)
local l = common.find_all(state.aliases, state.functions, namespace, name)
for _, ffqm in ipairs(l) do
local found
found, err = common.find_function_variant_from_fqm(ffqm, state, arg)
if found then
for _, v in ipairs(found) do
table.insert(variants, v)
end
end
end
if #variants > 0 then
return {
type = "function call",
called_name = name, -- name of the called function
paren_call = paren_call, -- was call with parantheses?
implicit_call = implicit_call, -- was call implicitely (no ! or parentheses)?
variants = variants, -- list of potential variants
argument = { -- wrap everything in a list literal to simplify later things (otherwise may be nil, single value, list constructor)
type = "map brackets",
expression = arg
}
}
else
return nil, err -- returns last error
end
end,
-- returns the function's signature text
signature = function(fn)
if fn.signature then return fn.signature end
local signature
local function make_param_signature(p)
local sig = p.name
if p.vararg then
sig = sig .. "..."
end
if p.alias then
sig = sig .. ":" .. p.alias
end
if p.type_constraint then
sig = sig .. "::" .. p.type_constraint
end
if p.default then
sig = sig .. "=" .. p.default
end
return sig
end
local arg_sig = {}
for j, p in ipairs(fn.params) do
arg_sig[j] = make_param_signature(p)
end
if fn.assignment then
signature = ("%s(%s) := %s"):format(fn.name, table.concat(arg_sig, ", "), make_param_signature(fn.assignment))
else
signature = ("%s(%s)"):format(fn.name, table.concat(arg_sig, ", "))
end
return signature
end,
-- same as signature, format the signature for displaying to the user and add some debug information
pretty_signature = function(fn)
return ("%s (at %s)"):format(common.signature(fn), fn.source)
end,
}
package.loaded[...] = common
expression = require((...):gsub("common$", "expression"))
return common

558
parser/expression.lua
View file

@ -1,558 +0,0 @@
local identifier_pattern, format_identifier, find, escape, find_function, parse_text, find_all, split, find_function_from_list, preparse
--- binop priority
local binops_prio = {
[1] = { ";" },
[2] = { ":=", "+=", "-=", "//=", "/=", "*=", "%=", "^=" },
[3] = { "," },
[4] = { "~?", "~", "#" },
[5] = { "=", ":" },
[6] = { "|", "&" },
[7] = { "!=", "==", ">=", "<=", "<", ">" },
[8] = { "+", "-" },
[9] = { "*", "//", "/", "%" },
[10] = { "::" },
[11] = {}, -- unary operators
[12] = { "^" },
[13] = { "!" },
[14] = {},
[15] = { "." }
}
local call_priority = 13 -- note: higher priority operators will have to deal with potential functions expressions
local implicit_call_priority = 12.5 -- just below call priority so explicit calls automatically take precedence
local pair_priority = 5
local implicit_multiply_priority = 9.5 -- just above / so 1/2x gives 1/(2x)
-- unop priority
local prefix_unops_prio = {
[1] = {},
[2] = {},
[3] = { "$" },
[4] = {},
[5] = {},
[6] = {},
[7] = {},
[8] = {},
[9] = {},
[10] = {},
[11] = { "-", "!" },
[12] = {},
[13] = {},
[14] = { "&" },
[15] = {}
}
local suffix_unops_prio = {
[1] = { ";" },
[2] = {},
[3] = {},
[4] = {},
[5] = {},
[6] = {},
[7] = {},
[8] = {},
[9] = {},
[10] = {},
[11] = {},
[12] = {},
[13] = { "!" },
[14] = {},
[15] = {}
}
local function get_text_in_litteral(s, start_pos)
local d, r
-- find end of string
start_pos = start_pos or 2
local i = start_pos
while true do
local skip
skip = s:match("^[^%\\\"]-%b{}()", i) -- skip interpolated expressions
if skip then i = skip end
skip = s:match("^[^%\\\"]-\\.()", i) -- skip escape codes (need to skip every escape code in order to correctly parse \\": the " is not escaped)
if skip then i = skip end
if not skip then -- nothing skipped
local end_pos = s:match("^[^%\"]-\"()", i) -- search final double quote
if end_pos then
d, r = s:sub(start_pos, end_pos-2), s:sub(end_pos)
break
else
return nil, ("expected \" to finish string near %q"):format(s:sub(i))
end
end
end
return d, r
end
local function random_identifier_alpha()
local r = ""
for _=1, 18 do -- that's live 10^30 possibilities, ought to be enough for anyone
if math.random(1, 2) == 1 then
r = r .. string.char(math.random(65, 90))
else
r = r .. string.char(math.random(97, 122))
end
end
return r
end
--- parse an expression
-- return expr, remaining if success
-- returns nil, err if error
local function expression(s, state, namespace, source, current_priority, operating_on)
s = s:match("^%s*(.*)$")
current_priority = current_priority or 0
if not operating_on then
-- number
if s:match("^%d*%.%d+") or s:match("^%d+") then
local d, r = s:match("^(%d*%.%d+)(.*)$")
if not d then
d, r = s:match("^(%d+)(.*)$")
end
return expression(r, state, namespace, source, current_priority, {
type = "number",
value = tonumber(d)
})
-- string
elseif s:match("^%\"") then
local d, r = get_text_in_litteral(s)
local l, e = parse_text(d, state, namespace, "string") -- parse interpolated expressions
if not l then return l, e end
return expression(r, state, namespace, source, current_priority, l)
-- text buffer
elseif s:match("^%%%[") then
local text = s:match("^%%(.*)$")
local v, r = parse_text(text, state, namespace, "text", "#~", true)
if not v then return nil, r end
return expression(r, state, namespace, source, current_priority, {
type = "text buffer",
text = v
})
-- paranthesis
elseif s:match("^%b()") then
local content, r = s:match("^(%b())(.*)$")
content = content:gsub("^%(", ""):gsub("%)$", "")
local exp
if content:match("[^%s]") then
local r_paren
exp, r_paren = expression(content, state, namespace, source)
if not exp then return nil, "invalid expression inside parentheses: "..r_paren end
if r_paren:match("[^%s]") then return nil, ("unexpected %q at end of parenthesis expression"):format(r_paren) end
else
exp = { type = "nil", value = nil }
end
return expression(r, state, namespace, source, current_priority, {
type = "parentheses",
expression = exp
})
-- list parenthesis
elseif s:match("^%b[]") then
local content, r = s:match("^(%b[])(.*)$")
content = content:gsub("^%[", ""):gsub("%]$", "")
local exp
if content:match("[^%s]") then
local r_paren
exp, r_paren = expression(content, state, namespace, source)
if not exp then return nil, "invalid expression inside list parentheses: "..r_paren end
if r_paren:match("[^%s]") then return nil, ("unexpected %q at end of list parenthesis expression"):format(r_paren) end
end
return expression(r, state, namespace, source, current_priority, {
type = "list brackets",
expression = exp
})
-- map parenthesis
elseif s:match("^%b{}") then
local content, r = s:match("^(%b{})(.*)$")
content = content:gsub("^%{", ""):gsub("%}$", "")
local exp
if content:match("[^%s]") then
local r_paren
exp, r_paren = expression(content, state, namespace, source)
if not exp then return nil, "invalid expression inside map parentheses: "..r_paren end
if r_paren:match("[^%s]") then return nil, ("unexpected %q at end of map parenthesis expression"):format(r_paren) end
end
return expression(r, state, namespace, source, current_priority, {
type = "map brackets",
expression = exp
})
-- identifier
elseif s:match("^"..identifier_pattern) then
local name, r = s:match("^("..identifier_pattern..")(.-)$")
name = format_identifier(name)
-- string:value pair shorthand using =
if r:match("^=[^=]") and pair_priority > current_priority then
local val
val, r = expression(r:match("^=(.*)$"), state, namespace, source, pair_priority)
if not val then return val, r end
local args = {
type = "list",
left = {
type = "string",
text = { name }
},
right = val
}
-- find compatible variant
local variant, err = find_function(state, namespace, "_=_", args, true)
if not variant then return variant, err end
return expression(r, state, namespace, source, current_priority, variant)
end
-- variables
-- if name isn't a valid variable, suffix call: detect if a prefix is valid variable, suffix _._ call is handled in the binop section below
local nl = split(name)
for i=#nl, 1, -1 do
local name_prefix = table.concat(nl, ".", 1, i)
local var, vfqm = find(state.aliases, state.variables, namespace, name_prefix)
if var then
if i < #nl then
r = "."..table.concat(nl, ".", i+1, #nl)..r
end
return expression(r, state, namespace, source, current_priority, {
type = "variable",
name = vfqm
})
end
end
-- functions. This is a temporary expression that will either be transformed into a reference by the &_ operator, or an (implicit) function call otherwise.
for i=#nl, 1, -1 do
local name_prefix = table.concat(nl, ".", 1, i)
local lfnqm = find_all(state.aliases, state.functions, namespace, name_prefix)
if #lfnqm > 0 then
if i < #nl then
r = "."..table.concat(nl, ".", i+1, #nl)..r
end
return expression(r, state, namespace, source, current_priority, {
type = "potential function",
called_name = name,
names = lfnqm
})
end
end
return nil, ("can't find function or variable named %q in namespace %q"):format(name, namespace)
end
-- prefix unops
for prio, oplist in ipairs(prefix_unops_prio) do
for _, op in ipairs(oplist) do
local escaped = escape(op)
if s:match("^"..escaped) then
local sright = s:match("^"..escaped.."(.*)$")
-- function and variable reference
if op == "&" then
local right, r = expression(sright, state, namespace, source, prio)
if not right then return nil, ("invalid expression after unop %q: %s"):format(op, r) end
if right.type == "potential function" then
return expression(r, state, namespace, source, current_priority, {
type = "function reference",
names = right.names
})
elseif right.type == "variable" then
return expression(r, state, namespace, source, current_priority, {
type = "variable reference",
name = right.name,
expression = right
})
else
-- find variant
local variant, err = find_function(state, namespace, op.."_", right, true)
if not variant then return variant, err end
return expression(r, state, namespace, source, current_priority, variant)
end
-- anonymous function
elseif op == "$" then
-- get eventual arguments
local params = "()"
if sright:match("^%b()") then
params, sright = sright:match("^(%b())(.*)$")
end
-- define function
local fn_name = ("%s🥸%s"):format(namespace, random_identifier_alpha())
local s, e = preparse(state, (":$%s%s\n\t@%s"):format(fn_name, params, fn_name), "", source)
if not s then return nil, e end
local fn = state.functions[fn_name][1]
-- parse return expression
local right, r = expression(sright, state, fn.namespace, source, prio)
if not right then return nil, ("invalid expression after unop %q: %s"):format(op, r) end
-- put expression in return line
for _, c in ipairs(fn.child) do
if c.type == "return" and c.expression == fn_name then
c.expression = right
end
end
-- return reference to created function
return expression(r, state, namespace, source, current_priority, {
type = "nonpersistent",
expression = {
type = "function reference",
names = { fn_name }
}
})
-- normal prefix unop
else
local right, r = expression(sright, state, namespace, source, prio)
if not right then return nil, ("invalid expression after unop %q: %s"):format(op, r) end
-- find variant
local variant, err = find_function(state, namespace, op.."_", right, true)
if not variant then return variant, err end
return expression(r, state, namespace, source, current_priority, variant)
end
end
end
end
return nil, ("no valid expression before %q"):format(s)
else
-- transform potential function/variable calls into actual calls automatically
-- need to do this before every other operator, since once the code finds the next operator it won't go back to check if this applied and assume it
-- didn't skip anything since it didn't see any other operator before, even if it's actually higher priority...
-- the problems of an implicit operator I guess
if implicit_call_priority > current_priority then
-- implicit call of a function. Unlike for variables, can't be cancelled since there's not any other value this could return, we don't
-- have first class functions here...
if operating_on.type == "potential function" then
local args, paren_call, implicit_call
local r = s
if r:match("^%b()") then
paren_call = true
local content, rem = r:match("^(%b())(.*)$")
content = content:gsub("^%(", ""):gsub("%)$", "")
r = rem
-- get arguments
if content:match("[^%s]") then
local err
args, err = expression(content, state, namespace, source)
if not args then return args, err end
if err:match("[^%s]") then return nil, ("unexpected %q at end of argument list"):format(err) end
end
else -- implicit call; will be changed if there happens to be a ! after in the suffix operator code
implicit_call = true
end
-- find compatible variant
local variant, err = find_function_from_list(state, namespace, operating_on.called_name, operating_on.names, args, paren_call, implicit_call)
if not variant then return variant, err end
return expression(r, state, namespace, source, current_priority, variant)
-- implicit call on variable reference. Might be canceled afterwards due to finding a higher priority operator.
elseif operating_on.type == "variable" or (operating_on.type == "function call" and operating_on.called_name == "_._") then
local implicit_call_variant, err = find_function(state, namespace, "_!", { type = "value passthrough" }, false, true)
if not implicit_call_variant then return implicit_call_variant, err end
return expression(s, state, namespace, source, current_priority, {
type = "implicit call if reference",
variant = implicit_call_variant,
expression = operating_on
})
end
end
-- binop
for prio, oplist in ipairs(binops_prio) do
if prio > current_priority then
-- cancel implicit call operator if we are handling a binop of higher priority
-- see comment a bit above on why the priority handling is stupid for implicit operators
local operating_on = operating_on
if prio > implicit_call_priority and operating_on.type == "implicit call if reference" then
operating_on = operating_on.expression
end
for _, op in ipairs(oplist) do
local escaped = escape(op)
if s:match("^"..escaped) then
local sright = s:match("^"..escaped.."(.*)$")
-- suffix call
if op == "!" and sright:match("^"..identifier_pattern) then
local name, r = sright:match("^("..identifier_pattern..")(.-)$")
name = format_identifier(name)
local args, paren_call
if r:match("^%b()") then
paren_call = true
local content, rem = r:match("^(%b())(.*)$")
content = content:gsub("^%(", ""):gsub("%)$", "")
r = rem
-- get arguments
if content:match("[^%s]") then
local err
args, err = expression(content, state, namespace, source)
if not args then return args, err end
if err:match("[^%s]") then return nil, ("unexpected %q at end of argument map"):format(err) end
end
end
-- add first argument
if not args then
args = operating_on
else
if args.type == "list" then -- insert as first element
local first_list = args
while first_list.left.type == "list" do
first_list = first_list.left
end
first_list.left = {
type = "list",
left = operating_on,
right = first_list.left
}
else
args = {
type = "list",
left = operating_on,
right = args
}
end
end
-- find compatible variant
local variant, err = find_function(state, namespace, name, args, paren_call)
if not variant then return variant, err end
return expression(r, state, namespace, source, current_priority, variant)
-- namespace
elseif op == "." and sright:match("^"..identifier_pattern) then
local name, r = sright:match("^("..identifier_pattern..")(.-)$")
name = format_identifier(name)
-- find variant
local args = {
type = "list",
left = operating_on,
right = { type = "string", text = { name } }
}
local variant, err = find_function(state, namespace, "_._", args, true)
if not variant then return variant, err end
return expression(r, state, namespace, source, current_priority, variant)
-- other binops
else
local right, r = expression(sright, state, namespace, source, prio)
if right then
-- list constructor (can't do this through a function call since we need to build a list for its arguments)
if op == "," then
return expression(r, state, namespace, source, current_priority, {
type = "list",
left = operating_on,
right = right
})
-- special binops
elseif op == ":=" or op == "+=" or op == "-=" or op == "//=" or op == "/=" or op == "*=" or op == "%=" or op == "^=" then
-- cancel implicit call on right variable
if operating_on.type == "implicit call if reference" then
operating_on = operating_on.expression
end
-- rewrite assignment + arithmetic operators into a normal assignment
if op ~= ":=" then
local args = {
type = "list",
left = operating_on,
right = right
}
local variant, err = find_function(state, namespace, "_"..op:match("^(.*)%=$").."_", args, true)
if not variant then return variant, err end
right = variant
end
-- assign to a function
if operating_on.type == "function call" then
-- remove non-assignment functions
for i=#operating_on.variants, 1, -1 do
if not operating_on.variants[i].assignment then
table.remove(operating_on.variants, i)
end
end
if #operating_on.variants == 0 then
return nil, ("trying to perform assignment on function %s with no compatible assignment variant"):format(operating_on.called_name)
end
-- rewrite function to perform assignment
operating_on.assignment = right
return expression(r, state, namespace, source, current_priority, operating_on)
elseif operating_on.type ~= "variable" then
return nil, ("trying to perform assignment on a %s expression"):format(operating_on.type)
end
-- assign to a variable
return expression(r, state, namespace, source, current_priority, {
type = ":=",
left = operating_on,
right = right
})
elseif op == "&" or op == "|" or op == "~?" or op == "~" or op == "#" then
return expression(r, state, namespace, source, current_priority, {
type = op,
left = operating_on,
right = right
})
-- normal binop
else
-- find variant
local args = {
type = "list",
left = operating_on,
right = right
}
local variant, err = find_function(state, namespace, "_"..op.."_", args, true)
if not variant then return variant, err end
return expression(r, state, namespace, source, current_priority, variant)
end
end
end
end
end
end
end
-- suffix unop
for prio, oplist in ipairs(suffix_unops_prio) do
if prio > current_priority then
-- cancel implit call operator if we are handling an operator of higher priority
-- see comment a bit above on why the priority handling is stupid for implicit operators
local operating_on = operating_on
if prio > implicit_call_priority and operating_on.type == "implicit call if reference" then
operating_on = operating_on.expression
end
for _, op in ipairs(oplist) do
local escaped = escape(op)
if s:match("^"..escaped) then
local r = s:match("^"..escaped.."(.*)$")
-- remove ! after a previously-assumed implicit function call
if op == "!" and operating_on.type == "function call" and operating_on.implicit_call then
operating_on.implicit_call = false
return expression(r, state, namespace, source, current_priority, operating_on)
-- normal suffix unop
else
local variant, err = find_function(state, namespace, "_"..op, operating_on, true)
if not variant then return variant, err end
return expression(r, state, namespace, source, current_priority, variant)
end
end
end
end
end
-- index / call
if call_priority > current_priority and s:match("^%b()") then
if operating_on.type == "implicit call if reference" then
operating_on = operating_on.expression -- replaced with current call
end
local args = operating_on
local content, r = s:match("^(%b())(.*)$")
content = content:gsub("^%(", ""):gsub("%)$", "")
-- get arguments
if content:match("[^%s]") then
local right, r_paren = expression(content, state, namespace, source)
if not right then return right, r_paren end
if r_paren:match("[^%s]") then return nil, ("unexpected %q at end of index/call expression"):format(r_paren) end
args = { type = "list", left = args, right = right }
end
local variant, err = find_function(state, namespace, "()", args, true)
if not variant then return variant, err end
return expression(r, state, namespace, source, current_priority, variant)
end
-- implicit multiplication
if implicit_multiply_priority > current_priority then
if s:match("^"..identifier_pattern) then
local right, r = expression(s, state, namespace, source, implicit_multiply_priority)
if right then
local args = {
type = "list",
left = operating_on,
right = right
}
local variant, err = find_function(state, namespace, "_*_", args, true)
if not variant then return variant, err end
return expression(r, state, namespace, source, current_priority, variant)
end
end
end
-- nothing to operate
return operating_on, s
end
end
package.loaded[...] = expression
local common = require((...):gsub("expression$", "common"))
identifier_pattern, format_identifier, find, escape, find_function, parse_text, find_all, split, find_function_from_list = common.identifier_pattern, common.format_identifier, common.find, common.escape, common.find_function, common.parse_text, common.find_all, common.split, common.find_function_from_list
preparse = require((...):gsub("expression$", "preparser"))
return expression

57
parser/expression/comment.lua Normal file
View file

@ -0,0 +1,57 @@
local primary = require("parser.expression.primary.primary")
local comment
comment = primary {
match = function(self, str)
return str:match("^%(%(")
end,
parse = function(self, source, str, limit_pattern)
local rem = source:consume(str:match("^(%(%()(.*)$"))
local content_list = {}
while not rem:match("^%)%)") do
local content
content, rem = rem:match("^([^%(%)]*)(.-)$")
-- cut the text prematurely at limit_pattern if relevant
if limit_pattern and content:match(limit_pattern) then
local pos = content:match("()"..limit_pattern) -- limit_pattern can contain $, so can't directly extract with captures
content, rem = source:count(content:sub(1, pos-1)), ("))%s%s"):format(content:sub(pos), rem)
source:increment(-2)
else
source:count(content)
end
table.insert(content_list, content)
-- nested comment
if rem:match("^%(%(") then
local subcomment
subcomment, rem = comment:parse(source, rem, limit_pattern)
table.insert(content_list, "((")
for _, c in ipairs(subcomment) do table.insert(content_list, c) end
table.insert(content_list, "))")
-- no end token after the comment
elseif not rem:match("^%)%)") then
-- single ) or (, keep on commentin'
if rem:match("^[%)%(]") then
local s
s, rem = source:count(rem:match("^([%)%(])(.-)$"))
table.insert(content_list, s)
-- anything other than end-of-line
elseif rem:match("[^%s]") then
error(("unexpected %q at end of comment"):format(rem), 0)
-- consumed everything until end-of-line, close your eyes and imagine the text has been closed
else
rem = rem .. "))"
end
end
end
rem = source:consume(rem:match("^(%)%))(.*)$"))
return table.concat(content_list, ""), rem
end
}
return comment

40
parser/expression/contextual/function_parameter.lua Normal file
View file

@ -0,0 +1,40 @@
local primary = require("parser.expression.primary.primary")
local identifier = require("parser.expression.primary.identifier")
local expression_to_ast = require("parser.expression.to_ast")
local ast = require("ast")
local FunctionParameter = ast.FunctionParameter
local operator_priority = require("common").operator_priority
local assignment_priority = operator_priority["_=_"]
local type_check_priority = operator_priority["_::_"]
return primary {
match = function(self, str)
return identifier:match(str)
end,
parse = function(self, source, str, limit_pattern, no_default_value)
local source_param = source:clone()
-- name
local ident, rem = identifier:parse(source, str)
-- type check
local type_check
if rem:match("^%s*::") then
local scheck = source:consume(rem:match("^(%s*::%s*)(.*)$"))
type_check, rem = expression_to_ast(source, scheck, limit_pattern, type_check_priority)
end
-- default value
local default
if not no_default_value then
if rem:match("^%s*=") then
local sdefault = source:consume(rem:match("^(%s*=%s*)(.*)$"))
default, rem = expression_to_ast(source, sdefault, limit_pattern, assignment_priority)
end
end
return FunctionParameter:new(ident, default, type_check):set_source(source_param), rem
end
}

7
parser/expression/contextual/function_parameter_no_default.lua Normal file
View file

@ -0,0 +1,7 @@
local function_parameter = require("parser.expression.contextual.function_parameter")
return function_parameter {
parse = function(self, source, str, limit_pattern)
return function_parameter:parse(source, str, limit_pattern, true)
end
}

49
parser/expression/contextual/parameter_tuple.lua Normal file
View file

@ -0,0 +1,49 @@
local primary = require("parser.expression.primary.primary")
local function_parameter = require("parser.expression.contextual.function_parameter")
local function_parameter_no_default = require("parser.expression.contextual.function_parameter_no_default")
local ast = require("ast")
local ParameterTuple = ast.ParameterTuple
return primary {
match = function(self, str)
return str:match("^%(")
end,
parse = function(self, source, str, limit_pattern)
local source_start = source:clone()
local parameters = ParameterTuple:new()
local rem = source:consume(str:match("^(%()(.*)$"))
-- i would LOVE to reuse the regular list parsing code for this, but unfortunately the list parsing code
-- itself depends on this and expect this to be available quite early, and it's ANNOYING
while not rem:match("^%s*%)") do
-- parameter
local func_param
func_param, rem = function_parameter:expect(source, rem, limit_pattern)
-- next! comma separator
if not rem:match("^%s*%)") then
if not rem:match("^%s*,") then
error(("unexpected %q at end of argument list"):format(rem), 0)
end
rem = source:consume(rem:match("^(%s*,)(.*)$"))
end
-- add
parameters:insert(func_param)
end
rem = rem:match("^%s*%)(.*)$")
-- assigment param
if rem:match("^%s*=") then
rem = source:consume(rem:match("^(%s*=%s*)(.*)$"))
local func_param
func_param, rem = function_parameter_no_default:expect(source, rem, limit_pattern)
parameters:insert_assignment(func_param)
end
return parameters:set_source(source_start), rem
end
}

16
parser/expression/primary/block_identifier.lua Normal file
View file

@ -0,0 +1,16 @@
local primary = require("parser.expression.primary.primary")
local ast = require("ast")
local Identifier, Call, ArgumentTuple = ast.Identifier, ast.Call, ast.ArgumentTuple
return primary {
match = function(self, str)
return str:match("^_")
end,
parse = function(self, source, str)
local source_start = source:clone()
local rem = source:consume(str:match("^(_)(.-)$"))
return Call:new(Identifier:new("_"), ArgumentTuple:new()):set_source(source_start), rem
end
}

205
parser/expression/primary/function_definition.lua Normal file
View file

@ -0,0 +1,205 @@
local primary = require("parser.expression.primary.primary")
local function_parameter_no_default = require("parser.expression.contextual.function_parameter_no_default")
local parameter_tuple = require("parser.expression.contextual.parameter_tuple")
local identifier = require("parser.expression.primary.identifier")
local expression_to_ast = require("parser.expression.to_ast")
local escape = require("common").escape
local ast = require("ast")
local Symbol, Definition, Function, ParameterTuple = ast.Symbol, ast.Definition, ast.Function, ast.ParameterTuple
local regular_operators = require("common").regular_operators
local prefixes = regular_operators.prefixes
local suffixes = regular_operators.suffixes
local infixes = regular_operators.infixes
local operator_priority = require("common").operator_priority
-- same as function_parameter_no_default, but allow wrapping in (evenetual) parentheses
-- in order to solve some priotity issues (_._ has higher priority than _::_, leading to not being possible to overload it with type filtering without parentheses)
local function_parameter_maybe_parenthesis = function_parameter_no_default {
match = function(self, str)
if str:match("^%(") then
return function_parameter_no_default:match(str:match("^%((.*)$"))
else
return function_parameter_no_default:match(str)
end
end,
parse = function(self, source, str, limit_pattern)
if str:match("^%(") then
str = source:consume(str:match("^(%()(.*)$"))
local exp, rem = function_parameter_no_default:parse(source, str, limit_pattern)
if not rem:match("^%s*%)") then error(("unexpected %q at end of parenthesis"):format(rem), 0) end
rem = source:consume(rem:match("^(%s*%))(.-)$"))
return exp, rem
else
return function_parameter_no_default:parse(source, str, limit_pattern)
end
end
}
-- signature type 1: unary prefix
-- :$-parameter exp
-- returns symbol, parameter_tuple, rem if success
-- return nil otherwise
local function search_prefix_signature(modifiers, source, str, limit_pattern)
for _, pfx in ipairs(prefixes) do
local prefix = pfx[1]
local prefix_pattern = "%s*"..escape(prefix).."%s*"
if str:match("^"..prefix_pattern) then
-- operator name
local rem = source:consume(str:match("^("..prefix_pattern..")(.*)$"))
local symbol = Symbol:new(prefix.."_", modifiers):set_source(source:clone():increment(-1))
-- parameters
local parameter
parameter, rem = function_parameter_maybe_parenthesis:expect(source, rem, limit_pattern)
local parameters = ParameterTuple:new()
parameters:insert(parameter)
return symbol, parameters, rem
end
end
end
-- signature type 2: binary infix
-- should be checked before suffix signature
-- :$parameterA + parameterB exp
-- returns symbol, parameter_tuple, rem if success
-- return nil otherwise
local function search_infix_signature(modifiers, source, str, limit_pattern)
if function_parameter_maybe_parenthesis:match(str) then
local src = source:clone() -- operate on clone source since search success is not yet guaranteed
local parameter_a, rem = function_parameter_maybe_parenthesis:parse(src, str, limit_pattern)
local parameters = ParameterTuple:new()
parameters:insert(parameter_a)
for _, ifx in ipairs(infixes) do
local infix = ifx[1]
local infix_pattern = "%s*"..escape(infix).."%s*"
if rem:match("^"..infix_pattern) then
-- operator name
rem = src:consume(rem:match("^("..infix_pattern..")(.*)$"))
local symbol = Symbol:new("_"..infix.."_", modifiers):set_source(src:clone():increment(-1))
-- parameters
if function_parameter_maybe_parenthesis:match(rem) then
local parameter_b
parameter_b, rem = function_parameter_maybe_parenthesis:parse(src, rem, limit_pattern)
parameters:insert(parameter_b)
source:set(src)
return symbol, parameters, rem
else
return
end
end
end
end
end
-- signature type 3: unary suffix
-- :$parameter! exp
-- returns symbol, parameter_tuple, rem if success
-- return nil otherwise
local function search_suffix_signature(modifiers, source, str, limit_pattern)
if function_parameter_maybe_parenthesis:match(str) then
local src = source:clone() -- operate on clone source since search success is not yet guaranteed
local parameter_a, rem = function_parameter_maybe_parenthesis:parse(src, str, limit_pattern)
local parameters = ParameterTuple:new()
parameters:insert(parameter_a)
for _, sfx in ipairs(suffixes) do
local suffix = sfx[1]
local suffix_pattern = "%s*"..escape(suffix).."%s*"
if rem:match("^"..suffix_pattern) then
-- operator name
rem = src:count(rem:match("^("..suffix_pattern..")(.*)$"))
local symbol = Symbol:new("_"..suffix, modifiers):set_source(src:clone():increment(-1))
source:set(src)
return symbol, parameters, rem
end
end
end
end
-- signature type 4: regular function
-- :$identifier(parameter_tuple, ...) exp
-- returns symbol, parameter_tuple, rem if success
-- return nil otherwise
local function search_function_signature(modifiers, source, str, limit_pattern)
if identifier:match(str) then
local name_source = source:clone()
local name, rem = identifier:parse(source, str, limit_pattern)
-- name
local symbol = name:to_symbol(modifiers):set_source(name_source)
-- parse eventual parameters
local parameters
if parameter_tuple:match(rem) then
parameters, rem = parameter_tuple:parse(source, rem)
else
parameters = ParameterTuple:new()
end
return symbol, parameters, rem
end
end
return primary {
match = function(self, str)
return str:match("^%::?[&@]?%$")
end,
parse = function(self, source, str, limit_pattern)
local source_start = source:clone()
local mod_const, mod_exported, rem = source:consume(str:match("^(%:(:?)([&@]?)%$)(.-)$"))
-- get modifiers
local constant, exported, persistent
if mod_const == ":" then constant = true end
if mod_exported == "@" then exported = true
elseif mod_exported == "&" then persistent = true end
local modifiers = { constant = constant, exported = exported, persistent = persistent }
-- search for a valid signature
local symbol, parameters
local s, p, r = search_prefix_signature(modifiers, source, rem, limit_pattern)
if s then symbol, parameters, rem = s, p, r
else
s, p, r = search_infix_signature(modifiers, source, rem, limit_pattern)
if s then symbol, parameters, rem = s, p, r
else
s, p, r = search_suffix_signature(modifiers, source, rem, limit_pattern)
if s then symbol, parameters, rem = s, p, r
else
s, p, r = search_function_signature(modifiers, source, rem, limit_pattern)
if s then symbol, parameters, rem = s, p, r end
end
end
end
-- done
if symbol then
-- parse expression
local right
s, right, rem = pcall(expression_to_ast, source, rem, limit_pattern, operator_priority["$_"])
if not s then error(("invalid expression after unop %q: %s"):format(self.operator, right), 0) end
-- return function
local fn = Function:new(parameters, right):set_source(source_start)
return Definition:new(symbol, fn):set_source(source_start), rem
end
end
}

40
parser/expression/primary/identifier.lua Normal file
View file

@ -0,0 +1,40 @@
local primary = require("parser.expression.primary.primary")
local Identifier = require("ast.Identifier")
local disallowed_set = (".~`^+-=<>/[]*{}|\\_!?,;:()\"@&$#%"):gsub("[^%w]", "%%%1")
local identifier_pattern = "%s*[^0-9%s'"..disallowed_set.."][^"..disallowed_set.."]*"
local common = require("common")
local trim, escape = common.trim, common.escape
-- for operator identifiers
local regular_operators = require("common").regular_operators
local operators = {}
for _, prefix in ipairs(regular_operators.prefixes) do table.insert(operators, prefix[1].."_") end
for _, infix in ipairs(regular_operators.infixes) do table.insert(operators, "_"..infix[1].."_") end
for _, suffix in ipairs(regular_operators.suffixes) do table.insert(operators, "_"..suffix[1]) end
-- all valid identifier patterns
local identifier_patterns = { identifier_pattern }
for _, operator in ipairs(operators) do table.insert(identifier_patterns, "%s*"..escape(operator)) end
return primary {
match = function(self, str)
for _, pat in ipairs(identifier_patterns) do
if str:match("^"..pat) then return true end
end
return false
end,
parse = function(self, source, str)
for _, pat in ipairs(identifier_patterns) do
if str:match("^"..pat) then
local start_source = source:clone()
local name, rem = source:count(str:match("^("..pat..")(.-)$"))
name = trim(name)
return Identifier:new(name):set_source(start_source), rem
end
end
end
}

42
parser/expression/primary/init.lua Normal file
View file

@ -0,0 +1,42 @@
--- try to parse a primary expression
local function r(name)
return require("parser.expression.primary."..name), nil
end
local primaries = {
r("number"),
r("string"),
r("text"),
r("parenthesis"),
r("function_definition"),
r("symbol"),
r("identifier"),
r("block_identifier"),
r("tuple"),
r("struct"),
-- prefixes
-- 1
r("prefix.semicolon"),
r("prefix.function"),
-- 2
r("prefix.return"),
-- 3.5
r("prefix.else"),
-- 11
r("prefix.negation"),
r("prefix.not"),
r("prefix.mutable"),
}
return {
-- returns exp, rem if expression found
-- returns nil if no expression found
search = function(self, source, str, limit_pattern)
for _, primary in ipairs(primaries) do
local exp, rem = primary:search(source, str, limit_pattern)
if exp then return exp, rem end
end
end
}

19
parser/expression/primary/number.lua Normal file
View file

@ -0,0 +1,19 @@
local primary = require("parser.expression.primary.primary")
local Number = require("ast.Number")
return primary {
match = function(self, str)
return str:match("^%d*%.%d+") or str:match("^%d+")
end,
parse = function(self, source, str)
local start_source = source:clone()
local d, r = str:match("^(%d*%.%d+)(.*)$")
if not d then
d, r = source:count(str:match("^(%d+)(.*)$"))
else
source:count(d)
end
return Number:new(tonumber(d)):set_source(start_source), r
end
}

31
parser/expression/primary/parenthesis.lua Normal file
View file

@ -0,0 +1,31 @@
-- either parentheses or nil ()
local primary = require("parser.expression.primary.primary")
local ast = require("ast")
local Nil = ast.Nil
local expression_to_ast = require("parser.expression.to_ast")
return primary {
match = function(self, str)
return str:match("^%(")
end,
parse = function(self, source, str)
local start_source = source:clone()
local rem = source:consume(str:match("^(%()(.*)$"))
local exp
if rem:match("^%s*%)") then
exp = Nil:new()
else
local s
s, exp, rem = pcall(expression_to_ast, source, rem, "%)")
if not s then error("invalid expression inside parentheses: "..exp, 0) end
if not rem:match("^%s*%)") then error(("unexpected %q at end of parenthesis"):format(rem), 0) end
end
rem = source:consume(rem:match("^(%s*%))(.*)$"))
return exp:set_source(start_source), rem
end
}

8
parser/expression/primary/prefix/else.lua Normal file
View file

@ -0,0 +1,8 @@
local prefix_quote_right = require("parser.expression.primary.prefix.prefix_quote_right")
local operator_priority = require("common").operator_priority
return prefix_quote_right {
operator = "~",
identifier = "~_",
priority = operator_priority["~_"]
}

35
parser/expression/primary/prefix/function.lua Normal file
View file

@ -0,0 +1,35 @@
local prefix = require("parser.expression.primary.prefix.prefix")
local parameter_tuple = require("parser.expression.contextual.parameter_tuple")
local escape = require("common").escape
local expression_to_ast = require("parser.expression.to_ast")
local ast = require("ast")
local Function, ParameterTuple = ast.Function, ast.ParameterTuple
local operator_priority = require("common").operator_priority
return prefix {
operator = "$",
priority = operator_priority["$_"],
parse = function(self, source, str, limit_pattern)
local source_start = source:clone()
local escaped = escape(self.operator)
local rem = source:consume(str:match("^("..escaped..")(.*)$"))
-- parse eventual parameters
local parameters
if parameter_tuple:match(rem) then
parameters, rem = parameter_tuple:parse(source, rem)
else
parameters = ParameterTuple:new()
end
-- parse expression
local s, right
s, right, rem = pcall(expression_to_ast, source, rem, limit_pattern, self.priority)
if not s then error(("invalid expression after unop %q: %s"):format(self.operator, right), 0) end
return Function:new(parameters, right):set_source(source_start), rem
end
}

9
parser/expression/primary/prefix/mutable.lua Normal file
View file

@ -0,0 +1,9 @@
local prefix = require("parser.expression.primary.prefix.prefix")
local operator_priority = require("common").operator_priority
return prefix {
operator = "*",
identifier = "*_",
priority = operator_priority["*_"]
}

9
parser/expression/primary/prefix/negation.lua Normal file
View file

@ -0,0 +1,9 @@
local prefix = require("parser.expression.primary.prefix.prefix")
local operator_priority = require("common").operator_priority
return prefix {
operator = "-",
identifier = "-_",
priority = operator_priority["-_"]
}

9
parser/expression/primary/prefix/not.lua Normal file
View file

@ -0,0 +1,9 @@
local prefix = require("parser.expression.primary.prefix.prefix")
local operator_priority = require("common").operator_priority
return prefix {
operator = "!",
identifier = "!_",
priority = operator_priority["!_"]
}

34
parser/expression/primary/prefix/prefix.lua Normal file
View file

@ -0,0 +1,34 @@
-- unary prefix operators, for example: the - in -5
local primary = require("parser.expression.primary.primary")
local escape = require("common").escape
local expression_to_ast = require("parser.expression.to_ast")
local ast = require("ast")
local Call, Identifier, ArgumentTuple = ast.Call, ast.Identifier, ast.ArgumentTuple
return primary {
operator = nil,
identifier = nil,
priority = nil,
match = function(self, str)
local escaped = escape(self.operator)
return str:match("^"..escaped)
end,
parse = function(self, source, str, limit_pattern)
local source_start = source:clone()
local escaped = escape(self.operator)
local sright = source:consume(str:match("^("..escaped..")(.*)$"))
local s, right, rem = pcall(expression_to_ast, source, sright, limit_pattern, self.priority)
if not s then error(("invalid expression after prefix operator %q: %s"):format(self.operator, right), 0) end
return self:build_ast(right):set_source(source_start), rem
end,
build_ast = function(self, right)
return Call:new(Identifier:new(self.identifier), ArgumentTuple:new(right))
end
}

11
parser/expression/primary/prefix/prefix_quote_right.lua Normal file
View file

@ -0,0 +1,11 @@
local prefix = require("parser.expression.primary.prefix.prefix")
local ast = require("ast")
local Call, Identifier, ArgumentTuple, Quote = ast.Call, ast.Identifier, ast.ArgumentTuple, ast.Quote
return prefix {
build_ast = function(self, right)
right = Quote:new(right)
return Call:new(Identifier:new(self.identifier), ArgumentTuple:new(right))
end
}

15
parser/expression/primary/prefix/return.lua Normal file
View file

@ -0,0 +1,15 @@
local prefix = require("parser.expression.primary.prefix.prefix")
local ast = require("ast")
local Return = ast.Return
local operator_priority = require("common").operator_priority
return prefix {
operator = "@",
priority = operator_priority["@_"],
build_ast = function(self, right)
return Return:new(right)
end
}

13
parser/expression/primary/prefix/semicolon.lua Normal file
View file

@ -0,0 +1,13 @@
local prefix = require("parser.expression.primary.prefix.prefix")
local operator_priority = require("common").operator_priority
return prefix {
operator = ";",
identifier = ";_",
priority = operator_priority[";_"],
build_ast = function(self, right)
return right
end
}

33
parser/expression/primary/primary.lua Normal file
View file

@ -0,0 +1,33 @@
local class = require("class")
return class {
new = false, -- static class
-- returns exp, rem if expression found
-- returns nil if no expression found
search = function(self, source, str, limit_pattern)
if not self:match(str) then
return nil
end
return self:parse(source, str, limit_pattern)
end,
-- return bool
-- (not needed if you redefined :search)
match = function(self, str)
return false
end,
-- return AST, rem
-- (not needed if you redefined :search)
parse = function(self, source, str, limit_pattern)
error("unimplemented")
end,
-- class helpers --
-- return AST, rem
expect = function(self, source, str, limit_pattern)
local exp, rem = self:search(source, str, limit_pattern)
if not exp then error(("expected %s but got %s"):format(self.type, str)) end
return exp, rem
end
}

78
parser/expression/primary/string.lua Normal file
View file

@ -0,0 +1,78 @@
-- note: this is reused in primary.text, hence all the configurable fields
local primary = require("parser.expression.primary.primary")
local StringInterpolation = require("ast.StringInterpolation")
local ast = require("ast")
local String = ast.String
local expression_to_ast = require("parser.expression.to_ast")
local escape = require("common").escape
local escape_code = {
["n"] = "\n",
["t"] = "\t",
-- everything else is identity by default
}
return primary {
type = "string", -- interpolation type - used for errors
start_pattern = "\"", -- pattern that start the string interpolation
stop_char = "\"", -- character that stops the string interpolation - must be a single character!
allow_implicit_stop = false, -- set to true to allow the string to be closed implicitely when reaching the end of the expression or limit_pattern
interpolation = StringInterpolation,
match = function(self, str)
return str:match("^"..self.start_pattern)
end,
parse = function(self, source, str, limit_pattern)
local interpolation = self.interpolation:new()
local stop_pattern = escape(self.stop_char)
local start_source = source:clone()
local rem = source:consume(str:match("^("..self.start_pattern..")(.-)$"))
while not rem:match("^"..stop_pattern) do
local text_source = source:clone()
local text
text, rem = rem:match("^([^%{%\\"..stop_pattern.."]*)(.-)$") -- get all text until something potentially happens
-- cut the text prematurely at limit_pattern if relevant
if self.allow_implicit_stop and limit_pattern and text:match(limit_pattern) then
local pos = text:match("()"..limit_pattern) -- limit_pattern can contain $, so can't directly extract with captures
text, rem = source:count(text:sub(1, pos-1)), ("%s%s%s"):format(self.stop_char, text:sub(pos), rem)
source:increment(-1)
else
source:count(text)
end
interpolation:insert(String:new(text):set_source(text_source))
if rem:match("^%{") then
local ok, exp
ok, exp, rem = pcall(expression_to_ast, source, source:consume(rem:match("^(%{)(.*)$")), "%}")
if not ok then error("invalid expression inside interpolation: "..exp, 0) end
if not rem:match("^%s*%}") then error(("unexpected %q at end of interpolation"):format(rem), 0) end
rem = source:consume(rem:match("^(%s*%})(.*)$"))
interpolation:insert(exp)
elseif rem:match("^\\") then
text, rem = source:consume(rem:match("^(\\(.))(.*)$"))
interpolation:insert(String:new(escape_code[text] or text))
elseif not rem:match("^"..stop_pattern) then
if not self.allow_implicit_stop or rem:match("[^%s]") then
error(("unexpected %q at end of "..self.type):format(rem), 0)
-- consumed everything until end-of-line, implicit stop allowed, close your eyes and imagine the text has been closed
else
rem = rem .. self.stop_char
end
end
end
rem = source:consume(rem:match("^("..stop_pattern..")(.*)$"))
return interpolation:set_source(start_source), rem
end
}

17
parser/expression/primary/struct.lua Normal file
View file

@ -0,0 +1,17 @@
local primary = require("parser.expression.primary.primary")
local tuple = require("parser.expression.primary.tuple")
local ast = require("ast")
local Struct = ast.Struct
return primary {
match = function(self, str)
return str:match("^%{")
end,
parse = function(self, source, str)
local l, rem = tuple:parse_tuple(source, str, "{", '}')
return Struct:from_tuple(l), rem
end
}

40
parser/expression/primary/symbol.lua Normal file
View file

@ -0,0 +1,40 @@
local primary = require("parser.expression.primary.primary")
local type_check = require("parser.expression.secondary.infix.type_check")
local identifier = require("parser.expression.primary.identifier")
local ast = require("ast")
local Nil = ast.Nil
return primary {
match = function(self, str)
if str:match("^%::?[&@]?") then
return identifier:match(str:match("^%::?[&@]?(.-)$"))
end
return false
end,
parse = function(self, source, str)
local mod_const, mod_export, rem = source:consume(str:match("^(%:(:?)([&@]?))(.-)$"))
local constant, persistent, type_check_exp, exported
-- get modifier
if mod_const == ":" then constant = true end
if mod_export == "&" then persistent = true
elseif mod_export == "@" then exported = true end
-- name
local ident
ident, rem = identifier:parse(source, rem)
-- type check
local nil_val = Nil:new()
if type_check:match(rem, 0, nil_val) then
local exp
exp, rem = type_check:parse(source, rem, nil, 0, nil_val)
type_check_exp = exp.arguments.list[2]
end
return ident:to_symbol{ constant = constant, persistent = persistent, exported = exported, type_check = type_check_exp }:set_source(source), rem
end
}

24
parser/expression/primary/text.lua Normal file
View file

@ -0,0 +1,24 @@
local string = require("parser.expression.primary.string")
local ast = require("ast")
local TextInterpolation = ast.TextInterpolation
return string {
type = "text",
start_pattern = "|%s?",
stop_char = "|",
allow_implicit_stop = true,
interpolation = TextInterpolation,
parse = function(self, source, str, limit_pattern)
local interpolation, rem = string.parse(self, source, str, limit_pattern)
-- restore | when chaining with a choice operator
if rem:match("^>") then
rem = "|" .. rem
source:increment(-1)
end
return interpolation, rem
end
}

41
parser/expression/primary/tuple.lua Normal file
View file

@ -0,0 +1,41 @@
local primary = require("parser.expression.primary.primary")
local ast = require("ast")
local Tuple = ast.Tuple
local expression_to_ast = require("parser.expression.to_ast")
local escape = require("common").escape
return primary {
match = function(self, str)
return str:match("^%[")
end,
parse = function(self, source, str)
return self:parse_tuple(source, str, "[", "]")
end,
parse_tuple = function(self, source, str, start_char, end_char)
local start_source = source:clone()
local rem = source:consume(str:match("^("..escape(start_char)..")(.*)$"))
local end_match = escape(end_char)
local l
if not rem:match("^%s*"..end_match) then
local s
s, l, rem = pcall(expression_to_ast, source, rem, end_match)
if not s then error("invalid expression in list: "..l, 0) end
end
if not Tuple:is(l) or l.explicit then l = Tuple:new(l) end -- single or no element
if not rem:match("^%s*"..end_match) then
error(("unexpected %q at end of list"):format(rem), 0)
end
rem = source:consume(rem:match("^(%s*"..end_match..")(.*)$"))
l.explicit = true
return l:set_source(start_source), rem
end,
}

9
parser/expression/secondary/infix/addition.lua Normal file
View file

@ -0,0 +1,9 @@
local infix = require("parser.expression.secondary.infix.infix")
local operator_priority = require("common").operator_priority
return infix {
operator = "+",
identifier = "_+_",
priority = operator_priority["_+_"]
}

9
parser/expression/secondary/infix/and.lua Normal file
View file

@ -0,0 +1,9 @@
local infix_quote_right = require("parser.expression.secondary.infix.infix_quote_right")
local operator_priority = require("common").operator_priority
return infix_quote_right {
operator = "&",
identifier = "_&_",
priority = operator_priority["_&_"]
}

23
parser/expression/secondary/infix/assignment.lua Normal file
View file

@ -0,0 +1,23 @@
local infix = require("parser.expression.secondary.infix.infix")
local escape = require("common").escape
local operator_priority = require("common").operator_priority
local ast = require("ast")
local Identifier, Assignment = ast.Identifier, ast.Assignment
return infix {
operator = "=",
identifier = "_=_",
priority = operator_priority["_=_"],
-- return bool
match = function(self, str, current_priority, primary)
local escaped = escape(self.operator)
return self.priority > current_priority and str:match("^"..escaped) and Identifier:is(primary)
end,
build_ast = function(self, left, right)
return Assignment:new(left, right)
end
}

24
parser/expression/secondary/infix/assignment_call.lua Normal file
View file

@ -0,0 +1,24 @@
local infix = require("parser.expression.secondary.infix.infix")
local escape = require("common").escape
local operator_priority = require("common").operator_priority
local ast = require("ast")
local Call = ast.Call
return infix {
operator = "=",
identifier = "_=_",
priority = operator_priority["_=_"],
-- return bool
match = function(self, str, current_priority, primary)
local escaped = escape(self.operator)
return self.priority > current_priority and str:match("^"..escaped) and Call:is(primary)
end,
build_ast = function(self, left, right)
left.arguments:set_assignment(right)
return Call:new(left.func, left.arguments) -- recreate Call since we modified left.arguments
end,
}

35
parser/expression/secondary/infix/assignment_with_infix.lua Normal file
View file

@ -0,0 +1,35 @@
local ast = require("ast")
local Call, Identifier, ArgumentTuple = ast.Call, ast.Identifier, ast.ArgumentTuple
local assignment = require("parser.expression.secondary.infix.assignment")
local assignment_call = require("parser.expression.secondary.infix.assignment_call")
local infixes = require("common").regular_operators.infixes
local generated = {}
for _, infix in ipairs(infixes) do
local operator = infix[1].."="
local identifier = "_=_"
local infix_identifier = "_"..infix[1].."_"
table.insert(generated, assignment {
operator = operator,
identifier = identifier,
build_ast = function(self, left, right)
right = Call:new(Identifier:new(infix_identifier), ArgumentTuple:new(left, right))
return assignment.build_ast(self, left, right)
end
})
table.insert(generated, assignment_call {
operator = operator,
identifier = identifier,
build_ast = function(self, left, right)
right = Call:new(Identifier:new(infix_identifier), ArgumentTuple:new(left, right))
return assignment_call.build_ast(self, left, right)
end
})
end
return generated

28
parser/expression/secondary/infix/call.lua Normal file
View file

@ -0,0 +1,28 @@
local infix = require("parser.expression.secondary.infix.infix")
local escape = require("common").escape
local identifier = require("parser.expression.primary.identifier")
local operator_priority = require("common").operator_priority
local ast = require("ast")
local Call, ArgumentTuple = ast.Call, ast.ArgumentTuple
return infix {
operator = "!",
identifier = "_!_",
priority = operator_priority["_!_"],
match = function(self, str, current_priority, primary)
local escaped = escape(self.operator)
return self.priority > current_priority and str:match("^"..escaped) and identifier:match(str:match("^"..escaped.."%s*(.-)$"))
end,
build_ast = function(self, left, right)
if Call:is(right) then
right.arguments:insert_positional(1, left)
return right
else
return Call:new(right, ArgumentTuple:new(left))
end
end
}

Some files were not shown because too many files have changed in this diff Show more