Update README.md

for some reason gmatch("[^\n]*") matches *two times* per line (one with the line, and an empty string) in Lua 5.1 and LuaJIT, but behave as expected in Lua 5.2/5.3...

LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2017-2019 Étienne "Reuh" Fildadut
+Copyright (c) 2017-2021 Étienne "Reuh" Fildadut
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
 

README.md

@@ -1,39 +1,59 @@
 Candran
 =======
-Candran is a dialect of the [Lua 5.3](http://www.lua.org) programming language which compiles to Lua 5.3 and Lua 5.1/LuaJit. It adds several useful syntax additions which aims to make Lua faster and easier to write, and a simple preprocessor.
+Candran is a dialect of the [Lua 5.4](http://www.lua.org) programming language which compiles to Lua 5.4, Lua 5.3, Lua 5.2, LuaJIT and Lua 5.1 compatible code. It adds several useful syntax additions which aims to make Lua faster and easier to write, and a simple preprocessor.
 
-Unlike Moonscript, Candran tries to stay close to the Lua syntax, and existing Lua code can run on Candran unmodified.
+Unlike Moonscript, Candran tries to stay close to the Lua syntax, and existing Lua code should be able to run on Candran unmodified.
 
 ````lua
 #import("lib.thing") -- static import
-#local debug or= false
+#local DEBUG = false
+
+#if DEBUG then
+#	define("log(...)", "print(...)") -- macro: calls to log() will be replaced with print() in compiled code
+#else
+#	define("log(...)", "") -- remove calls to log from the compiled code when DEBUG is true
+#end
+log("example macro") -- preprocessor macros
 
 local function calculate(toadd=25) -- default parameters
 	local result = thing.do()
 	result += toadd
-    #if debug then -- preprocessor conditionals
-        print("Did something")
-    #end
+	#if DEBUG then -- preprocessor conditionals
+		print("Did something")
+	#end
 	return result
 end
 
 let a = {
-	hey = true,
+	hey = 5 // 2, -- Lua 5.3+ syntax, that will be translated to work with the current Lua version
+
+	child = nil,
 
 	method = :(foo, thing) -- short function declaration, with self
 		@hey = thing(foo) -- @ as an alias for self
 	end,
 
 	selfReference = () -- short function declaration, without self
-		return a -- no need for a prior local declaration using let
+		return a -- no need for a prior local declaration when using let
 	end
 }
 
+const five = 5 -- shortcut for Lua 5.4 attributes 
+
 a:method(42, (foo)
 	return "something " .. foo
 end)
 
-local odd = [ -- table comprehension (kind of)
+local fn = a:method -- bundles an object and method in a function
+fn(42, (foo)
+	return "something" .. foo
+end)
+
+a.child?:method?() -- safe navigation operator
+
+local {hey, method} = a -- destructuring assignement
+
+local odd = [ -- table comprehension
 	for i=1, 10 do
 		if i%2 == 0 then
 			continue -- continue keyword
@@ -46,30 +66,46 @@ local count = [for i=1,10 i] -- single line statements
 
 local a = if condition then "one" else "two" end -- statement as expressions
 
-print("Hello %s":format("world")) -- methods calls on strings (and tables) litterals without enclosing parantheses
+print("Hello %s":format("world")) -- methods calls on strings (and tables) litterals without enclosing parentheses
+
+if f, err = io.open("data") then -- if condition with assignements
+	thing.process(f)
+else
+	error("can't open data: "..err)
+end
 
 ````
 
+**Current status**: Candran is heavily used in several of my personal projects and works as expected.
+
 Candran is released under the MIT License (see ```LICENSE``` for details).
 
 #### Quick setup
-Install Candran automatically using LuaRocks: ```sudo luarocks install rockspec/candran-0.8.0-1.rockspec```.
+Install Candran automatically using LuaRocks: ```sudo luarocks install candran```.
 
-Or manually install LPegLabel (```luarocks install LPegLabel```, version 1.5 or above), download this repository and use Candran through the scripts in ```bin/``` or use it as a library with the self-contained ```candran.lua```.
+Or manually install LPegLabel and argparse (```luarocks install lpeglabel```, version 1.5 or above, and ```luarocks install argparse```, version 0.7 or above), download this repository and use Candran through the scripts in ```bin/``` or use it as a library with the self-contained ```candran.lua```.
+
+You can optionally install lua-linenoise (```luarocks install linenoise```, version 0.9 or above) for an improved REPL, and luacheck (```luarocks install luacheck```, version 0.23.0 or above) to be able to use ```cancheck```. Installing Candran using LuaRocks will install linenoise and luacheck by default.
 
 You can register the Candran package searcher in your main Lua file (`require("candran").setup()`) and any subsequent `require` call in your project will automatically search for Candran modules.
 
+If you use LÖVE, some integration with Candran is detailled [here](https://github.com/Reuh/candran/wiki/L%C3%96VE).
+
 #### Editor support
 Most editors should be able to use their existing Lua support for Candran code. If you want full support for the additional syntax in your editor:
 * **Sublime Text 3**:
 	* [sublime-candran](https://github.com/Reuh/sublime-candran) support the full Candran syntax
-	* [SublimeLinter-candran-contrib](https://github.com/Reuh/SublimeLinter-contrib-candran) SublimeLinter plugin for Candran
+	* [SublimeLinter-cancheck-contrib](https://github.com/Reuh/SublimeLinter-contrib-cancheck) SublimeLinter plugin for Candran using ```cancheck```
+	* [SublimeLinter-candran-contrib](https://github.com/Reuh/SublimeLinter-contrib-candran) SublimeLinter plugin for Candran using ```canc -parse``` (only checks for syntaxic errors, no linting)
+* **VS Code**: [vscode-candran](https://github.com/Reuh/vscode-candran) basic support for the Candran syntax
 * **Atom**: [language-candran](https://atom.io/packages/language-candran) support the full Candran syntax
 
+For linting, if your editor support [luacheck](https://github.com/luarocks/luacheck), you should be able to replace it with ```cancheck``` (in this repository ```bin/cancheck```, or installed automatically if Candran was installed using LuaRocks), which is a wrapper around luacheck that monkey-patch it to support Candran.
+
 The language
 ------------
 ### Syntax additions
-After the preprocessor is run the Candran code is compiled to Lua. Candran code adds the folowing syntax to Lua:
+After the [preprocessor](#preprocessor) is run the Candran code is compiled to Lua. Candran code adds the folowing syntax to Lua 5.4 syntax:
 
 ##### Assignment operators
 * ````var += nb````
@@ -91,21 +127,21 @@ For example, a ````var += nb```` assignment will be compiled into ````var = var
 
 All theses operators can also be put right of the assigment operator, in which case ```var =+ nb``` will be compiled into ```var = nb + var```.
 
-If you feel like writing hard to understand code, right and left operator can be used at the same time.
+Right and left operator can be used at the same time.
 
-**Please note** that the Lua code `a=-1` will be compiled into `a = 1 - a` and not `a = -1`! Write good code, write spaced code: `a = -1` works as expected.
+**Please note** that the code `a=-1` will be compiled into `a = -1` and not `a = a - 1`, like in pure Lua. If you want the latter, spacing is required between the `=-` and the expression: `a=- 1`. Yes, this is also valid Lua code, but as far as I'm aware, nobody write code like this; people who really like spacing would write `a= - 1` or `a = - 1`, and Candran will read both of those as it is expected in pure Lua. This is the only incompatibility between Candran and pure Lua.
 
 ##### Default function parameters
 ```lua
 function foo(bar = "default", other = thing.do())
-    -- stuff
+	-- stuff
 end
 ```
-If an argument isn't provided or ```nil``` when the function is called, it will be automatically set to a default value.
+If an argument isn't provided or set to ```nil``` when the function is called, it will automatically be set to its default value.
 
 It is equivalent to doing ```if arg == nil then arg = default end``` for each argument at the start of the function.
 
-The default values can be complete Lua expressions, and will be evaluated each time the function is run.
+The default values can be any Lua expression, which will be evaluated in the function's scope each time the default value end up being used.
 
 ##### Short anonymous function declaration
 ```lua
@@ -119,7 +155,7 @@ end
 ```
 Anonymous function (functions values) can be created in a more concise way by omitting the ```function``` keyword.
 
-A ```:``` can prefix the parameters paranthesis to automatically add a ```self``` parameter.
+A ```:``` can prefix the parameters parenthesis to automatically add a ```self``` parameter.
 
 ##### `@` self aliases
 ```lua
@@ -148,8 +184,20 @@ let a = {
 
 Similar to ```local```, but the variable will be declared *before* the assignemnt (i.e. it will compile into ```local a; a = value```), so you can access it from functions defined in the value.
 
+This does not support Lua 5.4 attributes.
+
 Can also be used as a shorter name for ```local```.
 
+##### `const` and `close` variable declaration
+```lua
+const a = 5
+close b = {}
+
+const x, y, z = 1, 2, 3 -- every variable will be defined using <const>
+```
+
+Shortcut to Lua 5.4 variable attribute. Do not behave like `let`, as attributes require the variable to be constant and therefore can't be predeclared. Only compatibel with Lua 5.4 target.
+
 ##### `continue` keyword
 ```lua
 for i=1, 10 do
@@ -177,9 +225,9 @@ push "hey" -- Does *not* work, because it is a valid Lua syntax for push("hey")
 
 Add one or more value to the returned value list. If you use a `return` afterwards, the pushed values will be placed *before* the `return` values, otherwise the function will only return what was pushed.
 
-This keyword is mainly useful when used through implicit `push` with table comprehension and statement expressions.
+In particular, this keyword is useful when used through implicit `push` with table comprehension and statement expressions.
 
-**Please note** that, in order to stay compatible with vanilla Lua syntax, any `push` immediatly followed by a `"string expression"`, `{table expression}` or `(paranthesis)` will be interpreted as a function call. It's recommended to use the implicit `push` instead (when possible).
+**Please note** that, in order to stay compatible with vanilla Lua syntax, any `push` immediatly followed by a `"string expression"`, `{table expression}` or `(parenthesis)` will be interpreted as a function call. It's recommended to use the implicit `push` when possible.
 
 ##### Implicit `push`
 ```lua
@@ -197,7 +245,7 @@ local square = (x) x*x end -- function(x) return x*x end
 
 Any list of expressions placed *at the end of a block* will be converted into a `push` automatically.
 
-**Please note** that this doesn't work with `v()` function calls, because these are already valid statements. Use `push v()` instead.
+**Please note** that this doesn't work with `v()` function calls, because these are already valid statements. Use `push v()` in this case.
 
 ##### Table comprehension
 ```lua
@@ -224,12 +272,101 @@ You can write *any* code you want between `[` and `]`, this code will be run as
 
 Values returned by the function will be inserted in the generated table in the order they were returned. This way, each time you `push` value(s), they will be added to the table.
 
-The table generation function also have access to the `self` (or its alias `@`) variable, which is the table which is being created, so you can set arbitrary fields of the table.
+The table generation function also have access to the `self` variable (and its alias `@`), which is the table which is being created, so you can set any of the table's field.
+
+##### Destructuring assignement
+```lua
+t = { x = 1, y = 2, z = 3 }
+
+{x, y, z} = t -- x, y, z = t.x, t.y, t.z
+
+{x = o} = t -- o = t.x
+
+{["x"] = o} = t -- o = t["x"]
+
+-- Also works with local, let, for ... in, if with assignement, +=, etc.
+local {x, y} = t
+let {x, y} = t
+for i, {x, y} in ipairs{t} do end
+if {x, y} = t then end
+{x} += t -- x = x + t.x
+
+-- Works as expected with multiple assignement.
+a, {x, y, z}, b = 1, t, 2
+
+```
+
+Destruturing assignement allows to quickly extract fields from a table into a variable.
+
+This is done by replacing the variable name in any assignement with a table literal, where every item is the name of the field and assigned variable. It is possible to use a different field name than the variable name by naming the table item (`fieldName = var` or `[fieldExpression] = var`).
+
+
+##### Safe navigation operators
+```lua
+a = nil
+print(a?.b) -- nil
+
+a = {b=true}
+print(a?.b) -- true
+
+-- So instead of typing
+if object and object.child and object.child.isGreen then
+	-- stuff
+end
+-- you can type
+if object?.child?.isGreen then
+	-- stuff
+end
+
+-- The ?. operator does not break the whole chain; make sure to use the operator on each index.
+print(a?.undefined.field) -- a?.undefined returns nil, so this throws a "attempt to index a nil value"
+
+-- Other safe navigator operators behave similarly:
+print(a:method) -- nil if a is nil, other normal behaviour
+print(a["key"]) -- nil if a is nil, other normal behaviour
+print(a?()) -- nil if a is nil, other normal behaviour
+```
+
+Some operators can be prefixed by a `?` to turn into a safe version of the operator: if the base value if `nil`, the normal behaviour of the operator will be skipped and nil will be returned; otherwise, the operator run as usual. Is available safe dot index `?.`, safe array index `?[...]`, safe method stub `?:` and safe function call `?(...)`.
+
+##### If and while with assignement in the condition
+```lua
+if f, err = io.open("somefile") then -- condition if verified if f is a truthy value (not nil or false)
+	-- do something with f
+	f:close()
+elseif f2, err2 = io.open("anotherfile") then -- same behaviour on elseif
+	print("could not open somefile:", err) -- f and err stay in scope for the rest of the if-elseif-else block
+	-- do something with f2
+	f2:close()
+else
+	print("could not open somefile:", err)
+	print("could not open anotherfile:", err2)
+end
+-- f, err, f2 and err2 are now out of scope
+
+if (value = list[index = 2]) and yes = true then -- several assignements can be performed, anywhere in the expression; index is defined before value, yes is defined after these two. The condition is verified if both value and yes are thruthy.
+	print(index, value)
+end
+
+-- When used in a while, the expression is evaluated at each iteration.
+while line = io.read() do
+	print(line)
+end
+
+-- The assignement have the same priority as regular assignements, i.e., the lowest.
+if a = 1 and 2 then -- will be read as a = (1 and 2)
+elseif (a = 1) and 2 then -- will be read as (a = 1) and 2
+end
+```
+
+Assignements can be used in the condition of if, elseif and while statements. Several variables can be assigned; only the first will be tested in the condition, for each assignement. The assigned variables will be in scope the duration of the block; for if statements, they will also be in scope for the following elseif(s) and else.
+
+For while statements, the assigned expression will be reevaluated at each iteration.
 
 ##### Suffixable string and table litterals
 ```lua
-"some text":upper() -- same as ("some text"):upper() in Lua
-"string".upper -- the actual string.upper function. "string"["upper"] also works.
+"some text":upper() -- "SOME TEXT". Same as ("some text"):upper() in Lua.
+"string".upper -- the string.upper function. "string"["upper"] also works.
 
 {thing = 3}.thing -- 3. Also works with tables!
 [for i=0,5 do i*i end][3] -- 9. And table comprehensions!
@@ -238,10 +375,31 @@ The table generation function also have access to the `self` (or its alias `@`)
 someFunction"thing":upper() -- same as (someFunction("thing")):upper() (i.e., the way it would be parsed by Lua)
 ```
 
-String litterals, table litterals, and comprehensions can be suffixed with `:` method calls, `.` indexing, or `[` indexing, without needing to be enclosed in parantheses.
+String litterals, table litterals, and comprehensions can be suffixed with `:` method calls, `.` indexing, or `[` indexing, without needing to be enclosed in parentheses.
 
 **Please note**, that "normal" functions calls have priority over this syntax, in order to maintain Lua compatibility.
 
+##### Method stubs
+```lua
+object = {
+	value = 25,
+	method = function(self, str)
+		print(str, self.value)
+	end
+}
+
+stub = object:method
+
+object.method = error -- stub stores the method as it was when stub was defined
+object = nil -- also stores the object
+
+print(stub("hello")) -- hello	25
+```
+
+Create a closure function which bundles the variable and its method; when called it will call the method on the variable, without requiring to pass the variable as a first argument.
+
+The closure stores the value of the variable and method when created.
+
 ##### Statement expressions
 ```lua
 a = if false then
@@ -255,7 +413,7 @@ a, b, c = for i=1,2 do i end
 print(a, b, c) -- 1, 2, nil
 ```
 
-Candran allows to use `if`, `do`, `while`, `repeat` and `for` statements as expressions. Their content will be run as if they were run in a separate function which is immediatly run.
+`if`, `do`, `while`, `repeat` and `for` statements can be used as expressions. Their content will be run as if they were run in a separate function which is immediatly run.
 
 ##### One line statements
 ```lua
@@ -271,7 +429,9 @@ else -- "end" is always needed for else!
 end
 ```
 
-`if`, `elseif`, `for`, and `while` statements can be writtent without `do`, `then` or `end`, in which case they contain a single statement.
+`if`, `elseif`, `for`, and `while` statements can be written without `do`, `then` or `end`, in which case they contain a single statement.
+
+**Please note** that an `end` is always required for `else` blocks.
 
 ### Preprocessor
 Before compiling, Candran's preprocessor is run. It execute every line starting with a _#_ (ignoring prefixing whitespace, long strings and comments) as Candran code.
@@ -287,30 +447,78 @@ For example,
 
 Will output ````print("Bonjour")```` or ````print("Hello")```` depending of the "lang" argument passed to the preprocessor.
 
-The preprocessor has access to the following variables :
-* ````candran```` : the Candran library table.
-* ````output```` : the current preprocessor output string.
-* ````import(module[, [options])```` : a function which import a module. This should be equivalent to using _require(module)_ in the Candran code, except the module will be embedded in the current file. _options_ is an optional preprocessor arguments table for the imported module (current preprocessor arguments will be inherited). Options specific to this function: ```loadLocal``` (default ```true```): ```true``` to automatically load the module into a local variable (i.e. ```local thing = require("module.thing")```); ```loadPackage``` (default ```true```): ```true``` to automatically load the module into the loaded packages table (so it will be available for following ```require("module")``` calls).
-* ````include(filename)```` : a function which copy the contents of the file _filename_ to the output.
-* ````write(...)```` : write to the preprocessor output. For example, ````#print("hello()")```` will output ````hello()```` in the final file.
-* ```placeholder(name)``` : if the variable _name_ is defined in the preprocessor environement, its content will be inserted here.
-* ````...```` : each arguments passed to the preprocessor is directly available.
+The preprocessor has access to the following variables:
+* ````candran````: the Candran library table.
+* ````output````: the current preprocessor output string. Can be redefined at any time. If you want to write something in the preprocessor output, it is preferred to use `write(...)` instead of directly modifying `output`.
+* ````import(module[, [options])````: a function which import a module. This should be equivalent to using _require(module)_ in the Candran code, except the module will be embedded in the current file. Macros and preprocessor constants defined in the imported file (using `define` and `set`) will be made available in the current file. _options_ is an optional preprocessor arguments table for the imported module (current preprocessor arguments will be inherited). Options specific to this function:
+	* ```loadLocal``` (default ```true```): ```true``` to automatically load the module into a local variable (i.e. ```local thing = require("module.thing")```)
+	* ```loadPackage``` (default ```true```): ```true``` to automatically load the module into the loaded packages table (so it will be available for following ```require("module")``` calls).
+* ````include(filename)````: a function which copy the contents of the file _filename_ to the output.
+* ````write(...)````: write to the preprocessor output. For example, ````#write("hello()")```` will output ````hello()```` in the final file.
+* ```placeholder(name)```: if the variable _name_ is defined in the preprocessor environement, its content will be inserted here.
+* ```define(identifier, replacement)```: define a macro. See below.
+* ```set(identifier, value)```: set a preprocessor constant.
+* each arguments passed to the preprocessor is directly available in the environment.
 * and every standard Lua library.
 
+#### Macros
+
+Using `define(identifier, replacement)` in the preprocessor, you can define macros. `identifier` is expected to be string containing Candran/Lua code (representing either a identifier or a function call), and `replacement` can be either a string containing Candran/Lua code or a function.
+
+There are two types of macros identifiers: variables, which replace every instance of the given identifier with the replacement; and functions, which will replace every call to this function with the replacement, also replacing its arguments. The `...` will be replaced with every remaining argument. Macros can not be recursive.
+
+If `replacement` is a string, the macro will be replaced with this string, replacing the macros arguments in the string. If `replacement` is a function, the function will be called every time the macro is encoutered, with the macro arguments passed as strings, and is expected to return a string that will be used as a replacement.
+
+If `replacement` is the empty empty, the macro will simply be removed from the compiled code.
+
+```lua
+-- Variable macro
+#define("x", 42)
+print(x) -- 42
+
+-- Function macros
+#define("f(x)", "print(x)")
+f(42) -- replaced with print(42)
+
+#define("log(s, ...)", "print(s..": ", ...)")
+log("network", "error") -- network: error
+
+#define("debug()", "")
+debug() -- not present in complied code
+
+#define("_assert(what, err)", function(what, err)
+#	return "if "..what.." then error("..err..") end"
+#end)
+_assert(5 = 2, "failed") -- replaced with if 5 = 2 then error("failed") end
+```
+
+Candran provide some predefined macros by default:
+* `__STR__(expr)`: returns a string litteral representing the expression (e.g., `__STR__(5 + 2)` expands to `"5 + 2"`)
+* `__CONSTEXPR__(expr)`: calculate the result of the expression in the preprocessor, and returns a representation of the returned value, i.e. precalculate an expression at compile time
+You can disable these built-in macros using the `builtInMacros` compiler option.
+
 Compile targets
 ---------------
-Candran is based on the Lua 5.3 syntax, but can be compiled to both Lua 5.3 and Lua 5.1/LuaJit.
+Candran is based on the Lua 5.4 syntax, but can be compiled to Lua 5.4, Lua 5.3, Lua 5.2, LuaJIT, and Lua 5.1 compatible code.
 
-To chose a compile target, set the ```target``` option to ```lua53``` (default) or ```luajit``` in the option table when using the library or the command line tools.
+To chose a compile target, set the ```target``` option to  ```lua54```, ```lua53```, ```lua52```, ```luajit```, or ```lua51``` in the option table when using the library or the command line tools. Candran will try to detect the currently used Lua version and use it as the default target.
 
-Lua 5.3 specific syntax (bitwise operators, integer division) will automatically be translated in valid Lua 5.1 code, using LuaJIT's ```bit``` library if necessary. Unless you require LuaJIT's library, you won't be able to use bitwise operators with simple Lua 5.1 ("PUC Lua").
+Candran will try to translate Lua 5.4 syntax into something usable with the current target if possible. Here is what is currently supported:
+
+| Lua version     | Candran target | Integer division operator // | Bitwise operators                                     | Goto/Labels        | Variable attributes |
+| ---             | ---            | ---                          | ---                                                   | ---                | ---                 |
+| Lua 5.4         | lua54          | :white_check_mark:           | :white_check_mark:                                    | :white_check_mark: | :white_check_mark:  |
+| Lua 5.3         | lua53          | :white_check_mark:           | :white_check_mark:                                    | :white_check_mark: | X                   |
+| Lua 5.2         | lua52          | :white_check_mark:           | :white_check_mark: (32bit)                            | :white_check_mark: | X                   |
+| LuaJIT          | luajit         | :white_check_mark:           | :white_check_mark: (32bit)                            | :white_check_mark: | X                   |
+| Lua 5.1         | lua51          | :white_check_mark:           | :white_check_mark: if LuaJIT bit library is available (32bit) | X                  | X                   |
 
 **Please note** that Candran only translates syntax, and will not try to do anything about changes in the Lua standard library (for example, the new utf8 module). If you need this, you should be able to use [lua-compat-5.3](https://github.com/keplerproject/lua-compat-5.3) along with Candran.
 
 Usage
 -----
 ### Command-line usage
-The library can be used standalone through the ```canc``` and ```can``` utility:
+The library can be used standalone through the ```canc``` (for compiling Candran files) and ```can``` (for running Candran files directly) utilities:
 
 *	````canc````
 
@@ -320,66 +528,98 @@ The library can be used standalone through the ```canc``` and ```can``` utility:
 
 	Preprocess and compile each  _filename_ Candran files, and creates the assiociated ```.lua``` files in the same directories.
 
-	_options_ is of type ````-somearg -anotherarg thing=somestring other=5 ...````, which will generate a Lua table ```{ somearg = true, anotherarg = true, thing = "somestring", other = 5 }```.
+	_options_ is of type ````--no-map-lines -p --include module -d VAR 5````.
 
-    You can choose to use another directory where files should be written using the ```dest=destinationDirectory``` argument.
+	You can choose to use another directory where files should be written using the `--destination` or `-d` option: ```--destination destinationDirectory```.
 
-    You can choose the output filename using ```out=filename```. By default, compiled files have the same name as their input file, but with a ```.lua``` extension.
+	You can choose the output filename using `--output` or `-o` option: `--output filename`. By default, compiled files have the same name as their input file, but with a ```.lua``` extension.
 
-    ```canc``` can write to the standard output instead of creating files using the ```-print``` argument.
+	```canc``` can write to the standard output instead of creating files using the ```--print``` or `-p` argument.
 
-	You can choose to run only the preprocessor or compile using the ```-preprocess``` and ```-compile``` flags.
+	You can choose to run only the preprocessor or compile using the ```--preprocess``` and ```--compile``` flags.
 
-	You can choose to only parse the file and check it for syntaxic errors using the ```-parse``` flag. Errors will be printed to stderr in a similar format to ```luac -p```.
+	You can choose to only parse the file and check it for syntaxic errors using the ```--parse``` flag. Errors will be printed to stderr in a similar format to ```luac -p```.
 
-	The ```-ast``` flag is also available for debugging, and will disable preprocessing, compiling and file writing, and instead directly dump the AST generated from the input file(s) to stdout.
+	The ```--ast``` flag is also available for debugging, and will disable preprocessing, compiling and file writing, and instead directly dump the AST generated from the input file(s) to stdout.
 
 	Instead of providing filenames, you can use ```-``` to read from standard input.
 
-	* example uses :
+	You can change the compiler target using `--target` or `-t`: `--target luajit`.
 
-		````canc foo.can````
+	You can change the identation and newline string using `--indentation` and `--newline`: `--identation luajit`.
+
+	You can change Candran's built-in variable prefix using `--variable-prefix`: `--variable-prefix __CAN_`.
+
+	You can disable line mapping (error rewriting will not work) using `--no-map-lines`.
+
+	You can disable built-in macros using `--no-builtin-macros`.
+
+	You can define preprocessor constants using `--define` or `-D`: `--define VAR 5`. `VAR` will be available and set to 5 in the preprocessor. If you specify no value, it defaults to true.
+
+	You can statically import modules using `--import` or `-I`: `--import module`. The module will be imported in compiled files using `#import("module",{loadLocal=false})`.
+
+	You can disable error rewriting using `--no-rewrite-errors`.
+
+	You can change the chunkname using `--chunkname`: `--chunkname filename`. This will change the filenames are reported in errors. By default, try to use the current file name, or stdin when using `-`.
+
+	Use the ```-h``` or ```--help``` option to display the help text.
+
+	Example uses:
+
+	* ````canc foo.can````
 
 		preprocess and compile _foo.can_ and write the result in _foo.lua_.
 
-		````canc indentation="  " foo.can````
+	* ````canc --indentation "  " foo.can````
 
 		preprocess and compile _foo.can_ with 2-space indentation (readable code!) and write the result in _foo.lua_.
 
-		````canc foo.can -verbose -print | lua````
+	* ````canc foo.can -d verbose --print | lua````
 
-		preprocess _foo.can_ with _verbose_ set to _true_, compile it and execute it.
+		preprocess _foo.can_ with _verbose_ set to _true_ in the preprocessor, compile it and execute it.
 
-		````canc -parse foo.can````
+	* ````canc --parse foo.can````
 
 		checks foo.can for syntaxic errors.
 
 *   ```can```
 
-    Start a simplisitic Candran REPL.
+	Start a simplisitic Candran REPL. Will automatically call `candran.setup()`.
+
+	If you want a better REPL (autocompletion, history, ability to move the cursor), install lua-linenoise: ```luarocks install linenoise``` (automatically installed if Candran was installed using LuaRocks).
 
 *	````can [options] filename````
 
-    Preprocess, compile and run _filename_ using the options provided.
+	Preprocess, compile and run _filename_ using the options provided.
 
-	This will automatically register the Candran package searcher so required file will be compiled as they are needed.
+	This will automatically register the Candran package searcher using `candran.setup()`, so required Candran modules will be compiled as they are needed.
 
-	This command will use error rewriting if enabled.
+	This command will use error rewriting unless explicitely enabled (by setting the `rewriteErrors=false` option).
 
 	Instead of providing a filename, you can use ```-``` to read from standard input.
 
-### Library usage
-Candran can also be used as a Lua library. For example,
-````lua
-local candran = require("candran")
+	Use similar options as `canc`.
 
-local f = io.open("foo.can")
+	Use the ```-h``` or ```-help``` option to display the help text.
+
+*   ```cancheck```
+
+	Provides a linter and static analyzer with the exact same interface as [luacheck](https://github.com/luarocks/luacheck).
+
+	This requires luacheck: ```luarocks install luacheck``` (automatically installed if Candran was installed through LuaRocks).
+
+### Library usage
+Candran can also be used as a Lua library:
+````lua
+local candran = require("candran") -- load Candran
+
+local f = io.open("foo.can") -- read the file foo.can
 local contents = f:read("*a")
 f:close()
 
-local compiled = candran.make(contents, { lang = "fr" })
+local compiled = candran.make(contents, { DEBUG = true }) -- compile foo.can with DEBUG set to true
 
-load(compiled)()
+load(compiled)() -- execute!
 
 -- or simpler...
 candran.dofile("foo.can")
@@ -388,20 +628,19 @@ candran.dofile("foo.can")
 candran.setup()
 local foo = require("foo")
 ````
-Will load Candran, read the file _foo.can_, compile its contents with the argument _lang_ set to _"fr"_, and then execute the result.
 
-The table returned by _require("candran")_ gives you access to :
+The table returned by _require("candran")_ gives you access to:
 
-##### Compiler & preprocessor API
-* ````candran.VERSION```` : Candran's version string.
-* ````candran.preprocess(code[, options])```` : return the Candran code _code_, preprocessed with _options_ as options table.
-* ````candran.compile(code[, options])```` : return the Candran code compiled to Lua with _options_ as the option table.
-* ````candran.make(code[, options])```` : return the Candran code, preprocessed and compiled with _options_ as options table.
+##### Compiler & preprocessor
+* ````candran.VERSION````: Candran's version string (e.g. `"0.10.0"`).
+* ````candran.preprocess(code[, options])````: return the Candran code _code_, `macros` table. The code is preprocessed with the _options_ options table; `macros` is indented to be passed to `candran.compile` to apply the defined macros. In case of error, returns nil, error.
+* ````candran.compile(code[, options[, macros]])````: return the Candran code compiled to Lua with the _options_ option table and the macros `macros` (table returned by the preprocessor); or nil, err in case of error.
+* ````candran.make(code[, options])````: return the Candran code, preprocessed and compiled with the _options_ options table; or nil, err in case of error.
 
 ##### Code loading helpers
-* ```candran.loadfile(filepath, env, options)``` : Candran equivalent to the Lua 5.3's loadfile funtion. Will rewrite errors by default.
-* ```candran.load(chunk, chunkname, env, options)``` : Candran equivalent to the Lua 5.3's load funtion. Will rewrite errors by default.
-* ```candran.dofile(filepath, options)``` : Candran equivalent to the Lua 5.3's dofile funtion. Will rewrite errors by default.
+* ```candran.loadfile(filepath, env, options)```: Candran equivalent to the Lua 5.4's loadfile funtion. Will rewrite errors by default.
+* ```candran.load(chunk, chunkname, env, options)```: Candran equivalent to the Lua 5.4's load funtion. Will rewrite errors by default.
+* ```candran.dofile(filepath, options)```: Candran equivalent to the Lua 5.4's dofile funtion. Will rewrite errors by default.
 
 #### Error rewriting
 When using the command-line tools or the code loading helpers, Candran will automatically setup error rewriting: because the code is reformated when
@@ -420,11 +659,18 @@ If you are using the preprocessor ```import()``` function, the source Candran fi
 example.can:12(final.lua:5): attempt to call a nil value (global 'iWantAnError')
 ```
 
-* ```candran.messageHandler(message)``` : The error message handler used by Candran. Use it in xpcall to rewrite stacktraces to display Candran source file lines instead of compiled Lua lines.
+Please note that Candran can only wrap code directly called from Candran; if an error is raised from Lua, there will be no rewriting of Candran lines the stacktrace. These lines are indicated using `(compiled candran)` before the line number.
+
+If you want Candran to always wrap errors, you will need to wrap your whole code in a `xpcall`: `xpcall(func, candran.messageHandler)`.
+
+* ```candran.messageHandler(message[, noTraceback])```: the error message handler used by Candran. Given `message` the Lua error string, returns full Candran traceback where soure files and lines are rewritten to their Candran source. You can use it as is in xpcall as a message handler. If `noTraceback` is `true`, Candran will only rewrite `message` and not add a new traceback.
+
+Also note that the Candran message handler will add a new, rewritten, stacktrace to the error message; it can't replace the default Lua one. You will therefore see two stacktraces when raising an error, the last one being the Lua one and can be ignored.
 
 ##### Package searching helpers
-Candran comes with a custom package searcher which will automatically find, preprocesses and compile ```.can``` files. If you want to use Candran in your project without worrying about
-compiling the files, you can simply call
+Candran comes with a custom package searcher which will automatically find, preprocesses and compile ```.can``` files.
+
+If you want to use Candran in your project without worrying about compiling the files, you can simply call
 
 ```lua
 require("candran").setup()
@@ -432,38 +678,41 @@ require("candran").setup()
 
 at the top of your main Lua file. If a Candran file is found when you call ```require()```, it will be automatically compiled and loaded. If both a Lua and Candran file match a module name, the Candran file will be loaded.
 
-* ```candran.searcher(modpath)``` : Candran package searcher function. Use the existing package.path.
-* ```candran.setup()``` : Register the Candran package searcher, and return the `candran` table.
+* ```candran.searcher(modpath)```: Candran package searcher function. Use the existing package.path.
+* ```candran.setup()```: register the Candran package searcher (if not already done), and return the `candran` table.
 
 ##### Available compiler & preprocessor options
-You can give arbitrary options which will be gived to the preprocessor, but Candran already provide and uses these with their associated default values:
+You can give arbitrary options to the compiler and preprocessor, but Candran already provide and uses these with their associated default values:
 
 ```lua
-target = "lua53" -- Compiler target. "lua53" or "luajit" (default is automatically selected based on the Lua version used).
-indentation = "" -- Character(s) used for indentation in the compiled file.
-newline = "\n" -- Character(s) used for newlines in the compiled file.
+target = "lua53" -- compiler target. "lua54", "lua53", "lua52", "luajit" or "lua51" (default is automatically selected based on the Lua version used).
+indentation = "" -- character(s) used for indentation in the compiled file.
+newline = "\n" -- character(s) used for newlines in the compiled file.
 variablePrefix = "__CAN_" -- Prefix used when Candran needs to set a local variable to provide some functionality (example: to load LuaJIT's bit lib when using bitwise operators).
-mapLines = true -- If true, compiled files will contain comments at the end of each line indicating the associated line and source file. Needed for error rewriting.
-chunkname = "nil" -- The chunkname used when running code using the helper functions and writing the line origin comments. Candran will try to set it to the original filename if it knows it.
-rewriteErrors = true -- True to enable error rewriting when loading code using the helper functions. Will wrap the whole code in a xpcall().
+mapLines = true -- if true, compiled files will contain comments at the end of each line indicating the associated line and source file. Needed for error rewriting.
+chunkname = "nil" -- the chunkname used when running code using the helper functions and writing the line origin comments. Candran will try to set it to the original filename if it knows it.
+rewriteErrors = true -- true to enable error rewriting when loading code using the helper functions. Will wrap the whole code in a xpcall().
+builtInMacros = true -- false to disable built-in macros __*__
+preprocessorEnv = {} -- environment to merge with the preprocessor environement
+import = {} -- list of modules to automatically import in compiled files (using #import("module",{loadLocal=false}))
 ```
 
-You can change these values in the table `candran.default`.
+You can change the defaults used for these variables in the table `candran.default`.
 
-There are also a few function-specific options available, see the associated functions documentation for more information.
+There are also a few function-specific options available, see the preprocessor functions documentation for more information.
 
 ### Compiling the library
 The Candran library itself is written is Candran, so you have to compile it with an already compiled Candran library.
 
 The compiled _candran.lua_ should include every Lua library needed to run it. You will still need to install LPegLabel.
 
-This command will use the precompilled version of this repository (candran.lua) to compile _candran.can_ and write the result in _candran.lua_ :
+This command will use the precompilled version of this repository (_candran.lua_) to compile _candran.can_ and write the result in _candran.lua_:
 
 ````
 canc candran.can
 ````
 
-You can then run the tests on your build :
+You can then run the tests on your build:
 
 ````
 cd test

bin/can

@@ -1,45 +1,187 @@
-#!/bin/lua
-local candran = require("candran")
-local cmdline = require("lib.cmdline")
+#!/usr/bin/env lua
 
-local args = cmdline(arg)
+local candran = require("candran").setup()
+local util = require("candran.util")
+local argparse = require("argparse")
+local unpack = unpack or table.unpack
 
-if args.help or args.h then
-	print("Candran "..candran.VERSION.." interpreter by Reuh")
-	print("Usage: "..arg[0].." [options] filename")
-	print("Use - instead of a filename to read from the standard input.")
-	print("Default options:")
-	for opt, val in pairs(candran.default) do
-		if type(val) == "string" then val = val:gsub("\n", "\\n") end
-		print(("  %s=%q"):format(opt, val))
-	end
-	return
-end
+-- Parse args --
 
-if arg[#arg] == "-" then
-	local f, err = candran.load(io.read("*a"), "stdin", nil, args)
+local parser = argparse()
+	:name "can"
+	:description("Candran "..candran.VERSION.." interpreter by Reuh.")
+	:epilog "For more info, see https://github.com/Reuh/candran"
+
+parser:argument("filename", "Candran file to run. Use - to read from standard input. Start the REPL if no filename given.")
+	:args "?"
+
+util.cli.addCandranOptions(parser)
+
+local args = parser:parse()
+
+local options = util.cli.makeCandranOptions(args)
+
+-- Run --
+
+-- stdin
+if args.filename == "-" then
+	local f, err = candran.load(io.read("*a"), "stdin", nil, options)
 	if not f then
 		io.stderr:write("can: "..err.."\n")
 		os.exit(1)
 	end
-	f()
-elseif #args >= 1 then
-	candran.dofile(args[1], args)
-else -- REPL
-	print("Candran " .. candran.VERSION)
+	local r, e = xpcall(f, candran.messageHandler)
+	if not r then
+		io.stderr:write(e.."\n")
+		os.exit(1)
+	end
+-- file
+elseif args.filename then
+	local f, err = candran.loadfile(args.filename, nil, options)
+	if not f then
+		io.stderr:write("can: "..err.."\n")
+		os.exit(1)
+	else
+		local r, e = xpcall(f, candran.messageHandler)
+		if not r then
+			io.stderr:write(e.."\n")
+			os.exit(1)
+		end
+	end
+-- REPL
+else
+	candran.default = util.merge(candran.default, options)
+
+	-- Setup linenoise
+	local s, l = pcall(require, "linenoise")
+	if not s then -- pure Lua compatibility thingy
+		l = {
+			linenoise = function(prompt)
+				io.write(prompt)
+				local s, line = pcall(io.read)
+				if not s then
+					if line == "interrupted!" then
+						return nil
+					else
+						return nil, err
+					end
+				end
+				return line
+			end,
+			historyadd = function() end,
+			setcompletion = function() end,
+			sethints = function() end,
+			enableutf8 = function() end
+		}
+	end
+	local keywords = {
+		-- Lua
+		"and", "break", "do", "else", "elseif", "end", "false", "for", "function", "goto",
+		"if", "in", "local", "nil", "not", "or", "repeat", "return", "then", "true",
+		"until", "while",
+		-- Candran
+		"continue", "let", "push"
+	}
+	l.enableutf8()
+	l.setcompletion(function(comp, line)
+		local var = line:match("[a-zA-Z_][a-zA-Z_0-9]*$")
+		if var then
+			for _, k in ipairs(keywords) do
+				if k:match("^"..var) then
+					comp:add(line .. k:sub(#var+1))
+				end
+			end
+			for k in pairs(_ENV) do
+				if k:match("^"..var) then
+					comp:add(line .. k:sub(#var+1))
+				end
+			end
+		end
+	end)
+	l.sethints(function(line)
+		local var = line:match("[a-zA-Z_][a-zA-Z_0-9]*$")
+		if var then
+			for _, k in ipairs(keywords) do
+				if k:match("^"..var) then
+					return k:sub(#var+1), { color = 2, bold = true }
+				end
+			end
+			for k in pairs(_ENV) do
+				if k:match("^"..var) then
+					return k:sub(#var+1), { color = 2 }
+				end
+			end
+		end
+	end)
+
+	-- Introduction
+	print("Candran " .. candran.VERSION .. ", targeting " .. candran.default.target)
 	candran.setup()
+
+	-- check errors in static import
+	-- note: static imports will be run every line, as preprocessors macros and constants aren't kept between compilations...
+	do
+		local r, e = candran.load("local _", "stdin")
+		if not r then
+			print("In static import: "..e)
+			candran.default.import = {}
+		else
+			r, e = pcall(r)
+			if not r then
+				print("In static import: "..e)
+				candran.default.import = {}
+			end
+		end
+	end
+
+	-- REPL loop
+	local multiline = false -- true if wait for another line
+	local buffer
 	while true do
-		io.write("> ")
-		local line = io.read()
-		if line:match("^=") then
-			line = line:gsub("^=", "return tostring(") .. ")"
+		local line, err = l.linenoise(multiline and ">> " or "> ")
+
+		-- exit
+		if not line then
+			if not err then
+				if multiline then
+					multiline = false
+					line = ""
+				else
+					return
+				end
+			else
+				error(err)
+			end
 		end
 
-		local t = { pcall(candran.load, line, "stdin") }
-		if t[1] == false then
-			print(t[2])
+		-- history
+		if line:match("[^%s]") then
+			l.historyadd(line)
+		end
+
+		-- multiline
+		if multiline then
+			buffer = buffer .. "\n" .. line
+			multiline = false
 		else
-			t = { pcall(t[2]) }
+			buffer = line
+		end
+
+		-- print shortcut 
+		if buffer:match("^=") then
+			buffer = buffer:gsub("^=", "return tostring(") .. ")"
+		end
+
+		-- exec
+		local r, e = candran.load(buffer, "stdin")
+		if not r then
+			if e:match("expected '[end})]+' to close") then
+				multiline = true
+			else
+				print(e)
+			end
+		else
+			local t = { pcall(r) }
 			if t[1] == false then
 				print(t[2])
 			elseif #t > 1 then

bin/canc

@@ -1,50 +1,67 @@
-#!/bin/lua
+#!/usr/bin/env lua
+
 local candran = require("candran")
-local cmdline = require("lib.cmdline")
-local parse = require("lib.lua-parser.parser").parse
-local pp = require("lib.lua-parser.pp")
+local parse = require("candran.can-parser.parser").parse
+local pp = require("candran.can-parser.pp")
+local util = require("candran.util")
+local argparse = require("argparse")
 
-if #arg < 1 then
-	print("Candran "..candran.VERSION.." compiler by Reuh")
-	print("Usage: "..arg[0].." [options] filenames...")
-	print("Use - instead of filenames to read from the standard input. The output file will be named stdin.lua by default.")
-	print("Compiler options:")
-	print("  dest=\"directory\"  where compiled files should be written")
-	print("  out=\"name.lua\"    output filename. By default, will use the same name as the input file with a .lua extension.")
-	print("  -print            write to the standard output instead of creating files")
-	print("  -preprocess       only run the preprocessor")
-	print("  -compile          only run the compiler")
-	print("  -parse            only parse the file and prints errors to stdout")
-	print("  -ast              (for debugging purposes) only parse the files and dump the AST to stdout")
-	print("Default options:")
-	for opt, val in pairs(candran.default) do
-		if type(val) == "string" then val = val:gsub("\n", "\\n") end
-		print(("  %s=%q"):format(opt, val))
-	end
-	return
-end
+-- Parse args --
 
-local args = cmdline(arg)
+local parser = argparse()
+	:name "canc"
+	:description("Candran "..candran.VERSION.." compiler by Reuh.")
+	:epilog "For more info, see https://github.com/Reuh/candran"
 
-if arg[#arg] == "-" then
-	table.insert(args, io.stdin)
-end
+parser:argument("filename", "Candran files to compile. Use - to read from standard input; the output file will then be named stdin.lua by default.")
+	:args "+"
 
-for _, file in ipairs(args) do
+parser:group("Output options",
+	parser:option("-d --destination")
+		:description "Where compiled files should be written"
+		:argname "directory",
+
+	parser:option("-o --output")
+		:description "Output filename. (default: same name as the input file with a .lua extension)"
+		:argname "filename",
+
+	parser:flag("-p --print")
+		:description "Write to the standard output instead of creating files",
+
+	parser:flag("--preprocess")
+		:description "Only run the preprocessor",
+
+	parser:flag("--compile")
+		:description "Only run the compiler",
+
+	parser:flag("--parse")
+		:description "Only parse the file and prints syntax errors to stdout",
+
+	parser:flag("--ast")
+		:description"(for debugging purposes) Only parse the files and dump the AST to stdout"
+)
+
+util.cli.addCandranOptions(parser)
+
+local args = parser:parse()
+
+-- Compile --
+
+for _, file in ipairs(args.filename) do
 	-- Read
 	local dest, input
-	if file == io.stdin then
-		dest = args.out or "stdin.lua"
+	if file == "-" then
+		dest = args.output or "stdin.lua"
 
 		input = io.read("*a")
 
 		args.chunkname = "stdin"
 	else
-		dest = args.out or (file:gsub("%.can$", "")..".lua")
+		dest = args.output or (file:gsub("%.can$", "")..".lua")
 
 		local inputFile, err = io.open(file, "r")
 		if not inputFile then
-			io.stderr:write("canc: cannot open "..file..": "..err)
+			io.stderr:write("canc: cannot open "..file..": "..err.."\n")
 			os.exit(1)
 		end
 		input = inputFile:read("*a")
@@ -67,8 +84,10 @@ for _, file in ipairs(args) do
 	end
 
 	-- Compile and output
-	if args.dest then
-		dest = args.dest .. "/" .. dest
+	local options = util.cli.makeCandranOptions(args)
+
+	if args.destination then
+		dest = args.destination .. "/" .. dest
 	end
 
 	if not args.print then
@@ -77,13 +96,28 @@ for _, file in ipairs(args) do
 
 	local out = input
 	if args.preprocess then
-		out = candran.preprocess(out, args)
+		local r, err = candran.preprocess(out, options)
+		if not r then
+			io.stderr:write("canc: "..err.."\n")
+			os.exit(1)
+		end
+		out = r
 	end
 	if args.compile then
-		out = candran.compile(out, args)
+		local r, err = candran.compile(out, options)
+		if not r then
+			io.stderr:write("canc: "..err.."\n")
+			os.exit(1)
+		end
+		out = r
 	end
 	if args.compile == nil and args.preprocess == nil then
-		out = candran.make(input, args)
+		local r, err = candran.make(input, options)
+		if not r then
+			io.stderr:write("canc: "..err.."\n")
+			os.exit(1)
+		end
+		out = r
 	end
 
 	if args.print then

bin/cancheck

@@ -0,0 +1,215 @@
+#!/usr/bin/env lua
+
+--[[
+Based on luacheck: https://github.com/luarocks/luacheck
+
+The MIT License (MIT)
+
+Copyright (c) 2014 - 2018 Peter Melnichenko
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+]]
+
+-- Monkey patch Luacheck (tested against version 0.23.0) to support Candran files
+local candran = require("candran")
+local util = require("candran.util")
+
+-- set a function upvalues (if several names are given, will go up the upvalue chain)
+local function setupvalue(fn, val, name, ...)
+	for i=1, debug.getinfo(fn, "u").nups do
+		local n, v = debug.getupvalue(fn, i)
+		if n == name then
+			if not ... then
+				debug.setupvalue(fn, i, val)
+			else
+				setupvalue(v, val, ...)
+			end
+		end
+	end
+end
+
+-- escape a string to be used as a pattern
+local function escape(str)
+	return str:gsub("[^%w]", "%%%0")
+end
+
+-- returns a pattern that find start and stop position of a token
+local function pattern(token)
+	return "()"..escape(token).."()"
+end
+
+-- token aliases
+local tokenAlias = {
+	["self"] = { "@", ":" }
+}
+
+-- Patch checker
+local oldCheck = require("luacheck.check")
+local function check(can)
+	local lua, err = candran.make(can, {chunkname="_luacheck_source"})
+	-- Warnings
+	if lua then
+		local r = oldCheck(lua)
+		-- Calculate Candran file position.
+		if #r.warnings > 0 then
+			local lua_lines = {}
+			for l in (lua.."\n"):gmatch("([^\n]*)\n") do
+				table.insert(lua_lines, l)
+			end
+			local can_lines = {}
+			for l in (can.."\n"):gmatch("([^\n]*)\n") do
+				table.insert(can_lines, l)
+			end
+			for i=#r.warnings, 1, -1 do
+				local warning  = r.warnings[i]
+
+				-- calculating candran line
+				local lua_line = lua_lines[warning.line]
+
+				local source, line = lua_line:match(".*%-%- (.-)%:(%d+)$")
+				if source ~= "_luacheck_source" then -- line is from another file, discard
+					table.remove(r.warnings, i)
+
+				elseif source then
+					warning.can_line = tonumber(line)
+
+					-- do the same for prev_line
+					if warning.prev_line then
+						local s, l = lua_lines[warning.prev_line]:match(".*%-%- (.-)%:(%d+)$")
+						if s ~= "_luacheck_source" then
+							warning.prev_line = s..":"..l -- luacheck seems to do no validation on this, so we can redefine it to anything
+						elseif l then
+							warning.prev_line = l
+						end
+					end
+
+					-- Errors codes highlighting a identifier (alphanumeric+underscore not starting with a number):
+					-- 1xx: global variables
+					-- 2xx: unused variables
+					-- 3xx: unused values
+					-- 4xx: shadowing delarations
+					local isIdentifier = tonumber(warning.code) >= 100 and tonumber(warning.code) < 500
+
+					-- calculating candran column
+					local can_line = can_lines[warning.can_line]
+					local lua_token = lua_line:sub(warning.column, warning.end_column)
+					local token_pattern = pattern(lua_token) -- token finding pattern
+					-- the warning happens on the n-th instance of lua_token on this line
+					local lua_n = 1
+					for start in lua_line:gmatch(token_pattern) do
+						if start >= warning.column then
+							break
+						end
+						lua_n = lua_n + 1
+					end
+					-- Find associated candran token. If lua_n > can_nmax, the last found lua_token is used.
+					-- This approximation should work in like, 90% of cases.
+					local can_n = 1
+					local pos = 1
+					while can_n <= lua_n do
+						-- find first token or alias of this token
+						local start, stop = can_line:match(token_pattern, pos)
+						if tokenAlias[lua_token] then
+							for _, token in ipairs(tokenAlias[lua_token]) do
+								local nstart, nstop = can_line:match(pattern(token), pos)
+								if nstart and (not start or nstart < start) then
+									start, stop = nstart, nstop
+								end
+							end
+						-- for non aliases token that are identifier, check if match is a full identifier
+						-- (avoid things like `let e` matching the e in let)
+						elseif start and isIdentifier then
+							local prev = can_line:sub(start-1, start-1)
+							local next = can_line:sub(stop, stop)
+							if prev:match("[%w_]") or next:match("[%w_]") then
+								can_n = can_n - 1 -- skip this match
+							end
+						end
+						-- found
+						if start then
+							pos = stop
+							warning.can_column, warning.can_end_column = start, stop-1
+							can_n = can_n + 1
+						else
+							break
+						end
+					end
+
+					-- AFAIK, prev_column and prev_end_column are not displayed in any warning so we don't need to recalculate them for Candran.
+				end
+			end
+		end
+		return r
+	-- Syntax error
+	else
+		local line, column, msg = err:match(":(%d+):(%d+):%s*(.*)$")
+		local syntax_error = {
+			code = "011",
+			line = line,
+			column = column,
+			end_column = column,
+			msg = msg
+		}
+		return {
+			warnings = {syntax_error},
+			inline_options = {},
+			line_lengths = {},
+			line_endings = {}
+		}
+	end
+end
+package.loaded["luacheck.check"] = check
+
+local runner = require("luacheck.runner")
+local oldRunner = runner.new
+function runner.new(opts)
+	-- Disable max line length checking (it is compiled code...)
+	opts.max_line_length = false
+	opts.ignore = { "4[23]1/__CAN_.*" }
+	return oldRunner(opts)
+end
+
+-- Patch formatter
+local format = require("luacheck.format")
+local function format_location(file, location, opts)
+   local res = ("%s:%d:%d"):format(file, location.can_line or location.line, location.can_column or location.column)
+   if opts.ranges then
+      res = ("%s-%d"):format(res, location.can_end_column or location.end_column)
+   end
+   return res
+end
+setupvalue(format.builtin_formatters.plain, format_location, "format_event", "format_location")
+
+-- Fix some Luacheck messages and run
+local path = util.search("luacheck.main", {"lua"})
+if path then
+	local f = io.open(path, "r")
+	local code = f:read("*a")
+	f:close()
+
+	code = code:gsub(escape(" bug (please report at https://github.com/mpeterv/luacheck/issues)"), ", patched for Candran "..candran.VERSION.." bug. Please DO NOT report this bug to Luacheck") -- error text
+	           :gsub(escape("\"luacheck\","), "\"cancheck\",") -- command name
+	           :gsub("a linter and a static analyzer for Lua%.", "a linter and a static analyzer for Lua, patched for Candran "..candran.VERSION..".") -- help text
+
+	-- run
+	return load(code)()
+else
+	io.stderr:write("can't find luacheck.main\n")
+	os.exit(1)
+end

candran.can

@@ -1,35 +1,69 @@
-#import("lib.util")
-#import("lib.cmdline")
-
-#import("compiler.lua53")
-#import("compiler.luajit")
-
-#import("lib.lua-parser.scope")
-#import("lib.lua-parser.validator")
-#import("lib.lua-parser.pp")
-#import("lib.lua-parser.parser")
-
 local candran = {
-	VERSION = "0.8.0"
+	VERSION = "1.0.0"
 }
+package.loaded["candran"] = candran
+
+#import("candran.util")
+#import("candran.serpent")
+
+#import("compiler.lua54")
+#import("compiler.lua53")
+#import("compiler.lua52")
+#import("compiler.luajit")
+#import("compiler.lua51")
+
+#import("candran.can-parser.scope")
+#import("candran.can-parser.validator")
+#import("candran.can-parser.pp")
+#import("candran.can-parser.parser")
+
+local unpack = unpack or table.unpack
 
 --- Default options.
 candran.default = {
-	target = _VERSION == "Lua 5.1" and "luajit" or "lua53",
+	target = "lua54",
 	indentation = "",
 	newline = "\n",
 	variablePrefix = "__CAN_",
 	mapLines = true,
 	chunkname = "nil",
-	rewriteErrors = true
+	rewriteErrors = true,
+	builtInMacros = true,
+	preprocessorEnv = {},
+	import = {}
 }
 
+-- Autodetect version
+if _VERSION == "Lua 5.1" then
+	if package.loaded.jit then
+		candran.default.target = "luajit"
+	else
+		candran.default.target = "lua51"
+	end
+elseif _VERSION == "Lua 5.2" then
+	candran.default.target = "lua52"
+elseif _VERSION == "Lua 5.3" then
+	candran.default.target = "lua53"
+end
+
 --- Run the preprocessor
 -- @tparam input string input code
 -- @tparam options table arguments for the preprocessor. They will be inserted into the preprocessor environement.
--- @treturn output string output code
-function candran.preprocess(input, options={})
+-- @treturn[1] output string output code
+-- @treturn[1] macros registered macros
+-- @treturn[2] nil nil if error
+-- @treturn[2] error string error message
+function candran.preprocess(input, options={}, _env)
 	options = util.merge(candran.default, options)
+	local macros = {
+		functions = {},
+		variables = {}
+	}
+
+	-- add auto imports
+	for _, mod in ipairs(options.import) do
+		input =.. "#import(%q, {loadLocal=false})\n":format(mod)
+	end
 
 	-- generate preprocessor code
 	local preprocessor = ""
@@ -64,7 +98,8 @@ function candran.preprocess(input, options={})
 	preprocessor ..= "return output"
 
 	-- make preprocessor environement
-	local env = util.merge(_G, options)
+	local exportenv = {}
+	local env = util.merge(_G, options.preprocessorEnv)
 	--- Candran library table
 	env.candran = candran
 	--- Current preprocessor output
@@ -76,14 +111,17 @@ function candran.preprocess(input, options={})
 	-- @tparam modpath string module path
 	-- @tparam margs table preprocessor options to use when preprocessessing the module
 	env.import = function(modpath, margs={})
-		local filepath = assert(util.search(modpath), "No module named \""..modpath.."\"")
+		local filepath = assert(util.search(modpath, {"can", "lua"}), "No module named \""..modpath.."\"")
 
 		-- open module file
 		local f = io.open(filepath)
-		if not f then error("Can't open the module file to import") end
+		if not f then error("can't open the module file to import") end
 
 		margs = util.merge(options, { chunkname = filepath, loadLocal = true, loadPackage = true }, margs)
-		local modcontent = candran.preprocess(f:read("*a"), margs)
+		margs.import = {} -- no need for recursive import
+		local modcontent, modmacros, modenv = assert(candran.preprocess(f:read("*a"), margs))
+		macros = util.recmerge(macros, modmacros)
+		for k, v in pairs(modenv) do env[k] = v end
 		f:close()
 
 		-- get module name (ex: module name of path.to.module is module)
@@ -103,7 +141,7 @@ function candran.preprocess(input, options={})
 	-- @tparam file string filepath
 	env.include = function(file)
 		local f = io.open(file)
-		if not f then error("Can't open the file "..file.." to include") end
+		if not f then error("can't open the file "..file.." to include") end
 		env.write(f:read("*a"))
 		f:close()
 	end
@@ -119,40 +157,103 @@ function candran.preprocess(input, options={})
 			env.write(env[name])
 		end
 	end
+	env.define = function(identifier, replacement)
+		-- parse identifier
+		local iast, ierr = parser.parsemacroidentifier(identifier, options.chunkname)
+		if not iast then
+			return error("in macro identifier: %s":format(tostring(ierr)))
+		end
+		-- parse replacement value
+		if type(replacement) == "string" then
+			local rast, rerr = parser.parse(replacement, options.chunkname)
+			if not rast then
+				return error("in macro replacement: %s":format(tostring(rerr)))
+			end
+			-- when giving a single value as a replacement, bypass the implicit push
+			if #rast == 1 and rast[1].tag == "Push" and rast[1].implicit then
+				rast = rast[1][1]
+			end
+			replacement = rast
+		elseif type(replacement) ~= "function" then
+			error("bad argument #2 to 'define' (string or function expected)")
+		end
+		-- add macros
+		if iast.tag == "MacroFunction" then
+			macros.functions[iast[1][1]] = { args = iast[2], replacement = replacement }
+		elseif iast.tag == "Id" then
+			macros.variables[iast[1]] = replacement
+		else
+			error("invalid macro type %s":format(tostring(iast.tag)))
+		end
+	end
+	env.set = function(identifier, value)
+		exportenv[identifier] = value
+		env[identifier] = value
+	end
+
+	-- default macros
+	if options.builtInMacros then
+		env.define("__STR__(x)", function(x) return ("%q"):format(x) end)
+		local s = require("candran.serpent")
+		env.define("__CONSTEXPR__(expr)", function(expr)
+			return s.block(assert(candran.load(expr))(), {fatal = true})
+		end)
+	end
 
 	-- compile & load preprocessor
-	local preprocess, err = util.load(candran.compile(preprocessor, args), "candran preprocessor", env)
-	if not preprocess then error("Error while creating Candran preprocessor: " .. err) end
+	local preprocess, err = candran.compile(preprocessor, options)
+	if not preprocess then
+		return nil, "in preprocessor: "..err
+	end
+
+	preprocess, err = util.load(preprocessor, "candran preprocessor", env)
+	if not preprocess then
+		return nil, "in preprocessor: "..err
+	end
 
 	-- execute preprocessor
 	local success, output = pcall(preprocess)
-	if not success then error("Error while preprocessing file: " .. output) end
+	if not success then
+		return nil, "in preprocessor: "..output
+	end
 
-	return output
+	return output, macros, exportenv
 end
 
 --- Run the compiler
 -- @tparam input string input code
 -- @tparam options table options for the compiler
--- @treturn output string output code
-function candran.compile(input, options={})
+-- @tparam macros table defined macros, as returned by the preprocessor
+-- @treturn[1] output string output code
+-- @treturn[2] nil nil if error
+-- @treturn[2] error string error message
+function candran.compile(input, options={}, macros)
 	options = util.merge(candran.default, options)
 
 	local ast, errmsg = parser.parse(input, options.chunkname)
 
 	if not ast then
-		error("Compiler: error while parsing file: "..errmsg)
+		return nil, errmsg
 	end
 
-	return require("compiler."..options.target)(input, ast, options)
+	return require("compiler."..options.target)(input, ast, options, macros)
 end
 
 --- Preprocess & compile code
 -- @tparam code string input code
 -- @tparam options table arguments for the preprocessor and compiler
--- @treturn output string output code
+-- @treturn[1] output string output code
+-- @treturn[2] nil nil if error
+-- @treturn[2] error string error message
 function candran.make(code, options)
-	return candran.compile(candran.preprocess(code, options), options)
+	local r, err = candran.preprocess(code, options)
+	if r then
+		r, err = candran.compile(r, options, err)
+		if r then
+			return r
+		end
+	end
+	return r, err
 end
 
 local errorRewritingActive = false
@@ -161,7 +262,9 @@ local codeCache = {}
 -- Will rewrite errors by default.
 function candran.loadfile(filepath, env, options)
 	local f, err = io.open(filepath)
-	if not f then error("can't open the file: "..err) end
+	if not f then
+		return nil, "cannot open %s":format(tostring(err))
+	end
 	local content = f:read("*a")
 	f:close()
 
@@ -173,24 +276,29 @@ end
 function candran.load(chunk, chunkname, env, options={})
 	options = util.merge({ chunkname = tostring(chunkname or chunk) }, options)
 
-	codeCache[options.chunkname] = candran.make(chunk, options)
-	local f, err = util.load(codeCache[options.chunkname], options.chunkname, env)
+	local code, err = candran.make(chunk, options)
+	if not code then
+		return code, err
+	end
+
+	codeCache[options.chunkname] = code
+	local f
+	f, err = util.load(code, "=%s(%s)":format(options.chunkname, "compiled candran"), env)
 
 	-- Um. Candran isn't supposed to generate invalid Lua code, so this is a major issue.
 	-- This is not going to raise an error because this is supposed to behave similarly to Lua's load function.
 	-- But the error message will likely be useless unless you know how Candran works.
 	if f == nil then
-		return f, "Candran unexpectedly generated invalid code: "..err
+		return f, "candran unexpectedly generated invalid code: "..err
 	end
 
 	if options.rewriteErrors == false then
 		return f
 	else
 		return function(...)
-			local params = {...}
 			if not errorRewritingActive then
 				errorRewritingActive = true
-				local t = { xpcall(() return f(unpack(params)) end, candran.messageHandler) }
+				local t = { xpcall(f, candran.messageHandler, ...) }
 				errorRewritingActive = false
 				if t[1] == false then
 					error(t[2], 0)
@@ -217,20 +325,23 @@ end
 
 --- Candran error message handler.
 -- Use it in xpcall to rewrite stacktraces to display Candran source file lines instead of compiled Lua lines.
-function candran.messageHandler(message)
-	return debug.traceback(message, 2):gsub("(\n?%s*)([^\n]-)%:(%d+)%:", function(indentation, source, line)
+function candran.messageHandler(message, noTraceback)
+	message = tostring(message)
+	if not noTraceback and not message:match("\nstack traceback:\n") then
+		message = debug.traceback(message, 2)
+	end
+	return message:gsub("(\n?%s*)([^\n]-)%:(%d+)%:", function(indentation, source, line)
 		line = tonumber(line)
 
 		local originalFile
-		local strName = source:match("%[string \"(.-)\"%]")
+		local strName = source:match("^(.-)%(compiled candran%)$")
 		if strName then
 			if codeCache[strName] then
 				originalFile = codeCache[strName]
 				source = strName
 			end
 		else
-			local fi = io.open(source, "r")
-			if fi then
+			if fi = io.open(source, "r") then
 				originalFile = fi:read("*a")
 				fi:close()
 			end
@@ -238,7 +349,7 @@ function candran.messageHandler(message)
 
 		if originalFile then
 			local i = 0
-			for l in originalFile:gmatch("([^\n]*)") do
+			for l in (originalFile.."\n"):gmatch("([^\n]*)\n") do
 				i = i +1
 				if i == line then
 					local extSource, lineMap = l:match(".*%-%- (.-)%:(%d+)$")
@@ -260,18 +371,37 @@ end
 function candran.searcher(modpath)
 	local filepath = util.search(modpath, {"can"})
 	if not filepath then
-		return "\n\tno candran file in package.path"
+		if _VERSION == "Lua 5.4" then
+			return "no candran file in package.path"
+		else
+			return "\n\tno candran file in package.path"
+		end
 	end
-	return candran.loadfile(filepath)
+	return (modpath) -- 2nd argument is not passed in Lua 5.1, so a closure is required
+		local r, s = candran.loadfile(filepath)
+		if r then
+			return r(modpath, filepath)
+		else
+			error("error loading candran module '%s' from file '%s':\n\t%s":format(modpath, filepath, tostring(s)), 0)
+		end
+	end, filepath
 end
 
 --- Register the Candran package searcher.
 function candran.setup()
-	if _VERSION == "Lua 5.1" then
-		table.insert(package.loaders, 2, candran.searcher)
+	local searchers = if _VERSION == "Lua 5.1" then
+		package.loaders
 	else
-		table.insert(package.searchers, 2, candran.searcher)
+		package.searchers
 	end
+	-- check if already setup
+	for _, s in ipairs(searchers) do
+		if s == candran.searcher then
+			return candran
+		end
+	end
+	-- setup
+	table.insert(searchers, 1, candran.searcher)
 	return candran
 end
 

candran/can-parser/parser.lua

@@ -9,12 +9,12 @@ block: { stat* }
 stat:
     `Do{ stat* }
   | `Set{ {lhs+} (opid? = opid?)? {expr+} }   -- lhs1, lhs2... op=op e1, e2...
-  | `While{ expr block }                      -- while e do b end
+  | `While{ lexpr block }                      -- while e do b end
   | `Repeat{ block expr }                     -- repeat b until e
-  | `If{ (expr block)+ block? }               -- if e1 then b1 [elseif e2 then b2] ... [else bn] end
+  | `If{ (lexpr block)+ block? }              -- if e1 then b1 [elseif e2 then b2] ... [else bn] end
   | `Fornum{ ident expr expr expr? block }    -- for ident = e, e[, e] do b end
   | `Forin{ {ident+} {expr+} block }          -- for i1, i2... in e1, e2... do b end
-  | `Local{ {ident+} {expr+}? }               -- local i1, i2... = e1, e2...
+  | `Local{ {attributeident+} {expr+}? }      -- local i1, i2... = e1, e2...
   | `Let{ {ident+} {expr+}? }                 -- let i1, i2... = e1, e2...
   | `Localrec{ {ident} {expr} }               -- only used for 'local function'
   | `Goto{ <string> }                         -- goto str
@@ -36,10 +36,17 @@ expr:
   | `Op{ opid expr expr? }
   | `Paren{ expr }       -- significant to cut multiple values returns
   | `TableCompr{ block }
+  | `MethodStub{ expr expr }
+  | `SafeMethodStub{ expr expr }
+  | `SafeIndex{ expr expr }
   | statexpr
   | apply
   | lhs
 
+lexpr:
+    `LetExpr{ {ident+} {expr+}? }
+  | every node from expr
+
 statexpr:
     `DoExpr{ stat* }
   | `WhileExpr{ expr block }                    -- while e do b end
@@ -50,9 +57,9 @@ statexpr:
 
 apply:
     `Call{ expr expr* }
-  | `Invoke{ expr `String{ <string> } expr* }
+  | `SafeCall{ expr expr* }
 
-lhs: `Id{ <string> } | `Index{ expr expr }
+lhs: `Id{ <string> } | AttributeId{ <string> <string>? } | `Index{ expr expr } | ˇDestructuringId{ Id | Pair+ }
 
 opid:  -- includes additional operators from Lua 5.3 and all relational operators
     'add'  | 'sub' | 'mul'  | 'div'
@@ -106,7 +113,9 @@ local labels = {
   { "ErrDoFor", "expected 'do' after the range of the for loop" },
 
   { "ErrDefLocal", "expected a function definition or assignment after local" },
-  { "ErrDefLet", "expected a function definition or assignment after let" },
+  { "ErrDefLet", "expected an assignment after let" },
+  { "ErrDefClose", "expected an assignment after close" },
+  { "ErrDefConst", "expected an assignment after const" },
   { "ErrNameLFunc", "expected a function name after 'function'" },
   { "ErrEListLAssign", "expected one or more expressions after '='" },
   { "ErrEListAssign", "expected one or more expressions after '='" },
@@ -157,6 +166,10 @@ local labels = {
   { "ErrExprFKey", "expected an expression after '[' for the table key" },
   { "ErrCBracketFKey", "expected ']' to close the table key" },
 
+  { "ErrCBraceDestructuring", "expected '}' to close the destructuring variable list" },
+  { "ErrDestructuringEqField", "expected '=' after the table key in destructuring variable list" },
+  { "ErrDestructuringExprField", "expected an identifier after '=' in destructuring variable list" },
+
   { "ErrCBracketTableCompr", "expected ']' to close the table comprehension" },
 
   { "ErrDigitHex", "expected one or more hexadecimal digits after '0x'" },
@@ -170,6 +183,9 @@ local labels = {
   { "ErrCBraceUEsc", "expected '}' after the code point" },
   { "ErrEscSeq", "invalid escape sequence" },
   { "ErrCloseLStr", "unclosed long string" },
+
+  { "ErrUnknownAttribute", "unknown variable attribute" },
+  { "ErrCBracketAttribute", "expected '>' to close the variable attribute" },
 }
 
 local function throw(label)
@@ -255,25 +271,32 @@ local function insertIndex (t, index)
   return { tag = "Index", pos = t.pos, [1] = t, [2] = index }
 end
 
-local function markMethod(t, method)
+local function markMethod (t, method)
   if method then
     return { tag = "Index", pos = t.pos, is_method = true, [1] = t, [2] = method }
   end
   return t
 end
 
-local function makeIndexOrCall (t1, t2)
-  if t2.tag == "Call" or t2.tag == "Invoke" then
+local function makeSuffixedExpr (t1, t2)
+  if t2.tag == "Call" or t2.tag == "SafeCall" then
     local t = { tag = t2.tag, pos = t1.pos, [1] = t1 }
     for k, v in ipairs(t2) do
       table.insert(t, v)
     end
     return t
+  elseif t2.tag == "MethodStub" or t2.tag == "SafeMethodStub" then
+    return { tag = t2.tag, pos = t1.pos, [1] = t1, [2] = t2[1] }
+  elseif t2.tag == "SafeDotIndex" or t2.tag == "SafeArrayIndex" then
+    return { tag = "SafeIndex", pos = t1.pos, [1] = t1, [2] = t2[1] }
+  elseif t2.tag == "DotIndex" or t2.tag == "ArrayIndex" then
+    return { tag = "Index", pos = t1.pos, [1] = t1, [2] = t2[1] }
+  else
+    error("unexpected tag in suffixed expression")
   end
-  return { tag = "Index", pos = t1.pos, [1] = t1, [2] = t2[1] }
 end
 
-local function fixShortFunc(t)
+local function fixShortFunc (t)
     if t[1] == ":" then -- self method
         table.insert(t[2], 1, { tag = "Id", "self" })
         table.remove(t, 1)
@@ -283,12 +306,17 @@ local function fixShortFunc(t)
     return t
 end
 
-local function statToExpr(t) -- tag a StatExpr
+local function markImplicit (t)
+    t.implicit = true
+    return t
+end
+
+local function statToExpr (t) -- tag a StatExpr
     t.tag = t.tag .. "Expr"
     return t
 end
 
-local function fixStructure(t) -- fix the AST structure if needed
+local function fixStructure (t) -- fix the AST structure if needed
     local i = 1
     while i <= #t do
         if type(t[i]) == "table" then
@@ -309,7 +337,7 @@ local function fixStructure(t) -- fix the AST structure if needed
     return t
 end
 
-local function searchEndRec(block, isRecCall) -- recursively search potential "end" keyword wrongly consumed by a short anonymous function on stat end (yeah, too late to change the syntax to something easier to parse)
+local function searchEndRec (block, isRecCall) -- recursively search potential "end" keyword wrongly consumed by a short anonymous function on stat end (yeah, too late to change the syntax to something easier to parse)
     for i, stat in ipairs(block) do
         -- Non recursive statements
         if stat.tag == "Set" or stat.tag == "Push" or stat.tag == "Return" or stat.tag == "Local" or stat.tag == "Let" or stat.tag == "Localrec" then
@@ -404,7 +432,7 @@ local function searchEndRec(block, isRecCall) -- recursively search potential "e
     return nil
 end
 
-local function searchEnd(s, p, t) -- match time capture which try to restructure the AST to free an "end" for us
+local function searchEnd (s, p, t) -- match time capture which try to restructure the AST to free an "end" for us
     local r = searchEndRec(fixStructure(t))
     if not r then
         return false
@@ -412,7 +440,7 @@ local function searchEnd(s, p, t) -- match time capture which try to restructure
     return true, r
 end
 
-local function expectBlockOrSingleStatWithStartEnd(start, startLabel, stopLabel, canFollow) -- will try a SingleStat if start doesn't match
+local function expectBlockOrSingleStatWithStartEnd (start, startLabel, stopLabel, canFollow) -- will try a SingleStat if start doesn't match
     if canFollow then
         return (-start * V"SingleStatBlock" * canFollow^-1)
              + (expect(start, startLabel) * ((V"Block" * (canFollow + kw("end")))
@@ -424,16 +452,62 @@ local function expectBlockOrSingleStatWithStartEnd(start, startLabel, stopLabel,
     end
 end
 
-local function expectBlockWithEnd(label) -- can't work *optionnaly* with SingleStat unfortunatly
+local function expectBlockWithEnd (label) -- can't work *optionnaly* with SingleStat unfortunatly
     return (V"Block" * kw("end"))
          + (Cmt(V"Block", searchEnd) + throw(label))
 end
 
-local function maybeBlockWithEnd() -- same as above but don't error if it doesn't match
+local function maybeBlockWithEnd () -- same as above but don't error if it doesn't match
     return (V"BlockNoErr" * kw("end"))
          + Cmt(V"BlockNoErr", searchEnd)
 end
 
+local function maybe (patt) -- fail pattern instead of propagating errors
+  return #patt/0 * patt
+end
+
+local function setAttribute(attribute)
+  return function(assign)
+    assign[1].tag = "AttributeNameList"
+    for _, id in ipairs(assign[1]) do
+      if id.tag == "Id" then
+        id.tag = "AttributeId"
+        id[2] = attribute
+      elseif id.tag == "DestructuringId" then
+        for _, did in ipairs(id) do
+          did.tag = "AttributeId"
+          did[2] = attribute
+        end
+      end
+    end
+    return assign
+  end
+end
+
+local stacks = {
+  lexpr = {}
+}
+local function push (f)
+  return Cmt(P"", function()
+    table.insert(stacks[f], true)
+    return true
+  end)
+end
+local function pop (f)
+  return Cmt(P"", function()
+    table.remove(stacks[f])
+    return true
+  end)
+end
+local function when (f)
+  return Cmt(P"", function()
+    return #stacks[f] > 0
+  end)
+end
+local function set (f, patt) -- patt *must* succeed (or throw an error) to preserve stack integrity
+  return push(f) * patt * pop(f)
+end
+
 -- grammar
 local G = { V"Lua",
   Lua      = (V"Shebang"^-1 * V"Skip" * V"Block" * expect(P(-1), "Extra")) / fixStructure;
@@ -442,8 +516,9 @@ local G = { V"Lua",
   Block       = tagC("Block", (V"Stat" + -V"BlockEnd" * throw("InvalidStat"))^0 * ((V"RetStat" + V"ImplicitPushStat") * sym(";")^-1)^-1);
   Stat        = V"IfStat" + V"DoStat" + V"WhileStat" + V"RepeatStat" + V"ForStat"
               + V"LocalStat" + V"FuncStat" + V"BreakStat" + V"LabelStat" + V"GoToStat"
+              + V"LetStat" + V"ConstStat" + V"CloseStat"
               + V"FuncCall" + V"Assignment"
-              + V"LetStat" + V"ContinueStat" + V"PushStat"
+              + V"ContinueStat" + V"PushStat"
               + sym(";");
   BlockEnd    = P"return" + "end" + "elseif" + "else" + "until" + "]" + -1 + V"ImplicitPushStat" + V"Assignment";
 
@@ -451,12 +526,12 @@ local G = { V"Lua",
   BlockNoErr      = tagC("Block", V"Stat"^0 * ((V"RetStat" + V"ImplicitPushStat") * sym(";")^-1)^-1); -- used to check if something a valid block without throwing an error
 
   IfStat      = tagC("If", V"IfPart");
-  IfPart      = kw("if") * expect(V"Expr", "ExprIf") * expectBlockOrSingleStatWithStartEnd(kw("then"), "ThenIf", "EndIf", V"ElseIfPart" + V"ElsePart");
-  ElseIfPart  = kw("elseif") * expect(V"Expr", "ExprEIf") * expectBlockOrSingleStatWithStartEnd(kw("then"), "ThenEIf", "EndIf", V"ElseIfPart" + V"ElsePart");
+  IfPart      = kw("if") * set("lexpr", expect(V"Expr", "ExprIf")) * expectBlockOrSingleStatWithStartEnd(kw("then"), "ThenIf", "EndIf", V"ElseIfPart" + V"ElsePart");
+  ElseIfPart  = kw("elseif") * set("lexpr", expect(V"Expr", "ExprEIf")) * expectBlockOrSingleStatWithStartEnd(kw("then"), "ThenEIf", "EndIf", V"ElseIfPart" + V"ElsePart");
   ElsePart    = kw("else") * expectBlockWithEnd("EndIf");
 
   DoStat      = kw("do") * expectBlockWithEnd("EndDo") / tagDo;
-  WhileStat   = tagC("While", kw("while") * expect(V"Expr", "ExprWhile") * V"WhileBody");
+  WhileStat   = tagC("While", kw("while") * set("lexpr", expect(V"Expr", "ExprWhile")) * V"WhileBody");
   WhileBody   = expectBlockOrSingleStatWithStartEnd(kw("do"), "DoWhile", "EndWhile");
   RepeatStat  = tagC("Repeat", kw("repeat") * V"Block" * expect(kw("until"), "UntilRep") * expect(V"Expr", "ExprRep"));
 
@@ -464,17 +539,24 @@ local G = { V"Lua",
   ForNum    = tagC("Fornum", V"Id" * sym("=") * V"NumRange" * V"ForBody");
   NumRange  = expect(V"Expr", "ExprFor1") * expect(sym(","), "CommaFor") *expect(V"Expr", "ExprFor2")
             * (sym(",") * expect(V"Expr", "ExprFor3"))^-1;
-  ForIn     = tagC("Forin", V"NameList" * expect(kw("in"), "InFor") * expect(V"ExprList", "EListFor") * V"ForBody");
+  ForIn     = tagC("Forin", V"DestructuringNameList" * expect(kw("in"), "InFor") * expect(V"ExprList", "EListFor") * V"ForBody");
   ForBody   = expectBlockOrSingleStatWithStartEnd(kw("do"), "DoFor", "EndFor");
 
   LocalStat    = kw("local") * expect(V"LocalFunc" + V"LocalAssign", "DefLocal");
   LocalFunc    = tagC("Localrec", kw("function") * expect(V"Id", "NameLFunc") * V"FuncBody") / fixFuncStat;
-  LocalAssign  = tagC("Local", V"NameList" * (sym("=") * expect(V"ExprList", "EListLAssign") + Ct(Cc())));
+  LocalAssign  = tagC("Local", V"AttributeNameList" * (sym("=") * expect(V"ExprList", "EListLAssign") + Ct(Cc())))
+               + tagC("Local", V"DestructuringNameList" * sym("=") * expect(V"ExprList", "EListLAssign"));
 
   LetStat      = kw("let") * expect(V"LetAssign", "DefLet");
-  LetAssign    = tagC("Let", V"NameList" * (sym("=") * expect(V"ExprList", "EListLAssign") + Ct(Cc())));
+  LetAssign    = tagC("Let", V"NameList" * (sym("=") * expect(V"ExprList", "EListLAssign") + Ct(Cc())))
+               + tagC("Let", V"DestructuringNameList" * sym("=") * expect(V"ExprList", "EListLAssign"));
 
-  Assignment   = tagC("Set", V"VarList" * V"BinOp"^-1 * (P"=" / "=") * V"BinOp"^-1 * V"Skip" * expect(V"ExprList", "EListAssign"));
+  ConstStat       = kw("const") * expect(V"AttributeAssign" / setAttribute("const"), "DefConst");
+  CloseStat       = kw("close") * expect(V"AttributeAssign" / setAttribute("close"), "DefClose");
+  AttributeAssign = tagC("Local", V"NameList" * (sym("=") * expect(V"ExprList", "EListLAssign") + Ct(Cc())))
+                  + tagC("Local", V"DestructuringNameList" * sym("=") * expect(V"ExprList", "EListLAssign"));
+
+  Assignment   = tagC("Set", (V"VarList" + V"DestructuringNameList") * V"BinOp"^-1 * (P"=" / "=") * ((V"BinOp" - P"-") + #(P"-" * V"Space") * V"BinOp")^-1 * V"Skip" * expect(V"ExprList", "EListAssign"));
 
   FuncStat    = tagC("Set", kw("function") * expect(V"FuncName", "FuncName") * V"FuncBody") / fixFuncStat;
   FuncName    = Cf(V"Id" * (sym(".") * expect(V"StrId", "NameFunc1"))^0, insertIndex)
@@ -500,12 +582,19 @@ local G = { V"Lua",
   RetStat         = tagC("Return", kw("return") * commaSep(V"Expr", "RetList")^-1);
 
   PushStat         = tagC("Push", kw("push") * commaSep(V"Expr", "RetList")^-1);
-  ImplicitPushStat = tagC("Push", commaSep(V"Expr", "RetList"));
+  ImplicitPushStat = tagC("Push", commaSep(V"Expr", "RetList")) / markImplicit;
 
-  NameList  = tagC("NameList", commaSep(V"Id"));
+  NameList              = tagC("NameList", commaSep(V"Id"));
+  DestructuringNameList = tagC("NameList", commaSep(V"DestructuringId")),
+  AttributeNameList     = tagC("AttributeNameList", commaSep(V"AttributeId"));
   VarList   = tagC("VarList", commaSep(V"VarExpr"));
   ExprList  = tagC("ExpList", commaSep(V"Expr", "ExprList"));
 
+  DestructuringId          = tagC("DestructuringId", sym("{") * V"DestructuringIdFieldList" * expect(sym("}"), "CBraceDestructuring")) + V"Id",
+  DestructuringIdFieldList = sepBy(V"DestructuringIdField", V"FieldSep") * V"FieldSep"^-1;
+  DestructuringIdField     = tagC("Pair", V"FieldKey" * expect(sym("="), "DestructuringEqField") * expect(V"Id", "DestructuringExprField"))
+                           + V"Id";
+
   Expr        = V"OrExpr";
   OrExpr      = chainOp(V"AndExpr", V"OrOp", "OrExpr");
   AndExpr     = chainOp(V"RelExpr", V"AndOp", "AndExpr");
@@ -520,35 +609,39 @@ local G = { V"Lua",
   UnaryExpr   = V"UnaryOp" * expect(V"UnaryExpr", "UnaryExpr") / unaryOp
               + V"PowExpr";
   PowExpr     = V"SimpleExpr" * (V"PowOp" * expect(V"UnaryExpr", "PowExpr"))^-1 / binaryOp;
-
-  SimpleExpr = tagC("Number", V"Number")
-             + tagC("Nil", kw("nil"))
-             + tagC("Boolean", kw("false") * Cc(false))
-             + tagC("Boolean", kw("true") * Cc(true))
-             + tagC("Dots", sym("..."))
-             + V"FuncDef"
-             + V"ShortFuncDef"
-             + V"SuffixedExpr"
-             + V"StatExpr";
+  SimpleExpr  = tagC("Number", V"Number")
+              + tagC("Nil", kw("nil"))
+              + tagC("Boolean", kw("false") * Cc(false))
+              + tagC("Boolean", kw("true") * Cc(true))
+              + tagC("Dots", sym("..."))
+              + V"FuncDef"
+              + (when("lexpr") * tagC("LetExpr", maybe(V"DestructuringNameList") * sym("=") * -sym("=") * expect(V"ExprList", "EListLAssign")))
+              + V"ShortFuncDef"
+              + V"SuffixedExpr"
+              + V"StatExpr";
 
   StatExpr = (V"IfStat" + V"DoStat" + V"WhileStat" + V"RepeatStat" + V"ForStat") / statToExpr;
 
-  FuncCall  = Cmt(V"SuffixedExpr", function(s, i, exp) return exp.tag == "Call" or exp.tag == "Invoke", exp end);
+  FuncCall  = Cmt(V"SuffixedExpr", function(s, i, exp) return exp.tag == "Call" or exp.tag == "SafeCall", exp end);
   VarExpr   = Cmt(V"SuffixedExpr", function(s, i, exp) return exp.tag == "Id" or exp.tag == "Index", exp end);
 
-  SuffixedExpr      = Cf(V"PrimaryExpr" * (V"Index" + V"Invoke" + V"Call")^0
-                       + V"NoCallPrimaryExpr" * -V"Call" * (V"Index" + V"Invoke" + V"Call")^0
-                       + V"NoCallPrimaryExpr", makeIndexOrCall);
+  SuffixedExpr      = Cf(V"PrimaryExpr" * (V"Index" + V"MethodStub" + V"Call")^0
+                    + V"NoCallPrimaryExpr" * -V"Call" * (V"Index" + V"MethodStub" + V"Call")^0
+                    + V"NoCallPrimaryExpr", makeSuffixedExpr);
   PrimaryExpr       = V"SelfId" * (V"SelfCall" + V"SelfIndex")
                     + V"Id"
                     + tagC("Paren", sym("(") * expect(V"Expr", "ExprParen") * expect(sym(")"), "CParenExpr"));
   NoCallPrimaryExpr = tagC("String", V"String") + V"Table" + V"TableCompr";
   Index             = tagC("DotIndex", sym("." * -P".") * expect(V"StrId", "NameIndex"))
-                    + tagC("ArrayIndex", sym("[" * -P(S"=[")) * expect(V"Expr", "ExprIndex") * expect(sym("]"), "CBracketIndex"));
-  Call              = tagC("Call", V"FuncArgs");
-  Invoke            = tagC("Invoke", Cg(sym(":" * -P":") * expect(V"StrId", "NameMeth") * expect(V"FuncArgs", "MethArgs")));
+                    + tagC("ArrayIndex", sym("[" * -P(S"=[")) * expect(V"Expr", "ExprIndex") * expect(sym("]"), "CBracketIndex"))
+                    + tagC("SafeDotIndex", sym("?." * -P".") * expect(V"StrId", "NameIndex"))
+                    + tagC("SafeArrayIndex", sym("?[" * -P(S"=[")) * expect(V"Expr", "ExprIndex") * expect(sym("]"), "CBracketIndex"));
+  MethodStub        = tagC("MethodStub", sym(":" * -P":") * expect(V"StrId", "NameMeth"))
+                    + tagC("SafeMethodStub", sym("?:" * -P":") * expect(V"StrId", "NameMeth"));
+  Call              = tagC("Call", V"FuncArgs")
+                    + tagC("SafeCall", P"?" * V"FuncArgs");
+  SelfCall          = tagC("MethodStub", V"StrId") * V"Call";
   SelfIndex         = tagC("DotIndex", V"StrId");
-  SelfCall          = tagC("Invoke", Cg(V"StrId" * V"FuncArgs"));
 
   FuncDef   = (kw("function") * V"FuncBody");
   FuncArgs  = sym("(") * commaSep(V"Expr", "ArgList")^-1 * expect(sym(")"), "CParenArgs")
@@ -567,8 +660,12 @@ local G = { V"Lua",
 
   SelfId = tagC("Id", sym"@" / "self");
   Id     = tagC("Id", V"Name") + V"SelfId";
+  AttributeSelfId = tagC("AttributeId", sym"@" / "self" * V"Attribute"^-1);
+  AttributeId     = tagC("AttributeId", V"Name" * V"Attribute"^-1) + V"AttributeSelfId";
   StrId  = tagC("String", V"Name");
 
+  Attribute = sym("<") * expect(kw"const" / "const" + kw"close" / "close", "UnknownAttribute") * expect(sym(">"), "CBracketAttribute");
+
   -- lexer
   Skip     = (V"Space" + V"Comment")^0;
   Space    = space^1;
@@ -665,9 +762,23 @@ local G = { V"Lua",
   BinOp     = V"OrOp" + V"AndOp" + V"BOrOp" + V"BXorOp" + V"BAndOp" + V"ShiftOp" + V"ConcatOp" + V"AddOp" + V"MulOp" + V"PowOp";
 }
 
+-- used to parse macro indentifier in define() preprocessor function
+local macroidentifier = {
+  expect(V"MacroIdentifier", "InvalidStat") * expect(P(-1), "Extra"),
+
+  MacroIdentifier   = tagC("MacroFunction", V"Id" * sym("(") * V"MacroFunctionArgs" * expect(sym(")"), "CParenPList"))
+                    + V"Id";
+
+  MacroFunctionArgs = V"NameList" * (sym(",") * expect(tagC("Dots", sym("...")), "ParList"))^-1 / addDots
+                    + Ct(tagC("Dots", sym("...")))
+                    + Ct(Cc());
+
+}
+for k,v in pairs(G) do if macroidentifier[k] == nil then macroidentifier[k] = v end end -- copy other rules from main syntax
+
 local parser = {}
 
-local validator = require("lib.lua-parser.validator")
+local validator = require("candran.can-parser.validator")
 local validate = validator.validate
 local syntaxerror = validator.syntaxerror
 
@@ -682,4 +793,15 @@ function parser.parse (subject, filename)
   return validate(ast, errorinfo)
 end
 
+function parser.parsemacroidentifier (subject, filename)
+  local errorinfo = { subject = subject, filename = filename }
+  lpeg.setmaxstack(1000)
+  local ast, label, errpos = lpeg.match(macroidentifier, subject, nil, errorinfo)
+  if not ast then
+    local errmsg = labels[label][2]
+    return ast, syntaxerror(errorinfo, errpos, errmsg)
+  end
+  return ast
+end
+
 return parser

candran/can-parser/validator.lua

@@ -1,7 +1,7 @@
 --[[
 This module impements a validator for the AST
 ]]
-local scope = require "lib.lua-parser.scope"
+local scope = require "candran.can-parser.scope"
 
 local lineno = scope.lineno
 local new_scope, end_scope = scope.new_scope, scope.end_scope
@@ -158,16 +158,6 @@ local function traverse_call (env, call)
   return true
 end
 
-local function traverse_invoke (env, invoke)
-  local status, msg = traverse_exp(env, invoke[1])
-  if not status then return status, msg end
-  for i=3, #invoke do
-    status, msg = traverse_exp(env, invoke[i])
-    if not status then return status, msg end
-  end
-  return true
-end
-
 local function traverse_assignment (env, stm)
   local status, msg = traverse_varlist(env, stm[1])
   if not status then return status, msg end
@@ -238,6 +228,18 @@ local function traverse_goto (env, stm)
   return true
 end
 
+local function traverse_let (env, stm)
+  local status, msg = traverse_explist(env, stm[2])
+  if not status then return status, msg end
+  return true
+end
+
+local function traverse_letrec (env, stm)
+  local status, msg = traverse_exp(env, stm[2][1])
+  if not status then return status, msg end
+  return true
+end
+
 local function traverse_if (env, stm)
   local len = #stm
   if len % 2 == 0 then
@@ -266,18 +268,6 @@ local function traverse_label (env, stm)
   return true
 end
 
-local function traverse_let (env, stm)
-  local status, msg = traverse_explist(env, stm[2])
-  if not status then return status, msg end
-  return true
-end
-
-local function traverse_letrec (env, stm)
-  local status, msg = traverse_exp(env, stm[2][1])
-  if not status then return status, msg end
-  return true
-end
-
 local function traverse_repeat (env, stm)
   begin_loop(env)
   local status, msg = traverse_block(env, stm[1])
@@ -314,6 +304,8 @@ function traverse_var (env, var)
     status, msg = traverse_exp(env, var[2])
     if not status then return status, msg end
     return true
+  elseif tag == "DestructuringId" then
+    return traverse_table(env, var)
   else
     error("expecting a variable, but got a " .. tag)
   end
@@ -327,6 +319,22 @@ function traverse_varlist (env, varlist)
   return true
 end
 
+local function traverse_methodstub (env, var)
+  local status, msg = traverse_exp(env, var[1])
+  if not status then return status, msg end
+  status, msg = traverse_exp(env, var[2])
+  if not status then return status, msg end
+  return true
+end
+
+local function traverse_safeindex (env, var)
+  local status, msg = traverse_exp(env, var[1])
+  if not status then return status, msg end
+  status, msg = traverse_exp(env, var[2])
+  if not status then return status, msg end
+  return true
+end
+
 function traverse_exp (env, exp)
   local tag = exp.tag
   if tag == "Nil" or
@@ -344,15 +352,17 @@ function traverse_exp (env, exp)
     return traverse_op(env, exp)
   elseif tag == "Paren" then -- `Paren{ expr }
     return traverse_paren(env, exp)
-  elseif tag == "Call" then -- `Call{ expr expr* }
+  elseif tag == "Call" or tag == "SafeCall" then -- `(Safe)Call{ expr expr* }
     return traverse_call(env, exp)
-  elseif tag == "Invoke" then -- `Invoke{ expr `String{ <string> } expr* }
-    return traverse_invoke(env, exp)
   elseif tag == "Id" or -- `Id{ <string> }
          tag == "Index" then -- `Index{ expr expr }
     return traverse_var(env, exp)
+  elseif tag == "SafeIndex" then -- `SafeIndex{ expr expr }
+    return traverse_safeindex(env, exp)
   elseif tag == "TableCompr" then -- `TableCompr{ block }
     return traverse_tablecompr(env, exp)
+  elseif tag == "MethodStub" or tag == "SafeMethodStub" then -- `(Safe)MethodStub{ expr expr }
+    return traverse_methodstub(env, exp)
   elseif tag:match("Expr$") then -- `StatExpr{ ... }
     return traverse_statexpr(env, exp)
   else
@@ -399,8 +409,6 @@ function traverse_stm (env, stm)
     return traverse_break(env, stm)
   elseif tag == "Call" then -- `Call{ expr expr* }
     return traverse_call(env, stm)
-  elseif tag == "Invoke" then -- `Invoke{ expr `String{ <string> } expr* }
-    return traverse_invoke(env, stm)
   elseif tag == "Continue" then
     return traverse_continue(env, stm)
   elseif tag == "Push" then -- `Push{ <expr>* }

candran/serpent.lua

@@ -0,0 +1,163 @@
+--[[
+Serpent source is released under the MIT License
+
+Copyright (c) 2012-2018 Paul Kulchenko (paul@kulchenko.com)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
+]]
+local n, v = "serpent", "0.302" -- (C) 2012-18 Paul Kulchenko; MIT License
+local c, d = "Paul Kulchenko", "Lua serializer and pretty printer"
+local snum = {[tostring(1/0)]='1/0 --[[math.huge]]',[tostring(-1/0)]='-1/0 --[[-math.huge]]',[tostring(0/0)]='0/0'}
+local badtype = {thread = true, userdata = true, cdata = true}
+local getmetatable = debug and debug.getmetatable or getmetatable
+local pairs = function(t) return next, t end -- avoid using __pairs in Lua 5.2+
+local keyword, globals, G = {}, {}, (_G or _ENV)
+for _,k in ipairs({'and', 'break', 'do', 'else', 'elseif', 'end', 'false',
+  'for', 'function', 'goto', 'if', 'in', 'local', 'nil', 'not', 'or', 'repeat',
+  'return', 'then', 'true', 'until', 'while'}) do keyword[k] = true end
+for k,v in pairs(G) do globals[v] = k end -- build func to name mapping
+for _,g in ipairs({'coroutine', 'debug', 'io', 'math', 'string', 'table', 'os'}) do
+  for k,v in pairs(type(G[g]) == 'table' and G[g] or {}) do globals[v] = g..'.'..k end end
+
+local function s(t, opts)
+  local name, indent, fatal, maxnum = opts.name, opts.indent, opts.fatal, opts.maxnum
+  local sparse, custom, huge = opts.sparse, opts.custom, not opts.nohuge
+  local space, maxl = (opts.compact and '' or ' '), (opts.maxlevel or math.huge)
+  local maxlen, metatostring = tonumber(opts.maxlength), opts.metatostring
+  local iname, comm = '_'..(name or ''), opts.comment and (tonumber(opts.comment) or math.huge)
+  local numformat = opts.numformat or "%.17g"
+  local seen, sref, syms, symn = {}, {'local '..iname..'={}'}, {}, 0
+  local function gensym(val) return '_'..(tostring(tostring(val)):gsub("[^%w]",""):gsub("(%d%w+)",
+    -- tostring(val) is needed because __tostring may return a non-string value
+    function(s) if not syms[s] then symn = symn+1; syms[s] = symn end return tostring(syms[s]) end)) end
+  local function safestr(s) return type(s) == "number" and tostring(huge and snum[tostring(s)] or numformat:format(s))
+    or type(s) ~= "string" and tostring(s) -- escape NEWLINE/010 and EOF/026
+    or ("%q"):format(s):gsub("\010","n"):gsub("\026","\\026") end
+  local function comment(s,l) return comm and (l or 0) < comm and ' --[['..select(2, pcall(tostring, s))..']]' or '' end
+  local function globerr(s,l) return globals[s] and globals[s]..comment(s,l) or not fatal
+    and safestr(select(2, pcall(tostring, s))) or error("Can't serialize "..tostring(s)) end
+  local function safename(path, name) -- generates foo.bar, foo[3], or foo['b a r']
+    local n = name == nil and '' or name
+    local plain = type(n) == "string" and n:match("^[%l%u_][%w_]*$") and not keyword[n]
+    local safe = plain and n or '['..safestr(n)..']'
+    return (path or '')..(plain and path and '.' or '')..safe, safe end
+  local alphanumsort = type(opts.sortkeys) == 'function' and opts.sortkeys or function(k, o, n) -- k=keys, o=originaltable, n=padding
+    local maxn, to = tonumber(n) or 12, {number = 'a', string = 'b'}
+    local function padnum(d) return ("%0"..tostring(maxn).."d"):format(tonumber(d)) end
+    table.sort(k, function(a,b)
+      -- sort numeric keys first: k[key] is not nil for numerical keys
+      return (k[a] ~= nil and 0 or to[type(a)] or 'z')..(tostring(a):gsub("%d+",padnum))
+           < (k[b] ~= nil and 0 or to[type(b)] or 'z')..(tostring(b):gsub("%d+",padnum)) end) end
+  local function val2str(t, name, indent, insref, path, plainindex, level)
+    local ttype, level, mt = type(t), (level or 0), getmetatable(t)
+    local spath, sname = safename(path, name)
+    local tag = plainindex and
+      ((type(name) == "number") and '' or name..space..'='..space) or
+      (name ~= nil and sname..space..'='..space or '')
+    if seen[t] then -- already seen this element
+      sref[#sref+1] = spath..space..'='..space..seen[t]
+      return tag..'nil'..comment('ref', level) end
+    -- protect from those cases where __tostring may fail
+    if type(mt) == 'table' and metatostring ~= false then
+      local to, tr = pcall(function() return mt.__tostring(t) end)
+      local so, sr = pcall(function() return mt.__serialize(t) end)
+      if (to or so) then -- knows how to serialize itself
+        seen[t] = insref or spath
+        t = so and sr or tr
+        ttype = type(t)
+      end -- new value falls through to be serialized
+    end
+    if ttype == "table" then
+      if level >= maxl then return tag..'{}'..comment('maxlvl', level) end
+      seen[t] = insref or spath
+      if next(t) == nil then return tag..'{}'..comment(t, level) end -- table empty
+      if maxlen and maxlen < 0 then return tag..'{}'..comment('maxlen', level) end
+      local maxn, o, out = math.min(#t, maxnum or #t), {}, {}
+      for key = 1, maxn do o[key] = key end
+      if not maxnum or #o < maxnum then
+        local n = #o -- n = n + 1; o[n] is much faster than o[#o+1] on large tables
+        for key in pairs(t) do if o[key] ~= key then n = n + 1; o[n] = key end end end
+      if maxnum and #o > maxnum then o[maxnum+1] = nil end
+      if opts.sortkeys and #o > maxn then alphanumsort(o, t, opts.sortkeys) end
+      local sparse = sparse and #o > maxn -- disable sparsness if only numeric keys (shorter output)
+      for n, key in ipairs(o) do
+        local value, ktype, plainindex = t[key], type(key), n <= maxn and not sparse
+        if opts.valignore and opts.valignore[value] -- skip ignored values; do nothing
+        or opts.keyallow and not opts.keyallow[key]
+        or opts.keyignore and opts.keyignore[key]
+        or opts.valtypeignore and opts.valtypeignore[type(value)] -- skipping ignored value types
+        or sparse and value == nil then -- skipping nils; do nothing
+        elseif ktype == 'table' or ktype == 'function' or badtype[ktype] then
+          if not seen[key] and not globals[key] then
+            sref[#sref+1] = 'placeholder'
+            local sname = safename(iname, gensym(key)) -- iname is table for local variables
+            sref[#sref] = val2str(key,sname,indent,sname,iname,true) end
+          sref[#sref+1] = 'placeholder'
+          local path = seen[t]..'['..tostring(seen[key] or globals[key] or gensym(key))..']'
+          sref[#sref] = path..space..'='..space..tostring(seen[value] or val2str(value,nil,indent,path))
+        else
+          out[#out+1] = val2str(value,key,indent,nil,seen[t],plainindex,level+1)
+          if maxlen then
+            maxlen = maxlen - #out[#out]
+            if maxlen < 0 then break end
+          end
+        end
+      end
+      local prefix = string.rep(indent or '', level)
+      local head = indent and '{\n'..prefix..indent or '{'
+      local body = table.concat(out, ','..(indent and '\n'..prefix..indent or space))
+      local tail = indent and "\n"..prefix..'}' or '}'
+      return (custom and custom(tag,head,body,tail,level) or tag..head..body..tail)..comment(t, level)
+    elseif badtype[ttype] then
+      seen[t] = insref or spath
+      return tag..globerr(t, level)
+    elseif ttype == 'function' then
+      seen[t] = insref or spath
+      if opts.nocode then return tag.."function() --[[..skipped..]] end"..comment(t, level) end
+      local ok, res = pcall(string.dump, t)
+      local func = ok and "((loadstring or load)("..safestr(res)..",'@serialized'))"..comment(t, level)
+      return tag..(func or globerr(t, level))
+    else return tag..safestr(t) end -- handle all other types
+  end
+  local sepr = indent and "\n" or ";"..space
+  local body = val2str(t, name, indent) -- this call also populates sref
+  local tail = #sref>1 and table.concat(sref, sepr)..sepr or ''
+  local warn = opts.comment and #sref>1 and space.."--[[incomplete output with shared/self-references skipped]]" or ''
+  return not name and body..warn or "do local "..body..sepr..tail.."return "..name..sepr.."end"
+end
+
+local function deserialize(data, opts)
+  local env = (opts and opts.safe == false) and G
+    or setmetatable({}, {
+        __index = function(t,k) return t end,
+        __call = function(t,...) error("cannot call functions") end
+      })
+  local f, res = (loadstring or load)('return '..data, nil, nil, env)
+  if not f then f, res = (loadstring or load)(data, nil, nil, env) end
+  if not f then return f, res end
+  if setfenv then setfenv(f, env) end
+  return pcall(f)
+end
+
+local function merge(a, b) if b then for k,v in pairs(b) do a[k] = v end end; return a; end
+return { _NAME = n, _COPYRIGHT = c, _DESCRIPTION = d, _VERSION = v, serialize = s,
+  load = deserialize,
+  dump = function(a, opts) return s(a, merge({name = '_', compact = true, sparse = true}, opts)) end,
+  line = function(a, opts) return s(a, merge({sortkeys = true, comment = true}, opts)) end,
+  block = function(a, opts) return s(a, merge({indent = '  ', sortkeys = true, comment = true}, opts)) end }

candran/util.can

@@ -0,0 +1,125 @@
+local candran = require("candran")
+local util = {}
+
+function util.search(modpath, exts={})
+	for _, ext in ipairs(exts) do
+		for path in package.path:gmatch("[^;]+") do
+			local fpath = path:gsub("%.lua", "."..ext):gsub("%?", (modpath:gsub("%.", "/")))
+			local f = io.open(fpath)
+			if f then
+				f:close()
+				return fpath
+			end
+		end
+	end
+end
+
+function util.load(str, name, env)
+	if _VERSION == "Lua 5.1" then
+		local fn, err = loadstring(str, name)
+		if not fn then return fn, err end
+		return env ~= nil and setfenv(fn, env) or fn
+	else
+		if env then
+			return load(str, name, nil, env)
+		else
+			return load(str, name)
+		end
+	end
+end
+
+function util.recmerge(...)
+	local r = {}
+	for _, t in ipairs({...}) do
+		for k, v in pairs(t) do
+			if type(v) == "table" then
+				r[k] = util.merge(v, r[k])
+			else
+				r[k] = v
+			end
+		end
+	end
+	return r
+end
+
+function util.merge(...)
+	local r = {}
+	for _, t in ipairs({...}) do
+		for k, v in pairs(t) do
+			r[k] = v
+		end
+	end
+	return r
+end
+
+util.cli = {
+	-- add option to set Candran options to an argparse parser
+	addCandranOptions = function(parser)
+		parser:group("Compiler options",
+			parser:option("-t --target")
+				:description "Target Lua version: lua54, lua53, lua52, luajit or lua51"
+				:default(candran.default.target),
+
+			parser:option("--indentation")
+				:description "Character(s) used for indentation in the compiled file"
+				:default(candran.default.indentation),
+
+			parser:option("--newline")
+				:description "Character(s) used for newlines in the compiled file"
+				:default(candran.default.newline),
+
+			parser:option("--variable-prefix")
+				:description "Prefix used when Candran needs to set a local variable to provide some functionality"
+				:default(candran.default.variablePrefix),
+
+			parser:flag("--no-map-lines")
+				:description "Do not add comments at the end of each line indicating the associated source line and file (error rewriting will not work)"
+		)
+
+		parser:group("Preprocessor options",
+			parser:flag("--no-builtin-macros")
+				:description "Disable built-in macros",
+
+			parser:option("-D --define")
+				:description "Define a preprocessor constant"
+				:args("1-2")
+				:argname{"name", "value"}
+				:count("*"),
+
+			parser:option("-I --import")
+				:description "Statically import a module into the compiled file"
+				:argname("module")
+				:count("*")
+		)
+
+		parser:option("--chunkname")
+			:description "Chunkname used when running the code"
+
+		parser:flag("--no-rewrite-errors")
+			:description "Disable error rewriting when running the code"
+	end,
+
+	-- convert parsed arguments to a Candran options table
+	makeCandranOptions = function(args)
+		local preprocessorEnv = {}
+		for _, o in ipairs(args.define) do
+			preprocessorEnv[o[1]] = tonumber(o[2]) or o[2] or true
+		end
+
+		local options = {
+			target = args.target,
+			indentation = args.indentation,
+			newline = args.newline,
+			variablePrefix = args.variable_prefix,
+			mapLines = not args.no_map_lines,
+			chunkname = args.chunkname,
+			rewriteErrors = not args.no_rewrite_errors,
+			builtInMacros = not args.no_builtin_macros,
+			preprocessorEnv = preprocessorEnv,
+			import = args.import
+		}
+		return options
+	end
+}
+
+return util

compiler/lua51.can

@@ -0,0 +1,38 @@
+targetName = "Lua 5.1"
+
+states.continue = {} -- when in a loop that use continue
+
+CONTINUE_START = ()
+	return "local " .. var("break") .. newline() .. "repeat" .. indent() .. push("continue", var("break"))
+end
+CONTINUE_STOP = ()
+	return pop("continue") .. unindent() .. "until true" .. newline() .. "if " .. var("break") .. " then break end"
+end
+
+tags.Continue = ()
+	return "break"
+end
+tags.Break = ()
+	local inContinue = peek("continue")
+	if inContinue then
+		return inContinue .. " = true" .. newline() .. "break"
+	else
+		return "break"
+	end
+end
+
+-- Unsuported features
+tags.Goto = ()
+	error("target "..targetName.." does not support gotos")
+end
+tags.Label = ()
+	error("target "..targetName.." does not support goto labels")
+end
+
+#placeholder("patch")
+
+#local patch = output
+#output = ""
+#import("compiler.luajit", { preprocessorEnv = { patch = patch }, loadPackage = false })
+
+return luajit

compiler/lua52.can

@@ -0,0 +1,35 @@
+targetName = "Lua 5.2"
+
+APPEND = (t, toAppend)
+	return "do" .. indent() .. "local "..var("a")..", "..var("p").." = { " .. toAppend .. " }, #" .. t .. "+1" .. newline() .. "for i=1, #"..var("a").." do" .. indent() .. t .. "["..var("p").."] = "..var("a").."[i]" .. newline() .. ""..var("p").." = "..var("p").." + 1" .. unindent() .. "end" .. unindent() .. "end"
+end
+
+tags._opid.idiv = (left, right)
+	return "math.floor(" .. lua(left) .. " / " .. lua(right) .. ")"
+end
+tags._opid.band = (left, right)
+	return "bit32.band(" .. lua(left) .. ", " .. lua(right) .. ")"
+end
+tags._opid.bor = (left, right)
+	return "bit32.bor(" .. lua(left) .. ", " .. lua(right) .. ")"
+end
+tags._opid.bxor = (left, right)
+	return "bit32.bxor(" .. lua(left) .. ", " .. lua(right) .. ")"
+end
+tags._opid.shl = (left, right)
+	return "bit32.lshift(" .. lua(left) .. ", " .. lua(right) .. ")"
+end
+tags._opid.shr = (left, right)
+	return "bit32.rshift(" .. lua(left) .. ", " .. lua(right) .. ")"
+end
+tags._opid.bnot = (right)
+	return "bit32.bnot(" .. lua(right) .. ")"
+end
+
+#placeholder("patch")
+
+#local patch = output
+#output = ""
+#import("compiler.lua53", { preprocessorEnv = { patch = patch }, loadPackage = false })
+
+return lua53

compiler/lua53.can

@@ -1,565 +1,18 @@
-return function(code, ast, options)
-	--- Line mapping
-	local lastInputPos = 1 -- last token position in the input code
-	local prevLinePos = 1 -- last token position in the previous line of code in the input code
-	local lastSource = options.chunkname or "nil" -- last found code source name (from the original file)
-	local lastLine = 1 -- last found line number (from the original file)
+targetName = "Lua 5.3"
 
-	--- Newline management
-	local indentLevel = 0
-	-- Returns a newline.
-	local function newline()
-		local r = options.newline .. string.rep(options.indentation, indentLevel)
-		if options.mapLines then
-			local sub = code:sub(lastInputPos)
-			local source, line = sub:sub(1, sub:find("\n")):match("%-%- (.-)%:(%d+)\n")
-
-			if source and line then
-				lastSource = source
-				lastLine = tonumber(line)
-			else
-				for _ in code:sub(prevLinePos, lastInputPos):gmatch("\n") do
-					lastLine += 1
-				end
-			end
-
-			prevLinePos = lastInputPos
-
-			r = " -- " .. lastSource .. ":" .. lastLine .. r
-		end
-		return r
+-- Unsuported features
+tags.AttributeId = (t)
+	if t[2] then
+		error("target "..targetName.." does not support variable attributes")
+	else
+		return t[1]
 	end
-	-- Returns a newline and add one level of indentation.
-	local function indent()
-		indentLevel += 1
-		return newline()
-	end
-	-- Returns a newline and remove one level of indentation.
-	local function unindent()
-		indentLevel -= 1
-		return newline()
-	end
-
-	--- Module management
-	local required = {} -- { ["module"] = true, ... }
-	local requireStr = ""
-	-- Add the module "mod" to the list of modules to require, and load its field "field" (or the whole module if nil) into the variable "name".
-	local function addRequire(mod, name, field)
-		if not required[mod] then
-			requireStr ..= "local " .. options.variablePrefix .. name .. (" = require(%q)"):format(mod) .. (field and "."..field or "") .. options.newline
-			required[mod] = true
-		end
-	end
-
-	--- Variable management
-	-- Returns the prefixed variable name.
-	local function var(name)
-		return options.variablePrefix .. name
-	end
-
-	--- AST traversal helpers
-	local loop = { "While", "Repeat", "Fornum", "Forin" } -- loops tags
-	local func = { "Function", "TableCompr", "DoExpr", "WhileExpr", "RepeatExpr", "IfExpr", "FornumExpr", "ForinExpr" } -- function scope tags
-	-- Returns the first node or subnode from the list "list" which tag is in the list "tags", or nil if there were none.
-	-- Won't recursively follow nodes which have a tag in "nofollow".
-	local function any(list, tags, nofollow={})
-		local tagsCheck = {}
-		for _, tag in ipairs(tags) do
-			tagsCheck[tag] = true
-		end
-		local nofollowCheck = {}
-		for _, tag in ipairs(nofollow) do
-			nofollowCheck[tag] = true
-		end
-		for _, node in ipairs(list) do
-			if type(node) == "table" then
-				if tagsCheck[node.tag] then
-					return node
-				end
-				if not nofollowCheck[node.tag] then
-					local r = any(node, tags, nofollow)
-					if r then return r end
-				end
-			end
-		end
-		return nil
-	end
-
-	--- State stacks
-	-- Used for context-sensitive syntax.
-	local states = {
-		push = {} -- push stack variable names
-	}
-	-- Push a new value on top of the stack "name". Returns an empty string for chaining.
-	local function push(name, state)
-		table.insert(states[name], state)
-		return ""
-	end
-	-- Remove the value on top of the stack "name". Returns an empty string for chaining.
-	local function pop(name)
-		table.remove(states[name])
-		return ""
-	end
-	-- Returns the value on top of the stack "name".
-	local function peek(name)
-		return states[name][#states[name]]
-	end
-
-	--- Lua compiler
-	local tags
-	-- Recursively returns the compiled AST Lua code, set "forceTag" to override the tag type and pass additional arguments to the tag constructor if needed.
-	local function lua(ast, forceTag, ...)
-		if options.mapLines and ast.pos then
-			lastInputPos = ast.pos
-		end
-		return tags[forceTag or ast.tag](ast, ...)
-	end
-
-	--- Lua function calls writer
-	local UNPACK = (list, i, j) -- table.unpack
-		return "table.unpack(" .. list .. (i and (", " .. i .. (j and (", " .. j) or "")) or "") .. ")"
-	end
-	local APPEND = (t, toAppend) -- append values "toAppend" (multiple values possible) to t
-		return "do" .. indent() .. "local a = table.pack(" .. toAppend .. ")" .. newline() .. "table.move(a, 1, a.n, #" .. t .. "+1, " .. t .. ")" .. unindent() .. "end"
-	end
-
-	--- Tag constructors
-	tags = setmetatable({
-		-- block: { stat* } --
-		Block = (t)
-			local hasPush = peek("push") == nil and any(t, { "Push" }, func) -- push in block and push context not yet defined
-			if hasPush and hasPush == t[#t] then -- if the first push is the last statement, it's just a return
-				hasPush.tag = "Return"
-				hasPush = false
-			end
-			local r = ""
-			if hasPush then
-				r ..= push("push", var("push")) .. "local " .. var("push") .. " = {}" .. newline()
-			end
-			for i=1, #t-1, 1 do
-				r ..= lua(t[i]) .. newline()
-			end
-			if t[#t] then
-				r ..= lua(t[#t])
-			end
-			if hasPush and (t[#t] and t[#t].tag ~= "Return") then -- add return only if needed
-				r ..= newline() .. "return " .. UNPACK(var("push")) .. pop("push")
-			end
-			return r
-		end,
-
-		-- stat --
-
-		-- Do{ stat* }
-		Do = (t)
-			return "do" .. indent() .. lua(t, "Block") .. unindent() .. "end"
-		end,
-		-- Set{ {lhs+} (opid? = opid?)? {expr+} }
-		Set = (t)
-			if #t == 2 then
-				return lua(t[1], "_lhs") .. " = " .. lua(t[2], "_lhs")
-			elseif #t == 3 then
-				return lua(t[1], "_lhs") .. " = " .. lua(t[3], "_lhs")
-			elseif #t == 4 then
-				if t[3] == "=" then
-					local r = lua(t[1], "_lhs") .. " = " .. lua({ t[2], t[1][1], { tag = "Paren", t[4][1] } }, "Op")
-					for i=2, math.min(#t[4], #t[1]), 1 do
-						r ..= ", " .. lua({ t[2], t[1][i], { tag = "Paren", t[4][i] } }, "Op")
-					end
-					return r
-				else
-					local r = lua(t[1], "_lhs") .. " = " .. lua({ t[3], { tag = "Paren", t[4][1] }, t[1][1] }, "Op")
-					for i=2, math.min(#t[4], #t[1]), 1 do
-						r ..= ", " .. lua({ t[3], { tag = "Paren", t[4][i] }, t[1][i] }, "Op")
-					end
-					return r
-				end
-			else -- You are mad.
-				local r = lua(t[1], "_lhs") .. " = " .. lua({ t[2], t[1][1], { tag = "Op", t[4], { tag = "Paren", t[5][1] }, t[1][1] } }, "Op")
-				for i=2, math.min(#t[5], #t[1]), 1 do
-					r ..= ", " .. lua({ t[2], t[1][i], { tag = "Op", t[4], { tag = "Paren", t[5][i] }, t[1][i] } }, "Op")
-				end
-				return r
-			end
-		end,
-		-- While{ expr block }
-		While = (t)
-			local hasContinue = any(t[2], { "Continue" }, loop)
-			local r = "while " .. lua(t[1]) .. " do" .. indent()
-			if hasContinue then
-				r ..= "repeat" .. indent()
-			end
-			r .. = lua(t[2])
-			if hasContinue then
-				r ..= unindent() .. "until true"
-			end
-			r ..= unindent() .. "end"
-			return r
-		end,
-		-- Repeat{ block expr }
-		Repeat = (t)
-			local hasContinue = any(t[2], { "Continue" }, loop)
-			local r = "repeat" .. indent()
-			if hasContinue then
-				r ..= "repeat" .. indent()
-			end
-			r .. = lua(t[1])
-			if hasContinue then
-				r ..= unindent() .. "until true"
-			end
-			r ..= unindent() .. "until " .. lua(t[2])
-			return r
-		end,
-		-- If{ (expr block)+ block? }
-		If = (t)
-			local r = "if " .. lua(t[1]) .. " then" .. indent() .. lua(t[2]) .. unindent()
-			for i=3, #t-1, 2 do
-				r ..= "elseif " .. lua(t[i]) .. " then" .. indent() .. lua(t[i+1]) .. unindent()
-			end
-			if #t % 2 == 1 then
-				r ..= "else" .. indent() .. lua(t[#t]) .. unindent()
-			end
-			return r .. "end"
-		end,
-		-- Fornum{ ident expr expr expr? block }
-		Fornum = (t)
-			local r = "for " .. lua(t[1]) .. " = " .. lua(t[2]) .. ", " .. lua(t[3])
-			if #t == 5 then
-				local hasContinue = any(t[5], { "Continue" }, loop)
-				r ..= ", " .. lua(t[4]) .. " do" .. indent()
-				if hasContinue then
-					r ..= "repeat" .. indent()
-				end
-				r ..= lua(t[5])
-				if hasContinue then
-					r ..= unindent() .. "until true"
-				end
-				return r .. unindent() .. "end"
-			else
-				local hasContinue = any(t[4], { "Continue" }, loop)
-				r ..= " do" .. indent()
-				if hasContinue then
-					r ..= "repeat" .. indent()
-				end
-				r ..= lua(t[4])
-				if hasContinue then
-					r ..= unindent() .. "until true"
-				end
-				return r .. unindent() .. "end"
-			end
-		end,
-		-- Forin{ {ident+} {expr+} block }
-		Forin = (t)
-			local hasContinue = any(t[3], { "Continue" }, loop)
-			local r = "for " .. lua(t[1], "_lhs") .. " in " .. lua(t[2], "_lhs") .. " do" .. indent()
-			if hasContinue then
-				r ..= "repeat" .. indent()
-			end
-			r ..= lua(t[3])
-			if hasContinue then
-				r ..= unindent() .. "until true"
-			end
-			return r .. unindent() .. "end"
-		end,
-		-- Local{ {ident+} {expr+}? }
-		Local = (t)
-			local r = "local "..lua(t[1], "_lhs")
-			if t[2][1] then
-				r ..= " = "..lua(t[2], "_lhs")
-			end
-			return r
-		end,
-		-- Let{ {ident+} {expr+}? }
-		Let = (t)
-			local nameList = lua(t[1], "_lhs")
-			local r = "local " .. nameList
-			if t[2][1] then
-				if any(t[2], { "Function", "Table", "Paren" }) then -- predeclaration doesn't matter otherwise
-					r ..= newline() .. nameList .. " = " .. lua(t[2], "_lhs")
-				else
-					r ..= " = " .. lua(t[2], "_lhs")
-				end
-			end
-			return r
-		end,
-		-- Localrec{ {ident} {expr} }
-		Localrec = (t)
-			return "local function "..lua(t[1][1])..lua(t[2][1], "_functionWithoutKeyword")
-		end,
-		-- Goto{ <string> }
-		Goto = (t)
-			return "goto " .. lua(t, "Id")
-		end,
-		-- Label{ <string> }
-		Label = (t)
-			return "::" .. lua(t, "Id") .. "::"
-		end,
-		-- Return{ <expr*> }
-		Return = (t)
-			local push = peek("push")
-			if push then
-				local r = ""
-				for _, val in ipairs(t) do
-					r ..= push .. "[#" .. push .. "+1] = " .. lua(val) .. newline()
-				end
-				return r .. "return " .. UNPACK(push)
-			else
-				return "return "..lua(t, "_lhs")
-			end
-		end,
-		-- Push{ <expr*> }
-		Push = (t)
-			local var = assert(peek("push"), "no context given for push")
-			r = ""
-			for i=1, #t-1, 1 do
-				r ..= var .. "[#" .. var .. "+1] = " .. lua(t[i]) .. newline()
-			end
-			if t[#t] then
-				if t[#t].tag == "Call" or t[#t].tag == "Invoke" then
-					r ..= APPEND(var, lua(t[#t]))
-				else
-					r ..= var .. "[#" .. var .. "+1] = " .. lua(t[#t])
-				end
-			end
-			return r
-		end,
-		-- Break
-		Break = ()
-			return "break"
-		end,
-		-- Continue
-		Continue = ()
-			return "break"
-		end,
-		-- apply (below)
-
-		-- expr --
-
-		-- Nil
-		Nil = ()
-			return "nil"
-		end,
-		-- Dots
-		Dots = ()
-			return "..."
-		end,
-		-- Boolean{ <boolean> }
-		Boolean = (t)
-			return tostring(t[1])
-		end,
-		-- Number{ <string> }
-		Number = (t)
-			return tostring(t[1])
-		end,
-		-- String{ <string> }
-		String = (t)
-			return ("%q"):format(t[1])
-		end,
-		-- Function{ { ( `ParPair{ Id expr } | `Id{ <string> } )* `Dots? } block }
-		_functionWithoutKeyword = (t)
-			local r = "("
-			local decl = {}
-			if t[1][1] then
-				if t[1][1].tag == "ParPair" then
-					local id = lua(t[1][1][1])
-					indentLevel += 1
-					table.insert(decl, "if " .. id .. " == nil then " .. id .. " = " .. lua(t[1][1][2]) .. " end")
-					indentLevel -= 1
-					r ..= id
-				else
-					r ..= lua(t[1][1])
-				end
-				for i=2, #t[1], 1 do
-					if t[1][i].tag == "ParPair" then
-						local id = lua(t[1][i][1])
-						indentLevel += 1
-						table.insert(decl, "if " .. id .. " == nil then " .. id .. " = " .. lua(t[1][i][2]) .. " end")
-						indentLevel -= 1
-						r ..= ", " ..id
-					else
-						r ..= ", " .. lua(t[1][i])
-					end
-				end
-			end
-			r ..= ")" .. indent()
-			for _, d in ipairs(decl) do
-				r ..= d .. newline()
-			end
-			if t[2][#t[2]] and t[2][#t[2]].tag == "Push" then -- convert final push to return
-				t[2][#t[2]].tag = "Return"
-			end
-			local hasPush = any(t[2], { "Push" }, func)
-			if hasPush then
-				r ..= push("push", var("push")) .. "local " .. var("push") .. " = {}" .. newline()
-			else
-				push("push", false) -- no push here (make sure higher push don't affect us)
-			end
-			r ..= lua(t[2])
-			if hasPush and (t[2][#t[2]] and t[2][#t[2]].tag ~= "Return") then -- add return only if needed
-				r ..= newline() .. "return " .. UNPACK(var("push"))
-			end
-			pop("push")
-			return r .. unindent() .. "end"
-		end,
-		Function = (t)
-			return "function" .. lua(t, "_functionWithoutKeyword")
-		end,
-		-- Table{ ( `Pair{ expr expr } | expr )* }
-		Pair = (t)
-			return "[" .. lua(t[1]) .. "] = " .. lua(t[2])
-		end,
-		Table = (t)
-			if #t == 0 then
-				return "{}"
-			elseif #t == 1 then
-				return "{ " .. lua(t, "_lhs") .. " }"
-			else
-				return "{" .. indent() .. lua(t, "_lhs", nil, true) .. unindent() .. "}"
-			end
-		end,
-		-- TableCompr{ block }
-		TableCompr = (t)
-			return push("push", "self") .. "(function()" .. indent() .. "local self = {}" .. newline() .. lua(t[1]) .. newline() .. "return self" .. unindent() .. "end)()" .. pop("push")
-		end,
-		-- Op{ opid expr expr? }
-		Op = (t)
-			local r
-			if #t == 2 then
-				if type(tags._opid[t[1]]) == "string" then
-					r = tags._opid[t[1]] .. " " .. lua(t[2])
-				else
-					r = tags._opid[t[1]](t[2])
-				end
-			else
-				if type(tags._opid[t[1]]) == "string" then
-					r = lua(t[2]) .. " " .. tags._opid[t[1]] .. " " .. lua(t[3])
-				else
-					r = tags._opid[t[1]](t[2], t[3])
-				end
-			end
-			return r
-		end,
-		-- Paren{ expr }
-		Paren = (t)
-			return "(" .. lua(t[1]) .. ")"
-		end,
-		-- statexpr (below)
-		-- apply (below)
-		-- lhs (below)
-
-		-- statexpr --
-		_statexpr = (t, stat)
-			local hasPush = any(t, { "Push" }, func)
-			local r = "(function()" .. indent()
-			if hasPush then
-				r ..= push("push", var("push")) .. "local " .. var("push") .. " = {}" .. newline()
-			else
-				push("push", false) -- no push here (make sure higher push don't affect us)
-			end
-			r ..= lua(t, stat)
-			if hasPush then
-				r ..= newline() .. "return " .. UNPACK(var("push"))
-			end
-			pop("push")
-			r ..= unindent() .. "end)()"
-			return r
-		end,
-		-- DoExpr{ stat* }
-		DoExpr = (t)
-			if t[#t].tag == "Push" then -- convert final push to return
-				t[#t].tag = "Return"
-			end
-			return lua(t, "_statexpr", "Do")
-		end,
-		-- WhileExpr{ expr block }
-		WhileExpr = (t)
-			return lua(t, "_statexpr", "While")
-		end,
-		-- RepeatExpr{ block expr }
-		RepeatExpr = (t)
-			return lua(t, "_statexpr", "Repeat")
-		end,
-		-- IfExpr{ (expr block)+ block? }
-		IfExpr = (t)
-			for i=2, #t do -- convert final pushes to returns
-				local block = t[i]
-				if block[#block] and block[#block].tag == "Push" then
-					block[#block].tag = "Return"
-				end
-			end
-			return lua(t, "_statexpr", "If")
-		end,
-		-- FornumExpr{ ident expr expr expr? block }
-		FornumExpr = (t)
-			return lua(t, "_statexpr", "Fornum")
-		end,
-		-- ForinExpr{ {ident+} {expr+} block }
-		ForinExpr = (t)
-			return lua(t, "_statexpr", "Forin")
-		end,
-
-		-- apply --
-
-		-- Call{ expr expr* }
-		Call = (t)
-			if t[1].tag == "String" or t[1].tag == "Table" then
-				return "("..lua(t[1]) .. ")(" .. lua(t, "_lhs", 2) .. ")"
-			else
-				return lua(t[1]) .. "(" .. lua(t, "_lhs", 2) .. ")"
-			end
-		end,
-
-		-- Invoke{ expr `String{ <string> } expr* }
-		Invoke = (t)
-			if t[1].tag == "String" or t[1].tag == "Table" then
-				return "("..lua(t[1]).."):"..lua(t[2], "Id").."("..lua(t, "_lhs", 3)..")"
-			else
-				return lua(t[1])..":"..lua(t[2], "Id").."("..lua(t, "_lhs", 3)..")"
-			end
-		end,
-
-		-- lhs --
-		_lhs = (t, start=1, newlines)
-			local r
-			if t[start] then
-				r = lua(t[start])
-				for i=start+1, #t, 1 do
-					r ..= "," .. (newlines and newline() or " ") .. lua(t[i])
-				end
-			else
-				r = ""
-			end
-			return r
-		end,
-		-- Id{ <string> }
-		Id = (t)
-			return t[1]
-		end,
-		-- Index{ expr expr }
-		Index = (t)
-			if t[1].tag == "String" or t[1].tag == "Table" then
-				return "("..lua(t[1])..")["..lua(t[2]).."]"
-			else
-				return lua(t[1]).."["..lua(t[2]).."]"
-			end
-		end,
-
-		-- opid --
-		_opid = {
-			add = "+", sub = "-", mul = "*", div = "/",
-			idiv = "//", mod = "%", pow = "^", concat = "..",
-			band = "&", bor = "|", bxor = "~", shl = "<<", shr = ">>",
-			eq = "==", ne = "~=", lt = "<", gt = ">", le = "<=", ge = ">=",
-			["and"] = "and", ["or"] = "or", unm = "-", len = "#", bnot = "~", ["not"] = "not"
-		}
-	}, {
-		__index = (self, key)
-			error("don't know how to compile a "..tostring(key).." to Lua 5.3")
-		end
-	})
-
-	#placeholder("patch")
-
-	local code = lua(ast) .. newline()
-	return requireStr .. code
 end
+
+#placeholder("patch")
+
+#local patch = output
+#output = ""
+#import("compiler.lua54", { preprocessorEnv = { patch = patch }, loadPackage = false })
+
+return lua54

compiler/lua54.can

@@ -0,0 +1,898 @@
+local util = require("candran.util")
+
+local targetName = "Lua 5.4"
+
+local unpack = unpack or table.unpack
+
+return function(code, ast, options, macros={functions={}, variables={}})
+	--- Line mapping
+	local lastInputPos = 1 -- last token position in the input code
+	local prevLinePos = 1 -- last token position in the previous line of code in the input code
+	local lastSource = options.chunkname or "nil" -- last found code source name (from the original file)
+	local lastLine = 1 -- last found line number (from the original file)
+
+	--- Newline management
+	local indentLevel = 0
+	-- Returns a newline.
+	local function newline()
+		local r = options.newline..string.rep(options.indentation, indentLevel)
+		if options.mapLines then
+			local sub = code:sub(lastInputPos)
+			local source, line = sub:sub(1, sub:find("\n")):match(".*%-%- (.-)%:(%d+)\n")
+
+			if source and line then
+				lastSource = source
+				lastLine = tonumber(line)
+			else
+				for _ in code:sub(prevLinePos, lastInputPos):gmatch("\n") do
+					lastLine += 1
+				end
+			end
+
+			prevLinePos = lastInputPos
+
+			r = " -- "..lastSource..":"..lastLine..r
+		end
+		return r
+	end
+	-- Returns a newline and add one level of indentation.
+	local function indent()
+		indentLevel += 1
+		return newline()
+	end
+	-- Returns a newline and remove one level of indentation.
+	local function unindent()
+		indentLevel -= 1
+		return newline()
+	end
+
+	--- State stacks
+	-- Used for context-sensitive syntax.
+	local states = {
+		push = {}, -- push stack variable names
+		destructuring = {}, -- list of variable that need to be assigned from a destructure {id = "parent variable", "field1", "field2"...}
+		scope = {}, -- list of variables defined in the current scope
+		macroargs = {} -- currently defined arguemnts from a macro function
+	}
+	-- Push a new value on top of the stack "name". Returns an empty string for chaining.
+	local function push(name, state)
+		table.insert(states[name], state)
+		return ""
+	end
+	-- Remove the value on top of the stack "name". Returns an empty string for chaining.
+	local function pop(name)
+		table.remove(states[name])
+		return ""
+	end
+	-- Set the value on top of the stack "name". Returns an empty string for chaining.
+	local function set(name, state)
+		states[name][#states[name]] = state
+		return ""
+	end
+	-- Returns the value on top of the stack "name".
+	local function peek(name)
+		return states[name][#states[name]]
+	end
+
+	--- Variable management
+	-- Returns the prefixed variable name.
+	local function var(name)
+		return options.variablePrefix..name
+	end
+
+	-- Returns the prefixed temporary variable name.
+	local function tmp()
+		local scope = peek("scope")
+		local var = "%s_%s":format(options.variablePrefix, #scope)
+		table.insert(scope, var)
+		return var
+	end
+
+	-- indicate if currently processing a macro, so it cannot be applied recursively
+	local nomacro = { variables = {}, functions = {} }
+
+	--- Module management
+	local required = {} -- { ["full require expression"] = true, ... }
+	local requireStr = ""
+	-- Add the module "mod" to the list of modules to require, and load its field "field" (or the whole module if nil) into the variable "name".
+	local function addRequire(mod, name, field)
+		local req = "require(%q)%s":format(mod, field and "."..field or "")
+		if not required[req] then
+			requireStr ..= "local %s = %s%s":format(var(name), req, options.newline)
+			required[req] = true
+		end
+	end
+
+	--- AST traversal helpers
+	local loop = { "While", "Repeat", "Fornum", "Forin", "WhileExpr", "RepeatExpr", "FornumExpr", "ForinExpr" } -- loops tags (can contain continue)
+	local func = { "Function", "TableCompr", "DoExpr", "WhileExpr", "RepeatExpr", "IfExpr", "FornumExpr", "ForinExpr" } -- function scope tags (can contain push)
+
+	-- Returns the first node or subnode from the list "list" which tag is in the list "tags", or nil if there were none.
+	-- Won't recursively follow nodes which have a tag in "nofollow".
+	local function any(list, tags, nofollow={})
+		local tagsCheck = {}
+		for _, tag in ipairs(tags) do
+			tagsCheck[tag] = true
+		end
+		local nofollowCheck = {}
+		for _, tag in ipairs(nofollow) do
+			nofollowCheck[tag] = true
+		end
+		for _, node in ipairs(list) do
+			if type(node) == "table" then
+				if tagsCheck[node.tag] then
+					return node
+				end
+				if not nofollowCheck[node.tag] then
+					local r = any(node, tags, nofollow)
+					if r then return r end
+				end
+			end
+		end
+		return nil
+	end
+
+	-- Like any, but returns a list of every node found.
+	-- Order: in the order of the list, from the deepest to the nearest
+	local function search(list, tags, nofollow={})
+		local tagsCheck = {}
+		for _, tag in ipairs(tags) do
+			tagsCheck[tag] = true
+		end
+		local nofollowCheck = {}
+		for _, tag in ipairs(nofollow) do
+			nofollowCheck[tag] = true
+		end
+		local found = {}
+		for _, node in ipairs(list) do
+			if type(node) == "table" then
+				if not nofollowCheck[node.tag] then
+					for _, n in ipairs(search(node, tags, nofollow)) do
+						table.insert(found, n)
+					end
+				end
+				if tagsCheck[node.tag] then
+					table.insert(found, node)
+				end
+			end
+		end
+		return found
+	end
+
+	-- Returns true if the all the nodes in list have their type in tags.
+	local function all(list, tags)
+		for _, node in ipairs(list) do
+			local ok = false
+			for _, tag in ipairs(tags) do
+				if node.tag == tag then
+					ok = true
+					break
+				end
+			end
+			if not ok then
+				return false
+			end
+		end
+		return true
+	end
+
+	--- Lua compiler
+	local tags
+	-- Recursively returns the compiled AST Lua code, set "forceTag" to override the tag type and pass additional arguments to the tag constructor if needed.
+	local function lua(ast, forceTag, ...)
+		if options.mapLines and ast.pos then
+			lastInputPos = ast.pos
+		end
+		return tags[forceTag or ast.tag](ast, ...)
+	end
+
+	--- Lua function calls writer
+	local UNPACK = (list, i, j) -- table.unpack
+		return "table.unpack("..list..(i and (", "..i..(j and (", "..j) or "")) or "")..")"
+	end
+	local APPEND = (t, toAppend) -- append values "toAppend" (multiple values possible) to t
+		return "do"..indent().."local "..var("a").." = table.pack("..toAppend..")"..newline().."table.move("..var("a")..", 1, "..var("a")..".n, #"..t.."+1, "..t..")"..unindent().."end"
+	end
+	local CONTINUE_START = () -- at the start of loops using continue
+		return "do"..indent()
+	end
+	local CONTINUE_STOP = () -- at the start of loops using continue
+		return unindent().."end"..newline().."::"..var"continue".."::"
+	end
+	local DESTRUCTURING_ASSIGN = (destructured, newlineAfter=false, noLocal=false) -- to define values from a destructuring assignement
+		local vars = {}
+		local values = {}
+		for _, list in ipairs(destructured) do
+			for _, v in ipairs(list) do
+				local var, val
+				if v.tag == "Id" or v.tag == "AttributeId" then
+					var = v
+					val = { tag = "Index", { tag = "Id", list.id }, { tag = "String", v[1] } }
+				elseif v.tag == "Pair" then
+					var = v[2]
+					val = { tag = "Index", { tag = "Id", list.id }, v[1] }
+				else
+					error("unknown destructuring element type: "..tostring(v.tag))
+				end
+				if destructured.rightOp and destructured.leftOp then
+					val = { tag = "Op", destructured.rightOp, var, { tag = "Op", destructured.leftOp, val, var } }
+				elseif destructured.rightOp then
+					val = { tag = "Op", destructured.rightOp, var, val }
+				elseif destructured.leftOp then
+					val = { tag = "Op", destructured.leftOp, val, var }
+				end
+				table.insert(vars, lua(var))
+				table.insert(values, lua(val))
+			end
+		end
+		if #vars > 0 then
+			local decl = noLocal and "" or "local "
+			if newlineAfter then
+				return decl..table.concat(vars, ", ").." = "..table.concat(values, ", ")..newline()
+			else
+				return newline()..decl..table.concat(vars, ", ").." = "..table.concat(values, ", ")
+			end
+		else
+			return ""
+		end
+	end
+
+	--- Tag constructors
+	tags = setmetatable({
+		-- block: { stat* } --
+		Block = (t)
+			local hasPush = peek("push") == nil and any(t, { "Push" }, func) -- push in block and push context not yet defined
+			if hasPush and hasPush == t[#t] then -- if the first push is the last statement, it's just a return
+				hasPush.tag = "Return"
+				hasPush = false
+			end
+			local r = push("scope", {})
+			if hasPush then
+				r ..= push("push", var"push").."local "..var"push".." = {}"..newline()
+			end
+			for i=1, #t-1, 1 do
+				r ..= lua(t[i])..newline()
+			end
+			if t[#t] then
+				r ..= lua(t[#t])
+			end
+			if hasPush and (t[#t] and t[#t].tag ~= "Return") then -- add return only if needed
+				r ..= newline().."return "..UNPACK(var"push")..pop("push")
+			end
+			return r..pop("scope")
+		end,
+
+		-- stat --
+
+		-- Do{ stat* }
+		Do = (t)
+			return "do"..indent()..lua(t, "Block")..unindent().."end"
+		end,
+		-- Set{ {lhs+} (opid? = opid?)? {expr+} }
+		Set = (t)
+			-- extract vars and values
+			local expr = t[#t]
+			local vars, values = {}, {}
+			local destructuringVars, destructuringValues = {}, {}
+			for i, n in ipairs(t[1]) do
+				if n.tag == "DestructuringId" then
+					table.insert(destructuringVars, n)
+					table.insert(destructuringValues, expr[i])
+				else
+					table.insert(vars, n)
+					table.insert(values, expr[i])
+				end
+			end
+			--
+			if #t == 2 or #t == 3 then
+				local r = ""
+				if #vars > 0 then
+					r = lua(vars, "_lhs").." = "..lua(values, "_lhs")
+				end
+				if #destructuringVars > 0 then
+					local destructured = {}
+					r ..= "local "..push("destructuring", destructured)..lua(destructuringVars, "_lhs")..pop("destructuring").." = "..lua(destructuringValues, "_lhs")
+					return r..DESTRUCTURING_ASSIGN(destructured, nil, true)
+				end
+				return r
+			elseif #t == 4 then
+				if t[3] == "=" then
+					local r = ""
+					if #vars > 0 then
+						r ..= lua(vars, "_lhs").." = "..lua({ t[2], vars[1], { tag = "Paren", values[1] } }, "Op")
+						for i=2, math.min(#t[4], #vars), 1 do
+							r ..= ", "..lua({ t[2], vars[i], { tag = "Paren", values[i] } }, "Op")
+						end
+					end
+					if #destructuringVars > 0 then
+						local destructured = { rightOp = t[2] }
+						r ..= "local "..push("destructuring", destructured)..lua(destructuringVars, "_lhs")..pop("destructuring").." = "..lua(destructuringValues, "_lhs")
+						return r..DESTRUCTURING_ASSIGN(destructured, nil, true)
+					end
+					return r
+				else
+					local r  = ""
+					if #vars > 0 then
+						r ..= lua(vars, "_lhs").." = "..lua({ t[3], { tag = "Paren", values[1] }, vars[1] }, "Op")
+						for i=2, math.min(#t[4], #t[1]), 1 do
+							r ..= ", "..lua({ t[3], { tag = "Paren", values[i] }, vars[i] }, "Op")
+						end
+					end
+					if #destructuringVars > 0 then
+						local destructured = { leftOp = t[3] }
+						r ..= "local "..push("destructuring", destructured)..lua(destructuringVars, "_lhs")..pop("destructuring").." = "..lua(destructuringValues, "_lhs")
+						return r..DESTRUCTURING_ASSIGN(destructured, nil, true)
+					end
+					return r
+				end
+			else -- You are mad.
+				local r = ""
+				if #vars > 0 then
+					r ..= lua(vars, "_lhs").." = "..lua({ t[2], vars[1], { tag = "Op", t[4], { tag = "Paren", values[1] }, vars[1] } }, "Op")
+					for i=2, math.min(#t[5], #t[1]), 1 do
+						r ..= ", "..lua({ t[2], vars[i], { tag = "Op", t[4], { tag = "Paren", values[i] }, vars[i] } }, "Op")
+					end
+				end
+				if #destructuringVars > 0 then
+					local destructured = { rightOp = t[2], leftOp = t[4] }
+					r ..= "local "..push("destructuring", destructured)..lua(destructuringVars, "_lhs")..pop("destructuring").." = "..lua(destructuringValues, "_lhs")
+					return r..DESTRUCTURING_ASSIGN(destructured, nil, true)
+				end
+				return r
+			end
+		end,
+		-- While{ expr block }
+		While = (t)
+			local r = ""
+			local hasContinue = any(t[2], { "Continue" }, loop)
+			local lets = search({ t[1] }, { "LetExpr" })
+			if #lets > 0 then
+				r ..= "do"..indent()
+				for _, l in ipairs(lets) do
+					r ..= lua(l, "Let")..newline()
+				end
+			end
+			r ..= "while "..lua(t[1]).." do"..indent()
+			if #lets > 0 then
+				r ..= "do"..indent()
+			end
+			if hasContinue then
+				r ..= CONTINUE_START()
+			end
+			r ..= lua(t[2])
+			if hasContinue then
+				r ..= CONTINUE_STOP()
+			end
+			r ..= unindent().."end"
+			if #lets > 0 then
+				for _, l in ipairs(lets) do
+					r ..= newline()..lua(l, "Set")
+				end
+				r ..= unindent().."end"..unindent().."end"
+			end
+			return r
+		end,
+		-- Repeat{ block expr }
+		Repeat = (t)
+			local hasContinue = any(t[1], { "Continue" }, loop)
+			local r = "repeat"..indent()
+			if hasContinue then
+				r ..= CONTINUE_START()
+			end
+			r ..= lua(t[1])
+			if hasContinue then
+				r ..= CONTINUE_STOP()
+			end
+			r ..= unindent().."until "..lua(t[2])
+			return r
+		end,
+		-- If{ (lexpr block)+ block? }
+		If = (t)
+			local r = ""
+			local toClose = 0 -- blocks that need to be closed at the end of the if
+			local lets = search({ t[1] }, { "LetExpr" })
+			if #lets > 0 then
+				r ..= "do"..indent()
+				toClose += 1
+				for _, l in ipairs(lets) do
+					r ..= lua(l, "Let")..newline()
+				end
+			end
+			r ..= "if "..lua(t[1]).." then"..indent()..lua(t[2])..unindent()
+			for i=3, #t-1, 2 do
+				lets = search({ t[i] }, { "LetExpr" })
+				if #lets > 0 then
+					r ..= "else"..indent()
+					toClose += 1
+					for _, l in ipairs(lets) do
+						r ..= lua(l, "Let")..newline()
+					end
+				else
+					r ..= "else"
+				end
+				r ..= "if "..lua(t[i]).." then"..indent()..lua(t[i+1])..unindent()
+			end
+			if #t % 2 == 1 then
+				r ..= "else"..indent()..lua(t[#t])..unindent()
+			end
+			r ..= "end"
+			for i=1, toClose do
+				r ..= unindent().."end"
+			end
+			return r
+		end,
+		-- Fornum{ ident expr expr expr? block }
+		Fornum = (t)
+			local r = "for "..lua(t[1]).." = "..lua(t[2])..", "..lua(t[3])
+			if #t == 5 then
+				local hasContinue = any(t[5], { "Continue" }, loop)
+				r ..= ", "..lua(t[4]).." do"..indent()
+				if hasContinue then
+					r ..= CONTINUE_START()
+				end
+				r ..= lua(t[5])
+				if hasContinue then
+					r ..= CONTINUE_STOP()
+				end
+				return r..unindent().."end"
+			else
+				local hasContinue = any(t[4], { "Continue" }, loop)
+				r ..= " do"..indent()
+				if hasContinue then
+					r ..= CONTINUE_START()
+				end
+				r ..= lua(t[4])
+				if hasContinue then
+					r ..= CONTINUE_STOP()
+				end
+				return r..unindent().."end"
+			end
+		end,
+		-- Forin{ {ident+} {expr+} block }
+		Forin = (t)
+			local destructured = {}
+			local hasContinue = any(t[3], { "Continue" }, loop)
+			local r = "for "..push("destructuring", destructured)..lua(t[1], "_lhs")..pop("destructuring").." in "..lua(t[2], "_lhs").." do"..indent()
+			if hasContinue then
+				r ..= CONTINUE_START()
+			end
+			r ..= DESTRUCTURING_ASSIGN(destructured, true)..lua(t[3])
+			if hasContinue then
+				r ..= CONTINUE_STOP()
+			end
+			return r..unindent().."end"
+		end,
+		-- Local{ {attributeident+} {expr+}? }
+		Local = (t)
+			local destructured = {}
+			local r = "local "..push("destructuring", destructured)..lua(t[1], "_lhs")..pop("destructuring")
+			if t[2][1] then
+				r ..= " = "..lua(t[2], "_lhs")
+			end
+			return r..DESTRUCTURING_ASSIGN(destructured)
+		end,
+		-- Let{ {ident+} {expr+}? }
+		Let = (t)
+			local destructured = {}
+			local nameList = push("destructuring", destructured)..lua(t[1], "_lhs")..pop("destructuring")
+			local r = "local "..nameList
+			if t[2][1] then
+				if all(t[2], { "Nil", "Dots", "Boolean", "Number", "String" }) then -- predeclaration doesn't matter here
+					r ..= " = "..lua(t[2], "_lhs")
+				else
+					r ..= newline()..nameList.." = "..lua(t[2], "_lhs")
+				end
+			end
+			return r..DESTRUCTURING_ASSIGN(destructured)
+		end,
+		-- Localrec{ {ident} {expr} }
+		Localrec = (t)
+			return "local function "..lua(t[1][1])..lua(t[2][1], "_functionWithoutKeyword")
+		end,
+		-- Goto{ <string> }
+		Goto = (t)
+			return "goto "..lua(t, "Id")
+		end,
+		-- Label{ <string> }
+		Label = (t)
+			return "::"..lua(t, "Id").."::"
+		end,
+		-- Return{ <expr*> }
+		Return = (t)
+			local push = peek("push")
+			if push then
+				local r = ""
+				for _, val in ipairs(t) do
+					r ..= push.."[#"..push.."+1] = "..lua(val)..newline()
+				end
+				return r.."return "..UNPACK(push)
+			else
+				return "return "..lua(t, "_lhs")
+			end
+		end,
+		-- Push{ <expr*> }
+		Push = (t)
+			local var = assert(peek("push"), "no context given for push")
+			r = ""
+			for i=1, #t-1, 1 do
+				r ..= var.."[#"..var.."+1] = "..lua(t[i])..newline()
+			end
+			if t[#t] then
+				if t[#t].tag == "Call" then
+					r ..= APPEND(var, lua(t[#t]))
+				else
+					r ..= var.."[#"..var.."+1] = "..lua(t[#t])
+				end
+			end
+			return r
+		end,
+		-- Break
+		Break = ()
+			return "break"
+		end,
+		-- Continue
+		Continue = ()
+			return "goto "..var"continue"
+		end,
+		-- apply (below)
+
+		-- expr --
+
+		-- Nil
+		Nil = ()
+			return "nil"
+		end,
+		-- Dots
+		Dots = ()
+			local macroargs = peek("macroargs")
+			if macroargs and not nomacro.variables["..."] and macroargs["..."] then
+				nomacro.variables["..."] = true
+				local r = lua(macroargs["..."], "_lhs")
+				nomacro.variables["..."] = nil
+				return r
+			else
+				return "..."
+			end
+		end,
+		-- Boolean{ <boolean> }
+		Boolean = (t)
+			return tostring(t[1])
+		end,
+		-- Number{ <string> }
+		Number = (t)
+			return tostring(t[1])
+		end,
+		-- String{ <string> }
+		String = (t)
+			return "%q":format(t[1])
+		end,
+		-- Function{ { ( `ParPair{ Id expr } | `Id{ <string> } )* `Dots? } block }
+		_functionWithoutKeyword = (t)
+			local r = "("
+			local decl = {}
+			if t[1][1] then
+				if t[1][1].tag == "ParPair" then
+					local id = lua(t[1][1][1])
+					indentLevel += 1
+					table.insert(decl, "if "..id.." == nil then "..id.." = "..lua(t[1][1][2]).." end")
+					indentLevel -= 1
+					r ..= id
+				else
+					r ..= lua(t[1][1])
+				end
+				for i=2, #t[1], 1 do
+					if t[1][i].tag == "ParPair" then
+						local id = lua(t[1][i][1])
+						indentLevel += 1
+						table.insert(decl, "if "..id.." == nil then "..id.." = "..lua(t[1][i][2]).." end")
+						indentLevel -= 1
+						r ..= ", " ..id
+					else
+						r ..= ", "..lua(t[1][i])
+					end
+				end
+			end
+			r ..= ")"..indent()
+			for _, d in ipairs(decl) do
+				r ..= d..newline()
+			end
+			if t[2][#t[2]] and t[2][#t[2]].tag == "Push" then -- convert final push to return
+				t[2][#t[2]].tag = "Return"
+			end
+			local hasPush = any(t[2], { "Push" }, func)
+			if hasPush then
+				r ..= push("push", var"push").."local "..var"push".." = {}"..newline()
+			else
+				push("push", false) -- no push here (make sure higher push doesn't affect us)
+			end
+			r ..= lua(t[2])
+			if hasPush and (t[2][#t[2]] and t[2][#t[2]].tag ~= "Return") then -- add return only if needed
+				r ..= newline().."return "..UNPACK(var"push")
+			end
+			pop("push")
+			return r..unindent().."end"
+		end,
+		Function = (t)
+			return "function"..lua(t, "_functionWithoutKeyword")
+		end,
+		-- Table{ ( `Pair{ expr expr } | expr )* }
+		Pair = (t)
+			return "["..lua(t[1]).."] = "..lua(t[2])
+		end,
+		Table = (t)
+			if #t == 0 then
+				return "{}"
+			elseif #t == 1 then
+				return "{ "..lua(t, "_lhs").." }"
+			else
+				return "{"..indent()..lua(t, "_lhs", nil, true)..unindent().."}"
+			end
+		end,
+		-- TableCompr{ block }
+		TableCompr = (t)
+			return push("push", "self").."(function()"..indent().."local self = {}"..newline()..lua(t[1])..newline().."return self"..unindent().."end)()"..pop("push")
+		end,
+		-- Op{ opid expr expr? }
+		Op = (t)
+			local r
+			if #t == 2 then
+				if type(tags._opid[t[1]]) == "string" then
+					r = tags._opid[t[1]].." "..lua(t[2])
+				else
+					r = tags._opid[t[1]](t[2])
+				end
+			else
+				if type(tags._opid[t[1]]) == "string" then
+					r = lua(t[2]).." "..tags._opid[t[1]].." "..lua(t[3])
+				else
+					r = tags._opid[t[1]](t[2], t[3])
+				end
+			end
+			return r
+		end,
+		-- Paren{ expr }
+		Paren = (t)
+			return "("..lua(t[1])..")"
+		end,
+		-- MethodStub{ expr expr }
+		MethodStub = (t)
+			return "(function()"..indent() ..
+					"local "..var"object".." = "..lua(t[1])..newline()..
+					"local "..var"method".." = "..var"object".."."..lua(t[2], "Id")..newline() ..
+					"if "..var"method".." == nil then return nil end"..newline()..
+					"return function(...) return "..var"method".."("..var"object"..", ...) end"..unindent()..
+				"end)()"
+		end,
+		-- SafeMethodStub{ expr expr }
+		SafeMethodStub = (t)
+			return "(function()"..indent() ..
+					"local "..var"object".." = "..lua(t[1])..newline()..
+					"if "..var"object".." == nil then return nil end"..newline()..
+					"local "..var"method".." = "..var"object".."."..lua(t[2], "Id")..newline() ..
+					"if "..var"method".." == nil then return nil end"..newline()..
+					"return function(...) return "..var"method".."("..var"object"..", ...) end"..unindent()..
+				"end)()"
+		end,
+		-- statexpr (below)
+		-- apply (below)
+		-- lhs (below)
+
+		-- lexpr --
+		LetExpr = (t)
+			return lua(t[1][1])
+		end,
+
+		-- statexpr --
+		_statexpr = (t, stat)
+			local hasPush = any(t, { "Push" }, func)
+			local r = "(function()"..indent()
+			if hasPush then
+				r ..= push("push", var"push").."local "..var"push".." = {}"..newline()
+			else
+				push("push", false) -- no push here (make sure higher push don't affect us)
+			end
+			r ..= lua(t, stat)
+			if hasPush then
+				r ..= newline().."return "..UNPACK(var"push")
+			end
+			pop("push")
+			r ..= unindent().."end)()"
+			return r
+		end,
+		-- DoExpr{ stat* }
+		DoExpr = (t)
+			if t[#t].tag == "Push" then -- convert final push to return
+				t[#t].tag = "Return"
+			end
+			return lua(t, "_statexpr", "Do")
+		end,
+		-- WhileExpr{ expr block }
+		WhileExpr = (t)
+			return lua(t, "_statexpr", "While")
+		end,
+		-- RepeatExpr{ block expr }
+		RepeatExpr = (t)
+			return lua(t, "_statexpr", "Repeat")
+		end,
+		-- IfExpr{ (expr block)+ block? }
+		IfExpr = (t)
+			for i=2, #t do -- convert final pushes to returns
+				local block = t[i]
+				if block[#block] and block[#block].tag == "Push" then
+					block[#block].tag = "Return"
+				end
+			end
+			return lua(t, "_statexpr", "If")
+		end,
+		-- FornumExpr{ ident expr expr expr? block }
+		FornumExpr = (t)
+			return lua(t, "_statexpr", "Fornum")
+		end,
+		-- ForinExpr{ {ident+} {expr+} block }
+		ForinExpr = (t)
+			return lua(t, "_statexpr", "Forin")
+		end,
+
+		-- apply --
+
+		-- Call{ expr expr* }
+		Call = (t)
+			if t[1].tag == "String" or t[1].tag == "Table" then
+				return "("..lua(t[1])..")("..lua(t, "_lhs", 2)..")"
+			elseif t[1].tag == "Id" and not nomacro.functions[t[1][1]] and macros.functions[t[1][1]] then
+				local macro = macros.functions[t[1][1]]
+				local replacement = macro.replacement
+				local r
+				nomacro.functions[t[1][1]] = true
+				if type(replacement) == "function" then
+					local args = {}
+					for i=2, #t do
+						table.insert(args, lua(t[i]))
+					end
+					r = replacement(unpack(args))
+				else
+					local macroargs = util.merge(peek("macroargs"))
+					for i, arg in ipairs(macro.args) do
+						if arg.tag == "Dots" then
+							macroargs["..."] = [for j=i+1, #t do t[j] end]
+						elseif arg.tag == "Id" then
+							if t[i+1] == nil then
+								error("bad argument #%s to macro %s (value expected)":format(i, t[1][1]))
+							end
+							macroargs[arg[1]] = t[i+1]
+						else
+							error("unexpected argument type %s in macro %s":format(arg.tag, t[1][1]))
+						end
+					end
+					push("macroargs", macroargs)
+					r = lua(replacement)
+					pop("macroargs")
+				end
+				nomacro.functions[t[1][1]] = nil
+				return r
+			elseif t[1].tag == "MethodStub" then -- method call
+				if t[1][1].tag == "String" or t[1][1].tag == "Table" then
+					return "("..lua(t[1][1]).."):"..lua(t[1][2], "Id").."("..lua(t, "_lhs", 2)..")"
+				else
+					return lua(t[1][1])..":"..lua(t[1][2], "Id").."("..lua(t, "_lhs", 2)..")"
+				end
+			else
+				return lua(t[1]).."("..lua(t, "_lhs", 2)..")"
+			end
+		end,
+		-- SafeCall{ expr expr* }
+		SafeCall = (t)
+			if t[1].tag ~= "Id" then -- side effect possible, only evaluate each expr once (or already in a safe context)
+				return lua(t, "SafeIndex")
+			else -- no side effects possible
+				return "("..lua(t[1]).." ~= nil and "..lua(t[1]).."("..lua(t, "_lhs", 2)..") or nil)"
+			end
+		end,
+
+		-- lhs --
+		_lhs = (t, start=1, newlines)
+			local r
+			if t[start] then
+				r = lua(t[start])
+				for i=start+1, #t, 1 do
+					r ..= ","..(newlines and newline() or " ")..lua(t[i])
+				end
+			else
+				r = ""
+			end
+			return r
+		end,
+		-- Id{ <string> }
+		Id = (t)
+			local r = t[1]
+			local macroargs = peek("macroargs")
+			if not nomacro.variables[t[1]] then
+				nomacro.variables[t[1]] = true
+				if macroargs and macroargs[t[1]] then -- replace with macro argument
+					r = lua(macroargs[t[1]])
+				elseif macros.variables[t[1]] ~= nil then -- replace with macro variable
+					local macro = macros.variables[t[1]]
+					if type(macro) == "function" then
+						r = macro()
+					else
+						r = lua(macro)
+					end
+				end
+				nomacro.variables[t[1]] = nil
+			end
+			return r
+		end,
+		-- AttributeId{ <string> <string>? }
+		AttributeId = (t)
+			if t[2] then
+				return t[1] .. " <" .. t[2] .. ">"
+			else
+				return t[1]
+			end
+		end,
+		-- DestructuringId{ Id | Pair+ }
+		DestructuringId = (t)
+			if t.id then -- destructing already done before, use parent variable as id
+				return t.id
+			else
+				local d = assert(peek("destructuring"), "DestructuringId not in a destructurable assignement")
+				local vars = { id = tmp() }
+				for j=1, #t, 1 do
+					table.insert(vars, t[j])
+				end
+				table.insert(d, vars)
+				t.id = vars.id
+				return vars.id
+			end
+		end,
+		-- Index{ expr expr }
+		Index = (t)
+			if t[1].tag == "String" or t[1].tag == "Table" then
+				return "("..lua(t[1])..")["..lua(t[2]).."]"
+			else
+				return lua(t[1]).."["..lua(t[2]).."]"
+			end
+		end,
+		-- SafeIndex{ expr expr }
+		SafeIndex = (t)
+			if t[1].tag ~= "Id" then -- side effect possible, only evaluate each expr once (or already in a safe context)
+				local l = {} -- list of immediately chained safeindex, from deepest to nearest (to simply generated code)
+				while t.tag == "SafeIndex" or t.tag == "SafeCall" do
+					table.insert(l, 1, t)
+					t = t[1]
+				end
+				local r = "(function()"..indent().."local "..var"safe".." = "..lua(l[1][1])..newline() -- base expr
+				for _, e in ipairs(l) do
+					r ..= "if "..var"safe".." == nil then return nil end"..newline()
+					if e.tag == "SafeIndex" then
+						r ..= var"safe".." = "..var"safe".."["..lua(e[2]).."]"..newline()
+					else
+						r ..= var"safe".." = "..var"safe".."("..lua(e, "_lhs", 2)..")"..newline()
+					end
+				end
+				r ..= "return "..var"safe"..unindent().."end)()"
+				return r
+			else -- no side effects possible
+				return "("..lua(t[1]).." ~= nil and "..lua(t[1]).."["..lua(t[2]).."] or nil)"
+			end
+		end,
+
+		-- opid --
+		_opid = {
+			add = "+", sub = "-", mul = "*", div = "/",
+			idiv = "//", mod = "%", pow = "^", concat = "..",
+			band = "&", bor = "|", bxor = "~", shl = "<<", shr = ">>",
+			eq = "==", ne = "~=", lt = "<", gt = ">", le = "<=", ge = ">=",
+			["and"] = "and", ["or"] = "or", unm = "-", len = "#", bnot = "~", ["not"] = "not"
+		}
+	}, {
+		__index = (self, key)
+			error("don't know how to compile a "..tostring(key).." to "..targetName)
+		end
+	})
+
+	#placeholder("patch")
+
+	local code = lua(ast)..newline()
+	return requireStr..code
+end

compiler/luajit.can

@@ -1,13 +1,9 @@
+targetName = "LuaJIT"
+
 UNPACK = (list, i, j)
 	return "unpack(" .. list .. (i and (", " .. i .. (j and (", " .. j) or "")) or "") .. ")"
 end
-APPEND = (t, toAppend)
-	return "do" .. indent() .. "local a, p = { " .. toAppend .. " }, #" .. t .. "+1" .. newline() .. "for i=1, #a do" .. indent() .. t .. "[p] = a[i]" .. newline() .. "p = p + 1" .. unindent() .. "end" .. unindent() .. "end"
-end
 
-tags._opid.idiv = (left, right)
-	return "math.floor(" .. lua(left) .. " / " .. lua(right) .. ")"
-end
 tags._opid.band = (left, right)
 	addRequire("bit", "band", "band")
 	return var("band") .. "(" .. lua(left) .. ", " .. lua(right) .. ")"
@@ -33,8 +29,10 @@ tags._opid.bnot = (right)
 	return var("bnot") .. "(" .. lua(right) .. ")"
 end
 
+#placeholder("patch")
+
 #local patch = output
 #output = ""
-#import("compiler.lua53", { patch = patch, loadPackage = false })
+#import("compiler.lua52", { preprocessorEnv = { patch = patch }, loadPackage = false })
 
-return lua53
+return lua52

ideas.txt

@@ -6,9 +6,9 @@ To be implemented, theese need to:
     * are invalid vanilla Lua syntax.
     * are not ambigous with any vanilla Lua syntax.
 * be significantly useful compared to existing Candran/Lua code.
-* be useful without having to rewrite APIs specifically for Candran. Candran intends to make Lua easier, not supersede it.
+* be useful without having to rewrite APIs specifically for Candran. Candran intends to make Lua easier, not replace it.
 
-Example currently rejected ideas:
+Example rejected ideas:
 * Python-style function decorators (implemented in Candran 0.1.0):
     Useless 95% of the time because most Lua APIs applying something to a function are used like applySomething(someArg,func) instead of func=applySomething(someArg)(func).
     This could be adapted, but this will mean unecessary named functions in the environment and it will only works when the decorator returns the functions.
@@ -24,7 +24,7 @@ Please note that the following ideas are just random though and won't be necessa
 
 Feel free to open issues about ideas you find useful or want to improve. Otherwise, I will only implements them when I really need them or am bored.
 
-Actually, open issues for everything. I know of a couple persons using Candran but have no idea what they like and want with it.
+Actually, open issues for everything. I've heard that a couple of people use Candran, but have no idea what they like and want with it.
 
 * class keyword
 class Thing(parents...)
@@ -38,6 +38,8 @@ local a = new Thing()
 ->
 (TODO: define how classes work. May even use ClassCommons)
 
+Not very Lua-ey to impose how to make your classes?
+
 * try / except|catch / finally / else / other keywords
 try
     error("hey")
@@ -47,12 +49,7 @@ finally
     clean()
 end
 
-* Safe navigation operator
-local name = articles?[0].author?.name (?[] and ?. index opeators)
-or
-local zip = lottery.drawWinner?().address?.zipcode (expr? existence test suffix)
-
-See http://coffeescript.org/#existential-operator
+may be doable using if with assignement + pcall
 
 * static type checking
 local a = externalFunc() -- unknown
@@ -69,28 +66,43 @@ Will need to define stuff for a lot of Lua libraries (see the work already done
 
 While we're talking about static analysis, what about forking luacheck into candrancheck? Or at least make the line mapping work.
 
+Will require work.
+
 * array slicing
 local b = a[3:5:1]
 
+is it actually useful? even in python I rarely use it, apart from extracting a row or column for a matrix (and we don't have >1D arrays in Lua so...)
+
+OR return multiple value instead of a list?
+
+or list of incices:
+local a, b, c = l[1, 2, 3]
+
+how to handle hash table?
+local a, b, c = l.(a, b, c)
+or
+local a, b, c = l.a, .b, .c
+
+but
+local a, b, c = l[1], [2], [3]
+conflicts with table comprehension: change or use .[n]?
+
+or create some syntax akin to destructuring assignemnts but for numeric indexes:
+local [a, b, c] = t
+
 * Destructuring assignment
-local pos = { x = 5, y = 12 }
 
-local {x, y} = pos -- x, y = pos.x, pos.y
-local {a, b, x = x, y = y} = pos -- x, y = pos.x, pos.y, a = pos[1], b = pos[2]
-local {a, b, :x, :y} = pos -- shorthand for the above line. Or .x, .y
-local {:x.u} = pos OR {:x:u} OR {.x.u} -- u = pos.x.u
-local [x, y] = pos -- x, y = pos[0], pos[1]
-local x, y $= pos
-
-And in implicit assignments:
-for i, {x, y} in ipairs(positions) do
+Allow recursive destructing assignements
 
 * String interpolation
 Delimited by ``:
 `Hi, my name is ${@name}` -- -> "blaba" .. tostring(@name)
 Also allows multi-line with this maybe?
 
-* Other potential inspiration
-https://www.ruby-lang.org/fr/
+meh
 
-Well done, you're at the end of the file!
+* Other potential inspiration
+https://www.ruby-lang.org/
+
+* Lua 5.4 stuff
+const, to-be-closed variables: they're fun but as of now the syntax is awful

lib/cmdline.lua

@@ -1,126 +0,0 @@
--- started: 2008-04-12 by Shmuel Zeigerman
--- license: public domain
-
-local ipairs,pairs,setfenv,tonumber,loadstring,type =
-  ipairs,pairs,setfenv,tonumber,loadstring,type
-local tinsert, tconcat = table.insert, table.concat
-
-local function commonerror (msg)
-  return nil, ("[cmdline]: " .. msg)
-end
-
-local function argerror (msg, numarg)
-  msg = msg and (": " .. msg) or ""
-  return nil, ("[cmdline]: bad argument #" .. numarg .. msg)
-end
-
-local function iderror (numarg)
-  return argerror("ID not valid", numarg)
-end
-
-local function idcheck (id)
-  return id:match("^[%a_][%w_]*$") and true
-end
-
---[[------------------------------------------------------------------------
-Syntax:
-  t_out = getparam(t_in [,options] [,params])
-
-Parameters:
-  t_in:   table - list of string arguments to be processed in order
-          (usually it is the `arg' table created by the Lua interpreter).
-
-     * if an argument begins with $, the $ is skipped and the rest is inserted
-       into the array part of the output table.
-
-     * if an argument begins with -, the rest is a sequence of variables
-       (separated by commas or semicolons) that are all set to true;
-            example: -var1,var2  --> var1,var2 = true,true
-
-     * if an argument begins with !, the rest is a Lua chunk;
-            example: !a=(40+3)*5;b=20;name="John";window={w=600,h=480}
-
-     * if an argument contains =, then it is an assignment in the form
-       var1,...=value (no space is allowed around the =)
-         * if value begins with $, the $ is skipped, the rest is a string
-                 example: var1,var2=$         --> var1,var2 = "",""
-                 example: var1,var2=$125      --> var1,var2 = "125","125"
-                 example: var1,var2=$$125     --> var1,var2 = "$125","$125"
-         * if value is convertible to number, it is a number
-                 example: var1,var2=125       --> var1,var2 = 125,125
-         * if value is true of false, it is a boolean
-                 example: var1=false          --> var1 = false
-         * otherwise it is a string
-                 example: name=John           --> name = "John"
-
-     * if an argument neither begins with one of the special characters (-,!,$),
-       nor contains =, it is inserted as is into the array part of the output
-       table.
-
-  options (optional): a list of names of all command-line options and parameters
-     permitted in the application; used to check that each found option
-     is valid; no checks are done if not supplied.
-
-  params (optional): a list of names of all command-line parameters required
-     by the application; used to check that each required parameter is present;
-     no checks are done if not supplied.
-
-Returns:
-  On success: the output table, e.g. { [1]="./myfile.txt", name="John", age=40 }
-  On error: nil followed by error message string.
-
---]]------------------------------------------------------------------------
-return function(t_in, options, params)
-  local t_out = {}
-  for i,v in ipairs(t_in) do
-    local prefix, command = v:sub(1,1), v:sub(2)
-    if prefix == "$" then
-      tinsert(t_out, command)
-    elseif prefix == "-" then
-      for id in command:gmatch"[^,;]+" do
-        if not idcheck(id) then return iderror(i) end
-        t_out[id] = true
-      end
-    elseif prefix == "!" then
-      local f, err = loadstring(command)
-      if not f then return argerror(err, i) end
-      setfenv(f, t_out)()
-    elseif v:find("=") then
-      local ids, val = v:match("^([^=]+)%=(.*)") -- no space around =
-      if not ids then return argerror("invalid assignment syntax", i) end
-      if val == "false" then
-          val = false
-      elseif val == "true" then
-          val = true
-      else
-          val = val:sub(1,1)=="$" and val:sub(2) or tonumber(val) or val
-      end
-      for id in ids:gmatch"[^,;]+" do
-        if not idcheck(id) then return iderror(i) end
-        t_out[id] = val
-      end
-    else
-      tinsert(t_out, v)
-    end
-  end
-  if options then
-    local lookup, unknown = {}, {}
-    for _,v in ipairs(options) do lookup[v] = true end
-    for k,_ in pairs(t_out) do
-      if lookup[k]==nil and type(k)=="string" then tinsert(unknown, k) end
-    end
-    if #unknown > 0 then
-      return commonerror("unknown options: " .. tconcat(unknown, ", "))
-    end
-  end
-  if params then
-    local missing = {}
-    for _,v in ipairs(params) do
-      if t_out[v]==nil then tinsert(missing, v) end
-    end
-    if #missing > 0 then
-      return commonerror("missing parameters: " .. tconcat(missing, ", "))
-    end
-  end
-  return t_out
-end

lib/util.can

@@ -1,40 +0,0 @@
-local util = {}
-
-function util.search(modpath, exts={"can", "lua"})
-	for _, ext in ipairs(exts) do
-		for path in package.path:gmatch("[^;]+") do
-			local fpath = path:gsub("%.lua", "."..ext):gsub("%?", (modpath:gsub("%.", "/")))
-			local f = io.open(fpath)
-			if f then
-				f:close()
-				return fpath
-			end
-		end
-	end
-end
-
-function util.load(str, name, env)
-	if _VERSION == "Lua 5.1" then
-		local fn, err = loadstring(str, name)
-		if not fn then return fn, err end
-		return env ~= nil and setfenv(fn, env) or fn
-	else
-		if env then
-			return load(str, name, nil, env)
-		else
-			return load(str, name)
-		end
-	end
-end
-
-function util.merge(...)
-	local r = {}
-	for _, t in ipairs({...}) do
-		for k, v in pairs(t) do
-			r[k] = v
-		end
-	end
-	return r
-end
-
-return util

rockspec/candran-1.0.0-1.rockspec

@@ -2,29 +2,32 @@ rockspec_format = "3.0"
 
 package = "candran"
 
-version = "0.8.0-1"
+version = "1.0.0-1"
 
 description = {
 	summary = "A simple Lua dialect and preprocessor.",
 	detailed = [[
-		Candran is a dialect of the Lua 5.3 programming language which compiles to Lua 5.3 and Lua 5.1/LuaJit. It adds several useful syntax additions which aims to make Lua faster and easier to write, and a simple preprocessor.
-		Unlike Moonscript, Candran tries to stay close to the Lua syntax, and existing Lua code can run on Candran unmodified.
+		Candran is a dialect of the Lua 5.4 programming language which compiles to Lua 5.4, Lua 5.3, Lua 5.2, LuaJIT and Lua 5.1 compatible code. It adds several useful syntax additions which aims to make Lua faster and easier to write, and a simple preprocessor.
+		Unlike Moonscript, Candran tries to stay close to the Lua syntax, and existing Lua code should be able to run on Candran unmodified.
 	]],
 	license = "MIT",
 	homepage = "https://github.com/Reuh/candran",
 	issues_url = "https://github.com/Reuh/candran",
 	maintainer = "Étienne 'Reuh' Fildadut <fildadut@reuh.eu>",
-	labels = {"lpeg", "commandline"}
+	labels = { "lpeg", "commandline" }
 }
 
 source = {
 	url = "git://github.com/Reuh/candran",
-	tag = "v0.8.0"
+	tag = "v1.0.0"
 }
 
 dependencies = {
 	"lua >= 5.1",
-	"lpeglabel >= 1.5.0"
+	"lpeglabel >= 1.5.0",
+	"linenoise >= 0.9",
+	"luacheck >= 0.23.0",
+	"argparse >= 0.7.0"
 }
 
 build = {
@@ -33,6 +36,6 @@ build = {
 		candran = "candran.lua"
 	},
 	install = {
-		bin = { "bin/can", "bin/canc" }
+		bin = { "bin/can", "bin/canc", "bin/cancheck" }
 	}
 }

rockspec/candran-scm-1.rockspec

@@ -7,14 +7,14 @@ version = "scm-1"
 description = {
 	summary = "A simple Lua dialect and preprocessor.",
 	detailed = [[
-		Candran is a dialect of the Lua 5.3 programming language which compiles to Lua 5.3 and Lua 5.1/LuaJit. It adds several useful syntax additions which aims to make Lua faster and easier to write, and a simple preprocessor.
-		Unlike Moonscript, Candran tries to stay close to the Lua syntax, and existing Lua code can run on Candran unmodified.
+		Candran is a dialect of the Lua 5.3 programming language which compiles to Lua 5.3, LuaJIT and Lua 5.1 compatible code. It adds several useful syntax additions which aims to make Lua faster and easier to write, and a simple preprocessor.
+		Unlike Moonscript, Candran tries to stay close to the Lua syntax, and existing Lua code should be able to run on Candran unmodified.
 	]],
 	license = "MIT",
 	homepage = "https://github.com/Reuh/candran",
 	issues_url = "https://github.com/Reuh/candran",
 	maintainer = "Étienne 'Reuh' Fildadut <fildadut@reuh.eu>",
-	labels = {"lpeg", "commandline"}
+	labels = { "lpeg", "commandline" }
 }
 
 source = {
@@ -23,7 +23,10 @@ source = {
 
 dependencies = {
 	"lua >= 5.1",
-	"lpeglabel >= 1.5.0"
+	"lpeglabel >= 1.5.0",
+	"linenoise >= 0.9",
+	"luacheck >= 0.23.0",
+	"argparse >= 0.7.0"
 }
 
 build = {
@@ -32,6 +35,6 @@ build = {
 		candran = "candran.lua"
 	},
 	install = {
-		bin = { "bin/can", "bin/canc" }
+		bin = { "bin/can", "bin/canc", "bin/cancheck" }
 	}
 }