Reuh/candran
1
0
Fork 0
mirror of https://github.com/Reuh/candran.git synced 2025-10-27 17:59:30 +00:00
Code Releases Activity

Complete overhaul

Browse source 
- Changed name to Candran
- Do a real code parsing
    * Removed lexer.lua
    * Added LuaMinify
- Removed -- and ++ operators (see issue #2)
- Added decorators
- Preprocessor : renamed include to import and rawInclude to include
- Updated test.lua
- Updated table.lua
- Updated README.md
- Fixed tons of things
This commit is contained in:
Reuh 2015-02-14 20:23:39 +01:00
parent 18d3acf648
commit 1875ea31de
36 changed files with 11601 additions and 1582 deletions
Download patch file Download diff file Expand all files Collapse all files

198
README.md
View file

@ -1,144 +1,148 @@
Lune
====
Candran
=======
Candran is a dialect of the [Lua](http://www.lua.org) programming language which compiles to Lua. It adds a preprocessor and several useful syntax additions.
Lune is a simple [Lua](http://www.lua.org) dialect, which compile to normal Lua. It adds a preprocessor and some usefull syntax additions to the language, like += operators.
Lune code example :
Candran code example :
````lua
#local language = args.lang or "en"
local a = 5
a += 3
#if language == "fr" then
print("Le resultat est "..a)
#elseif language == "en" then
print("The result is "..a)
#end
#import("lib.thing")
#local debug = args.debug or false
local function debugArgs(func)
return function(...)
#if debug then
for _,arg in pairs({...}) do
print(arg, type(arg))
end
#end
return func(...)
end
end
@debugArgs
local function calculate()
local result = thing.do()
result += 25
return result
end
print(calculate())
````
This code will compile diffrently depending on the "lang" argument you pass to the compiler.
Syntax details
--------------
The language
------------
### Preprocessor
Before compiling, a preprocessor is run; it search the lines which start with a # and execute the Lune code after it.
Before compiling, Candran's preprocessor is run. It execute every line starting with a _#_ (ignoring whitespace) as Candran code.
For example,
````lua
#if args.lang == "fr" then
print("Ce programme a ete compile en francais")
print("Bonjour")
#else
print("This program was compiled in english")
print("Hello")
#end
````
Output ````print("Ce programme a ete compile en francais")```` or ````print("This program was compiled in english")```` depending of the "lang" argument.
Will output ````print("Bonjour")```` or ````print("Hello")```` depending of the "lang" argument passed to the preprocessor.
In the preprocessor, the following global variables are available :
* ````lune```` : the Lune library table
* ````output```` : the preprocessor output string
* ````include(filename)```` : a function which copy the contents of the file filename to the output and add some code so it is equivalent to :
The preprocessor has access to the following variables :
* ````candran```` : the Candran library table.
* ````output```` : the preprocessor output string.
* ````import(module[, autoRequire])```` : a function which import a module. This is equivalent to use _require(module)_ in the Candran code, except the module will be embedded in the current file. _autoRequire_ (boolean, default true) indicate if the module should be automaticaly loaded in a local variable or not. If true, the local variable will have the name of the module.
* ````include(filename)```` : a function which copy the contents of the file _filename_ to the output.
* ````print(...)```` : instead of writing to stdout, _print(...)_ will write to the preprocessor output. For example, ````#print("hello()")```` will output ````hello()````.
* ````args```` : the arguments table passed to the compiler. Example use : ````withDebugTools = args["debug"]````.
* and every standard Lua library.
````lua
filname = require("filename") or filename
````
### Syntax additions
After the preprocessor is run the Candran code is compiled to Lua. The Candran code adds the folowing syntax to Lua :
##### New assignment operators
* ````var += nb````
* ````var -= nb````
* ````var *= nb````
* ````var /= nb````
* ````var ^= nb````
* ````var %= nb````
* ````var ..= str````
except that the required code is actually embedded in the file.
* ````rawInclude(filename)```` : a function which copy the contents of the file filename to the output, whithout modifications
* ````print(...)```` : instead of writing to stdout, write to the preprocessor output; for example,
For example, a ````var += nb```` assignment will be compiled into ````var = var + nb````.
````lua
local foo = "hello"
#print("foo ..= ' lune')")
print(foo)
````
##### Decorators
Candran supports function decorators similar to Python. A decorator is a function returning another function, and allows easy function modification with this syntax :
````lua
@decorator
function name(...)
...
end
````
This is equivalent to :
````lua
function name(...)
...
end
name = decorator(name)
````
The decorators can be chained. Note that Candran allows this syntax for every variable, not only functions.
will output :
````lua
local foo = "hello"
foo = foo .. ' lune'
print(foo)
````
* ````args```` : the arguments table passed to the compiler. Example use :
````lua
argumentValue = args["argumentName"]
````
* And all the Lua standard libraries.
### Compiler
After the preprocessor, the compiler is run; it translate Lune syntax to Lua syntax. What is translated to what :
* ````var += nb```` > ````var = var + nb````
* ````var -= nb```` > ````var = var - nb````
* ````var *= nb```` > ````var = var * nb````
* ````var /= nb```` > ````var = var / nb````
* ````var ^= nb```` > ````var = var ^ nb````
* ````var %= nb```` > ````var = var % nb````
* ````var ..= str```` > ````var = var .. str````
* ````var++```` > ````var = var + 1````
* ````var--```` > ````var = var - 1````
Command-line usage
------------------
The library
-----------
### Command-line usage
The library can be used standalone :
lua lune.lua
* ````lua candran.lua````
Display a simple information text (version & basic command-line usage).
Display the information text (version and basic command-line usage).
lua lune.lua <input> [arguments]
* ````lua candran.lua <filename> [arguments]````
Output to stdout the Lune code compiled in Lua.
* arguments :
* input : input file name
* arguments : arguments to pass to the preprocessor (every argument is of type ````--<name> <value>````)
* example uses :
Output to stdout the _filename_ Candran file, preprocessed (with _arguments_) and compiled to Lua.
lua lune.lua foo.lune > foo.lua
_arguments_ is of type ````--somearg value --anotherarg anothervalue ...````.
compile foo.lune and write the result in foo.lua
* example uses :
lua lune.lua foo.lune --verbose true | lua
````lua candran.lua foo.can > foo.lua````
compile foo.lune with "verbose" set to true and execute it
preprocess and compile _foo.can_ and write the result in _foo.lua_.
Library usage
-------------
Lune can also be used as a normal Lua library. For example,
````lua candran.lua foo.can --verbose true | lua````
preprocess _foo.can_ with _verbose_ set to _true_, compile it and execute it.
### Library usage
Candran can also be used as a normal Lua library. For example,
````lua
local lune = require("lune")
local candran = require("candran")
local f = io.open("foo.lune")
local f = io.open("foo.can")
local contents = f:read("*a")
f:close()
local compiled = lune.make(contents, { lang = "fr" })
local compiled = candran.make(contents, { lang = "fr" })
load(compiled)()
````
will load Lune, read the file foo.lune, compile its contents with the argument "lang" set to "fr", and then execute the result.
Will load Candran, read the file _foo.can_, compile its contents with the argument _lang_ set to _"fr"_, and then execute the result.
Lune API :
* ````lune.VERSION```` : version string
* ````lune.syntax```` : syntax table used when compiling (TODO : need more explainations)
* ````lune.preprocess(code[, args])```` : return the Lune code preprocessed with args as argument table
* ````lune.compile(code)```` : return the Lune code compiled to Lua
* ````lune.make(code[, args])```` : return the Lune code preprocessed & compilled to Lua with args as argument table
The table returned by _require("candran")_ gives you access to :
* ````candran.VERSION```` : Candran's version string.
* ````candran.syntax```` : table containing all the syntax additions of Candran.
* ````candran.preprocess(code[, args])```` : return the Candran code _code_, preprocessed with _args_ as argument table.
* ````candran.compile(code)```` : return the Candran code compiled to Lua.
* ````candran.make(code[, args])```` : return the Candran code, preprocessed with _args_ as argument table and compilled to Lua.
Compiling Lune
--------------
Because the Lune compiler itself is written in Lune, you have to compile it with an already compiled version of Lune. This command will use the precompilled version in build/lune.lua to compile lune.lune and write the result in lune.lua :
### Compiling the library
The Candran library itself is written is Candran, so you have to compile it with an already compiled Candran library.
This command will use the precompilled version of this repository (build/candran.lua) to compile _candran.can_ and write the result in _candran.lua_ :
````
lua build/lune.lua lune.lune > lune.lua
lua build/candran.lua candran.can > candran.lua
````
You can then test your build :
You can then run the tests on your build :
````
cd tests
lua test.lua ../lune.lua
lua test.lua ../candran.lua
````

2764
build/candran.lua Normal file
View file

File diff suppressed because it is too large Load diff

784
build/lune.lua
View file

@ -1,784 +0,0 @@
#!/usr/bin/lua
--[[
Lune language & compiler by Thomas99.
LICENSE :
Copyright (c) 2014 Thomas99
This software is provided 'as-is', without any express or implied warranty.
In no event will the authors be held liable for any damages arising from the
use of this software.
Permission is granted to anyone to use this software for any purpose, including
commercial applications, and to alter it and redistribute it freely, subject
to the following restrictions:
1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software in a
product, an acknowledgment in the product documentation would be appreciated
but is not required.
2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.
]]
-- INCLUSION OF FILE "lib/lexer.lua" --
local function _()
--[[
This file is a part of Penlight (set of pure Lua libraries) - https://github.com/stevedonovan/Penlight
LICENSE :
Copyright (C) 2009 Steve Donovan, David Manura.
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in the
Software without restriction, including without limitation the rights to use, copy,
modify, merge, publish, distribute, sublicense, and/or sell copies of the Software,
and to permit persons to whom the Software is furnished to do so, subject to the
following conditions:
The above copyright notice and this permission notice shall be included in all copies
or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF
CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE
OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
]]
--- Lexical scanner for creating a sequence of tokens from text.
-- `lexer.scan(s)` returns an iterator over all tokens found in the
-- string `s`. This iterator returns two values, a token type string
-- (such as 'string' for quoted string, 'iden' for identifier) and the value of the
-- token.
--
-- Versions specialized for Lua and C are available; these also handle block comments
-- and classify keywords as 'keyword' tokens. For example:
--
-- > s = 'for i=1,n do'
-- > for t,v in lexer.lua(s) do print(t,v) end
-- keyword for
-- iden i
-- = =
-- number 1
-- , ,
-- iden n
-- keyword do
--
-- See the Guide for further @{06-data.md.Lexical_Scanning|discussion}
-- @module pl.lexer
local yield,wrap = coroutine.yield,coroutine.wrap
local strfind = string.find
local strsub = string.sub
local append = table.insert
local function assert_arg(idx,val,tp)
if type(val) ~= tp then
error("argument "..idx.." must be "..tp, 2)
end
end
local lexer = {}
local NUMBER1 = '^[%+%-]?%d+%.?%d*[eE][%+%-]?%d+'
local NUMBER2 = '^[%+%-]?%d+%.?%d*'
local NUMBER3 = '^0x[%da-fA-F]+'
local NUMBER4 = '^%d+%.?%d*[eE][%+%-]?%d+'
local NUMBER5 = '^%d+%.?%d*'
local IDEN = '^[%a_][%w_]*'
local WSPACE = '^%s+'
local STRING0 = [[^(['\"]).-\\%1]]
local STRING1 = [[^(['\"]).-[^\]%1]]
local STRING3 = "^((['\"])%2)" -- empty string
local PREPRO = '^#.-[^\\]\n'
local plain_matches,lua_matches,cpp_matches,lua_keyword,cpp_keyword
local function tdump(tok)
return yield(tok,tok)
end
local function ndump(tok,options)
if options and options.number then
tok = tonumber(tok)
end
return yield("number",tok)
end
-- regular strings, single or double quotes; usually we want them
-- without the quotes
local function sdump(tok,options)
if options and options.string then
tok = tok:sub(2,-2)
end
return yield("string",tok)
end
-- long Lua strings need extra work to get rid of the quotes
local function sdump_l(tok,options)
if options and options.string then
tok = tok:sub(3,-3)
end
return yield("string",tok)
end
local function chdump(tok,options)
if options and options.string then
tok = tok:sub(2,-2)
end
return yield("char",tok)
end
local function cdump(tok)
return yield('comment',tok)
end
local function wsdump (tok)
return yield("space",tok)
end
local function pdump (tok)
return yield('prepro',tok)
end
local function plain_vdump(tok)
return yield("iden",tok)
end
local function lua_vdump(tok)
if lua_keyword[tok] then
return yield("keyword",tok)
else
return yield("iden",tok)
end
end
local function cpp_vdump(tok)
if cpp_keyword[tok] then
return yield("keyword",tok)
else
return yield("iden",tok)
end
end
--- create a plain token iterator from a string or file-like object.
-- @param s the string
-- @param matches an optional match table (set of pattern-action pairs)
-- @param filter a table of token types to exclude, by default {space=true}
-- @param options a table of options; by default, {number=true,string=true},
-- which means convert numbers and strip string quotes.
function lexer.scan (s,matches,filter,options)
--assert_arg(1,s,'string')
local file = type(s) ~= 'string' and s
filter = filter or {space=true}
options = options or {number=true,string=true}
if filter then
if filter.space then filter[wsdump] = true end
if filter.comments then
filter[cdump] = true
end
end
if not matches then
if not plain_matches then
plain_matches = {
{WSPACE,wsdump},
{NUMBER3,ndump},
{IDEN,plain_vdump},
{NUMBER1,ndump},
{NUMBER2,ndump},
{STRING3,sdump},
{STRING0,sdump},
{STRING1,sdump},
{'^.',tdump}
}
end
matches = plain_matches
end
local function lex ()
local i1,i2,idx,res1,res2,tok,pat,fun,capt
local line = 1
if file then s = file:read()..'\n' end
local sz = #s
local idx = 1
--print('sz',sz)
while true do
for _,m in ipairs(matches) do
pat = m[1]
fun = m[2]
i1,i2 = strfind(s,pat,idx)
if i1 then
tok = strsub(s,i1,i2)
idx = i2 + 1
if not (filter and filter[fun]) then
lexer.finished = idx > sz
res1,res2 = fun(tok,options)
end
if res1 then
local tp = type(res1)
-- insert a token list
if tp=='table' then
yield('','')
for _,t in ipairs(res1) do
yield(t[1],t[2])
end
elseif tp == 'string' then -- or search up to some special pattern
i1,i2 = strfind(s,res1,idx)
if i1 then
tok = strsub(s,i1,i2)
idx = i2 + 1
yield('',tok)
else
yield('','')
idx = sz + 1
end
--if idx > sz then return end
else
yield(line,idx)
end
end
if idx > sz then
if file then
--repeat -- next non-empty line
line = line + 1
s = file:read()
if not s then return end
--until not s:match '^%s*$'
s = s .. '\n'
idx ,sz = 1,#s
break
else
return
end
else break end
end
end
end
end
return wrap(lex)
end
local function isstring (s)
return type(s) == 'string'
end
--- insert tokens into a stream.
-- @param tok a token stream
-- @param a1 a string is the type, a table is a token list and
-- a function is assumed to be a token-like iterator (returns type & value)
-- @param a2 a string is the value
function lexer.insert (tok,a1,a2)
if not a1 then return end
local ts
if isstring(a1) and isstring(a2) then
ts = {{a1,a2}}
elseif type(a1) == 'function' then
ts = {}
for t,v in a1() do
append(ts,{t,v})
end
else
ts = a1
end
tok(ts)
end
--- get everything in a stream upto a newline.
-- @param tok a token stream
-- @return a string
function lexer.getline (tok)
local t,v = tok('.-\n')
return v
end
--- get current line number. <br>
-- Only available if the input source is a file-like object.
-- @param tok a token stream
-- @return the line number and current column
function lexer.lineno (tok)
return tok(0)
end
--- get the rest of the stream.
-- @param tok a token stream
-- @return a string
function lexer.getrest (tok)
local t,v = tok('.+')
return v
end
--- get the Lua keywords as a set-like table.
-- So <code>res["and"]</code> etc would be <code>true</code>.
-- @return a table
function lexer.get_keywords ()
if not lua_keyword then
lua_keyword = {
["and"] = true, ["break"] = true, ["do"] = true,
["else"] = true, ["elseif"] = true, ["end"] = true,
["false"] = true, ["for"] = true, ["function"] = true,
["if"] = true, ["in"] = true, ["local"] = true, ["nil"] = true,
["not"] = true, ["or"] = true, ["repeat"] = true,
["return"] = true, ["then"] = true, ["true"] = true,
["until"] = true, ["while"] = true
}
end
return lua_keyword
end
--- create a Lua token iterator from a string or file-like object.
-- Will return the token type and value.
-- @param s the string
-- @param filter a table of token types to exclude, by default {space=true,comments=true}
-- @param options a table of options; by default, {number=true,string=true},
-- which means convert numbers and strip string quotes.
function lexer.lua(s,filter,options)
filter = filter or {space=true,comments=true}
lexer.get_keywords()
if not lua_matches then
lua_matches = {
{WSPACE,wsdump},
{NUMBER3,ndump},
{IDEN,lua_vdump},
{NUMBER4,ndump},
{NUMBER5,ndump},
{STRING3,sdump},
{STRING0,sdump},
{STRING1,sdump},
{'^%-%-%[%[.-%]%]',cdump},
{'^%-%-.-\n',cdump},
{'^%[%[.-%]%]',sdump_l},
{'^==',tdump},
{'^~=',tdump},
{'^<=',tdump},
{'^>=',tdump},
{'^%.%.%.',tdump},
{'^%.%.',tdump},
{'^.',tdump}
}
end
return lexer.scan(s,lua_matches,filter,options)
end
--- create a C/C++ token iterator from a string or file-like object.
-- Will return the token type type and value.
-- @param s the string
-- @param filter a table of token types to exclude, by default {space=true,comments=true}
-- @param options a table of options; by default, {number=true,string=true},
-- which means convert numbers and strip string quotes.
function lexer.cpp(s,filter,options)
filter = filter or {comments=true}
if not cpp_keyword then
cpp_keyword = {
["class"] = true, ["break"] = true, ["do"] = true, ["sizeof"] = true,
["else"] = true, ["continue"] = true, ["struct"] = true,
["false"] = true, ["for"] = true, ["public"] = true, ["void"] = true,
["private"] = true, ["protected"] = true, ["goto"] = true,
["if"] = true, ["static"] = true, ["const"] = true, ["typedef"] = true,
["enum"] = true, ["char"] = true, ["int"] = true, ["bool"] = true,
["long"] = true, ["float"] = true, ["true"] = true, ["delete"] = true,
["double"] = true, ["while"] = true, ["new"] = true,
["namespace"] = true, ["try"] = true, ["catch"] = true,
["switch"] = true, ["case"] = true, ["extern"] = true,
["return"] = true,["default"] = true,['unsigned'] = true,['signed'] = true,
["union"] = true, ["volatile"] = true, ["register"] = true,["short"] = true,
}
end
if not cpp_matches then
cpp_matches = {
{WSPACE,wsdump},
{PREPRO,pdump},
{NUMBER3,ndump},
{IDEN,cpp_vdump},
{NUMBER4,ndump},
{NUMBER5,ndump},
{STRING3,sdump},
{STRING1,chdump},
{'^//.-\n',cdump},
{'^/%*.-%*/',cdump},
{'^==',tdump},
{'^!=',tdump},
{'^<=',tdump},
{'^>=',tdump},
{'^->',tdump},
{'^&&',tdump},
{'^||',tdump},
{'^%+%+',tdump},
{'^%-%-',tdump},
{'^%+=',tdump},
{'^%-=',tdump},
{'^%*=',tdump},
{'^/=',tdump},
{'^|=',tdump},
{'^%^=',tdump},
{'^::',tdump},
{'^.',tdump}
}
end
return lexer.scan(s,cpp_matches,filter,options)
end
--- get a list of parameters separated by a delimiter from a stream.
-- @param tok the token stream
-- @param endtoken end of list (default ')'). Can be '\n'
-- @param delim separator (default ',')
-- @return a list of token lists.
function lexer.get_separated_list(tok,endtoken,delim)
endtoken = endtoken or ')'
delim = delim or ','
local parm_values = {}
local level = 1 -- used to count ( and )
local tl = {}
local function tappend (tl,t,val)
val = val or t
append(tl,{t,val})
end
local is_end
if endtoken == '\n' then
is_end = function(t,val)
return t == 'space' and val:find '\n'
end
else
is_end = function (t)
return t == endtoken
end
end
local token,value
while true do
token,value=tok()
if not token then return nil,'EOS' end -- end of stream is an error!
if is_end(token,value) and level == 1 then
append(parm_values,tl)
break
elseif token == '(' then
level = level + 1
tappend(tl,'(')
elseif token == ')' then
level = level - 1
if level == 0 then -- finished with parm list
append(parm_values,tl)
break
else
tappend(tl,')')
end
elseif token == delim and level == 1 then
append(parm_values,tl) -- a new parm
tl = {}
else
tappend(tl,token,value)
end
end
return parm_values,{token,value}
end
--- get the next non-space token from the stream.
-- @param tok the token stream.
function lexer.skipws (tok)
local t,v = tok()
while t == 'space' do
t,v = tok()
end
return t,v
end
local skipws = lexer.skipws
--- get the next token, which must be of the expected type.
-- Throws an error if this type does not match!
-- @param tok the token stream
-- @param expected_type the token type
-- @param no_skip_ws whether we should skip whitespace
function lexer.expecting (tok,expected_type,no_skip_ws)
assert_arg(1,tok,'function')
assert_arg(2,expected_type,'string')
local t,v
if no_skip_ws then
t,v = tok()
else
t,v = skipws(tok)
end
if t ~= expected_type then error ("expecting "..expected_type,2) end
return v
end
return lexer
end
local lexer = _() or lexer
-- END OF INCLUDSION OF FILE "lib/lexer.lua" --
-- INCLUSION OF FILE "lib/table.lua" --
local function _()
--[[
Lua table utilities by Thomas99.
LICENSE :
Copyright (c) 2014 Thomas99
This software is provided 'as-is', without any express or implied warranty.
In no event will the authors be held liable for any damages arising from the
use of this software.
Permission is granted to anyone to use this software for any purpose, including
commercial applications, and to alter it and redistribute it freely, subject
to the following restrictions:
1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software in a
product, an acknowledgment in the product documentation would be appreciated
but is not required.
2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.
]]
-- Copie récursivement la table t dans la table dest (ou une table vide si non précisé) et la retourne
-- replace (false) : indique si oui ou non, les clefs existant déjà dans dest doivent être écrasées par celles de t
-- metatable (true) : copier ou non également les metatables
-- filter (function) : filtre, si retourne true copie l'objet, sinon ne le copie pas
-- Note : les metatables des objets ne sont jamais re-copiées (mais référence à la place), car sinon lors de la copie
-- la classe de ces objets changera pour une nouvelle classe, et c'est pas pratique :p
function table.copy(t, dest, replace, metatable, filter, copied)
local copied = copied or {}
local replace = replace or false
local metatable = (metatable==nil or metatable) and true
local filter = filter or function(name, source, destination) return true end
if type(t) ~= "table" then
return t
elseif copied[t] then -- si la table a déjà été copiée
return copied[t]
end
local dest = dest or {} -- la copie
copied[t] = dest -- on marque la table comme copiée
for k, v in pairs(t) do
if filter(k, t, dest) then
if replace then
dest[k] = table.copy(v, dest[k], replace, metatable, filter, copied)
else
if dest[k] == nil or type(v) == "table" then -- si la clef n'existe pas déjà dans dest ou si c'est une table à copier
dest[k] = table.copy(v, dest[k], replace, metatable, filter, copied)
end
end
end
end
-- copie des metatables
if metatable then
if t.__classe then
setmetatable(dest, getmetatable(t))
else
setmetatable(dest, table.copy(getmetatable(t), getmetatable(dest), replace, filter))
end
end
return dest
end
-- retourne true si value est dans la table
function table.isIn(table, value)
for _,v in pairs(table) do
if v == value then
return true
end
end
return false
end
-- retourne true si la clé key est dans la table
function table.hasKey(table, key)
for k,_ in pairs(table) do
if k == key then
return true
end
end
return false
end
-- retourne la longueur exacte d'une table (fonctionne sur les tables à clef)
function table.len(t)
local len=0
for i in pairs(t) do
len=len+1
end
return len
end
-- Sépare str en éléments séparés par le pattern et retourne une table
function string.split(str, pattern)
local t = {}
local pos = 0
for i,p in string.gmatch(str, "(.-)"..pattern.."()") do
table.insert(t, i)
pos = p
end
table.insert(t, str:sub(pos))
return t
end
end
local table = _() or table
-- END OF INCLUDSION OF FILE "lib/table.lua" --
local lune = {}
lune.VERSION = "0.0.1"
lune.syntax = {
affectation = { ["+"] = "= %s +", ["-"] = "= %s -", ["*"] = "= %s *", ["/"] = "= %s /",
["^"] = "= %s ^", ["%"] = "= %s %%", [".."] = "= %s .." },
incrementation = { ["+"] = " = %s + 1" , ["-"] = " = %s - 1" },
}
-- Preprocessor
function lune.preprocess(input, args)
-- generate preprocessor
local preprocessor = "return function()\n"
local lines = {}
for line in (input.."\n"):gmatch("(.-)\n") do
table.insert(lines, line)
if line:sub(1,1) == "#" then
-- exclude shebang
if not (line:sub(1,2) == "#!" and #lines ==1) then
preprocessor = preprocessor .. line:sub(2) .. "\n"
else
preprocessor = preprocessor .. "output ..= lines[" .. #lines .. "] .. \"\\n\"\n"
end
else
preprocessor = preprocessor .. "output ..= lines[" .. #lines .. "] .. \"\\n\"\n"
end
end
preprocessor = preprocessor .. "return output\nend"
-- make preprocessor environement
local env = table.copy(_G)
env.lune = lune
env.output = ""
env.include = function(file)
local f = io.open(file)
if not f then error("can't open the file to include") end
local filename = file:match("([^%/%\\]-)%.[^%.]-$")
env.output = env.output ..
"-- INCLUSION OF FILE \""..file.."\" --\n"..
"local function _()\n"..
f:read("*a").."\n"..
"end\n"..
"local "..filename.." = _() or "..filename.."\n"..
"-- END OF INCLUDSION OF FILE \""..file.."\" --\n"
f:close()
end
env.rawInclude = function(file)
local f = io.open(file)
if not f then error("can't open the file to raw include") end
env.output = env.output .. f:read("*a").."\n"
f:close()
end
env.print = function(...)
env.output = env.output .. table.concat({...}, "\t") .. "\n"
end
env.args = args or {}
env.lines = lines
-- load preprocessor
local preprocess, err = load(lune.compile(preprocessor), "Preprocessor", nil, env)
if not preprocess then error("Error while creating preprocessor :\n" .. err) end
-- execute preprocessor
local success, output = pcall(preprocess())
if not success then error("Error while preprocessing file :\n" .. output .. "\nWith preprocessor : \n" .. preprocessor) end
return output
end
-- Compiler
function lune.compile(input)
local output = ""
local last = {}
for t,v in lexer.lua(input, {}, {}) do
local toInsert = v
-- affectation
if t == "=" then
if table.hasKey(lune.syntax.affectation, last.token) then
toInsert = string.format(lune.syntax.affectation[last.token], last.varName)
output = output:sub(1, -1 -#last.token) -- remove token before =
end
end
-- self-incrementation
if table.hasKey(lune.syntax.incrementation, t) and t == last.token then
toInsert = string.format(lune.syntax.incrementation[last.token], last.varName)
output = output:sub(1, -#last.token*2) -- remove token ++/--
end
-- reconstitude full variable name (ex : ith.game.camera)
if t == "iden" then
if last.token == "." then
last.varName = last.varName .. "." .. v
else
last.varName = v
end
end
last[t] = v
last.token = t
last.value = v
output = output .. toInsert
end
return output
end
-- Preprocess & compile
function lune.make(code, args)
local preprocessed = lune.preprocess(code, args or {})
local output = lune.compile(preprocessed)
return output
end
-- Standalone mode
if debug.getinfo(3) == nil and arg then
-- Check args
if #arg < 1 then
print("Lune version "..lune.VERSION.." by Thomas99")
print("Command-line usage :")
print("lua lune.lua <filename> [preprocessor arguments]")
return lune
end
-- Parse args
local inputFilePath = arg[1]
local args = {}
-- Parse compilation args
for i=2, #arg, 1 do
if arg[i]:sub(1,2) == "--" then
args[arg[i]:sub(3)] = arg[i+1]
i = i +1 -- skip argument value
end
end
-- Open & read input file
local inputFile, err = io.open(inputFilePath, "r")
if not inputFile then error("Error while opening input file : "..err) end
local input = inputFile:read("*a")
inputFile:close()
-- End
print(lune.make(input, args))
end
return lune

170
candran.can Normal file
View file

@ -0,0 +1,170 @@
--[[
Candran language, preprocessor and compiler by Thomas99.
LICENSE :
Copyright (c) 2015 Thomas99
This software is provided 'as-is', without any express or implied warranty.
In no event will the authors be held liable for any damages arising from the
use of this software.
Permission is granted to anyone to use this software for any purpose, including
commercial applications, and to alter it and redistribute it freely, subject
to the following restrictions:
1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software in a
product, an acknowledgment in the product documentation would be appreciated
but is not required.
2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.
]]
local candran = {
VERSION = "0.1.0",
syntax = {
assignment = { "+=", "-=", "*=", "/=", "^=", "%=", "..=" },
decorator = "@"
}
}
package.loaded["candran"] = candran
#import("lib.table")
#import("lib.LuaMinify.Util")
#import("lib.LuaMinify.Scope")
#import("lib.LuaMinify.ParseCandran")
#import("lib.LuaMinify.FormatIdentityCandran")
-- Preprocessor
function candran.preprocess(input, args)
-- generate preprocessor
local preprocessor = "return function()\n"
local lines = {}
for line in (input.."\n"):gmatch("(.-)\n") do
table.insert(lines, line)
-- preprocessor instructions (exclude shebang)
if line:match("^%s*#") and not (line:match("^#!") and #lines == 1) then
preprocessor ..= line:gsub("^%s*#", "") .. "\n"
else
preprocessor ..= "output ..= lines[" .. #lines .. "] .. \"\\n\"\n"
end
end
preprocessor ..= "return output\nend"
-- make preprocessor environement
local env = table.copy(_G)
env.candran = candran
env.output = ""
env.import = function(modpath, autoRequire)
local autoRequire = (autoRequire == nil) or autoRequire
-- get module filepath
local filepath
for _,search in ipairs(package.searchers) do
local loader, path = search(modpath)
if type(loader) == "function" and type(path) == "string" then
filepath = path
break
end
end
if not filepath then error("No module named \""..modpath.."\"") end
-- open module file
local f = io.open(filepath)
if not f then error("Can't open the module file to import") end
local modcontent = f:read("*a")
f:close()
-- get module name (ex: module name of path.to.module is module)
local modname = modpath:match("[^%.]+$")
--
env.output ..=
"-- IMPORT OF MODULE \""..modpath.."\" --\n"..
"local function _()\n"..
modcontent.."\n"..
"end\n"..
(autoRequire and "local "..modname.." = _() or "..modname.."\n" or "").. -- auto require
"package.loaded[\""..modpath.."\"] = "..(autoRequire and modname or "_()").." or true\n".. -- add to package.loaded
"-- END OF IMPORT OF MODULE \""..modpath.."\" --\n"
end
env.include = function(file)
local f = io.open(file)
if not f then error("Can't open the file to include") end
env.output ..= f:read("*a").."\n"
f:close()
end
env.print = function(...)
env.output ..= table.concat({...}, "\t") .. "\n"
end
env.args = args or {}
env.lines = lines
-- load preprocessor
local preprocess, err = load(candran.compile(preprocessor), "Preprocessor", nil, env)
if not preprocess then error("Error while creating preprocessor :\n" .. err) end
-- execute preprocessor
local success, output = pcall(preprocess())
if not success then error("Error while preprocessing file :\n" .. output .. "\nWith preprocessor : \n" .. preprocessor) end
return output
end
-- Compiler
function candran.compile(input)
local parse = require("lib.LuaMinify.ParseCandran")
local format = require("lib.LuaMinify.FormatIdentityCandran")
local success, ast = parse.ParseLua(input)
if not success then error("Error while parsing the file :\n"..tostring(ast)) end
local success, output = format(ast)
if not success then error("Error while formating the file :\n"..tostring(output)) end
return output
end
-- Preprocess & compile
function candran.make(code, args)
local preprocessed = candran.preprocess(code, args or {})
local output = candran.compile(preprocessed)
return output
end
-- Standalone mode
if debug.getinfo(3) == nil and arg then
-- Check args
if #arg < 1 then
print("Candran version "..candran.VERSION.." by Thomas99")
print("Command-line usage :")
print("lua candran.lua <filename> [preprocessor arguments]")
return candran
end
-- Parse args
local inputFilepath = arg[1]
local args = {}
for i=2, #arg, 1 do
if arg[i]:sub(1,2) == "--" then
args[arg[i]:sub(3)] = arg[i+1]
i = i + 1 -- skip argument value
end
end
-- Open & read input file
local inputFile, err = io.open(inputFilepath, "r")
if not inputFile then error("Error while opening input file : "..err) end
local input = inputFile:read("*a")
inputFile:close()
-- Make
print(candran.make(input, args))
end
return candran

121
lib/LuaMinify/CommandLineBeautify.lua Normal file
View file

@ -0,0 +1,121 @@
--
-- beautify
--
-- A command line utility for beautifying lua source code using the beautifier.
--
local util = require'Util'
local Parser = require'ParseLua'
local Format_Beautify = require'FormatBeautiful'
local ParseLua = Parser.ParseLua
local PrintTable = util.PrintTable
local function splitFilename(name)
--table.foreach(arg, print)
if name:find(".") then
local p, ext = name:match("()%.([^%.]*)$")
if p and ext then
if #ext == 0 then
return name, nil
else
local filename = name:sub(1,p-1)
return filename, ext
end
else
return name, nil
end
else
return name, nil
end
end
if #arg == 1 then
local name, ext = splitFilename(arg[1])
local outname = name.."_formatted"
if ext then outname = outname.."."..ext end
--
local inf = io.open(arg[1], 'r')
if not inf then
print("Failed to open '"..arg[1].."' for reading")
return
end
--
local sourceText = inf:read('*all')
inf:close()
--
local st, ast = ParseLua(sourceText)
if not st then
--we failed to parse the file, show why
print(ast)
return
end
--
local outf = io.open(outname, 'w')
if not outf then
print("Failed to open '"..outname.."' for writing")
return
end
--
outf:write(Format_Beautify(ast))
outf:close()
--
print("Beautification complete")
elseif #arg == 2 then
--keep the user from accidentally overwriting their non-minified file with
if arg[1]:find("_formatted") then
print("Did you mix up the argument order?\n"..
"Current command will beautify '"..arg[1].."' and overwrite '"..arg[2].."' with the results")
while true do
io.write("Confirm (yes/no): ")
local msg = io.read('*line')
if msg == 'yes' then
break
elseif msg == 'no' then
return
end
end
end
local inf = io.open(arg[1], 'r')
if not inf then
print("Failed to open '"..arg[1].."' for reading")
return
end
--
local sourceText = inf:read('*all')
inf:close()
--
local st, ast = ParseLua(sourceText)
if not st then
--we failed to parse the file, show why
print(ast)
return
end
--
if arg[1] == arg[2] then
print("Are you SURE you want to overwrite the source file with a beautified version?\n"..
"You will be UNABLE to get the original source back!")
while true do
io.write("Confirm (yes/no): ")
local msg = io.read('*line')
if msg == 'yes' then
break
elseif msg == 'no' then
return
end
end
end
local outf = io.open(arg[2], 'w')
if not outf then
print("Failed to open '"..arg[2].."' for writing")
return
end
--
outf:write(Format_Beautify(ast))
outf:close()
--
print("Beautification complete")
else
print("Invalid arguments!\nUsage: lua CommandLineLuaBeautify.lua source_file [destination_file]")
end

47
lib/LuaMinify/CommandLineLiveBeautify.lua Normal file
View file

@ -0,0 +1,47 @@
--
-- beautify.interactive
--
-- For testing: Lets you enter lines of text to be beautified to verify the
-- correctness of their implementation.
--
local util = require'Util'
local Parser = require'ParseLua'
local Format_Beautify = require'FormatBeautiful'
local ParseLua = Parser.ParseLua
local PrintTable = util.PrintTable
while true do
io.write('> ')
local line = io.read('*line')
local fileFrom, fileTo = line:match("^file (.*) (.*)")
if fileFrom and fileTo then
local file = io.open(fileFrom, 'r')
local fileTo = io.open(fileTo, 'w')
if file and fileTo then
local st, ast = ParseLua(file:read('*all'))
if st then
fileTo:write(Format_Beautify(ast)..'\n')
io.write("Beautification Complete\n")
else
io.write(""..tostring(ast).."\n")
end
file:close()
fileTo:close()
else
io.write("File does not exist\n")
end
else
local st, ast = ParseLua(line)
if st then
io.write("====== AST =======\n")
io.write(PrintTable(ast)..'\n')
io.write("==== BEAUTIFIED ====\n")
io.write(Format_Beautify(ast))
io.write("==================\n")
else
io.write(""..tostring(ast).."\n")
end
end
end

47
lib/LuaMinify/CommandLineLiveMinify.lua Normal file
View file

@ -0,0 +1,47 @@
--
-- CommandLineLiveMinify.lua
--
-- For testing: Lets you enter lines of text to be minified to verify the
-- correctness of their implementation.
--
local util = require'Util'
local Parser = require'ParseLua'
local Format_Mini = require'FormatMini'
local ParseLua = Parser.ParseLua
local PrintTable = util.PrintTable
while true do
io.write('> ')
local line = io.read('*line')
local fileFrom, fileTo = line:match("^file (.*) (.*)")
if fileFrom and fileTo then
local file = io.open(fileFrom, 'r')
local fileTo = io.open(fileTo, 'w')
if file and fileTo then
local st, ast = ParseLua(file:read('*all'))
if st then
fileTo:write(Format_Mini(ast)..'\n')
io.write("Minification Complete\n")
else
io.write(""..tostring(ast).."\n")
end
file:close()
fileTo:close()
else
io.write("File does not exist\n")
end
else
local st, ast = ParseLua(line)
if st then
io.write("====== AST =======\n")
io.write(PrintTable(ast)..'\n')
io.write("==== MINIFIED ====\n")
io.write(Format_Mini(ast)..'\n')
io.write("==================\n")
else
io.write(""..tostring(ast).."\n")
end
end
end

122
lib/LuaMinify/CommandLineMinify.lua Normal file
View file

@ -0,0 +1,122 @@
--
-- CommandlineMinify.lua
--
-- A command line utility for minifying lua source code using the minifier.
--
local util = require'Util'
local Parser = require'ParseLua'
local Format_Mini = require'FormatMini'
local ParseLua = Parser.ParseLua
local PrintTable = util.PrintTable
local function splitFilename(name)
table.foreach(arg, print)
if name:find(".") then
local p, ext = name:match("()%.([^%.]*)$")
if p and ext then
if #ext == 0 then
return name, nil
else
local filename = name:sub(1,p-1)
return filename, ext
end
else
return name, nil
end
else
return name, nil
end
end
if #arg == 1 then
local name, ext = splitFilename(arg[1])
local outname = name.."_min"
if ext then outname = outname.."."..ext end
--
local inf = io.open(arg[1], 'r')
if not inf then
print("Failed to open `"..arg[1].."` for reading")
return
end
--
local sourceText = inf:read('*all')
inf:close()
--
local st, ast = ParseLua(sourceText)
if not st then
--we failed to parse the file, show why
print(ast)
return
end
--
local outf = io.open(outname, 'w')
if not outf then
print("Failed to open `"..outname.."` for writing")
return
end
--
outf:write(Format_Mini(ast))
outf:close()
--
print("Minification complete")
elseif #arg == 2 then
--keep the user from accidentally overwriting their non-minified file with
if arg[1]:find("_min") then
print("Did you mix up the argument order?\n"..
"Current command will minify `"..arg[1].."` and OVERWRITE `"..arg[2].."` with the results")
while true do
io.write("Confirm (yes/cancel): ")
local msg = io.read('*line')
if msg == 'yes' then
break
elseif msg == 'cancel' then
return
end
end
end
local inf = io.open(arg[1], 'r')
if not inf then
print("Failed to open `"..arg[1].."` for reading")
return
end
--
local sourceText = inf:read('*all')
inf:close()
--
local st, ast = ParseLua(sourceText)
if not st then
--we failed to parse the file, show why
print(ast)
return
end
--
if arg[1] == arg[2] then
print("Are you SURE you want to overwrite the source file with a minified version?\n"..
"You will be UNABLE to get the original source back!")
while true do
io.write("Confirm (yes/cancel): ")
local msg = io.read('*line')
if msg == 'yes' then
break
elseif msg == 'cancel' then
return
end
end
end
local outf = io.open(arg[2], 'w')
if not outf then
print("Failed to open `"..arg[2].."` for writing")
return
end
--
outf:write(Format_Mini(ast))
outf:close()
--
print("Minification complete")
else
print("Invalid arguments, Usage:\nLuaMinify source [destination]")
end

347
lib/LuaMinify/FormatBeautiful.lua Normal file
View file

@ -0,0 +1,347 @@
--
-- Beautifier
--
-- Returns a beautified version of the code, including comments
--
local parser = require"ParseLua"
local ParseLua = parser.ParseLua
local util = require'Util'
local lookupify = util.lookupify
local LowerChars = lookupify{'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i',
'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r',
's', 't', 'u', 'v', 'w', 'x', 'y', 'z'}
local UpperChars = lookupify{'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I',
'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R',
'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z'}
local Digits = lookupify{'0', '1', '2', '3', '4', '5', '6', '7', '8', '9'}
local function Format_Beautify(ast)
local formatStatlist, formatExpr
local indent = 0
local EOL = "\n"
local function getIndentation()
return string.rep(" ", indent)
end
local function joinStatementsSafe(a, b, sep)
sep = sep or ''
local aa, bb = a:sub(-1,-1), b:sub(1,1)
if UpperChars[aa] or LowerChars[aa] or aa == '_' then
if not (UpperChars[bb] or LowerChars[bb] or bb == '_' or Digits[bb]) then
--bb is a symbol, can join without sep
return a .. b
elseif bb == '(' then
--prevent ambiguous syntax
return a..sep..b
else
return a..sep..b
end
elseif Digits[aa] then
if bb == '(' then
--can join statements directly
return a..b
else
return a..sep..b
end
elseif aa == '' then
return a..b
else
if bb == '(' then
--don't want to accidentally call last statement, can't join directly
return a..sep..b
else
return a..b
end
end
end
formatExpr = function(expr)
local out = string.rep('(', expr.ParenCount or 0)
if expr.AstType == 'VarExpr' then
if expr.Variable then
out = out .. expr.Variable.Name
else
out = out .. expr.Name
end
elseif expr.AstType == 'NumberExpr' then
out = out..expr.Value.Data
elseif expr.AstType == 'StringExpr' then
out = out..expr.Value.Data
elseif expr.AstType == 'BooleanExpr' then
out = out..tostring(expr.Value)
elseif expr.AstType == 'NilExpr' then
out = joinStatementsSafe(out, "nil")
elseif expr.AstType == 'BinopExpr' then
out = joinStatementsSafe(out, formatExpr(expr.Lhs)) .. " "
out = joinStatementsSafe(out, expr.Op) .. " "
out = joinStatementsSafe(out, formatExpr(expr.Rhs))
elseif expr.AstType == 'UnopExpr' then
out = joinStatementsSafe(out, expr.Op) .. (#expr.Op ~= 1 and " " or "")
out = joinStatementsSafe(out, formatExpr(expr.Rhs))
elseif expr.AstType == 'DotsExpr' then
out = out.."..."
elseif expr.AstType == 'CallExpr' then
out = out..formatExpr(expr.Base)
out = out.."("
for i = 1, #expr.Arguments do
out = out..formatExpr(expr.Arguments[i])
if i ~= #expr.Arguments then
out = out..", "
end
end
out = out..")"
elseif expr.AstType == 'TableCallExpr' then
out = out..formatExpr(expr.Base) .. " "
out = out..formatExpr(expr.Arguments[1])
elseif expr.AstType == 'StringCallExpr' then
out = out..formatExpr(expr.Base) .. " "
out = out..expr.Arguments[1].Data
elseif expr.AstType == 'IndexExpr' then
out = out..formatExpr(expr.Base).."["..formatExpr(expr.Index).."]"
elseif expr.AstType == 'MemberExpr' then
out = out..formatExpr(expr.Base)..expr.Indexer..expr.Ident.Data
elseif expr.AstType == 'Function' then
-- anonymous function
out = out.."function("
if #expr.Arguments > 0 then
for i = 1, #expr.Arguments do
out = out..expr.Arguments[i].Name
if i ~= #expr.Arguments then
out = out..", "
elseif expr.VarArg then
out = out..", ..."
end
end
elseif expr.VarArg then
out = out.."..."
end
out = out..")" .. EOL
indent = indent + 1
out = joinStatementsSafe(out, formatStatlist(expr.Body))
indent = indent - 1
out = joinStatementsSafe(out, getIndentation() .. "end")
elseif expr.AstType == 'ConstructorExpr' then
out = out.."{ "
for i = 1, #expr.EntryList do
local entry = expr.EntryList[i]
if entry.Type == 'Key' then
out = out.."["..formatExpr(entry.Key).."] = "..formatExpr(entry.Value)
elseif entry.Type == 'Value' then
out = out..formatExpr(entry.Value)
elseif entry.Type == 'KeyString' then
out = out..entry.Key.." = "..formatExpr(entry.Value)
end
if i ~= #expr.EntryList then
out = out..", "
end
end
out = out.." }"
elseif expr.AstType == 'Parentheses' then
out = out.."("..formatExpr(expr.Inner)..")"
end
out = out..string.rep(')', expr.ParenCount or 0)
return out
end
local formatStatement = function(statement)
local out = ""
if statement.AstType == 'AssignmentStatement' then
out = getIndentation()
for i = 1, #statement.Lhs do
out = out..formatExpr(statement.Lhs[i])
if i ~= #statement.Lhs then
out = out..", "
end
end
if #statement.Rhs > 0 then
out = out.." = "
for i = 1, #statement.Rhs do
out = out..formatExpr(statement.Rhs[i])
if i ~= #statement.Rhs then
out = out..", "
end
end
end
elseif statement.AstType == 'CallStatement' then
out = getIndentation() .. formatExpr(statement.Expression)
elseif statement.AstType == 'LocalStatement' then
out = getIndentation() .. out.."local "
for i = 1, #statement.LocalList do
out = out..statement.LocalList[i].Name
if i ~= #statement.LocalList then
out = out..", "
end
end
if #statement.InitList > 0 then
out = out.." = "
for i = 1, #statement.InitList do
out = out..formatExpr(statement.InitList[i])
if i ~= #statement.InitList then
out = out..", "
end
end
end
elseif statement.AstType == 'IfStatement' then
out = getIndentation() .. joinStatementsSafe("if ", formatExpr(statement.Clauses[1].Condition))
out = joinStatementsSafe(out, " then") .. EOL
indent = indent + 1
out = joinStatementsSafe(out, formatStatlist(statement.Clauses[1].Body))
indent = indent - 1
for i = 2, #statement.Clauses do
local st = statement.Clauses[i]
if st.Condition then
out = getIndentation() .. joinStatementsSafe(out, getIndentation() .. "elseif ")
out = joinStatementsSafe(out, formatExpr(st.Condition))
out = joinStatementsSafe(out, " then") .. EOL
else
out = joinStatementsSafe(out, getIndentation() .. "else") .. EOL
end
indent = indent + 1
out = joinStatementsSafe(out, formatStatlist(st.Body))
indent = indent - 1
end
out = joinStatementsSafe(out, getIndentation() .. "end") .. EOL
elseif statement.AstType == 'WhileStatement' then
out = getIndentation() .. joinStatementsSafe("while ", formatExpr(statement.Condition))
out = joinStatementsSafe(out, " do") .. EOL
indent = indent + 1
out = joinStatementsSafe(out, formatStatlist(statement.Body))
indent = indent - 1
out = joinStatementsSafe(out, getIndentation() .. "end") .. EOL
elseif statement.AstType == 'DoStatement' then
out = getIndentation() .. joinStatementsSafe(out, "do") .. EOL
indent = indent + 1
out = joinStatementsSafe(out, formatStatlist(statement.Body))
indent = indent - 1
out = joinStatementsSafe(out, getIndentation() .. "end") .. EOL
elseif statement.AstType == 'ReturnStatement' then
out = getIndentation() .. "return "
for i = 1, #statement.Arguments do
out = joinStatementsSafe(out, formatExpr(statement.Arguments[i]))
if i ~= #statement.Arguments then
out = out..", "
end
end
elseif statement.AstType == 'BreakStatement' then
out = getIndentation() .. "break"
elseif statement.AstType == 'RepeatStatement' then
out = getIndentation() .. "repeat" .. EOL
indent = indent + 1
out = joinStatementsSafe(out, formatStatlist(statement.Body))
indent = indent - 1
out = joinStatementsSafe(out, getIndentation() .. "until ")
out = joinStatementsSafe(out, formatExpr(statement.Condition)) .. EOL
elseif statement.AstType == 'Function' then
if statement.IsLocal then
out = "local "
end
out = joinStatementsSafe(out, "function ")
out = getIndentation() .. out
if statement.IsLocal then
out = out..statement.Name.Name
else
out = out..formatExpr(statement.Name)
end
out = out.."("
if #statement.Arguments > 0 then
for i = 1, #statement.Arguments do
out = out..statement.Arguments[i].Name
if i ~= #statement.Arguments then
out = out..", "
elseif statement.VarArg then
out = out..",..."
end
end
elseif statement.VarArg then
out = out.."..."
end
out = out..")" .. EOL
indent = indent + 1
out = joinStatementsSafe(out, formatStatlist(statement.Body))
indent = indent - 1
out = joinStatementsSafe(out, getIndentation() .. "end") .. EOL
elseif statement.AstType == 'GenericForStatement' then
out = getIndentation() .. "for "
for i = 1, #statement.VariableList do
out = out..statement.VariableList[i].Name
if i ~= #statement.VariableList then
out = out..", "
end
end
out = out.." in "
for i = 1, #statement.Generators do
out = joinStatementsSafe(out, formatExpr(statement.Generators[i]))
if i ~= #statement.Generators then
out = joinStatementsSafe(out, ', ')
end
end
out = joinStatementsSafe(out, " do") .. EOL
indent = indent + 1
out = joinStatementsSafe(out, formatStatlist(statement.Body))
indent = indent - 1
out = joinStatementsSafe(out, getIndentation() .. "end") .. EOL
elseif statement.AstType == 'NumericForStatement' then
out = getIndentation() .. "for "
out = out..statement.Variable.Name.." = "
out = out..formatExpr(statement.Start)..", "..formatExpr(statement.End)
if statement.Step then
out = out..", "..formatExpr(statement.Step)
end
out = joinStatementsSafe(out, " do") .. EOL
indent = indent + 1
out = joinStatementsSafe(out, formatStatlist(statement.Body))
indent = indent - 1
out = joinStatementsSafe(out, getIndentation() .. "end") .. EOL
elseif statement.AstType == 'LabelStatement' then
out = getIndentation() .. "::" .. statement.Label .. "::" .. EOL
elseif statement.AstType == 'GotoStatement' then
out = getIndentation() .. "goto " .. statement.Label .. EOL
elseif statement.AstType == 'Comment' then
if statement.CommentType == 'Shebang' then
out = getIndentation() .. statement.Data
--out = out .. EOL
elseif statement.CommentType == 'Comment' then
out = getIndentation() .. statement.Data
--out = out .. EOL
elseif statement.CommentType == 'LongComment' then
out = getIndentation() .. statement.Data
--out = out .. EOL
end
elseif statement.AstType == 'Eof' then
-- Ignore
else
print("Unknown AST Type: ", statement.AstType)
end
return out
end
formatStatlist = function(statList)
local out = ''
for _, stat in pairs(statList.Body) do
out = joinStatementsSafe(out, formatStatement(stat) .. EOL)
end
return out
end
return formatStatlist(ast)
end
return Format_Beautify

440
lib/LuaMinify/FormatIdentity.lua Normal file
View file

@ -0,0 +1,440 @@
require'strict'
require'ParseLua'
local util = require'Util'
local function debug_printf(...)
--[[
util.printf(...)
--]]
end
--
-- FormatIdentity.lua
--
-- Returns the exact source code that was used to create an AST, preserving all
-- comments and whitespace.
-- This can be used to get back a Lua source after renaming some variables in
-- an AST.
--
local function Format_Identity(ast)
local out = {
rope = {}, -- List of strings
line = 1,
char = 1,
appendStr = function(self, str)
table.insert(self.rope, str)
local lines = util.splitLines(str)
if #lines == 1 then
self.char = self.char + #str
else
self.line = self.line + #lines - 1
local lastLine = lines[#lines]
self.char = #lastLine
end
end,
appendToken = function(self, token)
self:appendWhite(token)
--[*[
--debug_printf("appendToken(%q)", token.Data)
local data = token.Data
local lines = util.splitLines(data)
while self.line + #lines < token.Line do
print("Inserting extra line")
self.str = self.str .. '\n'
self.line = self.line + 1
self.char = 1
end
--]]
self:appendStr(token.Data)
end,
appendTokens = function(self, tokens)
for _,token in ipairs(tokens) do
self:appendToken( token )
end
end,
appendWhite = function(self, token)
if token.LeadingWhite then
self:appendTokens( token.LeadingWhite )
--self.str = self.str .. ' '
end
end
}
local formatStatlist, formatExpr;
formatExpr = function(expr)
local tok_it = 1
local function appendNextToken(str)
local tok = expr.Tokens[tok_it];
if str and tok.Data ~= str then
error("Expected token '" .. str .. "'. Tokens: " .. util.PrintTable(expr.Tokens))
end
out:appendToken( tok )
tok_it = tok_it + 1
end
local function appendToken(token)
out:appendToken( token )
tok_it = tok_it + 1
end
local function appendWhite()
local tok = expr.Tokens[tok_it];
if not tok then error(util.PrintTable(expr)) end
out:appendWhite( tok )
tok_it = tok_it + 1
end
local function appendStr(str)
appendWhite()
out:appendStr(str)
end
local function peek()
if tok_it < #expr.Tokens then
return expr.Tokens[tok_it].Data
end
end
local function appendComma(mandatory, seperators)
if true then
seperators = seperators or { "," }
seperators = util.lookupify( seperators )
if not mandatory and not seperators[peek()] then
return
end
assert(seperators[peek()], "Missing comma or semicolon")
appendNextToken()
else
local p = peek()
if p == "," or p == ";" then
appendNextToken()
end
end
end
debug_printf("formatExpr(%s) at line %i", expr.AstType, expr.Tokens[1] and expr.Tokens[1].Line or -1)
if expr.AstType == 'VarExpr' then
if expr.Variable then
appendStr( expr.Variable.Name )
else
appendStr( expr.Name )
end
elseif expr.AstType == 'NumberExpr' then
appendToken( expr.Value )
elseif expr.AstType == 'StringExpr' then
appendToken( expr.Value )
elseif expr.AstType == 'BooleanExpr' then
appendNextToken( expr.Value and "true" or "false" )
elseif expr.AstType == 'NilExpr' then
appendNextToken( "nil" )
elseif expr.AstType == 'BinopExpr' then
formatExpr(expr.Lhs)
appendStr( expr.Op )
formatExpr(expr.Rhs)
elseif expr.AstType == 'UnopExpr' then
appendStr( expr.Op )
formatExpr(expr.Rhs)
elseif expr.AstType == 'DotsExpr' then
appendNextToken( "..." )
elseif expr.AstType == 'CallExpr' then
formatExpr(expr.Base)
appendNextToken( "(" )
for i,arg in ipairs( expr.Arguments ) do
formatExpr(arg)
appendComma( i ~= #expr.Arguments )
end
appendNextToken( ")" )
elseif expr.AstType == 'TableCallExpr' then
formatExpr( expr.Base )
formatExpr( expr.Arguments[1] )
elseif expr.AstType == 'StringCallExpr' then
formatExpr(expr.Base)
appendToken( expr.Arguments[1] )
elseif expr.AstType == 'IndexExpr' then
formatExpr(expr.Base)
appendNextToken( "[" )
formatExpr(expr.Index)
appendNextToken( "]" )
elseif expr.AstType == 'MemberExpr' then
formatExpr(expr.Base)
appendNextToken() -- . or :
appendToken(expr.Ident)
elseif expr.AstType == 'Function' then
-- anonymous function
appendNextToken( "function" )
appendNextToken( "(" )
if #expr.Arguments > 0 then
for i = 1, #expr.Arguments do
appendStr( expr.Arguments[i].Name )
if i ~= #expr.Arguments then
appendNextToken(",")
elseif expr.VarArg then
appendNextToken(",")
appendNextToken("...")
end
end
elseif expr.VarArg then
appendNextToken("...")
end
appendNextToken(")")
formatStatlist(expr.Body)
appendNextToken("end")
elseif expr.AstType == 'ConstructorExpr' then
appendNextToken( "{" )
for i = 1, #expr.EntryList do
local entry = expr.EntryList[i]
if entry.Type == 'Key' then
appendNextToken( "[" )
formatExpr(entry.Key)
appendNextToken( "]" )
appendNextToken( "=" )
formatExpr(entry.Value)
elseif entry.Type == 'Value' then
formatExpr(entry.Value)
elseif entry.Type == 'KeyString' then
appendStr(entry.Key)
appendNextToken( "=" )
formatExpr(entry.Value)
end
appendComma( i ~= #expr.EntryList, { ",", ";" } )
end
appendNextToken( "}" )
elseif expr.AstType == 'Parentheses' then
appendNextToken( "(" )
formatExpr(expr.Inner)
appendNextToken( ")" )
else
print("Unknown AST Type: ", statement.AstType)
end
assert(tok_it == #expr.Tokens + 1)
debug_printf("/formatExpr")
end
local formatStatement = function(statement)
local tok_it = 1
local function appendNextToken(str)
local tok = statement.Tokens[tok_it];
assert(tok, string.format("Not enough tokens for %q. First token at %i:%i",
str, statement.Tokens[1].Line, statement.Tokens[1].Char))
assert(tok.Data == str,
string.format('Expected token %q, got %q', str, tok.Data))
out:appendToken( tok )
tok_it = tok_it + 1
end
local function appendToken(token)
out:appendToken( str )
tok_it = tok_it + 1
end
local function appendWhite()
local tok = statement.Tokens[tok_it];
out:appendWhite( tok )
tok_it = tok_it + 1
end
local function appendStr(str)
appendWhite()
out:appendStr(str)
end
local function appendComma(mandatory)
if mandatory
or (tok_it < #statement.Tokens and statement.Tokens[tok_it].Data == ",") then
appendNextToken( "," )
end
end
debug_printf("")
debug_printf(string.format("formatStatement(%s) at line %i", statement.AstType, statement.Tokens[1] and statement.Tokens[1].Line or -1))
if statement.AstType == 'AssignmentStatement' then
for i,v in ipairs(statement.Lhs) do
formatExpr(v)
appendComma( i ~= #statement.Lhs )
end
if #statement.Rhs > 0 then
appendNextToken( "=" )
for i,v in ipairs(statement.Rhs) do
formatExpr(v)
appendComma( i ~= #statement.Rhs )
end
end
elseif statement.AstType == 'CallStatement' then
formatExpr(statement.Expression)
elseif statement.AstType == 'LocalStatement' then
appendNextToken( "local" )
for i = 1, #statement.LocalList do
appendStr( statement.LocalList[i].Name )
appendComma( i ~= #statement.LocalList )
end
if #statement.InitList > 0 then
appendNextToken( "=" )
for i = 1, #statement.InitList do
formatExpr(statement.InitList[i])
appendComma( i ~= #statement.InitList )
end
end
elseif statement.AstType == 'IfStatement' then
appendNextToken( "if" )
formatExpr( statement.Clauses[1].Condition )
appendNextToken( "then" )
formatStatlist( statement.Clauses[1].Body )
for i = 2, #statement.Clauses do
local st = statement.Clauses[i]
if st.Condition then
appendNextToken( "elseif" )
formatExpr(st.Condition)
appendNextToken( "then" )
else
appendNextToken( "else" )
end
formatStatlist(st.Body)
end
appendNextToken( "end" )
elseif statement.AstType == 'WhileStatement' then
appendNextToken( "while" )
formatExpr(statement.Condition)
appendNextToken( "do" )
formatStatlist(statement.Body)
appendNextToken( "end" )
elseif statement.AstType == 'DoStatement' then
appendNextToken( "do" )
formatStatlist(statement.Body)
appendNextToken( "end" )
elseif statement.AstType == 'ReturnStatement' then
appendNextToken( "return" )
for i = 1, #statement.Arguments do
formatExpr(statement.Arguments[i])
appendComma( i ~= #statement.Arguments )
end
elseif statement.AstType == 'BreakStatement' then
appendNextToken( "break" )
elseif statement.AstType == 'RepeatStatement' then
appendNextToken( "repeat" )
formatStatlist(statement.Body)
appendNextToken( "until" )
formatExpr(statement.Condition)
elseif statement.AstType == 'Function' then
--print(util.PrintTable(statement))
if statement.IsLocal then
appendNextToken( "local" )
end
appendNextToken( "function" )
if statement.IsLocal then
appendStr(statement.Name.Name)
else
formatExpr(statement.Name)
end
appendNextToken( "(" )
if #statement.Arguments > 0 then
for i = 1, #statement.Arguments do
appendStr( statement.Arguments[i].Name )
appendComma( i ~= #statement.Arguments or statement.VarArg )
if i == #statement.Arguments and statement.VarArg then
appendNextToken( "..." )
end
end
elseif statement.VarArg then
appendNextToken( "..." )
end
appendNextToken( ")" )
formatStatlist(statement.Body)
appendNextToken( "end" )
elseif statement.AstType == 'GenericForStatement' then
appendNextToken( "for" )
for i = 1, #statement.VariableList do
appendStr( statement.VariableList[i].Name )
appendComma( i ~= #statement.VariableList )
end
appendNextToken( "in" )
for i = 1, #statement.Generators do
formatExpr(statement.Generators[i])
appendComma( i ~= #statement.Generators )
end
appendNextToken( "do" )
formatStatlist(statement.Body)
appendNextToken( "end" )
elseif statement.AstType == 'NumericForStatement' then
appendNextToken( "for" )
appendStr( statement.Variable.Name )
appendNextToken( "=" )
formatExpr(statement.Start)
appendNextToken( "," )
formatExpr(statement.End)
if statement.Step then
appendNextToken( "," )
formatExpr(statement.Step)
end
appendNextToken( "do" )
formatStatlist(statement.Body)
appendNextToken( "end" )
elseif statement.AstType == 'LabelStatement' then
appendNextToken( "::" )
appendStr( statement.Label )
appendNextToken( "::" )
elseif statement.AstType == 'GotoStatement' then
appendNextToken( "goto" )
appendStr( statement.Label )
elseif statement.AstType == 'Eof' then
appendWhite()
else
print("Unknown AST Type: ", statement.AstType)
end
if statement.Semicolon then
appendNextToken(";")
end
assert(tok_it == #statement.Tokens + 1)
debug_printf("/formatStatment")
end
formatStatlist = function(statList)
for _, stat in ipairs(statList.Body) do
formatStatement(stat)
end
end
formatStatlist(ast)
return true, table.concat(out.rope)
end
return Format_Identity

601
lib/LuaMinify/FormatIdentityCandran.lua Normal file
View file

@ -0,0 +1,601 @@
--
-- CANDRAN
-- Based on the FormatIdentity.lua of LuaMinify.
-- Modified by Thomas99 to format valid Lua code from Candran AST.
--
-- Modified parts are marked with "-- CANDRAN" comments.
--
--[[
This file is part of LuaMinify by stravant (https://github.com/stravant/LuaMinify).
LICENSE :
The MIT License (MIT)
Copyright (c) 2012-2013
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
]]
--require'strict' -- CANDRAN : comment, useless here
-- CANDRAN : add Candran syntaxic additions
local candran = require("candran").syntax
require'lib.LuaMinify.ParseCandran'
local util = require'lib.LuaMinify.Util'
local function debug_printf(...)
--[[
util.printf(...)
--]]
end
--
-- FormatIdentity.lua
--
-- Returns the exact source code that was used to create an AST, preserving all
-- comments and whitespace.
-- This can be used to get back a Lua source after renaming some variables in
-- an AST.
--
local function Format_Identity(ast)
local out = {
rope = {}, -- List of strings
line = 1,
char = 1,
appendStr = function(self, str)
table.insert(self.rope, str)
local lines = util.splitLines(str)
if #lines == 1 then
self.char = self.char + #str
else
self.line = self.line + #lines - 1
local lastLine = lines[#lines]
self.char = #lastLine
end
end,
-- CANDRAN : options
appendToken = function(self, token, options)
local options = options or {} -- CANDRAN
self:appendWhite(token, options)
--[*[
--debug_printf("appendToken(%q)", token.Data)
local data = token.Data
local lines = util.splitLines(data)
while self.line + #lines < token.Line do
if not options.no_newline then self:appendStr('\n') end -- CANDRAN : options
self.line = self.line + 1
self.char = 1
end
--]]
if options.no_newline then data = data:gsub("[\n\r]*", "") end -- CANDRAN : options
if options.no_leading_white then data = data:gsub("^%s+", "") end
self:appendStr(data)
end,
-- CANDRAN : options
appendTokens = function(self, tokens, options)
for _,token in ipairs(tokens) do
self:appendToken( token, options ) -- CANDRAN : options
end
end,
-- CANDRAN : options
appendWhite = function(self, token, options)
if token.LeadingWhite then
self:appendTokens( token.LeadingWhite, options ) -- CANDRAN : options
--self.str = self.str .. ' '
end
end
}
local formatStatlist, formatExpr, formatStatement;
-- CANDRAN : added options argument
-- CANDRAN : options = { no_newline = false, no_leading_white = false }
formatExpr = function(expr, options)
local options = options or {} -- CANDRAN
local tok_it = 1
local function appendNextToken(str)
local tok = expr.Tokens[tok_it];
if str and tok.Data ~= str then
error("Expected token '" .. str .. "'. Tokens: " .. util.PrintTable(expr.Tokens))
end
out:appendToken( tok, options ) -- CANDRAN : options
tok_it = tok_it + 1
options.no_leading_white = false -- CANDRAN : not the leading token anymore
end
local function appendToken(token)
out:appendToken( token, options ) -- CANDRAN : options
tok_it = tok_it + 1
options.no_leading_white = false -- CANDRAN : not the leading token anymore
end
local function appendWhite()
local tok = expr.Tokens[tok_it];
if not tok then error(util.PrintTable(expr)) end
out:appendWhite( tok, options ) -- CANDRAN : options
tok_it = tok_it + 1
options.no_leading_white = false -- CANDRAN : not the leading token anymore
end
local function appendStr(str)
appendWhite()
out:appendStr(str)
end
local function peek()
if tok_it < #expr.Tokens then
return expr.Tokens[tok_it].Data
end
end
local function appendComma(mandatory, seperators)
if true then
seperators = seperators or { "," }
seperators = util.lookupify( seperators )
if not mandatory and not seperators[peek()] then
return
end
assert(seperators[peek()], "Missing comma or semicolon")
appendNextToken()
else
local p = peek()
if p == "," or p == ";" then
appendNextToken()
end
end
end
debug_printf("formatExpr(%s) at line %i", expr.AstType, expr.Tokens[1] and expr.Tokens[1].Line or -1)
if expr.AstType == 'VarExpr' then
if expr.Variable then
appendStr( expr.Variable.Name )
else
appendStr( expr.Name )
end
elseif expr.AstType == 'NumberExpr' then
appendToken( expr.Value )
elseif expr.AstType == 'StringExpr' then
appendToken( expr.Value )
elseif expr.AstType == 'BooleanExpr' then
appendNextToken( expr.Value and "true" or "false" )
elseif expr.AstType == 'NilExpr' then
appendNextToken( "nil" )
elseif expr.AstType == 'BinopExpr' then
formatExpr(expr.Lhs)
appendStr( expr.Op )
formatExpr(expr.Rhs)
elseif expr.AstType == 'UnopExpr' then
appendStr( expr.Op )
formatExpr(expr.Rhs)
elseif expr.AstType == 'DotsExpr' then
appendNextToken( "..." )
elseif expr.AstType == 'CallExpr' then
formatExpr(expr.Base)
appendNextToken( "(" )
for i,arg in ipairs( expr.Arguments ) do
formatExpr(arg)
appendComma( i ~= #expr.Arguments )
end
appendNextToken( ")" )
elseif expr.AstType == 'TableCallExpr' then
formatExpr( expr.Base )
formatExpr( expr.Arguments[1] )
elseif expr.AstType == 'StringCallExpr' then
formatExpr(expr.Base)
appendToken( expr.Arguments[1] )
elseif expr.AstType == 'IndexExpr' then
formatExpr(expr.Base)
appendNextToken( "[" )
formatExpr(expr.Index)
appendNextToken( "]" )
elseif expr.AstType == 'MemberExpr' then
formatExpr(expr.Base)
appendNextToken() -- . or :
appendToken(expr.Ident)
elseif expr.AstType == 'Function' then
-- anonymous function
appendNextToken( "function" )
appendNextToken( "(" )
if #expr.Arguments > 0 then
for i = 1, #expr.Arguments do
appendStr( expr.Arguments[i].Name )
if i ~= #expr.Arguments then
appendNextToken(",")
elseif expr.VarArg then
appendNextToken(",")
appendNextToken("...")
end
end
elseif expr.VarArg then
appendNextToken("...")
end
appendNextToken(")")
formatStatlist(expr.Body)
appendNextToken("end")
elseif expr.AstType == 'ConstructorExpr' then
-- CANDRAN : function to get a value with its applied decorators
local function appendValue(entry)
out:appendStr(" ")
if entry.Decorated then
for _,d in ipairs(entry.DecoratorChain) do
formatExpr(d)
out:appendStr("(")
end
end
formatExpr(entry.Value, { no_leading_white = true })
if entry.Decorated then
for _ in ipairs(entry.DecoratorChain) do
out:appendStr(")")
end
end
end
appendNextToken( "{" )
for i = 1, #expr.EntryList do
local entry = expr.EntryList[i]
if entry.Type == 'Key' then
appendNextToken( "[" )
formatExpr(entry.Key)
appendNextToken( "]" )
appendNextToken( "=" )
appendValue(entry) -- CANDRAN : respect decorators
elseif entry.Type == 'Value' then
appendValue(entry) -- CANDRAN : respect decorators
elseif entry.Type == 'KeyString' then
appendStr(entry.Key)
appendNextToken( "=" )
appendValue(entry) -- CANDRAN : respect decorators
end
appendComma( i ~= #expr.EntryList, { ",", ";" } )
end
appendNextToken( "}" )
elseif expr.AstType == 'Parentheses' then
appendNextToken( "(" )
formatExpr(expr.Inner)
appendNextToken( ")" )
else
print("Unknown AST Type: ", statement.AstType)
end
assert(tok_it == #expr.Tokens + 1)
debug_printf("/formatExpr")
end
formatStatement = function(statement)
local tok_it = 1
local function appendNextToken(str)
local tok = statement.Tokens[tok_it];
assert(tok, string.format("Not enough tokens for %q. First token at %i:%i",
str, statement.Tokens[1].Line, statement.Tokens[1].Char))
assert(tok.Data == str,
string.format('Expected token %q, got %q', str, tok.Data))
out:appendToken( tok )
tok_it = tok_it + 1
end
local function appendToken(token)
out:appendToken( str )
tok_it = tok_it + 1
end
local function appendWhite()
local tok = statement.Tokens[tok_it];
out:appendWhite( tok )
tok_it = tok_it + 1
end
local function appendStr(str)
appendWhite()
out:appendStr(str)
end
local function appendComma(mandatory)
if mandatory
or (tok_it < #statement.Tokens and statement.Tokens[tok_it].Data == ",") then
appendNextToken( "," )
end
end
debug_printf("")
debug_printf(string.format("formatStatement(%s) at line %i", statement.AstType, statement.Tokens[1] and statement.Tokens[1].Line or -1))
if statement.AstType == 'AssignmentStatement' then
local newlineToCheck -- CANDRAN : position of a potential newline to eliminate in some edge cases
for i,v in ipairs(statement.Lhs) do
formatExpr(v)
appendComma( i ~= #statement.Lhs )
end
if #statement.Rhs > 0 then
-- CANDRAN : get the assignment operator used (default to =)
local assignmentToken = "="
local candranAssignmentExists = util.lookupify(candran.assignment)
for i,v in pairs(statement.Tokens) do
if candranAssignmentExists[v.Data] then
assignmentToken = v.Data
break
end
end
appendNextToken(assignmentToken) -- CANDRAN : accept Candran assignments operators
--appendNextToken( "=" )
newlineToCheck = #out.rope + 1 -- CANDRAN : the potential newline position afte the =
if assignmentToken == "=" then
for i,v in ipairs(statement.Rhs) do
formatExpr(v)
appendComma( i ~= #statement.Rhs )
end
else
out.rope[#out.rope] = "= " -- CANDRAN : remplace +=, -=, etc. with =
for i,v in ipairs(statement.Rhs) do
if i <= #statement.Lhs then -- CANDRAN : impossible to assign more variables than indicated in Lhs
formatExpr(statement.Lhs[i], { no_newline = true }) -- CANDRAN : write variable to assign
out:appendStr(" "..assignmentToken:gsub("=$","")) -- CANDRAN : assignment operation
formatExpr(v) -- CANDRAN : write variable to add/sub/etc.
if i ~= #statement.Rhs then -- CANDRAN : add comma to allow multi-assignment
appendComma( i ~= #statement.Rhs )
if i >= #statement.Lhs then
out.rope[#out.rope] = "" -- CANDRAN : if this was the last element, remove the comma
end
end
end
end
end
end
-- CANDRAN : eliminate the bad newlines
if out.rope[newlineToCheck] == "\n" then
out.rope[newlineToCheck] = ""
end
elseif statement.AstType == 'CallStatement' then
formatExpr(statement.Expression)
elseif statement.AstType == 'LocalStatement' then
appendNextToken( "local" )
for i = 1, #statement.LocalList do
appendStr( statement.LocalList[i].Name )
appendComma( i ~= #statement.LocalList )
end
if #statement.InitList > 0 then
appendNextToken( "=" )
for i = 1, #statement.InitList do
formatExpr(statement.InitList[i])
appendComma( i ~= #statement.InitList )
end
end
elseif statement.AstType == 'IfStatement' then
appendNextToken( "if" )
formatExpr( statement.Clauses[1].Condition )
appendNextToken( "then" )
formatStatlist( statement.Clauses[1].Body )
for i = 2, #statement.Clauses do
local st = statement.Clauses[i]
if st.Condition then
appendNextToken( "elseif" )
formatExpr(st.Condition)
appendNextToken( "then" )
else
appendNextToken( "else" )
end
formatStatlist(st.Body)
end
appendNextToken( "end" )
elseif statement.AstType == 'WhileStatement' then
appendNextToken( "while" )
formatExpr(statement.Condition)
appendNextToken( "do" )
formatStatlist(statement.Body)
appendNextToken( "end" )
elseif statement.AstType == 'DoStatement' then
appendNextToken( "do" )
formatStatlist(statement.Body)
appendNextToken( "end" )
elseif statement.AstType == 'ReturnStatement' then
appendNextToken( "return" )
for i = 1, #statement.Arguments do
formatExpr(statement.Arguments[i])
appendComma( i ~= #statement.Arguments )
end
elseif statement.AstType == 'BreakStatement' then
appendNextToken( "break" )
elseif statement.AstType == 'RepeatStatement' then
appendNextToken( "repeat" )
formatStatlist(statement.Body)
appendNextToken( "until" )
formatExpr(statement.Condition)
-- CANDRAN : add decorator support (@)
elseif statement.AstType == 'DecoratedStatement' then
-- CANDRAN : list of the chained decorators
local decoratorChain = {statement}
-- CANDRAN : get the decorated statement
local decorated = statement.Decorated
while decorated.AstType == "DecoratedStatement" do
table.insert(decoratorChain, decorated)
decorated = decorated.Decorated
end
-- CANDRAN : write the decorated statement like a normal statement
formatStatement(decorated)
-- CANDRAN : mark the decorator token as used (and add whitespace)
appendNextToken(candran.decorator)
out.rope[#out.rope] = ""
-- CANDRAN : get the variable(s) to decorate name(s)
local names = {}
if decorated.AstType == "Function" then
table.insert(names, decorated.Name.Name)
elseif decorated.AstType == "AssignmentStatement" then
for _,var in ipairs(decorated.Lhs) do
table.insert(names, var.Name)
end
elseif decorated.AstType == "LocalStatement" then
for _,var in ipairs(decorated.LocalList) do
table.insert(names, var.Name)
end
else
error("Invalid statement type to decorate : "..decorated.AstType)
end
-- CANDRAN : redefine the variable(s) ( name, name2, ... = ... )
for i,name in ipairs(names) do
out:appendStr(name)
if i ~= #names then out:appendStr(", ") end
end
out:appendStr(" = ")
for i,name in ipairs(names) do
-- CANDRAN : write the decorator chain ( a(b(c(... )
for _,v in pairs(decoratorChain) do
formatExpr(v.Decorator)
out:appendStr("(")
end
-- CANDRAN : pass the undecorated variable name to the decorator chain
out:appendStr(name)
-- CANDRAN : close parantheses
for _ in pairs(decoratorChain) do
out:appendStr(")")
end
if i ~= #names then out:appendStr(", ") end
end
elseif statement.AstType == 'Function' then
--print(util.PrintTable(statement))
if statement.IsLocal then
appendNextToken( "local" )
end
appendNextToken( "function" )
if statement.IsLocal then
appendStr(statement.Name.Name)
else
formatExpr(statement.Name)
end
appendNextToken( "(" )
if #statement.Arguments > 0 then
for i = 1, #statement.Arguments do
appendStr( statement.Arguments[i].Name )
appendComma( i ~= #statement.Arguments or statement.VarArg )
if i == #statement.Arguments and statement.VarArg then
appendNextToken( "..." )
end
end
elseif statement.VarArg then
appendNextToken( "..." )
end
appendNextToken( ")" )
formatStatlist(statement.Body)
appendNextToken( "end" )
elseif statement.AstType == 'GenericForStatement' then
appendNextToken( "for" )
for i = 1, #statement.VariableList do
appendStr( statement.VariableList[i].Name )
appendComma( i ~= #statement.VariableList )
end
appendNextToken( "in" )
for i = 1, #statement.Generators do
formatExpr(statement.Generators[i])
appendComma( i ~= #statement.Generators )
end
appendNextToken( "do" )
formatStatlist(statement.Body)
appendNextToken( "end" )
elseif statement.AstType == 'NumericForStatement' then
appendNextToken( "for" )
appendStr( statement.Variable.Name )
appendNextToken( "=" )
formatExpr(statement.Start)
appendNextToken( "," )
formatExpr(statement.End)
if statement.Step then
appendNextToken( "," )
formatExpr(statement.Step)
end
appendNextToken( "do" )
formatStatlist(statement.Body)
appendNextToken( "end" )
elseif statement.AstType == 'LabelStatement' then
appendNextToken( "::" )
appendStr( statement.Label )
appendNextToken( "::" )
elseif statement.AstType == 'GotoStatement' then
appendNextToken( "goto" )
appendStr( statement.Label )
elseif statement.AstType == 'Eof' then
appendWhite()
else
print("Unknown AST Type: ", statement.AstType)
end
if statement.Semicolon then
appendNextToken(";")
end
assert(tok_it == #statement.Tokens + 1)
debug_printf("/formatStatment")
end
formatStatlist = function(statList)
for _, stat in ipairs(statList.Body) do
formatStatement(stat)
end
end
formatStatlist(ast)
return true, table.concat(out.rope)
end
return Format_Identity

364
lib/LuaMinify/FormatMini.lua Normal file
View file

@ -0,0 +1,364 @@
local parser = require'ParseLua'
local ParseLua = parser.ParseLua
local util = require'Util'
local lookupify = util.lookupify
--
-- FormatMini.lua
--
-- Returns the minified version of an AST. Operations which are performed:
-- - All comments and whitespace are ignored
-- - All local variables are renamed
--
local LowerChars = lookupify{'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i',
'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r',
's', 't', 'u', 'v', 'w', 'x', 'y', 'z'}
local UpperChars = lookupify{'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I',
'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R',
'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z'}
local Digits = lookupify{'0', '1', '2', '3', '4', '5', '6', '7', '8', '9'}
local Symbols = lookupify{'+', '-', '*', '/', '^', '%', ',', '{', '}', '[', ']', '(', ')', ';', '#'}
local function Format_Mini(ast)
local formatStatlist, formatExpr;
local count = 0
--
local function joinStatementsSafe(a, b, sep)
--print(a, b)
if count > 150 then
count = 0
return a.."\n"..b
end
sep = sep or ' '
local aa, bb = a:sub(-1,-1), b:sub(1,1)
if UpperChars[aa] or LowerChars[aa] or aa == '_' then
if not (UpperChars[bb] or LowerChars[bb] or bb == '_' or Digits[bb]) then
--bb is a symbol, can join without sep
return a..b
elseif bb == '(' then
print("==============>>>",aa,bb)
--prevent ambiguous syntax
return a..sep..b
else
return a..sep..b
end
elseif Digits[aa] then
if bb == '(' then
--can join statements directly
return a..b
elseif Symbols[bb] then
return a .. b
else
return a..sep..b
end
elseif aa == '' then
return a..b
else
if bb == '(' then
--don't want to accidentally call last statement, can't join directly
return a..sep..b
else
--print("asdf", '"'..a..'"', '"'..b..'"')
return a..b
end
end
end
formatExpr = function(expr, precedence)
local precedence = precedence or 0
local currentPrecedence = 0
local skipParens = false
local out = ""
if expr.AstType == 'VarExpr' then
if expr.Variable then
out = out..expr.Variable.Name
else
out = out..expr.Name
end
elseif expr.AstType == 'NumberExpr' then
out = out..expr.Value.Data
elseif expr.AstType == 'StringExpr' then
out = out..expr.Value.Data
elseif expr.AstType == 'BooleanExpr' then
out = out..tostring(expr.Value)
elseif expr.AstType == 'NilExpr' then
out = joinStatementsSafe(out, "nil")
elseif expr.AstType == 'BinopExpr' then
currentPrecedence = expr.OperatorPrecedence
out = joinStatementsSafe(out, formatExpr(expr.Lhs, currentPrecedence))
out = joinStatementsSafe(out, expr.Op)
out = joinStatementsSafe(out, formatExpr(expr.Rhs))
if expr.Op == '^' or expr.Op == '..' then
currentPrecedence = currentPrecedence - 1
end
if currentPrecedence < precedence then
skipParens = false
else
skipParens = true
end
--print(skipParens, precedence, currentPrecedence)
elseif expr.AstType == 'UnopExpr' then
out = joinStatementsSafe(out, expr.Op)
out = joinStatementsSafe(out, formatExpr(expr.Rhs))
elseif expr.AstType == 'DotsExpr' then
out = out.."..."
elseif expr.AstType == 'CallExpr' then
out = out..formatExpr(expr.Base)
out = out.."("
for i = 1, #expr.Arguments do
out = out..formatExpr(expr.Arguments[i])
if i ~= #expr.Arguments then
out = out..","
end
end
out = out..")"
elseif expr.AstType == 'TableCallExpr' then
out = out..formatExpr(expr.Base)
out = out..formatExpr(expr.Arguments[1])
elseif expr.AstType == 'StringCallExpr' then
out = out..formatExpr(expr.Base)
out = out..expr.Arguments[1].Data
elseif expr.AstType == 'IndexExpr' then
out = out..formatExpr(expr.Base).."["..formatExpr(expr.Index).."]"
elseif expr.AstType == 'MemberExpr' then
out = out..formatExpr(expr.Base)..expr.Indexer..expr.Ident.Data
elseif expr.AstType == 'Function' then
expr.Scope:ObfuscateVariables()
out = out.."function("
if #expr.Arguments > 0 then
for i = 1, #expr.Arguments do
out = out..expr.Arguments[i].Name
if i ~= #expr.Arguments then
out = out..","
elseif expr.VarArg then
out = out..",..."
end
end
elseif expr.VarArg then
out = out.."..."
end
out = out..")"
out = joinStatementsSafe(out, formatStatlist(expr.Body))
out = joinStatementsSafe(out, "end")
elseif expr.AstType == 'ConstructorExpr' then
out = out.."{"
for i = 1, #expr.EntryList do
local entry = expr.EntryList[i]
if entry.Type == 'Key' then
out = out.."["..formatExpr(entry.Key).."]="..formatExpr(entry.Value)
elseif entry.Type == 'Value' then
out = out..formatExpr(entry.Value)
elseif entry.Type == 'KeyString' then
out = out..entry.Key.."="..formatExpr(entry.Value)
end
if i ~= #expr.EntryList then
out = out..","
end
end
out = out.."}"
elseif expr.AstType == 'Parentheses' then
out = out.."("..formatExpr(expr.Inner)..")"
end
--print(">>", skipParens, expr.ParenCount, out)
if not skipParens then
--print("hehe")
out = string.rep('(', expr.ParenCount or 0) .. out
out = out .. string.rep(')', expr.ParenCount or 0)
--print("", out)
end
count = count + #out
return --[[print(out) or]] out
end
local formatStatement = function(statement)
local out = ''
if statement.AstType == 'AssignmentStatement' then
for i = 1, #statement.Lhs do
out = out..formatExpr(statement.Lhs[i])
if i ~= #statement.Lhs then
out = out..","
end
end
if #statement.Rhs > 0 then
out = out.."="
for i = 1, #statement.Rhs do
out = out..formatExpr(statement.Rhs[i])
if i ~= #statement.Rhs then
out = out..","
end
end
end
elseif statement.AstType == 'CallStatement' then
out = formatExpr(statement.Expression)
elseif statement.AstType == 'LocalStatement' then
out = out.."local "
for i = 1, #statement.LocalList do
out = out..statement.LocalList[i].Name
if i ~= #statement.LocalList then
out = out..","
end
end
if #statement.InitList > 0 then
out = out.."="
for i = 1, #statement.InitList do
out = out..formatExpr(statement.InitList[i])
if i ~= #statement.InitList then
out = out..","
end
end
end
elseif statement.AstType == 'IfStatement' then
out = joinStatementsSafe("if", formatExpr(statement.Clauses[1].Condition))
out = joinStatementsSafe(out, "then")
out = joinStatementsSafe(out, formatStatlist(statement.Clauses[1].Body))
for i = 2, #statement.Clauses do
local st = statement.Clauses[i]
if st.Condition then
out = joinStatementsSafe(out, "elseif")
out = joinStatementsSafe(out, formatExpr(st.Condition))
out = joinStatementsSafe(out, "then")
else
out = joinStatementsSafe(out, "else")
end
out = joinStatementsSafe(out, formatStatlist(st.Body))
end
out = joinStatementsSafe(out, "end")
elseif statement.AstType == 'WhileStatement' then
out = joinStatementsSafe("while", formatExpr(statement.Condition))
out = joinStatementsSafe(out, "do")
out = joinStatementsSafe(out, formatStatlist(statement.Body))
out = joinStatementsSafe(out, "end")
elseif statement.AstType == 'DoStatement' then
out = joinStatementsSafe(out, "do")
out = joinStatementsSafe(out, formatStatlist(statement.Body))
out = joinStatementsSafe(out, "end")
elseif statement.AstType == 'ReturnStatement' then
out = "return"
for i = 1, #statement.Arguments do
out = joinStatementsSafe(out, formatExpr(statement.Arguments[i]))
if i ~= #statement.Arguments then
out = out..","
end
end
elseif statement.AstType == 'BreakStatement' then
out = "break"
elseif statement.AstType == 'RepeatStatement' then
out = "repeat"
out = joinStatementsSafe(out, formatStatlist(statement.Body))
out = joinStatementsSafe(out, "until")
out = joinStatementsSafe(out, formatExpr(statement.Condition))
elseif statement.AstType == 'Function' then
statement.Scope:ObfuscateVariables()
if statement.IsLocal then
out = "local"
end
out = joinStatementsSafe(out, "function ")
if statement.IsLocal then
out = out..statement.Name.Name
else
out = out..formatExpr(statement.Name)
end
out = out.."("
if #statement.Arguments > 0 then
for i = 1, #statement.Arguments do
out = out..statement.Arguments[i].Name
if i ~= #statement.Arguments then
out = out..","
elseif statement.VarArg then
--print("Apply vararg")
out = out..",..."
end
end
elseif statement.VarArg then
out = out.."..."
end
out = out..")"
out = joinStatementsSafe(out, formatStatlist(statement.Body))
out = joinStatementsSafe(out, "end")
elseif statement.AstType == 'GenericForStatement' then
statement.Scope:ObfuscateVariables()
out = "for "
for i = 1, #statement.VariableList do
out = out..statement.VariableList[i].Name
if i ~= #statement.VariableList then
out = out..","
end
end
out = out.." in"
for i = 1, #statement.Generators do
out = joinStatementsSafe(out, formatExpr(statement.Generators[i]))
if i ~= #statement.Generators then
out = joinStatementsSafe(out, ',')
end
end
out = joinStatementsSafe(out, "do")
out = joinStatementsSafe(out, formatStatlist(statement.Body))
out = joinStatementsSafe(out, "end")
elseif statement.AstType == 'NumericForStatement' then
out = "for "
out = out..statement.Variable.Name.."="
out = out..formatExpr(statement.Start)..","..formatExpr(statement.End)
if statement.Step then
out = out..","..formatExpr(statement.Step)
end
out = joinStatementsSafe(out, "do")
out = joinStatementsSafe(out, formatStatlist(statement.Body))
out = joinStatementsSafe(out, "end")
elseif statement.AstType == 'LabelStatement' then
out = getIndentation() .. "::" .. statement.Label .. "::"
elseif statement.AstType == 'GotoStatement' then
out = getIndentation() .. "goto " .. statement.Label
elseif statement.AstType == 'Comment' then
-- ignore
elseif statement.AstType == 'Eof' then
-- ignore
else
print("Unknown AST Type: " .. statement.AstType)
end
count = count + #out
return out
end
formatStatlist = function(statList)
local out = ''
statList.Scope:ObfuscateVariables()
for _, stat in pairs(statList.Body) do
out = joinStatementsSafe(out, formatStatement(stat), ';')
end
return out
end
ast.Scope:ObfuscateVariables()
return formatStatlist(ast)
end
return Format_Mini

20
lib/LuaMinify/LICENSE.md Normal file
View file

@ -0,0 +1,20 @@
The MIT License (MIT)
Copyright (c) 2012-2013
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

2
lib/LuaMinify/LuaMinify.bat Normal file
View file

@ -0,0 +1,2 @@
@echo off
lua CommandLineMinify.lua %*

2
lib/LuaMinify/LuaMinify.sh Normal file
View file

@ -0,0 +1,2 @@
#!/bin/bash
lua CommandLineMinify.lua $@

1523
lib/LuaMinify/ParseCandran.lua Normal file
View file

File diff suppressed because it is too large Load diff

1411
lib/LuaMinify/ParseLua.lua Normal file
View file

File diff suppressed because it is too large Load diff

44
lib/LuaMinify/README.md Normal file
View file

@ -0,0 +1,44 @@
Lua Parsing and Refactorization tools
=========
A collection of tools for working with Lua source code. Primarily a Lua source code minifier, but also includes some static analysis tools and a general Lua lexer and parser.
Currently the minifier performs:
- Stripping of all comments and whitespace
- True semantic renaming of all local variables to a reduced form
- Reduces the source to the minimal spacing, spaces are only inserted where actually needed.
LuaMinify Command Line Utility Usage
------------------------------------
The `LuaMinify` shell and batch files are given as shortcuts to running a command line instance of the minifier with the following usage:
LuaMinify sourcefile [destfile]
Which will minify to a given destination file, or to a copy of the source file with _min appended to the filename if no output file is given.
LuaMinify Roblox Plugin Usage
-----------------------------
First, download the source code, which you can do by hitting this button:
![Click That](http://github.com/stravant/LuaMinify/raw/master/RobloxPluginInstructions.png)
Then copy the `RobloxPlugin` folder from the source into your Roblox Plugins directory, which can be found by hitting `Tools->Open Plugins Folder` in Roblox Studio.
Features/Todo
-------------
Features:
- Lua scanner/parser, which generates a full AST
- Lua reconstructor
- minimal
- full reconstruction (TODO: options, comments)
- TODO: exact reconstructor
- support for embedded long strings/comments e.g. [[abc [[ def ]] ghi]]
Todo:
- use table.concat instead of appends in the reconstructors

1570
lib/LuaMinify/RobloxPlugin/Minify.lua Normal file
View file

File diff suppressed because it is too large Load diff

BIN
lib/LuaMinify/RobloxPlugin/MinifyButtonIcon.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 355 B

93
lib/LuaMinify/RobloxPlugin/MinifyToolbar.lua Normal file
View file

@ -0,0 +1,93 @@
--
-- MinifyToolbar.lua
--
-- The main script that generates a toolbar for studio that allows minification of selected
-- scripts, calling on the _G.Minify function defined in `Minify.lua`
--
local plugin = PluginManager():CreatePlugin()
local toolbar = plugin:CreateToolbar("Minify")
local minifyButton = toolbar:CreateButton("", "Minify Selected Script", 'MinifyButtonIcon.png')
local toggleReplaceButton = toolbar:CreateButton("Replace", "If enabled, selected script will be REPLACED "..
"with a minified version",
'ReplaceButtonIcon.png')
local replace = false
toggleReplaceButton.Click:connect(function()
replace = not replace
toggleReplaceButton:SetActive(replace)
end)
minifyButton.Click:connect(function()
for _, o in pairs(game.Selection:Get()) do
if o:IsA('Script') then
--can't read linkedsource, bail out
if o.LinkedSource ~= '' then
Spawn(function()
error("Minify Plugin: Cannot Minify a script with a LinkedSource", 0)
end)
return
end
--see if it has been minified
if o.Name:sub(-4,-1) == '_Min' then
local original = o:FindFirstChild(o.Name:sub(1,-5))
if original then
local st, min = _G.Minify(original.Source)
if st then
game:GetService("ChangeHistoryService"):SetWaypoint("Minify `"..original.Name.."`")
if replace then
o.Source = min
original:Destroy()
else
o.Source = min
end
else
Spawn(function()
error("Minify Plugin: "..min, 0)
end)
return
end
else
if replace then
local st, min = _G.Minify(o.Source)
if st then
game:GetService("ChangeHistoryService"):SetWaypoint("Minify `"..original.Name.."`")
o.Source = min
else
Spawn(function()
error("Minify Plugin: "..min, 0)
end)
return
end
else
Spawn(function()
error("Minify Plugin: Missing original script `"..o.Name:sub(1,-5).."`", 0)
end)
end
end
else
local st, min = _G.Minify(o.Source)
if st then
game:GetService("ChangeHistoryService"):SetWaypoint("Minify `"..o.Name.."`")
if replace then
o.Source = min
o.Name = o.Name.."_Min"
else
local original = o:Clone()
original.Parent = o
original.Disabled = true
o.Name = o.Name.."_Min"
o.Source = min
end
else
Spawn(function()
error("Minify Plugin: "..min, 0)
end)
return
end
end
end
end
end)

BIN
lib/LuaMinify/RobloxPlugin/ReplaceButtonIcon.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 502 B

BIN
lib/LuaMinify/RobloxPluginInstructions.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 13 KiB

221
lib/LuaMinify/Scope.lua Normal file
View file

@ -0,0 +1,221 @@
--[[
This file is part of LuaMinify by stravant (https://github.com/stravant/LuaMinify).
LICENSE :
The MIT License (MIT)
Copyright (c) 2012-2013
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
]]
local Scope = {
new = function(self, parent)
local s = {
Parent = parent,
Locals = { },
Globals = { },
oldLocalNamesMap = { },
oldGlobalNamesMap = { },
Children = { },
}
if parent then
table.insert(parent.Children, s)
end
return setmetatable(s, { __index = self })
end,
AddLocal = function(self, v)
table.insert(self.Locals, v)
end,
AddGlobal = function(self, v)
table.insert(self.Globals, v)
end,
CreateLocal = function(self, name)
local v
v = self:GetLocal(name)
if v then return v end
v = { }
v.Scope = self
v.Name = name
v.IsGlobal = false
v.CanRename = true
v.References = 1
self:AddLocal(v)
return v
end,
GetLocal = function(self, name)
for k, var in pairs(self.Locals) do
if var.Name == name then return var end
end
if self.Parent then
return self.Parent:GetLocal(name)
end
end,
GetOldLocal = function(self, name)
if self.oldLocalNamesMap[name] then
return self.oldLocalNamesMap[name]
end
return self:GetLocal(name)
end,
mapLocal = function(self, name, var)
self.oldLocalNamesMap[name] = var
end,
GetOldGlobal = function(self, name)
if self.oldGlobalNamesMap[name] then
return self.oldGlobalNamesMap[name]
end
return self:GetGlobal(name)
end,
mapGlobal = function(self, name, var)
self.oldGlobalNamesMap[name] = var
end,
GetOldVariable = function(self, name)
return self:GetOldLocal(name) or self:GetOldGlobal(name)
end,
RenameLocal = function(self, oldName, newName)
oldName = type(oldName) == 'string' and oldName or oldName.Name
local found = false
local var = self:GetLocal(oldName)
if var then
var.Name = newName
self:mapLocal(oldName, var)
found = true
end
if not found and self.Parent then
self.Parent:RenameLocal(oldName, newName)
end
end,
RenameGlobal = function(self, oldName, newName)
oldName = type(oldName) == 'string' and oldName or oldName.Name
local found = false
local var = self:GetGlobal(oldName)
if var then
var.Name = newName
self:mapGlobal(oldName, var)
found = true
end
if not found and self.Parent then
self.Parent:RenameGlobal(oldName, newName)
end
end,
RenameVariable = function(self, oldName, newName)
oldName = type(oldName) == 'string' and oldName or oldName.Name
if self:GetLocal(oldName) then
self:RenameLocal(oldName, newName)
else
self:RenameGlobal(oldName, newName)
end
end,
GetAllVariables = function(self)
local ret = self:getVars(true) -- down
for k, v in pairs(self:getVars(false)) do -- up
table.insert(ret, v)
end
return ret
end,
getVars = function(self, top)
local ret = { }
if top then
for k, v in pairs(self.Children) do
for k2, v2 in pairs(v:getVars(true)) do
table.insert(ret, v2)
end
end
else
for k, v in pairs(self.Locals) do
table.insert(ret, v)
end
for k, v in pairs(self.Globals) do
table.insert(ret, v)
end
if self.Parent then
for k, v in pairs(self.Parent:getVars(false)) do
table.insert(ret, v)
end
end
end
return ret
end,
CreateGlobal = function(self, name)
local v
v = self:GetGlobal(name)
if v then return v end
v = { }
v.Scope = self
v.Name = name
v.IsGlobal = true
v.CanRename = true
v.References = 1
self:AddGlobal(v)
return v
end,
GetGlobal = function(self, name)
for k, v in pairs(self.Globals) do
if v.Name == name then return v end
end
if self.Parent then
return self.Parent:GetGlobal(name)
end
end,
GetVariable = function(self, name)
return self:GetLocal(name) or self:GetGlobal(name)
end,
ObfuscateLocals = function(self, recommendedMaxLength, validNameChars)
recommendedMaxLength = recommendedMaxLength or 7
local chars = validNameChars or "QWERTYUIOPASDFGHJKLZXCVBNMqwertyuioplkjhgfdsazxcvbnm_"
local chars2 = validNameChars or "QWERTYUIOPASDFGHJKLZXCVBNMqwertyuioplkjhgfdsazxcvbnm_1234567890"
for _, var in pairs(self.Locals) do
local id = ""
local tries = 0
repeat
local n = math.random(1, #chars)
id = id .. chars:sub(n, n)
for i = 1, math.random(0, tries > 5 and 30 or recommendedMaxLength) do
local n = math.random(1, #chars2)
id = id .. chars2:sub(n, n)
end
tries = tries + 1
until not self:GetVariable(id)
self:RenameLocal(var.Name, id)
end
end,
}
return Scope

116
lib/LuaMinify/Util.lua Normal file
View file

@ -0,0 +1,116 @@
--[[
This file is part of LuaMinify by stravant (https://github.com/stravant/LuaMinify).
LICENSE :
The MIT License (MIT)
Copyright (c) 2012-2013
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
]]
--
-- Util.lua
--
-- Provides some common utilities shared throughout the project.
--
local function lookupify(tb)
for _, v in pairs(tb) do
tb[v] = true
end
return tb
end
local function CountTable(tb)
local c = 0
for _ in pairs(tb) do c = c + 1 end
return c
end
local function PrintTable(tb, atIndent)
if tb.Print then
return tb.Print()
end
atIndent = atIndent or 0
local useNewlines = (CountTable(tb) > 1)
local baseIndent = string.rep(' ', atIndent+1)
local out = "{"..(useNewlines and '\n' or '')
for k, v in pairs(tb) do
if type(v) ~= 'function' then
--do
out = out..(useNewlines and baseIndent or '')
if type(k) == 'number' then
--nothing to do
elseif type(k) == 'string' and k:match("^[A-Za-z_][A-Za-z0-9_]*$") then
out = out..k.." = "
elseif type(k) == 'string' then
out = out.."[\""..k.."\"] = "
else
out = out.."["..tostring(k).."] = "
end
if type(v) == 'string' then
out = out.."\""..v.."\""
elseif type(v) == 'number' then
out = out..v
elseif type(v) == 'table' then
out = out..PrintTable(v, atIndent+(useNewlines and 1 or 0))
else
out = out..tostring(v)
end
if next(tb, k) then
out = out..","
end
if useNewlines then
out = out..'\n'
end
end
end
out = out..(useNewlines and string.rep(' ', atIndent) or '').."}"
return out
end
local function splitLines(str)
if str:match("\n") then
local lines = {}
for line in str:gmatch("[^\n]*") do
table.insert(lines, line)
end
assert(#lines > 0)
return lines
else
return { str }
end
end
local function printf(fmt, ...)
return print(string.format(fmt, ...))
end
return {
PrintTable = PrintTable,
CountTable = CountTable,
lookupify = lookupify,
splitLines = splitLines,
printf = printf,
}

39
lib/LuaMinify/strict.lua Normal file
View file

@ -0,0 +1,39 @@
-- From http://metalua.luaforge.net/src/lib/strict.lua.html
--
-- strict.lua
-- checks uses of undeclared global variables
-- All global variables must be 'declared' through a regular assignment
-- (even assigning nil will do) in a main chunk before being used
-- anywhere or assigned to inside a function.
--
local mt = getmetatable(_G)
if mt == nil then
mt = {}
setmetatable(_G, mt)
end
__STRICT = true
mt.__declared = {}
mt.__newindex = function (t, n, v)
if __STRICT and not mt.__declared[n] then
local w = debug.getinfo(2, "S").what
if w ~= "main" and w ~= "C" then
error("assign to undeclared variable '"..n.."'", 2)
end
mt.__declared[n] = true
end
rawset(t, n, v)
end
mt.__index = function (t, n)
if not mt.__declared[n] and debug.getinfo(2, "S").what ~= "C" then
error("variable '"..n.."' is not declared", 2)
end
return rawget(t, n)
end
function global(...)
for _, v in ipairs{...} do mt.__declared[v] = true end
end

60
lib/LuaMinify/tests/test_beautifier.lua Normal file
View file

@ -0,0 +1,60 @@
-- Adapted from Yueliang
package.path = "../?.lua;" .. package.path
local util = require'Util'
local Parser = require'ParseLua'
local Format = require'FormatBeautiful'
for w in io.lines("test_lines.txt") do
--print(w)
local success, ast = Parser.ParseLua(w)
if w:find("FAIL") then
--[[if success then
print("ERROR PARSING LINE:")
print("Should fail: true. Did fail: " .. tostring(not success))
print("Line: " .. w)
else
--print("Suceeded!")
end]]
else
if not success then
print("ERROR PARSING LINE:")
print("Should fail: false. Did fail: " .. tostring(not success))
print("Line: " .. w)
else
success, ast = Format(ast)
--print(success, ast)
if not success then
print("ERROR BEAUTIFYING LINE:")
print("Message: " .. ast)
print("Line: " .. w)
end
local success_ = success
success, ast = loadstring(success)
if not success then
print("ERROR PARSING BEAUTIFIED LINE:")
print("Message: " .. ast)
print("Line: " .. success_)
end
--print("Suceeded!")
end
end
end
print"Done!"
os.remove("tmp")
--[[
function readAll(file)
local f = io.open(file, "rb")
local content = f:read("*all")
f:close()
return content
end
local text = readAll('../ParseLua.lua')
local success, ast = Parser.ParseLua(text)
local nice
nice = Format(ast)
print(nice)
--]]

124
lib/LuaMinify/tests/test_identity.lua Normal file
View file

@ -0,0 +1,124 @@
package.path = "../?.lua;" .. package.path
local Parser = require'ParseLua'
local util = require'Util'
local FormatIdentity = require'FormatIdentity'
local FormatMini = require'FormatMini'
local FormatBeautiful = require'FormatBeautiful'
require'strict'
function readAll(file)
local f = io.open(file, "rb")
local content = f:read("*all")
f:close()
return content
end
local g_lexTime = 0
local g_parseTime = 0
local g_reconstructTime = 0
function reconstructText(text)
local preLex = os.clock()
local success, tokens, ast, reconstructed
success, tokens = Parser.LexLua(text)
if not success then
print("ERROR: " .. tokens)
return
end
local preParse = os.clock()
success, ast = Parser.ParseLua(tokens)
if not success then
print("ERROR: " .. ast)
return
end
local preReconstruct = os.clock()
local DO_MINI = false
local DO_CHECK = false
if DO_MINI then
success, reconstructed = FormatMini(ast)
else
success, reconstructed = FormatIdentity(ast)
end
if not success then
print("ERROR: " .. reconstructed)
return
end
local post = os.clock()
g_lexTime = g_lexTime + (preParse - preLex)
g_parseTime = g_parseTime + (preReconstruct - preParse)
g_reconstructTime = g_reconstructTime + (post - preReconstruct)
if DO_CHECK then
--[[
print()
print("Reconstructed: ")
print("--------------------")
print(reconstructed)
print("--------------------")
print("Done. ")
--]]
if reconstructed == text then
--print("Reconstruction succeeded")
else
print("Reconstruction failed")
local inputLines = util.splitLines(text)
local outputLines = util.splitLines(reconstructed)
local n = math.max(#inputLines, #outputLines)
for i = 1,n do
if inputLines[i] ~= outputLines[i] then
util.printf("ERROR on line %i", i)
util.printf("Input: %q", inputLines[i])
util.printf("Output: %q", outputLines[i])
break
end
end
end
end
end
--[*[
local files = {
"../ParseLua.lua",
"../FormatIdentity.lua",
"../Scope.lua",
"../strict.lua",
"../Type.lua",
"Test_identity.lua"
}
for _,path in ipairs(files) do
print(path)
local text = readAll(path)
reconstructText(text)
end
--]]
print("test_lines.txt")
local line_nr = 0
for text in io.lines("test_lines.txt") do
line_nr = line_nr + 1
if not text:find("FAIL") then
--util.printf("\nText: %q", text)
reconstructText(text)
end
end
reconstructText('function a(p,q,r,...) end')
util.printf("Lex time: %f s", g_lexTime)
util.printf("Parse time: %f s", g_parseTime)
util.printf("Format time: %f s", g_reconstructTime)

523
lib/LuaMinify/tests/test_lines.txt Normal file
View file

@ -0,0 +1,523 @@
; -- FAIL
local -- FAIL
local; -- FAIL
local = -- FAIL
local end -- FAIL
local a
local a;
local a, b, c
local a; local b local c;
local a = 1
local a local b = a
local a, b = 1, 2
local a, b, c = 1, 2, 3
local a, b, c = 1
local a = 1, 2, 3
local a, local -- FAIL
local 1 -- FAIL
local "foo" -- FAIL
local a = local -- FAIL
local a, b, = -- FAIL
local a, b = 1, local -- FAIL
local a, b = , local -- FAIL
do -- FAIL
end -- FAIL
do end
do ; end -- FAIL
do 1 end -- FAIL
do "foo" end -- FAIL
do local a, b end
do local a local b end
do local a; local b; end
do local a = 1 end
do do end end
do do end; end
do do do end end end
do do do end; end; end
do do do return end end end
do end do -- FAIL
do end end -- FAIL
do return end
do return return end -- FAIL
do break end -- FAIL
while -- FAIL
while do -- FAIL
while = -- FAIL
while 1 do -- FAIL
while 1 do end
while 1 do local a end
while 1 do local a local b end
while 1 do local a; local b; end
while 1 do 2 end -- FAIL
while 1 do "foo" end -- FAIL
while true do end
while 1 do ; end -- FAIL
while 1 do while -- FAIL
while 1 end -- FAIL
while 1 2 do -- FAIL
while 1 = 2 do -- FAIL
while 1 do return end
while 1 do return return end -- FAIL
while 1 do do end end
while 1 do do return end end
while 1 do break end
while 1 do break break end -- FAIL
while 1 do do break end end
repeat -- FAIL
repeat until -- FAIL
repeat until 0
repeat until false
repeat until local -- FAIL
repeat end -- FAIL
repeat 1 -- FAIL
repeat = -- FAIL
repeat local a until 1
repeat local a local b until 0
repeat local a; local b; until 0
repeat ; until 1 -- FAIL
repeat 2 until 1 -- FAIL
repeat "foo" until 1 -- FAIL
repeat return until 0
repeat return return until 0 -- FAIL
repeat break until 0
repeat break break until 0 -- FAIL
repeat do end until 0
repeat do return end until 0
repeat do break end until 0
for -- FAIL
for do -- FAIL
for end -- FAIL
for 1 -- FAIL
for a -- FAIL
for true -- FAIL
for a, in -- FAIL
for a in -- FAIL
for a do -- FAIL
for a in do -- FAIL
for a in b do -- FAIL
for a in b end -- FAIL
for a in b, do -- FAIL
for a in b do end
for a in b do local a local b end
for a in b do local a; local b; end
for a in b do 1 end -- FAIL
for a in b do "foo" end -- FAIL
for a b in -- FAIL
for a, b, c in p do end
for a, b, c in p, q, r do end
for a in 1 do end
for a in true do end
for a in "foo" do end
for a in b do break end
for a in b do break break end -- FAIL
for a in b do return end
for a in b do return return end -- FAIL
for a in b do do end end
for a in b do do break end end
for a in b do do return end end
for = -- FAIL
for a = -- FAIL
for a, b = -- FAIL
for a = do -- FAIL
for a = 1, do -- FAIL
for a = p, q, do -- FAIL
for a = p q do -- FAIL
for a = b do end -- FAIL
for a = 1, 2, 3, 4 do end -- FAIL
for a = p, q do end
for a = 1, 2 do end
for a = 1, 2 do local a local b end
for a = 1, 2 do local a; local b; end
for a = 1, 2 do 3 end -- FAIL
for a = 1, 2 do "foo" end -- FAIL
for a = p, q, r do end
for a = 1, 2, 3 do end
for a = p, q do break end
for a = p, q do break break end -- FAIL
for a = 1, 2 do return end
for a = 1, 2 do return return end -- FAIL
for a = p, q do do end end
for a = p, q do do break end end
for a = p, q do do return end end
break -- FAIL
return
return;
return return -- FAIL
return 1
return local -- FAIL
return "foo"
return 1, -- FAIL
return 1,2,3
return a,b,c,d
return 1,2;
return ...
return 1,a,...
if -- FAIL
elseif -- FAIL
else -- FAIL
then -- FAIL
if then -- FAIL
if 1 -- FAIL
if 1 then -- FAIL
if 1 else -- FAIL
if 1 then else -- FAIL
if 1 then elseif -- FAIL
if 1 then end
if 1 then local a end
if 1 then local a local b end
if 1 then local a; local b; end
if 1 then else end
if 1 then local a else local b end
if 1 then local a; else local b; end
if 1 then elseif 2 -- FAIL
if 1 then elseif 2 then -- FAIL
if 1 then elseif 2 then end
if 1 then local a elseif 2 then local b end
if 1 then local a; elseif 2 then local b; end
if 1 then elseif 2 then else end
if 1 then else if 2 then end end
if 1 then else if 2 then end -- FAIL
if 1 then break end -- FAIL
if 1 then return end
if 1 then return return end -- FAIL
if 1 then end; if 1 then end;
function -- FAIL
function 1 -- FAIL
function end -- FAIL
function a -- FAIL
function a end -- FAIL
function a( end -- FAIL
function a() end
function a(1 -- FAIL
function a("foo" -- FAIL
function a(p -- FAIL
function a(p,) -- FAIL
function a(p q -- FAIL
function a(p) end
function a(p,q,) end -- FAIL
function a(p,q,r) end
function a(p,q,1 -- FAIL
function a(p) do -- FAIL
function a(p) 1 end -- FAIL
function a(p) return end
function a(p) break end -- FAIL
function a(p) return return end -- FAIL
function a(p) do end end
function a.( -- FAIL
function a.1 -- FAIL
function a.b() end
function a.b, -- FAIL
function a.b.( -- FAIL
function a.b.c.d() end
function a: -- FAIL
function a:1 -- FAIL
function a:b() end
function a:b: -- FAIL
function a:b. -- FAIL
function a.b.c:d() end
function a(...) end
function a(..., -- FAIL
function a(p,...) end
function a(p,q,r,...) end
function a() local a local b end
function a() local a; local b; end
function a() end; function a() end;
local function -- FAIL
local function 1 -- FAIL
local function end -- FAIL
local function a -- FAIL
local function a end -- FAIL
local function a( end -- FAIL
local function a() end
local function a(1 -- FAIL
local function a("foo" -- FAIL
local function a(p -- FAIL
local function a(p,) -- FAIL
local function a(p q -- FAIL
local function a(p) end
local function a(p,q,) end -- FAIL
local function a(p,q,r) end
local function a(p,q,1 -- FAIL
local function a(p) do -- FAIL
local function a(p) 1 end -- FAIL
local function a(p) return end
local function a(p) break end -- FAIL
local function a(p) return return end -- FAIL
local function a(p) do end end
local function a. -- FAIL
local function a: -- FAIL
local function a(...) end
local function a(..., -- FAIL
local function a(p,...) end
local function a(p,q,r,...) end
local function a() local a local b end
local function a() local a; local b; end
local function a() end; local function a() end;
a -- FAIL
a, -- FAIL
a,b,c -- FAIL
a,b = -- FAIL
a = 1
a = 1,2,3
a,b,c = 1
a,b,c = 1,2,3
a.b = 1
a.b.c = 1
a[b] = 1
a[b][c] = 1
a.b[c] = 1
a[b].c = 1
0 = -- FAIL
"foo" = -- FAIL
true = -- FAIL
(a) = -- FAIL
{} = -- FAIL
a:b() = -- FAIL
a() = -- FAIL
a.b:c() = -- FAIL
a[b]() = -- FAIL
a = a b -- FAIL
a = 1 2 -- FAIL
a = a = 1 -- FAIL
a( -- FAIL
a()
a(1)
a(1,) -- FAIL
a(1,2,3)
1() -- FAIL
a()()
a.b()
a[b]()
a.1 -- FAIL
a.b -- FAIL
a[b] -- FAIL
a.b.( -- FAIL
a.b.c()
a[b][c]()
a[b].c()
a.b[c]()
a:b()
a:b -- FAIL
a:1 -- FAIL
a.b:c()
a[b]:c()
a:b: -- FAIL
a:b():c()
a:b().c[d]:e()
a:b()[c].d:e()
(a)()
()() -- FAIL
(1)()
("foo")()
(true)()
(a)()()
(a.b)()
(a[b])()
(a).b()
(a)[b]()
(a):b()
(a).b[c]:d()
(a)[b].c:d()
(a):b():c()
(a):b().c[d]:e()
(a):b()[c].d:e()
a"foo"
a[[foo]]
a.b"foo"
a[b]"foo"
a:b"foo"
a{}
a.b{}
a[b]{}
a:b{}
a()"foo"
a"foo"()
a"foo".b()
a"foo"[b]()
a"foo":c()
a"foo""bar"
a"foo"{}
(a):b"foo".c[d]:e"bar"
(a):b"foo"[c].d:e"bar"
a(){}
a{}()
a{}.b()
a{}[b]()
a{}:c()
a{}"foo"
a{}{}
(a):b{}.c[d]:e{}
(a):b{}[c].d:e{}
a = -- FAIL
a = a
a = nil
a = false
a = 1
a = "foo"
a = [[foo]]
a = {}
a = (a)
a = (nil)
a = (true)
a = (1)
a = ("foo")
a = ([[foo]])
a = ({})
a = a.b
a = a.b. -- FAIL
a = a.b.c
a = a:b -- FAIL
a = a[b]
a = a[1]
a = a["foo"]
a = a[b][c]
a = a.b[c]
a = a[b].c
a = (a)[b]
a = (a).c
a = () -- FAIL
a = a()
a = a.b()
a = a[b]()
a = a:b()
a = (a)()
a = (a).b()
a = (a)[b]()
a = (a):b()
a = a"foo"
a = a{}
a = function -- FAIL
a = function 1 -- FAIL
a = function a -- FAIL
a = function end -- FAIL
a = function( -- FAIL
a = function() end
a = function(1 -- FAIL
a = function(p) end
a = function(p,) -- FAIL
a = function(p q -- FAIL
a = function(p,q,r) end
a = function(p,q,1 -- FAIL
a = function(...) end
a = function(..., -- FAIL
a = function(p,...) end
a = function(p,q,r,...) end
a = ...
a = a, b, ...
a = (...)
a = ..., 1, 2
a = function() return ... end -- FAIL
a = -10
a = -"foo"
a = -a
a = -nil
a = -true
a = -{}
a = -function() end
a = -a()
a = -(a)
a = - -- FAIL
a = not 10
a = not "foo"
a = not a
a = not nil
a = not true
a = not {}
a = not function() end
a = not a()
a = not (a)
a = not -- FAIL
a = #10
a = #"foo"
a = #a
a = #nil
a = #true
a = #{}
a = #function() end
a = #a()
a = #(a)
a = # -- FAIL
a = 1 + 2; a = 1 - 2
a = 1 * 2; a = 1 / 2
a = 1 ^ 2; a = 1 % 2
a = 1 .. 2
a = 1 + -- FAIL
a = 1 .. -- FAIL
a = 1 * / -- FAIL
a = 1 + -2; a = 1 - -2
a = 1 * - -- FAIL
a = 1 * not 2; a = 1 / not 2
a = 1 / not -- FAIL
a = 1 * #"foo"; a = 1 / #"foo"
a = 1 / # -- FAIL
a = 1 + 2 - 3 * 4 / 5 % 6 ^ 7
a = ((1 + 2) - 3) * (4 / (5 % 6 ^ 7))
a = (1 + (2 - (3 * (4 / (5 % 6 ^ ((7)))))))
a = ((1 -- FAIL
a = ((1 + 2) -- FAIL
a = 1) -- FAIL
a = a + b - c
a = "foo" + "bar"
a = "foo".."bar".."baz"
a = true + false - nil
a = {} * {}
a = function() end / function() end
a = a() ^ b()
a = ... % ...
a = 1 == 2; a = 1 ~= 2
a = 1 < 2; a = 1 <= 2
a = 1 > 2; a = 1 >= 2
a = 1 < 2 < 3
a = 1 >= 2 >= 3
a = 1 == -- FAIL
a = ~= 2 -- FAIL
a = "foo" == "bar"
a = "foo" > "bar"
a = a ~= b
a = true == false
a = 1 and 2; a = 1 or 2
a = 1 and -- FAIL
a = or 1 -- FAIL
a = 1 and 2 and 3
a = 1 or 2 or 3
a = 1 and 2 or 3
a = a and b or c
a = a() and (b)() or c.d
a = "foo" and "bar"
a = true or false
a = {} and {} or {}
a = (1) and ("foo") or (nil)
a = function() end == function() end
a = function() end or function() end
a = { -- FAIL
a = {}
a = {,} -- FAIL
a = {;} -- FAIL
a = {,,} -- FAIL
a = {;;} -- FAIL
a = {{ -- FAIL
a = {{{}}}
a = {{},{},{{}},}
a = { 1 }
a = { 1, }
a = { 1; }
a = { 1, 2 }
a = { a, b, c, }
a = { true; false, nil; }
a = { a.b, a[b]; a:c(), }
a = { 1 + 2, a > b, "a" or "b" }
a = { a=1, }
a = { a=1, b="foo", c=nil }
a = { a -- FAIL
a = { a= -- FAIL
a = { a=, -- FAIL
a = { a=; -- FAIL
a = { 1, a="foo" -- FAIL
a = { 1, a="foo"; b={}, d=true; }
a = { [ -- FAIL
a = { [1 -- FAIL
a = { [1] -- FAIL
a = { [a]= -- FAIL
a = { ["foo"]="bar" }
a = { [1]=a, [2]=b, }
a = { true, a=1; ["foo"]="bar", }

61
lib/LuaMinify/tests/test_minifier.lua Normal file
View file

@ -0,0 +1,61 @@
-- Adapted from Yueliang
package.path = "../?.lua;" .. package.path
local util = require'Util'
local Parser = require'ParseLua'
local Format_Mini = require'FormatMini'
local line_nr = 0
for w in io.lines("test_lines.txt") do
line_nr = line_nr + 1
--print(w)
local success, ast = Parser.ParseLua(w)
if w:find("FAIL") then
--[[if success then
print("ERROR PARSING LINE:")
print("Should fail: true. Did fail: " .. tostring(not success))
print("Line: " .. w)
else
--print("Suceeded!")
end]]
else
if not success then
print("ERROR PARSING LINE:")
print("Should fail: false. Did fail: " .. tostring(not success))
print("Line: " .. w)
else
success, ast = Format_Mini(ast)
--print(success, ast)
if not success then
print("ERROR MINIFYING LINE:")
print("Message: " .. ast)
print("Line: " .. w)
end
success, ast = loadstring(success)
if not success then
print("ERROR PARSING MINIFIED LINE:")
print("Message: " .. ast)
print("Line nr: " .. line_nr)
print("Line: " .. w)
end
--print("Suceeded!")
end
end
end
print"Done!"
os.remove("tmp")
--[[
function readAll(file)
local f = io.open(file, "rb")
local content = f:read("*all")
f:close()
return content
end
local text = readAll('../ParseLua.lua')
local success, ast = Parser.ParseLua(text)
local nice
nice = Format_Mini(ast)
print(nice)
--]]

561
lib/LuaMinify/tests/test_parser.lua Normal file
View file

@ -0,0 +1,561 @@
-- Adapted from Yueliang
local source = [=[
; -- FAIL
local -- FAIL
local; -- FAIL
local = -- FAIL
local end -- FAIL
local a
local a;
local a, b, c
local a; local b local c;
local a = 1
local a local b = a
local a, b = 1, 2
local a, b, c = 1, 2, 3
local a, b, c = 1
local a = 1, 2, 3
local a, local -- FAIL
local 1 -- FAIL
local "foo" -- FAIL
local a = local -- FAIL
local a, b, = -- FAIL
local a, b = 1, local -- FAIL
local a, b = , local -- FAIL
do -- FAIL
end -- FAIL
do end
do ; end -- FAIL
do 1 end -- FAIL
do "foo" end -- FAIL
do local a, b end
do local a local b end
do local a; local b; end
do local a = 1 end
do do end end
do do end; end
do do do end end end
do do do end; end; end
do do do return end end end
do end do -- FAIL
do end end -- FAIL
do return end
do return return end -- FAIL
do break end -- FAIL
while -- FAIL
while do -- FAIL
while = -- FAIL
while 1 do -- FAIL
while 1 do end
while 1 do local a end
while 1 do local a local b end
while 1 do local a; local b; end
while 1 do 2 end -- FAIL
while 1 do "foo" end -- FAIL
while true do end
while 1 do ; end -- FAIL
while 1 do while -- FAIL
while 1 end -- FAIL
while 1 2 do -- FAIL
while 1 = 2 do -- FAIL
while 1 do return end
while 1 do return return end -- FAIL
while 1 do do end end
while 1 do do return end end
while 1 do break end
while 1 do break break end -- FAIL
while 1 do do break end end
repeat -- FAIL
repeat until -- FAIL
repeat until 0
repeat until false
repeat until local -- FAIL
repeat end -- FAIL
repeat 1 -- FAIL
repeat = -- FAIL
repeat local a until 1
repeat local a local b until 0
repeat local a; local b; until 0
repeat ; until 1 -- FAIL
repeat 2 until 1 -- FAIL
repeat "foo" until 1 -- FAIL
repeat return until 0
repeat return return until 0 -- FAIL
repeat break until 0
repeat break break until 0 -- FAIL
repeat do end until 0
repeat do return end until 0
repeat do break end until 0
for -- FAIL
for do -- FAIL
for end -- FAIL
for 1 -- FAIL
for a -- FAIL
for true -- FAIL
for a, in -- FAIL
for a in -- FAIL
for a do -- FAIL
for a in do -- FAIL
for a in b do -- FAIL
for a in b end -- FAIL
for a in b, do -- FAIL
for a in b do end
for a in b do local a local b end
for a in b do local a; local b; end
for a in b do 1 end -- FAIL
for a in b do "foo" end -- FAIL
for a b in -- FAIL
for a, b, c in p do end
for a, b, c in p, q, r do end
for a in 1 do end
for a in true do end
for a in "foo" do end
for a in b do break end
for a in b do break break end -- FAIL
for a in b do return end
for a in b do return return end -- FAIL
for a in b do do end end
for a in b do do break end end
for a in b do do return end end
for = -- FAIL
for a = -- FAIL
for a, b = -- FAIL
for a = do -- FAIL
for a = 1, do -- FAIL
for a = p, q, do -- FAIL
for a = p q do -- FAIL
for a = b do end -- FAIL
for a = 1, 2, 3, 4 do end -- FAIL
for a = p, q do end
for a = 1, 2 do end
for a = 1, 2 do local a local b end
for a = 1, 2 do local a; local b; end
for a = 1, 2 do 3 end -- FAIL
for a = 1, 2 do "foo" end -- FAIL
for a = p, q, r do end
for a = 1, 2, 3 do end
for a = p, q do break end
for a = p, q do break break end -- FAIL
for a = 1, 2 do return end
for a = 1, 2 do return return end -- FAIL
for a = p, q do do end end
for a = p, q do do break end end
for a = p, q do do return end end
break -- FAIL
return
return;
return return -- FAIL
return 1
return local -- FAIL
return "foo"
return 1, -- FAIL
return 1,2,3
return a,b,c,d
return 1,2;
return ...
return 1,a,...
if -- FAIL
elseif -- FAIL
else -- FAIL
then -- FAIL
if then -- FAIL
if 1 -- FAIL
if 1 then -- FAIL
if 1 else -- FAIL
if 1 then else -- FAIL
if 1 then elseif -- FAIL
if 1 then end
if 1 then local a end
if 1 then local a local b end
if 1 then local a; local b; end
if 1 then else end
if 1 then local a else local b end
if 1 then local a; else local b; end
if 1 then elseif 2 -- FAIL
if 1 then elseif 2 then -- FAIL
if 1 then elseif 2 then end
if 1 then local a elseif 2 then local b end
if 1 then local a; elseif 2 then local b; end
if 1 then elseif 2 then else end
if 1 then else if 2 then end end
if 1 then else if 2 then end -- FAIL
if 1 then break end -- FAIL
if 1 then return end
if 1 then return return end -- FAIL
if 1 then end; if 1 then end;
function -- FAIL
function 1 -- FAIL
function end -- FAIL
function a -- FAIL
function a end -- FAIL
function a( end -- FAIL
function a() end
function a(1 -- FAIL
function a("foo" -- FAIL
function a(p -- FAIL
function a(p,) -- FAIL
function a(p q -- FAIL
function a(p) end
function a(p,q,) end -- FAIL
function a(p,q,r) end
function a(p,q,1 -- FAIL
function a(p) do -- FAIL
function a(p) 1 end -- FAIL
function a(p) return end
function a(p) break end -- FAIL
function a(p) return return end -- FAIL
function a(p) do end end
function a.( -- FAIL
function a.1 -- FAIL
function a.b() end
function a.b, -- FAIL
function a.b.( -- FAIL
function a.b.c.d() end
function a: -- FAIL
function a:1 -- FAIL
function a:b() end
function a:b: -- FAIL
function a:b. -- FAIL
function a.b.c:d() end
function a(...) end
function a(..., -- FAIL
function a(p,...) end
function a(p,q,r,...) end
function a() local a local b end
function a() local a; local b; end
function a() end; function a() end;
local function -- FAIL
local function 1 -- FAIL
local function end -- FAIL
local function a -- FAIL
local function a end -- FAIL
local function a( end -- FAIL
local function a() end
local function a(1 -- FAIL
local function a("foo" -- FAIL
local function a(p -- FAIL
local function a(p,) -- FAIL
local function a(p q -- FAIL
local function a(p) end
local function a(p,q,) end -- FAIL
local function a(p,q,r) end
local function a(p,q,1 -- FAIL
local function a(p) do -- FAIL
local function a(p) 1 end -- FAIL
local function a(p) return end
local function a(p) break end -- FAIL
local function a(p) return return end -- FAIL
local function a(p) do end end
local function a. -- FAIL
local function a: -- FAIL
local function a(...) end
local function a(..., -- FAIL
local function a(p,...) end
local function a(p,q,r,...) end
local function a() local a local b end
local function a() local a; local b; end
local function a() end; local function a() end;
a -- FAIL
a, -- FAIL
a,b,c -- FAIL
a,b = -- FAIL
a = 1
a = 1,2,3
a,b,c = 1
a,b,c = 1,2,3
a.b = 1
a.b.c = 1
a[b] = 1
a[b][c] = 1
a.b[c] = 1
a[b].c = 1
0 = -- FAIL
"foo" = -- FAIL
true = -- FAIL
(a) = -- FAIL
{} = -- FAIL
a:b() = -- FAIL
a() = -- FAIL
a.b:c() = -- FAIL
a[b]() = -- FAIL
a = a b -- FAIL
a = 1 2 -- FAIL
a = a = 1 -- FAIL
a( -- FAIL
a()
a(1)
a(1,) -- FAIL
a(1,2,3)
1() -- FAIL
a()()
a.b()
a[b]()
a.1 -- FAIL
a.b -- FAIL
a[b] -- FAIL
a.b.( -- FAIL
a.b.c()
a[b][c]()
a[b].c()
a.b[c]()
a:b()
a:b -- FAIL
a:1 -- FAIL
a.b:c()
a[b]:c()
a:b: -- FAIL
a:b():c()
a:b().c[d]:e()
a:b()[c].d:e()
(a)()
()() -- FAIL
(1)()
("foo")()
(true)()
(a)()()
(a.b)()
(a[b])()
(a).b()
(a)[b]()
(a):b()
(a).b[c]:d()
(a)[b].c:d()
(a):b():c()
(a):b().c[d]:e()
(a):b()[c].d:e()
a"foo"
a[[foo]]
a.b"foo"
a[b]"foo"
a:b"foo"
a{}
a.b{}
a[b]{}
a:b{}
a()"foo"
a"foo"()
a"foo".b()
a"foo"[b]()
a"foo":c()
a"foo""bar"
a"foo"{}
(a):b"foo".c[d]:e"bar"
(a):b"foo"[c].d:e"bar"
a(){}
a{}()
a{}.b()
a{}[b]()
a{}:c()
a{}"foo"
a{}{}
(a):b{}.c[d]:e{}
(a):b{}[c].d:e{}
a = -- FAIL
a = a
a = nil
a = false
a = 1
a = "foo"
a = [[foo]]
a = {}
a = (a)
a = (nil)
a = (true)
a = (1)
a = ("foo")
a = ([[foo]])
a = ({})
a = a.b
a = a.b. -- FAIL
a = a.b.c
a = a:b -- FAIL
a = a[b]
a = a[1]
a = a["foo"]
a = a[b][c]
a = a.b[c]
a = a[b].c
a = (a)[b]
a = (a).c
a = () -- FAIL
a = a()
a = a.b()
a = a[b]()
a = a:b()
a = (a)()
a = (a).b()
a = (a)[b]()
a = (a):b()
a = a"foo"
a = a{}
a = function -- FAIL
a = function 1 -- FAIL
a = function a -- FAIL
a = function end -- FAIL
a = function( -- FAIL
a = function() end
a = function(1 -- FAIL
a = function(p) end
a = function(p,) -- FAIL
a = function(p q -- FAIL
a = function(p,q,r) end
a = function(p,q,1 -- FAIL
a = function(...) end
a = function(..., -- FAIL
a = function(p,...) end
a = function(p,q,r,...) end
a = ...
a = a, b, ...
a = (...)
a = ..., 1, 2
a = function() return ... end -- FAIL
a = -10
a = -"foo"
a = -a
a = -nil
a = -true
a = -{}
a = -function() end
a = -a()
a = -(a)
a = - -- FAIL
a = not 10
a = not "foo"
a = not a
a = not nil
a = not true
a = not {}
a = not function() end
a = not a()
a = not (a)
a = not -- FAIL
a = #10
a = #"foo"
a = #a
a = #nil
a = #true
a = #{}
a = #function() end
a = #a()
a = #(a)
a = # -- FAIL
a = 1 + 2; a = 1 - 2
a = 1 * 2; a = 1 / 2
a = 1 ^ 2; a = 1 % 2
a = 1 .. 2
a = 1 + -- FAIL
a = 1 .. -- FAIL
a = 1 * / -- FAIL
a = 1 + -2; a = 1 - -2
a = 1 * - -- FAIL
a = 1 * not 2; a = 1 / not 2
a = 1 / not -- FAIL
a = 1 * #"foo"; a = 1 / #"foo"
a = 1 / # -- FAIL
a = 1 + 2 - 3 * 4 / 5 % 6 ^ 7
a = ((1 + 2) - 3) * (4 / (5 % 6 ^ 7))
a = (1 + (2 - (3 * (4 / (5 % 6 ^ ((7)))))))
a = ((1 -- FAIL
a = ((1 + 2) -- FAIL
a = 1) -- FAIL
a = a + b - c
a = "foo" + "bar"
a = "foo".."bar".."baz"
a = true + false - nil
a = {} * {}
a = function() end / function() end
a = a() ^ b()
a = ... % ...
a = 1 == 2; a = 1 ~= 2
a = 1 < 2; a = 1 <= 2
a = 1 > 2; a = 1 >= 2
a = 1 < 2 < 3
a = 1 >= 2 >= 3
a = 1 == -- FAIL
a = ~= 2 -- FAIL
a = "foo" == "bar"
a = "foo" > "bar"
a = a ~= b
a = true == false
a = 1 and 2; a = 1 or 2
a = 1 and -- FAIL
a = or 1 -- FAIL
a = 1 and 2 and 3
a = 1 or 2 or 3
a = 1 and 2 or 3
a = a and b or c
a = a() and (b)() or c.d
a = "foo" and "bar"
a = true or false
a = {} and {} or {}
a = (1) and ("foo") or (nil)
a = function() end == function() end
a = function() end or function() end
a = { -- FAIL
a = {}
a = {,} -- FAIL
a = {;} -- FAIL
a = {,,} -- FAIL
a = {;;} -- FAIL
a = {{ -- FAIL
a = {{{}}}
a = {{},{},{{}},}
a = { 1 }
a = { 1, }
a = { 1; }
a = { 1, 2 }
a = { a, b, c, }
a = { true; false, nil; }
a = { a.b, a[b]; a:c(), }
a = { 1 + 2, a > b, "a" or "b" }
a = { a=1, }
a = { a=1, b="foo", c=nil }
a = { a -- FAIL
a = { a= -- FAIL
a = { a=, -- FAIL
a = { a=; -- FAIL
a = { 1, a="foo" -- FAIL
a = { 1, a="foo"; b={}, d=true; }
a = { [ -- FAIL
a = { [1 -- FAIL
a = { [1] -- FAIL
a = { [a]= -- FAIL
a = { ["foo"]="bar" }
a = { [1]=a, [2]=b, }
a = { true, a=1; ["foo"]="bar", }
]=]
package.path = "../?.lua;" .. package.path
local util = require'Util'
local Parser = require'ParseLua'
local Format_Mini = require'FormatMini'
local f = io.open("tmp", 'wb')
f:write(source)
f:close()
for w in io.lines("tmp") do
--print(w)
local success, ast = Parser.ParseLua(w)
if w:find("FAIL") then
if success then
print("ERROR PARSING LINE:")
print("Should fail: true. Did fail: " .. tostring(not success))
--print("Message: " .. ast)
print("Line: " .. w)
else
--print("Suceeded!")
end
else
if not success then
print("ERROR PARSING LINE:")
print("Should fail: false. Did fail: " .. tostring(not success))
print("Message: " .. ast)
print("Line: " .. w)
else
--print("Suceeded!")
end
end
end
print"Done!"
os.remove("tmp")

480
lib/lexer.lua
View file

@ -1,480 +0,0 @@
--[[
This file is a part of Penlight (set of pure Lua libraries) - https://github.com/stevedonovan/Penlight
LICENSE :
Copyright (C) 2009 Steve Donovan, David Manura.
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in the
Software without restriction, including without limitation the rights to use, copy,
modify, merge, publish, distribute, sublicense, and/or sell copies of the Software,
and to permit persons to whom the Software is furnished to do so, subject to the
following conditions:
The above copyright notice and this permission notice shall be included in all copies
or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF
CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE
OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
]]
--- Lexical scanner for creating a sequence of tokens from text.
-- `lexer.scan(s)` returns an iterator over all tokens found in the
-- string `s`. This iterator returns two values, a token type string
-- (such as 'string' for quoted string, 'iden' for identifier) and the value of the
-- token.
--
-- Versions specialized for Lua and C are available; these also handle block comments
-- and classify keywords as 'keyword' tokens. For example:
--
-- > s = 'for i=1,n do'
-- > for t,v in lexer.lua(s) do print(t,v) end
-- keyword for
-- iden i
-- = =
-- number 1
-- , ,
-- iden n
-- keyword do
--
-- See the Guide for further @{06-data.md.Lexical_Scanning|discussion}
-- @module pl.lexer
local yield,wrap = coroutine.yield,coroutine.wrap
local strfind = string.find
local strsub = string.sub
local append = table.insert
local function assert_arg(idx,val,tp)
if type(val) ~= tp then
error("argument "..idx.." must be "..tp, 2)
end
end
local lexer = {}
local NUMBER1 = '^[%+%-]?%d+%.?%d*[eE][%+%-]?%d+'
local NUMBER2 = '^[%+%-]?%d+%.?%d*'
local NUMBER3 = '^0x[%da-fA-F]+'
local NUMBER4 = '^%d+%.?%d*[eE][%+%-]?%d+'
local NUMBER5 = '^%d+%.?%d*'
local IDEN = '^[%a_][%w_]*'
local WSPACE = '^%s+'
local STRING0 = [[^(['\"]).-\\%1]]
local STRING1 = [[^(['\"]).-[^\]%1]]
local STRING3 = "^((['\"])%2)" -- empty string
local PREPRO = '^#.-[^\\]\n'
local plain_matches,lua_matches,cpp_matches,lua_keyword,cpp_keyword
local function tdump(tok)
return yield(tok,tok)
end
local function ndump(tok,options)
if options and options.number then
tok = tonumber(tok)
end
return yield("number",tok)
end
-- regular strings, single or double quotes; usually we want them
-- without the quotes
local function sdump(tok,options)
if options and options.string then
tok = tok:sub(2,-2)
end
return yield("string",tok)
end
-- long Lua strings need extra work to get rid of the quotes
local function sdump_l(tok,options)
if options and options.string then
tok = tok:sub(3,-3)
end
return yield("string",tok)
end
local function chdump(tok,options)
if options and options.string then
tok = tok:sub(2,-2)
end
return yield("char",tok)
end
local function cdump(tok)
return yield('comment',tok)
end
local function wsdump (tok)
return yield("space",tok)
end
local function pdump (tok)
return yield('prepro',tok)
end
local function plain_vdump(tok)
return yield("iden",tok)
end
local function lua_vdump(tok)
if lua_keyword[tok] then
return yield("keyword",tok)
else
return yield("iden",tok)
end
end
local function cpp_vdump(tok)
if cpp_keyword[tok] then
return yield("keyword",tok)
else
return yield("iden",tok)
end
end
--- create a plain token iterator from a string or file-like object.
-- @param s the string
-- @param matches an optional match table (set of pattern-action pairs)
-- @param filter a table of token types to exclude, by default {space=true}
-- @param options a table of options; by default, {number=true,string=true},
-- which means convert numbers and strip string quotes.
function lexer.scan (s,matches,filter,options)
--assert_arg(1,s,'string')
local file = type(s) ~= 'string' and s
filter = filter or {space=true}
options = options or {number=true,string=true}
if filter then
if filter.space then filter[wsdump] = true end
if filter.comments then
filter[cdump] = true
end
end
if not matches then
if not plain_matches then
plain_matches = {
{WSPACE,wsdump},
{NUMBER3,ndump},
{IDEN,plain_vdump},
{NUMBER1,ndump},
{NUMBER2,ndump},
{STRING3,sdump},
{STRING0,sdump},
{STRING1,sdump},
{'^.',tdump}
}
end
matches = plain_matches
end
local function lex ()
local i1,i2,idx,res1,res2,tok,pat,fun,capt
local line = 1
if file then s = file:read()..'\n' end
local sz = #s
local idx = 1
--print('sz',sz)
while true do
for _,m in ipairs(matches) do
pat = m[1]
fun = m[2]
i1,i2 = strfind(s,pat,idx)
if i1 then
tok = strsub(s,i1,i2)
idx = i2 + 1
if not (filter and filter[fun]) then
lexer.finished = idx > sz
res1,res2 = fun(tok,options)
end
if res1 then
local tp = type(res1)
-- insert a token list
if tp=='table' then
yield('','')
for _,t in ipairs(res1) do
yield(t[1],t[2])
end
elseif tp == 'string' then -- or search up to some special pattern
i1,i2 = strfind(s,res1,idx)
if i1 then
tok = strsub(s,i1,i2)
idx = i2 + 1
yield('',tok)
else
yield('','')
idx = sz + 1
end
--if idx > sz then return end
else
yield(line,idx)
end
end
if idx > sz then
if file then
--repeat -- next non-empty line
line = line + 1
s = file:read()
if not s then return end
--until not s:match '^%s*$'
s = s .. '\n'
idx ,sz = 1,#s
break
else
return
end
else break end
end
end
end
end
return wrap(lex)
end
local function isstring (s)
return type(s) == 'string'
end
--- insert tokens into a stream.
-- @param tok a token stream
-- @param a1 a string is the type, a table is a token list and
-- a function is assumed to be a token-like iterator (returns type & value)
-- @param a2 a string is the value
function lexer.insert (tok,a1,a2)
if not a1 then return end
local ts
if isstring(a1) and isstring(a2) then
ts = {{a1,a2}}
elseif type(a1) == 'function' then
ts = {}
for t,v in a1() do
append(ts,{t,v})
end
else
ts = a1
end
tok(ts)
end
--- get everything in a stream upto a newline.
-- @param tok a token stream
-- @return a string
function lexer.getline (tok)
local t,v = tok('.-\n')
return v
end
--- get current line number. <br>
-- Only available if the input source is a file-like object.
-- @param tok a token stream
-- @return the line number and current column
function lexer.lineno (tok)
return tok(0)
end
--- get the rest of the stream.
-- @param tok a token stream
-- @return a string
function lexer.getrest (tok)
local t,v = tok('.+')
return v
end
--- get the Lua keywords as a set-like table.
-- So <code>res["and"]</code> etc would be <code>true</code>.
-- @return a table
function lexer.get_keywords ()
if not lua_keyword then
lua_keyword = {
["and"] = true, ["break"] = true, ["do"] = true,
["else"] = true, ["elseif"] = true, ["end"] = true,
["false"] = true, ["for"] = true, ["function"] = true,
["if"] = true, ["in"] = true, ["local"] = true, ["nil"] = true,
["not"] = true, ["or"] = true, ["repeat"] = true,
["return"] = true, ["then"] = true, ["true"] = true,
["until"] = true, ["while"] = true
}
end
return lua_keyword
end
--- create a Lua token iterator from a string or file-like object.
-- Will return the token type and value.
-- @param s the string
-- @param filter a table of token types to exclude, by default {space=true,comments=true}
-- @param options a table of options; by default, {number=true,string=true},
-- which means convert numbers and strip string quotes.
function lexer.lua(s,filter,options)
filter = filter or {space=true,comments=true}
lexer.get_keywords()
if not lua_matches then
lua_matches = {
{WSPACE,wsdump},
{NUMBER3,ndump},
{IDEN,lua_vdump},
{NUMBER4,ndump},
{NUMBER5,ndump},
{STRING3,sdump},
{STRING0,sdump},
{STRING1,sdump},
{'^%-%-%[%[.-%]%]',cdump},
{'^%-%-.-\n',cdump},
{'^%[%[.-%]%]',sdump_l},
{'^==',tdump},
{'^~=',tdump},
{'^<=',tdump},
{'^>=',tdump},
{'^%.%.%.',tdump},
{'^%.%.',tdump},
{'^.',tdump}
}
end
return lexer.scan(s,lua_matches,filter,options)
end
--- create a C/C++ token iterator from a string or file-like object.
-- Will return the token type type and value.
-- @param s the string
-- @param filter a table of token types to exclude, by default {space=true,comments=true}
-- @param options a table of options; by default, {number=true,string=true},
-- which means convert numbers and strip string quotes.
function lexer.cpp(s,filter,options)
filter = filter or {comments=true}
if not cpp_keyword then
cpp_keyword = {
["class"] = true, ["break"] = true, ["do"] = true, ["sizeof"] = true,
["else"] = true, ["continue"] = true, ["struct"] = true,
["false"] = true, ["for"] = true, ["public"] = true, ["void"] = true,
["private"] = true, ["protected"] = true, ["goto"] = true,
["if"] = true, ["static"] = true, ["const"] = true, ["typedef"] = true,
["enum"] = true, ["char"] = true, ["int"] = true, ["bool"] = true,
["long"] = true, ["float"] = true, ["true"] = true, ["delete"] = true,
["double"] = true, ["while"] = true, ["new"] = true,
["namespace"] = true, ["try"] = true, ["catch"] = true,
["switch"] = true, ["case"] = true, ["extern"] = true,
["return"] = true,["default"] = true,['unsigned'] = true,['signed'] = true,
["union"] = true, ["volatile"] = true, ["register"] = true,["short"] = true,
}
end
if not cpp_matches then
cpp_matches = {
{WSPACE,wsdump},
{PREPRO,pdump},
{NUMBER3,ndump},
{IDEN,cpp_vdump},
{NUMBER4,ndump},
{NUMBER5,ndump},
{STRING3,sdump},
{STRING1,chdump},
{'^//.-\n',cdump},
{'^/%*.-%*/',cdump},
{'^==',tdump},
{'^!=',tdump},
{'^<=',tdump},
{'^>=',tdump},
{'^->',tdump},
{'^&&',tdump},
{'^||',tdump},
{'^%+%+',tdump},
{'^%-%-',tdump},
{'^%+=',tdump},
{'^%-=',tdump},
{'^%*=',tdump},
{'^/=',tdump},
{'^|=',tdump},
{'^%^=',tdump},
{'^::',tdump},
{'^.',tdump}
}
end
return lexer.scan(s,cpp_matches,filter,options)
end
--- get a list of parameters separated by a delimiter from a stream.
-- @param tok the token stream
-- @param endtoken end of list (default ')'). Can be '\n'
-- @param delim separator (default ',')
-- @return a list of token lists.
function lexer.get_separated_list(tok,endtoken,delim)
endtoken = endtoken or ')'
delim = delim or ','
local parm_values = {}
local level = 1 -- used to count ( and )
local tl = {}
local function tappend (tl,t,val)
val = val or t
append(tl,{t,val})
end
local is_end
if endtoken == '\n' then
is_end = function(t,val)
return t == 'space' and val:find '\n'
end
else
is_end = function (t)
return t == endtoken
end
end
local token,value
while true do
token,value=tok()
if not token then return nil,'EOS' end -- end of stream is an error!
if is_end(token,value) and level == 1 then
append(parm_values,tl)
break
elseif token == '(' then
level = level + 1
tappend(tl,'(')
elseif token == ')' then
level = level - 1
if level == 0 then -- finished with parm list
append(parm_values,tl)
break
else
tappend(tl,')')
end
elseif token == delim and level == 1 then
append(parm_values,tl) -- a new parm
tl = {}
else
tappend(tl,token,value)
end
end
return parm_values,{token,value}
end
--- get the next non-space token from the stream.
-- @param tok the token stream.
function lexer.skipws (tok)
local t,v = tok()
while t == 'space' do
t,v = tok()
end
return t,v
end
local skipws = lexer.skipws
--- get the next token, which must be of the expected type.
-- Throws an error if this type does not match!
-- @param tok the token stream
-- @param expected_type the token type
-- @param no_skip_ws whether we should skip whitespace
function lexer.expecting (tok,expected_type,no_skip_ws)
assert_arg(1,tok,'function')
assert_arg(2,expected_type,'string')
local t,v
if no_skip_ws then
t,v = tok()
else
t,v = skipws(tok)
end
if t ~= expected_type then error ("expecting "..expected_type,2) end
return v
end
return lexer

21
lib/table.lua
View file

@ -1,8 +1,8 @@
--[[
Lua table utilities by Thomas99.
Table utility by Thomas99.
LICENSE :
Copyright (c) 2014 Thomas99
Copyright (c) 2015 Thomas99
This software is provided 'as-is', without any express or implied warranty.
In no event will the authors be held liable for any damages arising from the
@ -23,6 +23,13 @@ to the following restrictions:
3. This notice may not be removed or altered from any source distribution.
]]
-- Diverses fonctions en rapport avec les tables.
-- v0.1.0
--
-- Changements :
-- - v0.1.0 :
-- Première version versionnée. Il a dû se passer des trucs avant mais j'ai pas noté :p
-- Copie récursivement la table t dans la table dest (ou une table vide si non précisé) et la retourne
-- replace (false) : indique si oui ou non, les clefs existant déjà dans dest doivent être écrasées par celles de t
-- metatable (true) : copier ou non également les metatables
@ -79,16 +86,6 @@ function table.isIn(table, value)
return false
end
-- retourne true si la clé key est dans la table
function table.hasKey(table, key)
for k,_ in pairs(table) do
if k == key then
return true
end
end
return false
end
-- retourne la longueur exacte d'une table (fonctionne sur les tables à clef)
function table.len(t)
local len=0

180
lune.lune
View file

@ -1,180 +0,0 @@
#!/usr/bin/lua
--[[
Lune language & compiler by Thomas99.
LICENSE :
Copyright (c) 2014 Thomas99
This software is provided 'as-is', without any express or implied warranty.
In no event will the authors be held liable for any damages arising from the
use of this software.
Permission is granted to anyone to use this software for any purpose, including
commercial applications, and to alter it and redistribute it freely, subject
to the following restrictions:
1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software in a
product, an acknowledgment in the product documentation would be appreciated
but is not required.
2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.
]]
#include("lib/lexer.lua")
#include("lib/table.lua")
local lune = {}
lune.VERSION = "0.0.1"
lune.syntax = {
affectation = { ["+"] = "= %s +", ["-"] = "= %s -", ["*"] = "= %s *", ["/"] = "= %s /",
["^"] = "= %s ^", ["%"] = "= %s %%", [".."] = "= %s .." },
incrementation = { ["+"] = " = %s + 1" , ["-"] = " = %s - 1" },
}
-- Preprocessor
function lune.preprocess(input, args)
-- generate preprocessor
local preprocessor = "return function()\n"
local lines = {}
for line in (input.."\n"):gmatch("(.-)\n") do
table.insert(lines, line)
if line:sub(1,1) == "#" then
-- exclude shebang
if not (line:sub(1,2) == "#!" and #lines ==1) then
preprocessor ..= line:sub(2) .. "\n"
else
preprocessor ..= "output ..= lines[" .. #lines .. "] .. \"\\n\"\n"
end
else
preprocessor ..= "output ..= lines[" .. #lines .. "] .. \"\\n\"\n"
end
end
preprocessor ..= "return output\nend"
-- make preprocessor environement
local env = table.copy(_G)
env.lune = lune
env.output = ""
env.include = function(file)
local f = io.open(file)
if not f then error("can't open the file to include") end
local filename = file:match("([^%/%\\]-)%.[^%.]-$")
env.output ..=
"-- INCLUSION OF FILE \""..file.."\" --\n"..
"local function _()\n"..
f:read("*a").."\n"..
"end\n"..
"local "..filename.." = _() or "..filename.."\n"..
"-- END OF INCLUDSION OF FILE \""..file.."\" --\n"
f:close()
end
env.rawInclude = function(file)
local f = io.open(file)
if not f then error("can't open the file to raw include") end
env.output ..= f:read("*a").."\n"
f:close()
end
env.print = function(...)
env.output ..= table.concat({...}, "\t") .. "\n"
end
env.args = args or {}
env.lines = lines
-- load preprocessor
local preprocess, err = load(lune.compile(preprocessor), "Preprocessor", nil, env)
if not preprocess then error("Error while creating preprocessor :\n" .. err) end
-- execute preprocessor
local success, output = pcall(preprocess())
if not success then error("Error while preprocessing file :\n" .. output .. "\nWith preprocessor : \n" .. preprocessor) end
return output
end
-- Compiler
function lune.compile(input)
local output = ""
local last = {}
for t,v in lexer.lua(input, {}, {}) do
local toInsert = v
-- affectation
if t == "=" then
if table.hasKey(lune.syntax.affectation, last.token) then
toInsert = string.format(lune.syntax.affectation[last.token], last.varName)
output = output:sub(1, -1 -#last.token) -- remove token before =
end
end
-- self-incrementation
if table.hasKey(lune.syntax.incrementation, t) and t == last.token then
toInsert = string.format(lune.syntax.incrementation[last.token], last.varName)
output = output:sub(1, -#last.token*2) -- remove token ++/--
end
-- reconstitude full variable name (ex : ith.game.camera)
if t == "iden" then
if last.token == "." then
last.varName ..= "." .. v
else
last.varName = v
end
end
last[t] = v
last.token = t
last.value = v
output ..= toInsert
end
return output
end
-- Preprocess & compile
function lune.make(code, args)
local preprocessed = lune.preprocess(code, args or {})
local output = lune.compile(preprocessed)
return output
end
-- Standalone mode
if debug.getinfo(3) == nil and arg then
-- Check args
if #arg < 1 then
print("Lune version "..lune.VERSION.." by Thomas99")
print("Command-line usage :")
print("lua lune.lua <filename> [preprocessor arguments]")
return lune
end
-- Parse args
local inputFilePath = arg[1]
local args = {}
-- Parse compilation args
for i=2, #arg, 1 do
if arg[i]:sub(1,2) == "--" then
args[arg[i]:sub(3)] = arg[i+1]
i = i +1 -- skip argument value
end
end
-- Open & read input file
local inputFile, err = io.open(inputFilePath, "r")
if not inputFile then error("Error while opening input file : "..err) end
local input = inputFile:read("*a")
inputFile:close()
-- End
print(lune.make(input, args))
end
return lune

110
tests/test.lua
View file

@ -1,17 +1,17 @@
print("=====================")
print("|| LUNE TESTS ||")
print("=====================")
print("========================")
print("|| CANDRAN TESTS ||")
print("========================")
local lune = dofile(arg[1] or "../build/lune.lua")
local candran = dofile(arg[1] or "../build/candran.lua")
-- test helper
local results = {} -- tests result
local function test(name, luneCode, result, args)
local function test(name, candranCode, result, args)
results[name] = { result = "not finished", message = "no info" }
local self = results[name]
-- make code
local success, code = pcall(lune.make, luneCode, args)
local success, code = pcall(candran.make, candranCode, args)
if not success then
self.result = "error"
self.message = "error while making code :\n"..code
@ -70,12 +70,12 @@ test("preprocessor print function", [[
#print("local a = true")
return a
]], true)
test("preprocessor include function", [[
#include("toInclude.lua")
return a
test("preprocessor import function", [[
#import("toInclude")
return toInclude
]], 5)
test("preprocessor rawInclude function", "a = [[\n#rawInclude('toInclude.lua')\n]]\nreturn a",
"a = 5\n")
test("preprocessor include function", "a = [[\n#include('toInclude.lua')\n]]\nreturn a",
"local a = 5\nreturn a\n")
test("+=", [[
local a = 5
@ -113,16 +113,84 @@ a ..= " world"
return a
]], "hello world")
test("++", [[
local a = 5
a++
return a
]], 6)
test("--", [[
local a = 5
a--
return a
]], 4)
test("decorator", [[
local a = function(func)
local wrapper = function(...)
local b = func(...)
return b + 5
end
return wrapper
end
@a
function c(nb)
return nb^2
end
return c(5)
]], 30)
test("decorator with arguments", [[
local a = function(add)
local b = function(func)
local wrapper = function(...)
local c = func(...)
return c + add
end
return wrapper
end
return b
end
@a(10)
function d(nb)
return nb^2
end
return d(5)
]], 35)
test("multiple decorators", [[
local a = function(func)
local wrapper = function(...)
local b = func(...)
return b + 5
end
return wrapper
end
local c = function(func)
local wrapper = function(...)
local d = func(...)
return d * 2
end
return wrapper
end
@a
@c
function e(nb)
return nb^2
end
return e(5)
]], 55)
test("multiple decorators with arguments", [[
local a = function(func)
local wrapper = function(...)
local b = func(...)
return b + 5
end
return wrapper
end
local c = function(mul)
local d = function(func)
local wrapper = function(...)
local e = func(...)
return e * mul
end
return wrapper
end
return d
end
@a
@c(3)
function f(nb)
return nb^2
end
return f(5)
]], 80)
-- results
print("=====================")

3
tests/toInclude.lua
View file

@ -1 +1,2 @@
a = 5
local a = 5
return a