Customize metaprocessor operators for DSL usage

[stefanos82]

After studying the examples/dsl2.lua's logic, I have been thinking.

Is it possible to implement a custom metaprocessor behavior and use that in place of standard one for a DSL I would like to design?

For instance, I might feel more comfortable with a syntax closer to C-domain languages, such as PHP and would like to borrow a couple of specific features from it and blend them with existing Lua-family syntax.

I will write an example to make things clearer, but don't take the pseudo-logic for granted; it would be used for demonstrative purposes:

From Lua-like syntax

## local function mul(res, a, b)
  #[res]# = #[a]# * #[b]#
## end

local a, b = 2, 3
local res = 0
#[mul]#(res, a, b)
print(res)

to something like the following:

v1

<?nl local function mul(res, a, b) ?>
  ${res} = ${a} * ${b}
<?nl end ?>

local a, b = 2, 3
local res = 0
${mul}(res, a, b)
print(res)

v2

<?nl
local function mul(res, a, b)
  ${res} = ${a} * ${b}
end
?>

local a, b = 2, 3
local res = 0
${mul}(res, a, b)
print(res)

[edubart]

The v1 is possible and is easy to do because you just did some little cosmetic changes (replacing tokens):

First create a test.nelua file, contained the required code to make that work:

##[[
-- Load the language syntax definitions
local syntaxdefs = require 'nelua.syntaxdefs'
-- Load the AST builder/parser
local aster = require 'nelua.aster'
local grammar = syntaxdefs.grammar
-- Patch the grammar with the new syntax
grammar = grammar:gsub(
  "PREPROCESS%s*<%-%-[^\n]+\n",
  "PREPROCESS <-- `<?` 'nl' {(!`?>` .)*} @`?>`\n")
grammar = grammar:gsub(
  "PreprocessExpr%s*<%=%=[^\n]+\n",
  "PreprocessExpr  <== `${` {@expr->0} @`}`\n")
-- Register the compiler
aster.register_syntax({
  extension = 'neluaphp',
  grammar = grammar,
  errors = syntaxdefs.errors,
  defs = syntaxdefs.defs,
})
]]

-- Load a file using new syntax.
-- Note that we must use relative path because we did not update the modules search path.
require './test.neluaphp'

Now create the test.neluaphp in the same folder:

<?nl local function mul(res, a, b) ?>
  ${res} = ${a} * ${b}
<?nl end ?>

local a, b = 2, 3
local res = 0
${mul}(res, a, b)
print(res)

And voila, works are you intended. I've choosen to patch the Nelua grammar with gsub to make this code small, but ideally you want to copy the whole grammar rules from syntaxdefs.lua and do the changes directly on the pasted rules.

The v2 has some issues in your design (using ${res} inside <?nl), so I did not make an example for it.

[stefanos82]

Oh wow, that as super easy lol!

The v2 has some issues in your design (using ${res} inside <?nl), so I did not make an example for it.

All I wanted from this was to access the Nelua's part from inside the metaprocessor without exiting it.

[stefanos82]

I have followed your advice and organized the code a bit; now it looks cleaner and more elegant:

pnel.nelua

##[[

local neluapeg = require './nelua.peg'

-- Load the language syntax definitions
local syntaxdefs = require 'nelua.syntaxdefs'
-- Load the AST builder/parser
local aster = require 'nelua.aster'

local grammar = neluapeg.grammar

-- Register the compiler
aster.register_syntax({
  extension = 'pnel',
  grammar = grammar,
  errors = syntaxdefs.errors,
  defs = syntaxdefs.defs,
})
]]

-- Load a file using new syntax.
-- Note that we must use relative path because we did not update the modules search path.
require './test.pnel'

test.pnel

<?nl local function mul(res, a, b) ?>
  ${res} = ${a} * ${b}
<?nl end ?>

local a, b = 2, 3
local res = 0
${mul}(res, a, b)
print(res)

nelua.peg

nelua = {}

local grammar = [==[
...
]==]

nelua.grammar = grammar

return nelua

My question to you @edubart is: how can I solve the issue with paths on the fly if I create a folder named grammars and place nelua.peg in it and require it without causing path breaking errors?