从 JavaScript 代码中获取令牌

Question

我想在 JavaScript/Node 中编写一个简单的解释器。我在生成令牌时遇到了障碍。

var code = 'if (a > 2 && b<4) c = 10;';

code.match(/\W+/g)
// [" (", " > ", " && ", "<", ") ", ";"]

code.match(/\w+/g)
// ["if", "a", "2", "b", "4", "elo"]

如图所示，W+ 让我获取特殊字符，w+ 让我获取单词。我想知道如何将它们放在一个数组中，如下所示：

// ["if", "(", "a", ">", "2", "&&", "b", "<", "4", ")", "c", "=", "10", ";"]

Answer 1

As shown, W+ lets me get special characters and w+ lets me get words. I wonder how to get those in one array, something like below:

试试这个

code.match(/\w+|\W+/g)

输出为

["if", " (", "a", " > ", "2", " && ", "b", "<", "4", ") ", "c", " = ", "10", ";"]

这也会 trim 代币

var tokens = code.match(/\w+|\W+/g).map(function(value){return value.trim()});

从 JavaScript 代码中获取令牌

Get tokens from a JavaScript code

javascript

regex

compiler-construction