规则语法说明

v1.0.6

Schema

• keyspace

  表空间 string

• class

  指定存储类 string

• parser

  原始数据格式类型 csv | text | json | grok | bitlog

• fields

  待解析数据数据结构，即在脚本钟可以直接使用的变量值。此设置需要配合parser使用。

  • csv

    <FIELD1>,<FIELD2>,<FIELD3>...

  • json

    <FIELD>

  • text

    <FIELD>

  • grok

    <SUBEXP1>,<SUBEXP2>,<SUBEXP3>...<EXP>

    需要先定义grok表达式

  • bitlog

    ID, FILE, LOGS

    固定三个变量，名称可以改变，但顺序必须是：主键，文件名称，日志内容（多行）

• fileds_type

  对应解析数据字段的类型，必须与fields一一对应。fields = json时此参数不起作用，可以设置的类型包括：

  INT 整形数字

  FLOAT 字符数字

  BOOL 布尔类型

  VAR 字符串类型

• ignore

  是否忽略规则 true | false

• mqltmpl

  自定义插入数据mql语句，支持prepare语句，如果设置为prepare语句，则需要配置mqltmpl_fields选项。

• mqltmpl_fields

  prepare模式插入需要的字段及顺序，字段名称按照逗号分割，配合mqltmpl选项一起使用。设置此选项则自动识别为prepare模式。可使用的名称可以来源于field也可以来源于var_<name>。

• cache_classes

  需要缓存到内存的类为classcache(<class>, <where>)访问，可指定多个class1, class2, class3...

• forward

  将原始数据转发到其他规则（在规则执行之前执行）。可以设置多个，用逗号分割。

• direct

  是否时直插数据模式 true | false。直插模式，lua中的代码将不会执行，直插模式需要设置mqltmpl和mqltmpl_fields。

• var_\

  当直插模式时direct = true，此变量用于mqltmpl_fields参数中可以使用的变量。字段名称可以是field也可以是常量，常量用单引号。变量支持类型如下：

  --var_<var_name>: field <field_name>
  --var_<var_name>: join <field_name1>, <field_name2>...
  --var_<var_name>: array <field_name1>, <field_name2>...
  --var_<var_name>: map <field_name1>:<field_name3>, <field_name2>:<field_name4>...
  

• retry

  此规则是否启用重试功能。true | false

• retry_total

  重试次数。不设置默认是：3

• retry_interval

  重试间隔（分钟）。不设置默认是：1

• log_level

  设置规则日志打印级别。默认是: trace。可选：trace | debug | info | warn | error | fatal

Functions

• global

  -- setclass: Set rule class
  -- params: <class string>
  -- return: None
  setclass('/test')
  
  -- setvar: Set gloabl cache var
  -- params: <key string>, <value any>
  -- return: None
  setvar('K1', 'V1')
  
  -- getvar: Get gloabl cache var
  -- params: <key string>
  -- return: <value any>
  local ret = getvar('ID1')
  
  -- retry: Retry this data, schema options must be set: --retry = true
  -- params: None
  -- return: None
  retry()
  
  -- lookup: Lookup extend info
  -- params: <extendName string>, <condition table>
  -- return: <result table>
  local ret = lookup('hostinfo', {key='mxsvr1'})
  
  -- setcache: Set table to LRU memory cache
  -- params: <key string>, <value table>
  -- return: None
  setcache('key1', {id='ID1', name='NAME1'})
  
  -- getcache: Set table from LRU memory cache, mql query if not exists and store to LRU memory cache
  -- params: <mql string>, <key string>
  -- return: <result table>
  local ret = getcache("select * from /test where id = 'ID1'", 'key1')
  
  -- gettimekey: Get odb 'day' and 'vtime' by timestamp
  -- params: <time timestamp (seconds or millionseconds)>, <isday bool (true: fill with current time)>
  -- return: <day string>, <vtime timestamp>
  local ret = gettimekey(1590994631)
  
  -- getcache: Get attributes of protocol message
  -- params: None
  -- return: <result table>
  local ret = getattr()
  
  -- getcache: Get hostname of protocol message
  -- params: None
  -- return: <result string>
  local ret = hostname()
  
  -- getcache: Get ipinfo of protocol message
  -- params: None
  -- return: <result table>
  local ret = ipinfo()
  
  -- dial: Send socket message
  -- params: <network string>, <address string>, <message string>
  -- return: None
  dial('udp', '127.0.0.1:514', 'hello')
  
  -- forward: Forward data to other rule
  -- params: <rule string>, <msg string>
  -- return: None
  forward('/matrix/rules/relation', '{"subject":"A", "predicate":"contain", "object":"B"}')
  
  -- cforward: Compress forward data to other rule
  -- params: <rule string>, <key string>, <msg string>, <compress number>
  -- return: None
  cforward('/matrix/rules/test', 'ID1', 'Test message', 5)
  
  -- dforward: Delay forward data to other rule
  -- params: <rule string>, <key string>, <msg string>, <delay number>, <times number>
  -- return: None
  dforward('/matrix/rules/test', 'ID1', 'Test message', 30, 3)
  
  -- classcache: Query class cache by conditions
  -- params: <class string>, <where string>
  -- where: Query logic expression
  --        ||
  --        &&
  --        = != > < >= <= ~= !~= is isnot contains
  --        + -
  --        * / %
  --        ^
  --        ( )
  -- return: <result table>
  local ret = classcache('/test', "id = 'ID1'")
  

• strings

  -- split: Split string
  -- params: <s string>, <sep string>
  -- return: <result table>
  local ret = strings.split('a,b,c', ',')
  
  -- replace: Replace string
  -- params: <s string>, <old string>, <new string>, <n int>
  -- return: <result table>
  local ret = strings.replace('Aaaa', 'a', 'A', -1) -- -1: all
  
  -- trim: Trim string space
  -- params: <s string>
  -- return: <result table>
  local ret = strings.trim('Aaaa')
  
  -- join: Join table to string
  -- params: <a table>, <sep string>
  -- return: <result table>
  local ret = strings.join({'a', 'b', 'c'}, ',')
  
  -- fields: Fields splits the string
  -- params: <s string>
  -- return: <result table>
  local ret = strings.fields('A  B C    D')
  

• time

  -- parse: Parse a time string into a number
  -- params: <layout string>, <time string>
  -- return: <time number>
  -- layout: 
      -- yyyy   (year)
      -- MM     (month)
      -- dd     (day)
      -- HH     (24hour)
      -- mm     (minute)
      -- ss     (second)
      -- SSS    (millinseconds)
      -- month  (month English abbreviations)
      -- week   (week English abbreviations)
      -- %y     (year)
      -- %M     (month)
      -- %d     (day)
      -- %H     (hour)
      -- %m     (minute)
      -- %s     (second)
  local ret = time.parse('yyyy-MM-dd HH:mm:ss.SSS', '2016-04-07 15:22:33.666')
  
  -- sleep: Sleep pauses the current goroutine for at least the duration d.
  -- params: <d string>
     -- Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h".
  -- return: None
  time.sleep('2s')
  

• log

  -- trace: Print trace message
  -- params: <v1 value>, <v2 value>...
  -- return: None
  log.trace('Hello', 123)
  
  -- debug: Print debug message
  -- params: <v1 value>, <v2 value>...
  -- return: None
  log.debug('Hello', 123)
  
  -- info: Print info message
  -- params: <v1 value>, <v2 value>...
  -- return: None
  log.info('Hello', 123)
  
  -- warn: Print warn message
  -- params: <v1 value>, <v2 value>...
  -- return: None
  log.warn('Hello', 123)
  
  -- error: Print error message
  -- params: <v1 value>, <v2 value>...
  -- return: None
  log.error('Hello', 123)
  
  -- fatal: Print fatal message
  -- params: <v1 value>, <v2 value>...
  -- return: None
  log.fatal('Hello', 123)
  

• odb

  -- mql: MQL access ODB
  -- params: <mql string>, <values any...>
  -- return: <result format: {data:[{}, {}...], meta:{}}>
  local ret = odb.mql('select id from /matrix/test where name = ? and age = ?', 'A', 10)
  
  -- mql: Search access ODB
  -- params: <search string>
  -- return: <result format: {data:[{}, {}...], meta:{}}>
  local ret = odb.search('wecise | print id | top 1')
  
  -- getid: Get object id by object table
  -- params: <object table>
  -- return: <id string>
  local id = odb.getid({column0='mxsvr1', class='/matrix/test'})
  

• appcontext

  -- rawget: Get raw cache, key-value mode
  -- params: <key string>
  -- return: <value string>
  local id = appcontext.rawget('testkey')
  
  -- rawset: Set raw cache, key-value mode
  -- params: <key string>, <value string>, <ttl second number>
  -- return: None
  appcontext.rawset('testkey', 'testvalue', 120)
  
  -- create: Create collection cache, collection mode
  -- params: <collection name string>, <value table>, <ttl second number>
  -- return: <id string>
  local id = appcontext.create('zoo', {name='monkey', room=1}, 120)
  
  -- getall: Get collection by condition
  -- params: <collection name string>, <query condition string>
  -- return: <result table>
  -- where: Query logic expression
  --        ||
  --        &&
  --        = != > < >= <= ~= !~= is isnot contains
  --        + -
  --        * / %
  --        ^
  --        ( )
  local list = appcontext.getall("zoo", "room == 1")
  
  -- getfirst: Get the first data of the collection
  -- params: <collection name string>
  -- return: <result table>
  local ret = appcontext.getfirst('zoo')
  
  -- getlast: Get the last data of the collection
  -- params: <collection name string>
  -- return: <result table>
  local ret = appcontext.getlast('zoo')
  
  -- get: Get data of the collection by id
  -- params: <collection name string>, <id string>
  -- return: <result table>
  local ret = appcontext.get('zoo', 'e898853a-74ee-11e8-941c-9eb6d01c8f95')
  
  -- update: Update data of the collection by id
  -- params: <collection name string>, <id string>, <value table>, <ttl number>
  -- return: None
  appcontext.update('zoo', 'e898853a-74ee-11e8-941c-9eb6d01c8f95', {name='monkey', room=2}, 120)
  
  -- deleteall: Delete collection all data
  -- params: <collection name string>
  -- return: None
  appcontext.deleteall('zoo')
  
  -- delete: Delete collection data by id
  -- params: <collection name string>, <id string>
  -- return: None
  appcontext.delete('zoo', 'e898853a-74ee-11e8-941c-9eb6d01c8f95')
  

• message

  -- send: Send message
  -- params: <source string>, <subject string>, <title string>, <msg string>
  -- return: None
  message.send('test_source', 'test_event', 'test_title', 'Test event message.')
  

• json

  -- parse: Parse string to table
  -- params: <json string>
  -- return: <result table>
  local obj = json.parse('{"name":"Tom", "age":12}')
  local arr = json.parse('[{"name":"Tom", "age":12}, {"name":"Jack", "age":10}]')
  
  -- string: json table to string
  -- params: <json table>
  -- return: <result string>
  local str = json.string({name='Tom', age=12})
  

• csv

  -- parse: Parse string to table
  -- params: <csv string>
  -- return: <result table>
  local str = 'Tom,12,BJ'..'\n'..'Jack,13,BJ'
  local arr = csv.parse(str)
  
  -- string: csv table to string (single line)
  -- params: <csv table>
  -- return: <result string>
  local str = csv.string({'Tom', 12, 'BJ'})
  

• omnibus

  -- charcount: Returns the number of characters in a string.
  -- params: <str string>
  -- return: <count number>
  local count = omnibus.charcount('ABCDE12345')
  
  -- clear: Removes the elements of an array.
  -- params: <table string>
  -- return: None
  clearTable = {'A', 'B', 'C'}
  omnibus.clear(clearTable)
  
  -- datetotime: Converts a string into a time data type.
  -- params: <datestr string>, <layout string>
  -- return: <timestamp number>
  local datetime = omnibus.datetotime('2020-09-29 23:43:11', 'yyyy-MM-dd HH:mm:ss')
  
  -- decimaltoascii: Converts a space-delimited decimal string into anASCII string.
  -- params: <decimal string>
  -- return: <ascii string>
  local asciistr = omnibus.decimaltoascii('104 101 108 108 111') -- asciistr: hello
  
  -- decimaltohex: Converts a decimal string into a hexadecimalstring. You can use it for straightforwardconversions with or without optional delimiterarguments. You can also use replacement delimiterarguments for more complicated conversions, forexample, to convert a dotted decimal MAC addressinto standard human-readable IEE802 format.
  -- params: <decimal string>, <oldstr string>, <newstr string>
  -- return: <hex string>
  local hexstr1 = omnibus.decimaltohex('9223372036854775807')  -- hexstr1: 7fffffffffffffff
  local hexstr2 = omnibus.decimaltohex('01.35.69.103.137.171', '.')  -- hexstr2: 01.23.45.67.89.ab
  local hexstr3 = omnibus.decimaltohex('01.35.69.103.137.171', '.', ':')  -- hexstr3: 01:23:45:67:89:ab
  
  -- extract: Returns the part of a string (which can be a field,element, or string expression) that matches theparenthesized section of the regular expression.
  -- params: <str string>, <regxp string>
  -- return: <str string>
  local exts = omnibus.extract('Access Port 123 ok', 'Port ([0-9]+)') -- exts: Port 123
  
  -- getdate: Returns the current date as a date data type.
  -- params: None
  -- return: <timestamp number>
  local timestamp = omnibus.getdate()
  
  -- getenv: Returns the value of an environment variable.
  -- params: <varname string>
  -- return: <value string>
  local timestamp = omnibus.getenv('PATH')
  
  -- gethostaddr: Returns the IP address of the host by using anaming service.
  -- params: <hostname string>
  -- return: <ip string>
  local timestamp = omnibus.gethostaddr('HOST1')
  
  -- gethostname: Returns the name of the host by using a naming service.
  -- params: <ip string>
  -- return: <hostname string>
  local timestamp = omnibus.gethostname('127.0.0.1')
  
  -- hextoascii: Converts a space-delimited hexadecimal string intoan ASCII string.
  -- params: <hexadecimal string>
  -- return: <ascii string>
  local asciistr = omnibus.hextoascii('68 65 6c 6c 6f') -- asciistr: hello
  
  -- hextodecimal: Converts a hexadecimal string into a decimalstring. You can use it for straightforwardconversions with or without optional delimiterarguments. You can also use replacement delimiterarguments for more complicated conversions, forexample, to convert a space-delimited hexadecimalIPv4 network address into a dotted decimaladdress.
  -- params: <hexadecimal string>, <old string>, <new string>
  -- return: <decimalstring string>
  local decstr1 = omnibus.hextodecimal('7FFFFFFFFFFFFFFF') -- decstr1: 9223372036854775807
  local decstr2 = omnibus.hextodecimal('C0 A8 0 1', ' ') -- decstr2: 192 168 0 1
  local decstr3 = omnibus.hextodecimal('C0 A8 0 1', ' ', '.') -- decstr3: 192.168.0.1
  
  -- int: Converts a numeric value into an integer.
  -- params: <n value>
  -- return: <num number>
  local num = omnibus.int('12345')
  
  -- length: Converts a numeric value into an integer.
  -- params: <str string>
  -- return: <len number>
  local strlength = omnibus.length('ABCDE12345')
  
  -- log: Enables you to log messages.
  -- params: <level string>, <msg string>
  -- return: None
  omnibus.log('DEBUG', 'Value is 1')
  
  -- lower: Converts an expression to lowercase.
  -- params: <str string>
  -- return: <str string>
  local str = omnibus.lower('ABC')
  
  -- ltrim: Removes white space from the left of anexpression.
  -- params: <str string>
  -- return: <str string>
  local str = omnibus.ltrim('   abc')
  
  -- match: Tests for an exact string match.
  -- params: <str1 string>, <str2 string>
  -- return: <ok bool>
  local str = omnibus.match(str1, 'abc')
  
  -- nmatch: Tests for a string match at the beginning of aspecified string.
  -- params: <str string>, <prefix string>
  -- return: <ok bool>
  local str = omnibus.nmatch(str, 'a')
  
  -- printable: Converts any non-printable characters in anexpression to a space character.
  -- params: <str string>
  -- return: <printstr string>
  local str = omnibus.printable(str)
  
  -- regmatch: Performs full regular expression matching of avalue in a regular expression in a string.
  -- params: <str string>, <reg string>
  -- return: <ok bool>
  local str = omnibus.regmatch(str, '^Name:[0-9]')
  
  -- rtrim: Removes white space from the right of anexpression.
  -- params: <str string>
  -- return: <str bool>
  local str = omnibus.rtrim('abc    ')
  
  -- scanformat: Converts an expression according to the availableformats, similar to the scanffamily of routines inC.
  -- params: <str string>, <format string>
  -- return: <v... values>
  local str, num = omnibus.scanformat('Hello int 123', '%s int %d')
  
  -- setvar: Set omnibus rule var
  -- params: <varname string>, <value any>
  -- return: None
  local str = omnibus.setvar('var1', '123')
  
  -- getvar: Get omnibus rule var
  -- params: <varname string>
  -- return: <value any>
  local str = omnibus.getvar('var1')
  
  -- setarray: Set omnibus rule array
  -- params: <varname string>, <value any>
  -- return: None
  local str = omnibus.setarray('var1', '123')
  
  -- getarray: Get omnibus rule array
  -- params: <varname string>
  -- return: <value any>
  local str = omnibus.getarray('var1')
  

Class field mapping

• 以下划线为开头的字段，都被认定为数据库映射字段，例如：

  _field1 = FIELD1
  _field2 = FIELD2
  _field3 = FIELD3
  .
  .
  .
  

• 特殊字段

  -- 是否忽略本条记录在规则之后插入到数据库，可根据条件判断设置。（规则运行不受影响，只影响规则之后的处理。）
  _IGNORE = <false | true>
  

Rule example

--class = /matrix/test
--parser = csv
--fields = FIELD1,FIELD2,FIELD3
--ignore = false

local ret = lookup('hostinfo', {key=FIELD1})

_field1 = FIELD1
_field2 = FIELD2
_field3 = FIELD3
_other = ret.val
_ctime = os.time()*1000

if _field1 = 'F1' then
    websocket.send('matrix', 'test', 'Test message.', 'auto')
end

Rule CSV direct example

--class = /matrix/test
--parser = csv
--fields = FIELD1,FIELD2,FIELD3
--fields_type = VAR, FLOAT, BOOL
--mqltmpl = insert into /matrix/test (id, testvalue, testvalue2) values (?, ?, ?) at ?
--mqltmpl_fields = id,testvalue,testvalue2,attime
--direct = true
--var_id = join FIELD1,'123'
--var_testvalue = array FIELD2,FIELD3,'ABC'
--var_attime = field FIELD3
--var_testvalue2 = map 'A':FIELD2,'B':FIELD3
--ignore = false

Rule JSON direct example

--class = /matrix/test
--parser = json
--fields = OBJECT
--mqltmpl = insert into /matrix/test (id, testvalue, testvalue2) values (?, ?, ?) at ?
--mqltmpl_fields = id,testvalue,testvalue2,attime
--direct = true
--var_id = join OBJECT.id,'123'
--var_testvalue = array OBJECT.name,OBJECT.content,'ABC'
--var_attime = field FIELD3
--var_testvalue2 = map 'A':OBJECT.name,'B':OBJECT.content
--ignore = false