External signal reception on FMZ platforms: extended API vs. built-in HTTP services

The Foreword

There are several articles in the platform's library about linking the Trading View webhook to signal-driven trading on systems outside the strategy, when the platform did not yet support the built-in HTTP service functionality of the JavaScript language.CommandRobotIn simple terms, an external HTTP/https signal request is sent to the FMZ platform, which then relay the signal to the FMZ platform as a policy interaction message notification to the policy program.

As the platform evolves, iterates and upgrades, many new features are updated. There are also new options for receiving external signals. All options have their own advantages, and we will explore this topic together in this article.

Expanded API with FMZ platform

The advantages of using this method to pair external systems are simplicity, security, and high stability of the platform-dependent extensible API interface.

How to receive external signals:

External system (Trading View webhook) > FMZ extended API service > Strategy

External system (Trading View webhook): For example, a PINE script running on Trading View can set an alarm that, when triggered, sends an HTTP request to the set webhook url as a signal. 2, FMZ extended API service: After successfully accessing the interface, the platform forwards the information, which is sent as an interactive message to the policy disk. 3, Policy Disks: The GetCommand function can be designed to listen to interactive messages in the policy disks and perform established operations after detecting the messages.

This is a step in the middle (platform switching) as opposed to using the built-in HTTP service to directly create service reception signals.

Built-in HTTP service policy

The platform supports the built-in HTTP service functionality of the JavaScript language, which allows you to directly create a concurrent service to listen to external signals. The advantages are: the created HTTP service is a separate thread, does not affect the logic of the main function, can listen to messages similar to the GetCommand function, listen directly to external signals, compared to the use of an extended API program, eliminating the middle loop.

How to receive external signals:

External system (Trading View webhook)

External system (Trading View webhook): For example, a PINE script running on Trading View can set an alarm that, when triggered, sends an HTTP request to the set webhook url as a signal. 2, Strategic Display: Strategy runs an HTTP service simultaneously and receives external signals directly.

This saves a step, but to improve security, it's best to configure the https service, which requires folding it down; it's a bit more of a hassle than using an extended API.

Testing code

The following policy will loop and send 10 HTTP/HTTPS requests per round to simulate external signals. The policy will then listen for "interactive messages" and "messages pushed by HTTP service threads". The policy will then match each external signal message with the received signal, detect if there is any signal loss, and calculate the time taken.

var httpUrl = "http://123.123.123.123:8088/CommandRobot"
var accessKey = ""
var secretKey = ""

function serverFunc(ctx) {
    var path = ctx.path()
    if (path == "/CommandRobot") {
        var body = ctx.body()
        threading.mainThread().postMessage(body)
        ctx.write("OK")
        // 200
    } else {
        ctx.setStatus(404)
    }
}

function createMsgTester(accessKey, secretKey, httpUrl) {
    var tester = {}
    
    tester.currentRobotId = _G()
    tester.arrSendMsgByAPI = []
    tester.arrSendMsgByHttp = []
    tester.arrEchoMsgByAPI = []
    tester.arrEchoMsgByHttp = []
    tester.idByAPI = 0
    tester.idByHttp = 0

    var sendMsgByAPI = function(msgByAPI, robotId, accessKey, secretKey) {
        var headers = {
            "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36",
            "Content-Type": "application/json"
        }
        HttpQuery(`https://www.fmz.com/api/v1?access_key=${accessKey}&secret_key=${secretKey}&method=CommandRobot&args=[${robotId},+""]`, {"method": "POST", "body": JSON.stringify(msgByAPI), "headers": headers})
    }

    var sendMsgByHttp = function(msgByHttp, httpUrl) {
        HttpQuery(httpUrl, {"method": "POST", "body": JSON.stringify(msgByHttp)})
    }

    tester.run = function() {
        var robotId = tester.currentRobotId

        for (var i = 0; i < 10; i++) {
            var msgByAPI = {"ts": new Date().getTime(), "id": tester.idByAPI, "way": "ByAPI"}            
            tester.arrSendMsgByAPI.push(msgByAPI)
            tester.idByAPI++
            threading.Thread(sendMsgByAPI, msgByAPI, robotId, accessKey, secretKey)

            var msgByHttp = {"ts": new Date().getTime(), "id": tester.idByHttp, "way": "ByHttp"}
            tester.arrSendMsgByHttp.push(msgByHttp)
            tester.idByHttp++
            threading.Thread(sendMsgByHttp, msgByHttp, httpUrl)
        }
    }

    tester.getEcho =function(msg) {
        if (msg["way"] == "ByAPI") {
            tester.arrEchoMsgByAPI.push(msg)
        } else {
            tester.arrEchoMsgByHttp.push(msg)
        }
    }

    tester.deal = function() {
        var tbls = []
        for (var pair of [[tester.arrEchoMsgByHttp, tester.arrSendMsgByHttp, "ByHttp"], [tester.arrEchoMsgByAPI, tester.arrSendMsgByAPI, "ByAPI"]]) {
            var receivedMessages = pair[0]
            var sentMessages = pair[1]
            var testType = pair[2]

            var receivedMap = new Map()
            receivedMessages.forEach(message => {
                receivedMap.set(message["id"], message)
            })
            
            var matchedPairs = []
            var timeDifferences = []
            for (var sentMessage of sentMessages) {
                var receivedMessage = receivedMap.get(sentMessage["id"])
                if (receivedMessage) {
                    matchedPairs.push([JSON.stringify(sentMessage), JSON.stringify(receivedMessage), receivedMessage["ts"] - sentMessage["ts"]])
                    timeDifferences.push(receivedMessage["ts"] - sentMessage["ts"])
                } else {
                    Log("no matched sentMessage:", sentMessage, "#FF0000")
                }
            }
            
            var averageTimeDifference = timeDifferences.reduce((sum, diff) => sum + diff, 0) / timeDifferences.length
            
            var tbl = {
                "type": "table",
                "title": testType + " / averageTimeDifference:" + averageTimeDifference,
                "cols": ["send", "received", "ts diff"],
                "rows": []
            }

            for (var pair of matchedPairs) {
                tbl["rows"].push(pair)
            }

            tbls.push(tbl)
            Log(testType, ", averageTimeDifference:", averageTimeDifference, "ms")
        }

        tester.arrSendMsgByAPI = []
        tester.arrSendMsgByHttp = []
        tester.arrEchoMsgByAPI = []
        tester.arrEchoMsgByHttp = []

        return tbls
    }

    return tester
}

function main() {
    __Serve("http://0.0.0.0:8088", serverFunc)

    var t = createMsgTester(accessKey, secretKey, httpUrl)
    while (true) {
        Log("测试开始...", "#FF0000")
        t.run()

        var beginTS = new Date().getTime()
        while (new Date().getTime() - beginTS < 60 * 1000) {
            var cmd = GetCommand()
            if (cmd) {
                try {
                    var obj = JSON.parse(cmd)
                    obj["ts"] = new Date().getTime()
                    t.getEcho(obj)
                } catch (e) {
                    Log(e)
                }
            }
            
            var msg = threading.mainThread().peekMessage(-1)
            if (msg) {
                try {
                    var obj = JSON.parse(msg)
                    obj["ts"] = new Date().getTime()
                    t.getEcho(obj)                
                } catch (e) {
                    Log(e)
                }
            }
        }
        Log("等待结束...", "#FF0000")
                
        var tbls = t.deal()
        LogStatus(_D(), "\n`" + JSON.stringify(tbls) + "`")
        Sleep(20000)
    }
}

If tested, it is necessary to fill in a specific server IP address, the FMZ platform's extension API KEY.

var httpUrl = "http://123.123.123.123:8088/CommandRobot"
var accessKey = "xxx"
var secretKey = "xxx"

1, the serverFunc function creates a concurrent HTTP service to listen to external signals. For external messages received by the extended API interface, the GetCommand function is used to listen.

• This is a message from the HTTP thread: byvar msg = threading.mainThread().peekMessage(-1)Listen to this.

• The interactive message forwarded by the extended API interface: byvar cmd = GetCommand()Listen to this.

The platform optimizes the underlying multi-threaded resource recovery mechanism to ensure that both the sending and receiving signal processes are unobstructed.ThreadOrexchange.GoSimultaneous functions, without having to explicitly wait for the simultaneous task to complete (e.g. join function, wait function, etc.), the system sub-layer automatically handles resource recovery (only hosts that require the latest version are supported).

    // 摘录代码片段，发送信号
    tester.run = function() {
        var robotId = tester.currentRobotId

        for (var i = 0; i < 10; i++) {
            var msgByAPI = {"ts": new Date().getTime(), "id": tester.idByAPI, "way": "ByAPI"}            
            tester.arrSendMsgByAPI.push(msgByAPI)
            tester.idByAPI++
            threading.Thread(sendMsgByAPI, msgByAPI, robotId, accessKey, secretKey)   // 并发调用，非阻塞

            var msgByHttp = {"ts": new Date().getTime(), "id": tester.idByHttp, "way": "ByHttp"}
            tester.arrSendMsgByHttp.push(msgByHttp)
            tester.idByHttp++
            threading.Thread(sendMsgByHttp, msgByHttp, httpUrl)                       // 并发调用，非阻塞
        }
    }

    // 摘录代码片段，接收信号
    var cmd = GetCommand()                              // 监听来自扩展API的消息，非阻塞
    var msg = threading.mainThread().peekMessage(-1)    // 监听来自自建Http服务的消息，使用了参数-1，非阻塞

Next, let's look at this test process, where the information is directly annotated in the code:

function main() {
    __Serve("http://0.0.0.0:8088", serverFunc)      // 在当前策略实例中，创建一个并发的http服务

    var t = createMsgTester(accessKey, secretKey, httpUrl)   // 创建一个用于测试管理的对象
    while (true) {                                           // 策略主循环开始
        Log("测试开始...", "#FF0000")
        t.run()                                              // 每次循环开始，调用测试管理对象的run函数，使用两种方式（1、通过扩展API发送信号，2、直接向当前策略创建的Http服务发送信号），每种方式并发发送10个请求

        var beginTS = new Date().getTime()
        while (new Date().getTime() - beginTS < 60 * 1000) {   // 循环检测来自扩展API的交互消息，循环检测来自自建Http服务的消息
            var cmd = GetCommand()
            if (cmd) {
                try {
                    var obj = JSON.parse(cmd)
                    obj["ts"] = new Date().getTime()        // 检测到交互消息，记录消息，更新时间为收到时间
                    t.getEcho(obj)                          // 记录到对应数组
                } catch (e) {
                    Log(e)
                }
            }
            
            var msg = threading.mainThread().peekMessage(-1)
            if (msg) {
                try {
                    var obj = JSON.parse(msg)
                    obj["ts"] = new Date().getTime()        // 检测到自建的Http服务收到的消息，更新时间为收到时间
                    t.getEcho(obj)                          // ...
                } catch (e) {
                    Log(e)
                }
            }
        }
        Log("等待结束...", "#FF0000")
                
        var tbls = t.deal()                                  // 根据记录的消息，配对，检查是否有未配对的消息，如果有说明有信号丢失
        LogStatus(_D(), "\n`" + JSON.stringify(tbls) + "`")
        Sleep(20000)
    }
}

The test results

Over a period of time, testing has shown that HTTP takes slightly less time on average than API.

The built-in HTTP service receives signals, this test method is not very strict, the request should come from the outside. For the sake of simplicity, this factor can be ignored. For the two ways of signal acquisition, the built-in HTTP service is reduced by one link, after all, it should be a little faster. For signal stability, it is more important that the signal is not lost or missed.

In this post, we have completed the implementation of an available built-in HTTP service template for receiving external Trading View signals, welcome to discuss, thank you for reading.