User guide

The resource loading... loading...

Custom Data Source

The backtesting system of the FMZ Quant Trading Platform supports custom data sources, the backtesting system uses the GET method to request a custom URL (publicly accessible URL) to obtain an external data source for backtest. The additional request parameters are as follows:

Parameter	Meaning	Explanation
symbol	Symbol Name	Spot market data, such as: `BTC_USDT`, futures market data, such as: `BTC_USDT.swap`, futures perpetual contract funding rate data, such as: `BTC_USDT.funding`, futures perpetual contract price index data, such as: `BTC_USDT.index`
eid	Exchanges	such as OKX, Futures_OKX
round	Data Accuracy	True means that the specific precision is defined in the data fed back by the custom data source. The request sent by the FMZ Quant Trading Platform Backtesting System to the custom data source is fixed as: `round=true`
period	K-line Data Period (Milliseconds)	such as: `60000` is a 1-minute period
depth	Depth Levels	1-20
trades	Whether Need to Split Data	true(1) / false(0)
from	Start Time	unix timestamp
to	End Time	unix timestamp
detail	Request data for symbol details	True means that it needs to be provided by a custom data source. The request sent by the FMZ Quant Trading Platform Backtesting System to the custom data source is fixed as: `detail=true`
custom	–	This parameter can be ignored

When the data source of the spot exchange and futures exchange objects is set to a custom data source (feeder), the backtesting system sends a request to the custom data source service:

http://customserver:9090/data?custom=0&depth=20&detail=true&eid=Bitget&from=1351641600&period=86400000&round=true&symbol=BTC_USDT&to=1611244800&trades=1
http://customserver:9090/data?custom=0&depth=20&detail=true&eid=Futures_OKX&from=1351641600&period=86400000&round=true&symbol=BTC_USDT.swap&to=1611244800&trades=1

Data Format

The returned format must be one of the following two formats (which will be recognized by the system automatically):

Simulation level Tick, the following is an example of JSON data:

{
    "detail": {
        "eid": "Binance",
        "symbol": "BTC_USDT",
        "alias": "BTCUSDT",
        "baseCurrency": "BTC",
        "quoteCurrency": "USDT",
        "marginCurrency": "USDT",
        "basePrecision": 5,
        "quotePrecision": 2,
        "minQty": 0.00001,
        "maxQty": 9000,
        "minNotional": 5,
        "maxNotional": 9000000,
        "priceTick": 0.01,
        "volumeTick": 0.00001,
        "marginLevel": 10
    },
    "schema":["time", "open", "high", "low", "close", "vol"],
    "data":[
        [1564315200000, 9531300, 9531300, 9497060, 9497060, 787],
        [1564316100000, 9495160, 9495160, 9474260, 9489460, 338]
    ]
}

Bot level tick, the following is an example of JSON data: Tick-level backtest data (contains information on the depth of the market, and the depth format is an array of [price, volume]. It can have multiple levels of depth, asks for price ascending order, bids for price descending order).

{
    "detail": {
        "eid": "Binance",
        "symbol": "BTC_USDT",
        "alias": "BTCUSDT",
        "baseCurrency": "BTC",
        "quoteCurrency": "USDT",
        "marginCurrency": "USDT",
        "basePrecision": 5,
        "quotePrecision": 2,
        "minQty": 0.00001,
        "maxQty": 9000,
        "minNotional": 5,
        "maxNotional": 9000000,
        "priceTick": 0.01,
        "volumeTick": 0.00001,
        "marginLevel": 10
    },
    "schema":["time", "asks", "bids", "trades", "close", "vol"],
    "data":[
        [1564315200000, [[9531300, 10]], [[9531300, 10]], [[1564315200000, 0, 9531300, 10]], 9497060, 787],
        [1564316100000, [[9531300, 10]], [[9531300, 10]], [[1564316100000, 0, 9531300, 10]], 9497060, 787]
    ]
}

Field	Description
detail	Detailed information on the requested data type,

including the name of the denominated currency, the name of the trading currency, the precision, the minimum order quantity, etc. | | schema | It specifies the attributes of the columns in the data array, which is case sensitive and is only limited to time, open, high, low, close, vol, asks, bids, trades| | data | The column structure, recorded data according to the schema settings.|

detail field

Field	Description
eid	Exchange Id, please note that the spot and futures of a
certain exchange have different eids.
symbol	Trading product code
alias	The symbol in the exchange corresponding to the current
trading product code
baseCurrency	Trading Currency
quoteCurrency	Denominated Currency
marginCurrency	Margin Currency
basePrecision	Transaction Currency Accuracy
quotePrecision	Pricing Currency Accuracy
minQty	Minimum Order Quantity
maxQty	Maximum Order Quantity
minNotional	Minimum Order Amount
maxNotional	Maximum Order Amount
priceTick	Price Jump
volumeTick	Minimum change value of order quantity (one jump in
order quantity)
marginLevel	Futures Leverage Value
contractType	For perpetual contracts set to: `swap`, the

backtest system will continue to send funding rate and price index requests |

Special column attributes asks, bids, trades:

Field	Description	Remarks
asks / bids	[[price, volume], …]	For example, the data in

the Live Trading Level Tick data example: [[9531300, 10]] | | trades | [[time,direction(0:buy,1:sell),price,volume], …] | For example, the data in the Live Trading Level Tick data example: [[1564315200000, 0, 9531300, 10]] |

When backtesting perpetual contracts on futures exchanges, custom data sources also require additional funding rate data and price index data. The backtesting system will continue to send requests for funding rates only when the requested market data is returned and the detail field in the returned structure contains the "contractType": "swap" key-value pair.

When the backtesting system receives funding rate data, it will continue to send requests for price index data.

The funding rate data structure is as follows:

{
    "detail": {
        "eid": "Futures_Binance",
        "symbol": "BTC_USDT.funding",
        "alias": "BTC_USDT.funding",
        "baseCurrency": "BTC",
        "quoteCurrency": "USDT",
        "marginCurrency": "",
        "basePrecision": 8,
        "quotePrecision": 8,
        "minQty": 1,
        "maxQty": 10000,
        "minNotional": 1,
        "maxNotional": 100000000,
        "priceTick": 1e-8,
        "volumeTick": 1e-8,
        "marginLevel": 10
    },
    "schema": [
        "time",
        "open",
        "high",
        "low",
        "close",
        "vol"
    ],
    "data": [
        [
            1584921600000,
            -16795,
            -16795,
            -16795,
            -16795,
            0
        ],
        [
            1584950400000,
            -16294,
            -16294,
            -16294,
            -16294,
            0
        ]
        // ...
    ]
}

The interval between adjacent periods is 8 hours
For example, Binance’s funding rate is updated every 8 hours. Why is the funding rate data -16795? Because like K-line data, in order to avoid the loss of floating point precision during network transmission, the data uses integer type; the funding rate data may also be negative.

An example of a funding rate data request from the backtesting system is:

http://customserver:9090/data?custom=0&depth=20&detail=true&eid=Futures_Binance&from=1351641600&period=86400000&round=true&symbol=BTC_USDT.funding&to=1611244800&trades=0

The price index data structure is as follows:


{
    "detail": {
        "eid": "Futures_Binance",
        "symbol": "BTC_USDT.index",
        "alias": "BTCUSDT",
        "baseCurrency": "BTC",
        "quoteCurrency": "USDT",
        "contractType": "index",
        "marginCurrency": "USDT",
        "basePrecision": 3,
        "quotePrecision": 1,
        "minQty": 0.001,
        "maxQty": 1000,
        "minNotional": 0,
        "maxNotional": 1.7976931348623157e+308,
        "priceTick": 0.1,
        "volumeTick": 0.001,
        "marginLevel": 10,
        "volumeMultiple": 1
    },
    "schema": [
        "time",
        "open",
        "high",
        "low",
        "close",
        "vol"
    ],
    "data": [
        [1584921600000, 58172, 59167, 56902, 58962, 0],
        [1584922500000, 58975, 59428, 58581, 59154, 0],
        // ...
    ]
}

An example of a price index data request sent by the backtesting system is:

http://customserver:9090/data?custom=0&depth=20&detail=true&eid=Futures_Binance&from=1351641600&period=86400000&round=true&symbol=BTC_USDT.index&to=1611244800&trades=0

Example for Custom Data Source

Specify the data source address, e.g., http://120.24.2.20:9090/data. The custom data source service program is written using Golang:

package main

import (
    "fmt"
    "net/http"
    "encoding/json"
)

func Handle (w http.ResponseWriter, r *http.Request) {
    // e.g. set on backtest DataSourse: http://xxx.xx.x.xx:9090/data

    // request: GET http://xxx.xx.x.xx:9090/data?custom=0&depth=20&detail=true&eid=OKX&from=1584921600&period=86400000&round=true&symbol=BTC_USDT&to=1611244800&trades=1
    //              http://xxx.xx.x.xx:9090/data?custom=0&depth=20&detail=true&eid=Futures_Binance&from=1599958800&period=3600000&round=true&symbol=BTC_USDT.swap&to=1611244800&trades=0
    fmt.Println("request:", r)

    // response
    defer func() {
        // response data
        /* e.g. data
        {
            "detail": {
                "eid": "Binance",
                "symbol": "BTC_USDT",
                "alias": "BTCUSDT",
                "baseCurrency": "BTC",
                "quoteCurrency": "USDT",
                "marginCurrency": "USDT",
                "basePrecision": 5,
                "quotePrecision": 2,
                "minQty": 0.00001,
                "maxQty": 9000,
                "minNotional": 5,
                "maxNotional": 9000000,
                "priceTick": 0.01,
                "volumeTick": 0.00001,
                "marginLevel": 10
            },
            "schema": [
                "time",
                "open",
                "high",
                "low",
                "close",
                "vol"
            ],
            "data": [
                [1610755200000, 3673743, 3795000, 3535780, 3599498, 8634843151],
                [1610841600000, 3599498, 3685250, 3385000, 3582861, 8015772738],
                [1610928000000, 3582499, 3746983, 3480000, 3663127, 7069811875],
                [1611014400000, 3662246, 3785000, 3584406, 3589149, 7961130777],
                [1611100800000, 3590194, 3641531, 3340000, 3546823, 8936842292],
                [1611187200000, 3546823, 3560000, 3007100, 3085013, 13500407666],
                [1611273600000, 3085199, 3382653, 2885000, 3294517, 14297168405],
                [1611360000000, 3295000, 3345600, 3139016, 3207800, 6459528768],
                [1611446400000, 3207800, 3307100, 3090000, 3225990, 5797803797],
                [1611532800000, 3225945, 3487500, 3191000, 3225420, 8849922692]
            ]
        }
        */
        
        // /* Simulation level Tick
        ret := map[string]interface{}{
            "detail": map[string]interface{}{
                "eid": "Binance",
                "symbol": "BTC_USDT",
                "alias": "BTCUSDT",
                "baseCurrency": "BTC",
                "quoteCurrency": "USDT",
                "marginCurrency": "USDT",
                "basePrecision": 5,
                "quotePrecision": 2,
                "minQty": 0.00001,
                "maxQty": 9000,
                "minNotional": 5,
                "maxNotional": 9000000,
                "priceTick": 0.01,
                "volumeTick": 0.00001,
                "marginLevel": 10,
            },
            "schema": []string{"time","open","high","low","close","vol"},
            "data": []interface{}{
                []int64{1610755200000, 3673743, 3795000, 3535780, 3599498, 8634843151},  // 1610755200000 : 2021-01-16 08:00:00
                []int64{1610841600000, 3599498, 3685250, 3385000, 3582861, 8015772738},  // 1610841600000 : 2021-01-17 08:00:00
                []int64{1610928000000, 3582499, 3746983, 3480000, 3663127, 7069811875},
                []int64{1611014400000, 3662246, 3785000, 3584406, 3589149, 7961130777},
                []int64{1611100800000, 3590194, 3641531, 3340000, 3546823, 8936842292},
                []int64{1611187200000, 3546823, 3560000, 3007100, 3085013, 13500407666},
                []int64{1611273600000, 3085199, 3382653, 2885000, 3294517, 14297168405},
                []int64{1611360000000, 3295000, 3345600, 3139016, 3207800, 6459528768},
                []int64{1611446400000, 3207800, 3307100, 3090000, 3225990, 5797803797},
                []int64{1611532800000, 3225945, 3487500, 3191000, 3225420, 8849922692},
            },
        }
        // */

        /* Bot level Tick
        ret := map[string]interface{}{
            "detail": map[string]interface{}{
                "eid": "Binance",
                "symbol": "BTC_USDT",
                "alias": "BTCUSDT",
                "baseCurrency": "BTC",
                "quoteCurrency": "USDT",
                "marginCurrency": "USDT",
                "basePrecision": 5,
                "quotePrecision": 2,
                "minQty": 0.00001,
                "maxQty": 9000,
                "minNotional": 5,
                "maxNotional": 9000000,
                "priceTick": 0.01,
                "volumeTick": 0.00001,
                "marginLevel": 10,
            },
            "schema": []string{"time", "asks", "bids", "trades", "close", "vol"},
            "data": []interface{}{
                []interface{}{1610755200000, []interface{}{[]int64{9531300, 10}}, []interface{}{[]int64{9531300, 10}}, []interface{}{[]int64{1610755200000, 0, 9531300, 10}}, 9497060, 787},
                []interface{}{1610841600000, []interface{}{[]int64{9531300, 15}}, []interface{}{[]int64{9531300, 15}}, []interface{}{[]int64{1610841600000, 0, 9531300, 11}}, 9497061, 789},                
            },
        }
        */

        b, _ := json.Marshal(ret)
        w.Write(b)
    }()
}
func main () {
    fmt.Println("listen http://localhost:9090")
    http.HandleFunc("/data", Handle)
    http.ListenAndServe(":9090", nil)
}

Test strategy, JavaScript example:

/*backtest
start: 2021-01-16 08:00:00
end: 2021-01-22 00:00:00
period: 1d
basePeriod: 1d
exchanges: [{"eid":"OKX","currency":"BTC_USDT","feeder":"http://120.24.2.20:9090/data"}]
args: [["number",2]]
*/

function main() {
    var ticker = exchange.GetTicker()
    var records = exchange.GetRecords()
    Log(exchange.GetName(), exchange.GetCurrency())
    Log(ticker)
    Log(records)
}

Save Backtest Settings Local Backtest Engine

Forums
|
API
|
Github
|
About
|
SiteMap