Similar to this question How to get all the historical data from chainlink price feeds?
I would like to retrieve the historical data of a particular price feed to be used off-chain for a large number of consecutive updates. However, according to the docs it seems like the round IDs are no longer incremental (i.e. you can't just call getRoundData(latest_round-_step) for steps in a certain range).
Is the solution to simply do a linear search backwards and try all entries smaller than the latest roundID, or is there a recursive way to achieve this right now (is the previous round ID saved somewhere)?
I've been recently working on ethereum-datafarm, which represents an open-source tool that parses events from Smart Contracts and stores them in csv format.
The tool was originally intedent to be used by economics/business students, who want to apply their knowledge on blockchain data but have hard times in setting up archive nodes or parsing bytes. Users can plug in any contract and start parsing. Under the hood, the Etherscan API is used, which is free to use up to quite generous limits.
A sample use case might be plugging in the Uniswap contracts, retrieve prices of certain pairs and then analyse them. Furthermore, it may be the foundation for network analyses, studies on Stablecoins, research on DAOs, goveranance and more.
Although magical beliefs (such as belief in luck and precognition) are presumably universal, the extent to which such beliefs are embraced likely varies across cultures. We assessed the effect of culture on luck and precognition beliefs in two large-scale multinational studies (Study 1: k = 16, N = 17,664; Study 2: k = 25, N = 4,024). Over and above the effects of demographic factors, culture was a significant predictor of luck and precognition beliefs in both studies. Indeed, when culture was added to demographic models, the variance accounted for in luck and precognition beliefs approximately doubled. Belief in luck and precognition was highest in Latvia and Russia (Study 1) and South Asia (Study 2), and lowest in Protestant Europe (Studies 1 and 2). Thus, beyond the effects of age, gender, education, and religiosity, culture is a significant factor in explaining variance in people’s belief in luck and precognition. Follow-up analyses found a relatively consistent effect of socio-economic development, such that belief in luck and precognition were more prevalent in countries with lower scores on the Human Development Index. There was also some evidence that these beliefs were stronger in more collectivist cultures, but this effect was inconsistent. We discuss the possibility that there are culturally specific historical factors that contribute to relative openness to such beliefs in Russia, Latvia, and South Asia.
That is from a new paper by Emily A. Harris, Taciano L. Milfont, and Matthew J. Hornsey, via the excellent Kevin Lewis.
In the past, FMZ officially released a perpetual grid strategy, which was popular among users, and the onlookers who traded TRX in real bots have gained a lot of profits in the past year with controllable risks. However, the perpetual grid strategy also has some problems:
It is necessary to set parameters such as initial price, grid spacing, grid value, long-short mode, etc. The settings are cumbersome and have a great impact on profits, making it difficult for novices to set.
The perpetual grid strategy has a high risk of short-selling, while the risk of long-selling is relatively low. Even if the grid value is set to a small value, it will not have a great impact on the short-selling price.
The perpetual contract grid can choose to only go long to avoid the risk of shorting, it seems okay so far. However, it needs to face the problem that the current price exceeds the initial price, resulting in a short position, and the initial price needs to be reset.
I wrote an article on the principle of the balance strategy and the comparison with the grid strategy before, and you can still refer to it now: https://www.fmz.com/digest-topic/5930. The balance strategy always holds positions with a fixed value ratio or value, sells some when it rises, and buys when it falls. It can be run with simple settings. Even if the currency price rises a lot, there is no risk of going short. The problem with the spot balance strategy is that the capital utilization is low, and there is no easy way to increase leverage. And perpetual contracts can solve the problem. If the total capital is 1000, 2000 can be held fixedly, which exceeds the original capital and improves the capital utilization. Another parameter is the adjustment ratio, which controls how much to sacle in or dump the position. If it is set to 0.01, it means that the position is dumped once for 1% increase and scaled in once for 1% decrease.
For beginners, the balance strategy is highly recommended. The operation is simple, just set a parameter of holding ratio or position value, and you can run it mindlessly without worrying about constant price increases. Those with certain experience can choose the grid strategy, and decide the upper and lower limits of fluctuations and the funds per grid, so as to improve the utilization of funds and obtain maximum profits.
In order to facilitate the backtesting of more trading pairs, this document will show the complete backtesting process, and users can adjust different parameters and trading pairs for comparison. (The version is Python3, and an agent is required to download the quotation. Users can download Anancoda3 by themselves or run it through Google's colab)
import requests
from datetime import date,datetime
import time
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import requests, zipfile, io
%matplotlib inline
## Current trading pairs
Info = requests.get('https://fapi.binance.com/fapi/v1/exchangeInfo')
symbols = [s['symbol'] for s in Info.json()['symbols']]
symbols = list(set(filter(lambda x: x[-4:] == 'USDT', [s.split('_')[0] for s in symbols]))-
set(['1000SHIBUSDT','1000XECUSDT','BTCDOMUSDT','DEFIUSDT','BTCSTUSDT'])) + ['SHIBUSDT','XECUSDT']
print(symbols)
#Get the function of the K-line of any period
def GetKlines(symbol='BTCUSDT',start='2020-8-10',end='2021-8-10',period='1h',base='fapi',v = 'v1'):
Klines = []
start_time = int(time.mktime(datetime.strptime(start, "%Y-%m-%d").timetuple()))*1000 + 8*60*60*1000
end_time = min(int(time.mktime(datetime.strptime(end, "%Y-%m-%d").timetuple()))*1000 + 8*60*60*1000,time.time()*1000)
intervel_map = {'m':60*1000,'h':60*60*1000,'d':24*60*60*1000}
while start_time < end_time:
mid_time = start_time+1000*int(period[:-1])*intervel_map[period[-1]]
url = 'https://'+base+'.binance.com/'+base+'/'+v+'/klines?symbol=%s&interval=%s&startTime=%s&endTime=%s&limit=1000'%(symbol,period,start_time,mid_time)
#print(url)
res = requests.get(url)
res_list = res.json()
if type(res_list) == list and len(res_list) > 0:
start_time = res_list[-1][0]+int(period[:-1])*intervel_map[period[-1]]
Klines += res_list
if type(res_list) == list and len(res_list) == 0:
start_time = start_time+1000*int(period[:-1])*intervel_map[period[-1]]
if mid_time >= end_time:
break
df = pd.DataFrame(Klines,columns=['time','open','high','low','close','amount','end_time','volume','count','buy_amount','buy_volume','null']).astype('float')
df.index = pd.to_datetime(df.time,unit='ms')
return df
By downloading the closing prices of all trading pairs from 2021 to the present, we can observe the changes in the overall market index: 2021 to 2022 is undoubtedly a bull market, and the index once rose by 14 times. It can be said that gold is everywhere, and many coins have risen hundreds of times. However, in 2022, the bear market that has lasted for half a year has begun, with the index plunging 80%, and dozens of coins have withdrawn by more than 90%. Such pump-and-dump reflects the enormous risk of grid strategies. The index is currently at around 3, which is still a 200% gain compared to the beginning of 2021, and it should be a relative bottom at the moment, considering the development of the market.
Currencies whose highest price has increased more than 10 times since the beginning of the year:
#Download closing prices for all trading pairs
start_date = '2021-1-1'
end_date = '2022-05-30'
period = '1d'
df_all = pd.DataFrame(index=pd.date_range(start=start_date, end=end_date, freq=period),columns=symbols)
for i in range(len(symbols)):
#print(symbols[i])
symbol = symbols[i]
df_s = GetKlines(symbol=symbol,start=start_date,end=end_date,period=period,base='api',v='v3')
df_all[symbol] = df_s[~df_s.index.duplicated(keep='first')].close
#The highest increase over the beginning of the year
max_up = df_all.max()/df_all.fillna(method='bfill').iloc[0]
print(max_up.map(lambda x:round(x,3)).sort_values().to_dict())
First of all, we use the simplest code to simulate the situation of falling all the way down, and see the liquidation price of different position values. Since the strategy always holds a long position, there is no risk in going up. The initial capital is 1000, the currency price is 1, and the adjustment ratio is 0.01. The results are as follows. It can be seen that the risk of long liquidation is not low. With 1.5 times leverage, it can resist a 50% decline. Given the current relative bottom situation, it is an acceptable risk.
#Still using the original backtesting engine
class Exchange:
def __init__(self, trade_symbols, fee=0.0004, initial_balance=10000):
self.initial_balance = initial_balance #Initial assets
self.fee = fee
self.trade_symbols = trade_symbols
self.account = {'USDT':{'realised_profit':0, 'unrealised_profit':0, 'total':initial_balance, 'fee':0}}
for symbol in trade_symbols:
self.account[symbol] = {'amount':0, 'hold_price':0, 'value':0, 'price':0, 'realised_profit':0,'unrealised_profit':0,'fee':0}
def Trade(self, symbol, direction, price, amount):
cover_amount = 0 if direction*self.account[symbol]['amount'] >=0 else min(abs(self.account[symbol]['amount']), amount)
open_amount = amount - cover_amount
self.account['USDT']['realised_profit'] -= price*amount*self.fee #Deduct the handling fees
self.account['USDT']['fee'] += price*amount*self.fee
self.account[symbol]['fee'] += price*amount*self.fee
if cover_amount > 0: #Close the position first
self.account['USDT']['realised_profit'] += -direction*(price - self.account[symbol]['hold_price'])*cover_amount #Profits
self.account[symbol]['realised_profit'] += -direction*(price - self.account[symbol]['hold_price'])*cover_amount
self.account[symbol]['amount'] -= -direction*cover_amount
self.account[symbol]['hold_price'] = 0 if self.account[symbol]['amount'] == 0 else self.account[symbol]['hold_price']
if open_amount > 0:
total_cost = self.account[symbol]['hold_price']*direction*self.account[symbol]['amount'] + price*open_amount
total_amount = direction*self.account[symbol]['amount']+open_amount
self.account[symbol]['hold_price'] = total_cost/total_amount
self.account[symbol]['amount'] += direction*open_amount
def Buy(self, symbol, price, amount):
self.Trade(symbol, 1, price, amount)
def Sell(self, symbol, price, amount):
self.Trade(symbol, -1, price, amount)
def Update(self, close_price): #Update of assets
self.account['USDT']['unrealised_profit'] = 0
for symbol in self.trade_symbols:
self.account[symbol]['unrealised_profit'] = (close_price[symbol] - self.account[symbol]['hold_price'])*self.account[symbol]['amount']
self.account[symbol]['price'] = close_price[symbol]
self.account[symbol]['value'] = abs(self.account[symbol]['amount'])*close_price[symbol]
self.account['USDT']['unrealised_profit'] += self.account[symbol]['unrealised_profit']
self.account['USDT']['total'] = round(self.account['USDT']['realised_profit'] + self.initial_balance + self.account['USDT']['unrealised_profit'],6)
First of all, we backtest the performance of the TRX balance strategy. The maximum retracement of TRX in this round of bear market is relatively small, so it has certain specificity. The data is selected from the 5min K-line from 2021 to the present, with an initial capital of 1000, the adjustment ratio is 0.01, the position value is 2000, and the handling fee is 0.0002.
The initial price of TRX was 0.02676U, and the highest price during the period reached 0.18U. It is currently around 0.08U, and the fluctuations are very violent. If you run the long-short grid strategy at the beginning, it is difficult to escape the result of short-selling. The balance strategies are less of a problem.
The final return of the backtest is 4524U, which is very close to the return of TRX at 0.18. The leverage is lower than 2 times from the beginning and finally lower than 0.4, and the possibility of liquidation is also getting lower and lowerr, during which there can be an opportunity to increase the value of the position. But below 2000U is always the same income. This is also one of the disadvantages of the balance strategy.
symbol = 'TRXUSDT'
df_trx = GetKlines(symbol=symbol,start='2021-1-1',end='2022-5-30',period='5m')
#Actual leverage of occupancy
(res_trx.value/(res_trx.profit+1000)).plot(figsize=(15,6),grid=True);
Let's backtest WAVES again. This currency is quite special. It rose from 6U to 60U at the beginning, and finally fell back to the current 8U. The final profit is 4945, far more than the profit of holding the currency unchanged.
symbol = 'WAVESUSDT'
df_waves = GetKlines(symbol=symbol,start='2021-1-1',end='2022-5-30',period='5m')
By the way, the performance of the grid strategy is backtested, the grid spacing is 0.01, and the grid value is 10. In the case of nearly 10 times of the increase, both WAVES and TRX have experienced huge drawdowns. Among them, WAVES has withdrawn 5000U, and TRX has also exceeded 3000U. If the initial capital is small, the positions alomst will be liquidated.
#Grid strategy
pct = 0.01
value = 10*pct/0.01
e = Exchange([symbol], fee=0.0002, initial_balance=1000)
init_price = df_waves.iloc[0].open
res_list = [] #For storing intermediate results
for row in df_waves.itertuples():
buy_price = (value / pct - value) / (value / (pct * init_price) + e.account[symbol]['amount'])
sell_price = (value / pct + value) / (value / (pct *init_price) + e.account[symbol]['amount'])
while row.low < buy_price:
e.Buy(symbol,buy_price,value/buy_price)
e.Update({symbol:row.close})
buy_price = (value / pct - value) / (value / (pct * init_price) + e.account[symbol]['amount']) #The buy order price, since it is a pending order transaction, is also the final matching price=
while row.high > sell_price:
e.Sell(symbol,sell_price,value/sell_price)
e.Update({symbol:row.close})
sell_price = (value / pct + value) / (value / (pct *init_price) + e.account[symbol]['amount'])
if int(row.time)%(60*60*1000) == 0:
e.Update({symbol:row.close})
res_list.append([row.time, row.close, e.account[symbol]['amount'],e.account[symbol]['amount']*row.close, e.account['USDT']['total']-e.initial_balance])
res_waves_net = pd.DataFrame(data=res_list, columns=['time','price','amount','value','profit'])
res_waves_net.index = pd.to_datetime(res_waves_net.time,unit='ms')
print(pct,e.account['USDT']['realised_profit']+e.account['USDT']['unrealised_profit'] ,round(e.account['USDT']['fee'],0))
This time, the backtest analysis used the 5min K-line, the fluctuations in the middle is not completely simulated, so the actual profits should be slightly higher. Overall, the balance strategy bears relatively small risk, not afraid of skyrocketing, and there is no need to adjust the parameters, it is relatively easy to use and suitable for novice users. The grid strategy is very sensitive to the initial price setting and requires some judgment of the market. In the long run, the risk of going short is high. The current round of bear market has been stable at the bottom for some time, many currencies are currently down more than 90% from their highs, if you are optimistic about some currencis, this is a good time to enter the market, you may want to open a balance strategy to buy the bottom, add a little leverage and get profits from volatility and price increase.
The Binance Thousand League Battle will provide free access to the perpetual balance strategy, and everyone is welcome to experience it.
Log in
