Crypto wallets are essentially software programs used to securely store, transfer, and receive digital assets such as cryptocurrencies and fiat currency. These wallets are typically given by exchange platforms that handle significant volumes of transactions on a daily basis. Exchange platforms, whether they have their own crypto wallets or not, provide an API service to largely access basic wallet information from user accounts with the implementation of private keys for security considerations. API service for this type of sensitive information can only be accessed under specific conditions to fulfill security standards.
With CryptoIndexSeries, we strive to supply as much of our clients' private wallet information as possible in order to build a well-designed and ordered portfolio analysis.
Because privacy is as important to us as it is to our clients, we are developing a software application that will automatically retrieve their account history information from their desired platforms - as much of them as we can provide - so that they can have a better understanding of their assets and wisely control their wealth. But wait, don't crypto exchange platforms have their own data models? How do we project them in a single application without accessing our clients' personal information? In other words, we created our own normalized model with a data set intersection. The main issue here was to ensure that we did not lose critical information from the incoming data when converting their model to ours. As previously said, each exchange platform has its unique data models; let us go into the specifics of each platform's wallet apps.
To begin, we'd like to discuss Binance, which is by far the most popular trading platform in Europe. Binance has 4.5M active daily visitors and 25M daily pageviews, according to a worldwide survey of Regulated United Europe (Licensed Crypto Exchanges in Europe). Binance was one of the most challenging exchange sites to acquire a multitude of information from. Binance has a high volume of operations every day, thus they must consider the quantity of demand on their API service, which puts a strain on their services and systems. To restrict and manage it better, they created a symbol requirement for endpoints with huge amounts of data to split the amount of data being fetched in smaller pieces, which sounds fine but is a nuisance for developers. Rate restriction is another feature that practically all exchange platforms provide in their services. It is used to limit the quantity of traffic generated by an IP address or account. Binance has a high volume of operations every day, thus they must consider the quantity of demand on their API service, which puts a strain on their services and systems. To restrict and manage it better, they created a symbol requirement for endpoints with huge amounts of data to split the amount of data being fetched in smaller pieces, which sounds fine but is a nuisance for developers. Rate restriction is another feature that practically all exchange platforms provide in their services. It is used to limit the quantity of traffic generated by an IP address or account.
We do not only collect data from exchanges; we also interact with blockchains, but this is an entirely other issue that has its own article because blockchains have very different techniques than exchange platforms, therefore we will not include blockchains here. While this is one of Crypto Index Series's newest projects, only Binance and Coinbase have been effectively included into the normalization process thus far. The list of exchange platforms and blockchains to be incorporated grows and will be added to our system once it is completely completed and available for usage by our clients.
To continue, we have Coinbase on our list, which is another powerful exchange platform for which we have been collecting public market and private customer data. Coinbase has 1.15M daily visitors and 4M daily pageviews, according to a worldwide survey of Regulated United Europe (Licensed Crypto Exchanges in Europe). The problematic element for Coinbase is not the same as it is for Binance; as previously noted, Binance demands symbols for certain of their endpoints, making it harder to compile whole histories, but Coinbase wants account ids within a customer profile. A crypto asset or a fiat money can be represented via an account. Each account has a source name, the amount of source saved, and some source information. This information may be utilized in other endpoints to acquire more data from client accounts; however, it is important to remember that this will be done automatically - not manually - so we must design rules to avoid exceeding rate restrictions. Coinbase was not the simplest to normalize; in comparison to Binance, they performed a terrible job with documentation and model design. Instead of incorporating the transaction type in a model, they specified a property as the type, requiring each endpoint to require a new data model to receive the data, and therefore a new logic to normalize each data model.
To learn more about some of the challenges we faced while developing account history applications, as well as specifics about some of the exchange platforms we integrated into our system, check out our article from last month, Crypto Index Series - Private Authentication Variations Among The Exchange Platforms.
Crypto Index Series Development Team
April 2023
References