Hi there. As some of you know, I have been leading the effort on the new Steem Engine DEX for the past few months. If you have ever used the current Steem Engine exchange, then you would be aware of its shortcomings and missing features.
One of the biggest complaints about the current DEX is speed. When you actually look underneath the hood of the current DEX, you will see that there is a lot of data needed to render some of the views. Opening up developer tools on the exchange view, for example, you will see request after request.
Other annoyances include having to click twice when signing in to the exchange (this annoys me too) and other various little caveats which sodden the experience a little bit.
Everything Reimagined
While the current exchange has served us well, Steem Engine 2.0 is totally rebuilt from the ground up. At a glance, this is what is different on a technical level.
- Developed using the Aurelia Javascript Framework, using TypeScript and modern web API's
- A contributor friendly codebase with a neatly organised structure, naming and conventions to find parts of the application.
- ESLint for code linting (ensure contributions are all to the same standard)
- Prettier for code formatting and standardisation
- New JSON Web Token (JWT) authentication layer (you actually log in to the DEX now)
- State management for handling client-side data
- HTTP/2 enabled API and resource requests to take advantage of multiplexing and other features in the updated HTTP specification which speed up web application performance.
- GraphQL for smarter data fetching. The site will make a smaller amount of requests for needed data. A data fetching cache layer where possible has been introduced to lessen the roundtrip latency of some queries.
- Eager and optimistic loading. Instead of making users wait for data to load, views will intelligently load data in a micro-app fashion where individual components on the screen load independently of one another.
- CSS Modules for safer styling of UI components
- Covered in unit tests (this is an ongoing effort) to produce smarter and safer deploys
Speed
We touched upon this in the opening paragraph, but the current DEX admittedly is slow at times. In the new DEX, we focused on consolidating requests and leveraging the power of GraphQL for instances where multiple sets of data are needed.
In most screens (especially the request heavy exchange view) we have been able to reduce the number of requests upwards of 80%. One such improvement is on the exchange view, two requests were being made; one for every single token with a limit of 1000 and one for every single token metric with a limit of 1000.
These two requests were absolutely massive from not only a payload perspective, but they're not cacheable because the SSC Javascript package uses POST requests to make RPC calls to the API. They contributed to the slowdown, so I set out to rework this in the new DEX.
Instead of making every token request, it should only request the current token it is on as well as the token it is paired with. At present, we only have STEEMP as the main pairing token, but there will be other pairs in the future. Why load all of those hundreds of other tokens if you only need one? In this one area, the speed of the data fetching request was increased 100x, a fraction of the payload size.
The general approach is less is more. Only querying for data that we need, when we need it. Instead of doing the filtering on the client, leveraging the API and its support for querying parameters to only request and filter what we need, passing it back to the API.
This is still ongoing work, but early results are already impressive.
100% TypeScript
While TypeScript does not guarantee complete safety, it offers some type of safety when it comes to working in a growing and distributed codebase. As the number of contributors grows, it is important that contributors are able to find their way around the codebase and the code documents itself.
Thanks to intellisense and types, anyone working on the Steem Engine codebase can be confident they're providing the right values to functions, get documented for arguments on functions and other features that TypeScript offers.
JSON Web Token (JWT) authentication layer
With the current DEX, you do not technically log in to the application. As many discovered, the username is just stored in localStorage and calls are made through Keychain or Steem Connect. In the new DEX we needed a way to verify users through a stronger authentication process, so JWT authentication was introduced creating authentication tokens using a series of token swaps and handshakes to ensure a user is who they say they are.
The reason for this change is KYC. While KYC is not a requirement, some upcoming features of the site will require users undergo KYC and we needed a safe way for users to upload KYC documents securely and verify that they are the only ones uploading their documents.
The introduction of JWT auth does not mean all operations will not ask for your permission. The Steem Keychain or Steem Connect service will still be used to authorise each and every single blockchain action, keeping you in control of your account and funds.
State management
When you are building large-scale web applications, client-side state management becomes crucial. When you are dealing with the troves of data that Steem Engine is, even more so. The new DEX is utilising state management in all areas from tokens to metrics and even the logged in user.
Data freely moves around the application in a safer manner, with the ability to trace the path of data and determine how, when and why it changed.
GraphQL
At present, the DEX uses a series of RPC calls via the sscjs client. While this works fine, it has limitations which prevent basic things such as caching being implemented. It also offers up challenges in terms of the number of requests being made to get data.
With GraphQL, we can query the Steem blockchain and Steem Engine specific data and bring it back in as many requests as we want. As mentioned in the speed section above, we are leaning on it for the exchange view to speed things up quite considerably.
GraphQL is the future of API's and integrating it with Steem and Steem Engine seemed like a worthy task. If you want to run your own GraphQL server for Steem and Steem Engine, the code is available for free here.
Disclaimer
Before getting into the UI parts of the new app, I just want to point out all of this is still a work in progress. While technically the exchange is quite complete now, there are still some UI parts that need some more thought and tweaking. What you see here might not look the same as the final version.
Localisation Localization 현지화 本土化
Some of you might know, but I am Australian and in Australia we speak British English. So, unlike our American friends we use "s" a lot in place of "z". The same thing goes for those in New Zealand, Great Britain and other Commonwealth countries that speak English.
Understandably, not everyone speaks English and not everyone likes being forced to use British or American style English. So, in the new DEX we've completely localised the UI so you get to decide. At launch we are going to have support for the following languages:
- American English
- British English
- Korean
- Chinese
Here is what the exchange UI looks like in Korean.
How about the tokens screen in Chinese (Simplified)?
The localisations in the app are not perfect, at times they might be wrong. This is where YOU come in. Do you speak a language other than English and would you like to see it supported in the new Steem Engine? We want your translations and improvements.
The process has been made as simple as possible for contributing locales. It's a simple JSON file with the key (original spelling and then the translation: https://github.com/steem-engine-exchange/steem-engine-dex/tree/master/src/locales
User Settings Screen
This is a feature that has been a long time coming, a user settings panel for Steem Engine. This area will allow you to manage personal information (part of the newly introduced KYC process), as well as various settings on the Steem Engine exchange relating to your account.
You will be able to hide zero balances from your wallet. You will also be able to toggle a mode which only shows your favourited tokens on the wallet screen. If you're like me, you're not really concerned with most tokens except for a select few, it will make things easier. These settings can also be changed on the wallet screen itself.
Favouriting tokens is as easy as typing the token name into the input area and selecting it. You can choose multiple tokens that you want to mark as favourites. You can also favourite tokens from the tokens page as well as the wallet screen as well.
New Exchange UI
If you watched Aggroed's Steemfest talk, then this screen will already be familiar to you. It is the most significant part of the new DEX where most of the work has been done on a technical level. The new exchange screen allows you to buy and sell Steem Engine tokens faster and with more information than ever before.
The biggest addition here is candlestick charts allowing you to visualise the rise and fall of your favourite Steem Engine tokens. The data only gives us a daily glance for now, but work is being done to allow for smaller intervals such as 5 minute, 15 minute and 1-hour charts.
There is still some work to be done on this screen in terms of colours, typography and spacing, so excuse the UI quirks that are visible for now.
New Wallet UI
The new wallet UI looks familiar as it is based on the old DEX design, only it has been given a refresh and had some improvements made. The icons now all display correctly and even on smaller screens.
Other Changes
I could go on all day talking about the new changes in the upcoming DEX, but we'll summarise here with a few other smaller (but still important points).
No test? No merge
The previous DEX had no tests or process for handling user contributions, so there were instances where contributions would sometimes break something on the DEX. While these contributions were tested, it can be impossible to know of side-effects in your application if there are no tests.
Going forward, while we still invite the community to make contributions to the DEX and we hope that the new codebase makes this not only easier but more straightforward, we will have a "no test, no merge" policy where all changes require accompanying tests and ensure that existing tests still pass.
This is an essential change as we have moved to a continuous deployment process where work is managed via GitFlow and then automatically released through Travis CI off of the master branch.
Continuous Deployment
Leading off of the last point, the new DEX is continuously deployed through Travis CI. Now that we have a proper testing process in place, we can confidently (to a certain degree) be sure that the DEX is in a functional and working state if the tests are passing.
The previous DEX required manual deployment, and admittedly, the process was flawed, allowing for potential errors in the deployment process.
Want to Contribute?
The code is public here on GitHub and anyone who would like to contribute, please reach out to me on Discord because the more the merrier. My Discord handle is beggars aka T-RexOfBinance#4504.
Stay tuned for part two where I will dive into some deeper technological aspects of the new Steem Engine.