[EN]
This post is intended to cover the server costs for SteemWorld.org and to enable me to continue my development in the future.
In case you missed the first part:
SteemWorld ~ Weekly Support ~ #1
[DE]
Dieser Post ist dafür gedacht die Serverkosten für SteemWorld.org zu decken und mir in Zukunft weitere Entwicklungen zu ermöglichen.
Falls du den ersten Teil verpasst haben solltest:
SteemWorld ~ Weekly Support ~ #1
Updates
The past week proceeded not exactly as I had planned before and I ended up with optimizing the implemented data caching methods for all clients on SteemWorld. Don't ask me how I got there, but I'm glad to announce that users now can open as many accounts as desired simultaneously without longer increasing the server load on my main data server.
As some of you might know, I don't make all required Steem RPC requests from each client to the connected nodes directly. I solved it in a way so that my server does the requests once a minute and stores the request results in a 'requests' table. The client (your browser) just sends a get_status request to my API once every 30 seconds and receives all data like new mentions, new followers and the cached Steem requests with only one request.
That takes much load from the public Steem RPC nodes, but of course, it increases the load on my side a bit, therefore I needed to make a few changes to the existing logic.
The following Steem API requests are being cached on my server:
const CACHED_GET_DYNAMIC_GLOBAL_PROPERTIES = 1;
const CACHED_GET_CURRENT_MEDIAN_HISTORY_PRICE = 2;
const CACHED_GET_REWARD_FUND = 3;
const CACHED_GET_HARDFORK_VERSION = 4;
const CACHED_GET_NEXT_SCHEDULED_HARDFORK = 5;
const CACHED_GET_ACCOUNT_COUNT = 6;
const CACHED_GET_WITNESS_COUNT = 7;
const CACHED_GET_TICKER = 8;
const CACHED_GET_CHAIN_PROPERTIES = 9;
A week ago my server always answered each get_status request with all cached data, because it didn't know what the current client exactly needs. That is now not longer the case and the clients send the required cached requests data IDs along the status request as shown here (abstract request):
{
"type" : "get_status",
"params" :
{
"ids" : [ 1, 2, 8 ],
"user" : "steemchiller",
"last_follow" : 94172775,
"last_mention" : 94238931
}
}
The example above would return all new mentions since the in field last_mention defined unix timestamp, all new followers since the in last_follow defined timestamp and all in the field ids defined cached requests (in this case GET_DYNAMIC_GLOBAL_PROPERTIES, GET_CURRENT_MEDIAN_HISTORY_PRICE and GET_TICKER).
After receiving the cached requests the client stores them in the browser's local storage, so that all other simultaneously opened SteemWorld tabs can share the same result and each of them knows about what is being required for the next get_status call. The good thing about using such a caching method is that it decreases the server load significantly and also the clients benefit from using more locally cached data, because they do not need to load blindly all data via internet connection each time.
Improved Content Security Policy (CSP) Headers
Special thanks go to @scorer for mentioning SteemWorld's bad CSP headers in my last post! To be honest, I didn't take care of the CSP prior to his comment, but I'm glad to have achieved a very good rating now:
Sites can be tested with:
https://securityheaders.com/?q=steemworld.org&hide=on&followRedirects=on
To Firefox users:
If you should encounter any issues when loading the site, please let me know and I will adapt the (now very strict) CSP settings. I say this, because when using Firefox there appear two logs in the Developer Console stating 'The page’s settings blocked the loading of a resource at eval/inline (“script-src”)', which don't make sense on my site (might just be a FF bug, but I will try to find out more about it in the coming days).
Continuing Main Mission
Building the Post Editor (yes, it can be used to create new posts and comments too) is still my main mission on a daily basis. There is not much to say about it currently, but I can tell you that I'm doing my best to get it running soon ;)
I wish you all a wonderful week!
Planned Open Source Steem Data Service (SDS)
- Runs on Linux, Mac and Windows
- Can be run locally or as public web service
- Enables each of us to analyze and work with the Steem blockchain data in a simple and very efficient way (SQL and custom queries can be used too)
- Syncing is possible via available public instances (very fast) or regular Steem nodes
- Provides automatic downloading of already parsed data (no need to start from block 1)
- No need to setup any database services (all file-based -> no layers in between -> lightning fast)
- Offers configurable datasets (if you just need a DB with all transfers for example, you can run an instance with only the desired parser module 'transfers' enabled)
- JS developers can easily implement own parser modules (just add a custom script in the modules path and the service will call the
parseBlock,parseTransaction,parseOperationandparseVirtualOperationmethods within your script for each block) - Also 'virtual operations' can be parsed (or downloaded from one of my servers that will provide all parsed datasets in nearly realtime)
- Highly compressed data (some crazy stuff and a bit of magic involved here, more details on this soon)
- Interfaces to other database systems planned
- Completely free to use, will be released under MIT license
In the making
( A )
- Post Viewer and Editor
- SDS
- More details in Coming Rewards (number of posts and comments, pie charts)
- Improved Coming Curation Rewards
- Thinking about if/when to start a witness node
( B )
- Different views, no limit in the Posts Overview (requires SDS)
- Account Operations without limits, custom date ranges and filters, jump to day x (requires SDS)
- Exact (separated SP, STEEM and SBD) All Time Rewards in the Stats Section (requires SDS)
- Rewards Overview Tool with searching capabilites (requires SDS)
( C )
- RC costs calculation
- New responsive SteemWorld that can be run locally (Open Source)
Links
EN | What is SteemWorld?
-> Welcome to SteemWorld.org!
DE | Was ist SteemWorld?
-> Willkommen auf SteemWorld.org!
Thank you for supporting my work!
Positive thoughts create a positive world. It all begins with connecting our thoughts with emotions. What we focus on is what becomes powerful. We are the creators of our world.
Much love,
