I'm not a dev. I've been very clear about this from day 1. That said I've been helping to get some pretty critical apps on this chain and I've had to figure out a decent understanding of how Steem works. I'd like you to know too, so here's some of the basics and I'll do my best to have it be readable by non-devs.
tl;dr - There's one program for nodes and data on the chain. As you scale up how many different things you want the node to do you increase the database size, ram, and replay time by turning on plugins. Curation/voting apps need full RPCs, but there's a good chance that apps on chain that aren't about voting can get away with Compact Broadcast nodes ($150 or less per month).
One of the core tenents of the liberty movement is the call for opensource and decentralized programs. Opensource means that anyone can come along and read the code and put it in their own code repository. Decentralized means that there isn't just one server or one team behind something with full control. The control is spread throughout the network by the users themselves.
Gaming provides some good examples of why that's important. Some Counter Strike Go guys would play a lot, get custom skins for their guns and avatars worth hundreds of thousands of dollars, speak their mind about the company or game, and then they'd get their gear banned from the market. So, they're out hundreds of thousands of dollars.
There's one central server and one authority. It's a small corporate dictatorship.
This could still happen in a decentralized system, but it's no longer 1 person or company making the call. It would have to be the whole ecosystem, and if the whole ecosystem didn't agree it either wouldn't go through or the ecosystem would fork and split into two groups. One group would allow the banning to take place, one group wouldn't allow the banning to take place, and because of the nature of opensource and decentralized all people in the community get their say!
In New England the phrase is "what's good for the goose is good for the gander." What's good for the leader is good for everybody. But obviously, that's not true in the case of the cs:go guy down a couple hundred thousand bucks. So, crypto is the oppostie. What's good for me is good for me, and that's what I'm gonna do.
Having a say in the network protects your ability to chose what's right for you, and having access to opensource code provides ultimate remedy for any decision you don't like. ie "Ok, I don't like that, I'm gonna take this code and go on my own. I'm gonna fork off!" You have a choice of how you want to participate.
So, now that you know some of the ethos let's look at how that plays out in the Steem ecosystem through the servers that run it?
A blockchain is a public database.
If you ever need a short non-techy description of blockchain use that. It's just a database that anyone can access. To make it public devs had to figure out a good way to store data. You have two options: store the whole thing every time you save or just save changes as they happen.
Option 1 where you save the whole file works pretty good for your college excel sheet. You do a chemistry lab, you get a few data tables. You save the whole thing. It takes two seconds, and it's pretty quick and works great.
The Steem blockchain has 36m Blocks and just shy of 1B transactions in it. You hit save on that database and it takes a long ass time. So, to save time people realized you don't really need to save everything, you just need to save changes to the file.
Think about it, if I have a database with 40 M entries but in a given time only 50 things change why save the whole thing? Just save what changed. That's how blockchains are designed.
To get your data onto the blockchain you have to use a node. A node is part hardware and part software. A node is a computer that's designed to store data for users on a network and running software meant to facilitate sharing that data.
On Steem you will generally hear talk of 4 types of nodes: Witness, Seed, Compact Broadcast Nodes (new), and RPCs. There's only one node software for Steemd. What's happening is that more and more plugins are getting turned on. The node gets beefier and can handle more types of operations for the users. So, in a sense there's only 1 node, but because some of the plugings really unlock new critical features we give the different nodes running different plugins of the same software their own name and describe what they can do.
Somehow new servers latching on to this public network have to get data so that they too can share what's going on. The whole point of the blockchain is that it's a decentralized repository of information that anyone in the public can access. Redundancy is key and everyone has to get the same data that wants it. Somehow you have to be able to get the data in the database onto your server and do it in a trusted way.
Seed nodes have all the blockchain data on them, they are synched with the chain to show that they are running the same version as everyone else so that you know the data isn't corrupted, and other servers can ping seed nodes and download all the data on the server.
Seed nodes are the lighest version of a node with just the one key function: Help other nodes get data to stay current with the blockchain.
Seed nodes help disseminate the blockchain, but something actually has to help build the blockchain. That's what witness nodes do. Witness servers add the witness plugin to a seed node. So, a witness node is a seed node + 1 plugin.
The community votes witnesses using their stake and now witnesses have the authority delegated to them by the community to add new information to the blockchain.
Our chain's heart beats every 3 seconds. A new block is stored on the chain every 3 seconds. Inside that block are all the transactions that happened in the last 3 seconds. We're averaging about 40 transactions per block these days.
What kinds of data are stored on the blockchain?
The witnesses check out the data, make sure it's signed with the appropriate keys so no counterfit transactions are made, and use consensus to decide if the block is valid.
If it's valid there's a short period of time where it's reversible, editable, or may not be processed and halt the chain. Assuming that it doesn't throw up any major flags then the block is added to the chain and is irreversibly bound to it.
So, Seeds get everyone the information on the chain. Witnesses actually build the chain. A compact braodcast node is one of the lightest ways to interact with the steem blockchain and tell witness nodes what data should be added.
A Compact Broadcast Node is a Seed node + 5 plugins. Here are the plugins that we're running on a Compact Broadcast Node-
plugin = condenser_api network_broadcast_api rc_api account_by_key database_api block_api
developers.steem.io has more info than what I'm providing here, but I'm trying to give a high level understanding of what these calls do.
network_broadcast_api Broadcast is basically a request to the witness servers to include a transaction. If I want to actually get my transaction recorded on the chain I have to broadcast it with my key. This plugin let's this server broadcast transactions to the full Steem network.
block_api Splinterlands and Steem-engine use their own database for a lot of things. What we do is we inspect every block on the Steem blockchain and look for transactions we care about. We take what we need for our database and forget about the rest. Block_api is what allows us to get all the information in a block.
account_by_key Required for looking up details about accounts, such as their balances, account keys/authorities, and witness information (such as signing key, and last price feed).
rc_api - With HF20 Steemit and the Witnesses introduced a resoruce credit system. If you want to know anythign about an account's current resource credits you have to run the rc_api plugin.
database_api Used to query information about accounts, transactions, and blockchain data. So, now instead of looking up all the transactions in a block it's looking up specific information from accounts and individual transactions.
condenser_api - This one is kind of the kitchen sink. There's a ton of data that's grouped under here.
[
{
"id": 0,
"name": "",
"owner": {
"weight_threshold": 0,
"account_auths": [],
"key_auths": [["", 0], ["", 0], ["", 0]]
},
"active": {
"weight_threshold": 0,
"account_auths": [],
"key_auths": [["", 0], ["", 0]]
},
"posting": {
"weight_threshold": 0,
"account_auths": [],
"key_auths": [["", 0], ["", 0], ["", 0]]
},
"memo_key": "",
"json_metadata": "",
"proxy": "",
"last_owner_update": "2017-03-16T21:17:51",
"last_account_update": "2017-03-16T23:10:36",
"created": "2016-03-24T17:00:21",
"mined": false,
"recovery_account": "",
"last_account_recovery": "1970-01-01T00:00:00",
"reset_account": "null",
"comment_count": 0,
"lifetime_vote_count": 0,
"post_count": 0,
"can_vote": false,
"voting_power": 0,
"last_vote_time": "2016-12-04T23:10:57",
"balance": "0.000 STEEM",
"savings_balance": "0.0 STEEM",
"sbd_balance": "0.0 SBD",
"sbd_seconds": "",
"sbd_seconds_last_update": "2018-04-10T04:08:36",
"sbd_last_interest_payment": "2018-03-15T17:58:24",
"savings_sbd_balance": "0.000 SBD",
"savings_sbd_seconds": "",
"savings_sbd_seconds_last_update": "1970-01-01T00:00:00",
"savings_sbd_last_interest_payment": "1970-01-01T00:00:00",
"savings_withdraw_requests": 0,
"reward_sbd_balance": "0.000 SBD",
"reward_steem_balance": "0.000 STEEM",
"reward_vesting_balance": "0.000000 VESTS",
"reward_vesting_steem": "0.000 STEEM",
"vesting_shares": "0.000 VESTS",
"delegated_vesting_shares": "0.000000 VESTS",
"received_vesting_shares": "0.000000 VESTS",
"vesting_withdraw_rate": "0.000000 VESTS",
"next_vesting_withdrawal": "1969-12-31T23:59:59",
"withdrawn": 0,
"to_withdraw": 0,
"withdraw_routes": 0,
"curation_rewards": 0,
"posting_rewards": 0,
"proxied_vsf_votes": [0, 0, 0, 0],
"witnesses_voted_for": 0,
"last_post": "2016-03-30T18:30:18",
"last_root_post": "2016-03-30T18:30:18",
"vesting_balance": "0.000 STEEM",
"reputation": "",
"transfer_history": [],
"market_history": [],
"post_history": [],
"vote_history": [],
"other_history": [],
"witness_votes": [],
"tags_usage": [],
"guest_bloggers": []
}
]
Well, a full node is a monster. It ranges in price, but can be fairly resource hungry. It also take forever to replay.
A compact broadcast node can be used by apps that don't need all the voting/curation features. So, Splinterlands and Steem Engine don't rely on account history. They have their own database for stuff like that, but they do need to get the data from the chain. So, we can use Compact Broadcast Nodes to get the data and fill up our own databases.
Currently we're running one of these on a Mira box, and we're adding a second one on a non-mira box. That should make it the tools more redundant and also substantially faster to get back up and running in the event the chain goes down for any reason.
When you get to "Full RPCs" they are essentially Seed nodes with many or all the plugins turned on. They can do more types of calls going further back in the blockchain history.
Now full nodes are running these plugins. If you're running apps that aren't voting or curation based then this isn't necessary. But programs that want to track all the voting and curation history of accounts will need a full node to operate.
These can operate with or without Mira. Mira makes them cheaper to run because it takes a lot of resources out of ram and puts it on a hard drive where it's drastically less expensive. It's also becoming somewhat customary to run full nodes split across more than 1 server so that Account History, which is one of the largest files in the database is on it's own server and then a second servers fulfills all the other parts of the data requests.
If the challenge is "I will need all the data available on the Steem blockchain to effectively run my application" then a full RPC is the tool that let's you do that.
basic plugins
plugin = witness
plugin = webserver p2p json_rpc
plugin = database_api network_broadcast_api condenser_api block_api rc_api
Account history
plugin = account_history_rocksdb
plugin = account_history_api
required for get_accounts / get_witness
plugin = account_by_key account_by_key_api
internal market history
plugin = market_history market_history_api
The key difference between full nodes and non-full nodes is the low memory setting. If you have low memory mode turned on, then regardless of what plugins you have enabled, it's simply not possible to view posts and things like that as well as some things related to voting.
Full Node:
Compact Broadcast Node
Witness Node
Seed Node
Jussi is software written by Steemit Inc to make nodes compatible and facilitate data calls. It allows node operators to route an API call to nodes that are optimized for the particular call, as if they are all hosted from the same place.
So, if you have 30k requests from Steem-Engine that only need get blocks and 10k requests that require account history for steemit.com Jussi can route all the Steem-Engine requests to a smaller box so that you're not using an expensive server to pull inexpensive requests.
https://developers.steem.io/services/#services-sbds
Is not a Steem specific tool. For the public at large it's an open source load balancing system designed to optomize limited computer resources. It helps process requests to not just dump all calls on one server, but instead take calls and route them to different servers so no one server is overloaded.
As an example: if there are 30k Steem-Engine requests and 10k condenser requests Jussi will route the Steem-Engine ones to less expensive servers, and Nginx could make sure that multiple less expensive servers are doing the work rather than 1 getting slammed and others sitting idle.
Specific to Steem though Nginx is mostly used as a web server designed to handle HTTP(s) requests and things like SSL or routing websockets to the right place. Essentially, it's a way to use 1 server for more than one purpose. Nginx makes it so that calls to a domain can get routed to the RPC part of a server or to the part that loads a website depending on what the call is asking for.
Now the same server can be used on Splinterlands to show you card art on the site or tell you who owns a specific level 6 pixie on the backend.
So, there you have it. A node is a server that runs the Steem Blockchain program (Daemon). Nodes can have a variety of plugins enabled depending on how many different types of data and how far back that data needs to go. As you increase plugins you get more data calls allowed, but you also increase the price and replay time.
These nodes are the lifeblood of the Steem ecosystem, and now you hopefully have a better understanding of what they do and how they do it.
I haven't shilled a product that I haven't helped build almost at all on this blockchain, but I'm extremely grateful to someguy123 and privex. They've helped me a ton, built tools with me, become my business partner, and I honestly think they are fantastic.
Some people tell me that I can get the same servers for less money per month by working with a different company. That may be true, but it doesn't come with the service. Most days as a witness that doesn't matter, but on the days the blockchain is broken nothing matters more. I know I'll get the help I need by someguy123 or his team by working with him. I'll also get good advice on how to build tools for exactly what I'm doing on Steem.
If you're considering biulding an app as you're realizing that the costs could be as little as $50-200 a month and the potential revenue is staggering I'd encourage you to check out Privex.io.
If you sign up help pay some of my server bills and use this affiliate link: https://www.privex.io/[email protected].
No, I'm not getting paid for this. Yes, I just like someguy123 and privex that much.