Link to the github repository
https://github.com/steemit/steem
Recently I have been interested in network analysis and I was examining the article Anatomy of Facebook where some parameters of the analysis of the Facebook network are discussed such as the cumulative degree distribution and the average degrees of separation.
Cumulative degree distribution: An important basic view of any social network is the cumulative degree distribution, which shows the percentage of individuals that have less than a given number of friends. As you can see above, only 10% of people have less than 10 friends, 20% have less than 25 friends, while 50% (the median) have over 100 friends. Meanwhile, because the distribution is highly skewed, the average friend count is 190.
Average degrees of separation: The average distance in 2008 was 5.28 hops, in 2011 was 4.74 and in 2016 was 3.57
I was wondering if I could do a similar analysis for Steem and compare it with the known Facebook values. To understand this analysis you have to think of Steem as a network where the nodes are the accounts and the links between followers and followed are the edges that connect the nodes of the network.
The information needed can be taken from the FOLLOWERS view of SteemSQL
In first place I will show some results that I have obtained using KNIME (an estimate of the cumulative IN and OUT degree distribution) but when I tried to calculate the average degrees of separation I found the difficulty of doing it. Researching on network analysis I discovered the NETWORKX Library in Python that I have used to estimate the average degrees of separation and to improve / confirm the previous results calculated with KNIME.
It should be clear that my analysis represents only an approximation since the processing capacity required to make a deep analysis of the entire network exceeds my possibilities.
Random networks vs scale‐free networks. A network is composed of nodes n, linked together by edges. The main parameter of network analysis is the degree k of a node (or connectivity), that is, the number of nodes to which a given node is linked.
The connectivity of random networks (on the left) is characterized by a Poisson distribution, whereas a scale-‐free (or clustered) network (on the right) is defined by a power law distribution. As a result, in a scale-‐free network, most nodes are sparsely linked (black nodes) while a few nodes are highly linked and constitute hubs in the network.
The size of this table is very large. It currently has 95,789,908 rows that represent the same number of edges between nodes or accounts.
To avoid extracting the entire table and enabling calculations of the python Networkx library in a reasonable time with standard hardware, I extracted a subset of the Followers table consisting of 8M records to make the analysis.
So, for this subset:
There are 226k unique nodes in the follower field and 77k unique nodes in the following field.
Some of these nodes or accounts are found in both fields as theses nodes have both types of behavior, follower and followed.
Seen in total, there are 274k unique accounts or nodes.
Expressing this data graphically in the form of a very simplified network, the following graph is obtained where it is appreciated that the behavior as a follower is the predominant one (72%) and the mixed behavior is the least (10.5%).
Grouping the data by the FOLLOWING field and counting the number of FOLLOWERS we obtain.
| NUM. FOLLOWERS | Number of nodes | % OF TOTAL |
|---|---|---|
| 1 | 7249 | 9.37% |
| 2 | 4736 | 6.12% |
| 3 | 2620 | 3.38% |
| 4 | 1586 | 2.05% |
| 5 | 1585 | 2.04% |
| 6 | 1856 | 2.39% |
| 7 | 2119 | 2.73% |
| 8 | 2183 | 2.82% |
| 9 | 2006 | 2.59% |
| =>10 | the rest | 66.51% |
The cumulative In-degree distribution shows the percentage of nodes/accounts that have less than a given number of followers. As you can see above, 33% of nodes/accounts have less than 10 FOLLOWERS, 45% have less than 20 followers, while 50% (the median) have over 25 followers. Meanwhile, because the distribution is highly skewed, the average FOLLOWER count is 103.44
Grouping the data by the FOLLOWER field and counting the number of FOLLOWED we obtain.
| NUM. FOLLOWED | Number of nodes | % OF TOTAL |
|---|---|---|
| 1 | 81924 | 36.13% |
| 2 | 31762 | 14.01% |
| 3 | 17920 | 7.905% |
| 4 | 12031 | 5.30% |
| 5 | 8644 | 3.81% |
| 6 | 6551 | 2.88% |
| 7 | 5417 | 2.38% |
| 8 | 4499 | 1.98% |
| 9 | 3781 | 1.66% |
| =>10 | The rest | 23.90% |
The cumulative Out-degree distribution shows the percentage of nodes/accounts that have less than a given number of followeed. As you can see above, 75% of nodes/accounts have less than 10 FOLLOWED, 84% have less than 20 followed, while 50% (the median) have over 3 followed. The average FOLLOWED count is 32.39
These results confirm the expected different behavior (with respect to the number of edges) that the nodes have in a scale-free network.
A node/account with followed behavior has statistically three times as many edges as a node/account with follower behavior
This is very clear looking at the nodes that have only one follower (9%) and the nodes that have a single followed (36%)
But as we know, a node/account can have followers and followed at the same time, the total number of edges being the sum of both. In this way we can talk about a mixed behavior.
In this case, the variable to be measured must be the total number of edges, called the degree of the node that corresponds to the concept of number of "friends" or "contacts" in a social network.
To find the average behavior in terms of degree I will use later the power of the Networkx library, but here I expose an intuitive way of characterizing the mixed-behavior using the averages (and other statistics) previously found for the behaviors like followed and follower separately.
That idea can be schematized in this key phrase:
"The average of the mixed behavior is the average of the averages of the behaviors like follower and followed."
Seen numerically perhaps it is easier to understand:
Finding the distribution of the mixed behavior from the distributions as follower or IN-DEGREE and as followed or OUT-DEGREE is not direct so I will obtain it later using the power of the Networkx library.
NetworkX is a Python package for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks. NetworkX is free software released under the BSD-new license.
The performance of Networkx is incredible but when processing a large network (large number of nodes and edges) the need for advanced hardware is required to calculate some of the characteristics of the network. That is why I have created a subgroup of data with 8M edges.
There are several ways to build a graph networkx "G" .One of them is to read a file in a certain format. Perhaps the easiest format that is the one I have used is the *** read_edgelist ***.
import networkx as nx
G = nx.read_edgelist ('edgelist_file.txt')
Where the file edgelist_file.txt consists of two columns, separated by a white space where the names of the nodes-accounts that form each edge or link are included.
follower1 followed0
follower2 followed0
follower3 followed0
.... etc
Knowing the number of edges and nodes of the created networkS is easy and fast using:
nx.number_of_nodes (G)
nx.number_of_edges (G)
| SUBSET | NUMER OF EDGES | NUMBER OF NODES |
|---|---|---|
| G | 7,943,889 | 274,224 |
NetworkX provides basic functionality for visualizing graphs, ( its main goal is to enable graph analysis rather than perform graph visualization).
nx.draw (G)
plt.show ()
I have been able to apply the nx.draw (G) function to a SMALL group of edges and nodes but it is enough to give an idea of the structure of the network shown in the following figure.
I have used two convergent ways to estimate this value. On the one hand, the AVERAGE SHORTEST PATH LENGTH as a lower value and the DIAMETER OF THE NETWORK as the higher value.
Average path length is a concept in network topology that is defined as the average number of steps along the shortest paths for all possible pairs of network.
For connected graph
nx.average_shortest_path_length(G)
The diameter is the maximum eccentricity. The eccentricity of a node v is the maximum distance from v to all other nodes in G.
nx.diameter(G)
Then, the estimated final value for the DEGREE OF SEPARATION FOR THE STEEM ECOSYSTEM should be within that range [3.61 - 6]
Adding more nodes of the network supposes in principle the possibility that the diameter of the network would grow but the edges of those new nodes added, according to what we know about the networks, would make the obtained value converges towards smaller values as has happened on Facebook in recent years. (4.74 in 2011 and 3.57 in 2016)
To know the real value for Steem we would need to do the calculations for the whole FOLLOWER view but I think the obtained value is coherent and reasonable.
Returning to the analysis of the joint (In-Out) degree distribution and its average value that I initially estimated at 67.5 I will recalculate these results using the Networkx library using the corresponding concepts of the DEGREE DISTRIBUTION and the AVERAGE DEGREE CONNECTIVITY
nx.average_degree_connectivity(G)
If we are analyzing an undirected and unweighted network, a node’s degree of connectivity will then simply be a summation of all the links that the node has.
Calculating...
degree_sequence = sorted([d for n, d in G.degree()], reverse=True) # degree sequence
degreeCount = collections.Counter(degree_sequence)
deg, cnt = zip(*degreeCount.items())
plt.bar(deg, cnt, width=0.90, color='b')
plt.title("Degree Histogram")
plt.ylabel("Count")
plt.xlabel("Degree")
From the values of this distribution the Cumulative Degree Distribution is calculated
Percentage of accounts with less than 20 friends / (followers + followed) is over 74.88%
50% of Steemit accounts have over 4-5 friends
NOTE:
It is necessary to understand that the subgroup of data used introduce different"effects" to the results: (1) Only accounts that have at least one friend (follower or follower) are being used, that is, those that do not have any friends are not included) (2) Only 8M edges of the existing 95M have been used. (3) The FOLLOWERS view is ordered by the following field, that is, it includes all the followers of a followed, but not necessarily all of the followed for a follower.
This last result of 4-5 friends (grade or number of edges or links) for 50% of Steem's accounts may seem surprisingly low. Maybe the real value (calculated using all 95M edges) is something different and possibly higher but not too far from the one obtained. If you examine your own number of friends (In+Out) this will be much higher than 4-5 but on the other hand there are a lot of accounts with very few links: 1 (29.5%), 2 (12.1%), 3 (6.7%), etc.
DATA SOURCE
I have used SteemSQL, a publicly available Microsoft SQL database containing all the Steem blockchain data held and managed by 
DATES
SQL MAIN QUERY
SELECT top 8000000 *
FROM Followers
ANALYSIS TOOL
I have used KNIME, a free and open-source data analytics, reporting and integration platform, to get, filter and manipulated data
KNIME WORKFLOW
SPIDER
The Scientific PYthon Development EnviRonment
NetworkX
A Python library for studying graphs and networks.