Grab Post from Steem API, Analyse Post using Datum Box Machine Learning, Initial Styling

Steem Analyser is a post analyzer that allows a user to enter a post URL and be presented with a list of results relating to the analysis of that content. It uses a collection of machine learning APIs to analyze a user's post. This release provides the foundation for further integration with multiple machine learning providers and deeper integration into the steem blockchain.

Within this release I have created the following features:

1. Ability to grab and display post
2. Integration into DatumBox machine learning API's to grab and display results for adult content, spam, author gender, commercial detection, educational material & sentiment.
3. Basic Styling

A screenshot of the overall experience which has currently been created can be seen below:

What & How was this created?

The first thing that I created within this release was the ability to grab a post and display it, this can be broken down into a few different parts such as:

1. Ajax Request to grab content
2. Define & Update
3. Convert Body from Markdown to HTML
4. Display Result & CSS

Firstly I created the following function which is used to grab the users post from the steem blockchain, in order to make a request I used the getContent call which requires you to pass in both author & permalink. In order to get the author a permalink I used the following regular expressions:

• Identify Permalink from URL: ()?([a-zA-Z0-9-]+)/?$/igm
• Identify Author from URL: /@[a-z][a-z0-9-]+[a-z0-9]/igm

For the request mentioned above, I also need to strip the '@' symbol from the match found with the above regular expression for the author. I did this by turning it into a string and then replacing the '@' with ''.

Following this, I then defined the items I need to display within my results and used them to create the HTML in index.html.

When grabbing the body of a post from the API it is displayed in markdown, in order to display this on my page correctly I needed to convert this to HTML, I did this using Remarkable.js using the following code:

   var md = new Remarkable();
   var body = md.render(body);

The function I used to grab the posts and generate the HTML for the post can be seen here:

Following that, I created multiple integrations into a machine learning provider called Datum Box (http://www.datumbox.com/machine-learning-api/). Within this release I created an integration for the following:

1. Sentiment
2. Commerical Detection
3. Spam Detection
4. Author Gender Detection
5. Educational Detection
6. Adult Content Detection

The method for each of these are the same, I simply passed the body of the post grabbed in getPosts into the function and was able to generate a result which I was able to pass into the HTML and display as a result for that post.

An example of one of the functions which are used to grab and create results for these integrations can be seen in the below screenshot:

A full list of the functions to generate the results from Databox are listed below:

https://github.com/tobias-g1/steem-post-insight/blob/master/js/adult.js
https://github.com/tobias-g1/steem-post-insight/blob/master/js/commerical.js
https://github.com/tobias-g1/steem-post-insight/blob/master/js/education.js
https://github.com/tobias-g1/steem-post-insight/blob/master/js/gender.js
https://github.com/tobias-g1/steem-post-insight/blob/master/js/sentiment.js
https://github.com/tobias-g1/steem-post-insight/blob/master/js/spam.js

Note: I'm aware that I could have done these in single function if I desired, however in future releases I believe I will begin to substitute some of the above for providers who are able to provide more detailed results.

Finally, I applied some basic styling to the page, I used a gridview in order to divide the page. This is a 12 column grid view provided by bootstrap, I do not use any other bootstrap styling within the project. I introduced some of the issues on reduced width windows/mobile when making adjustments to the overflow for the post columns vs the results and will resolve in a later release where I will design and implement a new responsive style.

Commits

The following commits are related to this release:

(Ordered Newest or Oldest)

https://github.com/tobias-g1/steem-post-insight/commit/8f2acf6de2ac92b99e72efa6cdaf53ef068eb5aa

https://github.com/tobias-g1/steem-post-insight/commit/0951578c730ea6e42c6424b4bebc3091366ec20a

https://github.com/tobias-g1/steem-post-insight/commit/a954469b7a571d51bb8713b9a2b01d8b5102e395

Current Limitations

1. Limited Validation on post URL field
2. Errors on reduced width windows & mobiles
3. Limited Integrations & Limitations in detail of results returned from API
4. Spelling Errors & Badly written copy

Roadmap

1. Mobile Ready Design
2. Animation and Custom CSS for results sets
3. Deeper integrations with other opensource machine learning providers for things such as keyword/entity detection/extraction, details sentiment analysis using a more relevant data set, spelling and grammar.
4. Ability to share results on steemit
5. General Clean Up

---

Posted on Utopian.io - Rewarding Open Source Contributors

---