Pen's Science and Programming #1, A Learning R Project - Part 1
Unix Epoch: 1563453215
So...
I'm an Electrical Engineer by education and first career... a homemaker, mostly, by second career... Anyway...
As an engineer, I have mostly worked with an open source numerical data analysis tool called GNU Octave, easily installable on my go-to FOSS desktop of Debian. This program duplicates much of the functionality of the proprietary program called Matlab which is quite expensive to obtain and my department in my prior place of employment would not appropriate the funds to purchase. So, I started working with and began to really love GNU Octave.
Over the last few years, I've performed a smattering of odd analysis for a few contract dollars here and there. However, I also discovered GNU R. R is used by many of the top level science and technology labs in the world. So, in an attempt to learn this awesome and fun tool, I purchased this book and got right to work placing on my shelf for the next 3 years.
Ahh! The best laid plans of mice ...... and men.
At the time the idea to learn R occurred to me, I was reading historical articles on pollen levels throughout the 20th century. Luckily, I have access to the historical New York Times online through my local library. And I discovered several articles detailing multi-year studies on pollen throughout the dust bowl era of the 1930s as well as the seeming downturn of pollen levels in the cooler 1970s. The articles were interesting, but the data contained in each analysis were not directly comparable. The pollen counts sometimes were only listed as a general level, such as: High, Moderate, Low. And sometimes actual numerical values were used. This mix of values and levels made it rather difficult to figure out what was going on historically. It also lead me to the thought that a min/max metric is probably not a good metric for pollen.
I discovered two seemingly decent online data sources for obtaining cost free pollen information:
Both of these sources of data have an email subscription service, and I subscribed to both. AAAAI has several data feeds, and I selected three locations across the Continental United States. So, at this point I have collected three complete years of pollen data from the following four locations:
- Southeast - Atlanta, GA
- Northwest - Eugene, OR
- Central - Kansas City, MO
- Mid-Atlantic - Springfield, NJ
I'm in the process of converting the email texts into pertinent tabular data, and will be utilizing my burgeoning database skills with postgresql along with GNU R to analyze what I've collected thus far. This will be an ongoing project, and I will be updating this series of posts from time to time. However, it is Summer here, and I have significant family responsibilities. So, the going may be slow until the kids go back to school in September.
Request for Suggestions
I've been looking around for a suitable blockchain enabled raw data repository. Ideally, I would like to distribute all my raw data as well as all programming and analysis coding. However, the Steemit platform seems ill-suited to that particular endeavor.