Author image

Malcolm Thirus

4 posts LinkedIn
Dirty Data Dancer, Spark Specialist, and an advocator of awesome AWS Applications. My experience ranges from Data Warehousing to Data Management and Data Exploration.

Get Your Head in the Clouds

There is a massive shift in mentality surrounding deployment and system maintenance. With cloud services handling the bulk of our DevOps for a fraction of the cost, companies are moving whole environments into the Cloud. But just how far can we take this? In the Amazon Webservices EcoSystem, we have

Apache Spark Datasets

With a Spark 2.0 release imminent, the previously experimental Datasets API will be a core feature. Spark Datasets were introduced in the 1.6 release as a bridge between the Object Oriented type safety of RDDs and the speed and optimization of Dataframes utilizing Spark SQL. Databricks has stated