Author Archives: Lucas Allen

Spark Dataframes and MLlib

Spark Dataframes and MLlib

spark-logoA couple of weeks ago, I got my first experience with Apache Spark. While I have yet to implement it with any meaningful problems, in my experience when working with a new tool or technology, just getting one’s feet wet can be crucial to getting a learning snowball rolling. Although Spark is primarily used for “big data” problems in data clusters, I have been experimenting with a very “small data” problem, a simple linear regression on California home prices. You can find the data set here. I’ve decided to put the resulting tutorial up on Tech Powered Math. Although there is nothing earth shattering in this post, I think some people will find it helpful for the following reasons.

  1. The method I used for working with the data is dataframes. Dataframes are a relatively new paradigm in Spark. They have only been available since Spark 1.3 in February 2015.
  2. I am using the Python API. While I suspect that PySpark is going to grow rapidly in popularity, there seem to be more resources for Scala at this time.
  3. I could find very few tutorials or even significant Q&A threads about using PySpark syntax and dataframes on Stack Overflow. That gives me cause to believe that even this simple tutorial about reading a CSV into Spark, doing some trivial data wrangling with dataframes, and performing a linear regression could be helpful to some individuals.

The code in the Jupyter Spark notebook below was completed with Spark 1.31.

Cost of College Infographic

If you’re a current college student, this infographic about the rapid rise in the cost of college will come as no surprise to you. However, if you’re in your mid-30’s and have kids like me, you may be shocked to learn how rapidly the cost of tuition, housing, and books has risen in the last… Continue Reading

Some TI-84 Plus CE Launch Models Will Require Immediate Upgrade

UPDATE (5/7/2015): Texas Instruments has released the new version of their TI-84 Plus CE operating system, version 5.0.1.0012. It can be downloaded from education.ti.com. You’ll also need the new version of TI-Connect. If you run into any problems on the upgrade, TI customer support can help you out at 1-800-TI-CARES.   Tomorrow, you’ll be able… Continue Reading

The Evolving TI-84 Plus Platform

The Evolving TI-84 Plus Platform

With a platform that’s been around as long as the TI-84, which is in and of itself the evolution of the TI-83 (launched in 1996), we’re bound to see models come and go. On Texas Instruments’ official website, I’ve noticed that certain models related to the TI-84 Plus were no longer featured. It also appears that… Continue Reading

TI-84 Plus CE Week

TI-84 Plus CE Week

It’s TI-84 Plus CE week here at Tech Powered Math. I’ve had a couple of weeks to interact with the new TI-84 Plus CE. I also recently had a nice informational chat with Texas Instruments’ reps about some TI-84 Plus CE news and how the TI-84 product line is evolving. All that added up to more than I… Continue Reading

Texas Instruments Launches STEM Behind Health

Texas Instruments Launches STEM Behind Health

It recently came to my attention that Texas Instruments launched a new initiative called STEM Behind Health. STEM Behind Health is designed to get students excited about health related careers. Activities from the initiative include a TI-Nspire document file as well as student worksheets and teacher notes. They were developed in conjunction with health care… Continue Reading