The National Oceanic and Atmospheric Administration (NOAA) has just partnered with Amazon Web Services to make a huge amount of historic and current radar data publicly available. In this tutorial we will show you how to use CartoDB to map this data to explore weather!

Weather effects us all in many different ways, be it knowing if its hot enough to go to the beach or getting advanced warning of a destructive storm approaching your city. One of the major world-wide organizations tasked with monitoring our weather is NOAA who among other things operate a radar network that covers all of the US.

We will walk you through how to get the data off Amazon S3 and into CartoDB to make both high-resolution polygons and torque-based animations using the NEXRad data.

Accessing the data.

The data lives on Amazons S3 services which is a great service for storing large amounts of data. To grab the data out of there we will need to know the structure of the data. The data lives in the “noaa- nexrad-level2” bucket and is split in to files and stored in a directory structure that follows a format of:

/Year/Month/Day/NEXRAD Station/filename

This allows us to grab just the data we want from the dataset. This can be done either via amazons command line utilities.

To list files available for a given year month day and station we can use
the command:

aws s3 ls --recursive s3://noaa-nexrad-level2/2010/01/01/`

and then download one of those files using the command:

  aws s3 cp {path} ./

You can also access the files via the AWS API kits for your favorite language or libraries like boto for python which we will show an example of in the
dynamics maps.

Static maps.

Once we have downloaded the data, we need to convert it to a format that CartoDB can load. Lets grab the data for Hurricane Arthur which hit off the coast of the North Carolina. Using the map above we see that the MHX station is close to the storm site so we can see the files available using this command:

aws s3 ls --recursive s3://noaa-nexrad-level2/2014/07/03/KMHX/

Let’s grab the file at 2014/07/03/KMHX/KMHX20140703_182118_V06.gz

aws s3 cp s3://noaa-nexrad-level2/2014/07/03/KMHX/KMHX20140703_182118_V06.gz ./

Now let’s open this in the climate toolkit. The first thing we want to do is select the elevation at which we’ll slice the data. Let’s take it as 0.54. This should look like this:

Frame the view using the zoom and move tools, and then we are ready to export the data to a shapefile. Click the data tab on the side then click on export. Enter in a location to save the data, select the elevation we had before, 0.54. When asked what the spatial extent is, just click next, it will grab this info from the view window. Similarly hit next on the following screen then start export. This will produce the shapefile for you. It will take a while.

It should have produced 4 files. One with the extension .dbf, .prj, .shp and .shx, select all of them and create a zip file from them. Next simply upload this
to CartoDB.

Once it’s loaded, head over to map view and select the positron dark basemap and then select the choropleth wizard. Set the Polygon Stroke to 0 and the column
to value.

You should now have an awesome map of our Hurricane!

Dynamic maps.

To make the map animate, we are going to have to use Torque which unfortunately doesn’t work just yet with polygons. Instead we are going to have to convert the data to a CSV and upload it to CartoDB.

We have created a python script to do just this which you can find here. You will need to install a few packages which we recomend you do with Anaconda.

conda install boto
conda install -c https://conda.anaconda.org/jjhelmus pyart
pip install pygeo

Once you have done this we can select the data for your dynamic map by changing this line:

keys = grab_list_of_files("2014", "07", "04", station="KMHX")

We have set it up here to grab all the data for Hurricane Arthur on the 4th of July 2014. Running the script will take a little while but will eventually produce a csv file called results.csv. Upload this to CartoDB and select Torque Cat as your map-type in the Wizard.

Select the CartoCSS panel and paste in the following CartoCSS.

Map {
-torque-frame-count:30;
-torque-animation-duration:5;
-torque-time-attribute:"date";
-torque-aggregation-function:"avg(value)";
-torque-resolution:1;
-torque-data-aggregation:linear;
}

#result2{
  comp-op: source-over;
  marker-fill-opacity: 1.0;
  marker-line-color: #FFF;
  marker-line-width: 0;
  marker-line-opacity: 1;
  marker-type: ellipse;
  marker-width: 1.2;
  marker-fill: #0F3B82;
}
#result2 [ value <= 47.0025244471123] {
   marker-fill: #B10026;
}
#result2 [ value <= 31.6189224706048] {
   marker-fill: #E31A1C;
}
#result2 [ value <= 28.5837251630989] {
   marker-fill: #FC4E2A;
}
#result2 [ value <= 26.3768800405451] {
   marker-fill: #FD8D3C;
}
#result2 [ value <= 24.6271245698714] {
   marker-fill: #FEB24C;
}
#result2 [ value <= 23.1575184005245] {
   marker-fill: #FED976;
}
#result2 [ value <= 21.8729825923365] {
   marker-fill: #FFFFB2;
}

You should now have an awesome animated version of the Hurricane Arthur map like this one:

We have only scratched the surface of what is possible with the NEXRad data in CartoDB. Have fun playing with this data which is being updated all the time.

Happy weather mapping!