Python | Tutorial: Temperature Advection

Introduction:

Temperature advection is a useful field for meteorologists and weather forecasters since it is an easy way to diagnose the dynamic state of the atmosphere.  Plotted on a map, it is particularly useful for identifying frontal boundaries.  In this blog post, you will learn how to:

• Use netCDF4 to load GFS data into arrays
• Subset data by geographic boundaries
• Use Cartopy transform_points to convert lat/lon to dx and dy
• Use Numpy to compute temperature advection
• Use Cartopy to plot temperature advection
• Use Scipy to smooth data for cleaner figures

Temperature Advection:
I'm not going to demonstrate how to derive the equation for temperature advection, but essentially, it's the -1 multiplied by the dot product between the horizontal wind vector and the temperature gradient:

 Temperature advection equation

To plot this on a map from gridded data, the first, and most important step is to "discretize" the above formula:

Discretized advection

Now, all we've done is changed the partial derivative to a finite change in temperature over a change in x.  So, basically, to plot temperature advection on a map, we just need the u and v components of the wind, and temperature at a specified model level, and Δx/Δy.  The only tricky part is getting Δx/Δy from common lat/lon coordinates associated with most gridded meteorological datasets.

 
Load the Data:
In this example, I'll load data from the GFS model available from the NOMADS data server directly into Python using the netCDF4 package.  The first step in building the script is to import all of the packages needed to plot temperature advection:

import numpy as np
from netCDF4 import Dataset as ds
from datetime import datetime as DT
from cartopy import crs as ccrs
from cartopy import feature as cfeature
from matplotlib import pyplot as plt 

from scipy.ndimage import gaussian_filter

Now that the packages are loaded into Python, we'll use the "datetime" module to make this script dynamic, updating to the current date no matter when it's run.  To do this, we simply copy the url for the GFS data into a string, and replace the date and time attributes with strings formatted to match the URL format.  Note, that there are numerous ways to format strings, I am simply most comfortable with the syntax used in the example below.

hour='00'
now=DT.now().strftime('%Y%m%d')
gfs_url='https://nomads.ncep.noaa.gov:9090/dods/gfs_0p25/gfs%s/gfs_0p25_%sz_anl'%(now,hour)

Basically, we chose the '00' hour model analysis, and put todays date into the yyyymmdd format.  These strings are then passed to the url such that it matches the current date.  Now that the url is set, it's time to load the data using the netCDF4 package.

data=ds(gfs_anal,'r')

For brevity, since this isn't a tutorial on using the netCDF package, I'll just give you the code to load the lat/lon/level variables into arrays.  Basically, we do that by accessing the "variables" dictionary of the dataset.

lon=dataset.variables['lon'][:]
lat=dataset.variables['lat'][:]
levs=dataset.variables['lev'][:]

Now, longitude, latitude and pressure levels are in arrays.  Since this is a large dataset, we'll want to subset the data to match geographic boundaries so we don't calculate temperature advection over the entire globe since we only care about a small region.  Since, in this instance, longitude and latitude are independent 1D vectors this is a simple task, simply choose lat/lon boundaries and use the Numpy "argmin" function which returns the index where a given operation is minimized:

boundbox=[-120+360,-65+360,20,55]
level=850 ##850 mb pressure level
x1,x2=np.argmin(np.abs(lon-boundbox[0])),np.argmin(np.abs(lon-boundbox[1]))
y1,y2=np.argmin(np.abs(lat-boundbox[2])),np.argmin(np.abs(lat-boundbox[3]))
z=np.argmin(np.abs(levs-level))

Basically, the array indices are identifed by taking the value of the absolute difference bewteen the array and the value and seeing where it's closest to zero. Now that the indices are found, load the wind and temperature data:

T=dataset.variables['tmpprs'][0,z,y1:y2+1,x1:x2+1]
U=dataset.variables['ugrdprs'][0,z,y1:y2+1,x1:x2+1]
V=dataset.variables['vgrdprs'][0,z,y1:y2+1,x1:x2+1]

Note that the +1 for the ending index accounts for the indexing structure of Python.

Coordinate Transforms and Computing Advection:
Transforming the lat/lon to physical x/y space to compute dx and dy is the critical challenge in this exercise.  Instead of crudly approximating a constant conversion factor or running a potentially slow and complicated haversine formula, we'll take advantage of cartopys "transform_points" function to transform the cylindrical lat/lon points to a coordinate system that uses x/y distance coordinates.  If you are unfamiliar with Cartopy, I strongly suggest you read through my introduction to cartopy post: Intro to Cartopy.  UTM is a good choice for smaller domains, I like the LambertConformal projection for this domain, simply set the central longitude argument to something close to the center of the domain.

proj=ccrs.LambertConformal(central_longitude=-90)
lon,lat=np.meshgrid(lon,lat)
output=proj.transform_points(ccrs.PlateCarree(),lon,lat)
x,y=output[:,:,0],output[:,:,1]

The transform points function is called from a defined projection object (in this example, the Lambert Conformal projection), and takes an input projection (PlateCarree, or cylindrical lat/lon coordinates), and then the x (lon) and y(lat), and (if applicable) z coordinates you are transforming.  It returns an array sized (nx,ny,3) where the 3 cooresponds to the x,y,z coordinates.  Note that prior to input into the function, the "meshgrid" function is used to reset the 1D lat/lon arrays into 2D arrays that have the same shape.  This is required for transform_points to work.

Now that the x,y coordinates are defined, the numpy "gradient" function is used to get the dx and dy for temperature advection (see Intro to Numpy script for more information on "gradient").

gradx=np.gradient(x,axis=1)
grady=np.gradient(y,axis=0)

Now, all the required pieces are required, and all that's left to do is calculate temperature advection:

Tadv=-(U*(np.gradient(T,axis=1)/gradx)+V*(np.gradient(T,axis=0)/grady))

Coordinate Transforms and Computing Advection:
The below code uses Cartopy to plot temperature advection:

skip=7

fig=plt.figure(figsize=(13,12))
ax=plt.subplot(111,projection=ccrs.PlateCarree())
Z=plt.contourf(x,y,Tadv*3600.,cmap='coolwarm',levels=np.linspace(-2.5,2.5,31),transform=proj,extend='both')
C=plt.contour(x,y,T-273.15,cmap='jet',levels=np.linspace(-25,20,10),transform=proj)
plt.barbs(x[::skip,::skip],y[::skip,::skip],U[::skip,::skip],V[::skip,::skip],length=4.5,transform=proj)
plt.colorbar(Z)
ax.set_extent(box,crs=ccrs.PlateCarree())
ax.add_feature(cfeature.COASTLINE.with_scale('50m'))
ax.add_feature(cfeature.STATES.with_scale('50m'))

This produces an image similar to the following:

Temperature Advection ( 00 UTC Nov 24 2019)

 

Note that advection is multiplied by 3600 to convert the units from K/s to a more managable K/hr.  Furthermore, note that the x/y arrays are input into the contourf function with the transform keyword set to the projection used when converting from lat/lon to xy.  It's messy, but you should be able to get an idea as to where the cold and warm air advection is on the map.  One way to clean up this figures is to use the "gaussian_filter" function from the SciPy data-anaylsis package to smooth out the data.  It's pretty straightforward to use, you simply supply the function with the array you want to smooth, and a "sigma" value to determine the degree of smoothing.

Tadv_smooth=gaussian_filter(Tadv, sigma=3.0)
T_smooth=gaussian_filter(T, sigma=3.0)

mslp=dataset.variables['prmslmsl'][0,y1:y2+1,x1:x2+1]
mslp_smooth=gaussian_filter(mslp, sigma=3.0)

In this code the temperature data, and temperature advection data is smoothed, and then the sea-level pressure  variable is loaded and smoothed as well to help visually connect the storm centers to the advection: Tweaking the plotting code to account for these changes:

fig=plt.figure(figsize=(13,12))
ax=plt.subplot(111,projection=ccrs.PlateCarree())
Z=plt.contourf(x,y,Tadv_smooth*3600.,cmap='coolwarm',levels=np.linspace(-2.5,2.5,31),transform=proj,extend='both')
C=plt.contour(x,y,T_smooth-273.15,cmap='jet',levels=np.linspace(-25,20,10),transform=proj)
C=plt.contour(x,y,mslp_smooth/100.,colors='k',levels=np.arange(900,1040,2),transform=proj)
plt.clabel(C,fmt='%i',fontsize=12)
plt.barbs(x[::skip,::skip],y[::skip,::skip],U[::skip,::skip],V[::skip,::skip],length=4.5,transform=proj)
plt.colorbar(Z)
ax.set_extent(box,crs=ccrs.PlateCarree())
ax.add_feature(cfeature.COASTLINE.with_scale('50m'))
ax.add_feature(cfeature.STATES.with_scale('50m'))
plt.show()

And we get the following:

Smoothed Temperature Advection ( 00 UTC Nov 24 2019)

In the above example, a nice cold front is seen extending along the southern Appalachian mountains from a low pressure center over Ohio, with a weak warm front over the Mid-Atlantic states.

That is pretty much it.  There are a few caveats with this calculation that I won't go into great detail over, since they are negligble for the purposes of the synoptic diagnostic tool.  I hope this tutorial was informative, and thanks for reading.

Read More

Quick Note about the future of GrADSaholic

 Quick Note:

Since I started this blog in 2013 (mostly out of the monotony of post grad-school unemployment), I've been consistently surprised at just how much engagement it's gotten from the community.  Even now, after several years of relative inactivity, this blog still has a consistent following and seems to serve a useful educational purpose.  While GrADS is still very popular in the atmospheric science and climate community (I know because I can spot a GrADS figure a mile away in a paper/presentation), I haven't used it much recently myself.  Which brings me to what I see for the future of GrADSaholic.  

In short, I don't foresee myself coming up with new GrADS specific educational content, but would like to use this platform to continue to provide fresh and relevant educational computer programming and data science resources to aspiring atmospheric and Earth scientists.

Like many in my field, I made the jump to the Python bandwagon due to its extremely user-friendly feel, broad community support, and seemingly infinite number of data-science tools and applications.  In the past, I've been extremely wary of shifting the focus of this blog from a GrADS focus to a Python focus, if for no other reason than the fact that there is already an enormous wealth of information about Python on the internet.  However, a growing number of recent experiences have led me to believe that shifting in this direction may be worthwhile.

So, going forward, this blog will no longer be restricted to GrADS tutorials and will rather evolve towards a more broad focus on "how to be a better data-scientist and code developer" with a heavy focus on atmospheric science and (for now) Python.  My goal is NOT to regurgitate or provide completely redundant information, but given the shear amount of Python information out there, some overlap is unavoidable. All of the GrADS scripts and tutorials will remain up on the site, and if I do find anything new or interesting to write about with respect to GrADS, I'll be sure to do so.

Thanks again for all of the community engagement and feedback over the years, and I hope I am able to continue being a valuable educational resource for the community.

Read More

Python | Tutorial: Blue and Black Marble Aurora Figure using Background images and Day/Night Delimiter

Introduction

One of the strengths of Python, and specifically Cartopy, is that it's pretty easy to combine many different datasets with varying geographic coordinates onto a single map.  In this tutorial, I'll combine background "Blue and Black" marble images from NASA with the forecast aurora data from the Space Weather Prediction Center (SWPC: https://www.swpc.noaa.gov/), and simulated cloud cover from the GFS to produce an aurora forecast map.  Now, since one of my main goals for this blog is not to simply regurgitate what already exists, if you are just looking to plot the aurora forecast, there is an excellent example (which I'll be building off of in this tutorial) of how to so in the Cartopy gallery: here.
This is a somewhat advanced tutorial, so I won't be focusing on trying to explain a lot of the plotting minutia or reasoning.  If you're just starting out it is recommended that you check out some of my beginner tutorials first: Introduction to Cartopy, and my post on how to read in NetCDF data.
In this tutorial you will learn how to:

• Load background images into Cartopy
• Use the Cartopy "Nightshade" feature to blend the NASA blue and black marble images
• Maskout shapefile geometries
• Load the aurora forecast from the SWPC using Numpy "loadtxt"
• Create a custom colormap using matplotlib "LinearSegmentedColorMap"
• Use the NearsidePerspective projection from Cartopy

The end result of the tutorial will be an image that looks similar to this, but will be different depending on the time of day, auroral activity, and cloud cover:

Final Product: Forecast Aurora Probability and simulated GFS clouds valid UTC time.

Loading the background images

The first step is to download the "Blue Marble" and "Black Marble" background images to load into the dataset.  These are pretty easy to grab, I will simply provide links to the geotiff 0.1 degree versions for each image.  Load these into a folder where you can access them using a Python script.

• Blue Marble: Here
• Black Marble: Here

If these don't work you can try the parent sites: Blue Marble here, and Black Marble here.

Now that you have the data, it's time to load it into Python.  Before that however, the required packages for the entire tutorial need to be imported:

try:
    from urllib2 import urlopen
except ImportError:
    from urllib.request import urlopen

from io import StringIO

import numpy as np
from datetime import datetime, timedelta
import cartopy.crs as ccrs
from cartopy import feature as cfeature
import matplotlib.pyplot as plt
from matplotlib.colors import LinearSegmentedColormap 

from cartopy.feature.nightshade import Nightshade
from matplotlib.path import Path
from cartopy.mpl.patch import geos_to_path
import netCDF4 as NC

Note the"try" statement is used to make the script compatible with both Python 2.7 and Python 3 urllib syntax.  Now, once the data is imported, I'm going to define a common function to load the "Marble" images, since they have identical lat/lon boundaries.

def load_basemap(map_path):
    img=plt.imread(map_path)
    img_proj = ccrs.PlateCarree()
    img_extent = (-180, 180, -90, 90)
    return img, img_proj,img_extent

You should be able to see that the above function is really straightforward.  The image is first loaded into an array using "imread" from matplotlib, and then the image extent is set to span the entire glob (which is obvious from looking at the image), and then the image projection information for Cartopy is set to PlateCarree, since we are working in lat/lon coordinate space.  To investigate a little further, we can call this function and look at the shape of the image array.
Assuming the image is in the same folder as your script...

map_path='RenderData.tif'
bm_img,bm_proj,bm_extent=load_basemap(map_path)
print(np.shape(bm_img))

returns: (1800, 3600, 3)

Notice that the array returned has 3 dimensions despite the fact that it is a 2D image: the 3rd column is where the individual red/green/blue information for the image is stored, where as the first 2 columns correspond to the y and x pixels.  To demonstrate this, I'll plot each channel separately without geographic information.

plt.figure(figsize=(8,14))
ax=plt.subplot(311)
plt.pcolormesh(bm_img[:,:,0],cmap='Reds')
ax=plt.subplot(312)
plt.pcolormesh(bm_img[:,:,1],cmap='Greens')
ax=plt.subplot(313)
plt.pcolormesh(bm_img[:,:,2],cmap='Blues')
plt.show()

This produces the following image:

RGB channels for the Blue Marble image (Yikes! It's upside down!)

Note that where all three channels are saturated, the combined RGB image will be white (e.g., Antarctica), where as where all three channels are faded, the RGB image will appear back (e.g., the Oceans).  Also notice how the image is upside down, this will be addressed in the next part.

The question, is how can we plot the combined RGB image from the 3 dimensional array.  Instead of using the "pcolormesh" or the "contourf" functions, the "imshow" function is used.  this function, is versitile and naturally takes a 3D array assuming the 3rd axis (if there is one) is the RGB channel.  It also, takes the extent argument, which defines the image corners, without needed corresponding coordinate arrays.

    plt.figure(figsize=(11,8))
    ax=plt.subplot(111,projection=ccrs.PlateCarree())
    ax.imshow(bm_img,extent=bm_extent,transform=bm_proj,origin='upper')
    plt.show()

This results in the following image:

Blue Marble

I won't go into detail on how to load the black marble, however, I will provide the code used to load both images as they are read into the final product.

bm_img,bm_proj,bm_extent=load_basemap('RenderData.tif')
nt_img,nt_proj,nt_extent=load_basemap('BlackMarble_2016_01deg_geo.tif')

Blend Images using the Day/Night Delimiter

This next section is the crux of the whole tutorial: Basically, I'll explain how to use Cartopy Nightshade (Note: requires version 0.17 or higher) to mask out the day time blue marble and show the night time black marble image.  For this, I'll use the matplotlib "Path" function to draw a polygon "path" connecting the coordinate information associated with the Nightshade function.

The Nightshade function is a neat function that allows you to draw the day/night delimiter on a glob for a given datetime object.  In the full example, the datetime object will be defined by the aurora forecast, but for demonstration of Nightshade, the datetime.now() object will be used.

For a simple demonstration, simply copying the code to generate the blue marble image, with only a couple of added lines, provided you are following the tutorial with the package imports written as above:

plt.figure(figsize=(11,8))
ax=plt.subplot(111,projection=ccrs.PlateCarree())
ax.imshow(bm_img,extent=bm_extent,transform=bm_proj,origin='upper')
ax.add_feature(Nightshade(datetime.utcnow()),zorder=2,alpha=0.8,color='r')
plt.title(datetime.utcnow().strftime('%c'))
plt.show()

This will essentially mask out the "night sky" over the blue marble image, it's that simple!

Night masked out.

Now the goal here is simple: replace the "red" masked out section with the black marble image.  To do that, I'll first define a function to clip regions within the shapefile polygon.

The first step is to define the Nightshade "shape" from the Nightshade function:

nshade=Nightshade(datetime.utcnow())
transform=nshade.crs

Now we have both the nightshade shape, and the nightshade coordinate reference system, which is needed to ensure the projection transform is performed correctly.  From the shape, we grab the geometries (i.e., the polygons, or in this case ... polygon; singular).

geoms=list(nshade.geometries())

Here is the tricky part: because the Nightshade coordinate reference system is NOT Plate Carree (it's a rotated pole projection for those interested), it's important that the transform is done correctly.  If you assume the transform is Plate Carree you will end up with a mask that is wrong.  However, the matplotlib path function requires that the transform is a matplotlib transform, and not a cartopy crs object.  So, to do this conversion, I will define a "dummy" transform when defining my Caropy subplot that equals the crs from the nightshade object.

fig = plt.figure(figsize=(10, 8))
ax = fig.add_subplot(1, 1, 1, projection=ccrs.PlateCarree(),transform=nshade.crs)

Now, we have a defined figure, and a matplotlib transform that matchs the nightshade object, we're ready to mask.  First we plot the blue and black marble images on the same subplot, and then use the geos_to_path and Path functions imported at the top of the script:

plt.figure(figsize=(11,8))
ax=plt.subplot(111,projection=ccrs.PlateCarree(),transform=nshade.crs)
im0=ax.imshow(bm_img,extent=bm_extent,transform=bm_proj,origin='upper',zorder=1) 
im1=ax.imshow(nt_img,extent=nt_extent,transform=nt_proj,origin='upper',zorder=2)
path = Path.make_compound_path(*geos_to_path(geoms))
im1.set_clip_path(path, transform=ax.get_transform())
plt.title(datetime.utcnow().strftime('%c'))
plt.show()

And the blended image is complete,note that the subplot transform is used in the set_clip_to_path function:

Blended image based on current datetime.

Now, it's time to plot the aurora forecast.

Loading and Plotting the Aurora Forecast

This section will mostly be a reproduction of the code demonstrated here, with some minor differences.  Esentially, Numpy loadtxt is used to load the ascii data from the SWPC into a 2 dimensional array and the array is plotted on the map.  The first step is to define a cool looking aurora color bar.  This a a direct reproduction from the Cartopy example because I think they did a nice job with the color scales.

def aurora_cmap():
    """Return a colormap with aurora like colors"""
    stops = {'red': [(0.00, 0.1725, 0.1725),
                     (0.50, 0.1725, 0.1725),
                     (1.00, 0.8353, 0.8353)],

             'green': [(0.00, 0.9294, 0.9294),
                       (0.50, 0.9294, 0.9294),
                       (1.00, 0.8235, 0.8235)],

             'blue': [(0.00, 0.3843, 0.3843),
                      (0.50, 0.3843, 0.3843),
                      (1.00, 0.6549, 0.6549)],

             'alpha': [(0.00, 0.0, 0.0),
                       (0.70, 1.0, 1.0),
                       (1.00, 1.0, 1.0)]}

    return LinearSegmentedColormap('aurora', stops)

Basically, a dictionary with 0-1 values for red (r), green (g), blue (b), and transparancy (alpha) is passed to the LinearSegmentedColormap function which defines a color map that basically linearly interpolates between each of the 9 "stops" defined by the user.

Now, that the colormap is defined, you can pull the aurora data from the SWPC and load it into an array:

    # To plot the current forecast instead, uncomment the following line
    url = 'http://services.swpc.noaa.gov/text/aurora-nowcast-map.txt'

    response_text = StringIO(urlopen(url).read().decode('utf-8'))
    aurora_prob = np.loadtxt(response_text)
    # Read forecast date and time
    response_text.seek(0)
    for line in response_text:
        if line.startswith('Product Valid At:', 2):
            dt = datetime.strptime(line[-17:-1], '%Y-%m-%d %H:%M')

Now, again this code is largely copied from the Cartopy example, but essentially you're reading the text information from the URL, first into the Numpy loadtxt function, which is a natural fit for this data, since the function, by default, assumes "#" is a comment, and "spaces" are delimiters, so in this instance, no additional arguments are required for loadtxt.  Then, the datetime associated with forecast is read into a datetime object.

Since it is known that the lat/lon data spans -90 to 90 and -180 to 180, we define lat/lon arrays based on the shape of the aurora data:

lons=np.linspace(-180,180,np.shape(aurora_prob)[1])
lats=np.linspace(-90,90,np.shape(aurora_prob)[0])

Finally, combining everything together to product the "cloud-free" aurora forecast:

plt.figure(figsize=(11,8))
ax=plt.subplot(111,projection=ccrs.PlateCarree(),transform=nshade.crs)
im0=ax.imshow(bm_img,extent=bm_extent,transform=bm_proj,origin='upper',zorder=1)
im1=ax.imshow(nt_img,extent=nt_extent,transform=nt_proj,origin='upper',zorder=2)
path = Path.make_compound_path(*geos_to_path(geoms))
im1.set_clip_path(path,transform=ax.get_transform())   
Z=ax.contourf(lons,lats,np.ma.masked_less(aurora_prob,1), levels=np.linspace(0,100,61), transform=ccrs.PlateCarree(),     zorder=5,cmap=aurora_cmap(),antialiased=True)
plt.title(datetime.utcnow().strftime('%c'))
plt.show()

Which gives you the following image:

Aurora Forecast with Plate Carree Projection

Finally, replacing the PlateCarree map projection with the NearsidePerspective projection, we get something a little bit closer to the final image:

fig = plt.figure(figsize=[10, 8])
ax = plt.axes(projection=ccrs.NearsidePerspective(-170, 45),transform=nshade.crs)
im0=ax.imshow(bm_img,extent=bm_extent,transform=bm_proj,origin='upper',zorder=1)
im1=ax.imshow(nt_img,extent=nt_extent,transform=nt_proj,origin='upper',zorder=2)
path = Path.make_compound_path(*geos_to_path(geoms))
im1.set_clip_path(path, transform=ax.get_transform())
Z=ax.contourf(lons,lats,np.ma.masked_less(aurora_prob,1), levels=np.linspace(0,100,61), transform=ccrs.PlateCarree(),

              zorder=5,cmap=aurora_cmap(),antialiased=True)
plt.title("Forecast Aurora Probabilty \n Valid: %s"%dt.strftime('%b/%d/%Y %H:%M UTC'),

          loc='left',fontweight='bold')

Orthographic / Nearside Perspective Projection

Adding A Cloud Forecast from the GFS

The final step in this tutorial is to add cloud cover from the GFS to the map.  This is unnecessary, if you're happy with the above image, but my personal thought is, you aren't going to see the aurora if it's cloudy, so having the forecast cloud cover is a nice overlay.  It's real easy to add the clouds, simply pull the GFS data from the NOMADS server using netCDF4, match the GFS model time to the aurora forecast time and overlay the total cloud cover variable on the map.  Again, my assumption is that if you've made it this far, you're comfortable working with GFS data using netCDF4 and I won't explain in detail why the following code works:

dhour='12' ## or 00, or 18 or 06, your choice
dtstr=dt.strftime('%Y%m%d')
gfs_path='http://nomads.ncep.noaa.gov:80/dods/gfs_0p25_1hr/gfs%s/gfs_0p25_1hr_%sz'%(dtstr,dhour)

gfs_data=NC.Dataset(gfs_path,'r')

gfslons=gfs_data.variables['lon']
gfslats=gfs_data.variables['lat']

tres=gfs_data.variables['time'].resolution
tmin=datetime.strptime(gfs_data.variables['time'].minimum,'%Hz%d%b%Y')
times=[timedelta(hours=float(i)*24.*tres)+tmin for i in range(len(gfs_data.variables['time'][:]))]

tidx=np.argmin(np.abs(np.array(times)-dt))

clouds=gfs_data.variables['tcdcclm'][tidx,:]

and then simply add the following code to the beneath the code used to load the background images and plot the aurora forecast:

ax.pcolormesh(gfslons[:],gfslats[:],np.ma.masked_less(clouds,75.),transform=ccrs.PlateCarree(),
    alpha=0.7,cmap='Greys_r',zorder=4,vmin=50,vmax=100)

And voila:

Final Product

Final Notes

That is all for this tutorial, it was a long one, but I think there is a lot of useful stuff in here.  A few final notes, reprojecting the background images can lead to really poor quality images, I find this particularly true for Lambert and PolarStereo projections, so be aware of that. 

Read More

Python | Tutorial: Intro to Cartopy

Introduction:

There are currently two main Python libraries for plotting geographic data on map: Cartopy and Basemap.  New users should use the Cartopy since support Cartopy will replace Basemap and support for Basemap is expected to wrap up in 2020.  Therefore, this tutorial will focus on Cartopy.  If you want to use Basemap (e.g., for plotting in "3D"), here are a couple of links to existing tutorials and examples to help get you started.  Furthermore, many concepts described here may be useful for understanding Basemap.

• Basemap Examples
• Basemap Tutorials 

Cartopy:

Cartopy is a geospatial plotting library built on top of Numpy and Matplotlib that makes plotting gridded data, shapefiles, and other geographic data on over 30 different map projections.  Furthermore, the Cartopy "transform" functionality makes it straightforward to convert data from one projection to another.  In this tutorial you will learn how to use Cartopy to:

• Plot GFS surface temperature data on a map
• Mask out land surfaces to plot a map of sea surface temperature and ice-cover from the GFS
• Plot a regional map of surface temperature with US states
• Use the transform function to plot GFS data on a different map projection

This tutorial accesses the NOMADS data server using the netCDF4 library, if you are unfamiliar with doing this, I recommend you see my tutorial on reading NetCDF data.

Getting Started - Grabbing GFS Data:
The following code is used to get started with this tutorial, essentially, it imports all of the required modules, pulls the GFS data from the NOMADS data server, and loads it into arrays.  Once the data is loaded, the remainder of the tutorial will focus on Cartopy. 

import numpy as np
from matplotlib import pyplot as plt
from cartopy import crs as ccrs

import cartopy.feature as cfeature

import netCDF4 as nc
import datetime as DT 

datetime=(DT.datetime.utcnow()-DT.timedelta(hours=6)).strftime('%Y%m%d')
fhour=20

nomads_path="https://nomads.ncep.noaa.gov:9090/dods/gfs_0p25/gfs{}/gfs_0p25_00z".format(datetime)

datafile=nc.Dataset(nomads_path)
lat=datafile.variables['lat'][:]
lon=datafile.variables['lon'][:] 

sfcT=datafile.variables['tmpsfc'] #Keep meta data for now!
sfcT_data=sfcT[fhour,:]

title=sfcT.long_name
time=datafile.variables['time']
tstart=DT.datetime.strptime(time.minimum,'%Hz%d%b%Y')
timestr=(tstart+DT.timedelta(hours=time.resolution*24.*fhour)).strftime('%c')

Now that your data is loaded, we initialize a figure, and subplot using matplotlibs "projection" keyword to link to the Cartopy library.  Once you've attached the Cartopy package to your subplot, there are a number of additional objects and classes you can add to your figure.  In the first example, I'll demonstrate by calling upon the "coastlines" attribute to plot continents on the map.

fig=plt.figure(figsize=(11,5))
ax=plt.subplot(111,projection=ccrs.PlateCarree())

Now, your figure and subplot are defined.  The Cartopy "PlateCarree()" projection is a basic cylindrical map projection that accepts coordinate information in the form of lat/lon coordinate pairs.  The lat/lon data can be either 1D or 2D, but essentially, if your coordinate information is lat/lon, your projection is PlateCarree().   You can pass 2 keywords into PlateCarree (central_longitude and globe), but in most instances, you won't need to.   Occasionally, I reset the central_longitude to 180, instead of it's default zero.

The code to plot the map is:

Z=plt.pcolormesh(lon,lat,sfcT_data,cmap='jet')

pos=ax.get_position()

plt.title(title,fontweight='bold',loc='left')
plt.title("Valid:{} UTC".format(timestr),loc='right')
ax.coastlines()

cbar_ax=fig.add_axes([pos.x1+0.01,pos.y0,0.015,pos.height])
cbar=plt.colorbar(Z,cax=cbar_ax)
cbar.set_label(title)

datafile.close()

plt.show()

And you'll get an image similar to this one:

GFS surface temperature from Cartopy

Now, lets say you wanted to mask out land areas, and add sea-ice to the map, to focus on the oceans.
Adding the sea-ice is easy, all it requires is for you to pull the icecsfc variable from the GFS data, and plot it overtop of the surface temperature map:

plt.pcolormesh(lon,lat,np.ma.masked_less(ice,0.1),vmin=0,vmax=1,cmap='Greys_r',zorder=3)

We use the numpy mask (ma) functionality to maskout gridcells where the ice cover is less than 10%.
To mask out land areas, we make use of the "Cartopy Feature" module, which handles shapefiles.  The Cartopy Feature module connects seemlessly to the Natural Earth Dataset, and allows for anyone to plot a host of different GIS datasets without needing to download individual shapefiles before hand.  The interface even has many pre-defined functions, including land.  This makes masking out the land areas, a breeze.

ax.add_feature(cfeature.LAND,zorder=4,color='gray')

In the above code, the "add_feature" function is used, and the pre-defined "LAND" feature from the Cartopy Feature package is supplied with a couple of basic keyword arguments.  Putting it all together:

fig=plt.figure(figsize=(11,5))
ax=plt.subplot(111,projection=ccrs.PlateCarree())
Z=plt.pcolormesh(lon,lat,sfcT_data-273.15,cmap='jet',vmin=-1,vmax=30.)
plt.pcolormesh(lon,lat,np.ma.masked_less(ice,0.1),vmin=0,vmax=1,cmap='Greys_r',zorder=3)
ax.add_feature(cfeature.LAND,zorder=4,color='gray')
pos=ax.get_position()

plt.title(title,fontweight='bold',loc='left')
plt.title("Valid:{} UTC".format(timestr),loc='right')
ax.coastlines(color='k',zorder=5)

cbar_ax=fig.add_axes([pos.x1+0.01,pos.y0,0.015,pos.height])
cbar=plt.colorbar(Z,cax=cbar_ax)
cbar.set_label(title)

datafile.close()

plt.show()

Running the above code will give you an image like this:

GFS SST and Sea-ice cover

Now, lets zoom in over the United States, and add states to the map.  To clip the extent, the "set_extent" function is used:

ax.set_extent([-125, -65, 25, 55],crs=ccrs.PlateCarree())

The list corresponds to lon/lat boundaries from [western-most longitude, eastern-most longitude, southern-most latitude, northern-most latitude] and the crs argument indicates which transform is being applied.  To add the states, you need to define the states from Natural Earth database. the "scale" argument

states = cfeature.NaturalEarthFeature(category='cultural', scale='50m', facecolor='none',
                         name='admin_1_states_provinces_shp',edgecolor='k')

 By changing the extent, and adding the states, you'll get a map that looks like this:

GFS SST centered over the United States

You'll notice that the "LAND" feature masks out the Great Lakes, which is unfortunate, and to my knowledge the only way to get both the land mask and the SST over lakes using GFS surface temperature data, is to use a Lakes shapefile with shapefile clipping, a technique that is beyond this simple introduction, but may be the subject of a future tutorial.

Finally, if you want to use a different map projection to plot the data, e.g., a North Polar Sterographic projection, you simply define a different projection when setting up your subplot, and use the "transform" keyword in the plotting function to specify that you are transforming your lon/lat coordinates from the PlateCarree() projection:

ax=plt.subplot(111,projection=ccrs.NorthPolarStereo())
Z=plt.pcolormesh(lon,lat,sfcT_data-273.15,cmap='jet',vmin=-1,vmax=30.,transform=ccrs.PlateCarree())
plt.pcolormesh(lon,lat,np.ma.masked_less(ice,0.1),vmin=0,vmax=1,cmap='Greys_r',zorder=3,transform=ccrs.PlateCarree())

ax.set_extent([-180, 180, 45, 90],crs=ccrs.PlateCarree()) 

Note, that we also reset the extent to focus on the Northern Latitudes.  You should get something that looks like this:

GFS Sea-ice and SST Polar Plot

That is the extent of this introductory tutorial on how to use Cartopy to work with atmospheric data.  You can learn more from the Cartopy Website, and get a list of different map projections here Cartopy Map Projections. Future tutorials on Cartopy will focus on working with shapefiles, transforming individual points, how to work with data without lat/lon coordinates, and how to put fine details on your map.

Read More