Building a startup is both difficult and rewarding work. It requires a team of dedicated and passionate people, with the tools and skills to face challenges head on, and re-imagine the possibilities of successful innovation. Being a product of these experiences, our team at CartoDB was proud to partner with The Startup Institute New York to elevate and educate new communities of innovators!

Earlier this month, we had the pleasure of facilitating a Strategy Hack workshop with the Startup Institute in New York City. The Startup Institute is the #1 career accelerator geared towards providing individuals with the skills, mindset, and network to succeed in jobs at startups. We were honored to once again bring powerful mapping tools to the table, and give back to our fellow innovation-pioneers by creating life-long learning opportunities challenging students to rethink mapping as applied to professional practice.

As you may have noticed, CartoDB’s commitment to educators and students has given us the opportunity to critically engage with these communities in making CartoDB as accessible as possible for many different user groups. Our two day learning experience at the Startup Institute allowed us to further explore how CartoDB is used in the classroom, while also pushing the envelope on how cloud-based mapping is applied by diverse groups of professionals. Some great maps resulted from the workshop!

Take a look at Stefanie Gray’s map of active spaces in New York City:

and this revealing visual by Albert Troszczynski about how socioeconomic demographic and education are linked to party votes:

Yikes!

While the collective experience at the Startup Institute was exciting and fruitful, our commitment to creating opportunities and elevating startups doesn’t end there. our recently announced CartoDB Start-Up Grants is an exciting opportunity for emerging companies and working individuals who build digital tools for social good to bring their work to new levels using the CartoDB platform. We are excited to collaborate with innovators seeking to develop mobile or web-applications, data visualizations, and data collection methods that engage with community positive and socially important topics. This initiative echos our climate grant program, and further strengthens our commitment to socially and environmentally progressive work.

Our goal is to provide free resources to startups around the world, so if you are working at an early stage startup that could benefit from beautiful maps or an amazing geospatial backend, don’t hesitate get in touch!

Happy Mapping!

Recently, CartoDB had the the chance to put our new space to work by hosting MaptimeNYC in our New York office. This exciting and informative session was a great opportunity to bring together mappers from across the city in a community-run and critically engaged environment.

Every two weeks in New York (and in other cities across the nation and world) a group of twenty or so mapping aficionados meet to share ideas and build community around mapping, and generate learning experiences around how to create functional, well-designed maps. Maptime is a great place for newbies and experienced mappers alike to come together and build collaboratively. People from all professions and interests are encouraged to bring mapping problems and barriers that are holding them back, and through our growing and active mapping community, participants can break through some of the barriers in constructive ways while making friends. Whether you are new to mapping, getting started with CartoDB, or a long-time GIS professional, Maptime creates a safe space to experiment, make mistakes, and lean on experienced peers. Plus… pizza. (You can’t beat mapping and pizza!)

Each session of Maptime is designed to showcase a skill or technique presented by a fellow mapper, then transition into problem-solving working sessions that tackle everything from CartoDB’s built in APIs, to Javascript, and many other facets of developing and mapping technologies. Our community of mappers here in New York and across the world are creating a strong culture of data visualization that CartoDB and our family of web-based mappers are proud to be embedded in.

Early December’s session featured the second part of a series on SQL using PostGIS, which many have been aching to improve their skills given its robust capabilities in CartoDB.

So we offer you part 2 of our SQL demo:

Built and fine-tuned by the group of fifteen that gathered at CartoDB HQ in NYC last week these handy SQL queries range from common to exotic, and inform a wide range of experienced users. We hope you find them useful!

Connecting a dots with lines: ST_Makeline

This is one for the n00bies. So you have a bunch of points, whether from a GPS or points you’ve created, and you want to connect them to make a path. Seems simple, but might not be as straightforward as you think.

Make sure you have: * Points on route - Here we’ve just made a few points on a blank table to make it simple.

• Order points (can be cartodb_id) to make the line connecting groupings of dots. These points can be route direction, segments, etc.
• If you have more than one type of line - say a route that goes through two directions or has different qualities - Connect lines with the statement: ST_Makeline

SELECTST_MakeLine(the_geomORDERBY_orderASC)ASthe_geom,routeFROMmaptimesql_pointsGROUPBYroute

From here you can create a new table when prompted on the CartoDB Interface.

Lines appear in new table:

Alternately, you can add in webmercator to create the line AND the cartodb_id to make it interactive. Try removing one or both of those to help you understand the function of them:

SELECTST_MakeLine(the_geom_webmercator,cartodb_idORDERBY_orderASC)ASthe_geom_webmercator,routeFROMmaptimesql_pointsGROUPBYroute

Points in a Geographic Area: ST_Contains

Let’s say you want to find all the points inside of a geographic area. The first thing you need is a table with polygons and a table with points. Here, we will use two CartoDB common data sets, Real Time Earthquakes (points) and US Counties (polygons), found at the top of your CartoDB dashboard.

In the end, this will take the counties and highlight the ones that have had earthquakes in the time-period defined by the earthquakes dataset.

Make sure you have: * Points table in CartoDB (in this case, earthquakes) * Polygons in a table (us_counties)

Use the_geom_webmercator to get the polygons and points merged on one table without having to select ‘create new table’.

SELECTus_counties.the_geom_webmercator,us_counties.cartodb_id,count(quakes.the_geom)AStotalFROMus_countiesJOINquakesONst_contains(us_counties.the_geom,quakes.the_geom)GROUPBYus_counties.cartodb_id

10 Closest Points to a Selected Latitude and Longitude: CDB_LatLng

One thing that SQL allows you to do is create compounded functions and compile multiple steps into a more simple statement. Here, we are using CDB_LatLng (latitude, Longitude)) to find the 10 closest points from a selected latlon. The original statement is found below and is unwieldy and difficult for new or intermediate users. CartoDB has created a few new statements to make life easier.

We will now find the ten earthquakes closest to Times Square.

Make sure you:

• Have a table with a lot of points. (You can use the earthquakes data from the previous section)
• Put the SQL statement below into the UI, changing ‘quakes’ to your table name
• Apply query

It should show you the ten closest earthquakes on that day to the point you noted. Use can use any latlon. Find out yours here.

SELECT*FROMquakesORDERBYthe_geom<->CDB_LatLng(42.5,-73)LIMIT10

Compare Rows of Data, Visualize Difference: lag( )

The lag() enables you to compare to rows of data and pull out the difference of those and display that difference visually. Here, we will use some self-created data to look at the time between stops on a New York tourist’s landmark tour.

What lag() does is to takes the data in a row and compare it to a determined previous row. lag (column, number of rows back). Useful in comparing data and extracting duration. Here’s a link to what’s happening in the background.

Make sure you:

• Have a journey that has points along a route. (can be multiple routes)
• Use this table to start
• Download the data and create a new table
• Look at the map to make sure everything is in place
• Put the SQL statement below into the UI, changing ‘landmark_visits’ to your table name
• Apply query
• Make new table from query

You should see lines running between the landmarks that can now be changed using a chloropleth for lines to show the difference in times between the journey segments.

SELECTcartodb_id,the_geom,description,extract(epochFROM(time_left-lag(time_left,1)OVER(ORDERBYtime_left)))ASduration_in_secondsFROMlandmark_visitsORDERBYtime_left

That wraps it up! We hope you learned a lot. SQL and PostGIS can be powerful tools for visualizing and analyzing data. If you want to learn more, check out our SQL and Post GIS Academy course and come out to Maptime NYC. Come learn new things, help grow our mapping community! We hope you learned a lot and as always:

Happy Mapping

We’re pleased to welcome Eric Bean to the Madrid contingent of the CartoDB team! He’ll be souping up our customer relations with a slurry of financial services, data analytics, and management skills!

With a midwestern-to-Madrid background, Eric toggled between both continents of our CartoDB teams. Following a few years of analytics consulting and statistical modeling on the West Coast, he now joins Business Development as an Account Manager at CartoDB’s Madrid headquarters.

Eric’s affection for geo info grew out of his childhood in small-town Iowa, where his father would routinely quiz him on political cartography and country capitals. Earning a D3 fútbol scholarship (not this D3, but this D3) to attend college in Los Angeles furthered his interest in economics, data analysis, and international law, which Eric supplemented with a 6-month study in Madrid. In his free time, Eric digs playing sports, finding outdoor adventures, and absorbing the social idiomatics that only avid travel and cultural exposure can provide.

Eric’s affection for travel and spanish culture brought him back to Madrid in pursuit of a Master’s in International Business last year.

Want to throw down with Eric to up your geo-foo?

• Check out this jetpunk capitals quiz online

Want to browse his favorite CartoDB maps?

• Check out the BBVA Economic Impact of MWC map if business is your bag
• Check out the Twitter Sunrise Map if you’re more into “incredibly aesthetically pleasing” (his words) CartoDBrilliance

What to chat more with Eric? * Reach out at eric@cartodb.com

Welcome to the team!

Today we hear from a friend from our NYC community, Chris Henrick. Chris is a MFA Design & Technology graduate student at Parsons, The New School For Design. He is a long time user and an expert on CartoDB. We recently saw The Northwest Bushwick Community Map, a project he has been developing to document ongoing change in one of our communities. We wanted to hear more! Read Chris’s description of the project below.

The Northwest Bushwick Community Map is meant to be a resource for local community organizations and tenants rights groups to easily access disparate information around land use, housing and urban development for the neighborhood of Bushwick in Brooklyn, NY. It is also intended to be a tool to inform NYC residents about what kinds of indicators may be used to predict new urban development and help prevent displacement of residents in their own neighborhoods. Recently I had the opportunity to work with the original author of the map and some fellow designers in the MFA DT program at Parsons to improve the map’s design and functionality.

Background

In the fall of 2014 two members of the Northwest Bushwick Community Group (NWB), Michael ‘Ziggy’ Mintz and Brigette Blood, met with several graduate students from MFA DT to discuss improvements to the beta-version of the map. Previously, the map used vanilla Leaflet.js with GeoJSON data created from NYC’s MapPLUTO and Department of Buildings permits open data. However the original map lacked a cohesive design and was slow in regards to loading of the data due to technical issues.

Map Improvements

To improve the interactivity of the map I chose to host the data on CartoDB and use the CartoDB.js library to handle loading, styling and toggling of the map’s data layers. When the user selects layers, CartoCSS and SQL code is passed to CartoDB to retrieve and style the data being rendered on the map. Hosting the data on CartoDB also allows for processing of the data in the CartoDB dashboard using PostgreSQL and PostGIS. This is beneficial as SQL scripts can be run to automate the processing of new data when the map needs to be updated.

To improve the context of the map (the Who, What, Where, When, Why and How) the MFA DT team asked Ziggy and Brigette to provide us with some real world stories that show how the data relates to the dwindling affordable housing stock and displacement of longtime residents in Bushwick. To accomplish this we used the Odyssey.js Javascript API to create an interactive slide show with the map which transitions between three of NWB’s stories. With a bit of hacking we were was able to use the changing of the slides as a jQuery event trigger to toggle the map’s data layers. This allows for the slide show to give background and context about the project while also relating seemingly abstract data in a visceral way.

Besides Cartodb.js and Odyssey.js we added some other features to the map to make it more useful for the Bushwick community. As there is a large Latino presence in Bushwick it was imperative to have the text of the map’s UI and the overall website toggle between English and Spanish. Additionally the non-map parts of the site were redesigned responsively using CSS Media Queries so that the other content such as “Get Help” can be easily viewed on a mobile device.

Future Plans

In the future NWB plans on adding participatory map data they’ve collected on housing vacancy and new development as well as stories of people who have experienced being displaced or harassed by their landlords. These features may be integrated with the current map or developed separately, but either way CartoDB and Odyssey will allow for the further creation of customized interactive map content to be readily achievable.

If you are interested in more, take a look at some of these links

• A preview of the map redesign
• Code for the map’s redesign
• The Northwest Bushwick Community Group’s home page

Thanks Chris for the awesome post about the great project!

This week’s map features the adventures of Varun Adibhatla on his quest to visualize the epic New York Century Bike Tour. A self-professed “ridestat geek,” Varun was inspired to experience his second time on the tour in a whole new way during his first week of class at New York University’s Center for Urban Science and Progress.

With the help of stalwart companion and fellow master’s student Patrick Atwater, Varun combined his knowledge of available data from a 2013 ride with his newfound interest in the science of cities to hit upon a curious new metric. The following map documents Varun’s first display of “bumpiness” data.

The New York Century Bike Tour from MapmyRide (Note: Varun collected 45 miles of Bumpiness Data before his phone fell off his bike).

Phase one of Varun’s bumpiness exploration involved using SensorLog to collect data from his iPhone. Between rough patches of road and New York’s notorious potholes, acquiring location and accelerometer data was that much more of a challenge.

But tech prevailed, and SensorLog provided data output in a no-nonsense csv file–perfect for CartoDB import. From here, Varun began tweaking data from every half-second across the following fields:

• time
• timestamp
• recordtime
• lat
• long
• alt
• speed
• course
• verticalAccuracy
• horizontalAccuracy
• locTimeStamp
• accelerationX
• accelerationY
• accelerationZ
• HeadingX
• HeadingY
• HeadingZ
• TrueHeading
• MagneticHeading
• HeadingAccuracy
• RotationX
• RotationY
• RotationZ
• motionYaw
• motionRoll
• motionPitch
• motionRotationRateX
• motionRotationRateY
• motionRotationRateZ
• motionUserAccelerationX
• motionUserAccelerationY
• motionUserAccelerationZ
• en0
• pdp_ip0
• DeviceOrientation
• State

Next came Varun’s neat trick. To derive a clean measure of bumpiness, he isolated the values from the motionUserAccelerationX, Y, and Z fields to calculate the magnitude of acceleration across the x, y, and z directions. The result? A single number that conveyed bumpiness.

SensorLog iPhone view showing Accelerometer data

SensorLog iPhone view showing Location Data

Timestamp + coordinates + bumpiness = three for three. Varun used Excel to rank these values and processed over 92,000 ranks, wherein the smallest rank conveyed the most bumpiness. He then split this data into four categories to evaluate rider experience:

CartoDB is loving “Bumpiness in NYC”. Our Category Torque tool was tailor-made for Varun’s project. The animation requires time series data, which Varun plugged into the above categories to convey bumpiness, and voila! This visualization was born:

Varun settled on a Stamen base-map to evoke “Sin City” and make the neon colors “pop.”

Extra kudos to Varun for getting “Bumpiness in NYC” recognized at NYU’S Data Service GIS Day. And huzzah for student mappers!

Happy Mapping!

Today we are excited to bring you a guest post from our friends at KimonoLabs. If you aren’t familiar with Kimono, they bring a set of tools that enable you to create APIs from data and information scattered throughout the Web. We highly recommend checking them out. Below, they will share with you the steps for turning a website into an API and then how to use CartoDB to map that API in some really interesting ways. This was posted originally over on Kimono’s blog here, thanks for the permission to repost!

Tutorial: Mapping your own location data in 10 minutes

Data is more accessible, tangible and interesting when you can visualize and interact with the figures on a page. That’s why we love teaming up with our friends over at CartoDB! Kimono is a smart web scraper that let’s you turn data on a website into an API – a structured feed of updating data. CartoDB let’s you take that data set and create beautiful interactive maps. In this post, we will use kimono to get over a year’s worth of bike trip data from New York City’s Citi Bike bike sharing program. We’ll then use cartoDB to plot our friend Andrew’s movements on a map. A big thanks to Andrew for riding his bike a lot and sharing his data with us!

Here’s all that you’ll need to build your own data-driven map:

• A kimono account (it’s free) and the kimono chrome extension
• A cartoDB account (it’s also free)
• A website with location data – we’ll use data from NYC’s Citi Bike bike sharing program, but you can capture anything you like (e.g., Uber, Lyft, public transportation routes)
• 10 minutes

Create an API

Navigate to the website with the data you want to map. For this example we are using data from Andrew’s Citi Bike account, which looks like this:

Click on the kimono chrome extension…

and the kimono toolbar will appear on top of the webpage:

Notice the flashing lock icon on the toolbar. This indicates that the page requires you to log in. Click on the lock icon.

Kimono will then direct you to the site’s login page (if you need to navigate further to get to the login page, click the navigation icon and go to the appropriate login page).

Once at the login page, you must identify the username, password and submit fields – this teaches kimono how to login to this site. Do this by clicking the username, password and submit icons on the toolbar and then clicking on the matching field on the webpage.

Click done and then enter your login credentials. Kimono securely stores the credentials so that it can access your data automatically on the schedule you specify.

Extract Data

Once you’ve completed the login cycle, you will see the original page with your data. Here we want to grab four types of data – the start station, start times, end station and end times. To do this, click on one of the start stations. Kimono will suggest other start stations to you. Click the check mark to accept all the start stations into your first data property. Click the check mark to accept all the start stations into your first data property. The number in the yellow circle…

…will increase to reflect the number of data points in that property.

Now click the grey plus…

…to add a new property and repeat this process with start times, end stations and end times. You can preview the structured data that kimono will extract in the data preview pane and make more granular adjustments in the data model view.

Click DONE to create an API. Select daily to make sure your data is refreshed every day. Once it’s done, click the link to check out your new API.

(Pro-tip: if you are having trouble selecting just the data you want, try clicking and dragging to select just the part you want and kimono will strip off extraneous text.)

Configure Your API

You can view the first page of data extracted on the API detail page for your new API.

The API we just set up only extracts from one page of rides. If you have several pages of data, you’ll need to configure your API to extract from multiple pages as well. To do this, go to the crawl setup tab, click on ‘crawl strategy’ to specify the type of crawl you want to do – select generated URL list in this case. Then, on the lower right you will see the URL generator.

Kimono has broken your source URL into its relevant sub-components. For us, the number after ‘trips’ specifies the page you’re on. To the right of the number parameter, click ‘range’ and specify 1 to 20 (instead of 20 use whatever number corresponds to the total number of pages of data that you have).

You will see a list of URLs generated by kimono below. Click save changes, then hit ‘start crawl above’. Once the crawl completes, go back to the data preview tab and download the CSV. Open it up in excel and remove the top row – the row that says ‘collection1’ to get it formatted for use with cartoDB.

Now that we have our structured data set, let’s start mapping our route data.

Geo-Code Your Data

Log in to your cartoDB account and select ‘tables’. Click the large plus on the right to add a new table.

Choose ‘data file’ and select the kimono csv file that we just downloaded. Once the data is loaded into cartoDB, click the drop-down next to the property with your start station data. Select ‘georeference’ to translate this into coordinates, i.e. latitude/longitude pairs.

Select referencing ‘by street address’, specify the city and country and hit continue. You’ll see a new column appear with latitude and longitude data for each station. We’re almost done.

Map the Results

At the top, click on ‘map view’ and click the wizard/wand icon on the right.

To create an animated map with categories, select ‘Torque Cat’, then use the drop down menus to set ‘time column’ to your start time property. Then set the category column to the end station and use the fields below to map colors to end stations by region, for example.

Ta-da! You’re done! You just built an awesome animated map. But, suppose you wanted to calculate a few more interesting things and plot the output? With kimono’s filter functions you can do just that. We’re beta testing this feature right now, so just email us at support@kimonolabs.com and we’ll give you early access to the feature.

Filter functions allow you to write JavaScript functions that operate on the data returned by your API. With filter functions enabled your APIs will now return the processed output. For example, we wrote a frequency function to count the number of times each station appears in the dataset in total, and how many times during the day and the night, allowing us to create a heatmap of where Andrew spends the most time, and how that changes by time of day. Once we’ve enabled you for filter functions, you can access them from the ‘advanced’ tab of your API.

Copy in our frequency function below, to start:

functiontransform(data,callback){varcollection=data.results.collection1;//shortcut for our collectionvartotalTimes={};//object for histogram for all times. Key is a string of station name.//helper function to return if is during day or night...between 7am and 7pm = day. //assumes a Date-able string as input vardayOrNight=function(date){vartime=newDate(date);if(19>time.getHours()&&time.getHours()>7){return'day';}else{return'night';}};//helps populate totalTimes with key and value pair of address and total, day, night timesvaraddToHistograms=function(station,date){//initialize for a given address if(!totalTimes.hasOwnProperty(station)){totalTimes[station]={'station':station,'total':1,'day':0,'night':0};if(dayOrNight(date)==='day'){totalTimes[station].day+=1;}else{totalTimes[station].night+=1;}}//add for a given addresselse{totalTimes[station].total+=1;if(dayOrNight(date)==='day'){totalTimes[station].day+=1;}else{totalTimes[station].night+=1;}}};//iterate through property2s (start destination) for collection1 and add them to totalTimesfor(vari=0;i<collection.length;i++){varstation=collection[i].property2;vardate=collection[i].property3;addToHistograms(station,date);}//do the same for end destinationfor(varj=0;j<collection.length;j++){station=collection[j].property4;date=collection[j].property5;addToHistograms(station,date);}//delete old datacollection.splice(0,collection.length);//pop off totalTimes by just the value, and add to the collection array (for csv formatting purposes)for(varkeyintotalTimes){if(totalTimes.hasOwnProperty(key)){collection.push(totalTimes[key]);}}callback(null,data);}

Using cartoDB’s bubble plot setting, we can quickly turn this into a heatmap of where Andrew spends his time.

That’s just a quick preview of some powerful maps you can build with kimono and cartoDB. We’re excited to see what you will build with the tools. Tell us what you create at contribute@kimonolabs.com and reach out to us at support@kimonolabs.com if you get stuck.

We’re pleased to welcome Francisco Dans to the CartoDB team! He’ll be animating bubbles everywhere with Torque.js.

Fran is from Galicia, one of the most lovely places in Spain and just landed from London where he has been working as a freelance developer (altough his background is map design) for mapping companies the last year and a half. He is an active member of the OSM community since 2012.

He is currently working on Torque.js, our library to render big timeseries data in the browser. We have pretty big plans for Torque this year, starting with new ways of rendering animated datasets like this:

Fran is also a petrolhead, loves motorbikes and dreams with each new Top Gear show. We hope he gets a proper car driving license this year :trollface:.

Follow his progress in GitHub and random stuff in Twitter

Welcome to the team!

Nope, that’s not an oil slick. Look again. The colors correspond to every American river based on which direction each one flows. But this visualization does more than showcase the rainbow-effect. It’s interactive. CartoDB’s Senior Data Scientist, Andrew Hill, brought U.S. geography to life by rendering a vector map of rivers that recieve rain every hour. Our friends at Fast Company like it, too.

Inspired by reading a post from Stephen Von Worley on Curbed SF, Andrew sourced U.S. Government satellite data to begin builiding his own colorful map. He harvested the work of Nelson Minar, whose tutorial project demonstrates how to make a web map with vector tiles, to convert a shape file into a csv for import into CartoDB.

This story has all of the elements we enjoy: open data, open-source, and wow-inducing visuals.

Interested in pretty maps DIY, but lacking blogosphere inspiration? Try importing the locations of +4,000 stream gaging stations and streamflow information from National Atlas, courtesy of CartoDB’s common data portal and accessible from your dashboard. Check out our ever-expanding options for more map ideas.

Happy Mapping!

Allow us to introduce the newest addition to our growing collection of cartographers, Michelle Chandra! She’s just joined our NY office where she specializes in showing CartoDB love to our global community.

After having moved around between NorCal, SoCal, and the Southwest, Michelle decided she was tired of awesome weather and moved to NYC to explore the intersection of experimental art and communicative technology at ITP-NYU.

With a professional background in content strategy, website management, and marketing, Michelle is going to fire-up our community team by hosting tutorials, curating data, creating cool maps, and generally documenting CartoDB awesomeness.

Ever seen a double rainbow? Well Michelle has, and she was so inspired by it’s beauty (although not as inspired as this guy) that she decided to create this incredible visualization!

As you can see, Michelle likes her maps “simple, elegant, and interesting,” and we’re excited to see what she cooks up next!

Welcome, Michelle!

We are excited to introduce Eva Cabanach who returns to Madrid from London to join the CartoDB team! Eva will be helping grow our community of users through dedicated outreach.

Eva is an avid traveller who trekked to China, Brussels, Paris, Berlin and Porto last year alone. Next up, she wants to finally visit New York City! When not travelling, Eva enjoys reading, shopping and supporting Real Madrid.

Although a native of Madrid, Eva spent the past 4 years enjoying all that London has to offer including it’s many hidden gems and shopping. London is her second favorite city - after Madrid, of course!

Eva’s professional background is in digital marketing. She most recently worked for two large agencies, and managed all the lifecycle campaigns for Paypal - across 24 different regions and 12 languages!

Eva is greatly inspired by the power of maps as a storytelling tool. Her favorite CartoDB maps are the animated ones, especially Torque maps.

Welcome Eva!

Today marks the beginning of the wind down for Google Maps Engine. As of today, the service is no longer taking new signups. Some good news is that we have been working closely with Google to create a CartoDB on Google Platform product. If you are looking for an alternative, please review our new offering. Below, you can also find a comparison of the two offerings, GME and CartoDB, and information about migrating your existing projects.

Over the years, Google Maps Engine (GME) has grown to become one of the most similar products to our own and one of our closest competitors. It is so similar in fact, that the question, “What is the difference between CartoDB and Google Maps Engine?”, might be one of the most common questions we get from customers. Today, we want to spend a minute to help answer that question.

We know that a lot of existing GME fans and users are going to be looking for a similar set of tools to maintain their projects without significant changes. If you are one of those GME fans or if you are thinking about using GME in a future project, let us take a moment now to explain why CartoDB is the obvious choice for your next project or as the new home for your existing one.

What do both services offer?

The services in CartoDB and GME are very similar - including a web interface to manage data, create maps, a set of APIS to access data, request maps, and create applications. Let’s take a look at the organization of those services.

While Google Maps API is here to stay, this morning’s news means that the Google Maps Engine UI and Google Maps Engine API are no longer available for new projects.

Editor and User Interface

The GME service provided a simple and minimalist interface for interacting with your hosted data and maps. At CartoDB, we offer a similar service. A major difference is that at CartoDB we have worked hard to refine the process of working with maps and geospatial data online through simple interfaces and intuitive interactions. We are constantly improving, measuring and refining those interfaces. The CartoDB Editor is far ahead of the GME interfaces in regard to usability and design.

The ease-of-use in the CartoDB Editor has quickly made it a favorite among university educators, data analysts, sales managers, and journalists. Maps are no longer just for for developers!

Input Data Formats

Prior to this morning, GME was one of only a few CartoDB alternatives that offered support for such a broad range of data formats and file types. GME and CartoDB support: point, line, polygon and raster import, as well as, analysis and rendering. In fact, compared to GME, CartoDB actually has a more comprehensive list of supported file extensions, including GeoJSON and GPX.

APIs and Platform

CartoDB and GME have become exciting services for power users because they expose full suites of APIs that allow the developers to leverage everything from data import to map creation. What this means is that most of the functionalities you see in the CartoDB Editor actually run on APIs that can be leveraged from your own applications and interfaces. The GME user interface was the same.

For users looking to leverage geo-processing capabilities, CartoDB offers everything available in Google Maps Engine and much, much more. CartoDB includes access to a library called PostGIS which enables a comprehensive list of GIS and data filtering functions including buffers, intersections, proximity searches, and more.

Authentication and Identity Management

In GME authentication is based on the Google Authentication system, including support for enterprise account. In CartoDB, we now give organizations on enterprise accounts the ability to use Google Authentication for single sign-on and identity management. If you are interested in other authentication connectors, such as LDAP, let us know and we can explain what is possible.

What types of maps can you make?

In terms of map design, GME took a slightly different approach when exposing customization tools to the user. Both services allow the user to interact with simple wizards to modify the style data on the map. When you dig deeper, GME had created a unique JSON structure for applying styles to maps. Alternatively, CartoDB exposes CartoCSS to users. CartoCSS is a more widely used map styling language and is immediately comfortable to anyone familiar with CSS.

For users switching to CartoDB from GME, the CartoCSS language will still allow for attribute and zoom dependant styling. It will also open up a wide range of new features to leverage for your maps, including better control of color, scale, and positioning.

How can you leverage the Google Maps API?

When looking for an alternative, the ability to use these APIs will be a key consideration for many GME fans. Fortunately we have built CartoDB so that the full set of tools available in the Google Maps API toolkit can be used with data hosted on our services. Any API you look to build on is included in the services - from Google basemaps, to StreetView, the Directions API, and Traffic API.

To help get you started we put together a set of examples that show how you can use custom data hosted on CartoDB with Google Maps API methods. Take a look at the full set of examples:

• Directions API (click on a point)
• CartoDB SQL API and heatmaps
• Many polygons and interactivity
• Animated maps

How do the services compare in performance and scalability?

Many users turned to GME in order to leverage the Google infrastructure for their own products. You might be surprised to know that CartoDB can actually run on the Google Cloud Platform (as well as other infrastructures) and that we have created a specific server cluster within Google Platform. If you are interested in using this feature, ask us about CartoDB on Google Cloud.

In addition to where CartoDB can be run, we have developed technology that enables us to provide world class performance and reliability in our services no matter who runs it. For example, some of the world’s most popular websites use CartoDB to regularly publish maps to their viewers. Our services scale and adapt to handle every user, here are some of the ways we do it:

• Multiple layers of caching: CartoDB performs caching operations at many different stages of data and map transfer, from the database all the way out to the user. At the final stage, CartoDB uses a next-generation CDN that scales with minimum latency and has direct connection to Google servers.

• Horizontally scaling renderers: Inside CartoDB infrastructure, multiple servers render the maps in parallel. During periods of peak load, new renders are added to the available pool.

• Scaling the database: For very large datasets or very dynamic applications, CartoDB can federate multiple servers at the database level to allow for parallel queries across multiple servers.

If you are interested in more details on the CartoDB on Google Cloud solution, ask us for an early look at our forthcoming white paper detailing the performance in real-world scenarios. We have seen sustained performance at more than 50,000 requests per second and updates every 10 seconds.

Finding the Right Data

Often one of the hardest parts to making a map is finding the right data to get it started. As a solution, Google Maps Engine offered public datasets that users could query. Similarly, CartoDB provides frequently requested data such as Natural Earth and USGS datasets for simple integration with your own data. On top of these curated datasets, CartoDB also gives users access to Twitter Firehose, MaxMind IP address database, and other sources of data to create maps from the most interesting data on the web.

Data Migration Options

There are three existing options for you to use. The process to manually download your data from GME and import it to CartoDB is straight-forward for many users. If you need more or have a lot of data, we have been working with two of our certified partners, AppGeo and HabitatSeven, to create a migration tool to help on the migration process. The tool authenticates with Google and using the ogr2ogr tool moves data from GME to CartoDB. It is as easy as selecting the tables you want to migrate and entering your API keys. Finally, the third option is FME

We will announce the migration tool very soon, stay tuned.

Conclusions

Today is the beginning of the end for the Google Maps Engine service. For users and developers that have grown to rely on the service, we want you to know that there is no need for stress. CartoDB provides an excellent service with viable alternatives for the complete GME product. Both services were built to be fast, scale to millions of requests, and simplify your development of geospatial projects.

We have been working with Google to provide a solid offering called CartoDB for Google Cloud and ensuring that GME customers will have all the power of Google Maps API available on CartoDB. We have already several projects migrating and several partners helping on the process.

Are you interested in moving from Google Maps Engine to CartoDB today? Get in touch to talk to one of our experts about using our custom migration tool to move your hosted data with ease. We can talk through your project needs and timelines and come up with a plan for your move to CartoDB right away.

We can’t wait to see what you build on top of Google Maps and CartoDB.

A common challenge we see facing our users is how to create dynamic maps with secure data. We have always had the ability to securely publish interactive maps from private data on CartoDB. Today, we go a step further. With the new Named Maps API, users can create dynamic maps, capable of being filtered and modified in front-end applications while maintaining the security of the data used to render those maps.

The Named Maps API gives you a way to define parameters that can be used in SQL and CartoCSS to change your visualizations on the fly. This allows you to pass parameters to filter the data on your map, or redefine the design based on user interactions. At the same time, it limits any viewer from passing unrestricted filters that could expose your data in unexpected ways.

If you are interested in learning more about the Named Maps API take a look at these great resources:

Named Maps Tutorial will take you from beginner to experienced user of the API in 30 minutes.

In a couple of weeks we will host a Named Maps Webinar that will show you a few more of the advanced methods exposed in the API.

Finally, our Named Maps Documentation is full of rich information for leveraging the API in your application today.

Keep the maps coming!

On Tuesday evening, President Barack Obama delivered his final State of the Union Address. This year marks the first year that the State of the Union address was made available as full text for perusal and commentary prior to the speech. In the spirit of this openness, we’ve paired a few pull quotes with some awesome maps from our community. This will kickstart a series of blogs wherein we’ll feature 5+ community maps based on a concept or theme.

Skim the speech text here, and follow along below to check out the State of the Union as measured in maps!

Economy

The topic of economy is so broad and important that it takes a major role in almost every political speech. In Obama’s final SOTU, he covered it from many angles, indicating that it is clearly a topic he knows we are all watching and thinking about carefully. As a major driver and indicator of the economy, the topic of employment came up again and again. In an interesting series of maps, the Wall Street Journal showed us how U.S. job opportunity has been changing over time. Take a look at this example, showing the change from 2013-2014.

Another map we found interesting is the one below, created by the International Business Times last September. The map shows the at-the-time ongoing protests by fast-food workers around the country, aimed at raising the hourly wage.

Healthcare

There is no question healthcare has been a focal point of Obama’s presidency. In a story published in November, the Wall Street Journal highlighted what America was talking about during the interim election. In the map below, the WSJ focused on healthcare, a topic highlighted during the election by many Republican candidates. If you are interested in maps showing the intended impact of Obamacare, also take a look at this series of maps from the IBTimes in 2013, Obamacare: How Effective Will It Be In Each State?

Education

Throughout his discourse on education, Obama detailed the need for more and easier access to higher education, stating, “[b]y the end of this decade, two in three job openings will require some higher education. Two in three.” We thought the contrast between the two maps below was relevant. In the first, the IBTimes published a map of High School Rankings around the country. In the second map, published by one of our users, we can see the concentrations of the country’s top ten public and private universities…

Environment

Obama took a moment to highlight his administration’s commitment to the environment and the fact that they have “set aside more public lands and waters than any administration in history”. This is very relevant to the work of the Global Forest Watch, which aims to make the data around deforestation available for journalists, policy makers, scientists and others. While not always perfect, protected areas are a great step to protecting our forests.

Learn more about public land use and deforestation globally here.

Equal Opportunity

Same-sex marriage is just one component of the equal opportunity struggle. Al Jazeera America created the map below in the process of tracking the status of same-sex marriage across the country.

Women’s right to choose

The President approached the topic of womens’ right to choose gingerly, saying, “We still may not agree on a woman’s right to choose, but surely we can agree it’s a good thing that teen pregnancies and abortions are nearing all-time lows, and that every woman should have access to the health care she needs.” This is a topic some of our users addressed thoroughly. Take a look at this complete interactive piece developed for the 40th Aniversary of Roe V Wade.

Learn a bit more about that map in this feature on its authors.

BONUS: Campaign Zinger

I have no more campaigns to run. My only agenda for the next two years is the same as the one I’ve had since the day I swore an oath on the steps of this Capitol — to do what I believe is best for America.

~ President Obama, January 20th, 2015

And in case you missed it, check out the reaction to this quote:

Finally, we’d be remiss if we didn’t include Twitter’s map of the SOTU reaction, available on Vox

Read more

If you want to read more about State of the Union, check out the ABC News coverage or the Atlantic’s Split in the State of the Union piece. Here’s some interesting history, datasets, and data science related to the SOTU addresses over the years.

Stay tuned for a map review of the address this week, and look out for our upcoming series, Roughly 5 maps to explain [XYZ]. Thanks for reading!

Happy mapping!

As many know, CartoDB’s commitment to elevating efforts in conservation, climate change, and the environment has shaped our practice and given way to many exciting mapping projects such as the creative and impressive Carbon Calculator and the Global Forest Watch Map that allows users to monitor deforestation in real time.

Recently, we came across a very interesting case-study article in conservation and invasive species mitigation published by the international weekly journal of science, Nature. The article, titled Invasive species: The 18-km2 rat trap and authored by Henry Nicholls, depicts a narrative of passionate and dedicated conservation through invasive species eradication (specifically rats) in the Galapagos archipelago off of the Ecuadorian mainland coast. As the article states: “The rodents feed on the eggs and young of seabirds, land birds and reptiles, and have brought several species — including the rare Pinzón giant tortoise (Chelonoidisduncanensis) — to the brink of extinction…[the effort] promises to allow unique species to flourish again and, building on the prior removal of feral pigs and goats from much of the archipelago, to make Ecuador a world leader in the eradication of invasive species”.

Fueled by our passion for cutting-edge mapping technology and the environment, Nature’s article inspired us to take this beautiful static map of the conservation effort phases and create an interactive and versatile version that allows for in depth exploration using the CartoDB platform!

Check it out!

Map Annotations

The CartoDB Rat Race Map serves as a wonderful example of just how CartoDB can be used in journalism, planning, and information design in a broad sense. One of the key features of this map is the annotation styling that allows any CartoDB user to place text information directly related to specific Geo-spatial locations on the map. This often overlooked built in feature offers versatile labeling capabilities that anyone can use!

This is how it works:

In the upper left hand side of your CartoDB visualization editor (not the table editor), There is an option to “Add Element”. This drop down menu contains CartoDB’s built in capabilities for labels, text boxes, images and annotations.

Clicking on add annotation item, generates a movable element that attaches itself to the geographic location on the map. This feature remains fixed on the specific latitude and longitude and moves with them map, unlike the label, text, and image options.

The built in styling interface allows for your annotations to be fully customizable and comes with a wide range of features to get the most out of your maps.

The user interface allows users to modify the text, color, orientation, size, bounding box, and many other styling options to fit the needs of the map and story being told. The styling example above, generates this annotation:

in the styling interface, the option “Zoom (min-max)” adds an additional level of customization to your annotations by designating the parameters where your annotations are the most prominent. In the example above, the parameters are set from zoom level 8 to 28. This means that the opacity and prominence of the annotation is reduced outside of these boundaries. Here is what the same annotations look like at zoom level 7:

As you continue to explore and create with CartoDB don’t forget to share your maps, stories, and data visualizations with us @cartodb on Twitter. We’re always excited to see new maps and interesting visualizations that we can share with our mapping communities!

Happy Mapping!

The open data movement is sweeping the globe like winter storm Juno has swept the Northeast. Yet when individual cities want to make their data available online, they don’t want to end up with a data portal that’s too unique a snowflake — and thus too hard to use and maintain.

From traffic crash reports to zoning maps, opening up data is making it easier for everybody — officials, citizens, companies and media — to understand their cities and make better decisions.

That’s why today we’re excited to announce that data portal service OpenData.City is now integrated with CartoDB.

OpenData.City is built on CKAN, the popular open source platform running data portals like Data.gov. CKAN is battle-tested, secure, full-featured, and flexible.

OpenData.City is designed with small and medium sized cities in mind, making the process of setting up and running an open data portal straightforward and headache-free.

“With OpenData.City and CartoDB, city officials can create useful maps on the fly,” says Joel Natividad, CEO of Ontodia, the data wrangling and publishing company behind OpenData.City.

“Almost all city data has a spatial component, and while CKAN is incredibly powerful, the standard spatial features are quite limited. We wanted to give users a strong, flexible toolset for viewing and publishing their datasets.”

With OpenData.City, you can upload datasets via a series of simple wizards that collect all the right details and validate the inputs for accuracy. You can create flexible dashboards to highlight key metrics and track changes over time. And you can search for, filter, and publish data in all sorts of ways.

Want to learn more about OpenData.City and CartoDB? Check out the overview and sign up for our introductory webinar on Wednesday, February 11 at 11am EST.

Unlock that data!

Join us in welcoming our latest cartographer, Aurelia Moser, to the CartoDB team in New York City! Aurelia is a librarian, developer and cartographer by trade who builds communities around code through education and outreach.

Aurelia recently wrapped up a Knight-Mozilla Open News Fellowship which had her traveling around the world speaking at many conferences and hackathons. She is passionate about open data, cybersecurity and privacy, hacking-for-good and code. She is an active member of the NYC meetup community where she develops curricula for Girl Develop It NYC and co-organizes Nodebots-NYC, a meetup for developers to program robots with Node.js.

Originally from the Midwest, Aurelia attended undergrad at the University of Wisconsin, Madison and spent a few years in the south of France in grad school and as a teacher before returning to NYC five years ago.

Aurelia believes maps are one of the most intuitive types of visualizations in their ability to reconcile both the physical and digital worlds, and because of the ease with which people are more readily able to understand them.

One of Aurelia’s favorite side projects is her online radio show,Stereo Semantics, in which she generates playlists based on the semantic web of connections.

Aurelia also loves making lists and project management tools, some of her favorites include:

1. thenews.im - For keeping posted on hacker/designer news
2. the point - Extension for commenting and archiving collaboratively on articles
3. reporter - Slick interface and graphs for regular data review, can export your data at the end of the year, as a csv to run feltron-esque reports on your daily life and (un)productivity.
4. task warrior - an awesome cli app for task management

Welcome Aurelia!

CartoDB is excited to participate in the American Museum of Natural History’s Educator’s Evening on January 30. We’re excited because we know that secondary and primary students will do amazing things with maps.

What makes CartoDB an exemplary addition to the classroom is it’s flexible interface for collaboration, visually stunning graphics, and it’s ease of use. Literally any explorer can become a map maker.

Last Fall, local New York City high school students used CartoDB and open datasets to explore their own questions related to the impacts of climate change on New York City.

Using various techniques, the students familiarized themselves with how to use and tell stories with a broad range of data. More importantly, the students were able to engage in a discussion about the maps, apply critical thinking to interpret the stories the maps were telling, and demonstrate a new understanding of how effective, confusing, or misleading maps can be.

“Working in CartoDB allowed these students to think about and interact with data in a way they never had before. We as educators learned that a tool such as CartoDB could effectively be incorporated into our curriculum,” observed Hannah Jaris, Senior Coordinator at AMNH.

Each student walked away with a new interest in cartography and the concepts of visualizing data in new ways.

The final projects (group and individual) consisted of analysis of the impacts of Hurricane Sandy on New York City, a close examination of the 100 year flood zones around Manhattan - or the areas that have a 1% chance of flooding each year in the year 2050, and the location of FEMA Evacuation Zones related to our highest need populations.

You can review the students amazing works here.

CartoDB will be at AMNH tonight to chat and demonstrate the power of map making in the classroom.

Happy mapping!

In case you were trapped on a moutain, in a blizzard so thick you couldn’t see your hands in front of your face, and just got back to civilization this morning, let us say the Superbowl happened last night. The team from the East Coast beat the team from the West Coast in a game where many fans held their breath until only 10 seconds were left.

If you love football, or if you don’t love football but you love maps, we want to show you a couple of maps that use CartoDB to show us some neat things about the Superbowl last night.

Twitter

In the first map, Simon Rogers assembled all the tweets from across the web and across the globe sent during the game. Using Torque he shows how different moments in the game change the patterns of conversation. One thing you might notice from this map is that the only thing that could get Seattle to stop talking about themselves was the half-time show.

Google Trends

The second map - Google shows us their data for the most searched teams on the web. You can see from the geographic breakdown of how people align with their favorite teams. Vizzuality built the map using CartoDB’s SQL API to create GeoJSON for client-side mapping. The results are beautiful.

Trending maps

We love seeing these awesome maps happening, keep them coming!

We’re excited to announce that Carlos Tallón is joining CartoDB. Carlos was previously lead designer at Minijuegos.com. He has excelent taste in design and aesthetics and awesome iconography skills. Carlos will be helping our community grow and make the CartoDB Editor even simpler to use.

Carlos is from Aspe, a small minicipality in the eastern part of Spain and which has nothing to do with Aspen, Colorado. Although he works remotely from there, he enjoys visiting the Madrid Office and beating people at ping-pong.

As a plus, Carlos usually brings cakes and chocolate directly from the ovens of his family’s bakery.

Design has been always part of our DNA, so we are thrilled about this new addition to our team.

Welcome, Carlos!

FOSS4G and PGCUS are the premiere conference series for the open data movement which seeks to bridge society, developers, educators, activists, companies, and governments together on the subject of free and open sources for geospatial software.

CartoDB will be participating at the FOSS4G-North America and PGConf US 2015. This year CartoDB will sponsor both events and will also have booths.

FOSS4G-NA will be held March 9-12 in San Francisco. CartoDB team members will be available to answer any questions while demoing CartoDB’s latest innovations.

Our Map Scientist, Andy Eschbacher, will be giving a presentation called ‘Everyone’s a Geographer.’ Andy plans to share his expertise in education and mapping. The session is designed for beginners but not limited to the novice.

In the afternoon, check-in with CartoDB’s CSO - Chief Science Officer, Andrew Hill. Andrew will be hosting a discussion titled, ‘What is a Map?’ Andrew’s talk focuses on how open data has changed the way we map.

Although not a member of the CartoDB team, Nick Martinelli will be forging ahead with a very cool presentation on ’Carto CSS and UTF Grids for Collaborative Mapping’ - using FOSS4G.

PGConf US will be held on March 25-27 in New York City. This will be CartoDB’s third time in attendance. We believe that the PGConf US can help move PostgreSQL into a new era of usage, but we need your help in bringing in attendees.

Join the CartoDB team and see how we hosted a version of PostgreSQL and PostGIS to enable incredible map making. Attendees can also meet our CEO, Javier de la Torre, to learn how our company is expanding in the NYC and U.S. community.

CartoDB is offering a special discount on attendance price when you tweet, blog, or comment CartoDB on social media - just mention the code: CARTO10.

CartoDB will also have some exclusive CartoDB swag in the fund raising auction!

PGConf US 2015 is a not for profit conference hosted by the local NYC PostgreSQL User Group (NYCPUG) and run by the United States PostgreSQL Association.

View CartoDB’s role in past conferences here.