Greetings mobile data mappers! This is a very special post coming to you from the Xamarin Evolve conference in sunny Florida. If you are attending, make sure you visit CartoDB’s booth (#118) and meet me in person!

Now, let’s get back to our regularly scheduled programming in our mobile series, with an introduction to a non-traditional type of map, indoor maps.

In many cases, indoor maps are very similar to “ordinary” maps but on a smaller scale. For example, the recent article by The L.A. Times, charts every shot ever taken by Kobe Bryant on the basketball court. The interactive graphic was done using standard mapping methods and data in a creative way with CartoDB technology. You’ll notice that the visualization is an interactive chart and not a geographical map. This type of spatial visualization can also be done on mobile devices.

With CartoDB’s mobile tools and SDK, you get some special features and methods specifically developed with indoor maps in mind:

1) Ground overlays from raster images allow raster image based indoor plans to be used as map backgrounds. You may have CAD drawings or just nice artwork.

2) You can use 3D Polygons. These are just like polygons, but with height and 3D rendering, and customizable colors. They can be added to your visualization as vector elements. You can use them to show rooms, walls, or as 2.5D or “shoebox models,” and color each individually and add interactions. For bigger datasets, thousands of polygons, we suggest the use of vector tiles instead of just the vector layer.

This Google Glass app uses 3D polygons to map the Mobile World Congress venue:

3) For most advanced cases we can show a full indoor 3D model. This requires nice 3D content, which is hard and quite expensive to create, even if you use free tools like SketchUp. The models must not have too detailed of a geometry. You can use textures to give a very realistic perspective to building floors and also outdoor looks. See our demo screencast video on how this might look in your app.

For indoor positioning there is no global standard solution. We don’t provide indoor positioning, but there are many organizations that do. Some companies are focussed on proximity only, and some even give geographical location coordinates. I would check outstartups like: indoo.rs (Bluetooth beacons and WiFi), Gimbal (proximity), and IndoorAtlas (geomagnetic field, no special beacons).

As indoor map data providers we suggest checking out Micello, and also HERE for licensing data. Some other vendors who deal with indoor maps (Google, Apple) unfortunately just use them in their solutions and do not give data away. You can import your 3rd party data into the CartoDB platform and then use it with the mobile SDK as described in an earlier blog post in the series.

Happy mobile mapping!

Next week CartoDB will be at Qonnections 2016 as a Silver Sponsor for the first time. Come visit booth #801 to see our demos and learn how CartoDB can enhance your Qlik^® business by providing even more value through location intelligence.

The CartoDB platform expands the value of Qlik^® investments by shortening time-to-insight in products and solutions with real-time geo-optimized analysis, dynamic mapping with multiple layers, and the ability to process and visualize millions of data-points. We also have a great reseller program with training and revenue shares.

Some of our top customers and partners like Boston Consulting Group, Twitter, and Bloomberg choose CartoDB because of our core differentiators:

• Scalability - Easily visualize and analyze millions of locations in the cloud or on premise with multiple layers and custom styling

• Batteries included - Geocoding, routing, isolines, demographics, augmentation, segmentation, geostatistical algorithms, and boundaries are all built in

• Democratization - We provide the best learning curve experience, allowing you to quickly master location intelligence without having a black belt in GIS

• Embeddability - Build on top of our fast platform APIs. Leverage them to add advanced geo-analytical and mapping capabilities into your existing platforms and apps

• Mobile - Native mobile SDKs for iOS, Android, and Windows. Bring location intelligence to any device

Check out this map we made that shows Qlik^® partner and customer locations along with their optimized driving routes to Qonnections 2016:

By combining CartoDB with Qlik^®, organizations can turn all of their data, including location data, into actionable insights across all aspects of their business. We look forward to seeing you at Qonnections 2016—drop us a note if you want to hear more.

At CartoDB we are fierce champions for open data! That is why our Grants for Good Program is supporting organizations and volunteers who advocate for more access to open data using innovative technology and crowdsourcing techniques. CartoDB work with stakeholders who are using open data and maps for research, business intelligence, deep analysis and better communication. We are happy to introduce you to our newest grantees who are blazing the trail in these spaces.

Bath: Hacked

Bath: Hacked is an established, community run open data project based in South West England. Their goal is to publish and use open data for the benefit of the local community, residents, and visitors to Bath and North East Somerset. Bath: Hacked has been working hard to not just publish data, but also run events and meetups to encourage the thriving tech community to build interesting and useful things using that data. Most of that data has a geographic element, which means that they are big fans of CartoDB! “This support will let us scale up our ambition for the types of services we are able to build. And, as a community owned and operated project, this type of investment lets us continue our mission,” says Bath: Hacked community member. They’ve used CartoDB to visualize traffic accidents, chart business growth in the area, map local population demographics, and area classifications from satellite imagery. They even produced a video tutorial that demonstrates how to use CartoDB with data from our datastore!

“We’re now designing our next tool, which will use our growing collection of open geographical data to build a tool that explores who owns and manages land in our area. This will support residents in requesting support and maintenance from our local government and housing associations, and also help community groups identify green spaces that could become the site for the next local community growing space,” says Bath: Hacked community member.

Datenna: China Innovation Intelligence

Datenna tracks, visualizes, and maps Chinese data related to science, technology, and innovation. They are in the process of building an interactive SaaS solution to offer services, like competitor intelligence and technology matchmaking, directly to their customers and help free the world from static reports.

Datenna wants to show all 300 State Key Labs (SKL), laboratories at Chinese universities and research institutes with special status, on an interactive map. Information collected about these SKLs include: their Chinese and English names, description, location (GPS), keywords, technology category, founding date, and parent organization.

They want users to be able to filter SKLs based on technology categories, explore how the amount of SKLs grew over time, and see which areas in China have more SKLs. Datenna has plans to expand to Chinese Engineering Research Centers (another 200~300 organizations) and to other similar facilities.

Datenna’s mapping project is in early stages but they have already put CartoDB to good use!

Bath: Hacked and Datenna are connecting citizens to the power of data and allowing for deeper insight into the innovation and technology sectors… Keep up the great work!

Happy Data Mapping!

In practice, journalist require the most trusty and reliable information to disseminate messages to the public. There is no better way to do that than by aggregating facts, or data, and analyzing and providing that analysis in the most accessible way possible. To do so, journalists across sectors and media need a technology that is compatible with the realities of the modern newsroom and provides data analysis on the go. CartoDB is that technology for the new media sector.

CartoDB will be attending the fourth annual Jornada de Periodismo de Datos conference, organized by Open Knowledge Spain on data journalism and open data. This year, talks, workshops, and discussions will be focused on displaying data visually, digital literacy, open data, and security and privacy.

CartoDB’s Ramiro Aznar will be giving an introductory workshop on CartoDB for journalist and Ernesto Martínez will be hosting an advanced workshop. Join Erik Escoffier, as he talks about all the latest CartoDB news. Additionally, we’ll be hosting a happy hour later in the day. You can view the agenda for Friday, May 6here.

As the U.S. continues to ramp up for the presidential elections this year, journalist and media outlets have a lot at stake with keeping the public informed and up-to-date with the latest in campaign news. How can your news outlet distinguish itself from the rest of the politico noise out there? Data-driven visualizations, that’s how! Join our ‘Mapping Elections’ webinar, tomorrow, hosted by Erik to learn how to build the most visually stunning elections maps.

And don’t forget to register for our Geo-Jour Newsletter to stay up-to-date on new resources, projects, and events for the media sector.

Happy Data Mapping!

Lately, there has been a lot happening at CartoDB related to the visual exploration of location data. We’ve been researching, for quite a while, new ideas for visualizing thematic information on the web. Apart from color, we might have something to share with you soon, we are investing time experimenting with different methods for symbolizing data, and a set of tools to drill-down into other (non-visual) attributes of the data.

Some of the results from those experimentations come from our Senior Digital Cartographer, Mamata Akella, who inspired by maps like this and this, came up with an uncomplicated and astonishing idea, using SVG markers and a simple set of CartoCSS rules, to create a “Data Mountain Map”. What is so neat about this, is that with some simple CartoCSS tricks she achieved a great way of creating perspective maps that avoid some of the typical problems that occur with Bubble Maps when visualizing this kind of data.

This, together with deep-insights.js helps us detect patterns and outliers. For example, you can explore a county near Santa Fe, where people spend under 20% of their income on rent, or locations that appear more expensive like San Francisco, Los Angeles, NYC, and DC and discover that people spend a lot on rent, but it still remains under 30% of their household income. On the other hand, if we filter to see where people spend most of their income on rent we see places like Miami, where they spend more than 40% of their income.

Even as an experimental project, we can see how well it works for visualizing clusters of high values (mountains) without hiding the lower values. It also plays very well with colors, white space, and creates a nice effect when tweaking the filters.

Creating your own Data Mountains Map

Create a grid with your data

First, you’ll need to snap your data to a grid. You can do that with just a simple SQL query using the ST_SnapToGrid method available in CartoDB. You can see that we are re-projecting the data to an Albers Equal Area projection, centered on the US, all in the same query. Isn’t that amazing?

SELECTcartodb_id,percent_household_income_spent_on_rent,median_rent,ST_Transform(ST_SnapToGrid(the_geom,0.5,0),5070)ASthe_geom_webmercatorFROMyour_table_nameORDERbyST_x(the_geom)DESC

Note that in this case, we are selecting the percent_household_income_spent_on_rent and the median_rent columns since they will be the variables that we will use to change the size and color of the triangles in our CartoCSS.

Add some styling

Second, as you can see below, we use CartoCSS conditional styling for changing the height of the triangles based on the values in median_rent, and changing the color depending on the values of the percent_household_income_spent_on_rent column.

#your_layer_name{marker-line-width:0;marker-width:4;marker-allow-overlap:true;[zoom<=4]{marker-width:2.5;}marker-file:url(http://com.cartodb.users-assets.production.s3.amazonaws.com/maki-icons/triangle-18.svg);}#your_layer_name[median_rent<=1659]{marker-height:80;[zoom<=4]{marker-height:40;}[zoom>=6]{marker-height:160;}[zoom>=7]{marker-height:320;}}#your_layer_name[median_rent<=868]{marker-height:40;[zoom<=4]{marker-height:20;}[zoom>=6]{marker-height:80;}[zoom>=7]{marker-height:160;}}#your_layer_name[median_rent<=672]{marker-height:20;[zoom<=4]{marker-height:10;}[zoom>=6]{marker-height:40;}[zoom>=7]{marker-height:80;}}#your_layer_name[median_rent<=518]{marker-height:10;[zoom<=4]{marker-height:5;}[zoom>=6]{marker-height:20;}[zoom>=7]{marker-height:40;}}#your_layer_name[median_rent<=367]{marker-height:5;[zoom<=4]{marker-height:2.5;}[zoom>=6]{marker-height:5;}[zoom>=7]{marker-height:10;}}#your_layer_name[percent_household_income_spent_on_rent<=50]{marker-fill:#751b4f;}#your_layer_name[percent_household_income_spent_on_rent<=36.7]{marker-fill:#b7187b;}#your_layer_name[percent_household_income_spent_on_rent<=31.2]{marker-fill:#e05966;}#your_layer_name[percent_household_income_spent_on_rent<=25.6]{marker-fill:#f9952d;}#your_layer_name[percent_household_income_spent_on_rent<=20]{marker-fill:#FFDAA7;}

And that should be all! Create your “Data Mountain Map” and let us know how it looks. We would love to see what some of the best uses of this type of visualization are.

Happy (Infographic) Mapping!

When visualizing, on a map, tens of thousands or millions of points in a dataset, data points may be spread all around the world, or be confined to a single country, region, city, or even some smaller area. In any case, there will be a zoom level in which the whole dataset is represented in a single 256x256 pixel tile. The “great mother of all tiles”, tile 0/0/0, will inevitably have to show the whole collection of points.

Rendering all the points in that tile is not only a tough task, but it isn’t optimal at all. Moving all that data will take quite some time, and drawing it will be a daunting task. If it does get completely painted, most of the work will be useless because you cannot possibly depict millions of features in a 256x256 pixel grid – many features will be stacked on top of others occluding them.

Of course, some children of that tile will also be packed with too-many features, so we will have similar problems at higher zoom levels. Eventually, we may get to a zoom level where the features have spread so much that each tile gets only a few features per tile rendered efficiently. ​ That was one of the main obstacles to visualizing huge amounts of data. Today, we have put some techniques in place at CartoDB to circumvent this problem, all with a click - or an sql function if you are brave enough.

Take a look at the map below. We’ve imported NYC Taxi pick-up data (over 13 million rows) into CartoDB and styled it in a way that streets and buildings are almost unnecessary.

Now when you import a large dataset it will be analyzed to determine at which zoom level tiles can be densely packed with features. Then, for that level and all levels lower than its reduced version of the dataset, (‘overviews’) are constructed containing fewer features as to avoid overloading the tiler.

The way the number of features is reduced is by aggregating them into “overview-features” that represent many of the original features and have aggregated values for their attributes. This aggregation is done so that at most one feature will be located at each of the tile’s pixels, which both limits the maximum number of features the tile will have to deal with and also maintains
the original visual aspect of the tile (since all the discernible positions, in the tile are used).

The process is repeated for all smaller zoom levels, so that we have for each one a reduction of the data targeted to the level’s tile size. This allows the dataset to be represented, without substantial changes in its aspect, efficiently. Take a look at how easy it is to identify hotspots of data such as airports. ​

And, don’t worry about your columns. We’ve chosen to average numeric quantities over all the features that are represented in a single point. For text attributes the multiple values aggregated are shown only if there are only a few, otherwise there’s no efficient way to handle them, so an asterisk (*) is used as a placeholder to show that multiple different values of the attribute exists at the location. ​ On the other hand, sometimes numerical or textual attributes represent discrete categories, rather than a continuum of possible values. In this case, the aggregation methods mentioned are not useful. When building overviews the columns for which a few different values seem to exist are handled differently, and it is the ‘mode’ (most frequent value) which is stored as the aggregate value for the attribute.

Rendering and viewing all the features in a single tile from a large dataset no longer has to be a pain. With Data Overviews you can derive the necessary insights you need without compromising your data. This is a big step towards a platform where location intelligence can be done out of the box, no matter how much data needs to be handled.

Please don’t hesitate in contacting us if you have any questions or feedback about the data overviews feature.

Happy data mapping!

Next month CartoDB will be at Alteryx Inspire 2016 Conference, in San Diego, CA from June 6-9, as a platinum sponsor! Come visit booth #P2 to view our demos and learn how CartoDB can enhance your data workflows and analysis by providing even more value through location intelligence.

At Inspire, business leaders and data analysts from around the globe converge to share real world examples of how to get the most value from self-service data analytics.

Join us for a two hour workshop to see how Alteryx and CartoDB can expand the value of location data for a wider audience by making workflows simpler for business analysts while visualizing workflow outputs on beautifully dynamic maps.

You’ll also hear from Boston Consulting Group about using tools in ways that are best for clients, including how they’re using Alteryx with CartoDB to combine data blending into dynamic maps to quickly and iteratively find the right answers.

We hope to see you at Inspire 2016—Drop us a note if you want to hear more.

Happy data mapping and see you in San Diego!

From the very beginning, CartoDB has focused and researched how to make visual exploration of location data easier and more compelling. For most of that research, we rely on standard languages that are easy to use and learn, such as SQL and CartoCSS.

Now, we are taking CartoCSS a step further and adding new capabilities centered on data-driven styling. We’ve done this before with torque.js.

Introducing Turbo-Carto

Turbo-Carto is an open source CartoCSS pre-processor that enables functions to be added to CartoCSS that can be evaluated asynchronously. For example, Turbo-Carto allows you to create color and symbol size ramps with just a single line of code, so you don’t need to worry anymore about calculating the correct bins for your thematic map. It does it for you.

/* Creating a color ramp with Turbo Carto */marker-fill:ramp([your_column_name],colorbrewer(Greens));/* Changing symbol sizes with Turbo Carto */marker-width:ramp([your_column_name],4,18,6jenks));/* Where 4 is the minimum size, 18 the max size, 6, the number of buckets, and jenks
 the quantification method */

Not only does Turbo-Carto save you time when writing conditions on CartoCSS, but it also calculates the buckets for you and keeps you connected to the data. For example, if you filter a dataset previously styled with a color ramp, it will recalculate the bins depending on the filtered sample or even depending on the data that you are seeing in the bounding box. Analysts can leverage this with Deep Insights to find new outliers or interesting values on their maps, while keeping a statistically correct approach.

Starting today, Turbo-Carto is available through CartoDB.js and the Maps API and will be integrated soon in the CartoDB Editor.

Isn’t it cool? Take a look at the repo, give it a try, and let us know what you think.

Happy data mapping!

How we view, collect, and process data has gone through a revolutionary transformation. Roughly 45 years ago, the most widely disseminated image of the Earth was captured by NASA, The Blue Marble. Eight years ago, smartphones were invented – forever changing the way we take images and store and collect data. Now, companies like CartoDB are ushering in the age of data analysis.

Recently, our co-founder and CEO, Javier de la Torre sat down with Forbes Magazine and True Bridge to dicuss how almost all forms of data have a location component and how analyzing location data changes the way we act and react in the world.

In the first of a two-part series, Forbes explores how a new generation of tech consists entirely of data - imaging, aggregation, and analysis. In part-two, CartoDB, along with our friends at the World Resources Institute and Global Forest Watch, are highlighted for the use of location intelligence to make better business decisions and the world a better place.

Partnerships with organizations, like WRI and GFW, help broaden the understanding of location data, democratize access to that data, and help visualize the earth in a whole new way.

Javier and our friends at Forbes make it clear that imaging and location data are only going to become more relevant from here, and perhaps as common a resource for everyday decision making as a Google search.

Read more about how CartoDB is changing the game of data analysis and making location data a household name.

Happy location data mapping!

Now that you’ve mastered indoor mapping, let me introduce you to another great benefit of the mobile SDK. One of the key advantages of vector-basedmaps is that you can re-style them on the client side using just one single map source. This is especially useful for offline map package cases.

Basemap style file format

Since version 3.0, Nutiteq vector basemaps have used Mapnik XML as a style language. Mapnik XML defines what layers are visible at which zoom levels, what are colors, line styles, and text – basically, everything about how a map should look. It is a well-known language that has been used for years.

There are style editing tools, like Mapbox Studio Classic, that can be used to edit or generate the file from CartoCSS styling. Specifically, for the mobile SDK, we need XML and several additional assets as extra files – fonts and icons for POIs, and images for line patterns. We use style.xml as the main style name and bundle all assets to a single zip file and use this as a style package that is referred to in the code.

Some of the style assets, e.g. font files, can be big. Our default setting, with all the international characters (Chinese, Korean, etc.), is about 7MB, which is much larger than the whole SDK binary. If you have several similar styles in your app then you do not want to repeat them. Luckily, one style zip can include several style XMLs, which reuse similar assets and our SDK can read each style from it.

Dynamic style parameters

Generally, the loaded style is immutable. You’ll need to edit it in an XML file level and it cannot be changed in code. Sometimes this is not good because, you do not want to have very many similar styles just to change your map language. We have added one feature to Mapnik XML as our custom addition – style parameters.

These are specific variables in style that you can change in code. Usage of the variables must be predefined in XML, then the code can change their value. In our style we have pre-defined, for example, the following parameters:

• Map text language
• 3D building on/off
• 3D text orientation: Billboard or flat

Styling using CartoCSS

The mobile SDK, since 3.3.0, supports an additional map styling language–CartoCSS. This is much more compact, and is compatible with CartoDB’s general map styling tools.

In our downloads page you can find a package with 3 ready-made styles:
Grey, Bright (generic/universal), and Dark. These are compatible with our
nutiteq.osm-v2 online and offline data source. Read more technical details about map styling from our mobile developer site.

Happy mobile data mapping!

When visualizing, on a map, tens of thousands or millions of points in a dataset, data points may be spread all around the world, or be confined to a single country, region, city, or even some smaller area. In any case, there will be a zoom level in which the whole dataset is represented in a single 256x256 pixel tile. The “great mother of all tiles”, tile 0/0/0, will inevitably have to show the whole collection of points.

Rendering all the points in that tile is not only a tough task, but it isn’t optimal at all. Moving all that data will take quite some time, and drawing it will be a daunting task. If it does get completely painted, most of the work will be useless because you cannot possibly depict millions of features in a 256x256 pixel grid – many features will be stacked on top of others occluding them.

Of course, some children of that tile will also be packed with too-many features, so we will have similar problems at higher zoom levels. Eventually, we may get to a zoom level where the features have spread so much that each tile gets only a few features per tile rendered efficiently. ​ That was one of the main obstacles to visualizing huge amounts of data. Today, we have put some techniques in place at CartoDB to circumvent this problem, all with a click - or an sql function if you are brave enough.

Take a look at the map below. We’ve imported NYC Taxi pick-up data (over 13 million rows) into CartoDB and styled it in a way that streets and buildings are almost unnecessary.

Now when you import a large dataset it will be analyzed to determine at which zoom level tiles can be densely packed with features. Then, for that level and all levels lower than its reduced version of the dataset, (‘overviews’) are constructed containing fewer features as to avoid overloading the tiler.

The way the number of features is reduced is by aggregating them into “overview-features” that represent many of the original features and have aggregated values for their attributes. This aggregation is done so that at most one feature will be located at each of the tile’s pixels, which both limits the maximum number of features the tile will have to deal with and also maintains
the original visual aspect of the tile (since all the discernible positions, in the tile are used).

The process is repeated for all smaller zoom levels, so that we have for each one a reduction of the data targeted to the level’s tile size. This allows the dataset to be represented, without substantial changes in its aspect, efficiently. Take a look at how easy it is to identify hotspots of data such as airports. ​

And, don’t worry about your columns. We’ve chosen to average numeric quantities over all the features that are represented in a single point. For text attributes the multiple values aggregated are shown only if there are only a few, otherwise there’s no efficient way to handle them, so an asterisk (*) is used as a placeholder to show that multiple different values of the attribute exists at the location. ​ On the other hand, sometimes numerical or textual attributes represent discrete categories, rather than a continuum of possible values. In this case, the aggregation methods mentioned are not useful. When building overviews the columns for which a few different values seem to exist are handled differently, and it is the ‘mode’ (most frequent value) which is stored as the aggregate value for the attribute.

Rendering and viewing all the features in a single tile from a large dataset no longer has to be a pain. With Data Overviews you can derive the necessary insights you need without compromising your data. This is a big step towards a platform where location intelligence can be done out of the box, no matter how much data needs to be handled.

Please don’t hesitate in contacting us if you have any questions or feedback about the data overviews feature.

Happy data mapping!

Next month CartoDB will be at Alteryx Inspire 2016 Conference, in San Diego, CA from June 6-9, as a platinum sponsor! Come visit booth #P2 to view our demos and learn how CartoDB can enhance your data workflows and analysis by providing even more value through location intelligence.

At Inspire, business leaders and data analysts from around the globe converge to share real world examples of how to get the most value from self-service data analytics.

Join us for a two hour workshop to see how Alteryx and CartoDB can expand the value of location data for a wider audience by making workflows simpler for business analysts while visualizing workflow outputs on beautifully dynamic maps.

You’ll also hear from Boston Consulting Group about using tools in ways that are best for clients, including how they’re using Alteryx with CartoDB to combine data blending into dynamic maps to quickly and iteratively find the right answers.

We hope to see you at Inspire 2016—Drop us a note if you want to hear more.

Happy data mapping and see you in San Diego!

From the very beginning, CartoDB has focused and researched how to make visual exploration of location data easier and more compelling. For most of that research, we rely on standard languages that are easy to use and learn, such as SQL and CartoCSS.

Now, we are taking CartoCSS a step further and adding new capabilities centered on data-driven styling. We’ve done this before with torque.js.

Introducing Turbo-Carto

Turbo-Carto is an open source CartoCSS pre-processor that enables functions to be added to CartoCSS that can be evaluated asynchronously. For example, Turbo-Carto allows you to create color and symbol size ramps with just a single line of code, so you don’t need to worry anymore about calculating the correct bins for your thematic map. It does it for you.

/* Creating a color ramp with Turbo Carto */marker-fill:ramp([your_column_name],colorbrewer(Greens));/* Changing symbol sizes with Turbo Carto */marker-width:ramp([your_column_name],4,18,6jenks));/* Where 4 is the minimum size, 18 the max size, 6, the number of buckets, and jenks
 the quantification method */

Not only does Turbo-Carto save you time when writing conditions on CartoCSS, but it also calculates the buckets for you and keeps you connected to the data. For example, if you filter a dataset previously styled with a color ramp, it will recalculate the bins depending on the filtered sample or even depending on the data that you are seeing in the bounding box. Analysts can leverage this with Deep Insights to find new outliers or interesting values on their maps, while keeping a statistically correct approach.

Starting today, Turbo-Carto is available through CartoDB.js and the Maps API and will be integrated soon in the CartoDB Editor.

Isn’t it cool? Take a look at the repo, give it a try, and let us know what you think.

Happy data mapping!

How we view, collect, and process data has gone through a revolutionary transformation. Roughly 45 years ago, the most widely disseminated image of the Earth was captured by NASA, The Blue Marble. Eight years ago, smartphones were invented – forever changing the way we take images and store and collect data. Now, companies like CartoDB are ushering in the age of data analysis.

Recently, our co-founder and CEO, Javier de la Torre sat down with Forbes Magazine and True Bridge to dicuss how almost all forms of data have a location component and how analyzing location data changes the way we act and react in the world.

In the first of a two-part series, Forbes explores how a new generation of tech consists entirely of data - imaging, aggregation, and analysis. In part-two, CartoDB, along with our friends at the World Resources Institute and Global Forest Watch, are highlighted for the use of location intelligence to make better business decisions and the world a better place.

Partnerships with organizations, like WRI and GFW, help broaden the understanding of location data, democratize access to that data, and help visualize the earth in a whole new way.

Javier and our friends at Forbes make it clear that imaging and location data are only going to become more relevant from here, and perhaps as common a resource for everyday decision making as a Google search.

Read more about how CartoDB is changing the game of data analysis and making location data a household name.

Happy location data mapping!

Now that you’ve mastered indoor mapping, let me introduce you to another great benefit of the mobile SDK. One of the key advantages of vector-basedmaps is that you can re-style them on the client side using just one single map source. This is especially useful for offline map package cases.

Basemap style file format

Since version 3.0, Nutiteq vector basemaps have used Mapnik XML as a style language. Mapnik XML defines what layers are visible at which zoom levels, what are colors, line styles, and text – basically, everything about how a map should look. It is a well-known language that has been used for years.

There are style editing tools, like Mapbox Studio Classic, that can be used to edit or generate the file from CartoCSS styling. Specifically, for the mobile SDK, we need XML and several additional assets as extra files – fonts and icons for POIs, and images for line patterns. We use style.xml as the main style name and bundle all assets to a single zip file and use this as a style package that is referred to in the code.

Some of the style assets, e.g. font files, can be big. Our default setting, with all the international characters (Chinese, Korean, etc.), is about 7MB, which is much larger than the whole SDK binary. If you have several similar styles in your app then you do not want to repeat them. Luckily, one style zip can include several style XMLs, which reuse similar assets and our SDK can read each style from it.

Dynamic style parameters

Generally, the loaded style is immutable. You’ll need to edit it in an XML file level and it cannot be changed in code. Sometimes this is not good because, you do not want to have very many similar styles just to change your map language. We have added one feature to Mapnik XML as our custom addition – style parameters.

These are specific variables in style that you can change in code. Usage of the variables must be predefined in XML, then the code can change their value. In our style we have pre-defined, for example, the following parameters:

• Map text language
• 3D building on/off
• 3D text orientation: Billboard or flat

Styling using CartoCSS

The mobile SDK, since 3.3.0, supports an additional map styling language–CartoCSS. This is much more compact, and is compatible with CartoDB’s general map styling tools.

In our downloads page you can find a package with 3 ready-made styles:
Grey, Bright (generic/universal), and Dark. These are compatible with our
nutiteq.osm-v2 online and offline data source. Read more technical details about map styling from our mobile developer site.

Happy mobile data mapping!

CartoDB is constantly engaging with and supporting our community of mappers, developers, open data champions, development practitioners, students, educators, journalists, and innovative companies. We offer workshops and webinars to organizations, companies, and groups who want to learn how to harness the power of data by mastering our mapping platform.

In the past couple months we have given workshops to a diverse range of mappers, from Columbia University students, designers, architects, journalists at the Wall Street Journal, and Code For Newark (CFN). Recently, CartoDB sponsored The Staten Island Bus Hackathon, organized by the NYU Rudin Center, TransitCenter, and the MTA, as well as the HackNewark Hackathon. The results have been exceptionally positive!

To further foster CFN’s development and coding skills, CartoDB conducted a ground-up workshop covering the basic skills necessary to add a mapping component to an app the team was developing. Shortly after, CartoDB sponsored the HackNewark Hackathon held at Newark City Hall in New Jersey. Three teams participated in the hackathon and were given the challenge of addressing a social issue in Newark through the creation of an app.

Code for Newark created Newark Thrives!, an app which connects parents and their children to youth programs in their community. The app allows the user to filter programs by area of focus and also provides transportation information and other important location specific data. Ivan Quan, of Newark Thrives!, recently described his experience with CFN and leveraging technology to create change in his community.

“We all started participating in Code for Newark hack nights around the same time with very little experience in programming. Together we’re a cohort of average citizens. It’s really amazing nowadays the access we have to open source tools that can be customized to fit the needs of a local community. For us, this is proof that ordinary people can be a part of something that can directly improve the outcomes of their neighbors’ lives. This wouldn’t have been possible if it weren’t for the collaborative environment that Code for Newark offers. For us, this is just the beginning!”

CartoDB and our Community team is eager to teach our software to many more groups, companies, government agencies, and organizations, no matter where you are in the world. Please contact Tyler Bird tbird@cartodb.com and Santiago Giraldo santiago@cartodb.com to inquire about requesting, planning or teaming up on a workshop, webinar, or hackathon!

Alternatively, if you think you already have the skills needed to start carrying out insightful and impactful mapping projects, but need some additional support, please see our Grants for Good Program!

Happy Data Mapping!

When your organization has locations all over the country, or world, sometimes it’s hard to keep up with the progress of different targeted marketing campaigns. How you analyze all that data to determine the impact your organization is having (or not), makes all the difference in coordinating your next move.

CartoDB helps marketing and communications departments save time and money by adding location intelligence and spatial analytics to their campaigns, identifying where, when, and how a campaign has impact. Kroton, a private educational organization with over 100 campuses in the primary, secondary, and post-secondary segment, fused geographic analysis and marketing statistics to discover where their campaigns were most successful, as well as potential risk from competitors all across Brazil.

Using CartoDB’s location and data analysis platform, Kroton created visualizations to supplement their reports and represent hundreds of locations, as well as integrating CartoDB’s technology into their business intelligence dashboard. In addition to the basic display, Kroton added other miscellaneous information for geomarketing analysis, such as: school locations, competitors, population by census tract, income by census tract, economic profile of students, and distance between the student to the competitor.

Thanks to our capabilities and features, Kroton was able to focus on the creation of a customizable and real-time visualizations that stays updated and easily integrates into their existing platforms. Now, Kroton can see the impact of its campaigns and compare what works and doesn’t, for better, faster decisions.

Discover how Kroton used CartoDB to develop an easy-to-use and integratable data-driven analysis platform to perform geomarketing analytics:

Learn more!

CartoDB’s location intelligence enriches communications and marketing departments with deep insights from geographical and customer data.

We really hope to see more successful cases like this one!

Happy data mapping!

Today we are releasing a new CartoDB technology called the Data Observatory which gives our users access to some of the world’s most valuable location data.

The Data Observatory makes it easier than ever before to analyze and extract insights from your own location data by attaching data-driven context to each point location or polygon region. The technology is packed with hundreds of different measurements of the world and continues to grow each week. Now you can learn about your data’s underlying relationships with demographics, industries, housing, and other topics of interest.

CartoDB isn’t only about mapping, it just happens to do it very well. CartoDB is really all about extracting value from location data. Often, value comes from visualizing data on a map, but we find that the greatest value can come from exploring the underlying patterns within your data. Uncovering those patterns and performing the best location-driven analyses requires location-driven context–such as how many people exist at a location or the trends in house prices in an area. Data Observatory resources make it easy to gain context from your location data.

A few examples of the resources available in the Data Observatory include:

• Multiple resolutions of population segmentation
• Many scales and regions of boundary data
• Housing and rental price data

An example of how quick it is to create new data with the Data Observatory

Why is the Data Observatory exciting?

The Data Observatory has organized and blended many different data sources and then made them available for you to easily access through CartoDB. Let’s take a look a three big reasons why we think you’ll be excited about the Data Observatory.

1. Revealing secrets through location data

We believe your location data is keeping secrets from you. Behind every data point on a map are thousands of dimensions of data that can help you learn why that point exists in that location. You can discover those secrets if you know where to look. The Data Observatory is reducing the time and effort it normally takes people to uncover the hidden measures behind their location data. To do so, we’ve built search and discovery mechanisms that don’t exist anywhere else. For example, you can use your own location data to quickly search for relevant measures and then augment your own data with the results.

2. Making the world’s best data even better

The world is full of amazing location data, but it is pretty hard to use and even harder to get into the perfect form for your use-case. Wherever possible we are going the extra mile to make the Data Observatory useful for whatever needs you have. One example is the data licensing, which adopts the most open and flexible licenses possible, so you can feel safe using this data for your own business endeavors. Another example can be found in our boundary data, where we handle many of the complexities and even make variants (e.g. statistical boundaries versus boundaries suitable for cartography) available to fit different needs at different times.

3. Living, breathing data

The Data Observatory isn’t a single release that will grow old over time, it’s an evolving feature of CartoDB. The Data Observatory is kept up-to-date with the authoritative sources of data, and as a CartoDB user you will get access as quick as possible to new sources. We are also expanding the available data each week, moving into new themes of data and new regions of the world. We will regularly update you with changes through our newsletter and here on the blog. But you can always get the latest by visiting the Data Observatory documentation.

Get started

The Data Observatory is built right into CartoDB, which means you will be able to access tons of great data almost as though it is already in your account. The first release of the Data Observatory is available to Enterprise accounts through the CartoDB Editor and Platform and we are working to expand access over time.

If you are itching to get access to the Data Observatory right now, contact our team—we’d love to assist you.

get in touch with our team today

Also stay tuned to our blog and Twitter feed all week as we release interesting stories told through CartoDB and the Data Observatory.

Happy data mapping!

Recently, we announced our partnership with Mapzen. We are commercializing and integrating some of the great services they are building like routing, geocoding, and global basemap data services. As part of this collaboration, we’ll be converging our technology efforts, and today we are one step closer. We are pleased to announce the support of the Mapzen Vector Tile Service into our native Mobile SDK.

The Mapzen Vector Tile Service delivers worldwide coverage of OpenStreetMap base layer data with the latest updates. Vector rendering works great without having to worry about WebGL support on desktop browsers because it is optimized for the native Mobile SDK.

There are great advantages to rendering vector tiles on the SDK. For example, you can dynamically change the styles and rotate labels. The way you define the style of a map for the CartoDB Mobile SDK is, of course, by using CartoCSS, a great styling language we use across our entire platform.

We have also created an additional set of styles that work with the layers and structure of Mapzen Vector Tiles to enable its usage in our Mobile SDK. You can use these styles as an example to further customize your tiles. Our Mobile SDK allows for the dynamic changing of styles, so you can also do so programmatically.

Remember, various vector tiles usually have a different data structure inside. We have worked with Mapzen to ensure you have a set of styles ready to use with their service. Check out the latest style package - nutibright-v3.zip (from developer.nutiteq.com/downloads)! We’ve bundled the general “bright” map style variations for Mapzen Vector Tiles and our previous format.

This makes usage of Mapzen vector tiles quite simple. Just follow these three steps:

1. Load styling from nutibright-v3.zip, make sure you create MBVectorTileDecoder with “mapzen” as the style name parameter.
2. Use HTTP Datasource with Mapzen tile URL. Here is what you’ll find from Mapzen site: https://vector.mapzen.com/osm/all/{z}/{x}/{y}.mvt?api_key=vector-tiles-xxxxxxx
3. In this URL use your api_key from Mapzen developer site (you need to register there). When defining HTTP datasource, use 0 as min. zoom and 16 as max. zoom.

Code samples for it are in our sample apps for Android.

In order to support Mapzen Vector Tiles we had to made some adjustments on our core, so you need to use the latest Nutiteq Maps SDK 3.3.0 (by CartoDB) to use their tiles.

From a technology perspective this allows our SDK to make use of a great source of Vector Tiles, and gives developers the option to use CartoCSS to style them. CartoCSS on vector tiles, what else could you be looking for. ;)

Happy mobile data mapping!

As you might have already heard, the Data Observatory just launched to help provide CartoDB users with a universe of data. One of the reasons we built the Data Observatory is because getting the third-party data you need is oftentimes the hardest part of analyzing your own data. Data wrangling shouldn’t be such a big roadblock to mapping and analyzing your world.

Some of the most desirable and most used datasets are also the hardest to handle. Datasets like the US Census are rich with information but can be large and distributed. What most people want, though, is just a snapshot of some boundaries and particular measures. For instance, if you were telling a story about unemployment in a city, you might want to grab Census Tracts for that city, the number of unemployed people for each tract, and the number of people 16 and over for each tract (as the denominator). All of this is no easy feat given the size of the datasets provided by the US Census.

For today’s post, I want to look at how we can use CartoDB’s Data Observatory to answer questions when we don’t already have any data at all. Further, I want to only focus on just the region that I’m interested in, and avoid spending time working with data and geometries that are not used in my analysis.

Let’s explore San Francisco’s current housing price boom.

We’ll pull data for San Francisco’s real estate reality provided by Zillow and available in the Data Observatory. For the boundaries, we will work with the US Census generated ZIP code Tabulation Areas (ZCTA) boundaries, which correspond to ZIP codes for most purposes. To go beyond displaying one measure, we will pull the most recent median housing value per square foot and compare it to the same locations from three years ago to show the explosive growth in the region.

Getting Appropriate Boundaries

To get started we’ll need the boundaries that we’re interested in visualizing. Depending on the scale of your map (national, state, county) you will want different geometries. Looking at Zillow, they provide median home values in particular geographical regions, the most convenient of which is ZIP codes.

To find the available boundaries, we can use the function OBS_GetAvailableBoundaries which will list all of the IDs for boundaries available through the Data Observatory at the location we need them. Since we’re focused on San Francisco, we just pass the approximate location of the city and filter by descriptions which contain the word ‘zip’, like so:

SELECT*FROMOBS_GetAvailableBoundaries(CDB_LatLng(37.7749,-122.4194))WHEREdescriptionILIKE'%zip%'

The results look like this:

This will produce a temporary table of all boundaries we can use that intersect that point. We want the ZCTA boundary since we’re focused on ZIP codes. I’ll also opt for the water-clipped version since we don’t need to do any analysis with the geometries–we are only using them for visualization this time. The boundary_id that we need is: us.census.tiger.zcta5_clipped. You can see the data associated with these boundaries in this ID: they come from US Census Tiger geometries, are ZCTA boundaries, and are clipped (meaning water-clipped instead of statistical).

Now that we have our boundary ID, we can get our boundaries with the Data Observatory function OBS_GetBoundariesByPointAndRadius, which requires a center, a radius (in meters), and the boundary ID we’re interested in. I’ll populate an empty table called sf_housing which has a column for geometries (the_geom), and another for storing the text id of the geometry.

INSERTINTOsf_housing(the_geom,geoid)SELECT*FROMOBS_GetBoundariesByPointAndRadius(CDB_LatLng(37.7749,-122.4194),-- San Francisco, CA25000*1.609,-- 25 mile radius (= 25 km * conversion to miles)'us.census.tiger.zcta5_clipped'-- use water-clipped geometries for visualization)

Checking out our table, we’ll find it filled with all of the ZCTA boundaries in the region we’re interested in, but there’s no data for the geometries yet!

Getting Housing Value Data

Our next step is getting the housing price data for each of the ZIP code areas. The Data Observatory makes that possible through the wonderfully versatile OBS_GetMeasure. The reason that the Data Observatory splits these the work into these two pieces is so that you have complete flexibility about what data you blend and what data you ignore. With hundreds of measures and growing, this flexibility will keep your work easy.

To find the measures that would be of interest, we’ll use the OBS_Search function through CartoDB’s SQL tray, like so:

SELECT*FROMOBS_Search('zillow')

In CartoDB, it will look like this:

This gives back several IDs of interest, but let’s choose us.zillow.AllHomes_MedianValuePerSqft. This entry is just what it says, median home value per square foot for “All Homes” (i.e., single-family, condominium and co-operative homes). Let’s look at the most recent timestamp (March 2016).

We’ll put all of this information into OBS_GetMeasure and populate a new column with the results:

UPDATEsf_housingSETmedian_value_per_sqft_2016=OBS_GetMeasure(the_geom,-- specify the place'us.zillow.AllHomes_MedianValuePerSqft',-- specify the measure'area',-- specify normalization (this gives back raw measure)'us.census.tiger.zcta5',-- specifies level of geometries'2016-03'-- specifies when in time)

Visualizing this in CartoDB, we get the following map:

Now we can easily see the median housing value per square foot easily visualized on a map – and we didn’t have to track down the datasets to do so!

Calculating Historical Changes

Now that we have recent housing price data for San Francisco, let’s compare it to data from March, 2013 to look at the change in home value over time. To do this, we can calculate the current price divided by the historical price to get the ratio of change. Subtracting one from this ratio and multiplying by 100% gives us the percentage change of 2016 as compared to 2013.

We can accomplish this with the Data Observatory as follows (after creating a new numeric column):

UPDATEsf_housingSETpercent_change_median_value_2016_2013=100.0*(median_value_per_sqft_2016/OBS_GetMeasure(ST_PointOnSurface(the_geom),'us.zillow.AllHomes_MedianValuePerSqft','area','us.census.tiger.zcta5','2013-03')-1)

Now that we have our data, let’s map it! In under 5 minutes we have created a pretty cool map of housing value change in SF starting with an empty dataset!

We went from having no data to creating a value-added maps in under five minutes. And this is just the beginning of what can be done with the Data Observatory.

What’s next?

We hope you are starting to see the power of the Data Observatory. We see the Data Observatory as a source of enrichment and a way to make your analyses more powerful in CartoDB.

There’s so much more to explore, so watch our blog and Twitter for more. Also, read our documentation and checkout our catalog of available data to get started.

What maps and datasets will you make from an empty table? Share your “something from nothing” maps with us!

Happy data mapping!