Quantcast
Channel: CARTO Blog
Viewing all 820 articles
Browse latest View live

Gravity Models: Strategically Attracting Business

$
0
0
Gravity Models

As part of our initiative to provide strategic solutions for business, CartoDB is thrilled to announce Gravity Models—a function that users can add to their analysis–that predicts the amount of people who will come to your location based on the presence of competing locations.

The first Gravity Model, for retail analysis, is based on Reilly’s Economic Law of Retail Gravitation, which suggests that customers travel longer distances to larger retail centers the higher the level of attraction the shopping center has. Attractiveness, is analogous to size (mass) in the physical law of gravity. Based on Reilly’s Law, businesses can make informed decisions about their next move. Imagine, for example, that one of your competitors has a store in city X, and that this business enjoys a position of market dominance in the region. If your company opens its own store in neighboring city Y, will this new competition attract enough traffic away from city X to turn a profit?

Another approach that can be applied is Huff’s Law, which takes multiple attributes and retail centers into consideration. This model’s approach to market penetration assumes that there is a spatial variation in the proportion of households served by a store due to competition. The trade area is conceptualized as a probability surface that represents the likelihood of customer patronage.

The idea is to create a probability surface that is based on a spatial interaction model that includes variables such as distance, attractiveness, and competition. The probability surface can be contoured to produce regions of patronage probability, which can then be further used as weights in the preparation of a market profile.

In this visualization you can see various mall locales in Madrid. Simply choose a mall from the list and begin your analysis! We’ve selected Xanadú shopping center, in this example, to determine how many customers are attracted to shopping in this particular mall and their distribution.

The dashboard histogram widgets update in real-time to reflect population density, potential customers, probability of patronage, distance to mall, and average age. The color ramp is dynamic, which gives you a color scale and range that best reflects your insights. Simply adjust the probability of patronage histogram to accurately reflect your minimum and maximum values denoting the meaning of your color choice.

For example, if you adjust the probability of patronage histogram to reflect approximately 25-50%, you’ll notice that the Xanadú shopping center’s surrounding areas remain a deep purple indicating their high probability of patronage. The surrounding regions that are yellow have the lowest probability of mall visits. You’ll also notice that the total population is a little over 19,000 with more than 6,600 potential customers, with most of the consumers located directly around the shopping center.

Now imagine you are a commercial real-estate speculator and you want to determine where your next mall should be, how big you should make it, and the best ROI in terms of attracting people. Select the new mall function on the right and and then select a location on the map of where you’d like your new mall to be, the cursor places a dot (colored based on the size of your mall). You can adjust the attributes in your dashboard to reflect how many square meters you’d like to make your new shopping center and start performing your strategic analysis. You can also drag your dot to a different location and begin recalculating and adjusting variables.

The function behind all the color craziness is CDB_Gravity, which you can find in this repo. This is a simplified version of Huff’s Law that can be used with different parameters, such as weights or population.

Gravity Models is a powerful way to make decisions and solve problems that can be applied to various sectors including migration trends, trade flows, transportation demand forecasting, commercial and tourism attractions, and trade areas, and CartoDB is already offering the solution!

Happy data mapping!


Headed to the Esri UC? Come get the scoop on the latest and greatest with our CEO & the CartoDB team!

$
0
0
ESRI UC

It’s that time of year again! As hoards of geospatial professionals migrate to the Esri User Conference in San Diego this year, we at CartoDB are getting ready once again to celebrate the geospatial industry and foster a culture of location analysis that looks towards the future. The truth is, we’ve been so focused on creating new technology, creating strong open source alliances, and even reframing how we access and use data that this year’s time in San Diego will be a time to share innovation and build strong relationships.

Our very own CEO, Javier de la Torre, accompanied by some of CartoDB’s other brilliant minds will be dropping knowledge and showing off some of our latest technology personally, while we beat the heat in style poolside of course, at the Marriott Marquis San Diego Marina. From Sunday June 26th, through Thursday June 30st, we’ll be available for meetings with open arms (and margaritas) to embrace even the weariest Esri UC traveler. We’re really looking forward to seeing you there, so if you want to meet, drop us a line at asimcox@cartodb.com.

Let’s celebrate you, the geospatial community

What’s better than a party on Tuesday or Wednesday? A celebration on Monday! Start the week off right (and hopefully without a hangover) with CartoDB’s Celebration Happy Hour of All Things Geospatial! Celebrating is about sharing moments with the people that matter most. We’re bringing the festivities to you, right next to the convention center at the Hilton San Diego Bayfront, Fox News Grill. Come enjoy great company, conversation, and drinks on us in an intimate and fun open outdoor setting with gorgeous views of the bay. This event is for the Esri UC community and to show how much we love you – Good times guaranteed.

Here’s the deets:

  • When:
    Monday, June 27th, 2016
    7:00PM-10:00PM
  • Where:
    Hilton San Diego Bayfront, Fox Sports Grill
    1 Park Blvd
    San Diego, CA 92101
  • RSVPhere!

The truth doesn’t have to hurt.

Truth is, the geospatial community has always been near and dear to our hearts. They are in fact, one of the main reasons we relentlessly strive to bring excellence to spatial practices through open source projects and collaborations that foster a culture of accessibility and innovation. We even push to break down barriers for our users with easy to use tools such as our ArcGIS™ Connector. We do it because you deserve it, and because together we can make strides towards a better, brighter location technology future.

Happy data mapping!

Large Number of Points on Mobile Map

$
0
0
Large number of points on mobile map

Lately we’ve been talking a lot about vector maps and how to better utilize the capabilities of our Mobile SDK when rendering your maps for your devices.

A question that is often asked is, how can you add thousands or even millions of points to your map. Visualizing this many data points can pose some specific challenges. For example, if you add them to a VectorLayer, via in-memory LocalVectorDataSource, the results can appear like this:

vector layer points

There are several issues with this technique. You’ll notice the visual overloading of the map, in addition to technical memory and speed issues. You may even end up with crashing and “Out-of-Memory” situations.

Fortunately, we have not just one but several ways to resolve these challenges!

  1. Show a small number of POIS and display only relevant categories.
  2. Use point objects, instead of marker or balloon pop-ups, if possible. Marker is best for map tilting (2.5D view) because the markers are shown as billboards. However, point is much faster and memory-efficient because it does not have the additional necessary logic.
  3. Optimize marker bitmap sizes by making them as small as you can.
  4. Use in-storage or an online data source instead of the in-memory data source, LocalVectorDataSource. This way, you load only visible objects to memory. Our Mobile SDK standard package has an interface VectorDataSource, which can be implemented by application. You can use your own database (e.g. SQLite) or an on-line API to load, and try not to load everything to memory at once.
  5. Upload your data to your cartodb.com account and request data using vector or raster tiles, torque, or our special CartoDBSQLDataSource. We have a ready-made online data source using the CartoDB platform! See our earlier blog post on how to consume CartoDB via the SDK.
  6. Use vector tiles for the most effective big data rendering and offline usage. Vector tiles can be generated by CartoDB. For offline use you’ll need to extract the tiles into an MBTiles file and use it with MBTtilesTileDataSource.
  7. Use the Mobile SDK GIS Extension, which provides ready-made in-storage data sources for ESRI Shape, KML, and many other geodata file formats. For very big datasets you may still need to use other tips from this list, e.g. pre-convert data to vector tile mbtiles.
  8. Use point or marker clusters to reduce visual overload of markers. Our Mobile SDK provides a special interface and implementation.Note: this requires loading all the data to memory, because clusters are created dynamically. This works best with medium sized datasets (>50,000 objects).

P.S. These and many other Mobile SDK features will be shown in a webinar on June 29. RSVP today!

Happy mobile data mapping!

Celebrating Gay Pride with CartoDB!

$
0
0

Last Saturday night the largest mass shooting in U.S. history happened in an LGBTQ nightclub in Orlando – an act of hate killing 50 and injuring 50 more.
It was an act of hate towards gays that has brought the community together, once again, to show their strength in numbers. We want to offer deep condolences to the families and friends of the victims and show our support for LGBTQ people everywhere.

But it is also time to celebrate, with it being Gay Pride month and all. LGBTQ people around the world have great reason to be together this year as an act of solidarity and CartoDB wants to help you find the celebrations happening.

What’s amazing about this map is to remember the history of Gay Pride celebrations. LGBTQ pride has its roots in New York City. In 1970, a few, bold people stood up and marched for rights. They walked down Christopher Street, through the city streets to reinforce the bold activism of the Stonewall riots exactly one year prior.

Forty-six years later, we have this map– an inventory of worldwide celebrations of Pride and of a struggle that is making bold progress year-after-year.

We’ve got some more maps planned for next week as NYC Pride rolls out, so stay tuned!

Happy data mapping!

3 ways that the Data Observatory will make your data analysis better

$
0
0

If you follow us on Twitter at all you have probably seen our announcement of the Data Observatory. We are excited about the release and also excited about what it brings to the CartoDB community. While the Data Observatory looks like it is all about data, that’s the tip of the iceberg. Like we talked about previously, The Data Observatory is about prediction and all spatial data analysis for that matter. The reason the Data Observatory is important for CartoDB and the future of our platform is that the data we are publishing underpins exciting spatial analysis you will be able to use in the near future.

Let’s look at a few ways that the Data Observatory can make data analysis better!

1. Predict unknown possibilities from your known data

We’ve been spending a lot of time lately researching ways to predict future scenarios from data. The analyses we are working with all rely on special and powerful properties only contained in location data. The Data Observatory is where we’ll make that much of that underlying data available.

2. Find similar populations across huge geographic space

Many of the measures contained within the Data Observatory are available at high-resolutions over large areas. So you can know the same thing about a population in New York as a population across the country in San Diego. Using the property of sameness allows clients to build models to find new clients or new populations they are trying to reach. We hope soon you will all be able to access these methods.

3. Spend less time fighting with data and more time learning its secrets

One thing we’ve heard again and again before the Data Observatory was that getting many of these high-value datasets is hard. Many of our clients have shared this struggle. The Data Observatory is the first step toward making that process easier than ever before. There are many different ways you can access the Data Observatory, from accessing single measures at a point location to pulling entire sets of boundaries based on a bounding box. If you’ve ever fought with data before, you are going to love the Data Observatory.

Happy data mapping!

CartoDB wants to Congratulate the #DJA 2016 Winners

$
0
0

CartoDB is always happy to support journalists and researchers in their efforts to tell stories through data analysis and visualization. In the past we’ve hosted events like, GeoJourNews for journalists and new media to showcase their work, share lessons learned, and discuss new trends in data journalism.

Last week, the Global Editors Network, a community of top editors and innovators, celebrated data journalism at the fifth Data Journalism Awards. The competition rewards organizations for producing distinguished and outstanding work in the field of journalism, worldwide. With over 450 projects submitted from 50 countries, this is an exceptional opportunity to share ideas and find out new ways to solve problems through data journalism.

We want to congratulate all the participants for their hard work and are also very proud to see some of the winners using CartoDB maps in their articles!

BuzzFeed employed our technology to show how agents of the FBI and the Department of Homeland Security (DHS) monitor America from the sky.

'America from the sky

The infamous Panama Papers ended up among the winners with an amazing project about the politicians, criminals, and the rogue industry that hides their cash, which was spread around the world on social media and news outlets with different maps like this one:

Al Jazeera America did an excellent job on this map to report the train accident on May 8th traveling from Washington D.C. to New York’s Penn Station.

This competition is proof that journalism and new media can leverage data and location intelligence with powerful tools like CartoDB.

We really hope to see more success stories like these ones!

Happy data mapping!

Gravity Models: Predicting Attraction by Population Size

$
0
0
Gravity Models

In our previous blog post we introduced Gravity Models, a useful function that allows businesses and organizations to choose the location of their next store or mall. This time, we are going to apply Huff’s Law to populated areas to study their attraction based on population as an extension to the gravity model of migration.

According to this law, the probability of patronage to a specific business is dependent on the distance from the patron’s home to that business, the attractiveness of that business based on the size of the surrounding population, and adjusted by the presence of other potential businesses in the neighborhood.

When tasked with selecting a new location for your next store or business you can use these models as a way to predict potential success for profit and popularity. For example, in this visualization, by selecting Zaragoza, from the drop down list of cities in the top right selector, and zooming in, you’ll notice several factors:

  • Placement: Managers typically place their businesses in locations with more than 20,000 inhabitants. In the case of Zaragoza, there is a potential customer count of ~1 million. If these results align with the project’s requirements, you can adjust attributes and make an even more strategic decision based on the analysis disccussed in the previous post.
  • Traffic forecast: If you narrow the probability range to above 90%, the tool renders the populated places where people are willing to commute to Zaragoza to shop. By checking the top places, you can decide where to add new bus lines.
  • Understanding the demand for public services requires forecast analysis: In order to scale up the current public service provision in Zaragoza, you can have an initial estimate of the potential number of citizens that will use these services in upcoming years: ~1 million vs. the current 680,000 inhabitants.

Gravity Models make decision making processes easy when location is key to your businesses success. Stay tuned for our next installment on Gravity Models and how you can optimize your strategic location-based business decisions.

Happy data mapping!

The Mobile SDK Powers a Popular Traffic Jam Navigation App in Japan

$
0
0
SKD powers a Popular Traffic Jam Navigation App

Flictec, Inc., is a Japanese mobile application development company. They have been avid users of our Mobile SDK and the CartoDB platform for many years. One of the solutions they provide is an advanced traffic and navigation application 渋滞ナビ or Jyutai Navi (Japanese for traffic jam navigator), with millions of happy end-users you can call it the “Waze of Japan.”

How it works:

Probe data is collected from users smartphones and recorded into CartoDB, so they can get traffic insights by spatially analyzing probe data with road networks. Flictec has been collecting data since 2013 and the total number of collected records has reached more than 10 billion. Currently, they add 30 million records per day that span across Japan. Flictec also keeps all of its data and uses it to predict future traffic jams. Now, they have 8TB in CartoDB, making it one of the largest CartoDB single database instances. To avoid slowing they do partitioning every 12 hours and use additional databases for map-matching analysis.

Here is real time probe data from Jun 13, 2016 at 8 a.m. to 9 a.m., as recorded in CartoDB, for the center of Tokyo:

Real Data, center of Tokyo

Each data point indicates location, speed, and bearing. Using CartoDB, they’ve plotted the very slowest cars as red, slower cars as orange, faster as blue, and the fastest as green.

Jyutai Navi provides the following key features:

  1. A traffic information map (powered by our Mobile SDK)
    • All data is saved in CartoDB on the cloud
    • Traffic layers are drawn using CartoDB Maps API
    • POI and labels are taken from the server through the CartoDB SQL API
  2. Car navigation
    • Routes are calculated on the server. Their routing engine considers traffic situations and finds the fastest route
    • Route lines are drawn on the map as a layer of the Mobile SDK
    • The app uses 6 layers at the same time and loads a lot of objects but our SDK is still really fast!

This is a very exciting use case for CartoDB! Seeing a customer merge our web and mobile technologies, long before we were united, and really stretch our technical limits in several ways, is very inspiring to us. We wish them the very best in their solution and will keep looking for other cases like this!

どうもありがとうございました

P.S. These and many other Mobile SDK features will be shown in a webinar next Wednesday, so don’t forget to register!

Happy mobile data mapping!


Connecting ESRI and CartoDB Together Now for Everyone

$
0
0

Next week we will be in San Diego for the ESRI User Conference. To get the week started right, we are booking meetings to show off our latest technology and hosting a Happy Hour on Monday, June 27th at 7:00 pm right next door to the convention center at the Fox Sports San Diego Grill. We hope to see you there!

Global Forest Watch

Many organizations are using ESRI technology and CartoDB together to create the best geospatial applications. One great example we are particularly proud of is the Global Forest Watch Initiative developed by Vizzuality, where ESRI and CartoDB are both partners. GIS specialist make use of ESRI software to collect and produce spatial data that is shared on an ArcGIS Server. From there CartoDB syncs and caches the data to provide the application development framework for their website. With technologies like Torque, CartoDB provides a great capability to build the best User Experience for understanding deforestation.

This is a great way to see how CartoDB and ESRI technologies are already working together in complex and diverse environments, but this is just one way to connect the two technologies. Let’s look at different workflows and if you have any questions let us know.

Synchronizing ArcGIS Server with CartoDB

If you are using an Enterprise GIS, and have an ArcGIS Server, you can leverage CartoDB with our ArcGIS™ Connector. This connector, enabled in your CartoDB account, allows you to connect to public ArcGIS server services, and import them into CartoDB. The connector accesses the services through the REST Endpoint that ArcGIS Server provides out of the box. Our connector can ingest both Map and Feature services that are hosted on ArcGIS Server.

The ArcGIS connector for CartoDB was only available for enterprise plans. Starting today it will be available for all CartoDB plans, including Free Plans.

Share on Twitter

Using the Connector is as simple as Copy and Paste. You simply go the the REST Endpoint URL like this one, and copy the layer URL that you want to ingest into the box below.

ArcGIS import

Once imported, the data can be used for your work in CartoDB. Also, the connection between ArcGIS Server and CartoDB is dynamic, meaning that when your services add new data, remove data, or change attributes from your geodatabase, the table in CartoDB updates as well. You can choose to automatically update the datasource from every hour, or do it manually every 15 minutes. This provides a great synchronization mechanism where CartoDB federates to ArcGIS for content.

ArcGIS import

Click here to find out more about the Connector for ArcGIS.

Using CartoDB OSM basemaps in ArcGIS Desktop, ArcGIS Server or AGOL

CartoDB offers a number of free basemaps to use for creating your own maps. We provide specially designed basemaps for data visualization with tones that make your data pop. We offer them in raster format and now vector format through our partnership with Mapzen.

They are free to use with almost no limitations, but if you need commercial terms we have them available. You can use them directly as “Tile Layer” with a standard URL format or you can connect to them as WMTS Server.

The WMTS Capabilities document can be found here.

On ArcGIS Online for example add them like this:

ArcGIS online

One great capability of these basemaps is that they can be used to make label sandwiches! So your labels always stay on top of the data.

Publishing maps from ArcGIS Desktop to CartoDB

If you are using ArcGIS Desktop to analyze and manage geospatial data, you can upload that data into CartoDB for free using a great tool from GKudos. This tool leverages our SQL API in conjunction with Python to load data from ArcGIS Desktop into CartoDB. The tool is available as a toolbox in ArcMap and ArcCatalog:

Import Data

Here is a link to the ArcGIS Desktop Connector to CartoDB from GKudos.

Why use CartoDB together with Esri?

Now that you have learned the different ways to make Esri and CartoDB work together, you might wonder why people do it in the first place. The answer is because they can. We love Open Standards and Open products that can interoperate! These are some aspects of integration that users take into account:

Cost-effective for data that updates frequently

CartoDB’s dynamic capabilities for generating maps represent a cost effective solution to organizations that need to publish data that changes quickly. Because CartoDB maps render dynamically and automatically cache on demand, we do not charge for the number of updates. That can provide very significant savings in situations where data changes every day, hour or minute!

However with ArcGIS Online you get charged by number of tiles generated. In the Esri model you generate tiles every time you make a change. Therefore because tiling the entire map is expensive, they charge you based on how many you create, and that increases costs quickly when your data changes often.

Credits/Map Views on web applications can be scary

One issue with usage based pricing is that often you don’t know how many times you are going to use the system or apps to create or modify your maps. Additionally, you don’t know how popular your maps or applications are going to be. With a credit system your price can skyrocket before you know it.

That’s why we have simple predictable prices that let you know how much you are going to pay. And we do not penalize you based on your success! We want your maps to be as popular as possible. This why there are no credits or mapviews limits on our service.

Simple APIs for developers

Our goal with the CartoDB platform is to enable anyone to develop spatial applications. We believe that by using technologies like CartoCSS and SQL we make geospatial technology accessible to many more developers. By adhering to web development standards means faster development times and improved time to value with an overall better user experience of applications.

Lock-in free. Open Source

CartoDB source code is Open Source. That means the same code we run on our cloud service, you can get it and run it on your own servers. We have extra modules to increase performance and security on our cloud, but the software is fundamentally the same. Why is this important for our customers? It provides trust. You can develop on top of CartoDB knowing that you will have the freedom to run your own service if you want or need to. It provides a NO lock-in policy that many organizations appreciate. If we are providing a great service you will still prefer us for running your servers as we can provide a more cost effective solution and better quality.

Join us for the CartoDB happy hour on Monday at ESRI User Conference!

We mentioned it above, but it’s worth mentioning again. We are having a Happy hour at the Fox Sports San Diego Grill, 2 minutes away from the San Diego Convention Center. RSVP here!

Fox Sports San Diego Grill

We’re very excited to celebrate the geo community. It’s going to be a great time full of drinks, food and fun. We hope to see everyone there!

Happy data mapping!


All product names, logos, and brands are property of their respective owners. All company, product and service names used in this website are for identification purposes only. Use of these names, logos, and brands does not imply endorsement.

ESRI, ArcGIS, ArcGIS Online, and ArcGIS Server are registered trademarks of the Environmental Systems Research Institute Inc.

Normalize All the Things with the Data Observatory

$
0
0

The data we collect about the world is rarely as clean as we would like it to be. Often times data is noisy or incomplete and because of this we draw false conclusions from our data.

One of the biggest sources of potential error in geodata comes from not properly normalizing our data. There are a plethora of maps out there that claim to show some notable effect a population has on something, or the effect any given person, place or thing may have on a population. However, in reality these maps might just show us where a large number of people are. One of the worst offenders of these kinds of maps are those made with social media data.

In this post we will show you how you can use the Data Observatory to minimize the influence of population on your maps to let the true insight shine through.

Twitter

Let’s grab a dataset using CartoDB’s Twitter Connector and pick something we know is geographically distributed. Let’s say we are trying to analyze a social media campaign that the restaurant Chick-fil-A is running. We collect tweets from the data and produce the following twitter map:

We are not able to derive much insight from this map. There are too many points on top of each other. Let’s try aggregating this to counties and produce a choropleth of the number of tweets in each county.

We can use the Data Observatory to do this in one shot using the following SQL:

WITHcountiesAS(SELECT*FROMOBS_GetBoundariesByGeometry(ST_MakeEnvelope(-130.61,21.21,-64.16,54.62,4326),'us.census.tiger.county'))SELECTcount(*)AStweet_count,counties.the_geomASthe_geomWHEREST_Within(twitter_chickgila.the_geom,counties.the_geom)GROUPBYcounties.the_geom

Great! Now we have a better view of which counties are tweeting about Chick-fil-A more than others, but it is still not a particularly useful metric. Let’s take Albany County, for example, which has a population of around 304,000 people, and compare it to Allegany County, which has a population of about 50,000. We shouldn’t be surprised if we see more tweets in Albany because there are a lot more people there to tweet.

We really want the number of tweets per person in each county and not the number of tweets per county. With the Data Observatory we can easily update our table to show a cholorpleth map normalized by the total number of people in each county:

SELECTthe_geom,tweet_count/(ST_Area(the_geom::geography)*1000.0*1000.0*OBS_GetUsCensusMeasure(ST_PointOnSurface(the_geom),'Total Population','Area','us.census.tiger.county
      ) as count_per_person
  FROM twitter_chickfila_choropleth

Applying a normalization query gives us a much better view of where our social media campaign is having an effect.

Normalizing other things

The example above shows how important it is to properly normalize your data for maximum insight. Geodata combined with the Data Observatory makes this really easy, and with the large number of measures available in the catalogue you should be able to normalize most datasets. For example, if you had a dataset of the amount of money people spend on their pets in each county, a map showing the percentage of an individual’s income spent on their pets. No problem! All we need to do is normalize by the median income in each of those regions.

Normalizing your data and your maps will help you create projects that are anything but normal! We can’t wait to see the many different ways you will apply this strategy to your maps!

Happy (normalized) data mapping!

New Academy Lesson: How to Choose Map Colors

$
0
0
Analagous and Complementary Colors

Do you want to go beyond using default color schemes for your map, but don’t know where to start? Or do you need to make your map more legible, but aren’t sure how? We’ll show you!

Learn how to make beautiful maps that tell a clear story, like the one above created by LifeWatch INBO. Check out our new Map Academy Intermediate Design lesson: How to Choose Map Colors, Part 1.

We explain the basics of color theory, and give you specific guidelines for picking colors. That includes designing for your color-blind audience members! Then we show you many different ways to implement your palette with CartoCSS. We’ll also help you avoid inflicting effects like this on your audience.

This lesson is the latest in our Intermediate Design course. It’s the first of a two-part series on working with color. The second part is coming soon! It’ll show you how to apply the skills you learned in Part 1 to create complete color palettes. It will also show you which palette type is best for the specific kind of data you have.

We hope you find the new lesson useful. Share your color palettes with us in the lesson comments! If you have feedback on this lesson or want to suggest another, please let us know at support@cartodb.com.

Happy data mapping!

Helping the eDNA project to Keep Montana's Rivers Free of Invasive Species

$
0
0

There is a growing concern that Montana’s waterways are at risk of new invasive aquatic species, which are known to spread rapidly through US rivers and lakes, negatively affecting water quality and altering ecosystems and natural habitats.

Researchers and investigators have been dedicating more attention to this issue and they are now using new tools to better understand the level of impact.

As we can see in this CartoDB visualization made by Ken Rand, some of these invasive species have already made their way to Montana:

To avoid these aquatic species from spreading to new areas and reproducing in Montana’s canals, Luikart decided to leverage CartoDB’s location intelligence and find out where the most affected areas are so that he can make better decisions on how to respond.

In order to achieve this goal, Luikart and his team took eDNA analysis from waterways, simplified the inspection and made early detection of invasive species easier.

Also, by using CartoDB’s platform they are tracking the evolution of each specie from 1902 up until recent years to fully understand the reality of the situation in the US.

Edna project

If you are interested in discovering more about what CartoDB’s geospatial data analysisis able to achieve, have a look at this page.

Happy data mapping!

The Data Observatory's reach expands to England & Wales

$
0
0


We are proud to announce the availability of demographic data for England and Wales from the Office for National Statistics in the United Kingdom.

With much focus right now on the Brexit results and what they will mean to the people, we felt it was an important time to make this data available in the Data Observatory. The release represents some of the richest demographic data available in the area and it is now easy to access and blend with your own data.

In England and Wales, the Data Observatory now includes:

  • Population and general demographics (age, gender, etc.)
  • Occupation and economic involvement
  • (In Wales) health characteristics
  • Languages spoken
  • Race & ethnicity
  • Religion

We’ve already had the opportunity to use the release to enrich data in some of our analysis projects. Take a look.

Twitter anticipates Brexit

In this dashboard, it’s possible to see which segments of the UK population tweeted for or against Brexit. Segments are based off the same demographic information now in the Observatory:

Mapping Brexit results

In this dashboard, the results of the Brexit vote can be drilled down alongside a subset of demograhpic characteristics:

Discover how the Data Observatory enhances your location-data through advanced analysis methods in our recorded webinar. Watch it as many times as you need to!

Happy data mapping!

CartoDB wants to Congratulate the #DJA 2016 Winners

$
0
0

CartoDB is always happy to support journalists and researchers in their efforts to tell stories through data analysis and visualization. In the past we’ve hosted events like, GeoJourNews for journalists and new media to showcase their work, share lessons learned, and discuss new trends in data journalism.

Last week, the Global Editors Network, a community of top editors and innovators, celebrated data journalism at the fifth Data Journalism Awards. The competition rewards organizations for producing distinguished and outstanding work in the field of journalism, worldwide. With over 450 projects submitted from 50 countries, this is an exceptional opportunity to share ideas and find out new ways to solve problems through data journalism.

We want to congratulate all the participants for their hard work and are also very proud to see some of the winners using CartoDB maps in their articles!

BuzzFeed employed our technology to show how agents of the FBI and the Department of Homeland Security (DHS) monitor America from the sky.

'America from the sky

The infamous Panama Papers ended up among the winners with an amazing project about the politicians, criminals, and the rogue industry that hides their cash, which was spread around the world on social media and news outlets with different maps like this one:

Al Jazeera America did an excellent job on this map to report the train accident on May 8th traveling from Washington D.C. to New York’s Penn Station.

This competition is proof that journalism and new media can leverage data and location intelligence with powerful tools like CartoDB.

We really hope to see more success stories like these ones!

Happy data mapping!

Gravity Models: Predicting Attraction by Population Size

$
0
0
Gravity Models

In our previous blog post we introduced Gravity Models, a useful function that allows businesses and organizations to choose the location of their next store or mall. This time, we are going to apply Huff’s Law to populated areas to study their attraction based on population as an extension to the gravity model of migration.

According to this law, the probability of patronage to a specific business is dependent on the distance from the patron’s home to that business, the attractiveness of that business based on the size of the surrounding population, and adjusted by the presence of other potential businesses in the neighborhood.

When tasked with selecting a new location for your next store or business you can use these models as a way to predict potential success for profit and popularity. For example, in this visualization, by selecting Zaragoza, from the drop down list of cities in the top right selector, and zooming in, you’ll notice several factors:

  • Placement: Managers typically place their businesses in locations with more than 20,000 inhabitants. In the case of Zaragoza, there is a potential customer count of ~1 million. If these results align with the project’s requirements, you can adjust attributes and make an even more strategic decision based on the analysis disccussed in the previous post.
  • Traffic forecast: If you narrow the probability range to above 90%, the tool renders the populated places where people are willing to commute to Zaragoza to shop. By checking the top places, you can decide where to add new bus lines.
  • Understanding the demand for public services requires forecast analysis: In order to scale up the current public service provision in Zaragoza, you can have an initial estimate of the potential number of citizens that will use these services in upcoming years: ~1 million vs. the current 680,000 inhabitants.

Gravity Models make decision making processes easy when location is key to your businesses success. Stay tuned for our next installment on Gravity Models and how you can optimize your strategic location-based business decisions.

Happy data mapping!


The Mobile SDK Powers a Popular Traffic Jam Navigation App in Japan

$
0
0
SKD powers a Popular Traffic Jam Navigation App

Flictec, Inc., is a Japanese mobile application development company. They have been avid users of our Mobile SDK and the CartoDB platform for many years. One of the solutions they provide is an advanced traffic and navigation application 渋滞ナビ or Jyutai Navi (Japanese for traffic jam navigator), with millions of happy end-users you can call it the “Waze of Japan.”

How it works:

Probe data is collected from users smartphones and recorded into CartoDB, so they can get traffic insights by spatially analyzing probe data with road networks. Flictec has been collecting data since 2013 and the total number of collected records has reached more than 10 billion. Currently, they add 30 million records per day that span across Japan. Flictec also keeps all of its data and uses it to predict future traffic jams. Now, they have 8TB in CartoDB, making it one of the largest CartoDB single database instances. To avoid slowing they do partitioning every 12 hours and use additional databases for map-matching analysis.

Here is real time probe data from Jun 13, 2016 at 8 a.m. to 9 a.m., as recorded in CartoDB, for the center of Tokyo:

Real Data, center of Tokyo

Each data point indicates location, speed, and bearing. Using CartoDB, they’ve plotted the very slowest cars as red, slower cars as orange, faster as blue, and the fastest as green.

Jyutai Navi provides the following key features:

  1. A traffic information map (powered by our Mobile SDK)
    • All data is saved in CartoDB on the cloud
    • Traffic layers are drawn using CartoDB Maps API
    • POI and labels are taken from the server through the CartoDB SQL API
  2. Car navigation
    • Routes are calculated on the server. Their routing engine considers traffic situations and finds the fastest route
    • Route lines are drawn on the map as a layer of the Mobile SDK
    • The app uses 6 layers at the same time and loads a lot of objects but our SDK is still really fast!

This is a very exciting use case for CartoDB! Seeing a customer merge our web and mobile technologies, long before we were united, and really stretch our technical limits in several ways, is very inspiring to us. We wish them the very best in their solution and will keep looking for other cases like this!

どうもありがとうございました

P.S. These and many other Mobile SDK features will be shown in a webinar next Wednesday, so don’t forget to register!

Happy mobile data mapping!

Connecting Esri and CartoDB Now Together for Everyone

$
0
0

We will be in San Diego next week for the 2016 Esri User Conference. To hit the ground running, we are scheduling meetings to show off our latest technology, as well as hosting a Happy Hour on Monday, June 27th at 7:00 p.m., right next door to the convention center, at the FOX Sports San Diego Grill. We hope to see you there!

Global Forest Watch

Many organizations use Esri and CartoDB together to create powerful geospatial applications. An example we are particularly proud of is the Vizzuality-developed application Global Forest Watch, an initiative in which Esri and CartoDB are both partners. In this project, GIS specialists make use of Esri software to collect and produce spatial data that is shared on ArcGIS™ for Server. From there, CartoDB syncs and caches the data to provide the application development framework for the website. With technologies like Torque, CartoDB enables an intuitive user experience for those interested in understanding deforestation.

GFW is a great use case about how CartoDB and Esri are already working together in complex and diverse environments, but this is just one of the ways the two technologies can be connected. Now let’s take a look at some different workflows; if you have any questions, please let us know.

Synchronizing ArcGIS™ Server with CartoDB

If you are using an Enterprise GIS and have ArcGIS™ for Server, you can already leverage CartoDB with our ArcGIS™ Connector. This connector, enabled in your CartoDB account, allows you to connect to public ArcGIS™ for Server services and import them into CartoDB. The connector accesses the services through the REST Endpoint that ArcGIS™ for Server provides out of the box. CartoDB’s connector can ingest both map and feature services that are hosted on ArcGIS™ for Server.

The ArcGIS™ connector for CartoDB was only available for enterprise plans. Starting today it will be available for all CartoDB plans, including Free Plans.

Share on Twitter

Using the Connector is as simple as copy and paste: you simply go a REST Endpoint URL, like this one, and copy the layer URL that you would like to ingest into the box below.

ArcGIS import

Once imported, the data can be used for your work in CartoDB. Also, as the connection between ArcGIS™ for Server and CartoDB is dynamic, the table in CartoDB updates when your services add new data, remove data, or change attributes from your geodatabase. You can choose to automatically update the datasource every hour, or do it manually every 15 minutes. This provides a great synchronization mechanism whereby CartoDB federates to ArcGIS™ for content.

ArcGIS import

Please, click here to read more about the Connector for ArcGIS™.

Using CartoDB OSM basemaps in ArcGIS™ Desktop, ArcGIS™ for Server, or AGOL

CartoDB offers a number of free basemaps for your data visualizations; we also provide custom-designed basemaps with tones that make your data pop. We offer them in raster format and now in vector format through our partnership with Mapzen.

They are free to use with almost no limitations, but if you need commercial terms we have these available. You can use the basemaps directly as “Tile Layers” with a standard URL format, or you can connect to them as a WMTS Server.

The WMTS Capabilities document can be found here.

On ArcGIS™ Online, add them like this:

ArcGIS online

One great capability of these basemaps is that they can be used to make label sandwiches! This way your labels can always stay on top of the data.

Publishing maps from ArcGIS™ Desktop to CartoDB

If you are using ArcGIS™ Desktop to analyze and manage geospatial data, you can upload that data into CartoDB for free, using a great tool from GKudos. This tool leverages our SQL API in conjunction with Python to load data from ArcGIS™ Desktop into CartoDB. The tool is available as a toolbox in ArcMap™ and ArcCatalog™:

Import Data

Here is a link to the ArcGIS™ Desktop Connector to CartoDB by GKudos.

Why use CartoDB together with Esri?

Now that you have learned the different ways to make Esri and CartoDB work together, you might wonder why people do it in the first place. The answer is because they can. We love Open Standards and Open products that can interoperate! These are some aspects of integration that users take into account:

Cost-effective for data that updates frequently

CartoDB’s dynamic capabilities for generating maps represent a cost-effective solution to organizations that need to publish data that changes quickly. Because CartoDB maps render dynamically and automatically cache on demand, we do not charge for the number of updates. That can provide very significant savings in situations where data changes every day, hour, or minute!

However with ArcGIS™ Online you get charged by number of tiles generated. In the Esri model, you generate tiles every time you make a change; because tiling an entire map is expensive, they charge you based on how many you create, and this increases costs quickly when your data changes often.

Credits/Map Views on web applications can be scary

One issue with usage-based pricing is that often you don’t know how many times you are going to use the system or apps to create or modify your maps. Additionally, you don’t know how popular your maps or applications are going to be. With a credit system your price can skyrocket before you know it.

That’s why we have simple, predictable prices that let you know how much you are going to pay. And we do not penalize you based on your success! We want your maps to be as popular as possible—this why there are no credits or mapview limits on our service.

Simple APIs for developers

Our goal with the CartoDB platform is to enable anyone to develop spatial applications. We believe that by using technologies like CartoCSS and SQL we make geospatial technology accessible to many more developers. Adhering to web development standards means faster development times and improved time-to-value, with an overall better user experience of applications.

Lock-in free. Open Source

CartoDB source code is open source. This means that the same code we run on our cloud service, you can get and run on your own servers. We have extra modules to increase performance and security on our cloud, but the software is fundamentally the same. Why is this important for our customers? It provides trust. You can develop on top of CartoDB knowing that you will have the freedom to run your own service if you want or need to. It provides a NO lock-in policy that many organizations appreciate. If we are providing a great service you will still prefer for us to run your servers, as we can provide a cost-effective solution and better quality.

Join us for the CartoDB happy hour on Monday at Esri User Conference!

We mentioned this above, but it’s worth saying again. We are having a happy hour at the FOX Sports San Diego Grill, 2 minutes away from the San Diego Convention Center. Please RSVP here!

Fox Sports San Diego Grill

We’re very excited to celebrate the geo community. It’s going to be a great time full of drinks, food, and fun. We hope to see everyone there!

Happy data mapping!


All product names, logos, and brands are property of their respective owners. All company, product, and service names used in this website are for identification purposes only. Use of these names, logos, and brands does not imply endorsement.

Esri, ArcCatalog, ArcGIS, ArcGIS Online, ArcMap and ArcGIS for Server are registered trademarks of the Environmental Systems Research Institute, Inc.

Normalize All the Things with the Data Observatory

$
0
0

The data we collect about the world is rarely as clean as we would like it to be. Often times data is noisy or incomplete and because of this we draw false conclusions from our data.

One of the biggest sources of potential error in geodata comes from not properly normalizing our data. There are a plethora of maps out there that claim to show some notable effect a population has on something, or the effect any given person, place or thing may have on a population. However, in reality these maps might just show us where a large number of people are. One of the worst offenders of these kinds of maps are those made with social media data.

In this post we will show you how you can use the Data Observatory to minimize the influence of population on your maps to let the true insight shine through.

Twitter

Let’s grab a dataset using CartoDB’s Twitter Connector and pick something we know is geographically distributed. Let’s say we are trying to analyze a social media campaign that the restaurant Chick-fil-A is running. We collect tweets from the data and produce the following twitter map:

We are not able to derive much insight from this map. There are too many points on top of each other. Let’s try aggregating this to counties and produce a choropleth of the number of tweets in each county.

We can use the Data Observatory to do this in one shot using the following SQL:

WITHcountiesAS(SELECT*FROMOBS_GetBoundariesByGeometry(ST_MakeEnvelope(-130.61,21.21,-64.16,54.62,4326),'us.census.tiger.county'))SELECTcount(*)AStweet_count,counties.the_geomASthe_geomWHEREST_Within(twitter_chickgila.the_geom,counties.the_geom)GROUPBYcounties.the_geom
Twitter Map

Great! Now we have a better view of which counties are tweeting about Chick-fil-A more than others, but it is still not a particularly useful metric. Let’s take Albany County, for example, which has a population of around 304,000 people, and compare it to Allegany County, which has a population of about 50,000. We shouldn’t be surprised if we see more tweets in Albany because there are a lot more people there to tweet.

We really want the number of tweets per person in each county and not the number of tweets per county. With the Data Observatory we can easily update our table to show a choropleth map normalized by the total number of people in each county:

SELECTthe_geom,tweet_count/(ST_Area(the_geom::geography)*1000.0*1000.0*OBS_GetUsCensusMeasure(ST_PointOnSurface(the_geom),'Total Population','Area','us.census.tiger.county
      ) as count_per_person
  FROM twitter_chickfila_choropleth

Applying a normalization query gives us a much better view of where our social media campaign is having an effect.

Normalizing other things

The example above shows how important it is to properly normalize your data for maximum insight. Geodata combined with the Data Observatory makes this really easy, and with the large number of measures available in the catalogue you should be able to normalize most datasets. For example, if you had a dataset of the amount of money people spend on their pets in each county, a map showing the percentage of an individual’s income spent on their pets. No problem! All we need to do is normalize by the median income in each of those regions.

Normalizing your data and your maps will help you create projects that are anything but normal! We can’t wait to see the many different ways you will apply this strategy to your maps!

Happy (normalized) data mapping!

New Academy Lesson: How to Choose Map Colors

$
0
0
Analagous and Complementary Colors

Do you want to go beyond using default color schemes for your map, but don’t know where to start? Or do you need to make your map more legible, but aren’t sure how? We’ll show you!

Learn how to make beautiful maps that tell a clear story, like the one above created by LifeWatch INBO. Check out our new Map Academy Intermediate Design lesson: How to Choose Map Colors, Part 1.

We explain the basics of color theory, and give you specific guidelines for picking colors. That includes designing for your color-blind audience members! Then we show you many different ways to implement your palette with CartoCSS. We’ll also help you avoid inflicting effects like this on your audience.

This lesson is the latest in our Intermediate Design course. It’s the first of a two-part series on working with color. The second part is coming soon! It’ll show you how to apply the skills you learned in Part 1 to create complete color palettes. It will also show you which palette type is best for the specific kind of data you have.

We hope you find the new lesson useful. Share your color palettes with us in the lesson comments! If you have feedback on this lesson or want to suggest another, please let us know at support@cartodb.com.

Happy data mapping!

Helping the eDNA project to Keep Montana's Rivers Free of Invasive Species

$
0
0

There is a growing concern that Montana’s waterways are at risk of new invasive aquatic species, which are known to spread rapidly through US rivers and lakes, negatively affecting water quality and altering ecosystems and natural habitats.

Researchers and investigators have been dedicating more attention to this issue and they are now using new tools to better understand the level of impact.

As we can see in this CartoDB visualization made by Ken Rand, some of these invasive species have already made their way to Montana:

To avoid these aquatic species from spreading to new areas and reproducing in Montana’s canals, Luikart decided to leverage CartoDB’s location intelligence and find out where the most affected areas are so that he can make better decisions on how to respond.

In order to achieve this goal, Luikart and his team took eDNA analysis from waterways, simplified the inspection and made early detection of invasive species easier.

Also, by using CartoDB’s platform they are tracking the evolution of each specie from 1902 up until recent years to fully understand the reality of the situation in the US.

Edna project

If you are interested in discovering more about what CartoDB’s geospatial data analysisis able to achieve, have a look at this page.

Happy data mapping!

Viewing all 820 articles
Browse latest View live