Map of the Week: Making Rent

Property values are a persistent obsession for New Yorkers, as are the arbitrary constraints to dwelling well (dwelling? yes). Between navigating the rental quagmire of online listings to toggling the various Trulia, StreetEasy, and Zillow helper services, you can tumble down a rabbit hole of the worst options imaginable (or at least, publishable). In the bizzare world of renting, we can help you through the looking glass.

This Map of the Week features the work of a familiar face to CartoDB, Chris Henrick, who between juggling GIS projects and grad school also co-organizes Maptime NYC. His MFA thesis project “Am I Rent Stabilized?” combines code, maps, and critical thinking about urban planning and renters rights - all of which we love and welcome at CartoDB! He currently lives in Brooklyn, NY, where we locals join him in an obsession with rent rates and regulations. Read on to discover more of the mechanics behind his impressive web app!

Am I Rent Stabilized? is a web application that encourages New York City tenants to find out if their landlord may be illegally overcharging them for a rent stabilized apartment and if so, motivates them to take action. The development of the app was spearheaded by the lack of enforcement of rent regulation laws in NYC by the local and state government. It is an attempt at using open data as a prompt for civic action, rather than solely for visualization and analysis. The app asks the user to input their address and borough, then checks it against a database of properties that are likely to have rent stabilized apartments. From here the app recommends a course of action and informs the user of their nearest tenants rights group so they can get help. The app features a responsive UI that is mobile friendly and its content can be toggled to either Spanish or Chinese, for non-english speakers.

DHCR & MapPLUTO Data Processing

Am I Rent Stabilized? uses a database I created that lives on CartoDB, which enables the website to work as a fully funcitonal web-app using the CartoDB SQL API and CartoDB.js library. However, I did a lot of data processing on my local machine before importing the data into CartoDB.

1. Processing the DHCR Rent Stabilized Building List

Excel workbooks obtained from a Freedom of Information Law request by the NY Department of Housing and Community Renewal were normalized, stacked, and converted to a Comma Separated Value (CSV) file format using a Node JS script. This allowed the data to be geocoded at once and then imported into a PostgreSQL database where it could be analyzed with the NYC MapPLUTO GIS tax lot data.

1. Geocoding the Processed DHCR data

A Python script was then used to obtain values for each property’s Borough - Block - Lot number (BBL), Building Identificaiton Number (BIN), and latitude - longitude coordinates from the NYC GeoClient API. A property’s street address and borough are passed to the GeoClient API, which then returns a plethora of useful information about the property such as the BBL, BIN, latitude and logitude values.

1. Determining NYC Properties That Are Likely Rent Stabilized

After processing and geocoding the DHCR data it was imported into a PostgreSQL database using CSVkit’s csvsql command as follows: csvsql --db:///nyc_pluto --insert dhcr_rs_geocoded.csv --table dhcr_rs

From here PostgreSQL was then used to analyze the data. Here is a link to the entire SQL code, but the most important queries are the following:

-- select the number of properties in the dhcr list the nyc map pluto data
-- returns 47,130 rows
SELECTCount(a.bbl)FROM(SELECTDISTINCTbblFROMmap_pluto2014v2)ASaINNERJOIN(SELECTDISTINCTON(bbl)bblFROMdhcr_rs_w_bblsWHEREbblISNOTNULL)ASbONa.bbl=b.bbl;-- select the number of properties in the dhcr list not in the map pluto "likely rent-stabilized" query
-- returns 12,549 rows
CREATETABLEmap_pluto2014v2_likely_rsASSELECTCOUNT(a.bbl)FROM(SELECTDISTINCTbblFROMmap_pluto2014v2WHEREyearbuilt<1974ANDunitsres>=6AND(ownernameNOTILIKE'new york city housing authority'orownernameNOTILIKE'nycha')ANDbldgclASsNOTILIKE'r%')ASaLEFTJOIN(SELECTDISTINCTbblFROMdhcr_rs_w_bblsWHEREbblISNOTNULL)ASbONa.bbl=b.bblWHEREb.bblISNULL;

These two queries tell us: A. what properties in the MapPLUTO tax lot data match the DHCR’s rent-stabilized building list, and B. what other properties are likely to have rent-stabilized apartments but aren’t on the DHCR list. From here I created a table that combines data from both queries as well as a flag that states whether or not the property is listed in the DHCR data.

CREATETABLEmap_pluto_not_dhcrASSELECTnot_dhcr.address,not_dhcr.unitsres,not_dhcr.borough,not_dhcr.ownername,not_dhcr.zipcode,not_dhcr.yearbuilt,not_dhcr.geom,not_dhcr.cd,not_dhcr.council,not_dhcr.bbl::bigintFROM(SELECTa.*FROM(SELECT*FROMmap_pluto2014v2WHEREyearbuilt<1974ANDunitsres>=6AND(ownernamenotILIKE'new york city housing authority'orownernamenotILIKE'nycha')ANDbldgclassnotILIKE'r%')ASaLEFTJOIN(SELECT*FROMdhcr_rs_w_bblsWHEREbblISNOTNULL)ASbONa.bbl=b.bblWHEREb.bblISNULL)ASnot_dhcr;CREATETABLEmap_pluto_dhcr_rsASSELECTdhcr.address,dhcr.unitsres,dhcr.borough,dhcr.ownername,dhcr.zipcode,dhcr.yearbuilt,dhcr.geom,dhcr.cd,dhcr.council,dhcr.bbl::bigintFROM(SELECTc.address,c.unitsres,c.borough,c.ownername,c.zipcode,c.yearbuilt,c.bbl,c.cd,c.council,c.geomFROMmap_pluto2014v2c,(SELECTDISTINCTbblFROMdhcr_rs_w_bblsWHEREbblISNOTNULL)dWHEREc.bbl=d.bbl)ASdhcr;-- I then added a column to identify properties that are registered or not registered with the DHCR:
ALTERTABLEmap_pluto_not_dhcraddcolumnregisteredboolean;UPDATEmap_pluto_not_dhcrsetregistered=false;ALTERTABLEmap_pluto_dhcr_rsaddcolumnregisteredboolean;UPDATEmap_pluto_dhcr_rssetregistered=true;-- now these two tables can be combined AND have a boolean value for whether or not they are in the DHCR's rent-stabilized buildings list.
-- 59,679 rows total.
DROPTABLEmap_pluto2014v2_likely_rs;CREATETABLEmap_pluto_likely_rsASSELECT*FROMmap_pluto_not_dhcrUNIONSELECT*FROMmap_pluto_dhcr_rs;-- check to make sure the data looks good:
SELECTCount(*)FROMmap_pluto_likely_rsWHEREregisteredISNULL;-- returns 0 rows
SELECTCount(DISTINCTbbl)FROMmap_pluto_likely_rs;-- returns 59,679 rows
SELECTCount(*)FROMmap_pluto_likely_rsWHEREgeomISNULL;-- returns 0 rows
SELECTSum(unitsres)AStotal_res_unitsFROMmap_pluto_likely_rs;--returns1,962,469

1. Further Data Processing Using CartoDB Lastly, the data was imported into CartoDB and some final tweaks to the data were made.

--  Remove all properties owned by the NYC Housing Authority that were missed.
--  This involved doing a spatial intersect with polygon centroids created from
-- a shapefile of NYCHA properties from 2011 to determine all spellings of "NYCHA"SELECTDISTINCTa.ownernameFROMmap_pluto_likely_rsa,nycha_centroidsbwhereST_Intersects(a.the_geom,b.the_geom)ORDERBYownername-- remove properties that are obviously owned by NYCHA
DELETEFROMmap_pluto_likely_rsWHEREownernameLIKE'NYC HOUSING%';DELETEFROMmap_pluto_likely_rsWHEREownernameILIKE'new york city%';DELETEFROMmap_pluto_likely_rsWHEREownernameLIKE'NYC CITY HSG%';DELETEFROMmap_pluto_likely_rsWHEREownername='CITY OF NEW YORK';DELETEFROMmap_pluto_likely_rsWHEREownernameLIKE'N Y C H A%';DELETEFROMmap_pluto_likely_rsWHEREownernameLIKE'N.Y.C. HOUSING AUTHOR%';DELETEFROMmap_pluto_likely_rsWHEREownernameLIKE'N Y C HOUSING AUTHORI%';DELETEFROMmap_pluto_likely_rsWHEREownername='NY HOUSING AUTHORITY';DELETEFROMmap_pluto_likely_rsWHEREownername='NEW YRK CTY HSG AUTHR';-- pgsql2shp converted boolean value of the "registered" column to T / F,
-- so I changed the valuse to 'yes' / 'no' for CartoDB Infowindows
UPDATEmap_pluto_likely_rssetregistered='no'WHEREregistered='F';UPDATEmap_pluto_likely_rssetregistered='yes'WHEREregistered='T';-- change boro codes to actual names for CartoDB Infowindows
UPDATEmap_pluto_likely_rssetborough='Queens'WHEREborough='QN';UPDATEmap_pluto_likely_rssetborough='Brooklyn'WHEREborough='BK';UPDATEmap_pluto_likely_rssetborough='Staten Island'WHEREborough='SI';UPDATEmap_pluto_likely_rssetborough='Bronx'WHEREborough='BX';UPDATEmap_pluto_likely_rssetborough='Manhattan'WHEREborough='MN';

Creating Catchment Area Polygons For Local Tenants Rights Organizations

In order to inform a user as to whether or not any local tenants rights organizations are operating within their neighborhood, custom polygon geospatial data was created to respresent each of the 94 organization’s service areas.

First, a list of Community Based Housing Organizations was scraped from an HTML table on the DHCR’s website using a Python script. Organizations that operate in the boroughs / counties that make up NYC were pulled out from the scraped data into a new table.

For these 94 organizations, polygon data was manually created representing each organization’s service area. Reference polygon geospatial data sources used to create the service areas include Pediatcities NYC Neighborhood boundaries, NYC Planning Neighborhood Tabulation Areas, U.S. Census Zipcode Tabulation Areas, and NYC Planning Community District boundaries. This data was copied and in some cases aggregated (dissolved) into a new dataset using MAPublisher, a GIS plug-in for Adobe Illustrator. In some cases boundaries had to be drawn by hand, such as for the Cooper Square Committee, which operates within a very specific area in the East Village of Manhattan. Once completed, the polygon data was joined to the DHCR Community Housing Based Organizations for NYC and then exported to a shapefile format.

The data was then imported into CartoDB for use with Am I Rent Stabilized. When a user’s address is geocoded, a point in polygon SQL query is made via PostGIS to the data in CartoDB.

For example:

SELECT*FROMnyc_tenants_rights_service_areasWHEREST_Contains(nyc_tenants_rights_service_areas.the_geom,ST_GeomFromText('Point(-73.917104 40.694827)',4326));

If a user’s address is within a group’s cachment area, that group’s information is passed into a modal in the app that displays information such as the group’s website url, phone number, contact person, and/or address. As this data varies from group to group, a Handlebars.js helper function is used to check if the data exists before passing it to the Handlebars HTML template.

varH=Handlebars;H.registerHelper('each',function(context,options){varret="";for(vari=0,j=context.length;i<j;i++){ret=ret+options.fn(context[i]);}returnret;});H.registerHelper('if',function(conditional,options){if(conditional){returnoptions.fn(this);}else{returnoptions.inverse(this);}});

Learn More!

Interested in learning more about the project? Tweet Chris (@chrishenrick) your thoughts and questions, check out the links below, and review his cross-posting on Urban Omnibus!

• Github Repo for Am I Rent Stabilized
• Visualization of NYC properties that likely have rent-stabilized apartments
• DHCR Rent Stabilized Building List (note: this is just a list of addresses that have one or more rent stabilized apartments, not the actual apartment numbers)
• NYC Likely Rent-Stabilized GIS data
• NYC Local Tenants Rights Groups Service Areas

Thanks for reading, and happy renting!