Untitled

ArcConsole

Mon, 25 Mar 2013 20:50:08 -0400

I just released something that I’ve been working on recently: ArcConsole. Check out a demo.

It’s a simplified, intuitive API console for the ArcGIS REST API. It uses awesome JavaScript libraries like Backbone, Backbone Forms, Chosen, Bootstrap, Leaflet and Leaflet Draw to take some of the pain out of querying data from the standard ArcGIS REST API.

The ArcGIS REST API is really powerful and the City of Philadelphia (which I work for) is using it more in our open data efforts. We’ve got some big time datasets on it like all of the Police Dept’s Part I crimes since the beginning of 2006 and the Office of Property Assessments’ recent actual value initiative which set new values for almost every property in the city. But, it can be frustrating for developers and other users that are new to querying the API to get started. This application attempts to make that on-boarding a bit easier by stripping away some of the advanced features that we don’t see being used very often. If the user needs those advanced features, like non-Web Mercator spatial references, they can use the standard API console at any time. But with this application the user doesn’t have to navigate to the service and layer that they need, they just select the data set that they want from the top drop-down and the rest of the parameters and the application constructs the URL for them. There’s a static JSON file that defines all of those endpoints and additional information like a description and links to metadata can be included as well. Plus the ability to draw the geometry that you want to query on instead of the frustration of trying to the object string just right is a big step up in my opinion.

The app’s still a work in progress and we’ll look to improve it over time. With some server-side code, it could be extended into a full-blown data extractor. There’s a lot still to do and you can check out the code at the City of Philadelphia’s GitHub page.

Adding a repo to the official City of Philadelphia GitHub page was especially satisfying for me. The City of Philadelphia is on GitHub. And I have some of my code in it. How cool is that?

PHL Crime Mapper Post Mortem

Tue, 22 Jan 2013 00:22:00 -0500

It’s been a little over a week since I released PHL Crime Mapper and the response has been beyond my wildest dreams. Here’s a few select quotes that made me proud:

“…a handy tool…”

“…gives easiest look yet at crimes by neighborhood”

“…wins points for simplicity…”

“…nifty…”

“…handy interface for seeing where serious crimes are being committed.”

“…crisp, easy to use…the simplicity and refinement of design is excellent.”

I was even on the radio to talk about it!

I like seeing words like “handy” and “simplicity” because that’s definitely what I was going for. The app strives to do one thing and hopefully do it well.

I also got a lot of feedback by email from community leaders and others about how they found the app useful. I didn’t think of it before, but this would be really helpful for community organizations and neighborhood watch groups. I’ll have to think about ways to get the word out to them in the future. Overall so far I’ve gotten a lot of feedback that has solidified how I should move the development forward.

A few things have been identified that need improvement. Some users had issues with using the application in so far as drawing an area and changing the date range. So tonight I added a “Draw” button to the black box on the left that explains how to use the app. In the process I actually decreased the height of that box by removing some copy from the last sentence. So that’s good.

The other thing that I have to work on is finding a date selector that is easy to use. I really like the current one but I’ve actually witnessed myself a few people having trouble figuring out how it works. I may find a different one (although I hate the calendar picker ones) or at least put a hover over the slider that explains how to use it.

Beyond that I’ve thought about some long term bigger additions that could really make the app awesome. I also have to work on making sure the codebase remains maintainable by using Backbone and maybe porting the JavaScript to CoffeeScript. I’ve never used CoffeeScript before so this would be a good way to learn.

Finally, some closing stats:

30% of visits from mobile devices (including tablets). This number was a bit surprising to me, but it shouldn’t be. There was a mobile version but a healthy amount of these hits were on tablets, which the app currently doesn’t support at all, which has been been driving me crazy. I have to spend some time soon adding that support.
The app was viewed in 46 states and 54 countries (whoa)!
Internet Explorer, Chrome and Safari split 70% of the traffic down the middle, Firefox got 16% and a bucketful of other browsers got the rest. Great to see Internet Explorer not in the lead.
Most toggled crime: Thefts. Which makes sense because there are so many of them.
The Philly.com article accounted for about 63% of traffic, which is amazing. Interestingly enough, a fifth of that came in through the mobile version of their website.

The traffic has died down, so now it’s time to get back to work on making the application better long term.

PHL Crime Mapper

Mon, 14 Jan 2013 07:33:00 -0500

Last month the City of Philadelphia released data on Part I (violent and serious property) crimes over the last six years. Today I’m releasing a web application that uses that data to help folks visualize these crimes in their area: PHL Crime Mapper.

On a computer (non-mobile device), the application allows users to select a time frame and draw a polygon around the area that they want to view the crimes for. They can then toggle on/off different types of crimes and can always change their data range or start over again.

On a smartphone, users select either a 3, 5 or 10 block search area (approximate) around their location. A tablet version is coming soon, for now the app should be pretty worthless on iPads and the like. It was important to me that there was a version of this app that worked on smartphones and that it was just not the desktop version shrunken down.

The ultimate vision with this is to allow users to select the region they want to know about and get a text or email alert weekly/monthly of all the new crimes in that area. It’ll take some time to get there, but I think the app really becomes useful at that point.

I showed this to my friends over the weekend and one of them asked what the objective is. I don’t really have one and I don’t know how people will use it specifically. If they’re like me they’ll draw a polygon around their house to view the recent crimes in their neighborhood. Perhaps they’ll change the data range to see how crimes have changed over time. To me the more you know the better, especially when it comes to crime in your city and the Police Department thankfully releasing this data lets it to happen.

It was fun working on this project and I’m pleased with the results for now although there are some UX and loading time improvements that have to be made. This was the first time that I got to work with Leaflet extensively, which is 100% awesome. There were a few steps along the way where I thought that I would be stuck because Leaflet has a smaller API than other client-side mapping libraries, but it always had the perfect solution. Where Leaflet cannot offer the functionality there are some great plugins that give you more, including the awesome Leaflet Draw that I am using here. I may have to post again later about Leaflet. I have always been a fan of the ‘Toner’ basemap from Stamen Design and I finally get to use it here. Overall the app is pretty fast once it’s initially loaded, due to the fast Esri REST server that I’m calling for the data (and geometry services) from and the markers being Font Awesome CSS sprites.

I hope folks find this useful. The code is up on GitHub and pull requests and new issues are welcome. It’ll be interesting to see how people use it and I’m definitely open to making changes to make it more useful and easier to use.

Know When To Go To Sleep

Sat, 05 Jan 2013 01:09:07 -0500

Recently I’ve been working on some side projects late into the night. If I don’t have any commitments on a weekend I may stay up late working on something and then sleep in the next day. Because I can, and it’s awesome.

But I also have to learn when it’s time to stop and go the f**k to sleep.

A few nights ago I was working on one of these projects and got stuck on one thing that wasn’t working even though I KNEW it should have been. After struggling and trying to debug for 45 minutes, I gave up, commented my battle in the code and went to bed.

Tonight I came back to that code and fixed my problem in less than a minute. It made so much sense that I had a function call inside a loop when it shouldn’t have been. With a clear head I caught the mistake almost instantaneously, but a few nights before I toiled away forever to no avail, cursing computers the whole time.

The same thing just happened to me tonight: some parameter in a POST request isn’t working and it’s not working even though it SHOULD BE. So for now I’ll just go to sleep, dream of successful Ajax calls and come back to it when the sun is up.

My 2013 New Year's resolutions

Mon, 31 Dec 2012 18:37:00 -0500

So it’s about that time again, isn’t it? For the past few years I’ve tried to make and keep a list of New Year’s Resolutions. Problem is that I’ve been changing my blogging platform so much that I can never link back to review it a year later. Here’s to hoping this time is different.

2012 was an awesome year for me, probably the best of my life. Here’s what happened to me this year:

Got married
Got a great job that I really enjoy
Got to visit Paris, London and Brussels on our honeymoon. The first time I’ve ever been outside of the United States
Got my Master’s Degree (pending, but I’m done with classes)

While all of this was awesome, it’s all stuff that I knew was going to happen and was already planning for in 2011. The challenge in 2013 is to keep the momentum going after getting a lot of the big life things done. So here’s what I’m shooting for in 2013.

Skills

Learn HTML5 and the mobile web. I’ve started this already and it’s clearly where the future of technology is going. A sub-goal here is that every app I launch this year will look great on smartphones, tablets and traditional desktops and a majority of them will be designed for mobile first.
Learn Android. Yeah yeah, you can make more money in iOS, but Android is going to continue to spread like a disease. Long term this is a really good skill to have. Measurement: release at least one app in the Google Play store. I’ve already got pretty solid ideas for three, I just have to wrap my mind around how it all fits together.
Learn functional programming by the way of Scala. I’ve started this already by beginning the Coursera’s Scala class. How awesome is it to learn a programming language from its creator?! End goal: write one substantial application in Scala.
Contribute to at least 3-5 open source projects. This could be creating and maintaining something that others find useful or having pull requests accepted for other projects.
Attend five hackathons. They’re a good way to stretch yourself, code as a team under pressure and learn from others.
Learn D3.js. Like really learn it, not just make a few bar charts. Make something cool. At least five original, substantial data visualizations.

Personal

Run 750 miles. In 2010 I ran 1,000 miles, but I wasn’t able to replicate that in 2011 due to injuries and just a lack of free time. It’s not the energy that’s the problem but devoting 10+ hours a week to running can be challenging. But 750 miles is a pretty healthy number.
Less distractions. I’m constantly swamped in Google Reader like it’s my job. I’ll have to make some subscription cuts to get my unread count down and maybe sometimes “Mark all as read” button. Also, cancel my magazine subscriptions (Wired and Fast Company). There just isn’t enough time in the day to read everything I want to read and I have to get used to that.
Read 12 books. In 2010 (man, I had free time in 2010) I read 24 books, which isn’t insane, just two books a month. While devoting time to the above I think I can manage one book a month. Hopefully more fiction than technology books to give myself a break.

Self Quantification

Get back onto Daytum. In the second half of 2011 I sort of fell off with Daytum which I was putting a lot of data into every day. In 2013 I resolve to be more dedicated to it, but not for everything but only the important stuff. Also, I’ll work towards visualizing the large dataset that I currently have about myself.
Write five days a week in 750 Words. Writing in a journal helps me reflect, especially at the beginning of end of a day.
Use at least one passive logging technique. Selfspy looks interesting.

And that’s about it. Some of this kind of scares me, but I guess that’s the point. I doubt that anyone will find this interesting, but I have to put this list out publicly to motivate myself to get to work. I’ll check back in the beginning of 2014.

Monitor crapped out, but kind of cool. No filter.

Sat, 29 Dec 2012 01:42:37 -0500

Monitor crapped out, but kind of cool. No filter.

Did The Cavs Win Last Night?

Sun, 16 Dec 2012 11:05:00 -0500

So yesterday I finished up a little web app that I’ve been working on over the last few weeks when I get a chance: Did The Cavs Win Last Night? A while back I came across Is there a Giants game today? and thought it was a great idea in its simplicity. I’m definitely a fan of one-purpose single-page apps because they can be really good at the very one thing that they are trying to do.

So I created something similar for the NBA that not only tells you when the next game is, but tells you if your team won the night before. And not only that, but it’ll give you fresh stories and videos about the game. I envisioned folks waking up the day after a game to see how their team did the night before. In the past I maybe went to NBA.com or ESPN.com to get the score and some single wrap-up, but now with this app it’s possible to get a bunch of viewpoints in one place.

The app is entirely static. When a user accesses it it checks if there was a game yesterday by calling a JSON file of the team’s schedule. If there was, it’ll ask the Twitter account @SimpleNBAScores for the score. It then pulls up stories from blogs and websites and videos on Youtube about the game to give a more complete view of what happened and what people are saying about it.

It was fun to make and it gave me a chance to try out a few new things like: media queries (pretty simple since this was one column), Canvas (just one element but quadratic curves can be tricky) and Yeoman, which I can’t live without now. I also have it hosted on S3, which is new to me but was pretty easy and working well so far. Deploying through their web client is a pain, but I think I’ll try to make a command line application that streamlines that a bit.

While planning this project I did a lot of research into how I can get sports scores on the client-side and basically there are none besides that Twitter account. ESPN has an API, but the general public isn’t given access to scores and you need a key anyway so it wouldn’t work on the client. All other APIs for sports info are paid. There is one way person that found a ESPN live score stream and he created RSS feeds to scrap it, but I couldn’t make jQuery GET requests to it. So there’s almost nothing, which is strange to me because sports scores are so ubiquitous. You’d think that it would be in the NBA’s best interest to put out a robust API, why not, but I guess there’s too much money at stake. For now I’ll have to use the Twitter feed even though it’s using a depreciated non-auth version of the API. If that goes down I’ll probably have to create my own API that all version of Did The… can use.

Anyway, I hope people find it useful. The code is on GitHub and I’ve included instructions on how to setup it up for other teams. Hopefully folks will fork it and deploy a version of their team!

I don’t know why, but I always love seeing when public...

Thu, 13 Dec 2012 00:04:44 -0500

I don’t know why, but I always love seeing when public displays aren’t working properly. Something about revealing the curtain to show…a regular old Windows PC!

New blog!

Wed, 12 Dec 2012 23:58:49 -0500

Well here’s another try for me with maintaining a blog. This time I’m using the excellent Syte platform. It’s made in Django and built to be deployed primarily on Heroku, which I’m quite the fan of. It’s based around the idea of using services to display all of your content - for example I’m writing my blog posts in Tumblr (when it’s up). Consequently there’s no database, which is completely awesome to me. It’s really a forward-thinking way of personal publishing on the internet and I hope to add some pull requests to improve it when I get a chance.

So let’s see where this blog new website goes. I’ll probably be writing mostly about technology and projects I’m working on, but maybe about music and basketball too. I hope to launch a new app, which I will make sure to share with you, loyal reader(s).