SpatialDev 3.0

April 16th, 2013 admin No comments

The blog has been silent for too long and its time to change that. We launched our updated web site last week amid rapid changes.  Over the past year we’ve:

  • Added more awesome people. There are 12 of us now!
  • We moved in to our new and improved office space
  • Launched dozens of applications on the web.
  • Continued to work with great clients like NASA,  IFPRI, OpenGeo, eCityGov, The City of Seattle, and the list goes on.

Stay tuned for more news!  Come find us on Facebook here.

    Categories: SpatialDev Tags:

    The new Northwest Maps website sets a new standard for municipal information services.

    October 11th, 2011 admin No comments

    For the past 15 years, delivery of location-based community information has been fragmented across departments and delivered through archaic and inconsistent user interfaces. In short, user needs and their experience weren’t considered so much. Over the past year, we’ve had a chance to work with the eCityGov alliance to create a new information service that was focused on the end user and how they could efficiently access the information they need.

    The new NWMap.net landing page

    We are very proud of the new NWMaps.net, launched this past June. It takes the best conventions of web-design and provides an efficient means to access a plethora of community information from property-based information to community demographics, from environmental resource information to commercial property listings. NWMaps.net is a one-stop shop for the citizens living in the central Puget Sound Region.

    Based on ESRI’s ArcGIS Server and Javascript API,  NWMaps.net starts with a data schema common across the member cities and an aggregation/validation process scripted in python that unites each city’s data into a common database.

    The application has a true service oriented architecture that not only incorporates web map services, but mashes in several other web services (permits, parks and commercial property listings) from third-party sources. This architecture not only allows for extensibility when other data come online, it also provides a means by which the data and services can be exploited directly via their REST endpoints by other applications.

    The real advantage of NWMaps comes from the user experience it provides. Site layout is simple. There is a Home page, a Results page and a Map view Page. The home page provides a single search box that will accept addresses, place names or parcel numbers. As far as we know, this is the first time this kind of custom search has been provided. We think it responds to users’ expectations for a Google-like search experience, including type-ahead suggestions and search criteria disambiguation.

    Flexible search via a single box

    In addition to the flexible parameters, users can pre-select the composition of their results by choosing a “Quick Search” option. If one is selected, related themes are promoted on the results page, putting the specifics they are looking for front and center.

    Construction Activity Quick Search at NWMaps.net

    Quick searches allow users to focus their results

    While the information accessible to the user is voluminous, we did not want to overwhelm them with pages and pages of results, so each result topic is contained in a results panel, collapsed by default. While the information is initially hidden, feedback on the number of results is given (where relevant) on the results bar.

    Results grouped by theme

    While the name of the site is NWMaps, the map doesn’t dominate the experience… unless the user wants it to. In the normal site flow, the user enters a search on the home page and once taken to the results page is presented with a set of “map snapshots” that give the user map-based visualizations of information within the immediate vicinity of their search result. If the user wants to have a “traditional” web map experience, they can go directly to the map view which incorporates both the unified search and a set of map tools that support a variety of specialized tasks.

    The Map View page and specialized tools

    This post is already getting long, but there is a lot more to share about the NWMaps.net which I’ll include in a follow up post. In the meantime, Check out the new NWMaps.net, we think it’s pretty cool.

    A Web Mapping Solution for Resource-constrained NGOs

    July 12th, 2011 admin No comments

    Non-governmental Organizations are often on the front lines of service and innovation for international development. Operating as not-for-profit entities dependent on the largess of donors, they are natural economizers and that often means they forego investments in tech tools like GIS, mobile and web solutions. The ESRI non-profit program is going to shepherd in a significant expansion in the use of GIS in international development. The cost barrier has been removed but it will be some time before the human capital gets to the point where it can fully exploit the new tools available. SpatialDev recently had the opportunity to collaborate with a project designed to raise awareness of land tenure issues as they relate to people living in extreme poverty. The project wanted a series of maps that helped to tell the story of how natural endowments of the land, climate and land use provide context to the discussion if land rights. Embedded below is a prezi used for a panel discussion, “Facing Reality in Developing Countries: Working with Research Partners with Limited Resources”, at the 2011 ESRI International User Conference in San Diego.

    Getting Spatial Data in to Tableau

    April 18th, 2011 admin No comments

    If you haven’t yet see Tableau Software run over to their web site and check it out now.  In a nutshell Tableau software is a desktop tool that allows users to quickly analyze reams of data, from multiple formats extremely fast. You can use it to create some remarkable visualizations from data that is probably already in your database. Of course the first thing we wanted to try is visualizing and analyzing spatial data.

    Tableau is not (nor does it pretend to be) a GIS tool. But it does have some basic ability to represent spatial data. Tableau requires you to flatten out the data by essentially pulling out each vertex in to its own row:

    Default Tabluea Spatial Data TableThe tricky part to this turned out to be the VERTEX_TYPE. Any geospatial veteran will immediately recognize that this column is meant to identify island polygons in the data. However nobody at Tableau could us what the valid range of values were,  so we had to guess.  FME seemed to be the most logical tool to use to try to flatten out the data.

    Using a combination of ListBuilders, Choppers and ListExploders I was able to get 80% of the way towards building a ‘Tableau Loader’ which could take any vector data and flatten it out to almost any RDBMS.

    However this really does not handle the mysterious vertex type. The transformation simply treats everything as interior polygons which results on strange spikes and lots of self-intersecting lines any time it encounters a interior lake or other real world island. The challenge is in trying to determine the point order for interior and exterior polygons. After several hours on the phone with Tableau and working with four different tech people, no body could tell me what the proper ordering for the vertex points should be. Since this work there have been a few more examples posted which should all work with relatively simple data.

    We shifted gears back to trying to represent point information. Since a lot of the data we deal with on a regular basis is raster based, its relatively easy to represent as points. Using this approach we were very quickly able to generate some simple dashboards on sub-national level population statistics.

    After the visualization is created using the desktop software its a snap to publish out to their hosted, public facing server. You can see our little sample here.  It would be great (and probably not hard) to connect Tableau directly to a spatial database such as PostGIS or SQL Spatial. With a little more effort it should be able to connect to FME for more complex data sources or even include some licensed FME components in to the Tableau desktop.

    Categories: SpatialDev Tags: , ,

    SERVIR – Himalaya

    April 3rd, 2011 admin No comments

    A few weeks back I had the opportunity to visit ICIMOD on behalf of the SERVIR program. ICIMOD is the International Center for Integrated Mountain Development  that serves eight member countries of the Hindu Kush-Himalayan region.

    ICIMOD is the latest institution to join the SERVIR program as a regional node (the other nodes include RCMRD in Kenya and CATHALC in Panama). The week in Kathmandu was spent taking a deep dive in to the technical issues and challenges associated with deploying sophisticated mapping applications and geo-processing services in this part of the world. The joint team dug into geodatabase models, ISO metadata profiles and dataset publishing amongst other topics.

    ICIMOD already has a talented group of developers and the MENRIS division already hosts some great applications including the Mountain GeoPortal. Now that the partnership with SERVIR is firmly in place, the plan is to create some specific thematic and regionally focused mapping applications. Several environmentally focused systems are planned that will initially focus on addressing fire mitigation, land use change and the cryosphere.

    Of course it wasn’t all work. We we’re hosted to a traditional, five-course Nepali dinner in one of the nicest hotels in Kathmandu. The local team was also kind enough to show us around the city for some quick visits to famous Buddhist and Hindu temples. SpatialDev is looking forward to continuing work with the team at ICIMOD and SERVIR.

    Categories: Uncategorized Tags: ,

    Some good tools for the bootstrapped startup

    November 27th, 2010 admin No comments

    SpatialDev’s first year was a great learning experience in how to efficiently run (and hopefully grow) a small business. One of things that helped us was a great set of tools to do everything from managing our financials to testing our products. This post gives my review (in random order) of some of the tools we’ve tried and the ones we’re sticking with.

    Tools

    Collaboration and Content Management
    We started using BaseCamp back when SpatialDev was still just a pub based conspiracy. It is simple, cheap and worked beautifully when we were starting up. Now that we’ve got larger and more complex projects, we find ourselves wishing for a few things BaseCamp doesn’t have… like document folders. It’s also a hard sell with clients who are invested in SharePoint (a product that makes me want to cry every time I try to use it). BaseCamp is great for “mastering” our key project and product documents, but we’re pragmatic and not religious users. One of the coolest things about BaseCamp is the ecology that has grown up around it. We’ve been using a bunch of cool tools built on Rails that use the BaseCamp API and like BaseCamp, they are cheap and for the most part just plain work.

    Timekeeping
    Anyone looking for a simple timekeeping system to use with or without BaseCamp integration should seriously consider Tick . It’s a very straightforward timekeeping system that can draw projects and work-breakdown structures (WBS) from BaseCamp and comes with mobile apps to enable you your crew to log time on the go. It is super simple to set up projects, generate billing details to accompany invoices, and keep track of burn.

    SourceCode Control and Bug Tracking
    We use Springloops which is basically a hosted subversion service that has straightforward Rails UI and basecamp integration. Another Rails web service that we use is 16 Bugs. Its easy to set up and provides a simple interface that testers of all backgrounds  can use. It doesn’t do anything fancy like manage the testing process, it just does bugs tracking and resolution and does it well.

    Virtualization
    I’m a Mac guy at heart and one of the things I always wanted to do was have a Mac as my primary computer at work. I use a 27” iMac loaded with RAM running Parallels . I haven’t used a PC emulator in quite some time and am totally blown away with the features and performance of Parallels. With the screen real estate I have on the iMac, I can truly run in parallel with Win 7 and Snow Leopard. This way, I can get access to all of the Windows-only software I need and stay in the Mac world where I belong.

    Cloud Storage and Infrastructure
    We’ve used Box.net and it works fine, but the business customer pricing is a little high for what you get. The desktop integration with Dropbox is cool and we used that as well. These are great tools and work well, but we can do the same thing with S3, so we’ve started moving to Amazon Web Services for all of our storage and flexible infrastructure needs. Now that Azure is part of the Bizspark program, we may check that out as well.

    Dedicated Server Hosting
    We still use dedicated servers for production hosting and we’ve been happy with what GoDaddy provides. It is also pretty slick to have your domain registrar and hosting provider one in the same. As ActionMappr continues to grow, we’ll move to provisioning new client sites through AWS.

    Software Design
    Our Interactive Art Director uses the heavy duty tools to design and create UIs, but from time to time, we need to draw up a workflow, a context diagram, system block diagram, a crude wireframe, what have you. Visio is good, but I have found OmniGraffle to be way better, despite the weird name. The shapes behave like I expect them to, the guides for placing things are more helpful in getting things straight, and the ability to apply styling is easier. The best thing about this software though is the availability of stencils that are directly relevant to what we do. There are a ton of free stencils developed by the user community and available at “Graffletopia” . You can find stencils for iPad, Android, Napkin UML and tons of other stuff that relates well to agile development.

    Business Productivity
    One of the biggest boosts to our start-up business has been access to all of Microsoft’s products through their BizSpark program. Microsoft gets its share of criticism for many of the things it does, but this is a very generous program that really makes bootstrapping possible. Other software companies create high barriers to entry for using their products and miss out on the good will and loyalty to be had by helping a small company get off the ground. Kudos to Microsoft for BizSpark!

    Being a bootstrapped startup is always an exercise in resourcefulness and endurance. We found some great tools that alleviated some of the pain of building from scratch. For others considering  a start up or searching for tools to relieve pain, I hope this helps.

    Categories: SpatialDev Tags: , ,

    African Ecosystems:Clip, Zip & Ship

    November 11th, 2010 admin No comments

    When we first started work on the Ecosystems application we had just a few goals in mind
    a) Highlight the new Ecosystems data recently made available from the USGS.
    b) Make the data as simple as possible to download.
    c) Kick the tires of ArcGIS Server for serving very large raster data sets from the central office in Kenya.

    We started with the core Ecosystems data but quickly expanded to include additional agricultural and infrastructure data such as Google Map Maker Roads and Open Street Map roads for sub-Saharan Africa. We ended up with over thirty gigabtyes of spatial data (both raster and vector) stored across multiple file and SDE geodatabases.
    The site uses a combination of custom python scripts in conjunction with the ESRI ArcGIS Server 9.3.1 javascript API. Downloading data is as simple as selecting one or multiple layers, entering an e-mail address and defining an area on the map. Because of the large file sizes the defined areas are limited to about the size of Mali.  The system sends the user an e-mail with a download link to the server at RCMRD (Regional Center for Mapping Resources for Development) in Kenya. The downloads include an ArcGIS MXD,  the basic metadata and in some cases a data license.

    The biggest challenge during the development process was probably physically moving the data, in the form of file geodatabases, to Africa. The ten hour time difference between Seattle and Nairobi didn’t help either. Another on-going problem is the frequent power outages in Nairobi which routinely break the connection between the application server and database server. A new uninterruptible power supply should be on-line soon.

    In the near future we hope to support additional non-ESRI download formats. We will also be increasing the number of data sources from RCMRDs vast catalog of African spatial data. A huge thanks to the data providers including as the USGS, Open Street Map, Google for all the content. And a huge thanks the local team at RCMRD and the SERVIR program for making this all come together.

    Categories: SpatialDev Tags: , , , ,

    FME User Group Meeting

    September 29th, 2010 admin No comments

    Jubal Harpster from SpatialDev will be speaking at the upcoming FME user meeting in Seattle.  The meeting is not just a sales pitch by Safe Software but a great chance to see what others in and around Seattle are doing with FME. Jubal will be talking about some ongoing work trying to orchestrate and audit multiple FME work flows using a combination of FME and Python. If time allows we may even get a chance to look at recent work building XFMaps to create geography from the Foursquare API.

    Categories: SpatialDev Tags: , ,

    SpatialDev & ECityGov.Net

    September 15th, 2010 admin No comments

    We are pleased to announce that SpatialDev has been selected to parter with the eCityGov Alliance to develop the next generation NWMaps.net GIS Internet Mapping Platform and Portal. The current site – in service to 10 communities now for over eight years – needs some repairs, data updates and a technology overhaul. The primary goal of the project is to create citizen-facing information services and assembling a new and improved base map.

    The SpatialDev team is doubly excited because this announcement comes as we mark one year as a company.

    In the coming months we will be working closely with the eCityGov Alliance to define new and innovative approaches to serving location-based municipal information. The new NWMaps.net will also integrate more seamlessly with other eCityGov services such as property and permit information. Ultimately, the project will provide an intuitive and user-friendly website for all seeking information about their community to make decisions affecting their families and business.

    Categories: SpatialDev Tags: , , , ,

    More about ActionMappr and a request…

    August 17th, 2010 admin No comments

    We keep making progress on ActionMappr and look forward to launching soon. Part of our process of getting to launch is customer discovery, trying to find some really good use cases and pain points where ActionMappr can really create value. Our experience with ActionMappr is that these aren’t too hard to find… they’re too hard to filter down to just a few. So we’ve decided to try and cast our net a bit wider and get more feedback to focus ActionMappr where it can have the most impact. We’ll also be putting a survey up on the ActionMappr site to solicit some specific feedback for our effort. Attached to this post is a one page profile sheet (hi-res) for ActionMappr that we hope will get around to explain what we are trying to do and get the attention of some more folks. Please cross post and tweet this.

    ActionMappr Overview

    Categories: SpatialDev Tags: ,