IBM-Netezza and ESRI: Geospatial Data Analysis

I have always paid a lot of attention to the world of GIS and its applications. Today, ESRI is the major player dominating the world of GIS, thanks to its innovative products and understanding of the needs of its customers. This is why the announcement made in October by Esri and IBM-Netezza is so important. I'll try to explain here why and what it means.

One of the challenges of the systems handling geographical data is the analysis of that data when it reaches certain amount. While current databases can store as many terabytes of information as we need, and storage is becoming cheaper and cheaper, the problem arise when we need to analyze this data, as it takes a lot of resources and a lot of time -not only analyzing it, also moving the data-, making some analysis just impossible to perform. What usually happens is that customers only analyze samples of data, so the results achieved are not satisfactory at all.

This is when Netezza and ESRI partner together to find a solution to this problem. When you put together the leader in data-warehouse analysis and the leader in GIS solutions, something wonderful has to come out!

This announcement brings you the promise of an easy way of analysis of massive amounts of geospatial data and all in a timely fashion. But, this is something you already can achieve with Netezza...

So, what does all it mean? I'll try to explain it in detail, answering some typical questions.

What about Netezza and Spatial data, does Netezza support it?

Yes, Netezza is able to store spatial data, as it uses the Open Geospatial Consortium Simple Feature Specification. So, Netezza is able to store and understand natively spatial data: point, lines, polygons, multipoint, multiline, multypolygon...

What about Spatial Functions?

Today you can find about 90 spatial operations included in Netezza, executed as any other SQL function, but taking profit of the massive parallel multiprocessor architecture of Netezza. In upcoming versions, the number and complexity of functions, as long as the kind of spatial data stores, will increase.

So, does it work only with ESRI?

Not only with ESRI, as it is a standard implementation, it works with any GIS that fulfills the standard, including opensource products, but the relation with ESRI is special.

Why is the relation with ESRI Special?

The relation with ESRI is a long term one, as ESRI supports Netezza as data storage from a long time, and ESRI provides a native connector for Netezza (ArcView, ArcCatalog, etc.).

Ok, but, what is the cost of it?

There is no additional cost. It is included in Analytics, and Analytics is included in Netezza. You buy Netezza, and it is all included, at no additional cost: Analytics and Spatial. No hidden costs, no surprises.

So, what are ESRI and IBM-Netezza announcing?

A very interesting advance in geospatial analysis. So far, ESRI or any other GIS software can use Netezza as a repository of their data, taking advantage of the astounding speed Netezza provides serving queries. But, in this case, what ESRI and IBM are announcing is the inclusion of several ESRI libraries into the Analytics package of Netezza. This way, instead of extracting data from Netezza and then perform the complex calculations in the server containing the ESRI software, part of those calculations can be done in-database, inside Netezza, as it already will contain the libraries and functions to be performed by ESRI: at the end of the day, Netezza is a kind of parallel supercomputer, so, why not taking advantage of it? And what is more interesting, at no additional cost.

If you want to download a paper describing Spatial in detail, follow this link, and register for free.

Impressed? You could be more impressed, as you may ask for a trial for free (or just call your local IBM-Netezza representative).

As Netezza says: your data, your facilities, our appliance.

Sun memorabilia

During my tenure at Sun, I was able to collect a series of stuff identified with the Sun brand. I guess that many customers just came to our shows to get some of the archetypal Sun mugs, starting with the classical Java mug

Mugs against "Technology Craziness" (I guess this is a post-dot-com mug)

And of course the mugs delivered for the local edition of the Java Expo, a re-known event, probably the most famous in the IT sector in Spain in the early 2000s

The other mug is a general branding mug with the inscription "Conecta 2" (Connected).

Another fine mug is this pair advertising the "Mainframe Rehosting" program from Sun, that tried to erode the IBM's mainframe market (with no success, by the way), but the mugs are just great (as the mainframes are...)

Another mug I like very much is this one, from StorageTek, including an anti-sliding base and prepared to keep your coffee warm

A fine collection I'm proud to have, as you can see

In another occasion I'll show you my t-shirts collection (some of my mates thought Sun was an apparel company...). To close this post, I'll show you another amazing Sun piece of merchandising the Sun car:

Well, it is smaller than it seems, but is quite useful though:

Hope you enjoyed.

Marathon Woman!

In many of the companies I've worked for, my mates were very concerned about their fitness, and in some of them they were real fans of running, participating in amateur marathons and popular races.
At IBM it is different: as everything we do at IBM, we take things quite seriously. I can say that my mates take running almost as a profession, and I give you here an example. I'd like you to meet Pilar:

Pilar, training with her husband Galo

She is not only a marathon woman, she is a work mate (a very busy one!) and a mother of family, a person I really admire, full of energy and optimism. You can follow her on her blog, where she is describing how she is preparing her New York Marathon (she finds time to attend meetings at the office, presentations with her sponsor, training for the competition, and of course quality time for her family, just amazing!). And not only that, after competing in New York, she will go to THE marathon: the Athens Marathon! I really envy her.
Go Pilar, we support you!

Dennis M. Ritchie (1941 - 2011)

During this few weeks we have been shocked by the death of one of the most charismatic personalities in the computer industry and in the world in general, as he is credited of having changed our habits and of having brought the technology to the masses. Yes, I'm talking of Steve Jobs. But he has not been the only great loss we have had these days, we have lost one of the men who made possible the dream of Steve Jobs, I'm talking of Dennis MacAlistair Ritchie.

I'm not going to start a discussion of who was more important, Jobs or Ritchie, my opinion is that without men like Jobs the technology would not be spread as a commodity today, and men like Ritchie would not be possible; and without the Ritchies of the world it would not be possible the scientific advance in computer science, and thus there would be no Jobs to woo users with fantastic devices. So, both have their role in the business, and no one is to be underestimated, but, as Jobs has received plenty of attention, I think Ritchie deserves this little homage.

Maybe a usual reader of this blog already know about the work of Dennis Ritchie, but I just want to underline his two most important contributions, or at least the most famous inventions of this great man.

The first contribution is the co-creation of the Unix operating system. Ritchie worked at Bell Labs with other geniuses (Thomson, Kernigan, McIllroy and Ossanna). The first implementation of Unix was done in a PDP-7 machine from Digital, and was written in machine code.

As this was a relatively poor performing machine, they asked permission to port the Unix OS they have created to a more powerful PDP-11 machine. Although the original porting of the system was done also in machine code, sooner they wrote most of the core of the operating system in C language, a language that has been developed entirely by Dennis Ritchie between 1969 and 1973.

This was quite important: they proved that an operating system could be written in a high level language (although some other attempts were made before, they were not so successful), so it became a much more easier task than doing it in machine code and could add more features to the OS. And it was a way to facilitate the porting of the operating system to any other computer: as it was not written in machine code, with a few minor changes it could be compiled in any system they wanted Unix to be run.

And the rest, well, is history now. Unix spread out of the Bell Labs and can be considered as the backbone of the Internet as we know it, as most servers (and enterprise server outside the Internet, like Netezza for example) run any variant of Unix (AIX, Solaris, HP-UX), or its clones like Linux (you know, Linux Is Not UniX, but it was clearly inspired by it). Even in the mass consumer market we can see Unix: Apple's operating systems for the iPhone, iPad, MacBooks... are all based on Unix.

And what to say about the C language. Until the advent of Java (not counting its variant C++) it was the most used programming language, present everywhere, in every Windows flavor and application, and it is still in good shape, very close to this first position as the most used programming language.

In summary, Internet, the IT industry, would not be as we know it without Dennis Ritchie. Thanks you Dennis.

Afterword: Of course Jobs did a great job spreading the technologies created by the Ritchies of the world, as I said. Just a couple of things about Jobs. A link to its famous phrase "good artists copy, great artists steal", by the way, stolen from Picasso...and probably misinterpreted too. And a recommendation of a movie, "Pirates of Silicon Valley". Maybe it is not a master piece, but shows very well, in my opinion, the beginnings and the personalities of Jobs and in a second level, of Bill Gates.