Category: technology

With INTERDUBS growing in the US solidly it was time to start to add another dimension to its growth. It is an interesting experience to go through the same motions again. Just on a different continent. Luckily we found a great data center partner. It is pretty cool these days that one can get a virtual test server within minutes. Of course we are building real machines again for the real install. After some research we found some great vendors: ISP Proshop served us extremely well for cases, cables and the like. We found that Alternate.de has very decent inventory in terms of high end server parts.

Some observations along the way:

Calling a vendor can mean that they already have your order on their screen. Before they pick up the phone (after the 1st ring). Caller-ID plus decent software makes this possible.

Ordering parts it can happen that they arrive 16 hours after you did so. Standard shipping. 5 Euros.

Payment is done electronically. Online, bank account to bank account. Securely, since you have a little key generating device. And no credit company sits in the middle, getting their 3 percent, just because banks didn’t get their act together.

Prices are horribly high. Almost 20% tax on top of things.

Another drawback: If you are fond of those plastic packing chips (who isn’t) you will come up empty. Crumpled recycled card board works just as well it seems.

So I wrote a script that will save me a minute. I pretty much assumed that I wrote it, just because I like writing code, and this task was just something that fit into the timeslot before dinner. I chalked up the twenty minutes it took as wasted time. Others check Facebook, I write a script that can be done before the next thing on the schedule.

As i said this one will only save a minute. But it will do so every day. Still no big deal, I thought. But -funny as it goes- it finished it a minute early, so I came to realize that I will have saved six hours after a year. Yes, in my head it takes takes 60 seconds to compute 365 / 60. Anyway: after two years I get one more day in Hawaii. That’s actually not bad at all for something squeezed in before dinner.

It also gets to show how bad we are actually in estimating what the impact our actions is. I didn’t start out to save a work day in two years. I simply had twenty minutes to fill and a repeating task that could be sped up. Guess I got lucky. Again.

“It usually takes about 12 to 18 months to build a new center,” she said. “We’re cutting that down to less than a year.”

from a NY Times article about Microsoft and Google and their respective data center operations.

It is interesting that after years in corporate culture people start saying this kind of thing and feel that there is nothing wrong with it.

The article poses the question whether google benefits from looking at each level of the technology stack and inventing where needs are. It does not come to a conclusive answer. I think it is rather obvious: Google was able to reduce its capex spending simply when it felt oppotune to do so. To my knowledge and own experience there hasn’t been any noticeable impact on the Google useability by this reduction in spending. I would guess google simple turned down the pace innovation while the influx of new equipment was slowing.

On a -in comparison- microscopic scale I experience the benefits of looking at the entire technology stack first hand. Part of what runs INTERDUBS is of the shelve, and other parts are enhanced, customized and severely optimized. Some we even actually build ourselves. We constantly look at the running service and identify room for improvement. Be it, in the user experience, or how efficient internals work. Having an understanding of the entire system on all levels lets us identify clearly where enhancements should be made. Each of these steps might only add a couple of percentage points. Having metrics and detailed information about all aspects of the system at all times not only give us visibility into which areas are to be tunesd and enchanced next. It also reveals, that all those little optimizations add up into a configuration ,that is faster by dimensions than the un-altered and generic one would have been.

Having this culture of change and constant optimization is allot of fun. I was plain scared having to do this on live data and a running service. But the goal was that INTERDUBS is available 24/7. And it turns out that technology – used in the right way – is able to do this now. It is literally flying the airplane and rebuilding it in the same time. You start in LA in a 707 and land in New York on a A380.

The main purpose of the API for INTERDUBS was to let my clients use it programmatically. Their system control INTERDUBS in a way that fits best into their workflow. That is what the API was made for, and it works. Every day.

Interestingly there are other, somewhat more surprisingly, benefits to having an Application Programmable Interface to a system too. This week I had a discussion with a client where they outlined specific needs in respect to their data that they have in INTERDUBS. They could have updated things manually, but that would have taken a very long time. By using the API myself it actually took me not that long to implement their needs. I spent only slightly more time on it than the actual discussion took that we had about their needs. This was very nice to see. And actually quiet unexpected. If the next client needs something similar I will be able to solve this in five minutes.