INTERDUBS support call: A client needs a folder for each US state and in there a specific set of sub folders. We don’t have that button!
What is awesome is that our clients just call before they start making something like this manually. There must be a better way. And of course there is.
17 minutes after the call the client had the desired list of folders. APIs are a nice thing to have. But even better are clients who call when things could
be improved.
INTERDUBS crossed the 1.5 Million file mark yesterday. Big numbers are hard to imagine. Simplifying matters a bit one can assume that each file at least replaces one DVD. If one would put those 1.5 million DVDs side by side on a shelf then it would be 14 miles long. It was a while back that I walked that distance. But I still remember vividly that it took a while.
Not using those DVDs saved 500 metric tons of CO2 as well.
sources:
INTERDUBS file counter
Until yesterday we had a perfect uptime history. 1,375 days online, and no interruptions. Today that changed: Starting from 9:16am PDT INTERDUBS.com was not reachable for 19 minutes.
Absolutely our fault. Of all possible scenarios it is actually our preferred one since we can fix it. We did and this problem will never occur again. Still sucks to having lost our outage virginity. All outages are avoidable, and so was this one. And -as usual- it was the lack of imagination that caused us to not see this coming.
What happened?
An upload with 550Mb/s triggered an automated protection system that took a network interface offline. It should have only impacted the one address using that amount of traffic. But it was doing it’s job wrong and shot in both directions: Rendering us unreachable.
This system is meant to guard INTERDUBS against malicious brute force attacks. Not a bad idea, if implemented right. 550Mb/s is of course still very very far away from the limit of our network capabilities. It was the volume and specific traffic pattern that caused the emergency shut off. We did neither envision nor test this specific condition. No two ways around this: Our fault. Embarrassing. Please accept our apologies (and money, if you like – see below for details).
We re-configured the system and are confident todays outage will never happen again.
Since we were unaware of the bug in the system configuration it took us a little while to identify the cause. Since the system is responsible for security we also had to spend a couple minutes testing the changes that then became the fix. 19 Minutes is a long time for an outage. Looking at what needed to get done to bring us back online we feel that we did OK. Not great, but OK. Of course there is also room for improvement, and we started to implement those changes today.
INTERDUBS overall uptime dropped now from 100% to 99.99904%.
For the month we are down to 99.95%. Well below the 99.999% we promise. None of our clients has to pay for INTERDUBS this month. If they choose so. A simple email is enough: we will discount the whole month.
Since counting nines of uptime is not really what most people want to think about, we decided that effective immediately we change our policy: we now guarantee 100% uptime. No longer ‘only’ 99.999%. If a client feels that INTERDUBS wasn’t there for them when they needed it, then they don’t pay. Simple as that.
This page documents all outages and service disruptions we ever experienced
This is Scott Salinas Music reel. It is an awesome reel. Scott used INTERDUBS for it. His company Critical Mass switched to INTERDUBS just a couple of weeks ago.
We are very happy to see how quickly they are able to make custom looks with our system. This one here, he made to match his personal site..
They save a bundle too we hear.
And -as so often- he has a point.
Just saw the Boards Summit opening reel:
They made a big deal about the QR Code.
But INTERDUBS clients can use QR Codes since January
We like it that way.
Back in the day an electron beam was running across the TV screen. NTSC was running with 30 and PAL with 25 frames a second. If the beam would go line by line the screen would flicker. The solution was, to let it run twice over the screen for each frame: Once for all odd lines (1,3,5 etc) and then again for all the even ones (2,4,6). That looked better. It is called ‘interlaced’. Each of these passes is a ‘field’.
Film cameras liked to run at 24 frames per second. Cinema does not flicker since each frames is shown twice, but that is not the point here.
When you have 24 fps footage and your TV runs at 30fps, what do you do? The solution was to insert a so called 3:2 pulldown to make 30 frames out of 24. This was done based on 60 fields to make it look smooth.
Interlacing is dead. There are no electron beams going over glass tubes to make images to speak of.
If you like to compress an NTSC spot that was shot on film, and that has the 3:2 pulldown in it, then you should go back to the 24fps version first. Since I could not find anything that worked I developed this. In 1998. Then, in 2008, I needed it again, and so I looked again. Much to my surprise, nothing really worked the way it should be. Many tools have the button to do an ‘inverse telecine’. But none detect cuts and deal with changing cadence patterns. So, I wrote it again. This time based on quicktime.
I decided to give it away: 32none is a free tool now.
Enjoy.
With INTERDUBS growing in the US solidly it was time to start to add another dimension to its growth. It is an interesting experience to go through the same motions again. Just on a different continent. Luckily we found a great data center partner. It is pretty cool these days that one can get a virtual test server within minutes. Of course we are building real machines again for the real install. After some research we found some great vendors: ISP Proshop served us extremely well for cases, cables and the like. We found that Alternate.de has very decent inventory in terms of high end server parts.
Some observations along the way:
Calling a vendor can mean that they already have your order on their screen. Before they pick up the phone (after the 1st ring). Caller-ID plus decent software makes this possible.
Ordering parts it can happen that they arrive 16 hours after you did so. Standard shipping. 5 Euros.
Payment is done electronically. Online, bank account to bank account. Securely, since you have a little key generating device. And no credit company sits in the middle, getting their 3 percent, just because banks didn’t get their act together.
Prices are horribly high. Almost 20% tax on top of things.
Another drawback: If you are fond of those plastic packing chips (who isn’t) you will come up empty. Crumpled recycled card board works just as well it seems.
I would get the New York Yellow Pages from audible if Jeff Heusser would read them. Sorry that do talk so much in this podcast.
So I wrote a script that will save me a minute. I pretty much assumed that I wrote it, just because I like writing code, and this task was just something that fit into the timeslot before dinner. I chalked up the twenty minutes it took as wasted time. Others check Facebook, I write a script that can be done before the next thing on the schedule.
As i said this one will only save a minute. But it will do so every day. Still no big deal, I thought. But -funny as it goes- it finished it a minute early, so I came to realize that I will have saved six hours after a year. Yes, in my head it takes takes 60 seconds to compute 365 / 60. Anyway: after two years I get one more day in Hawaii. That’s actually not bad at all for something squeezed in before dinner.
It also gets to show how bad we are actually in estimating what the impact our actions is. I didn’t start out to save a work day in two years. I simply had twenty minutes to fill and a repeating task that could be sped up. Guess I got lucky. Again.