spam, too much of it

BlogsNow

Sunday, 2am Pacific, 5am Eastern, the blogosphere is active.
Hyperactive. Just that it’s all spam. Most of those so called splogs are hosted at googles blogspot.com aka Blogger.

Blogger is the biggest hosting outlet for real as well as for spam blogs. I find it very hard to believe that Blogger could not do more against spam blogs. They certainly have the technology. The six billion dollar question is, why do they let all this spam happen. First: They can. Google knows how to store vast amounts of data safe and cheap. Probably cheaper than anybody else. GFS and commodity hardware create an unbeatable combination that creates amazingly low costs per byte.
The second part of my explaination why 90% of all blogspot.com subdomains are junk is slightly more evil:

Blogger knows what’s spam and what is not. They know which IPs added which content. Those other weblogs that are not spam are an invaluable resourse. Blogger can tell it’s cousin googlebot where to go, and, more importantly where not to.
Others like Yahoo or MSN can not. They have to crawl and evaluate all that junk in order to find the gems created by all those real bloggers. This creates considerable costs for Googles wannabe competition.
“Don’t be evil” they said. Looking at a random blogspot blog these days it sounds more like: “Don’t be to evil to spammers”

ping

BlogsNow

A couple of months ago BlogsNow stopped accepting pings.
Pings are these little messages basicallly saying “hello Service XYZ here is weblog ABC, I have new content”.
The mother of all these ping servers is weblogs.com. Just got sold to Verisign.
I started to ignore pings, since they were mostly spam.
That did not stop them: Right now I am getting half a million pings a a day at BlogsNow. I would guess that 99.9 % of them are spam. Maybe I have a look one day and then use them to build a blacklist …

four million blogs

BlogsNow

BlogsNow had a little hickup. An old index issue finally prevented one of it’s main tables to accept new entries. The easiest remedy was to delete the last four million weblogs that had been added.
They will come back, I am sure.

no more ads

BlogsNow

Turned off the ads on BlogsNow. Don’t feel like it right now.

kassandra

BlogsNow history

This is what I was clicking on on Sunday 4:46pm PDT. Found it on BlogsNow. Which simply means that lots of bloggers had linked to this. Which means that the information was not only there, it also had been read, understood and repeated. Another link of that day was this National Geographic Article from last October.

During last years Tsunami there was allot of talk about the fact that the early warning systems in the Pacific knew about the event before people died.
This seems to be a reoccuring theme. To see that people die, because they lack something as cheap and ubiqitous (sp?) as information is not understandable. We only think that information is free and generally available. It is an illusion. How many bloggers where in the superdome?

Another angle to the same aspect of this very sad story:

Growing up in a front country of the cold war, we were aware of what was needed for which situation. One of the things we always had was a radio that was running on batteries. Even as a kid of 10 years I knew that this would be a crucial tool of information in case something should go wrong.
If only the population of New Orleans would have listened to battery driven radios. Could have. Would have. We hear that allot during these sad days.


URGENT - WEATHER MESSAGE
NATIONAL WEATHER SERVICE NEW ORLEANS LA
1011 AM CDT SUN AUG 28 2005

DEVASTATING DAMAGE EXPECTED

HURRICANE KATRINA
A MOST POWERFUL HURRICANE WITH UNPRECEDENTED
STRENGTH...RIVALING THE INTENSITY OF HURRICANE CAMILLE OF 1969.

MOST OF THE AREA WILL BE UNINHABITABLE FOR WEEKS...PERHAPS LONGER. AT
LEAST ONE HALF OF WELL CONSTRUCTED HOMES WILL HAVE ROOF AND WALL
FAILURE. ALL GABLED ROOFS WILL FAIL...LEAVING THOSE HOMES SEVERELY
DAMAGED OR DESTROYED.

THE MAJORITY OF INDUSTRIAL BUILDINGS WILL BECOME NON FUNCTIONAL.
PARTIAL TO COMPLETE WALL AND ROOF FAILURE IS EXPECTED. ALL WOOD
FRAMED LOW RISING APARTMENT BUILDINGS WILL BE DESTROYED. CONCRETE
BLOCK LOW RISE APARTMENTS WILL SUSTAIN MAJOR DAMAGE...INCLUDING SOME
WALL AND ROOF FAILURE.

HIGH RISE OFFICE AND APARTMENT BUILDINGS WILL SWAY DANGEROUSLY...A
FEW TO THE POINT OF TOTAL COLLAPSE. ALL WINDOWS WILL BLOW OUT.

AIRBORNE DEBRIS WILL BE WIDESPREAD...AND MAY INCLUDE HEAVY ITEMS SUCH
AS HOUSEHOLD APPLIANCES AND EVEN LIGHT VEHICLES. SPORT UTILITY
VEHICLES AND LIGHT TRUCKS WILL BE MOVED. THE BLOWN DEBRIS WILL CREATE
ADDITIONAL DESTRUCTION. PERSONS...PETS...AND LIVESTOCK EXPOSED TO THE
WINDS WILL FACE CERTAIN DEATH IF STRUCK.

POWER OUTAGES WILL LAST FOR WEEKS...AS MOST POWER POLES WILL BE DOWN
AND TRANSFORMERS DESTROYED. WATER SHORTAGES WILL MAKE HUMAN SUFFERING
INCREDIBLE BY MODERN STANDARDS.

THE VAST MAJORITY OF NATIVE TREES WILL BE SNAPPED OR UPROOTED. ONLY
THE HEARTIEST WILL REMAIN STANDING...BUT BE TOTALLY DEFOLIATED. FEW
CROPS WILL REMAIN. LIVESTOCK LEFT EXPOSED TO THE WINDS WILL BE
KILLED.

AN INLAND HURRICANE WIND WARNING IS ISSUED WHEN SUSTAINED WINDS NEAR
HURRICANE FORCE...OR FREQUENT GUSTS AT OR ABOVE HURRICANE FORCE...ARE
CERTAIN WITHIN THE NEXT 12 TO 24 HOURS.

ONCE TROPICAL STORM AND HURRICANE FORCE WINDS ONSET...DO NOT VENTURE
OUTSIDE!

new look

BlogsNow

BlogsNow has changed significantly. Now that I have more views active based on Version 2 I changed the design a little bit as well.
The update frequency has been changed to sixty seconds. Why not:
The new database design is pleasently fast.
I also droped the reference to “Version 2”. I certainly looks differently enough for people to realize that this is indeed new.

a sad occasion

BlogsNow history media

This is a sad day. The terror in London is on everybodies mind.

BlogsNow was written to be a fast reflector of what is going on in the world. Right now 64 out of the 100 links are related to the events in London. The top 17 links are all about it.

Here how other tools look right now:

technorati
people search allot for it, but the link list still focuses on yesterdays olympic nomination

blogdex
as usual blogdex has no clue, and it will be like this for a while

daypop
same here

BlogsNow
I wish I could have done this comparision with a more positive event.

3 hours later:
Server crashes. Again. Now I know that it is mysqlhotcopy when making a backup. While running the mysql repair I run out of disk space. All those bin-log files. Then I am stupid again and ctrl-c the repair. It would have waited for disk space. Then I tried to move the mysql data dir to another disk. Which takes some while. Then mysql does not want to start from that disk. Since I have no time for a dive into the manual I move things back and start it again. With the result that BlogsNow now shows results from 24 hours ago. Maybe I should learn something here?

Right now I like blogdex’ Version of the latest news much better. It is yesterdays. Yesterday was better than today.

the new news media

BlogsNow history

as BlogsNow fills with London links

gizillion blogs

BlogsNow internet technology

BlogsNow Version 2 started with a clean and new database. During it’s one year of operation Version 1 had seen close to 7 Million weblogs. BlogsNow follows ping lists like most other tools. These list became more and more a resource for spammers to inject their content. BlogsNow Version 2 jumped from three to almost four Million weblogs within one week. It turned out that two IP addresses alone had created 600,000 new ‘blogs’. All of them made just to spam whomever they can.

Many websites tracking weblogs will claim how many weblogs they track. It appears as if those 11 Million you find right now are actually an accumulation of all weblogs seen, regardless if active or not. And, at least a certain amount, of bogus blogs only created for spam should be takn into account.

Those inflated numbers are being used wherever people like to put an extra boost on the blog phenomen. There are definitely millions of blogs.
But the active blogging community may be just a few hundred thousand people.

BlogsNow view on the Zeitgeist

BlogsNow

Just added six more views to BlogsNow. Most of the Version 1 views are back now.
And then some.
see for your self