In the Wetware post last week on “A New Way to Fight Blog Comment Spam” I proposed methods that would prevent robots from posting comments. Kalsey commented that there are clear indications that many spam comments are actually posted manually, rather than by robots, rendering my proposed functionality obviously useless in these cases.
The day after Kalsey’s comment, I was reading a paper by Richard Gatarski called “Artificial Consumers: A Role for Computers as Subjects in Consumer-Related Marketing“. In the paper Richard makes convincing arguments that computers are in fact consumers in our world, as they – among other consumer characteristics – consume bandwidth, processing power and information and interact with humans and each other. Richard’s presentation slides give a quick overview of the main concepts.
These two accounts had me thinking about the role that robots play on the Internet. Even more than Richard I’m now not only convinced that robots should be considered consumers, but that they are arguable the most important consumers that visit many websites.
Robots are the visitors that find our sites in the sea of information on the Web, read them through and “spread the word”. Looking at the referrer log for Wetware I see that a large portion of my referred visits are coming from pages created by robots: search engines, automated blog listings, etc. Every day I repeatedly ask my favorite robot – Googlebot – where I can find useful information on various subjects, indirectly by using Google.
This said, I wonder whether we pay these visitors enough attention. I brought the subject up with my good friend Gudjon Mar Gudjonsson. During the last few weeks I have been helping him organizing a conference and he’s been very careful in designing the conference website so that it will be robot friendly. The methods he uses are in no way trying to exploit or bluff the robots. It is simply the correct use of meta tags, meaningful naming of files and folders and using the title tag and header tags for relevant information.
In Gudjon’s own words: “You need to think like a spider when you design your site”, and thanks to his spider-like thinking, our Digital Reykjavik conference pages rank high in searches for several of our speakers. Googling for Esther Dyson ranks the Digital Reykjavik site around #35 out of over 60,000 matches, and a Google search for Gordon Graylish puts his Digital Reykjavik profile (#1) even before Intel’s own page about him (#2).
Enough about that before you start thinking this blog entry is only to rank the conference pages even higher 🙂 The thing is that there are robot users out there and the bulk of them is there with good intentions and hence welcome to visit. In a comment to my aforementioned blog comment spam entry, Richard Gatarski envisions a robot that might search other blogs or sites for related content and post comments about it for useful cross referencing. Methods like captchas would prevent these friendly robots from making their message and discourage people from attempting to design such genius creatures, as just the hassle to find a work around for the captchas might be harder than actually making the robot do what it is otherwise meant to do.
In a similar manner – even though most spam mail is definitely sent by robots, that is not to say that we wouldn’t welcome emails from a robot with good intentions. In fact we often receive such mails, e.g. in correspondence with companies (it might not be Jeff himself that sends all the mails from Amazon that have his signature), and I would be more than happy to receive a friendly email from a robot that pointed out a spleling, grammatical or even factual error in one of my blog entries.
I wonder what can be done to serve both the needs of us humans to be free from the bugging bots and the rights of friendly robots to live a healthy social life? One way might be to have some sort of a certification body issuing robot IDs and a protocol that would allow a visiting robot to tell the web server “Hi, I’m Spelly the robot. Can I come in? If I misbehave you can send my number to RobbieSign and they will keep me on a short leash from now on”.