About Us | Corante Research NEW! | Reprints | Advertise | Contact Us | Services


INDUSTRY NEWS today's headlines

Law & Policy
Personal Technology
Venture Capital


Instant Messaging

WEBLOG COLUMNS today's highlights

Amateur Hour
the "me" in media

The Bottom Line
the economics of IT

Brain Waves
neurons, bits & genes

nodes & networks

the politics of IP

Corante on Blogging
in media res

Got Game?
the future of play

creativity & innovation

In the Pipeline
drug discovery

Living Code
biology & information

social software

Moore's Lore
new technology

Open Mind
open source

About this site

Here we'll explore the various economic and financial principles that impact the business of technology, keeping up to date on the various ideas, theories, trends and numbers, dispelling the silly buzzwords, slogans and fads and generally trying to understand how recent developments affect this industry going forward and may help divine what's going on and where things may be headed. Among the topics we'll touch on: regulatory issues, intellectual property, network effects, the general economy, productivity and more.

About this editor


Arnold Kling has a Ph.D. in economics from MIT; founded homefair.com, one of the very first commercial websites, in 1994; separated from Homefair in January 2000 after it was sold to Homestore; is author of Under the Radar: Starting Your Internet Business without Venture Capital

and is an essayist. Please send any comments, as well as suggestions for what we might point to from this page, to us at econ@corante.com

Corante.com Navigation

About Us
Contact Us
Press Coverage

Advertising Info
Corante Research
Buy Reprints
Custom Services

Privacy Policy
Terms of Use

powered by WebCrimson

THE BOTTOM LINE: the economics of IT

By Arnold Kling

Posted Friday, February 28, 2003

VC-funded business vs. Mom & Pop

Zimran Ahmed writes

Here's something I recently learned in school: venture capitalists hate to go up against Mom-and-Pop businesses. They hate it because Mom-and-Pop businesses are tough as nails, and will refuse to exit a market long after purely financial companies have pulled the plug.

It's no fun competing against a nonprofit.  In the 1990's, it was the VC's funding nonprofits, and it was those of us who were self-funded who felt like we were screwed.

In fact, mom-and-pop businesses typically are in markets with low ambiguity and VC's go into markets with high ambiguity, so they don't go up against one another.  See Amar Bhide, The Origin and Evolution of New Businesses.

All this B-school jive is being brought up in the context of the Google-Blogger merger, an event whose cosmic significance eludes me. 

But then, I could just be slow.  I still have not figured out what all the fuss over the AOL-TimeWarner merger was about.

. . . . . .

No-demand Computing?

That's was the way I ridiculed IBM's "On-demand computing" concept. I still think that only a moron would buy it.  However, today, news.com reports

IBM's Global Services Division will help the Paris-based company consolidate its computers and data storage equipment, and eventually will make arrangements for AXA to purchase computing power and storage capacity on a per-unit basis as those resources are needed, IBM said in a statement...

The contract is similar to IBM's $2 billion deal with auto-parts maker Visteon, announced earlier this month, and a $5 billion contract with J.P. Morgan Chase, made public in December.

Looks like I was wrong.  I should have remembered the old cliche.  No corporate information officer ever was fired for being a moron.

. . . . . .

Non-Bayesian Spam Filtering

Here is a report by Jon Fortt on a non-Bayesian spam filter.

MailFrontier faced off against rival spam-blocker Cloudmark in an anti-spam challenge at Demo, and MailFrontier won, hands down -- it blocked 85 percent of spam, and only 1 percent of the time categorized a legitimate e-mail as junk mail.

That performance is unacceptably bad.  If you watch a spam filter classify 1 percent of your legitimate email as spam, you would soon realize that you would never use it. 

With Bayesian filters, like PopFile, you can get much lower rates of both Type I and Type II errors.

. . . . . .

Privacy, Security, and Architecture II

A friend of mine who works as a lawyer at a trade association complained that during an internal email discussion of how to respond to an email from a hostile consumer group, someone accidentally forwarded the internal email to the consumer group.  He was frustrated that there was no technical solution for this.

My first reaction was to send my friend a link to James Grimmelmann's piece on accidental privacy spills, which argues that there is no technical solution. 

However, it seems that where there is a will there is a way.  Microsoft has an idea for embedding rights management into email.  The kicker, though, is that it is not compatible with anything currently in use.

[it] relies on the proposed Extensible Rights Markup Language (XrML) standard, an XML-based language that is heavily backed by Microsoft but has yet to attract broad industry support. While Office 2003, Microsoft's Office update scheduled for mid-2003, supports XrML and will work with RMS, older versions of Microsoft Office, including the currently available Office XP, won't work with the technology.

For some reason, I get this picture in my head of how this project was pitched to senior management:  "with this technology, your email could not be read by an adversary in an anti-trust proceeding..."

. . . . . .

Posted Wednesday, February 26, 2003

Warning to Suits

This is what the Geeks think of you.  (Pointer courtesy of Doc Searls)


. . . . . .

The Star-Maker Machinery

Brother Jolliffe gave me this pointer.  I have no idea how accurate it is.  It takes a hypothetical band that has a Gold record album--half a million copies sold.

Royalties $764,100
Band's recording cost $150,000
Promotional Video $100,000
Producer $203,760
Manager $46,551
Lawyer/Agent $101,880
Leaves the Band $161,909
The story's kicker is this:
That's not bad money, but it's split four ways, or $40,477.25 each, about the same as a city sanitation worker with two years' experience, without health benefits, vacation and retirement fund. But with, of course, groupies.
I'm a little surprised that the music publisher doesn't pick up the recording cost or the cost of the promotional video.  And it seems to me that the producer's take is a little rich.  And if it were book publishing, the lawyer-agent would be getting a share of the "band's" net, not of the gross.  But the basic point that bands do not necessarily get rich from big albums is probably right.
Still, I think that bands will miss the star-maker machinery when it's gone.  In ten years, bands will be looking back with nostalgia to the days when a successful musician could earn as much as a sanitation worker.

. . . . . .

Data Mining Abuse

As you know, I am in favor of using data mining to deal with terrorism

On the other hand, because I believe that data mining is a powerful weapon, I want to see its use carefully regulated, legislated, and limited.  When I see a story like this, about credit card databases being used to locate people who buy child porn, I am ready to march with the civil libertarians. 

I feel that as a parent of three daughters, I have all the tools I need to fend off child pornography.  I don't need the credit card industry to snoop on its own behalf or on behalf of the government. 

I believe that this country needs a reasoned debate on the use of data mining.  As long as people simply march under banners ("absolute privacy" on the one hand or "stop child pornography" on the other), we will get the worst possible outcome.  We will see data mining used to embarrass people who look at the "wrong" web sites, and we will see it not used to connect the dots to prevent terrorism.

. . . . . .

Posted Tuesday, February 25, 2003

Bayes' Theorem and Homeland Security

I try to explain Bayes' Theorem here.   I think that Bayes' Theorem is something that everyone should try to grasp, even if you do not buy my thesis that it is an argument for data mining as a method of fighting terrorism.

. . . . . .

Spam Filtering Update

At this point, all of the Type I errors (when Popfile classifies legitimate email as spam) are graphics-heavy web-generated group emails.  For example, I belong to a book club where for some reason they use Evite.  This generates a spam-like email.  I'm sure that Popfile will learn to accept these emails, but my guess is that the cost will be a lot of type II errors--classifying spam emails as good, because they look like Evite-generated email.  One more reason for my book club to not use Evite--it's a time-waster, anyway.

Anyway, I'm just about to the point where I could use Popfile for all my email sorting.  I continue to have to disable the process "POP3trap" which a reader told me is part of Trend Micro P-cillin anti-virus software, not part of PopFile.

. . . . . .

Posted Monday, February 24, 2003

Moore's Law Reading

Since our economy is pretty much dominated by Moore's Law, it pays to be informed about it.  So this article by Jon "Hannibal" Stokes is worth study.  Pulling out one tiny quote:

Shrinking transistor densities are what give you smaller, cheaper, and lower power chips with the same amount of performance as their larger, more expensive, more power hungry predecessors. So a company that decides to install a rack of clustered "blade" servers in place of one monolithic piece of server hardware is by no means "opting out of Moore's Law." They're simply choosing to take advantage of it in a different way. 

Well, maybe one more tiny quote:

It isn't that people really want less computing power--they just want less of it in one place. Instead of a mammoth 200 million-transistor CPU on their desktop, they'll want 500 million transistors spread out over a laptop, a desktop, a PDA, and some type of wearable. Furthermore, they'll want all 500 million transistors to be as inexpensive and energy efficient as possible. This way of exploiting Moore's Law, then, brings us back to Moore's original vision of cheap, ubiquitous computing. 

Thanks to TechCentralStation.com's Nick Schulz for another great pointer.

. . . . . .

Differing with Bill

An interesting interview with Bill Gates.  I'll pull out this quote:

We call it the PC-plus era. And the reason is that the role of the PC is still quite central. You know, when you want to do your homework or fill out your tax return or really see what the news stories are, that much visual area is very beneficial...

So particularly for document creation you know when you get small like this we're all getting creative about small keyboards and little handwriting things but creativity really needs the full-screen device.

...Now, there's intelligence in the digital camera. And, yet, even in a rare case, you might take a photo and directly send it to the printer going around the PC. But in most cases you want to organize it, save it, send it off to various people and so then you really do want the PC to get involved. So the more you have digital cameras in fact, the more you have these things (holding up a Pocket PC) the more important the PC becomes.

It might work out that way.  But I could also see an alternative scenario, in which what is coming is not the PC-plus era, but the remote-control era.  In the latter scenario, you carry around a remote control that interacts with lots of devices out there, including storage (so you can transfer your photos directly to network-based storage, rather than going home and loading them onto your PC). 

As I've said before, I think that the remote control is not a big-screen device, but my guess is that it needs to have a bigger screen than what you see on a cell phone.

. . . . . .

Economics of Intellectual Property

Zimran Ahmed captures the intuition of many economists.

Since the Internet has lowered distribution and reproduction costs, bad IP laws are more costly now than they were in the past. 150 year copyright terms just didn't matter much before Napster. A new IP regime needs to understand the economics of production for different types of ideas and tailor the right laws for the right circumstances. Personally, I'd like to see shorter copyright, no patents on business processes or software, and longer patents for drugs.

That intuition seems right.  It's a bit harder to spell out exactly the economic factors behind it.

For more, see my other blog.

. . . . . .


Sister Wentworth cites an article by Declan McCullagh, who says that Geeks should stay out of the lobbying game, except to try to repeal some bad past legislation.  She disagrees, asking rhetorically,

Will the invisible hand plus an uphill battle--or many--to repeal sections of current & future dinosaur-industry backed laws truly be enough to protect fair use and innovation in the digital sphere? Or is this simply wishful (libertarian) thinking?

My prediction is that the Geeks will win.  They will treat the Suits as damage, and route around them.

And the riff that she quotes from Dave Winer is not the right argument to make.  He is saying that the music industry is immoral because artists are not well paid. 

Mark my words.  Ten years from now, today's musicians will look back on this period as a golden age for compensation.  The decline of the legacy music industry is going to make it harder, not easier, to earn a living in music.  The breakup of the star-maker machinery will increase our national well-being, but the social benefit will accrue to consumers, not artists.  See Equilibrium in the Market for Rock-n-Roll.  (UPDATE...or see this piece from Michael Wolff, which brother Jolliffe forwarded with some reluctance.  It sort of says what I've said in this post.)

. . . . . .

Posted Sunday, February 23, 2003

Privacy, Security, and Architecture

It's unusual for a long Web essay to be worth reading in its entirety, but set aside some time for this piece by James Grimmelmann.  He meditates on an "accidental privacy spill" in which a journalist's private summary of a conference gets forwarded and winds up onthe web.  A sample nugget from Grimmelmann:

people make secure systems insecure. Not out of malice, or even out of laziness. People make secure systems insecure because insecure systems do what people want and secure systems don't.

Another interesting point:

On a technical level, privacy and copyright are isomorphic problems. Information is to be shared with certain people and not with others.

Grimmelmann argues convincingly that there is no good technical/legal solution to ensure email privacy.  The isomorphism he describes would imply that there is no technical/legal solution to deal with copyright.  Sic transit gloria Creative Commons.

Thanks to Ed Felten for the pointer, although sister Wentworth also cites it.  So you may have read it already.  If you haven't, do so now.  I give it 5 stars.

. . . . . .

Web-based Bayesian Spam Filtering

If you're one of those folks who likes to use a web-based email program, then you might look at Oddpost.  They claim to have implemented Bayesian spam filtering.  If you have tried it and care to report on the results, let me know. 

Meanwhile, for the folks who use client-based email, I'll continue to update you on my experiences with PopFile.

. . . . . .

PopFile anti-spam update

I'm having pretty good luck with PopFile.  In any given day, I might get 10 legitimate emails, with PopFile tossing one into the spam category about every four days.  I will get about 70 spam emails, with PopFile tossing one of them into the "ok" category about once a day.  So type I error rate is about 2 or 3 percent (still too high) and type II error rate is less than 2 percent (much better than with rule-based filter.)  In fact, the type II error rate is so low that when my rule-based filter says that something is spam but Popfile disagrees, I open the mail to see what is going on.  Today, for example, Popfile accepted an email that was unsolicited but was a notification about stuff that I really wanted to read!

I think that the reason that the Type I error rate is so high is that I don't get enough legitimate email to give it sufficient examples yet.  Maybe in a few more weeks...

Also, email from mailing lists poses an interesting problem.  If you subscribe to an email list, chances are a lot of the mail you get from the list does not interest you.  Yet it is not spam.  It would be interesting to try to train Popfile by setting up separate buckets for this sort of mail.  I have not done that, and instead I tend to unsubscribe to lists that for me have a high noise-signal ratio.  But it is possible that by creating more buckets in Popfile (right now, I just have two--legitimate email or spam) I could subscribe to email lists with a high noise-signal ratio and train Popfile to find the signal.

. . . . . .

Mooning Microsoft

Somebody coined that term to describe Netscape's bluster, which ultimately proved to be self-defeating.  And now we have Jonathan Peterson's interview with Marc Canter.

Oh God dude - these people are so FAR from that - it's ridiculous!  They might have researchers like Nathan Myrvold worrying about that - but it's almost like they HAVE to apply for patents, do futuristic stuff, etc. - because they need to spend the R&D dollars.

All they're ever gonna do is let others launch new ideas in the market and wait till that technology matures - and then do their thing to claim it as their own.

Oh God dude - let your product speak for you.  I'm from the old school that tends to think of trash talking and real accomplishment as being inversely related.

. . . . . .

Posted Friday, February 21, 2003

How to Free Spectrum

If you think that the answer is a wireless "commons," think again.

imagine what might happen if a company like Intel could buy a large swath of spectrum. In the next decade, Intel would like to sell lots of software-defined radios, which are a critical component in spread-spectrum wireless. As owners of spectrum, they would want to make sure that software-defined radios could achieve their maximum potential, which in turn would mean that Intel would make its spectrum available for spread-spectrum wireless use.

. . . . . .

Posted Thursday, February 20, 2003

Suits 3, Geeks 2

That's the way I read the outcome of the FCC action on local phone "competition."

As you probably know, Michael Powell was on the losing end.  He wanted to free the Baby Bells from state regulators who can impose arbitrary wholesale rates that the Bells must charge "competitors" for use of the phone lines.

What the state-regulatory system means is that a company that does not have a single engineer can enter the phone business.  Once they run the regulatory guantlet, they can sell phone service over the local phone company's lines.  This does reduce retail phone rates, but not as efficiently as simply regulating retail phone rates directly.

It is a "win" for federalism in the same sense that localizing cable TV regulation is a win for federalism.  In both cases, it makes lots of work for lawyers, lobbyists, and local officials, with abundant opportunities to corrupt the process.  It stimulates a revolving door (or revolving trough) for influence-peddling Suits, as they go from government to the semi-private sector and consulting.

Whether the Bells end up winners or losers is unclear.  They have their own Suits, who know how to work the fine print. 

I don't think that ordinary consumers had a horse in this race.  Our best hope is that wireless Geeks will be able to route around the Suits and introduce genuine phone competition.  Let's hope that Powell's colleagues on the FCC don't try to stop that one.

UPDATE:  I agree with this comment from brother Blankenhorn:

The Bells were given an invitation to suicide. Build new fiber, they were told, and you own full control of it. But technology won't wait for fiber construction.

Given that the last mile will be wireless, anyone who builds out fiber-to-the-home today would be nuts.

. . . . . .

Posted Wednesday, February 19, 2003

Suits and Geeks

I think Sarah Stirland captures the "out of it" feel of a room full of suits.

Fortunately, the geeks treat the suits as damage and route around them.

. . . . . .

Michael Malone, Troll

When a blogger engages in a wild rant on a popular subject, Slashdotters call it a troll.  Michael Malone does a troll with this Jeremiad on Moore's Law.

He blames Moore's Law for the dotcom bubble, which is ridiculous.  The dotcom bubble was not a technology miscalculation.  It was a business miscalculation.  What I call the McKinsey business strategy--regardless of the question, the answer is to sacrifice profits for market share--became the winning recipe for an IPO.

Malone suggests that consumers will not pay more for better chip performance.  But that was never the point of Moore's Law.  Moore's Law was that the most economical chip density would double every eighteen months.  People will find that achieving a given level of performance will be less expensive using the latest chips.

In theory, consumers could be satiated at a given level of performance, so that they do not want to upgrade to the next chip.  However, I think that practical applications of Bayesian filtering will give consumers a reason to want that upgrade.

Thanks to Nick Schulz of TechCentralStation.com, for pointing me to Malone's article.

. . . . . .

Posted Tuesday, February 18, 2003

Bayes is Cool

I was really upset a few years ago when my daughter's textbook for AP Statistics did not include Bayes' Theorem, which is an elementary statistical theorem that is one of my all-time favorite mathematical equations.  If a statistics text ignores Bayes, that is a problem.

But as articles like this illustrate, Bayes is cool nowadays.

The technology will be embedded in future Microsoft software and is intended to let computers and cell phones automatically filter messages, schedule meetings without their owners' help and derive strategies for getting in touch with other people.

The article's explanation of Bayes' Theorem is way off.  The article says that Bayes said that you use the past to predict the future.  That's not what the theorem is about. 

Bayes' theorem says that you can sort spam email (for example) by looking not just for phrases that are commonly associated with spam, such as "enlarge your penis."  Equally important is the absence of phrases that are commonly associated with legitimate email, such as "following up on this morning's meeting."  It is by scoring an email's correlation with past legitimate email as well as past spams, and comparing the two scores, that Bayesian email filters become very powerful. 

. . . . . .

Spam Filter Update

I figured out how to get POPfile, the Bayesian spam filtering program, to work reliably on my Windows XP machine.  The problem is that when I start my computer, a process called POP3trap.exe starts running which seems to convince POPfile that it does not really need to start up properly.  So when I start my computer, I give it the old 3-finger salute, shut down POP3trap.exe, and then start POPfile.  Then I start my email program, and things work fine.

At this point, POPfile has seen about 250 spam emails and fewer than 50 good emails.  It seems to make no Type II errors (classifying spam as good mail), but it still makes an occasional Type I error (classifying good email as spam).  Assuming that after another 100 or so good emails it stops making any Type I errors, it will have lived up to my high expectations for Bayesian spam filters.

To paraphrase Bruce Sterling William Gibson, Bayesian filtering to overcome spam is here.  It's just not widely distributed yet.

UPDATE:  Thanks to Zimran Ahmed for correcting me--I always get Gibson and Sterling mixed up on that quote!

. . . . . .

Posted Monday, February 17, 2003

More Toll Roads

I can safely predict that one of the important applications for "the era of sentient things" will be more toll roads.  It looks like London's camera-enforced congestion tax is off to a good start.

There were complaints and small demonstrations around London, but traffic was lighter than normal and did not appear to back up around the edges of the restricted zone, as expected by opponents of Mayor Ken Livingstone's charging plan.

My guess is that "easy-pass" systems are getting very inexpensive to operate, and we will see a lot more toll roads in the U.S. over the next five years.

. . . . . .

Posted Sunday, February 16, 2003

Microsoft's Challenges

David Stutz lists every conceivable challenge facing Microsoft:

(1) The shift from PC-centric computing to Internet-based computing

(2) Open-source software

(3) New hardware

I think that (1) and (3) are the important ones.  In what Howard Rheingold calls "the era of sentient things," Intel will flood the world with chip radios.  It seems unlikely that these will depend on Microsoft software. 

I think that the new battle is for the hiptop.  What device are people going to carry with them to interact with the sentient things that start to permeate everyday life?  Will it be a cell phone, as Rheingold believes?  Will it be a Tablet PC?  Will it be something in between? 

My guess is that battery life will prove to be an important issue, which puts the optimal hiptop device closer to a cell phone than to a Tablet PC.  Which means that Microsoft does not have any particular leverage going into the battle.

. . . . . .

Posted Friday, February 14, 2003

Thinking about Security Databases

People who want to apply their brains and not just their knees to the issues of government use of data mining for security purposes might want to read this paper by Paul Rosenzweig.

Ultimately, it is not persuasive to argue that the prospect of increased efficiency and increased governmental power is so great a threat to civil liberties that all efforts toward an enhanced capacity for information fusion should be abandoned.

Rosenzweig's opponent in a forthcoming debate will be Timothy Lynch, who writes,

We can either retain our freedom or throw it away in an attempt to make ourselves safe.

In my view, we can retain our freedom without restricting the intelligence of those who are trying to prevent terrorism

UPDATE:  Instapundit quotes with approval on the topic of nanotechnology, "I don't want the science to slow down. I want the ethics to catch up."

That's the way I feel about using databases in national security.  I don't want the science to slow down.  I want the ethics, in the form of better auditing and checks and balances on government use of the information, to catch up.

. . . . . .

Identify People, not Devices

I agree with Eric Norlin that digital identity is an important topic, and I'll pass along his pointer to this article.

the Commerce Department recommended that the United States participate in an emerging electronic numbering system, known as ENUM, that will allow people to use one identifier for many different purposes, including mobile phones, e-mail, instant messaging and faxes.

Right now, identity in the Internet and the broader telecom sphere is device-centric.  Phones have numbers.  Computers have IP addresses.  People are anonymous, and they have to work constantly to identify and re-identify themselves.

One can imagine instead a world in which identity belongs to people, and the devices always recognize the people.  In some ways, this will turn things upside-down.  Instead of dealing with the problems of anonymity (hard-to-trace spam, hard-to-use electronic shopping systems), we will be dealing with the problems of non-anonymity, including privacy protection.

. . . . . .

Posted Thursday, February 13, 2003

The High Cost of Free Software

As I mentioned, I am trying to use PopFile, an anti-spam program that you download to your machine to act as a front-end (or proxy server, to be a bit more technical) to your email program. 

The documentation is terse, and I feel that it was sheer luck that I figured out how to even start the program.  Other than that, set-up was quick and painless.

I would say that if you can get PopFile to work reliably enough, then you *definitely* should use it.   The idea is brilliant, and it's pretty nicely done. I've barely started to train the Bayesian filters, and they already work better than the rule-based filters that I put into my regular email program (although I can keep using my old filters without having to make any changes, so for now I have two layers of filter going).

Because I hate spam and I love statistical filtering, I will keep at it with PopFile.  But I have to say, if it were anything else, I would have ditched it.  But the program's operation is only intermittent.  If I click on "check mail," over half the time I fail to get a connection all the way through PopFile to my Internet Service Provider.  The net result is that checking mail has become much more time-consuming, rather than less.

To me, this illustrates one of the costs of open-source software.  Proprietary software gets tested on popular configurations, and most of the bugs get worked out before it gets released. 

Proprietary software has bugs, too, as we all know.  But my experience has been that the time that I spend trying to install, configure, and overcome bugs with proprietary software is less than the time I spend doing those things with open source software. 

In the contest between open source and proprietary software, I'm in the camp that says "there is room for both."   In this case, I think there is room for a proprietary solution.

The developers of Popfile definitely have my thanks.  But I'd rather give them money--and get a reliable product in return.

. . . . . .

My Day

I went to a conference on spectrum regulation.  The main thing I got out of it was a photo of Kevin Werbach and Sarah Lai Stirland.  Your honor, I believe that this picture definitely will prove that I did *not* punch Ms. Stirland in the face.

Other than that, Sarah and I left the conference wondering what it was all about.  Maybe she'll post more.

In my TCS essay today, I suggested--and I'm serious about this--that government officials should be blogging.  Maybe Michael Powell will take the bait.

Finally, I've been trying to work with Popfile, a Bayesian spam filter.  More comments on that shortly.

. . . . . .

Posted Wednesday, February 12, 2003

Data Mining or Duct Tape

This week, there was a run on duct tape, as Americans tried to protect themselves in an environment in which we do not know who among us is a terrorist or where they might strike.  Meanwhile, as Heather MacDonald points out, the campaign against the attempt to use database technology to fight terrorism is roaring ahead.

Data-mining is not an "untested and controversial intelligence procedure," pace Senator Russell Feingold, who has introduced a bill to ban it. The Treasury Department's Financial Crimes Enforcement Network has long used data-mining to search for evidence of money-laundering, and is now charged with searching for terrorist financing networks. The technique also aids the early detection of infectious disease epidemics through searches of hospital databases. 

I would add that data mining is the basis for the efficient provision of consumer credit in this country, including home mortgages.  It is the basis of the most effective email spam filtering techniques (commonly called Bayesian filters).

The knee-jerk hostility to data mining technology is winning for now.  But Senator Feingold and his colleagues are not completely against using technology to fight terrorism.  As far as I know, nobody has introduced a bill to ban duct tape. 

. . . . . .

Posted Tuesday, February 11, 2003

Coke Goes Flat

Another one of my blog siblings (I still owe him a welcome punchTM) found a speech by a Coke advertising executive that was idiotic, and dutifully reports it as such.  Basically, the adman's idea is to promote synergy by linking his successful brand with a bunch of media industry losers.  Like that strategy worked so well for AOL.

. . . . . .

Telecom Needs Decontrol

Now that Corante has a real journalist covering the telecom arena, among other things, I feel less obliged to report the news and more free to just be cranky.  In one of today's posts, my sister blog points to a speech by FCC RINO Kevin J. Martin.  He talks about "new fiber."  That rang a bell with me.  An alarm bell.

When I was just a lad working as a economic research assistant, we were in the middle of a government-run oil market, where "new oil" was to be priced differently from "old oil," among other things.  It was brilliant in theory, and unworkable in practice.  As a result, the United States appeared to be in an "energy crisis," until President Reagan dismantled the regulatory regime and let the market sort out the oil situation.  

Michael Powell wants to do for telecom what Reagan did for oil.  Kevin Martin still thinks that regulatory agencies can fine-tune a superior outcome.  But government-managed competition of the type they are attempting is once again beguiling in theory and nightmarish in practice.  Mr. Powell is the the one to root for in this "mudwresting match." 

(RINO stands for "Republican In Name Only," as apparently Mr. Martin intends to vote with the Democrats on the FCC.)

. . . . . .

Non-Bayesian Spam Filtering

Kevin Marks of Mediagora emails me with a question about Bayesian spam filtering:

does it have to be Bayesian? The latent semantic mapping OS X mail uses works great for me.

See what Tim Oren says about it:


The Tim Oren piece is very good, and I recommend visiting the link. 

I am not at all dogmatic about using the Bayesian formula.  There are lots of good statistical techniques out there.  I'd recommend picking a technique that is computationally rapid and implemented with a no-brainer user interface, even if it "inferior" by some theoretical argument. 

All statistical techniques are subject to the problem that they could degrade over time as spammers change methods.  But that is more an issue for spam filtering at the server level than at the individual level.  My individual spam filter is less likely to degrade, in part because the cost/benefit to the spammer of reverse engineering my personal filtering weights is prohibitive.

So I would like to see a combination of server-based filters (to take the load of spam off of ISP's) and individual filters (to make it cost-ineffective for spammers to try to evade filtering).  The server-based filters might be rule-based rather than statistical, to save on computation. 

Or is there a way to change Internet email from a client-server application to a peer-to-peer application?  If that were feasible (and I scarcely know what I am talking about), it might make it easier to allocate resources to filtering spam at the network level.

. . . . . .

Posted Monday, February 10, 2003

Electrical Grid Broadband:  Panacea or Kludge?

Reading this story, I couldn't tell.  It quoted some some enthusiastic supporters, but then deeper in the story it said this:

The technology works like this: data is carried either by fiber-optic or telephone lines to skip disruptive high-voltage lines, then is injected into the power grid downstream, onto medium-voltage wires.

Because signals can only make it so far before breaking apart, special electronic devices on the line catch packets of data, then reamplify and repackage them before shooting them out again.

Sounds difficult and expensive.

. . . . . .

Another Music Industry Revenue Model

Nick Schultz of TechCentralStation.com pointed me to this story

Clear Channel Concerts, the nation's largest concert promoter, has ambitious plans to record live CDs of its shows and sell them to patrons within five minutes after those shows end.

Think about this in contrast with standard music publishing.  Instead of publishing and distributing CD's to retailers "on spec" (hoping that they will sell), you would be publishing CD's to order.  Note also that the publisher would not be one of the traditional music publishers, which is likely to raise all sorts of conflicts within the industry.

. . . . . .

The Language Barrier

If the Internet creates trade in information services, I argue that this may create a trade barrier involving language.

For now, the Internet revolution is boosting the economic prospects of the English speakers of the world. This includes the countries where English is the native language, as well as the people in other countries who happen to be educated in English. People who never learn English may be destined to spend their lives on the wrong side of the language barrier.

. . . . . .

A Music Industry Model

David Strom passes along a college freshman's economic model for the music industry.

if the record companies are going to succeed at their own P2P network, they have to offer music at a rate lower than the retail market. This will move people away from the old way of buying music and
force them into doing it the new way through the simple laws of economics. But they are saving the costs of moving physical product, so it is a win-win situation. The record companies have to adjust their profit projections, however, in order to bring their costs in line with what people will be willing to pay.

My recommendation is to charge a fee somewhere in the range of $5 to $20 a month for use of the network -- in other words for all you download and not per song. This is the right price point; about the cost of one CD in today's market and within the threshold of pain for most members of my generation.

I actually think that the record companies could earn higher profits with this kind of subscription model, because of high volume, low costs, and the possibility of add-on services.

. . . . . .

Future of Books?

Prashant Kothari talks about the future of books and book publishing.

eBooks are great for non-linear content (reference works, scholarly and academic texts, religious books and such like), for PDA-type reading material (training manuals, business and self-help books) but are still very much a work-in-progress for works of fiction (at least, for the average reader).

I think that the overall trend is for people to read more sources, and less from any one source. 

I think that a lot of traditional nonfiction books will die, because it takes the authors too long to convey their ideas.  Even books that I recently read that I would recommend highly, such as Steven Pinker's The Blank Slate, Howard Rheingold's Smart Mobs, and Joel Mokyr's The Gifts of Athena, all suffer from being considerably longer than I would have liked.

Increasingly, I see books as a free-time indulgence.  Reading a book is like going to the beach (and those activities are indeed linked together in many people's minds).  But "practical" reading involves much shorter pieces.

So I think that e-reading will take off.  But e-books won't.

. . . . . .

Posted Saturday, February 8, 2003

Corante Connected

In the spirit of "Corante is a Big Tent" (i.e., they even let me in), let me welcome Sarah Lai Stirland by seasoning her opening connected manifesto with a bit of bottom-line cynicism.

All entities in all sectors of a society should be able to communicate directly with each other without external control.

If you believe that, I've got some spam mailing lists I'd like to sell your name to.

Information has a social function. It should not be viewed as propaganda nor a commodity, nor should it be controlled by the power structures of the market or the state. Information should contribute to reducing ignorance and preconceptions.

Here is my idea of reducing ignorance and preconceptions.

Communication should occur through horizontal mutually beneficial exchange of information, not through vertical transfer from those who have control of a medium to passive receivers.

Not everyone would agree with this bald-face boldface assertion.  For example, see Audiences scale, communities don't.

Technologies should be reviewed as to their potential impact on a society and power structures within them.

Reviewed by whom?  By some unelected international bureaucrats, as in the EU or the UN? 


Let the market review technologies.  To paraphrase Churchill, the market is the second-worst review mechanism.  As for the worst, all the others are tied.


Let me close by saying congratulations to Corante on landing its newest blogger.  I plan to be a regular reader.

. . . . . .

Posted Friday, February 7, 2003

Where is the innovation in email?

Consider this rant from Bob Frankston.

Simple, and anonymous over-the-wire encryption of email would allow us to assume the kind of confidentiality we associate with first class mail (in the US). This is a standard far below the high security systems which try to keep mail encrypted even when stored locally and which also attempt to provide authentication. We simply want to keep the messages from prying eyes. We also need a delivery confirmation mechanism which doesn't make the recipient feel spied-upon. 

Email is the most popular Internet application, and yet innovation is incredibly slow.  I would pay for an email client that uses Bayesian filters.  I agree with Frankston that a feature that sent a notice back to the sender that the mail passed the Bayesian filter would be great.   

. . . . . .

Rheingold Again

Some days, I feel like this blog is pointless.  All you have to do to keep up with current technology is go to www.smartmobs.com.  Meanwhile, I'm just passing along Howard Rheingold's pointers.  Here's another one.

Researchers around the world are working on ways to create mid-air messaging systems that let you post or read comments people have left tied to a particular location.

The "mid-air" messages will hang in the air until someone walks past carrying a device, a phone, handheld computer or laptop, capable of receiving them.

As colleague Dana Blankenhorn points out, small, inexpensive radio chips are really going to have a big impact.  It will take several years, but it is a big deal.


. . . . . .

Posted Tuesday, February 4, 2003

Identity and the Future

One of Howard Rheingold's favorite phrases is "How to recognize the future when it lands on you."  Eric Norlin is trying to help.  He surveys activity on the part of various levels of government in the area of improving digital identity.

The end result: the government becomes a hotbed of innovation for massive identity infrastructure -- in a way that only the government can.

The reason: like the massive project that is space travel, identity infrastructure on this scale will require the bureaucratic structure of a government program to survive its developmental phases...

Thanks to Doc Searls for the pointer.

. . . . . .

Posted Monday, February 3, 2003

The Joys of Telecom "Competition"

From Joel on software:

Much as I hate to encourage monopolistic local telcos, the only thing worse than dealing with a local telco directly is dealing with another idiot bureaucratic company who themselves have no choice but to deal with the local telco. Our next office space will be wired by Verizon DSL, thank you very much.

This is what happens when "competition" has no infrastructure and instead adds nothing but another layer of bureaucracy.  It can only happen in the artificial environment of telecom regulation.

. . . . . .

Posted Sunday, February 2, 2003

Music Industry Stupidity

Criminalizing their customers.  Janis Ian weighs in,

In RIAA vs. Verizon, the U.S. District Court for the District of Columbia ruled that anyone suspected of downloading so-called "infringing" files on the Internet -- usually an MP3 of a song -- could be sued. No evidence is required. An accuser fills out a form for a court clerk and the machinery is set in motion.

The record companies say this decision will mean more money for musicians, but they have it backward.

. . . . . .

Music Industry Wisdom

A lot of wisdom is packed into this article by John Snyder of Artist House Records.  For example,

Record companies are not logical, righteous entities. They are ramshackle, profit-driven enterprises. They act in their perceived best interests, and they act ruthlessly and, in many cases, irrationally. The people who run them still have their e-mail printed out by their secretaries. We have to wait for the next generation to take over, the "software" generation, the generation of people who don't remember growing up without a computer around. I would argue that the future of music is multimedia, the future of multimedia is DVD, and the future of music companies is software. In five years, record labels will be software companies and I don't think they know that yet.

The whole article is worth reading.  Twice.


. . . . . .

Copyright 2002-2003 Arnold Kling. All rights reserved. Terms of use

Archives, RSS & Email

The archives for this section have recently been made available. Click here for access to them.

Also, enter your email address below to subscribe to The Bottom Line by email!

powered by Bloglet

Subscribe to the Bottom Line! - RSS feed

Recent Essays

Paying for Information

Stock Options and Start-Ups

Genetic Engineering

Corporate Profits

Is Blogging a Fad?

Index with Summaries


Zimran Ahmed
Smart Mobs
Brad DeLong
Dan Gillmor
Paul Krugman
The Speculist
Edward Felten
Jakob Nielsen
Andrew Odlyzko
Tomalak's Realm
Megan McArdle
Doc Searls
Clay Shirky
TechCentral Station
Hal Varian

News Sources

Boston Globe
Business 2.0
CNET - News.com
Harvard Bus. Review
Mercury News
New York Times
San Fran. Chronicle
Tech Review
Wall Street Journal
Washington Post
ZDNet News


Context Magazine
Cowles Foundation
Chron. of Econ Journals
The Dismal Scientist
Economics of Networks
Encyclopedia of Economics
History of Economics
The Information Economy
Library of Economics
McKinsey Quarterly
NUA Surveys
Strategy + Business
Working Knowledge


American Enterprise Institute
Berkman Center
Brookings Institution
Cato Institute
Economic Policy Institute
Frasier Institute
Heritage Foundation
National Center for Public Policy Research
Progressive Policy Institute