|
|
|
|
|
|
|
|
|
Posted Friday, February 28, 2003 VC-funded business vs. Mom & Pop
It's no fun competing against a nonprofit. In the 1990's, it was the VC's funding nonprofits, and it was those of us who were self-funded who felt like we were screwed. In fact, mom-and-pop businesses typically are in markets with low ambiguity and VC's go into markets with high ambiguity, so they don't go up against one another. See Amar Bhide, The Origin and Evolution of New Businesses. All this B-school jive is being brought up in the context of the Google-Blogger merger, an event whose cosmic significance eludes me. But then, I could just be slow. I still have not figured out what all the fuss over the AOL-TimeWarner merger was about. . . . . . .
No-demand Computing?That's was the way I ridiculed IBM's "On-demand computing" concept. I still think that only a moron would buy it. However, today, news.com reports
Looks like I was wrong. I should have remembered the old cliche. No corporate information officer ever was fired for being a moron. . . . . . .
Non-Bayesian Spam FilteringHere is a report by Jon Fortt on a non-Bayesian spam filter.
That performance is unacceptably bad. If you watch a spam filter classify 1 percent of your legitimate email as spam, you would soon realize that you would never use it. With Bayesian filters, like PopFile, you can get much lower rates of both Type I and Type II errors. . . . . . .
Privacy, Security, and Architecture IIA friend of mine who works as a lawyer at a trade association complained that during an internal email discussion of how to respond to an email from a hostile consumer group, someone accidentally forwarded the internal email to the consumer group. He was frustrated that there was no technical solution for this. My first reaction was to send my friend a link to James Grimmelmann's piece on accidental privacy spills, which argues that there is no technical solution. However, it seems that where there is a will there is a way. Microsoft has an idea for embedding rights management into email. The kicker, though, is that it is not compatible with anything currently in use.
For some reason, I get this picture in my head of how this project was pitched to senior management: "with this technology, your email could not be read by an adversary in an anti-trust proceeding..." . . . . . .
Posted Wednesday, February 26, 2003 Warning to SuitsThis is what the Geeks think of you. (Pointer courtesy of Doc Searls)
. . . . . .
The Star-Maker MachineryBrother Jolliffe gave me this pointer. I have no idea how accurate it is. It takes a hypothetical band that has a Gold record album--half a million copies sold.
The story's kicker is this:
I'm a little surprised that the music publisher doesn't pick up the recording cost or the cost of the promotional video. And it seems to me that the producer's take is a little rich. And if it were book publishing, the lawyer-agent would be getting a share of the "band's" net, not of the gross. But the basic point that bands do not necessarily get rich from big albums is probably right.
Still, I think that bands will miss the star-maker machinery when it's gone. In ten years, bands will be looking back with nostalgia to the days when a successful musician could earn as much as a sanitation worker.
. . . . . .
Data Mining AbuseAs you know, I am in favor of using data mining to deal with terrorism. On the other hand, because I believe that data mining is a powerful weapon, I want to see its use carefully regulated, legislated, and limited. When I see a story like this, about credit card databases being used to locate people who buy child porn, I am ready to march with the civil libertarians. I feel that as a parent of three daughters, I have all the tools I need to fend off child pornography. I don't need the credit card industry to snoop on its own behalf or on behalf of the government. I believe that this country needs a reasoned debate on the use of data mining. As long as people simply march under banners ("absolute privacy" on the one hand or "stop child pornography" on the other), we will get the worst possible outcome. We will see data mining used to embarrass people who look at the "wrong" web sites, and we will see it not used to connect the dots to prevent terrorism. . . . . . .
Posted Tuesday, February 25, 2003 Bayes' Theorem and Homeland SecurityI try to explain Bayes' Theorem here. I think that Bayes' Theorem is something that everyone should try to grasp, even if you do not buy my thesis that it is an argument for data mining as a method of fighting terrorism. . . . . . .
Spam Filtering UpdateAt this point, all of the Type I errors (when Popfile classifies legitimate email as spam) are graphics-heavy web-generated group emails. For example, I belong to a book club where for some reason they use Evite. This generates a spam-like email. I'm sure that Popfile will learn to accept these emails, but my guess is that the cost will be a lot of type II errors--classifying spam emails as good, because they look like Evite-generated email. One more reason for my book club to not use Evite--it's a time-waster, anyway. Anyway, I'm just about to the point where I could use Popfile for all my email sorting. I continue to have to disable the process "POP3trap" which a reader told me is part of Trend Micro P-cillin anti-virus software, not part of PopFile. . . . . . .
Posted Monday, February 24, 2003 Moore's Law ReadingSince our economy is pretty much dominated by Moore's Law, it pays to be informed about it. So this article by Jon "Hannibal" Stokes is worth study. Pulling out one tiny quote:
Well, maybe one more tiny quote:
Thanks to TechCentralStation.com's Nick Schulz for another great pointer. . . . . . .
Differing with BillAn interesting interview with Bill Gates. I'll pull out this quote:
It might work out that way. But I could also see an alternative scenario, in which what is coming is not the PC-plus era, but the remote-control era. In the latter scenario, you carry around a remote control that interacts with lots of devices out there, including storage (so you can transfer your photos directly to network-based storage, rather than going home and loading them onto your PC). As I've said before, I think that the remote control is not a big-screen device, but my guess is that it needs to have a bigger screen than what you see on a cell phone. . . . . . .
Economics of Intellectual PropertyZimran Ahmed captures the intuition of many economists.
That intuition seems right. It's a bit harder to spell out exactly the economic factors behind it. For more, see my other blog. . . . . . .
SuitwatchSister Wentworth cites an article by Declan McCullagh, who says that Geeks should stay out of the lobbying game, except to try to repeal some bad past legislation. She disagrees, asking rhetorically,
My prediction is that the Geeks will win. They will treat the Suits as damage, and route around them. And the riff that she quotes from Dave Winer is not the right argument to make. He is saying that the music industry is immoral because artists are not well paid. Mark my words. Ten years from now, today's musicians will look back on this period as a golden age for compensation. The decline of the legacy music industry is going to make it harder, not easier, to earn a living in music. The breakup of the star-maker machinery will increase our national well-being, but the social benefit will accrue to consumers, not artists. See Equilibrium in the Market for Rock-n-Roll. (UPDATE...or see this piece from Michael Wolff, which brother Jolliffe forwarded with some reluctance. It sort of says what I've said in this post.) . . . . . .
Posted Sunday, February 23, 2003 Privacy, Security, and ArchitectureIt's unusual for a long Web essay to be worth reading in its entirety, but set aside some time for this piece by James Grimmelmann. He meditates on an "accidental privacy spill" in which a journalist's private summary of a conference gets forwarded and winds up onthe web. A sample nugget from Grimmelmann:
Another interesting point:
Grimmelmann argues convincingly that there is no good technical/legal solution to ensure email privacy. The isomorphism he describes would imply that there is no technical/legal solution to deal with copyright. Sic transit gloria Creative Commons. Thanks to Ed Felten for the pointer, although sister Wentworth also cites it. So you may have read it already. If you haven't, do so now. I give it 5 stars. . . . . . .
Web-based Bayesian Spam FilteringIf you're one of those folks who likes to use a web-based email program, then you might look at Oddpost. They claim to have implemented Bayesian spam filtering. If you have tried it and care to report on the results, let me know. Meanwhile, for the folks who use client-based email, I'll continue to update you on my experiences with PopFile. . . . . . .
PopFile anti-spam updateI'm having pretty good luck with PopFile. In any given day, I might get 10 legitimate emails, with PopFile tossing one into the spam category about every four days. I will get about 70 spam emails, with PopFile tossing one of them into the "ok" category about once a day. So type I error rate is about 2 or 3 percent (still too high) and type II error rate is less than 2 percent (much better than with rule-based filter.) In fact, the type II error rate is so low that when my rule-based filter says that something is spam but Popfile disagrees, I open the mail to see what is going on. Today, for example, Popfile accepted an email that was unsolicited but was a notification about stuff that I really wanted to read! I think that the reason that the Type I error rate is so high is that I don't get enough legitimate email to give it sufficient examples yet. Maybe in a few more weeks... Also, email from mailing lists poses an interesting problem. If you subscribe to an email list, chances are a lot of the mail you get from the list does not interest you. Yet it is not spam. It would be interesting to try to train Popfile by setting up separate buckets for this sort of mail. I have not done that, and instead I tend to unsubscribe to lists that for me have a high noise-signal ratio. But it is possible that by creating more buckets in Popfile (right now, I just have two--legitimate email or spam) I could subscribe to email lists with a high noise-signal ratio and train Popfile to find the signal. . . . . . .
Mooning MicrosoftSomebody coined that term to describe Netscape's bluster, which ultimately proved to be self-defeating. And now we have Jonathan Peterson's interview with Marc Canter.
Oh God dude - let your product speak for you. I'm from the old school that tends to think of trash talking and real accomplishment as being inversely related. . . . . . .
Posted Friday, February 21, 2003 How to Free SpectrumIf you think that the answer is a wireless "commons," think again.
. . . . . .
Posted Thursday, February 20, 2003 Suits 3, Geeks 2That's the way I read the outcome of the FCC action on local phone "competition." As you probably know, Michael Powell was on the losing end. He wanted to free the Baby Bells from state regulators who can impose arbitrary wholesale rates that the Bells must charge "competitors" for use of the phone lines. What the state-regulatory system means is that a company that does not have a single engineer can enter the phone business. Once they run the regulatory guantlet, they can sell phone service over the local phone company's lines. This does reduce retail phone rates, but not as efficiently as simply regulating retail phone rates directly. It is a "win" for federalism in the same sense that localizing cable TV regulation is a win for federalism. In both cases, it makes lots of work for lawyers, lobbyists, and local officials, with abundant opportunities to corrupt the process. It stimulates a revolving door (or revolving trough) for influence-peddling Suits, as they go from government to the semi-private sector and consulting. Whether the Bells end up winners or losers is unclear. They have their own Suits, who know how to work the fine print. I don't think that ordinary consumers had a horse in this race. Our best hope is that wireless Geeks will be able to route around the Suits and introduce genuine phone competition. Let's hope that Powell's colleagues on the FCC don't try to stop that one. UPDATE: I agree with this comment from brother Blankenhorn:
Given that the last mile will be wireless, anyone who builds out fiber-to-the-home today would be nuts. . . . . . .
Posted Wednesday, February 19, 2003 Suits and GeeksI think Sarah Stirland captures the "out of it" feel of a room full of suits. Fortunately, the geeks treat the suits as damage and route around them. . . . . . .
Michael Malone, TrollWhen a blogger engages in a wild rant on a popular subject, Slashdotters call it a troll. Michael Malone does a troll with this Jeremiad on Moore's Law. He blames Moore's Law for the dotcom bubble, which is ridiculous. The dotcom bubble was not a technology miscalculation. It was a business miscalculation. What I call the McKinsey business strategy--regardless of the question, the answer is to sacrifice profits for market share--became the winning recipe for an IPO. Malone suggests that consumers will not pay more for better chip performance. But that was never the point of Moore's Law. Moore's Law was that the most economical chip density would double every eighteen months. People will find that achieving a given level of performance will be less expensive using the latest chips. In theory, consumers could be satiated at a given level of performance, so that they do not want to upgrade to the next chip. However, I think that practical applications of Bayesian filtering will give consumers a reason to want that upgrade. Thanks to Nick Schulz of TechCentralStation.com, for pointing me to Malone's article. . . . . . .
Posted Tuesday, February 18, 2003 Bayes is CoolI was really upset a few years ago when my daughter's textbook for AP Statistics did not include Bayes' Theorem, which is an elementary statistical theorem that is one of my all-time favorite mathematical equations. If a statistics text ignores Bayes, that is a problem. But as articles like this illustrate, Bayes is cool nowadays.
The article's explanation of Bayes' Theorem is way off. The article says that Bayes said that you use the past to predict the future. That's not what the theorem is about. Bayes' theorem says that you can sort spam email (for example) by looking not just for phrases that are commonly associated with spam, such as "enlarge your penis." Equally important is the absence of phrases that are commonly associated with legitimate email, such as "following up on this morning's meeting." It is by scoring an email's correlation with past legitimate email as well as past spams, and comparing the two scores, that Bayesian email filters become very powerful. . . . . . .
Spam Filter UpdateI figured out how to get POPfile, the Bayesian spam filtering program, to work reliably on my Windows XP machine. The problem is that when I start my computer, a process called POP3trap.exe starts running which seems to convince POPfile that it does not really need to start up properly. So when I start my computer, I give it the old 3-finger salute, shut down POP3trap.exe, and then start POPfile. Then I start my email program, and things work fine. At this point, POPfile has seen about 250 spam emails and fewer than 50 good emails. It seems to make no Type II errors (classifying spam as good mail), but it still makes an occasional Type I error (classifying good email as spam). Assuming that after another 100 or so good emails it stops making any Type I errors, it will have lived up to my high expectations for Bayesian spam filters. To paraphrase UPDATE: Thanks to Zimran Ahmed for correcting me--I always get Gibson and Sterling mixed up on that quote! . . . . . .
Posted Monday, February 17, 2003 More Toll RoadsI can safely predict that one of the important applications for "the era of sentient things" will be more toll roads. It looks like London's camera-enforced congestion tax is off to a good start.
My guess is that "easy-pass" systems are getting very inexpensive to operate, and we will see a lot more toll roads in the U.S. over the next five years. . . . . . .
Posted Sunday, February 16, 2003 Microsoft's ChallengesDavid Stutz lists every conceivable challenge facing Microsoft: (1) The shift from PC-centric computing to Internet-based computing (2) Open-source software (3) New hardware I think that (1) and (3) are the important ones. In what Howard Rheingold calls "the era of sentient things," Intel will flood the world with chip radios. It seems unlikely that these will depend on Microsoft software. I think that the new battle is for the hiptop. What device are people going to carry with them to interact with the sentient things that start to permeate everyday life? Will it be a cell phone, as Rheingold believes? Will it be a Tablet PC? Will it be something in between? My guess is that battery life will prove to be an important issue, which puts the optimal hiptop device closer to a cell phone than to a Tablet PC. Which means that Microsoft does not have any particular leverage going into the battle. . . . . . .
Posted Friday, February 14, 2003 Thinking about Security DatabasesPeople who want to apply their brains and not just their knees to the issues of government use of data mining for security purposes might want to read this paper by Paul Rosenzweig.
Rosenzweig's opponent in a forthcoming debate will be Timothy Lynch, who writes,
In my view, we can retain our freedom without restricting the intelligence of those who are trying to prevent terrorism.
That's the way I feel about using databases in national security. I don't want the science to slow down. I want the ethics, in the form of better auditing and checks and balances on government use of the information, to catch up. . . . . . .
Identify People, not DevicesI agree with Eric Norlin that digital identity is an important topic, and I'll pass along his pointer to this article.
Right now, identity in the Internet and the broader telecom sphere is device-centric. Phones have numbers. Computers have IP addresses. People are anonymous, and they have to work constantly to identify and re-identify themselves. One can imagine instead a world in which identity belongs to people, and the devices always recognize the people. In some ways, this will turn things upside-down. Instead of dealing with the problems of anonymity (hard-to-trace spam, hard-to-use electronic shopping systems), we will be dealing with the problems of non-anonymity, including privacy protection. . . . . . .
Posted Thursday, February 13, 2003 The High Cost of Free SoftwareAs I mentioned, I am trying to use PopFile, an anti-spam program that you download to your machine to act as a front-end (or proxy server, to be a bit more technical) to your email program. The documentation is terse, and I feel that it was sheer luck that I figured out how to even start the program. Other than that, set-up was quick and painless. I would say that if you can get PopFile to work reliably enough, then you *definitely* should use it. The idea is brilliant, and it's pretty nicely done. I've barely started to train the Bayesian filters, and they already work better than the rule-based filters that I put into my regular email program (although I can keep using my old filters without having to make any changes, so for now I have two layers of filter going). Because I hate spam and I love statistical filtering, I will keep at it with PopFile. But I have to say, if it were anything else, I would have ditched it. But the program's operation is only intermittent. If I click on "check mail," over half the time I fail to get a connection all the way through PopFile to my Internet Service Provider. The net result is that checking mail has become much more time-consuming, rather than less. To me, this illustrates one of the costs of open-source software. Proprietary software gets tested on popular configurations, and most of the bugs get worked out before it gets released. Proprietary software has bugs, too, as we all know. But my experience has been that the time that I spend trying to install, configure, and overcome bugs with proprietary software is less than the time I spend doing those things with open source software. In the contest between open source and proprietary software, I'm in the camp that says "there is room for both." In this case, I think there is room for a proprietary solution. The developers of Popfile definitely have my thanks. But I'd rather give them money--and get a reliable product in return. . . . . . .
My DayI went to a conference on spectrum regulation. The main thing I got out of it was a photo of Kevin Werbach and Sarah Lai Stirland. Your honor, I believe that this picture definitely will prove that I did *not* punch Ms. Stirland in the face. Other than that, Sarah and I left the conference wondering what it was all about. Maybe she'll post more. In my TCS essay today, I suggested--and I'm serious about this--that government officials should be blogging. Maybe Michael Powell will take the bait. Finally, I've been trying to work with Popfile, a Bayesian spam filter. More comments on that shortly. . . . . . .
Posted Wednesday, February 12, 2003 Data Mining or Duct TapeThis week, there was a run on duct tape, as Americans tried to protect themselves in an environment in which we do not know who among us is a terrorist or where they might strike. Meanwhile, as Heather MacDonald points out, the campaign against the attempt to use database technology to fight terrorism is roaring ahead.
I would add that data mining is the basis for the efficient provision of consumer credit in this country, including home mortgages. It is the basis of the most effective email spam filtering techniques (commonly called Bayesian filters). The knee-jerk hostility to data mining technology is winning for now. But Senator Feingold and his colleagues are not completely against using technology to fight terrorism. As far as I know, nobody has introduced a bill to ban duct tape. . . . . . .
Posted Tuesday, February 11, 2003 Coke Goes FlatAnother one of my blog siblings (I still owe him a welcome punchTM) found a speech by a Coke advertising executive that was idiotic, and dutifully reports it as such. Basically, the adman's idea is to promote synergy by linking his successful brand with a bunch of media industry losers. Like that strategy worked so well for AOL. . . . . . .
Telecom Needs DecontrolNow that Corante has a real journalist covering the telecom arena, among other things, I feel less obliged to report the news and more free to just be cranky. In one of today's posts, my sister blog points to a speech by FCC RINO Kevin J. Martin. He talks about "new fiber." That rang a bell with me. An alarm bell. When I was just a lad working as a economic research assistant, we were in the middle of a government-run oil market, where "new oil" was to be priced differently from "old oil," among other things. It was brilliant in theory, and unworkable in practice. As a result, the United States appeared to be in an "energy crisis," until President Reagan dismantled the regulatory regime and let the market sort out the oil situation. Michael Powell wants to do for telecom what Reagan did for oil. Kevin Martin still thinks that regulatory agencies can fine-tune a superior outcome. But government-managed competition of the type they are attempting is once again beguiling in theory and nightmarish in practice. Mr. Powell is the the one to root for in this "mudwresting match." (RINO stands for "Republican In Name Only," as apparently Mr. Martin intends to vote with the Democrats on the FCC.) . . . . . .
Non-Bayesian Spam FilteringKevin Marks of Mediagora emails me with a question about Bayesian spam filtering:
The Tim Oren piece is very good, and I recommend visiting the link. I am not at all dogmatic about using the Bayesian formula. There are lots of good statistical techniques out there. I'd recommend picking a technique that is computationally rapid and implemented with a no-brainer user interface, even if it "inferior" by some theoretical argument. All statistical techniques are subject to the problem that they could degrade over time as spammers change methods. But that is more an issue for spam filtering at the server level than at the individual level. My individual spam filter is less likely to degrade, in part because the cost/benefit to the spammer of reverse engineering my personal filtering weights is prohibitive. So I would like to see a combination of server-based filters (to take the load of spam off of ISP's) and individual filters (to make it cost-ineffective for spammers to try to evade filtering). The server-based filters might be rule-based rather than statistical, to save on computation. Or is there a way to change Internet email from a client-server application to a peer-to-peer application? If that were feasible (and I scarcely know what I am talking about), it might make it easier to allocate resources to filtering spam at the network level. . . . . . .
Posted Monday, February 10, 2003 Electrical Grid Broadband: Panacea or Kludge?Reading this story, I couldn't tell. It quoted some some enthusiastic supporters, but then deeper in the story it said this:
Sounds difficult and expensive. . . . . . .
Another Music Industry Revenue ModelNick Schultz of TechCentralStation.com pointed me to this story.
Think about this in contrast with standard music publishing. Instead of publishing and distributing CD's to retailers "on spec" (hoping that they will sell), you would be publishing CD's to order. Note also that the publisher would not be one of the traditional music publishers, which is likely to raise all sorts of conflicts within the industry. . . . . . .
The Language BarrierIf the Internet creates trade in information services, I argue that this may create a trade barrier involving language.
. . . . . .
A Music Industry ModelDavid Strom passes along a college freshman's economic model for the music industry.
I actually think that the record companies could earn higher profits with this kind of subscription model, because of high volume, low costs, and the possibility of add-on services. . . . . . .
Future of Books?Prashant Kothari talks about the future of books and book publishing.
I think that the overall trend is for people to read more sources, and less from any one source. I think that a lot of traditional nonfiction books will die, because it takes the authors too long to convey their ideas. Even books that I recently read that I would recommend highly, such as Steven Pinker's The Blank Slate, Howard Rheingold's Smart Mobs, and Joel Mokyr's The Gifts of Athena, all suffer from being considerably longer than I would have liked. Increasingly, I see books as a free-time indulgence. Reading a book is like going to the beach (and those activities are indeed linked together in many people's minds). But "practical" reading involves much shorter pieces. So I think that e-reading will take off. But e-books won't. . . . . . .
Posted Saturday, February 8, 2003 Corante ConnectedIn the spirit of "Corante is a Big Tent" (i.e., they even let me in), let me welcome Sarah Lai Stirland by seasoning her opening connected manifesto with a bit of bottom-line cynicism.
If you believe that, I've got some spam mailing lists I'd like to sell your name to.
Here is my idea of reducing ignorance and preconceptions.
Not everyone would agree with this bald-face boldface assertion. For example, see Audiences scale, communities don't.
Reviewed by whom? By some unelected international bureaucrats, as in the EU or the UN?
Let the market review technologies. To paraphrase Churchill, the market is the second-worst review mechanism. As for the worst, all the others are tied.
Let me close by saying congratulations to Corante on landing its newest blogger. I plan to be a regular reader. . . . . . .
Posted Friday, February 7, 2003 Where is the innovation in email?Consider this rant from Bob Frankston.
Email is the most popular Internet application, and yet innovation is incredibly slow. I would pay for an email client that uses Bayesian filters. I agree with Frankston that a feature that sent a notice back to the sender that the mail passed the Bayesian filter would be great. . . . . . .
Rheingold AgainSome days, I feel like this blog is pointless. All you have to do to keep up with current technology is go to www.smartmobs.com. Meanwhile, I'm just passing along Howard Rheingold's pointers. Here's another one.
As colleague Dana Blankenhorn points out, small, inexpensive radio chips are really going to have a big impact. It will take several years, but it is a big deal.
. . . . . .
Posted Tuesday, February 4, 2003 Identity and the FutureOne of Howard Rheingold's favorite phrases is "How to recognize the future when it lands on you." Eric Norlin is trying to help. He surveys activity on the part of various levels of government in the area of improving digital identity.
Thanks to Doc Searls for the pointer. . . . . . .
Posted Monday, February 3, 2003 The Joys of Telecom "Competition"From Joel on software:
This is what happens when "competition" has no infrastructure and instead adds nothing but another layer of bureaucracy. It can only happen in the artificial environment of telecom regulation. . . . . . .
Posted Sunday, February 2, 2003 Music Industry StupidityCriminalizing their customers. Janis Ian weighs in,
. . . . . .
Music Industry WisdomA lot of wisdom is packed into this article by John Snyder of Artist House Records. For example,
The whole article is worth reading. Twice.
. . . . . .
Copyright 2002-2003 Arnold Kling. All rights reserved. Terms of use |
|
|
