The Black Hats must be gloating

Posted by: Steve Hamm on July 29, 2005

What a miserable week for software security! First, on Wednesday, a smartass researcher outed a problem in Cisco router software at the Black Hat security confab in Las Vegas—potentially exposing the Internet to massive attacks and outages.Then, on Friday, just five days after Microsoft launched its new anti-counterfieting software, hackers publicized a way to bypass the stuff. They call software security a cat and mouse game. This week, the mice won.

The Black Hat conference blow-up is really disturbing. According to published reports, what happened was Michael Lynn, who started off the week as a security researcher at Internet Security Systems, defied ISS and Cisco by putting on a presentation at the conference that explosed a flaw in older versions of Cisco's Internet Operating System. He was apparently quit. Cisco sued him and the conference organizers.The matter was settled out of court Thursday when Lynn agreed never to repeat the information he imparted in his Black Hat presentation and handed over any Cisco software code he had.

Hey, it's good to expose flaws in software so they can be fixed. But, typically you tell the software maker about them first, and give them plenty of time to fix them, so their products can be patched before much harm is done. Then it's okay for you to publicize the flaw to show how smart you are and get good press for the security firm you work for. I don't know all the details behind the story, so I may be all wet. But, based on what has been published so far, I'd say Lynn crossed way over the line.

Some in the blogosphere have hammered Cisco for suing. They call the company heavy handed. I think not. By the time Cisco sued, it was probably too late to put the genie back into the bottle. But now, at least, anybody who plans this sort of caper in the future might think better of it.

Re Microsoft's latest glitch. All I can say is, with $1 billion or so in profits flowing in per month, you'd think Microsoft would be able to get its software right more often, and avoid looking foolish.

TrackBack URL for this entry: http://blogs.businessweek.com/mt/mt-tb.cgi/

Reader Comments

James

July 29, 2005 05:44 PM

On some mainstream tech site (not a blog) I read that Cisco had been notified earlier this year of the problem but had yet to take any action. Sounded to me like this case is no different than the same methodology employed against Microsoft when they drag their feet other than the company he worked for got cold feet.

T. McClain

July 29, 2005 06:27 PM

All the other articles out there say Michael Lynn quit his job, not that he was fired. Do you have some inside information to say otherwise?

Nick Levay

July 29, 2005 06:35 PM

You are all wet. You have many factual errors in your post.

Mike Lynn did not get fired, he resigned. Lynn had been working with Cisco for months. Cisco was going to be presenting with him until they changed their minds in the immediate run-up to the presentation. Lynn also gave his presentation in a manor that did not reveal technical details that would allow anyone to exploit the bug without doing the same research he did to discover it.

MaxieZ

July 29, 2005 06:41 PM

If you don't know the details you should really do your homework before printing libel.

Here are a few facts for you to look up.

A). Cisco has known about this since April. They released a patch for it at the time and have since been using newer software on their releases.

B). ISS originally wanted to come out with it without speaking to Cisco and Mike stopped them.

C). Only after all this time with Cisco not emphasising it as a critical patch that he felt the need to speak.

D). Mike did not get fired. ISS told him he couldn't speak. He resigned so he could give the speech.

Marie

July 29, 2005 06:45 PM

You should get your facts straight before putting things in print.

Marc Hedlund

July 29, 2005 06:55 PM

This post is pretty severely misinformed. Lynn quit, he wasn't fired. He gave Cisco exactly the notice you ask for of the flaws he reported.

There's a good background on the story at Wired: http://www.wired.com/news/politics/0,1283,68356,00.html?tw=wn_tophead_3

As a matter of opinion, I strongly disagree that Lynn is the one who potentially exposed the Internet to "massive attacks and outages," as you claim. Instead, by making the problem publicly known, he is giving systems administrators everywhere the opportunity to update their systems and to avoid the threat to which they were already vulnerable. If, as Lynn claims, "known bad people" were already working on exploits, he should be getting credit for raising a needed alarm.

I think you should correct the factual errors in your post.

latiff

July 29, 2005 07:05 PM

He didn't get fired. He quit. And he gave cisco plenty of time. Since he told them about the problem in april. I don't like the tone of this article at all. This guy quit his job, because he wanted people to know about the vulnerability. You can't just keep everything locked up forever. And the way you say that he handed out cisco code makes me sick. Yes, the documents did have cisco code in them, but not that much. You made it sound like all he did was hand out proprietory code. But from what i've read, the documents also had explanations in them. Actually, here is a URL to a 35 page PDF documentation of some of his findings. http://www.infowarrior.org/users/rforno/lynn-cisco.pdf

I have a feeling that someone is paying you to say what you're actually saying. You maybe mis-informed. I support this guy with all my strength. You just can't do what cisco did.

A purveyor of real journalism

July 29, 2005 07:14 PM

Steve Hamm... Hrm...

I remember when journalists had ethics, and actually did investigation before they wrote anything.

But then again, I don't think you are a journalist, writer, or any other news worthy person.

Some clarifications for your article.

1. Mike Lynn was not Fired. He resigned before the presentation.

2. He DID give the exploit to Cisco in early 2005. They had more than enough time to "fix" the flaws, however, Cisco didn't have the brain power employed to realize the threat, and wouldn't believe that Mike was correct. Cisco held by their OLD addage that the IOS could never be hacked.

3. There are over 100 articles on this as of the time of your posting, and yes, you are all wet. You obviously read one article, and made up your mind.

4. The fact that ISS and Cisco both tried to cover up this "flaw" in the architecture of the IOS for their own immoral and unethical reasons never makes your article. I really hope that you review the true issue at hand, as I agree with you, the events this week at Blackhat are VERY DISTURBING... A security researcher tried for several months to do the RIGHT thing, and when a cover up was forced down his throat, he quit, and presented the information, not for personal gain, but to let everyone know the underhanded politics at ISS and Cisco.

Perhaps you would be so kind as to do your research, and really out the "bad guys" instead of taking the side of the Huge Organizations that tried in vain to keep this flaw under wraps...

James

July 29, 2005 07:17 PM

First, Michael Lynn was NOT fired. He submitted his resignation several hours before his presentation.

Second, the "flaw" you describe either shows you have no understanding of what was actually presented or you are misinformed. Lynn gave a display of how to execute arbitrary code on a Cisco router using a known exploit. The important part of this display was _code execution_, something which until this point had never been done on Cisco routers before. The exploit he used to demonstrate this method was a _known_ vulnerability that Cisco, ISS, and Lynn had addressed months before in a _responsible_ manner. His display did not include any technical details, and it would be impossible for someone to duplicate his efforts based on the information he gave nor in the information Cisco removed before the presentation.

Lastly, Lynn and ISS did give Cisco warning on both the initial vulnerability, a IPv6 issue, and the ability to execute arbitrary code on the router. This was done in April. At this time ISS _and_ Cisco approved the issue for publication at Black Hat and support Lynn in this endevor. On Monday, before his Wednesday presentation, support was withdrawn with little reason given to Lynn. The suggestion was made that the research was "incomplete", something Lynn knew to be untrue. This suggestion to the public by ISS besmudged Lynn's reputation in the security world.

Fact Checker

July 29, 2005 07:41 PM

It's truly sad that Mr. Hamm, who writes
for a major magazine, has fallen into the
"who needs the basic facts to write a story
when I can be more sensational if I make it
up" style of journalism. By his own hand
he admits to not knowing 'all the details'
and he obviously couldn't be bothered to
find them out.

Fact: Mr. Lynn, a lead R & D researcher at
ISS, had been working on the Cisco project
with ISS' blessing for about 6 months.
Cisco was aware of the project and only in
the last few weeks began to show any
regrets regarding the talk.

Fact: ISS sent Mr. Lynn to the Black Hat
Conference specifically to give the talk
on the Cisco vulnerability and it was ISS
provided and vetted materials that were
included in the CD and published materials
which were to be given to all conference attendees (those CDs and bound copies were
taken by Cisco who used razor blades to cut
out Lynn's presenation materials from the
bound copies and Black Hat's elaborately
produced CDs were replaced by plain CDs
with a clear file label, burned by Cisco
minus Mr. Lynn's information).

Fact: Mr. Lynn WAS NOT fired by ISS. Lynn
believed the information so important to
the internet security community that he
formally RESIGNED his position with ISS
hours before giving his talk so as NOT to
compromise ISS.

Fact: Only AFTER both companies reached
a settlement on Thursday with Mr. Lynn
that saw him signing an agreement to never
again present or give information on the
Cisco IOS flaw, did Lynn receive word
that the FBI was now involved regarding an
ISS criminal lawsuit because the same
presentation materials that ISS had given
to Black Hat to publish weeks ago, are now all over the internet.

Mr. Hamm, I am not a journalist, nor have
I conversed with anyone who has been at
Black Hat or in Las Vegas this week. I do
not even work in the computer field but I
managed to find out the above facts and I
verified them from multiple legitimate
sources. Since you are paid to do what I
did for free, in my own time; I suggest
you forego the name calling (smart ass
researcher? have you ever met Mr. Lynn?)
and your own 'creative storytelling' and
try a little REAL research. Maybe you
could actually gain some credibility with
community you supposedly write for?!

Colin

July 29, 2005 08:17 PM

It seems like a correction of facts may be due! (http://radar.oreilly.com/archives/2005/07/businessweek_bl_1.html)

Jon Nyx

July 29, 2005 08:34 PM

"I don't know all the details behind the story, so I may be all wet."

You, sir, are all wet. If you had bothered to learn the details before writing your piece (a failing all too common in most segments of the news/infotainment industry these days), you would have known the following:

- Mr. Lynn, through ISS, had reported the initial problem to Cisco months ago.

- Cisco patched one bug, but did nothing to fix the basic problem (a heap overflow). In fact, the next (modular) version of IOS would have made the problem even worse.

- Cisco was not going to tell anyone about this potentially devastating problem, even though...

- ...Cisco had originally agreed to have one of their engineers host the presentation with Mr. Lynn. Cisco then reversed their decision 48hrs before the Black Hat convention, filed restraining orders all around and even hired temps to rip materials out of convention program booklets printed as early as May.

- Mr. Lynn quit his job with ISS so he could give the presentation. He felt the problem he had found was too important not to disclose, especially in light of the fact that non-US bad guys are currently discussing potential exploits for the same problem via underground webboards.

- According to other, more accurate and careful journalists (who are actually attending the conventions), Mr. Lynn has been thanked all week by senior members of our intelligence, infosec, IT and military communities who attended both BlackHat and DefCon. He his also being hailed as a hero on infosec websites and mailing lists.

Your reporting on this issue, Mr. Hamm, is sloppy and you do a disservice to your readers, who are advised to go elsewhere if they wish to know exactly what happened between Mr. Lynn, Cisco and ISS. This is an important matter.

You also owe an apology to Mr. Lynn, and if this piece is indicative of your reporting as a whole, perhaps you should consider other career opportunities.

Sincerely,

--jX

Kristi

July 29, 2005 11:13 PM

I think you have to be the biggest idiot on earth. Cisco knew about the problem, IIS knew about the problem. They were burying their heads.

The CISCO code was stolen and has been passed around left and right.

The guy is a HERO, not a smart ass.

Of course, an idiot like you would never understand that.

unbeliever

July 30, 2005 12:06 AM

It _is_ a cat and mouse game, but the mice is us, I'm afraid.

lamo

July 30, 2005 03:30 PM

I didn't realize that Business Week was attempting to compete with the tabloids for making stuff up.

This is an utterly amazing piece of journalistic fabrication.

I suppose we now know why your name is Hamm, eh?

wayne

July 30, 2005 05:12 PM

ahh.. thank you for article mr hamm.
helps me whittle down my feeds and bookmarks

Nate

July 30, 2005 11:11 PM

don't know all the details about publishing articles, so I may be all wet, but I thought fact checking was apart of your job...not your readers.

Matthew Murphy

July 30, 2005 11:44 PM

Horrible post. Lynn wouldn't have been exposing *anyone* to attacks had Cisco announced the issue in a security bulletin when they released their update last April. The fact that they had a bulletin available a mere two days after Lynn's talk illustrates that they could easily have avoided any problem by simply acknowledging that what Lynn found *was* a security risk.

Although Lynn may have defied ISS and Cisco, he was not an employee for either when he did it. I was at the conference where Mr. Lynn's presentation took place. All accounts (including the "published reports" you cite) document the fact that Lynn resigned.

Lynn *did* give Cisco plenty of time, as evidenced by the fact that they plugged the flaw *IN APRIL*. They simply neglected to inform users that the fix wasn't a simple bug, but a buffer overrun that could be easily exploited to crash or compromise their routers.

You say your entry is "based on what's been published so far", but it's obvious you didn't *READ* any of the published information, because no source that I've come across (and I attended the event) has said anything close to what you've written.

The "sort of caper" that Lynn pulled off was to inform users that the dozens of remotely-exploitable buffer overruns that Cisco has been passing off as "denial of service" attacks for years are in fact exploitable to compromise routers running IOS, even the latest versions.

Frankly, I'm ready to thank him and shake his hand. I wish I had the opportunity to offer Mr. Michael Lynn a nice job and a six-figure salary as a reward for his integrity in, essentially, sacrificing his job (and possibly his career) to inform network admins that their networks were at risk. Not for lack of updates, but for lack of accurate information from Cisco.

You forget... Cisco had (and took) a chance to issue a patch. They simply refused to admit that the issue was a vulnerability, and repeatedly tried to coerce involved parties into not reporting that fact when it became obvious that it was known to be an exploitable flaw that they were dealing with.

Even your comments on the Microsoft issue are inaccurate. The anti-piracy measure you cite is strictly designed to verify activation information. Like all anti-piracy measures, it can be dodged, as it runs on a PC under the pirate's control. The pirate can simply nullify the software as it runs. This is something common to all anti-piracy measures. They won't stop a pirate with the time to fake the necessary information, or implement the necessary counters.

Unfortunately for Microsoft, the technology just doesn't exist to make software pirate-proof, though MS has some of the best tools in the industry (both in terms of effectiveness, and in terms of low interference for legitimate users) for making it pirate-resistant.

Mr. Hamm, remove this post, and next time... try commenting on an issue you understand. Security clearly isn't such an issue.

Matthew Murphy
Independent Vulnerability Researcher
Attendee, Black Hat Briefings USA 2005

Neville Aga

July 31, 2005 01:54 AM


I was there and saw the presentation. Steve Hamm's facts and comments are totally wrong. The subsequent comments from people defending Mr Lynn are basically correct.

The presentation was intelligent and carefully delivered. Nothing was given to an attendee so they could go out and exploit a router. The slides now circulating on the internet are not the slides he presented. He blacked out critical exploit details when he presented the talk.

Also, some words he said in the conference that you will not read in the slides need careful consideration:

- Cisco source code has been stolen at least twice.

- A substantial portion of his research came from English translations of Chinese hacking web sites.

- Cisco is considering changing their memory structure in a way that would make a router worm out of a vulnerability like this significantly easier.


By bringing these things to light in a responsible manner (I guarantee you that had you seen the talk you would have regarded it as professional and responsible), he did us all a great service.

Chuck Han

July 31, 2005 10:09 AM

I'm most interested in whether BW has or will publish Mr. Hamm's article in its hardcopy version of the magazine:

If BW goes ahead and prints it anyway after reviewing these posts and the facts, it pretty much shows that BW is just paying lip-service by allowing people to comment on online articles.

If BW has printed the article and does not prominently retracted it (again, in the hardcopy version of the magazine), it AGAIN pretty much shows that BW is just paying lip-service by allowing people to comment on online articles.

Finally, if the editors of BW don't post an online retraction of Mr. Hamm's article, it YET AGAIN pretty much shows that BW is just paying lip-service by allowing people to comment on online articles.

Jack Reynolds

July 31, 2005 10:56 AM

Shame on Business Week for not insisting that Steve Hamm double check his facts before printing an article. No wonder many people do not trust main stream media...

jack

Jonathan Metcalfe

August 1, 2005 04:30 AM

Boy, you're getting hammered aren't you Steve? How on Earth did you get this job writing such rubbish? People, this just tells me the guy is a lazy ass, and can't be stuffed checking the facts before writing.

Lyn

August 1, 2005 07:22 AM

You think BW would have the sense to take an obviously BS article down by now, but then you'd be wrong.

Steve Hamm

August 1, 2005 09:58 AM

Yikes! I reread a bunch of the news stories about this incident, and the blogs. Seems to me there are precious few "facts" that have been firmly established. I'm trying to get Cisco to help clarify things. If you know how I can get in touch with Michael Lynn to get his side of the story, tell me.

Oddball's girl

August 1, 2005 12:02 PM

You're very wet, and if you wanted to talk to Mr. Lynn, you should have done so before posting this hugely misleading and anti-factual article. As it is now, I'd be very surprised indeed if he was willing to talk to anyone who so grossly misrepresented the events.

Jim Dermitt

August 1, 2005 12:10 PM

TippingPoint, a division of networking giant 3Com, plans to pay researchers for information about unannounced vulnerabilities in major systems and software. If you find something wrong, you might as well get paid for it. And there is plenty of wrong to find. If you can find a better router, buy it! Kim Zetter at Wired wrote a piece called Router Is a Ticking Bomb. It has an interview with Mr. Lynn. There is still plenty of time. I guess the people who should be fixing the problem, will fix it and that will be that. I hope that Mr. Lynn is part of that solution. The FBI is reportedly investigating. This story has been a big deal all weekend. The Internet is still working, so I guess all isn't lost. With all of the other security issues, you can't be too careful. Corporations take risks and you have crappy insecure products out there. Sales and marketing people keep parroting how secure stuff is. It seems to me that Lynn is guity of telling the truth. He has no future as a used car salesman, for sure. Michael Lynn demonstrated mechanisms to remotely compromise Cisco routers and run malign code. Who's can say that he was the only person who could do this? I'm glad he did what he did, before somebody else did it with a different intent or created a bigger problem.

spider

August 1, 2005 01:56 PM

"I reread a bunch of the news stories about this incident, and the blogs. Seems to me there are precious few "facts" that have been firmly established."

What an absurd comment. No one who posted a comment in response to your article contradicted the other. None of the articles I've read contradict each other. Yet you seem to insist that there are no known facts about this incident. Nice way to downplay your blunder. And why would Michael Lynn want to speak to you after this?

Wordlackey

August 1, 2005 05:06 PM

What I find most interesting about this article is its bias toward the business community rather than to actual security of the internet backbone.

In theory, the concept of notifying vendors of their code holes and problems and allowing them ample time to correct them is fine. But I know of cases where such notification has been ignored for years without a patch or even a security advisory. Merely hoping no one will exploit a flaw isn't a security measure. Nor is suing anyone who discusses the problem in public. These are tactics designed to intimidate people in the industry rather than correct flaws.

Shabbadabbacadabramabbaslabba

August 2, 2005 02:13 PM

You stupid, Cisco stupid. I denowntze my sertufukashun. ISS no good hakcer asecueity. they stupid too. This website stupid tooooo, you no like good tart frame for certains!!!!!

Brydon

August 2, 2005 10:02 PM

I don't think that this BW post should be taken down, it's the controversy that gets people all excited, I didn't know about this issue until one of my employees who is an ex-Cisco contractor told me. I heartily agree with Chuck Han who posted here saying BW should not publish this article in their print version. There should definitely be a post from Steve Hamm with a correction once he has all the facts. "I'm trying to get Cisco to help clarify things." Mr Hamm, I sincerely hope that they come through for you and tell you straight what's up because you're looking quite foolish right now :-)

Alastair

August 3, 2005 07:36 AM

Your article is so far removed from the truth, you should consider a job with either Cisco or ISS ...if you aren't already working for them!!! Woolly-headed thinking and ill-informed judgements like yours will probably be right in line with their employment policies. Lynn did the right thing. Hopefully ISS's and Cisco's customers will all file lawsuits against the companies for knowingly exposing their customers to potential security risks... Let's hope so!!!!

Richard Schwartz

August 3, 2005 04:23 PM

You're trying to get Cisco to help clarify things? Cisco!?? Talk about a "Yikes!"

They're the ones trying to keep things from being clarified! Why else would they force Lynn to stop speaking about it publicly!? Come on... use some common sense here. Get some independent experts in the field of computer security, with no connection to Cisco, ISS, or Mr. Lynn, to clarify things. Preferably, get someone who was there to give you an accurate report on the facts.

The computer industry has no agreed-upon standards for disclosure of security risks. All that we can rely on is the integrity of corporations and individuals in weighing the consequences of their actions. Unfortunately, the proprietary interests of even the most reputable corporations are guarded closely (and misguidedly in some cases) by their attorneys, so the integrity of individuals is of primary importance. Now, the fact that Cisco brought in the lawyers in this case shouldn't be considered damning, but it does have to be considered significant -- at least in terms of what you are going to get from them when you go to them for "clarification". All you will get from them is their attorney-approved statements intended to protect their own best interests.

In considering disclosure issues, reputable professionals will consider the likelihood that the "bad guys" already know as much or more than the "good guys", and whether or not innocent users can be provided with ways to cope with the vulerabilities once they are exposed. Furthermore, they will provide only as much information as is necessary to convince the innocent users to implement the patches that are available as soon as possible. Lacking any accepted standard, it must be the majority opinion of the computer security community that serves as judge of whether Mr. Lynn balanced the issues and the consequences of his actions properly.

What you will find if you talk to some independent experts who where there, I believe, is that while a few who attended the conference believe that Mr. Lynn may have "crossed the line", the vast majority believe he very deliberately and expertly did not cross the line, and that he proved beyond a shadow of a doubt that the previously known vulerability in Cisco's product is worse than anyone -- especially Cisco -- had publicly acknowledged.

oen met een k

August 15, 2005 03:36 PM

Really a shame to post such an uninformed piece of writing. What was Steve Hamm's source of information?
Looks like he only bothered with the Cisco / ISS Press release.

Tim Wright

January 12, 2006 11:02 AM

It is a shame that you are so stupid! I was at Blackhat when this came out, and Mike made it clear that he is a white hat. Cisco knew about this problem and did not what it let out to the public.

You need to do a little more research before you post garbage. It is people like you that give security researchers a bad name.

To quote Napolean Dynamite "God... so stupid!!!"

Post a comment

 

About

BusinessWeek writers Peter Burrows, Cliff Edwards, Olga Kharif, Aaron Ricadela, Douglas MacMillan, and Spencer Ante dig behind the headlines to analyze what’s really happening throughout the world of technology. One of the first mainstream media tech blogs, Tech Beat covers everything from tech bellwethers like Apple, Google, and Intel and emerging new leaders such as Facebook to new technologies, trends, and controversies.

Categories

 

BW Mall - Sponsored Links

Buy a link now!