Security vendors question accuracy of AV tests
503690
(InfoWorld) - Antivirus software is frequently tested for performance, so picking a top product should be straightforward: Select the No. 1 vendor whose software kills off all of the evil things circulating on the Internet. You're good to go then, right? Not necessarily. The increasing complexity of security software is causing vendors to gripe that current evaluations do not adequately test other technologies in the products designed to protect machines. Relations between vendors and testing organizations are generally cordial but occasionally tense when a product fails a test. Representatives in both camps agree that the testing regimes need to be overhauled to give consumers a more accurate view of how different products compare. "I don't think anyone believes the tests as they are run now ... are an accurate reflection of how one product relates to the other," said Mark Kennedy, an antivirus engineer with Symantec. Representatives of Symantec, F-Secure,Â and Panda SoftwareÂ agreed last month at the International Antivirus Testing Workshop in Reykjavik, Iceland, to design a new testing plan that would better reflect the capabilities of competing products. They hope all security vendors will agree on a new test that can be applied industrywide, Kennedy said. A preliminary plan should be drawn up by September, Kennedy said. One of the most common tests involves running a set of malicious software samples through a product's antivirus engine. The antivirus engine contains indicators, called signatures, that enable it to identify harmful software. But antivirus products have changed over the last couple years, and "now many products have other ways of detecting and blocking malware," said Toralv Dirron, security lead system engineer for McAfee Inc. Signature-based detection is important, but an explosion in the number of unique malicious software programs created by hackers is threatening its effectiveness. As a result, vendors have added overlapping defenses to catch malware. Vendors are employing behavioral detection technology, which may identify a malicious program if it undertakes a suspicious action on a machine. A user may unwittingly download a malicious software program that is not detected through signatures. But if the program starts sending spam, the activity can be identified and halted. Also, a program can be halted if it tries to exploit a buffer overflow vulnerability, where an error in memory can allow a bad program to run. Host-based, intrusion-prevention systems, which can employ firewalls and packet inspection techniques, can also stop attacks. The ways in which a computer can be infected also make comprehensive testing complex. For example, users may infect their computers by opening malicious e-mail attachments or visiting harmful Web sites designed to exploit known vulnerabilities in a Web browser. The different modes of attack also involve different defenses, all of which would need to be tested to arrive at an accurate ranking, analysts said. By contrast, signature-based tests can take as little as five minutes. "This is a very basic test," said Andreas Marx of AV-Test.org, who wrote his master's degree thesis on antivirus testing. "It's easy, and it's cheap." Other concerns remain, over sample sets of malicious software, the age of the samples and the relative threat those samples pose on the Internet as they become older. Security vendors also think tests should check how well security applications remove bad programs, a process that can affect a computer's performance. For vendors, a failed test can be embarrassing, since the testing companies often issue news releases highlighting the latest results. Testing companies make money in various ways. AV-Test.org is often commissioned by technology magazines such as PC World (a magazine owned by IDG). Virus Bulletin licenses its logo to companies for use in promotional material and publishes a monthly online magazine. Earlier this month, Virus Bulletin announced that its latest round of testing produced some "big-name failures," including products from Kaspersky Lab and Grisoft SRO. The company's VB100 tests antivirus engines against malware samples collected by the Wildlist Organization International, a group of security researchers who collect and study malware. To pass the VB100, products must detect all samples. Kaspersky briefly removed a signature for a worm out of its product for "optimization" purposes on the day of the test, wrote Roel Schouwenberg, senior research engineer for Kaspersky, in an e-mail. The signature has since been put back in, he said. "Obviously, we would have rather passed than failed," Schouwenberg wrote. "Had the test been conducted a day earlier or a day later, we would have passed." Similarly, F-Secure initially failed its test also because of a technicality, but the failed rating was later reversed. All vendors are told after testing which samples they failed to detect, thus most end up adding signatures to their products. So what should a user do? John Hawes, a technical consultant for Virus Bulletin, cautioned that the signature-based tests are "not enormously representative of the way things are in the real world." But Hawes also noted that signature-based tests can indicate the reliability and consistency of a vendor's software. Virus Bulletin also writes reviews of AV suites, which take into account aspects such as usability, which may be just as important as detection for consumers. The company is developing more advanced tests that will test new security technologies. AV-Test.org is already performing more comprehensive tests, although it uses between 30 to 50 malware samples, a much smaller sample set compared to the Wildlist, which uses more than 600,000 samples, Marx said. Those tests may give a better indication of how a security software suite performs. At a bare minimum, through, users should install some security software, as computers without it can face high risks, Marx said. Several free suites are available that may be fine for light Internet use, he said. Ironically, Marx doesn't use any antivirus software. That's because AV-Test.org collects malware for its testing, most of which comes through e-mail from other researchers. "I'm getting about 1,000 viruses a day," he said. "It [antivirus software] would be counterproductive." Â
671001
The U.S. International Trade Commission (ITC) has voted to investigate claims by Trend Micro of competitors&#39; patent infringement involving antivirus products.Trend Micro filed a trade complaint Nov. 21 against fellow cybersecurity vendors Barracuda Networks of Campbell, California; Panda Software International, based in Spain; and Panda Distribution of Glendale, California.Trend Micro&#39;s complaint accuses the three companies of infringing its patent for virus detection and removal apparatus for computer networks. The technology &quot;represents a dramatic departure from the traditional antivirus methods of safeguarding individual computers,&quot; the company says in its complaint.The Trend Micro complaint accuses Barracuda of importing antivirus software that infringes its patent. The complaint says Barracuda uses code from the open-source antivirus product ClamAV, which is written in part in Europe and Australia. Barracuda also imports hardware components, the complaint says.Panda&#39;s products also contain code imported into the U.S., the Trend Micro complaint says.A Barracuda spokeswoman said the complaint is curious. &quot;It is interesting that the ITC has decided to take up an investigation, given Barracuda Networks manufactures all of its products in the United States and does not import anything of significance,&quot; said Kylie Heintz.A Panda representative didn&#39;t immediately respond to a request for comments.The ITC case will be referred to an administrative law judge, who will schedule and hold an evidentiary hearing. The judge will make an initial determination as to whether there is a violation of section 337 of the Tariff Act of 1930. That initial determination is subject to commission review.The ITC will make a final determination in the investigation at the earliest practicable time, the commission said in a news release. Within 45 days, the ITC will set a target date for completing the investigation.
509953
During a weeklong period in mid-November, security experts with Symantec observed roughly 65,000 new applications being downloaded onto the computers of customers participating in a new research project -- based on their analysis of the software, as many as 60 percent of the programs were malicious.The involved timeframe represented a relative high point for the percentage of unknown applications being downloaded by Symantec's project participants, and the basis for the company's assessment of the programs as malicious was predicated largely on the programs' use of obfuscated naming conventions.However, the numbers point to a disturbing trend that the researchers say may force the security company to change its fundamental approach for warding off threats -- that being that the number of malicious applications coming to life on the Web appears to be outpacing the volume of legitimate programs.With malware authors using fuzzing tools to find holes in popular applications such as Web browsers, and testing their work against commercial anti-virus (AV) products to ensure that the attacks evade detection by the tools, leading researchers at Symantec admit that defending against threats using traditional methods has become something of a losing battle."The reality is that most new malware is going undetected by commercial security products, and not just Symantec's, but we have to recognize that like all other AV products we are probably missing a sizeable amount of this malware," said Carey Nachenberg, a member of the company's Security Response team who also wears the title of Symantec Fellow."Eventually we write [virus] signatures and get those out to customers, but it appears that a sizeable proportion of this malware never gets detected," he said. "Instead of distributing one copy of each malware program to thousands of people, they're producing a copy for as few as two or three people and then re-writing it; so, if we get one version we can remove it from a few computers, but not all the variants. The problem with this is that there is the potential over time for almost everyone to have some form of infestation, maybe in only a few years time."The trend toward malware authors using small runs of attacks to evade detection and hook as many victims as possible, known as server-side polymorphism, is forcing Symantec to reassess how it goes about protecting its users.Since it can't hope to keep up with every flavor of threat that is being created, traditional countermeasures such as the use of malware signatures or behavioral heuristics will need to be augmented with new tactics, Nachenberg said.One such alternative is the use of the same distributed data collection capabilities that Symantec is using to track the proliferation of malware. By creating a system of file and Web site reputation by studying applications usage patterns among its customers, the researcher said, Symantec hopes to use a community approach to help people determine which programs they decide to use, or avoid.Much as many people turn to the reviews section on Amazon.com or the buyer feedback system on eBay to get a real-world take on products before they decide to buy, Nachenberg contends that by watching how people are using various applications the security vendor can use a process of elimination for weeding out malware from legitimate software.If only a few people among the millions of Symantec customers who could contribute usage data to such a program were utilizing some application in question, it would be prudent to recommend that people avoid the program until its nature has been better determined, he said.Using opt-out tools that provide anonymous feedback on applications that were built into Symantec's existing Norton AntiVirus and Internet Security 2008 products, the company is already gathering the type of data necessary to create such a system of recommendation."Right now this is just a long-term research project, but we hope that as we get more users involved in the system, we can truly get a better idea of what is on people's computers so that we can identify malicious software based on the demographics of who is using it, versus what it does," Nachenberg said. "We're hoping to get more clarity through the large base of users we have; by collecting this data we should be able to get the most comprehensive view of the usage patterns to derive reputation information for everything they use."Faced with questions over potential privacy issues driven by Symantec's ability to watch just who is using what applications and how, the researcher reiterated that users must be made aware of the data collection, allowed to opt-out, and guaranteed that all the information aggregation is done in an anonymous fashion.By offering users the ability to decide whether or not to use an application based on demographics, versus simply blocking programs based on its own observations, the company will also give people more freedom to determine what tools they feel are appropriate to use, he said."If we know that only five people are using a program, given the tens of millions of users we ultimately hope to have in the system, we can be totally objective and recommend that people wait until it is scrutinized further before using it," the researcher said. "We will need to have some manual process for white-listing programs as well, but we think that using this approach we can deliver a reasonable amount of quality with a low false positive rate."If the volume of new malware strains arriving on the Web continues to outpace the proliferation of legitimate programs, Nachenberg said that AV vendors including Symantec may need to move to a white-listing approach in general, and focus more attention on identifying good applications instead of trying to chase down all the bad."If there is less software to analyze that is good, it makes more sense to spend our time scanning for good programs and simply telling our users to avoid everything else," he said. "We're considering models where we can produce the world's largest up-to-date white list of software, but it's not something we can put together in a year; maybe in two-to-three years time."
611897
(InfoWorld) - Security researcher Joanna Rutkowska has demonstrated several methods that sophisticated rootkits can use to hide from even the most reliable detection method currently available -- hardware-based products that read a system's RAM. Rutkowska is a researcher with security firm Coseinc Advanced Malware Labs. She recently outlined several ways of getting around the UAC (User Account Control) feature introduced in Windows Vista. Several researchers have identified problems with UAC. The demonstration, given at the Black Hat security conference in Arlington, Va., indicates that if a rootkit is advanced enough, there currently is no way it can be reliably detected, Rutkowska said. Rootkits are designed to hide some activity from observers and have recently been used to conceal the presence of Trojans and hacker backdoors -- not to mention Sony BMG's copy-protection software. Several hardware-based systems exist for acquiring an image of a computer's RAM, the most reliable way to detect the presence of certain kinds of rootkits, Rutkowska said. Those include Tribble, Komoku's CoPilot, and RAM Capture Tool from BBN Technologies, but she said none are particularly easy to find. Rutkowska's findings mean system designers may need to come up with a system architecture better suited to forensic analysis, such as an interface dedicated to memory acquisition. "We live in the 21st century, but apparently can't reliably read the memory of our computers," she said at the presentation. "Maybe we should rethink the design of our computer systems, so that they were somehow verifiable." Rutkowska's presentation outlined three types of attacks, one that crashes the machine when RAM acquisition is attempted, a "covering attack" that allows the acquisition tool to see only garbage when it inspects certain parts of physical memory, and a "full replacing attack" that allows the tool to see false information when it scans parts of physical memory. The denial-of-service attack could leave investigators legally liable for crashing the system, Rutkowska said. The "covering attack" might allow investigators to see the malware's "hooks" in some cases, but would make it impossible for investigators to analyze the malware. And the "full replacing attack" could simply mean that the malware is never detected at all, she said.
424521
The software maker has acknowledged it was wrong when it told users certain file formats were insecure.
405976
Letters: Going deeper on the question of security and operating systems.
51877
InfoWorld - Antivirus software is frequently tested for performance, so picking a top product should be straightforward: Select the No. 1 vendor whose software kills off all of the evil things circulating on the Internet. You&#39;re good to go then, right? Not necessarily.
597105
Company says replicating network latency, bandwidth and packet loss problems enables better lab testing.
620220
The rating system Edwin Mier, Randall Birdsall and Rodney Thayer used in their tests of Cisco and Avaya systems.
555292
Online technology retailer Geeks.com displays a security vendor's "Hacker Safe" logo on its Web site. But now it's warning customers of a possible data compromise.
769408
In 1980, I signed a deal with a company to market a product I was developing. The contract required me to turn over the source code, which I did. One day I went to a meeting at the office of the company, and there on the product manager's desk, right next to a door that opened to the outside, was a floppy labeled "Dave Winer's source code" in big letters.  If you own a Mac you may soon find out what that felt like. It's amazing to me that the tech blogosphere doesn't treat Apple's policy re broken hard disks as the huge gaping security hole that it is.  Think about it. We worry about bad people getting their hands on little pieces of data that, when added together, give them the power to be you in banking and credit transactions.  Think about what you would do if your laptop was stolen. Well, if you own a Mac and its hard disk goes bad, and you make the mistake of bringing it to Apple for service, you will turn over all that data to Apple. Not "may" or "might" but "will." What Apple in turn does with that data is none of your affair. They don't sign anything or offer any guarantees that they won't sell the disk to a data miner. Think it can't happen or that it's unlikely? I don't gain much comfort from your feeling of security. I've been writing about this issue since December 22. Usually when I write something critical of Apple, the results are mixed. Some people are supportive, but far more people attack. This time the ratio is very different. Almost everyone who has commented gets that there's a huge problem here. Can we solve this problem? Of course we can.  Some say that other vendors do this too. That gives me less comfort, not more. That means there's no escaping this crazy way of treating user's confidentiality. Ultimately it hurts the vendors because people can't use their computers for things the manufacturers say we can.  Further, it's got to be an issue for the banks, brokerage firms, credit agencies. If you are a newspaper and you employ reporters and they use a computer, how exactly are you guarding the confidentiality of your sources? If you're a confidential source, don't you have an interest when the reporter gives their computer manufacturer all their data to do with as they please? Imagine what you would do if it turned out there was a bug in a Netgear or Linksys router that allowed, under special circumstances, a mailicous person to gain access to the full content of your hard disk at any time. Would you have a problem with that? This is worse than Microsoft's neglect of malware that got me to stop using their computers. In that case it was Microsoft being neglegent. This time Apple itself is the cause of the problem. It's as if they planted a virus in their operating system that entitled them, under special circumstances, completely out of the user's control, to gain access to everything on your disk, with as much time as they want, with no way for you to prevent or even detect the intrusion.  See also: My letter to Steve Jobs.
312429
You can't cheat an honest person, they say. Like generations of scammers before them, some malware writers are taking that "advice" to heart, releasing their Trojan software and keyloggers as "cracked" versions of Vista oon peer-to-peer service. Who's going to turn them in, after all -- a would-be pirate?
653092
(InfoWorld) - On Aug. 14, IT security training and research authority SANS Institute will convene its inaugural set of exams for software developers seeking to attain its new secure coding certifications. The rise of such initiatives -- and increasing adoption of source code vulnerability scanning tools among internal software development teams -- are finally making a difference in overall applications security, some end users and industry experts contend. According to Allan Paller, director of research for SANS, based in Bethesda, Md., large businesses have moved aggressively in the last 18 months to push their applications developers to incorporate security testing into almost every facet of their work. As a result, he said, companies are finally beginning to realize significant applications security gains as developers become more careful and enlist new tools to drive vulnerabilities out of their code. "There's no older question in security than when will developers finally figure out a way to eliminate buffer overflows, everyone as asked that at some time or another, the difference is that developers themselves have discovered that they have become targets," said Paller. "People traditionally haven't thought about designing applications with the idea that someone else would be attacking them, and that's been a major discovery." It wasn't until roughly two years ago -- when more comprehensive systems software patching programs and the arrival of stronger intrusion prevention systems (IPS) pushed hackers to move their work to the applications level -- that developers were finally forced to face unwavering demands from management to clean up their code, said the expert. The internal secure development initiatives that arose from those mandates are finally bearing fruit, he said. "As soon as developers started thinking that someone would be trying to misuse everything they designed, things started to change, but it's taken time since it wasn't on their radar 18 months ago," said Paller. "Some people obviously started doing this work years ago, but it never became a trend until attackers came directly after the applications." In addition to offering security training and certification tests for programming languages including .Net and Perl, SANS is working with 60 universities to inject secure coding programming into their software development curriculums. Along with a more aggressive approach to security on the part of developers, Paller cited the arrival of more powerful applications code scanning tools as key to the recent improvements. According to a research report published by Stamford, Conn.-based Gartner in May 2007, approximately 60 percent of all IT organizations will have made security vulnerability detection an "integral" part of their software development process in 2010. By that time, an estimated 40 percent of organizations will have enlisted the help of a vendor marketing both source code scanning and Web application vulnerability testing tools, the research company said. For organizations such as the United States Agency for International Development (USAID) -- an independent federal government agency that carries out economic and humanitarian efforts under guidance from the U.S. Secretary of State -- the work to improve development security goes slowly, said Bill Geimer, a program manger in the organization. Trying to improve the security of applications development spread across the globe is no easy task, he said, but with the help of some new tools the effort is making headway. "We don't have the software development budget of Amazon.com or eBay, it's an incremental process, but we've found that developers are responding more positively to security assessment reports than we had originally imagined," Geimer said. "When it comes down to it, these people don't want to develop insecure code or get their Web sites defaced," he said. "The first phase is to fix issues before they reach production, and to do that we're trying to drive secure tools and methods into their hands." One of the technologies USAID is arming its developers with is Atlanta-based SPI Dynamics' Assessment Management Platform (AMP), used to track and measure Web applications security risks. Brian Cohen, chief executive of SPI, said that his company has been trying to sell its applications testing tools to developers for over two years, but he has only seen adoption begin to grow significantly since mid-2006. Prior to that time, the only consistent demand for the technology came in the form of partnerships with software development platform providers including IBM, Mercury and Microsoft, he said. "The message just didn't resonate with end users unless it was something built directly into one of the popular development platforms," Cohen said. "Now we're finally seeing more development groups ask for this technology for themselves; at a high level, organizations would prefer just to block vulnerabilities after the fact, but they've finally learned that they can't afford to maintain that type of approach." Despite the growing predisposition toward the use of secure development automation tools among businesses, the ages-old question of what types of problems should be handled by software designers -- and which should be tackled by IT security specialists, remains a stumbling block, according to the executive. "There's still a lack of understanding among security workers and development teams as to who is responsible for what," said Cohen. "But hopefully we'll see experience and education lead to new progress in that areas as well." Mike Weider -- founder and chief technology officer at rival Web application security testing tools vendor Watchfire, Waltham, Mass. -- said that targeted attacks against online transactional and customer service systems at retailers and financial services companies have served as a wakeup call. With the fear of having their Web sites hacked and facing sharp criticism from regulators and customers hanging over their heads, companies are finally embracing vulnerability testing applications designed specifically for use by developers, he said. In mid-April, Watchfire added features in its new AppScan 7.5 release aimed specifically at users in the development realm of Web applications quality assurance. Tired of paying high-priced consultants to ferret-out vulnerabilities on a cyclical basis, more large organizations than ever are creating security testing teams within their software development departments, he said. "Many of these companies have thousands of applications and developers, and only small teams of security professionals available to do all the work necessary to track and remove all the errors, so security teams are reaching out to development to take on some of this daunting workload," said Wieder. "Everyone from the government to financial services firms and retailers are looking at the regulatory environment and seeing the writing on the wall in terms of being forced to take responsibility for breaches; this is pushing the work down to developers who are creating demand for new tools." At least one Watchfire customer said that the availability of such security testing technologies is allowing it to make headway in instilling new secure coding practices among its developers. "We used to hire a third party to do vulnerability assessment of production level code, then we would report their findings to business unit, and unless something was high-level it would just get put on a release schedule," said Ethan Stieger, chief security officer at automotive market intelligence firm Polk Global Automotive, based in Southfield, Mich. "This technology allows us to fix things when they're being developed, which is the right way to do things, if you wait until production, that's a lot more invasive in environment, and the security issues are exposed to the outside world," he said. "We're still tying to sell this effort internally, but now we can do so on the commercial value of keeping potential attacks at bay."
386296
The facts of web servers don't necessarily match the myths when it comes to security.
371222
Sniffing out malicious code On-demand application security testing firm Veracode has added detection for backdoors and malicious code to its services. The addition aims to tap into concerns about the integrity of code developed by outsourced contractors.â¦
684924
Spyware is rife and virus infection commonplace yet many home users reckon they are safe from online threats. An AOL/National Cyber Security Alliance (NCSA) Online Safety Study - conducted by technical experts
670395
(InfoWorld) - Trying to lock down your company's applications and protect your systems from attack? If so, security scanners and source-code analysis tools are not up to the job -- despite vendor claims to the contrary. "There's an awful lot of marketing spiel, people introducing technology tools that are sold as silver bullets," said Mark Curphey, vice president of professional services at McAfee's Foundstone division, in an interview. "The reality is, in a large enterprise, those things generally don't work." "Technology is increasing at such a fast and crazy pace, but security technology isn't keeping up with it. With application security it's even worse," Curphey said. Security tools, such as code scanners, are able to detect just 1 percent to 2 percent of vulnerabilities in an application, leaving "gaping holes" behind, he said. Curphey, who has worked on security for U.S. financial institutions, is speaking this week at the Hack In The Box Security Conference (HITB) in Kuala Lumpur, Malaysia. Software vendors and companies must get back to basics to secure their applications. "What the smart banks are doing about this is they're building security into their development life cycles and that's exactly what Microsoft has done," he said. Since being hit by a series of high-profile vulnerabilities in its software, Microsoft hasÂ significantly improved the securityÂ of the software it produces, resulting in fewer critical vulnerabilities being reported. "Microsoft are going to be the security vendor of choice in the next couple of years," Curphey said. "There's been a phenomenal turnaround in that company." While a company's IT systems may never be perfectly secure, IT managers should be able to identify and mitigate risks. "At the end of the day, security is about building something that's secure enough to do business on," Curphey said. To help protect their systems, companies need to be sure they are hiring IT staff with the right set of technical skills, setting business processes that determine how to respond to attacks on the system, and providing staff with the tools they need to respond when attacks happen, he said. HITB runs through Thursday, Sept. 21.ADVERTISEMENTMAKE SOA REAL WITH IBM WEBSPHERE PODCASTStreamline processes and maximize efficiency within a proven service oriented architecture. Take back control of your business processes with IBM WebSphere.