Why Patch Management is a Moving Target

Whether you use a centralized patch management system for your organization or rely on less sophisticated measures such as manual patching, you will often find that patch management is a constantly moving target. Patch management is a fundamental security task, but yet it seems to be one of the hardest in which to achieve a consistently high security “score.” And patching is only part of the issue. Reporting metrics that allow you to see tangible results are not always easy to obtain.

The data required for many measures of Cyber-security health or “scorecards” is often more readily pulled from a centralized system, and often organizations wish to report how many nodes have 100% of their security patches installed. This includes possibly a very large number of devices, depending on the size of your organization, including servers, desktops, routers, and switches. It also makes sense to focus efforts only on patches that are 30 days old and older, and that have not been superseded or replaced. The reason for this is because you are still testing the newest patches, and if you are in a very large organization have not yet had time to fully deploy all the new patches. Additionally, the older the patch, the greater the risk if that hole is still not closed. Plus, it doesn’t make sense to include patches that have been superseded by newer patches, as your scoring metrics would give erroneous results if those patches were included.

The purpose of this article is to address an example of an organizational patch status improvement effort and illustrate findings of an experiment to improve these patch statuses. This project was specifically aimed at the Microsoft specific security patches for the Windows XP Professional operating system, and all references to patches in this article will be geared toward those patches. This report will discuss the findings of that project, and describe ways that other programs can use to improve their own patching statuses. The results and recommendations are geared toward a fairly large organization. Your mileage may vary.

The Project:

I and my team recently performed an analysis of patching statuses to determine how to improve patch statuses for our entire organization. We belong to a fairly large organization with many business units, each business unit having their own IT staffs that manage their computers. One of the goals of this project was to look at ways to improve patching statuses and to document specifics concerning any anomalies that were found. By pushing out specific types of patches, and analyzing the results of those patch deployments, I was able to put together some strategies to help others with their own patching efforts. The analysis of patching efforts was performed as follows:

1) Concentrate on the "low hanging fruit" by focusing on the Microsoft critical security patches that are greater than 30 days old, and with the highest incidences of "Not Patched" statuses. This was done in phases:

Phase 1: Push out the outstanding Microsoft Office Service Packs plus Windows Defender signature files. These were chosen because they represented the largest number of un-patched vulnerabilities in the environment, as seen in the image below:

Phase 2: Push out the new Office patches offered as a result of Phase 1 completion above. This is important because applying a service pack will usually result in the computer needing additional patches that apply only to the new service pack level on the machine.

Phase 3: Push out the top 5 "not patched" patches resulting from phases 1 and 2 above.

Phase 4: Push out any remaining patches as appropriate.

2) Identify patches that are deploying successfully, but are not showing as "Patched" in our patch management system. This will include verifying that the patch is applied by using Microsoft Security Baseline Analyzer (MBSA) and Windows/Microsoft Updates.

3) Compile a list of patches that are having deployment detection signature problems and submit to the patch management system engineering for assistance with detection signatures.

4) Identify computers on the patch management system that are not checking in to get their patches. This will include looking at deployment reports to see which computers have not checked in no more than 24 hours after the deployment has been sent.

Findings:

As shown in the image below, patching statuses tend to fluctuate dramatically from day to day. This can be caused by machines falling in and out of patch status due to new patches being released to replace older versions of the same patch. For example, the Windows Defender DAT files are released approximately every three days. Rather than releasing a new patch each time, our patch management system simply replaces the existing patch with a new revision of the same patch. As the new revision is released, the computers fall out of patch status because they have the older DAT files. As the IT staffs push out the DAT files, the patching statuses go back up.

Some more specific examples of patching issues include:

Patches That Change Frequently:

The Microsoft Windows Defender DAT Files: These definition files are released by Microsoft approximately every three days. Since they are categorized as Critical-01 patches, they cause the patch statuses to fluctuate significantly every time they are released, and then again when they are subsequently deployed. This patch was the single largest reason why patch statuses greatly fluctuated from day to day.

Patches That Cause Other Patches to be Applicable:

Service Packs: Once installed, these patches tend to make the computer detect as needing additional patches. Some patches may only apply to a newer service pack level, and were thus not applicable to the machine until the latest service pack was installed.

Patches With Deployment Issues:

Microsoft Office Patches: These patches in particular were found to have a number of difficulties when deployed. In some cases, the patch is deployed, and completes successfully according to the patch management server’s deployment status. Even though the patch deployed successfully, the patch did not apply because it produced an error message that it could not find the Office installation files. In other cases the patch fails, for the same reason as stated above. Ensuring that the installation files have not been removed manually, or through the Disk Cleanup procedure typically resolves this issue. In some cases, it was necessary to uninstall and reinstall Microsoft Office, again ensuring that the Office Installation files are not removed.

Microsoft .NET Framework 1.1 SP1: This specific patch typically fails when being deployed. The reason for failure was found to have been on computers that also have the .NET Framework 1.1 Hotfix (kb928366) installed. The resolution is to go to Add/Remove Programs and remove this hotfix, deploy the .NET Framework 1.1 SP1 patch. The computer will then likely show up as needing the MS07-040 patch. Deploy MS07-040 if needed.

Patches With Detection Issues:

MS08-018 for Microsoft Project: This patch is not supposed to apply to versions of Project 2003 that have service pack 3 applied, but our patch management system incorrectly identifies the computers with Office 2003, SP3 as needing it. This is still an open issue with engineering and will hopefully be resolved soon.

Patch Management System Housekeeping Issues:

If you are using a centralized patch management system, and you are using the various reporting features to obtain your patch statuses, then it is important to take a look at housekeeping. One important thing I found in my testing was that simply deleting stale accounts out of the patch management system increased patch statuses.
The below image is an example of how much difference in patching status can be achieved just by doing housekeeping and nothing else. The patch status for the month of April was taken at the end of the month, and the patch status being shown for May was taken at the beginning of the month after clearing out all the dead computer accounts:

The result was that patching statuses for every business unit (BU) except one improved, with an overall improvement going from 42% to 58% just by clearing out dead computer accounts.

Recommendations for Improving Patch Statuses:

If you are a large organization, use a centralized patch management system. The ability to gather data on the whole organization is vital to enabling you to keep track of gaps in patching efforts.

Make sure that your centralized patch management system is being properly maintained, in terms of housekeeping. Get those stale computer accounts out of there.
Start small. Break your patching efforts into pieces, and go for the “low hanging fruit’ first. Look for the patches where the most computers need them, and start there. If you have a lot of these in your environment, break them up into groups and deploy them over several deployments if needed.

Test, test, test! If you are trying to bring an entire organization up from a dismal patching status, don’t try to push them all at once, and be sure to perform testing to make sure to discover if any patches break anything.

When pushing out service packs or roll-ups, be aware that installation of a patch of this type will often result in additional patches being applicable that were not applicable previously because of the new configuration.

Monitor patch deployments and subsequent detection results. In cases where patches deploy successfully but detect as still not patched, check to see what error messages are occurring during the deployment. In the case of Microsoft Office patches erring out, for example, ensure that the Office installation files have not been inadvertently removed from the computer.
Develop a patching routine and communicate this with your end users. Get them used to the fact that you will usually be pushing out patches the same night of the month (if they are in your central offices) and to leave their computers on that night. For remote users that receive their patches through your centralized patching system, make sure they are aware that patches will be coming to them on a certain day and give them instructions for how to properly receive the patch:

Example:

When coming in through the corporate VPN to replicate email or other databases, ensure they leave the computer on long enough to receive patches on the day you deploy them.

Other Follow-up Action:

Remember: Patch management is not something that you do once to get caught up then forget about. You have to treat patching as a constantly moving target, and always follow-up on patching efforts. Get into the habit of always keeping an eye on patch statuses and results of patch deployments.

Determine if an application is the mandated or authorized solution to be used. Sometimes you find that you are chasing patches for products that are no longer in use or maybe even not even authorized on your systems. Why patch a product that isn’t even needed? Removing it is more secure and less time consuming than patching it.

Continue to monitor patching efforts and publish lists of those patches which remain as the most likely to be causing degraded patch status.

Assist IT staffs with troubleshooting computer detection, discovery, and patch assessment issues that may exist. It could be that the patch assessments on a certain machine are out of date and not even accurate.

Monitor patch management and security discussion forums such as the patchmanagement.org listserv. If a particular patch is causing breakages or deployment issues, this is where you will find out about it the quickest.

Wrapping It All Up:

Getting a handle on patching statuses can be a real challenge for a large and geographically dispersed organization. A centralized patch management can greatly assist your efforts, particularly if you are in a large organization. Break your patching effort up into phases, and go for the “low hanging fruit” to get caught up. Be sure to continuously monitor deployments and patching statuses, and address issues where the deployments are not starting as they should, or the patch is not detecting as it should.

Laptop Security Starts with Physical Security

There has been a lot in the news lately about laptops getting stolen, and the resulting exposure of personal and other sensitive information. Protection of personally identifiable information (PII) has become a very hot topic lately, and there have been many instances in the news where PII has been exposed because of a stolen laptop. In fact, the Office of Management and Budget (OMB) in 2006 released a memo requiring government agencies to implement procedures to encrypt all agency sensitive data on laptop and other portable computing devices. This includes PDAs, Blackberries, cell phones, flash drives, and other easily stolen removable storage media. This article will be primarily discussing the loss of sensitive or personal information due to a stolen laptop or other device owned by an employer. But we could very well be discussing personal laptops and devices as well, because these security measures will apply to anything that contains data, is small, and can be easily lost or stolen. And in many cases the loss of your own personal data can be just as devastating to you as losing something that contained the data of others.

Much of what is being discussed to solve this problem involves implementing technological solutions. For example, laptops can be encrypted using something as simple as Windows’ built-in file and folder encryption, Windows Vista’s built-in BitLocker tool, or a wide variety of other full-drive encryption solutions. Blackberries can already be password protected and encrypted, and many flash drives come with built-in software to encrypt them. But using these technologies, while providing an extra layer of protection, will help protect after the loss event occurs, they do nothing to prevent the loss. Data security is more about being proactive than it is about being reactive.

These technologies offer a valid and useful solution to this problem, to be sure. But I think people are overlooking a very fundamental non-technical solution that can really go a long way to preventing these exposures – physical security. I was talking with a colleague recently, and she brought up a very valid point – if people would just do more to prevent these thefts in the first place, then we wouldn’t be where we are today, with so many instances of people winding up in the news because they allowed a laptop to be stolen from them. She said, and I strongly agree, that physical security is completely being overlooked. In fact, I would go so far as to say that the advent of all these technological solutions is actually giving people more of a reason to be less careful about protecting their laptops and other devices from theft. And all these technological solutions protect you after the fact. What ever happened to being proactive and using some prevention to avoid the theft in the first place?

How many times have we heard that a laptop has been stolen from a car? “But the car was locked,” “I was only gone for a few minutes,” “It was hidden in the back seat.” It only takes a fraction of a second to smash a window. And the thieves are getting clever and using electronic devices to help them detect if a car has a laptop inside. They can then be very selective about their targets, and easily do a “smash and grab” in very little time. “The laptop was stolen from my house. The house was locked. What could I have done?” This looks like a less preventable issue than having it stolen from a car, but let’s takes a look at what they have in common, and what the underlying issues are. Then, we will come up with some methods that can be used to protect them in each case.

Standard of Care: To being with, let’s look at the fundamental issue – if you are going to wind up in the news, it is because you did something to allow the personal information about many people to become compromised, or you were careless with a company’s secrets. The media could care less if you had your personal laptop stolen and your checkbook register, latest term paper, and resume where the only things that got stolen. If you are carrying around a laptop or PDA with a lot of PII and/or a company’s proprietary information, however, it means that you either have a piece of equipment provided by your employer, or you were keeping that information on your own personal equipment. First, I’ll discuss the later – what do your company policies say about you storing business information on your own personal computer? They don’t have a policy? That’s another issue, and I won’t cover that in this article. But even if they don’t have a policy, what does common sense tell you about it? You shouldn’t do it, period!

Now let’s look at the former – your company provided your laptop and PDA for you, and you will need to surrender it upon request. It is provided for your use to perform company business. Your employer paid for it, and hopefully they have policies about your responsibilities towards safeguarding it. This is where the commonly heard term “standard of care” comes in. Your standard of care in protecting this equipment is far greater than the standard of care you most likely exercise in protecting your personal computing equipment. You are not only responsible for protecting the equipment itself, but you are responsible for protecting the data on it as well. This may be the data about thousands of people or the trade secrets about your company’s newest product! Losing it may wind up costing you much more than just the embarrassment of media attention. Your company can be sued, and you can be sued. Or worse – federal or other regulations may have been violated, and you and your employer could wind up facing criminal charges. Termination, jail time, fines, and a long miserable process of dealing with the unwanted attention are some potential outcomes. Those ideas alone should instill a new sense of urgency in your thoughts about “standard of care” and “due diligence.”

So what can be done? This is the relatively simple part because laptops, PDAs, flash drives, and such are small – they should be easy to protect. Here are some ideas that you may find useful while taking your laptop out and about, or even just leaving it in your home, hotel, or dorm room.

Physical Protection in the Car: A laptop is light – put it in a carrying case and take it with you – just don’t leave it in the car. Is it really that tough to have to take your computer case into Wal-Mart with you? If it is, then why are you running all these errands? Take the laptop home, lock it up (see the next section), and then go shopping. I know, I know: Wal-Mart is just on the way home, and with the high price of gas, it is much more economical to stop off on the way home and pick up a few things. That’s a decision you have to make – but remember what I told you about “standard of care.” You have an obligation to safeguard this equipment and the data on it. Be prepared to take the necessary steps to protect it.

My colleague had a clever idea: She said that if you absolutely must leave it in the car, buy a computer cable and secure it. I’ll add to that, put the cable in the trunk, secure it to the frame, then secure the laptop to the cable, in the trunk. The one thing to remember is that thieves who break into cars don’t usually have a whole lot of time to spend trying to get around physical security devices such as cables. They are looking for targets of opportunity – the “low hanging fruit” so to speak. If they smash a window in broad daylight, they need to get in and get out quickly. A cable presents a significant delay, and more chances for them to get caught. If it’s in the trunk they can’t even see it in plain view, making it that much more difficult. But again, do you really need to leave it in the car? I am now putting on my “electronics geek” hat and will tell you that leaving a laptop in a car in either extreme heat or extreme cold, or leaving it exposed to the sun, is just wrong on so many levels. Forget my 30+ years of experience working with electronics. You are damaging your computer, or at the very least shortening its life by doing that!

Physical Protection in the Home, Hotels, and Dorm Rooms: There are a variety of inexpensive cables and other devices you can buy to protect laptops these days. Cables that do everything from simply physically locking down the device, to emitting an alarm when cut or broken, can be purchased and easily installed. If you are going to leave that employer owned equipment in your house, secure it to the desk. Better yet, how about locking those things up? Remember, thieves look for the low hanging fruit. If they break into your house, they aren’t going to hang out finding ways to get into secured cabinets or safes, and wait for the police to show up – they need to get in and get out. A locked filing cabinet inside a locked office does not present them with an easy target, but it shows that you were practicing due diligence in protecting these items should some brazen criminal decide to take the time to break into those secured areas.

If you’re in a hotel, it probably means that you are on travel for your job. That being the case, it should be just a simple matter of fact that you are taking your computer with you when you leave for the day for your conference or other meetings. If you are in a hotel on a pleasure trip, then why, oh why do you have your computer with you? OK – you’re probably a workaholic geek like me. In that case, then the above applies. Or ask the hotel to lock it up in their safe while you’re gone. The standard of care is then at least partially on them.

College students – even though I have been primarily focusing on employer owned equipment and data, I just have to mention you in this article also. Many of you live in dorm rooms and have computers. While the level of sensitivity of your data isn’t nearly at the level of what I have been discussing so far, can you really afford to lose that paper that is due tomorrow, and that you have been working on all night? Does your dorm room have a steady stream of visitors? Do you know all the people who your roommate invites in? Get a computer cable and lock that thing to your desk! Even if it’s a big desktop computer – lock it!

The University of Arizona has a great security poster that gives some good tips on security in the dorm room:

University of Arizona Security Posters:
http://security.arizona.edu/index.php?id=780


Physical Protection While Out and About: It is easy to let down your guard when going to the coffee shop, waiting for a flight in an airport, or just hanging out in the park. These settings all provide classic examples of how computers get stolen. In one example, a television commercial depicts a guy sitting in a coffee shop, turns around to look at a girl, then turns back – the laptop is stolen! The punch-line is “what now?!” What now, indeed? How many times do you go to the coffee shop, leave your laptop on a table, and go back to the counter to get your coffee and a donut? All it takes is for you to turn your back for a moment and for your laptop to then go missing.

You wouldn’t leave your wallet lying on a table while you go off to do something else, would you? As was stated in a 2004 Security Watch article by Robert Vamosi “…you should think of your laptop sitting on the table as a thousand dollars in cold cash; you wouldn't turn your back on that, would you?” Protect your laptop like you would your wallet or purse. Don’t take the thing out unless you are ready to use it, and you can be there to physically protect it. Robert also mentioned carrying laptops in non-descript bags. A great big black “Dell” bag is a good indicator that you are carrying a laptop. Use a padded backpack or something a little more plain.


Physical Protection While In the Office: We can’t discount security in the office or take for granted that just because your equipment is located in an office building it will be safe. First of all, just because it is in an office building, are you sure your employer’s policies don’t still hold you responsible for lost or stolen equipment? Start out by finding out what the policies are. Then, if they don’t already do so, ask your employer to purchase a security cable to secure that employer owned laptop. A number of recent articles have indicated that many, if not most, security threats come from within the organization. This can include coworkers or building custodial staff. How many people have access to your work area? If you are in a typical cube-farm, then nothing is secure. All of your work area is fair game for people to cruise around looking for easy targets.

If you are going to leave a laptop in the office or cubicle overnight, then lock as many things between public access and your equipment as possible. If it’s an enclosed office, and you are able to, lock the door. Secure the laptop with a cable or lock it in a file cabinet. Don’t lock it in one of those cubicle cupboards that someone can just lift off of the wall to get to the contents, but a file cabinet that is solid on all sides. Lock up any PDAs, flash drives, or portable storage units that you don’t take home with you. And since we’re talking about securing data in all of its forms, put away and lock up any paper, CDs, disks, or any other things that have sensitive information on it. Many organizations have a “clean desk” policy in place. And no, this doesn’t mean to take 409 and wipe down your desk every day. It means to put away and secure all items containing information: PDAs, paperwork, sticky notes, micro-film, secret decoder rings, everything!

An important note about those cables: If you do take your laptop home with you, don’t leave the cable just laying there on the desk with the combination dialed in. All someone has to do is come by, test the unlatching mechanism, and if it works, they can then look to see what the combination is. And dialing one of the numbers to one digit off won’t do it either. Set the dial to all zeros – don’t leave any clues at all. If you leave the combination dialed in, or close to it, on that cable, it doesn’t matter if you lock that laptop with the cable or not. The potential thief then has the combination and can just come back later. If you do use a combination lock instead of a key lock, change the combination periodically, just as you would change your network password periodically.


Wrapping It All up:

There are a wide variety of technologies now available to protect the data on your laptop or PDA should it get lost or stolen. But those things protect the data after the fact, provided they are in place and functioning. You still lose hours of hard work and an expensive piece of equipment. The real goal is to use some prevention and keep the asset from being lost or stolen in the first place.

Don’t be in such a hurry while running your errands that you leave an unsecured laptop in a car. Windows can be smashed and the laptop taken in seconds. Are you aware of your surroundings? When you leave the laptop on a table in a coffee shop, are you sure it will be there when you return? How about in hotel and dorm rooms? Are you sure the housekeeping staff is completely honest? Are your dorm room roommates having a lot of visitors? There are so many variables and so many possibilities to have equipment go missing.

Physical security is a preventive measure that should be taken seriously. Don’t rely solely on technologies to make data unobtainable through encryption – keep it from getting stolen and exposed in the first place. There a re a variety of low-tech to no-tech solutions to keep you from losing your equipment.. Cables, keeping the item with you, good file cabinets and locked doors will all add a significant measure of protection and security. It all begins at the lowest layer – physical security!


Additional Resources:

Security Watch: How to Protect Your Laptop While on the Road
http://reviews.cnet.com/4520-3513_7-5145310-1.html

Washington Post – “OMB Sets Guidelines for Federal Laptop Security”
http://www.washingtonpost.com/wp-dyn/content/article/2006/06/27/AR2006062700540.html

Security Posters:
http://www.us-cert.gov/reading_room/distributable.html

Georgetown University Safe and Secure Computing Quick Start Guide:
http://www3.georgetown.edu/security/10574.html

University of Arizona Security Posters:
http://security.arizona.edu/index.php?id=780

IA Newsletter – Defense in Depth
http://iac.dtic.mil/iatac/download/Vol3_No2.pdf

Information Security Magazine - Laptop Security:
http://infosecuritymag.techtarget.com/articles/february01/features_laptop_security.shtml

SearchSecurity.Com - Elements of a Security Program:
http://searchsecurity.techtarget.com/tip/0,289483,sid14_gci1210562,00.html

NIST SP800-100:
http://csrc.nist.gov/publications/nistpubs/800-100/SP800-100-Mar07-2007.pdf