Monday, July 19, 2010

SCCM is ConfigMgr and diagnosing BITS transfer errors

I'm told that SCCM should actually be referred to as ConfigMgr, as Microsoft doesn't actually own the SCCM trademark. Personally, not working for Microsoft, I could care less, but to make life easier on people, I'm going to start tagging posts with both terms.

On to the meat of things, which is how to diagnose ConfigMgr BITS transfer errors in Windows 7, and what a common error you might encounter is. When you configure a ConfigMgr site system for app distribution on a Server 2008 or 2008 R2 machine, it sets up IIS and enables BITS, the Background Intelligent Transfer Service. BITS is neat in that it can trickle data down to a machine, allowing it to download files without chewing up all of the user's apparent bandwidth. BITS works over http and https, which is why IIS is required for distribution points. BITS usually works pretty well, downloading packages quickly from the nearest distribution point when the user's network connection is idle.

Sometimes, however, a system may hang up on a BITS download. I'm not speaking here about the common failures of ConfigMgr downloads, such as IIS misconfigurations (WebDAV needs to be enabled and properly configured, for instance), location issues, impersonation issues (if a user is logged on, the distribution point is selected and accessed using the user's account as opposed to the computer account; this also means that if you have multiple sites, the site for the domain in which the user account lives is used), or the like. No, a BITS error appears when you begin a download and it stalls at a particular percent, seemingly at random. You can check on BITS problems in two ways -- using PowerShell, or by using bitsadmin. Either one needs to be run as an administrator to be used for ConfigMgr troubleshooting. Unfortunately, the PowerShell cmdlets for BITS are functionally useless, as they can basically only tell you a job's rough status. Therefore, we must delve into the deprecated bitsadmin command.

The first step is to find the job of interest. To do this, run "bitsadmin /list /allusers", which will list all of the BITS jobs on the system. ConfigMgr's jobs are named "CCMDTS Job" and will look something like this:

{7DDD0DB1-33BC-4EFE-BE0E-E904B362D55F} 'CCMDTS Job' TRANSFERRING 0 / 2 811168 /

The first field is the job's GUID. The second field is the name, while the third is the status. The fourth field lists the number of files completely downloaded out of the number of files in the job, and the fifth field lists the number of bytes downloaded out of the number of bytes in the job. This job is healthy; it's listed as status TRANSFERRING. A job can also be in the TRANSFERRED state, which means it's done (and ConfigMgr usually cleans those up for you), or in the ERROR state.

To find out why a job has ERRORed out, you can look at it with "bitsadmin /info GUID /verbose", where GUID is the GUID of the job. A list of files will appear under the JOB FILES header of the output. Each line shows the source file URL and the destination path. The error is usually on the last file downloaded, which is the first one in the list; any error will appear on that line. For instance, you might get an http 404 error, which means that the file isn't available on the distribution point. You can verify that by copy-pasting the source URL into any web browser. A 404 error can be due to a failure of the package to copy (though usually this'll result in ConfigMgr's LocationServices throwing a fit and blocking the download), but sometimes, it's a case of missing a specific fiddly bit in ConfigMgr setup on IIS 7.0.

You see, IIS 7.0 blocks certain extensions from being downloaded from the server. It essentially makes them invisible if they're not handled by a type handler, which is good if, for instance, your legacy web application uses an Access database. You probably wouldn't want that to be downloaded, and it probably needs to be accessible by the web application's service account, so IIS 7.0 just makes it impossible to download. These filters are set up in the %windir%\system32\inetsrv\config\applicationHost.config file. Search for the requestFiltering section, and you can see the entries. If any file in your ConfigMgr application package has an extension that's in that list -- perhaps because it uses an Access database -- LocationServices will allow the download to occur, but the BITS transfer will fail because the file is "unavailable," IIS having made it invisible. The fix is simple and documented here:

  • Change the "allowed" "false" to "true" for the extensions in question, then 
  • Run "iisreset /restart" from your nearest friendly command prompt

Be aware that this makes your IIS configuration less secure, though if you're like most people, your distribution point servers only run IIS for DP duties, so this is an irrelevant concern. Now that you've done this, you can go back to your machine and run "bitsadmin /restart GUID", which will force the job to restart from the error state. If nothing else is blocked, you're in business!

Other BITS errors to be aware of are:

  • http 500: An Internal Server Error means you should check the Application event log on the DP server. If this is the first time it's happened, recycling the application pool for the DP site and restarting the BITS service on the server should fix the issue. If not, make sure BITS is installed properly and WebDAV is properly configured.
  • http 403: Make sure the user attempting to access the DP has the required rights to do so; this means that the security on the distribution point location should permit proper groups (usually, DPMachine\Users) read access. This should really only be an issue in a cross-forest configuration, where the default configuration of the Users group on the DP machine doesn't include authenticated users from the other forest. My advice here is to not cross forest boundaries with your DPs, especially as AD Site boundaries behave strangely across forests. Avoiding cross-forest DPs is a key part of a KISS design for ConfigMgr; if you need to do this, set up different ConfigMgr sites in the forests and wire them together in a site hierarchy.