SharePoint 2007: Upgrading and Errors 6482, 6398 and 10016

Problem

I had completed a deployment of MOSS 2007 Enterprise onto a Windows Server 2003 Enterprise Edition machine and was now installing upgrades.  After installing both WSS3.0 SP1 and MOSS SP1, I checked the system's event logs and was nonplussed to see DCOM errors filling up the System log and Windows SharePoint Services and Office SharePoint Server events filling the Application log:

Event Type:       ErrorEvent Source:      DCOM
Event Category:   None
Event ID:            10016
Date:                  3/26/2011
Time:                  12:57:26 PM
User:                  CORP\spservice
Computer:          2003K1
Description:
The application-specific permission settings do not grant Local Activation permission for the COM Server application with CLSID {3D42CCB1-4665-4620-92A3-478F47389230} to the user CORP\spservice SID (S-1-5-21-1450330090-4081781163-1502405366-1106).  This security permission can be modified using the Component Services administrative tool.
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Event Type:      ErrorEvent Source:     Office SharePoint Server
Event Category:  Office Server Shared Services
Event ID:            6482
Date:                  3/26/2011
Time:                 12:55:58 PM
User:                 N/A
Computer:         2003K1
Description:
Application Server Administration job failed for service instance Microsoft.Office.Server.Search.Administration.SearchServiceInstance (c3e25954-acf6-49d1-a800-d3dcb60cff9a). Reason: Retrieving the COM class factory for component with CLSID {3D42CCB1-4665-4620-92A3-478F47389230} failed due to the following error: 80070005.
Techinal Support Details:
System.UnauthorizedAccessException: Retrieving the COM class factory for component with CLSID {3D42CCB1-4665-4620-92A3-478F47389230} failed due to the following error: 80070005.
   at System.Runtime.Remoting.RemotingServices.AllocateUninitializedObject(RuntimeType objectType)
   at System.Runtime.Remoting.RemotingServices.AllocateUninitializedObject(Type objectType)
   at System.Runtime.Remoting.Activation.ActivationServices.CreateInstance(Type serverType)
   at System.Runtime.Remoting.Activation.ActivationServices.IsCurrentContextOK(Type serverType, Object[] props, Boolean bNewObj)
   at Microsoft.Office.Server.Search.Administration.Gatherer.get_AdminObject()
   at Microsoft.Office.Server.Search.Administration.Gatherer.ProvisionGlobalProperties()
   at Microsoft.Office.Server.Search.Administration.SearchServiceInstance.Synchronize()
   at Microsoft.Office.Server.Administration.ApplicationServerJob.ProvisionLocalSharedServiceInstances(Boolean isAdministrationServiceJob)
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Event Type:      ErrorEvent Source:    Windows SharePoint Services 3
Event Category: Timer
Event ID:           6398
Date:                 3/26/2011
Time:                12:56:26 PM
User:                N/A
Computer:         2003K1
Description:
The Execute method of job definition Microsoft.Office.Server.Search.Administration.IndexingScheduleJobDefinition (ID da517b18-c29d-447e-b1f6-73d0d688d9c0) threw an exception. More information is included below.
Retrieving the COM class factory for component with CLSID {3D42CCB1-4665-4620-92A3-478F47389230} failed due to the following error: 80070005. For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

The DCOM error 10016 recurred approximately every 30 seconds, and the Office SharePoint Server error 6482 and Windows SharePoint Server Timer error 6398 recurred approximately every 60 seconds.  Below are the exact steps I took - in sequential order - that eventually and permanently resolved this error.  I hope this will assist you in your troubleshooting as well.  Good luck!

Troubleshooting

1. I began by restarting IIS.  Checking the event logs afterwards found that all of the errors were continuing.
2. Stopped and started the Windows SharePoint Services Timer service.  Checking the event logs afterwards found all of the errors continuing.
3. Re-entered the service account credentials for the Windows SharePoint Services Timer service.  Still no change.
4. Verified that CORP\spservice was a member of the local WSS_ADMIN_WPG group.
5. Installed WSS3 SP2 and MOSS SP2.  Still no change.
6. Installed hot fix 923028 for .NET 2.0 [4] as recommended by [1].  Same errors.
7. I then revised my Internet search approach somewhat, this time searching the actual error text rather than its number and found [5].  This proved the key article to help me resolve this error.
8. As discussed in [5], stopped both search services: Central Administration > Operations Tab > Services on Server:  Office SharePoint Server Search and Windows SharePoint Services Help Search.
9. Started Office SharePoint Server Search first.  Doing so requires that you reconfigure the service.  I reconfigured the service using the same parameters used previously including using the same account.  This served started without issue.
10. Attempted to start Windows SharePoint Services Help Search. Doing so requires that you reconfigure the service.  I reconfigured the service using the same parameters used previously including using the same account. After clicking OK, I experienced an error: WSS_Search_2003K1 on 2003K1\SQLEXPRESS contains user-defined schema.  Databases must be empty before they can be used.  Delete all of the tables stored procedures and other objects or use a different database.  This error message was encouraging, as it gave additional detail on the actual cause of the error.

11. I went back to the configuration page (for this service), this time entering a different name for the database: WSS_Search2_2003K1 instead of WSS_Search_2003K1.  When I clicked OK, there was no error and I was returned to the Services on Server page which indicated that the Windows SharePoint Services Help Search had started.
12. Navigated to the Shared Services Administration page, and then clicked on Search Settings.  This navigated me to another error:

13. Clicked the link provided: Configure an indexer and a search database for this Shared Services Provider.  This took me to the Edit Shared Services Provider page. 
14. I left everything default except for reentering the service account password; and when I scrolled to the bottom, I noticed that no indexer was selected so I made that selection and then clicked OK. 
15. Navigating back to the Shared Services Administration: SharedServices1 page, this time, when I click on Search settings, I was taken to the Search Settings page without issue.
16. As a last measure, I reset all crawled content.
17. I then checked the server Application and System logs: no more occurrences of errors 10016, 6398 or 6482.  After eight hours, these errors still hadn't recurred.
18. Issue resolved!  Happy computing!

References

1. Official Blog of the SharePoint Product Gurus: Timer Job Shocker
2. Microsoft TechNet Enterprise Search: How to stop Office SharePoint Server Search Service
3. SharePoint Errors: 6482, 7888, 6398 and 7076
4. FIX: You may be unable to manage IIS by using Server Manager if two threads access IIS at the same time
5. Application Server Admnistration job failed for service instance Microsoft.Office.Server.Search.Administration.SearchServiceInstance - The device is not ready
6. Event ID Errors: 6398, 6482, and 6641 for Share Point
7. The product expiration date is activated incorrectly when you install 2007 Microsoft Office servers Service Pack 2
8. SharePoint, ASP.NET and more: Error event ID 6398 and 6482 – about security rights of OSearch service
9. The SharePoint Users Group: Making MOSS Work for You
10. EventID.NET
11. Event ID 10016, KB 920783, and the WSS_WPG Group
12. Event ID error messages 10016 and 10017 are logged in the System log after you install Windows SharePoint Services 3.0

Notes

• I suspect that the actual error fix lies somewhere among steps 11-17.  Unfortunately, I didn't check the server event logs each time after executing a task.
• There is a flaw in SP2 that sets your licensing to trial mode during SP2 installation.  The fix is simple: re-enter your existing license.