SharePoint 2013: How to Rebuild an OWA farm

Introduction

This posting walks through the process of rebuilding Office Web App (OWA) services for a SharePoint 2013 farm.  Rebuilding an OWA farm is simple and quick and is sometimes the most efficient method for recovering from severe errors or service failures. 

This posting assumes a single-server internal OWA farm that exposes its service over HTTP (not HTTPS) for an internal corporate environment and that has Microsoft Office volume licensing.  It also assumes an internal SharePoint 2013 farm that uses authentication over HTTP and does not employ encrypted connections.

Procedure

1. Remove SharePoint Bindings
1. Login to any server in the farm that hosts SharePoint 2013, using the Setup User Administrator account (eg, spAdmin).

   NOTE: always use the SharePoint Setup User Administrator account for any SharePoint installation and configuration activities.

2. Open a SharePoint Management Shell with elevated privileges (as administrator).
3. Execute the following:

   Remove-SPWOPIBinding -All:$True

   This removes the Office application bindings that SharePoint has to the OWA server.
4. Logout of the SharePoint server.
2. Remove the OWA server from the farm
1. Login to the OWA server using an administrator account (no need to use the SharePoint Admin account - this is not really a SharePoint server).
2. Open a PowerShell window with elevated privileges (as administrator).
3. Execute the following:

   Import-Module -Name OfficeWebApps
   Remove-OfficeWebAppsMachine

   This doesn't uninstall the OWA software; it only removes the local OWA server from the OWA farm.

   NOTE: if you have more than one OWA server in the OWA farm, you must first execute this command on each of the child OWA servers first. Then you can execute this command on the final, master OWA server.

4. Remain logged onto the OWA server.  There's still one more thing to do.
3. Re-build the OWA Server
1. Uninstall the software using standard Windows uninstallation method.

   NOTE: you must uninstall the OWA software and rebuild fresh due to how the updates were previously installed. Uninstalling the OWA software will also get rid of the updates.

2. Restart the OWA server.
3. Logon using an administrator account (no need to use the SharePoint Admin account).
4. Re-install OWA.
5. Re-install updates.
6. Restart the OWA server.
4. Re-build the OWA farm
1. Login to the OWA server again using an administrator account.
2. Open a PowerShell window with elevated privileges (as administrator).
3. Execute the following:

   Import-Module -Name OfficeWebApps
   New-OfficeWebAppsFarm -InternalURL "http://servername" -AllowHttp -EditingEnabled

   where servername is the actual name of the OWA server host machine. Do not use the fully qualified name. Executing this command builds a new OWA farm.
4. Next, execute the following:

   Get-OfficeWebAppsMachine

   This returns the Roles and health status of the local OWA server. For my OWA farm, I wanted the roles to be All and health status Healthy. 

   NOTE: if the status returned is Unhealthy, verify that the HTTP-Activation feature has been added for both .NET 3.5 and .NET 4.5.  If not, add these features and then check status again.  It may take afew minutes for this feature change to have an affect.

5. Logout of the OWA server.
6. Open any browser on your desktop machine (assuming it can connect to the OWA server), and then connect to the following:

   http:/servername/hosting/discovery

   If everything's working correctly, you will see the service discovery file in XML format.
7. Close the browser.
5. Add SharePoint Bindings
1. Login to any SharePoint farm server (ie, hosting SharePoint) using your farm's SharePoint Setup User Administrator account.
2. Open a SharePoint Management Shell with elevated privileges (as administrator).
3. Execute the following:

   Get-SPWOPIBinding

   This should return nothing, since all WOPI bindings should have been removed back in step 3. If it doesn't, execute Remove-SPWOPIBinding.
4. In this same SharePoint Management shell, execute the following:

   New-SPWOPIBinding -ServerName <WacServerName> -AllowHTTP

   where ServerName is the actual name of the OWA server host machine. Do not use the fully qualified name. This command builds the bindings in SharePoint to the OWA service application.
5. In this same SharePoint Management shell, execute the following:

   Get-SPWOPIBinding

   This should now return a list of Office applications and actions and their bindings.
6. In the same SharePoint Management shell, execute the following:

   Get-SPWOPIZone

   This should return internal-http, given the WOPI authentication type that was configured in step 22, "-AllowHTTP."
7. In the same SharePoint Management shell, execute the following:

   (Get-SPSecurityTokenServiceConfig).AllowOAuthOverHttp

   This should return True, if you are not using HTTPS for connections to your farm.
8. OWA services should be available immediately, and you can test right away.

References

• Remove-OfficeWebAppsMachine
• Remove-SPWOPIBinding
• Import-Module
• Fun configuring Office web apps 2013 (OWA)
• Install Office Web Apps 2013 for SharePoint 2013 – with PDF Preview
• Installing SharePoint 2013 Office Web Applications Server
• A quick-guide to setting up OWA with SharePoint 2013 – start to finish
• Office Web Apps 2013 multi servers (NLB) Installation and Deployment for SharePoint 2013 step by step guide
• How to remove Office Web Apps 2013 with Service Pack 1 and apply Security Update 2878219
• General
• Plan Office Web Apps Server
• Deploy Office Web Apps Server
• Configure Office Web Apps for SharePoint 2013
• Windows PowerShell for Office Web Apps Server
• Apply software updates to Office Web Apps Server

SharePoint 2013: how to remove a cache host from an unrecoverable farm server

Introduction

This posting walks through the steps necessary for removing a distributed cache host from a SharePoint farm server that has been removed, terminated, or is otherwise unrecoverable.

Procedure

1. Preparation
1. Identify the name of the unrecoverable cache host (eg, SERVER1).
2. Identify the connection string used by the distributed cache service to connect to the farm database:
1. On any farm server hosting SharePoint, launch a command prompt with elevated privileges.
2. Execute regedit.
3. Navigate to: HKLM\SOFTWARE\Microsoft\AppFabric\V1.0\Configuration.
4. Open a new NotePad instance.
5. Copy the contents of the ConnectionString parameter to NotePad.
6. Remove the last parameter of this string, or ";Enlist=False" including the ";".
7. Save this text; you'll need it later.
2. Connect
1. Login to a SharePoint farm server as the SharePoint Setup User Administrator account.
2. Launch the SharePoint 2013 Management Shell with elevated privileges.
3. Check cache status
1. Execute Use-CacheCluster and then Get-CacheHost to check current distributed cache status.
4. Unregister the cache host:
1. Execute Unregister-CacheHost -HostName "YourHostName"
2. At the prompt for ProviderType, enter SPDistributedCacheClusterProvider.
3. At the prompt for ConnectionString, enter the string you saved back in Preparation step 1.7.  Unregistration takes only a few seconds.
5. Check cache status
1. Execute Get-CacheHost to check current distributed cache status.  The target cache host should no longer be listed.
2. Execute Stop-CacheCluster followed by Start-CacheCluster: these commands should complete without any errors and without the target cache host being listed.

References

• Manage the Distributed Cache service in SharePoint Server 2013
• Delete Cache Host, SharePoint 2013
• AppFabric Caching PowerShell Cmdlets: this is the complete reference for all PowerShell commands used for AppFabric.
• Use-CacheCluster
• Get-CacheHost
• Unregister-CacheHost
• Stop-CacheCluster
• Start-CacheCluster
• Sample: RemoveCacheHost Script (Windows Server Appfabric Caching)
• Windows Server AppFabric Caching Deployment and Management Guide: this was written for SharePoint 2010, but is still an outstanding reference.  This link takes you to the downloadable document.
• AppFabric 1.1 Caching Deployment and Management Guide

Notes

• Stop-CacheCluster: There's no need to bring down the cache cluster before unregistering the target cache host.  if you do first bring it down, and then try to execute the Unregister-CacheHost command, you may experience an error:

  Unregister-CacheHost : ErrorCode<ClusterNotInitialized>: SubStatus<ES0001>:No cluster configuration is present in target.

  .
• Unregister-CacheHost: Don't use the Unregister-CacheHost format presented in the Windows Server AppFabric Caching Deployment and Management Guide, page 13.  This won't work.  Instead, use the steps I've provided in the Unregister the cache host section, above.
• Servers in Farm: you may need to remove the target server from the farm.
• ConnectionString: you can also use PowerShell to get this value.  To do so: open a SharePoint management shell with elevated privileges.  Then execute the following:

  Get-ItemProperty "HKLM:\SOFTWARE\Microsoft\AppFabric\V1.0\Configuration"

  Note that you are using "HKLM" and not "HKEY_LOCAL_MACHINE".

SharePoint 2013 OWA: Server Error: We're sorry. An Error has occurred. We've logged the error...

Problem

Users attempt to view online a Microsoft Office document in a site library, but experience the following error:

This error is seen when attempting to view any type of Office document.  You then login to the Office Web Apps 2013 server, view its Application log, and find it rapidly filling up with multiple versions of Event IDs 1000 and 1026, such as:

Log Name:      Application
Source:        Application Error
Date:          [date/time]
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      [OWA servername]
Description:
Faulting application name: BroadcastWatchdog_App.exe, version: 
15.0.4502.1000, time stamp: 0x512d264c
Faulting module name: KERNELBASE.dll, version: 6.2.9200.16864, 
time stamp: 0x531d34d8
Exception code: 0xe0434352
Fault offset: 0x0000000000047b8c
Faulting process id: 0x1b54
Faulting application start time: 0x01cfa51426c90393
Faulting application path: C:\Program Files\Microsoft Office Web 
Apps\BroadcastServicesWatchdog_App\BroadcastWatchdog_App.exe
Faulting module path: C:\Windows\system32\KERNELBASE.dll
Report Id: 6525fef4-1107-11e4-93ff-00155d38891e
Faulting package full name: 
Faulting package-relative application ID: 
Event Xml:

and

Log Name:      Application
Source:        Application Error
Date:          [date/time]
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      [OWA servername]
Description:
Faulting application name: Microsoft.Office.Excel.Server.EcsWatchdog.exe, 
version: 15.0.4511.1000, time stamp: 0x5164ab86
Faulting module name: KERNELBASE.dll, version: 6.2.9200.16864, 
time stamp: 0x531d34d8
Exception code: 0xe0434352
Fault offset: 0x0000000000047b8c
Faulting process id: 0x49c
Faulting application start time: 0x01cfa51428fbb3ad
Faulting application path: C:\Program Files\Microsoft Office Web
Apps\ExcelServicesEcsWatchdog\Microsoft.Office.Excel.Server.EcsWatchdog.exe
Faulting module path: C:\Windows\system32\KERNELBASE.dll
Report Id: 66efc6c6-1107-11e4-93ff-00155d38891e
Faulting package full name: 
Faulting package-relative application ID: 
Event Xml:

and

Log Name:      Application
Source:        .NET Runtime
Date:          [date/time]
Event ID:      1026
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      [OWAservername]
Description:
Application: WordViewerAppManagerWatchdog.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.TypeInitializationException
Stack:
   at Microsoft.Office.Web.Common.ServiceInstanceFinder.
      GetLocalAgentInstance(Microsoft.Office.Web.Common.OfficeServiceType)
   at Microsoft.Office.Web.Common.WatchdogHelper.PrepareRegistrations
      (Microsoft.Office.Web.Common.OfficeServiceType)
   at Microsoft.Office.Web.Common.WatchdogHelper.WatchMachines
      (Microsoft.Office.Web.Common.OfficeServiceType, CheckServiceInstance, 
         Microsoft.Office.Web.Common.OfficeServiceType, System.String)
   at Microsoft.Office.Web.WordViewerWatchdog.Program.Main()
Event Xml:

and many others of a similar nature.  You recall that you recently installed updates to the OWA server, via Systems Center Configuration Manager (SCCM), and that some of these involved OWA.

Discussion

Applying updates to Office Web Apps server via SCCM or Windows Updates is not supported and may cause service application failure and loss of OWA functionality to all your users.  Recovery from this situation involves rebuilding the entire OWA service application and its bindings to the SharePoint farm; as well as modifications to your organization's Windows patching SOP. 

Total Solution Time Estimate: 3 hours.

Solution

1. Login to any server in the farm that hosts SharePoint 2013, under the Setup User Administrator account (eg, spAdmin). 

   NOTE: always use the SharePoint Setup User Administrator account for any SharePoint installation and configuration activities.

2. Open a SharePoint Management Shell with elevated privileges (as administrator).
3. Execute the following:

   Remove-SPWOPIBinding

   This removes the Office application bindings that SharePoint has to the OWA server.
4. Logout of the SharePoint server.
5. Login under any administrator account to the OWA server.
6. Open a PowerShell window with elevated privileges (as administrator).
7. Execute the following:

   Import-Module -Name OfficeWebApps
   Remove-OfficeWebAppsMachine

   This doesn't uninstall the OWA software; it only removes the local OWA server from the OWA farm. 

   NOTE: if you have more than one OWA server in the OWA farm, you must first execute this command on each of the child OWA servers first.  Then you can execute this command on the final, master OWA server.

8. Uninstall the software using standard Windows uninstallation method.

   NOTE: you must uninstall the OWA software and rebuild fresh due to how the updates were previously installed.  Uninstalling the OWA software will also get rid of the updates.

9. Restart the OWA server.
10. Re-install OWA.
11. Re-install updates.
12. Restart the OWA server.
13. Login to the OWA server under an administrator account.
14. Open a PowerShell window with elevated privileges (as administrator).
15. Execute the following:

    Import-Module -Name OfficeWebApps
    New-OfficeWebAppsFarm -InternalURL "http://servername" -AllowHttp -EditingEnabled

    where servername is the actual name of the OWA server host machine.  Do not use the fully qualified name.  Executing this command creates a new OWA farm.
16. Next, execute the following:

    Get-OfficeWebAppsMachine

    This returns the Roles and health status of the local OWA server.  For my OWA farm, I wanted the roles to be All and health status Healthy.
17. Logout of the OWA server.
18. Open any browser, and then connect to the following:

    http:/servername/hosting/discovery

    If everything's working correctly, you will see the service discovery file in XML format.
19. Login to any SharePoint farm server (hosting SharePoint), using your farm's SharePoint Setup User Administrator account.
20. Open a SharePoint Management Shell with elevated privileges (as administrator).
21. Execute the following:

    Get-SPWOPIBinding

    This should return nothing, since all WOPI bindings should have been removed back in step 3.  If it doesn't, execute Remove-SPWOPIBinding.
22. In this same SharePoint Management shell, execute the following:

    New-SPWOPIBinding -ServerName <WacServerName> -AllowHTTP

    where ServerName is the actual name of the OWA server host machine. Do not use the fully qualified name.  This command builds the bindings in SharePoint to the OWA service application.
23. In this same SharePoint Management shell, execute the following:

    Get-SPWOPIBinding

    This should now return a list of Office applications and actions and their bindings.
24. In the same SharePoint Management shell, execute the following:

    Get-SPWOPIZone

    This should return internal-http, given the WOPI authentication type that was configured in step 22, "-AllowHTTP."
25. In the same SharePoint Management shell, execute the following:

    (Get-SPSecurityTokenServiceConfig).AllowOAuthOverHttp

    This should return True, if you are not using HTTPS for connections to your farm.
26. OWA services should be available immediately, and you can test right away.

References

• Office Web Apps Server
• Deploy Office Web Apps Server
• Configure Office Web Apps for SharePoint 2013
• Apply software updates to Office Web Apps Server
• Plan for administrative and service accounts in SharePoint 2013
• Plan for least-privileged administration in SharePoint 2013
• Windows PowerShell for Office Web Apps Server
• Remove-SPWOPIBinding
• Remove-OfficeWebAppsMachine
• New-OfficeWebAppsFarm
• Get-OfficeWebAppsMachine
• Get-SPWOPIBinding
• New-SPWOPIBinding
• Get-SPWOPIZone
• Get-SPSecurityTokenServiceConfig
• Office Updates
• Fun configuring Office web apps 2013 (OWA)
• A quick-guide to setting up OWA with SharePoint 2013 – start to finish: lots of good and accurate content to help you get started.
• Install, Configure & Monitor Office Web Apps 2013 for SP 2013: good section on troubleshooting and monitoring.
• How to remove Office Web Apps 2013 with Service Pack 1 and apply Security Update 2878219: this was the primary reference I used to guide my recovery efforts.

Notes

• This solution assumes a single-server internal OWA farm that exposes its service over HTTP (not HTTPS) for an internal corporate environment and that has Microsoft Office volume licensing.
• It also assumes an internal SharePoint 2013 farm that uses authentication over HTTP and does not employ encrypted connections.

SharePoint 2013: How to expand server disks on HyperV

Introduction

This posting walks through the process of expanding a SharePoint Server system disk for Windows Server 2012 servers hosted on Hyper-V.

Procedure

1. Compile a listing:
1. Name of SharePoint server
2. Current disk sizes
2. Shutdown all servers on this list (PowerShell: Shutdown /s)
3. Launch Hyper-V Manager
4. Connect to the appropriate Hyper-V host.
5. Under Virtual Machines, select the SharePoint server.
6. In the Actions panel, click Edit Disk...
7. Navigate to the folder containing the virtual disk file, and then select this file.
8. Click Open.
9. Click Next.
10. Select the Expand option, and then click Next.
11. Click Finish.

References

• Expanding Virtual Hard Disks with Hyper-V: this posting is old but remains accurate.

Notes

• The expanded disk is available immediately.

SharePoint 2013: How to Restore Content Databases

Introduction

This posting walks you through the process of restoring one or more content databases to a SharePoint Server 2013 farm web application.  This procedure can be used to restore a production farm web application or even to "refresh" a 2013 development environment with the latest content from the production site, via just the production site content databases or a backup of these databases.  It assumes a farm having a single SQL Server database.

Procedure

1. Identify the web application that you want to restore.
2. Identify the content database(s) of this web application and their location.
3. Logon as administrator (not the farm administrator) to the SQL Server of the target farm.
4. Create a local folder, and copy to this folder the full backups of the content databases that you want to restore.
5. Logout.
6. Login as the Farm Setup User Administrator account (eg, spAdmin) to a SharePoint server of the target farm.
7. Launch Central Administration
8. Navigate to Application Management > Databases > Manage content databases.
9. Select the appropriate web application from the Web Application dropdown.
10. Click on the title of a content database.
11. Enable the option, Remove content database, and then click OK at the popup prompt.
12. Click OK again.
13. Logout.
14. Login to the target farm's SQL Server as administrator.
15. Launch SQL Server Management Studio with elevated privileges (right-click).
16. In Object Explorer, expand the tree to find the content databases.
17. Right-click, point to Tasks and then click Detach...
18. Select the database to be detached.
19. Enable the Drop Connections option, and then click OK.
20. Repeat for each content database associated with this web application.
21. Open Windows Explorer, and then navigate to the folder containing the SQL Server databases.  This will usually be of the form:

    [Drive]:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\DATA

22. Rename the content database(s) and associated transaction log files.
23. In SQL Server Management Studio, in Object Explorer, right-click Database, and then click Restore Database...
24. Select Device, and then click the ellipses "..."
25. Click the Add button.
26. Navigate to the folder where you placed the content database(s) that you want to restore.
27. Select a content database.
28. Click OK.
29. Click OK again.
30. Wait until the restore operation is completed.
31. In Object Explorer, right-click Databases, and then click Refresh.
32. Verify that the newly restored database is listed.
33. Repeat steps 23-32 for each content database.
34. Exit SQL Server Management Studio.
35. Logout of the SQL Server host.
36. Login, as the farm Setup User Administrator account to any SharePoint server in the target farm.
37. Launch an instance of the SharePoint 2013 Management Shell with elevated privileges (right-click).
38. Execute the following script for each content database you need to restore:

    Mount-SPContentDatabase "[content database name]" -DatabaseServer "[database server name or alias]" -WebApplication [URL]

    NOTE: use Mount-SPContentDatabase; not Restore-SPFarm.  You are "restoring" a content database, not a site.

39. After completing these mount operations, the restored web application should become available immediately.

References

• Restore content databases in SharePoint 2013
• Mount-SPContentDatabase
• Restore-SPFarm

Notes

• The terminology here is understandably confusing.  The procedure presented in this posting effectively accomplishes a restore operation.  However, technically speaking, you are really only performing unmount and mount operations on content databases. 
• Also, Attach and Detach operations performed on SQL Server accomplish different things  than performing these operations through Central Administration.  A Detach operation performed through Central Administration detaches the content database from the SharePoint farm; it does not detach it from SQL Server.
• Don't worry if your content databases contain multiple site collections distributed among multiple content databases that these won't be properly integrated back into one web application: just mount them all up and SharePoint Server will figure out the rest.
• While you can also mount the content databases through Central Administration, I have found from experience that after doing it this way, not all my site collections will become visible and I may have to unmount and mount a content database several times this way before things finally work out. Why this is, I don't know.

SharePoint 2013: Start-CacheHost : Cannot start service AppFabricCachingService on computer

Problem

You have two AppFabric hosts in your SharePoint Server 2013 farm and want to add a third.  After adding this host, you find that the new host status is DOWN.  You try starting the service first using the usual Start- CacheHost command, but this fails.  When you try starting it through the Services control panel, it does start up, but then stops after a few minutes.  You then check the CacheClusterConfiguration file and see all three hosts there and configured correctly.  But when you check the health stats (Get-CacheClusterHealth), only the two original hosts are shown.

Solution

• After adding a new cache host, stop and then start the CacheCluster.

References

• Adding a local SharePoint 2013 (development server) as a cache host to AppFabric’s cache cluster
• Configuring Multiple Distributed Cache Servers in SharePoint 2013
• AppFabric cache not clearing on cluster restart
• Stop-CacheCluster
• Restart-CacheCluster
• Start-CacheHost
• AppFabric Caching and SharePoint: Concepts and Examples (Part 1)
• AppFabric Caching and SharePoint: Concepts and Examples (Part 2)
• AppFabric 1.1 Cache - Problem with Start-CacheCluster
• Server Unavailability Troubleshooting (Windows Server AppFabric Caching)

Notes

• Thanks to Marco van Wieren and his post for providing the clue needed for solving this.
• Here's the process I went through to add the instance and troubleshoot:
  
  1. I first added the instance by executing the following script on the target new AppFabric host:

     
     $SPFarm = Get-SPFarm 
     $cacheClusterName = "SPDistributedCacheCluster_" + $SPFarm.Id.ToString()
     $cacheClusterManager = [Microsoft.SharePoint.DistributedCaching.Utilities.SPDistributedCacheClusterInfoManager]::Local
     $cacheClusterInfo = $cacheClusterManager.GetSPDistributedCacheClusterInfo($cacheClusterName)
     $instanceName ="SPDistributedCacheService Name=AppFabricCachingService"
     $serviceInstance = Get-SPServiceInstance | ? {($_.Service.Tostring()) -eq $instanceName -and ($_.Server.Name) -eq $env:computername}
     if([System.String]::IsNullOrEmpty($cacheClusterInfo.CacheHostsInfoCollection)) {Add-SPDistributedCacheServiceInstance; 
     $cacheClusterInfo.CacheHostsInfoCollection}
     

  2. After this, I executed Use-CacheCluster followed by Get-CacheHost to check service status. Note that I did this while logged into the new AppFabric host machine.  The results were that my two existing hosts were UP while the new one was DOWN.  I then executed the following script to spot check the new host's service instance status:

     $instanceName ="SPDistributedCacheService Name=AppFabricCachingService"
     $serviceInstance = Get-SPServiceInstance | ? {($_.service.tostring()) -eq $instanceName -and ($_.server.name) -eq $env:computername}
     $serviceInstance

     The result was that the new service instance status is Online. 
  3. I then decided to check the service instance status of all hosts in the farm using this script:

     Get-SPServiceInstance | ? {($_.service.tostring()) -eq "SPDistributedCacheService Name=AppFabricCachingService"} | select Server, Status

     The results were that all service instances were Online.
  4. I then started the AppFabric service manually, via the Services control panel on the new host.  I noted that the correct service account was configured (spService); and when I clicked Start, the service started up without issue.
  5. Still logged into this machine, I again executed Use-CacheCluster followed by Get-CacheHost to check service application status.  This time, the results were UNKNOWN for all.  I then logged into one of the existing AppFabric hosts and executed Use-CacheCluster followed by Get-CacheHost to check service status.  The results were UNKNOWN for the new host and UP for the two existing hosts.
  6. I waited a minute or two to give time for the caching service to warm up and synchronize.  I then repeated steps 4 - 5, but experienced the same outcome.
  7. I repeated steps 4 - 5, but this time from the SharePoint Management Shell, executing Start-CacheHost.  This immediately resulted in the error:

     Start-CacheHost : Cannot start service AppFabricCachingService on computer.

  8. I then checked the cluster configuration by exporting the configuration file using this script:

     export-cacheclusterconfig C:\CacheClusterConfig.txt

     All three hosts were identified in this file and the configuration information for each seemed to be correct.
  9. Next, I checked the health metrics by executing this script:

     Use-CacheCluster
     Get-CacheClusterhealth

     Oddly, this showed that the two existing hosts were perfectly healthy, but it did not mention anything about the new one I had just added.
  10. I then checked all of the caches using this script:

      Get-Cache

      The results of this were a listing of the names of all of the caches, but this listing only showed the caches for the two existing hosts and nothing for the new one.
  11. Next, I checked the configuration of the new host using this script:

      Get-CacheHostConfig

      The result showed a CacheSize of 1124 MB, which was significantly higher than any of the CacheSizes for the other hosts.
  12. Thinking that this might be the issue, I stopped the cluster, set the new CacheSize and then started the cluster using this script:

      Stop-CacheCluster
      Set-CacheHostConfig -Hostname OCS-VS-BAT13D1 -CachePort 22233 -CacheSize 300
      Start-CacheCluster

  13. I then checked the service status of all hosts, using Get-CacheHost, and this time all instances reported back UP status.
  14. It wasn't clear to me whether it was setting the CacheSize to a lower value or stopping and starting the cache cluster that resolve the issue.  So, I stopped the cluster, set the CacheSize back to 1124 MB, and then started the cluster and checked service statuses again: they were all up.
  15. Therefore, it was stopping and starting the cache cluster that enabled the new host to fully integrate with the cluster.

SharePoint 2013: there was an error during installation

Problem

You are attempting to install SharePoint Server 2013 prerequisites to a Windows Server 2012 virtual machine on Hyper-V.  The server has Internet access.  You run PrerequisiteInstaller as Administrator.  Part way through the installation process, the prerequisite installer stops and displays the usual prerequisite installation error dialog:

Troubleshooting

1. Action: Reviewed prerequisite installer log file.
1. Results: saw the following error in this file:

   Error: The tool was unable to install Application Server Role, Web Server (IIS) Role.

2. Action: Searched for this error text.
1. Results: references 1 and 2 (below) seemed to suggest for my environment that this error involved PrerequisiteInstaller not being able to access the Internet to download appropriate prerequisites. 
3. Action: opened browser and connected to known accessible website.
1. Result: successfully connected to website.
2. Observation: Internet connectivity established, but noted that IE Enhanced Security Settings were enabled.  Perhaps IE Enhanced Security Settings might play a role.
4. Action: disabled IE Enhanced Security Settings, and then re-ran PrerequisiteInstaller.
1. Result: still failed.
2. Observation: perhaps Firewall played a role.
5. Action: Checked Firewall settings for target server and also for production server currently running older instance of SharePoint (2010).
1. Results: firewall was set to default settings on target server and on server currently running older instance. Recalled that a development instance was successfully installed without changing default firewall settings.
2. Observation: firewall settings not likely the cause.  Internet connectivity not likely the cause.
6. Action: searched again for error text.
1. Results: reference 3 seemed to suggest that the issue may be permissions-related; considered tweaking GPO, but wanted to avoid making such modifications.  References 4, 5 and 6 seemed to indicate that for some reason the PrerequisiteInstaller was not able to install the Application Server and Web Server (IIS) roles on Windows Server 2012 and that therefore these should be installed manually.
7. Actions: Added Application Server and Web Server roles per reference 6; then re-ran PrerequisiteInstaller.
1. Results: still failed, but noted that PrerequisiteInstaller seemed to run for longer period before failing.
8. Actions: searched again for error text.
1. Results: reference 7 seemed to indicate that perhaps just need to run aspnet_regiis -enable -i, but for .NET version 4.0.  I recalled that I have had to do this for previous 2010 installations that experienced problems on installation and this resolved it.
9. Action: ran aspnet_regiis -enable -i command, at console, as administrator.
1. Results: the command did not fail, but simply presented help instructions.
10. Action: searched again for error text.
1. Results: found reference 8.  Copy of PrerequisiteInstaller log file was displayed in reference.  This log file included the original error message I identified above (see step 1, above).  Additionally, it highlighted an error message that I had apprently overlooked: Error when enabling ASP.NET v4.0.30319.  This reference also had a link to another reference, 9, which was touted as the solution.
11. Action: Performed the steps in reference 9.  These steps added ASP.NET 3.5. 
1. Results: Installed.
12. Action: restarted PrerequisiteInstaller.exe.
1. Results: completed without issue.

Solution

• Prior to running the PrerequisiteInstaller, install the Application and Web Server roles manually.  During the installation of the Web Server role, be sure to check the ASP.NET 3.5 feature.

References

1. SharePoint 2013: Install Prerequisites Offline or Manually on Windows Server 2012 - A Comprehensive Guide
2. The Products Preparation Tool in SharePoint Server 2013 may not progress past "Configuring Application Server Role, Web Server (IIS) Role
3. SharePoint 2013 Pre requisites install fail, Error: The tool was unable to install Application Server Role, Web Server (IIS) Role
4. SharePoint 2013 pre-requisite: Application and Web Server Role configuration error
5. Installing SharePoint 2013 on Windows Server 2012 R2 *RTM*
6. Installing SharePoint 2013 on Windows Server 2012 R2 Preview
7. The tool was unable to install Application Server Role, Web Server (IIS) Role
8. Trying to install SharePoint 2013 on server 2012 Unable to install the application to web server IIS
9. IIS 8.0 Using ASP.NET 3.5 and ASP.NET 4.5
10. SharePoint 2013 SP1 support in Windows Server 2012 R2
11. Error The tool was unable to install Application Server Role, Web Server IIS Role Last return code 0X41D=1053.

SharePoint 2013: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))

Problem

You attempt to run a PowerShell script against the SharePoint 2013 farm My Site web application and experience the following error message appearing for each website the script iterates through:

Get-SPWeb : Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))
At line:1 char:79
+ Get-SPWebApplication http:/[YourWebAppURL] | Get-SPSite -Limit All | Ge ...
+                                                                               ~~
    + CategoryInfo          : InvalidData: (Microsoft.Share....SPCmdletGetWeb:SPCmdletGetWeb) [Get
   -SPWeb], UnauthorizedAccessException
    + FullyQualifiedErrorId : Microsoft.SharePoint.PowerShell.SPCmdletGetWeb

You verify that you are running the script in the SharePoint Management Shell as Administrator.  You also verify that the user account you are logged in as is a member of the Farm Administrators group and that it is the Primary site collection administrator for the root site collection of the My Site web application.

Solution

• Configure a new User Policy for the My Site web application that grants your administrator account Full Control to all the site collections in this web application:
1. Login to the farm's Central Administration as a farm administrator.
2. Navigate to: Application Management > Web Applications > Manage Web Applications.
3. Select the target web application.  This will enable the options on the Web Applications ribbon.
4. Click the User Policy button.
5. Click Add Users.
6. Click Next.
7. Enter a username, select Full Control, and then click Finish.  When adding the username, be sure to prefix it with i:0#.w|.  So, for example, DOMAIN\John.Smith would be entered as:

   i:0#.w|DOMAIN\john.smith

8. Open a fresh SharePoint Management Shell as Administrator and re-run the script.

References

• Choose administrators and owners for the administration hierarchy in SharePoint 2013

Notes

• An account added to the SharePoint Farm Administrators group is not automatically granted access to a farm web application or to user sites.  A SharePoint Farm Administrator can, however, add himself to a site collection's Site Collection Administrators group and thereby gain administrative access to that site collection and all of its subsites.
• Administrators of site collections and sites must be members of the Site Collection Administrators group or of the site Administrators group.  A member of the Site Collection Administrators group has administrative access to all sites and subsites within the site collection.
• This issue can also affect the results you get when you use PowerShell to harvest My Site metrics.  For example, running this command

  Get-SPWebApplication http:/[MySiteURL] | Get-SPSite -Limit All | Select URL, @{Expression={$_.Usage.Storage/1024/1024}}

  will return a list of "0" for the storage values for all My Sites for which your administrator account is not configured as the Site Collection Administrator.

SharePoint 2010: An exception of type Microsoft.SharePoint.Administration.SPUpdatedConcurrencyException was thrown

Problem

You restored a content database to your 2010 farm, and then attempted to run psconfig to configure the database.  The attempt fails with the following error:

An exception of type Microsoft.SharePoint.Administration.SPUpdatedConcurrencyException was thrown.  Additional exception information: An update conflict has occurred, and you must re-try this action.  The object...

You check the trace log and see the same message.

Solution

• First, verify that the SharePoint 2010 Timer service is started.  If it's stopped, this is likely the issue.  If it's not, try the next series of steps.
• Perform the following steps:
1. Open an elevated command prompt, and then execute stsadm -o setproperty -pn command-line-upgrade-running -pv No, followed by an iisreset
   
2. Next, restart the SharePoint 2010 Timer service.

References

• SPUpdatedConcurrencyException While Installing SharePoint 2010 Service Pack 1

Notes

• Upgrades will not succeed if the timer service is stopped.

SharePoint 2010: Databases running in compatibility range, upgrade recommended

Problem
You find the following entry in the SharePoint 2010 Central Administration Review problems and solutions All Reports listing:

Title	Databases running in compatibility range, upgrade recommended
Severity	2 - Warning
Category	Configuration
Explanation	The following databases have versions that are older than the current SharePoint software, but are within the backwards compatible range: ...
Remedy	To achieve optimal results from these databases, use Upgrade-SPContentDatabase to upgrade Content databases, or psconfig.exe to upgrade other databases.  For more information about this rule, see "http://go.microsoft.com/fwlink/?LinkID=142697".
Failing Servers
Failing Services	SPTimerService (SPTimerV4)
Rule Settings	View

 
This problem can also occur for the BDC database.  Performing a Psconfig upgrade does not resolve it.  However, a simple solution is available.

Solution

• Content Database: In an elevated SharePoint 2010 management shell, execute the PowerShell command presented in the message.
• BDC Database: In an elevated SharePoint 2010 management shell, execute the following command: (Get-SPDatabase | ?{$_.type -eq "Microsoft.SharePoint.BusinessData.SharedService.BdcServiceDatabase"}).Provision(). 

References

• Upgrade-SPContentDatabase
• "Database is in compatibility range and upgrade is recommended" for BDCServiceDatabase after March 2013 Upgrade

Notes

• If this fails, verify that the SharePoint 2010 Timer service has been started.
• The above solutions also work for SharePoint 2013.
• Thanks to Markus Hintner for the solution to this problem when it occurs for the BDC database.