SharePoint 2013: The operation failed because the server could not access the distributed cache

Problem

You navigate to your My Site, click on Newsfeed, and then select Following: nothing appears - this after specifically and successfully selecting several items for following.  You then navigate back to your My Site, click on Newsfeed, and then click on Everyone.  You then see the following error message appear on this page:

There was a problem retrieving the latest activity. Please try again later.

Technical Details

SharePoint returned the following error: The operation failed because the server could not access the distributed cache. Internal type name: Microsoft.Office.Server.Microfeed.MicrofeedException. Internal error code: 55. Contact your system administrator for help in resolving this problem.

You then commence troubleshooting.

Troubleshooting

1. Verify Repeatability: "followed" some additional items; returned to Newsfeed page; and then clicked Everyone: experienced same error. Repeated process again with same result.
2. Research: searched on error message text: found Microsoft TechNet blog posting.
3. Review ULS: triggered issue again, noting time.  Then checked ULS logs on WFEs. Found on first WFE checked.  Found series of 41 entries, from process initiation to closure.  Some examples:

   Unable to write service call usage entry.
    
   MossClientBase_WCFLogging:: MossClientBase.ExecuteOnChannel - Unrecognized exception: System.ServiceModel.FaultException`1[Microsoft.Office.Server.UserProfiles.FeedCacheFault]: Unexpected exception in FeedCacheService.GetPublishedFeed: Object reference not set to an instance of an object.. (Fault Detail is equal to Microsoft.Office.Server.UserProfiles.FeedCacheFault).
   
   Exception occured in ExecuteOnChannel: System.ServiceModel.FaultException`1[Microsoft.Office.Server.UserProfiles.FeedCacheFault]: Unexpected exception in FeedCacheService.GetPublishedFeed: Object reference not set to an instance of an object.. (Fault Detail is equal to Microsoft.Office.Server.UserProfiles.FeedCacheFault).
   
   There were many examples like these. However, the one that seemed more informative was this one (see highlighted):
   
   Microsoft.Office.Server.Social.SPSocialFeedManager.GetFeed: Microsoft.Office.Server.Social.SPSocialException: The operation failed because the server could not access the distributed cache. Internal type name: Microsoft.Office.Server.Microfeed.MicrofeedException. Internal error code: 55. at Microsoft.Office.Server.Social.SPSocialUtil.TryTranslateExceptionAndThrow(Exception exception) at Microsoft.Office.Server.Social.SPSocialUtil.InvokeWithExceptionTranslation[T](ISocialOperationManager target, String name, Func`1 func) at Microsoft.Office.Server.Social.SPSocialFeedManager.<>c__DisplayClass48`1.<s2sinvoke>b__47() at Microsoft.Office.Server.Social.SPSocialUtil.InvokeWithExceptionTranslation[T](ISocialOperationManager target, String name, Func`1 func)
   
   The statement, "...server could not access the distributed cache...," occurred frequently in many other ULS event messages.

4. Review UPA Connection Permissions: farm, application, and admin accounts all provisioned with Full Control.
5. Review IIS Application Pool Identities on WFEs: noted that one application pool used the general farm's service service account (e.g., spService).
6. Review IIS Application Pool Identities on APP: noted that farm's service service account used as application pool identity for six service applications, including the UPA service.
7. Grant account UPA connection full control: provisioned the farm's service service account with UPA Full Control connection permission.
8. Check My Site News Feed: navigated back to My Site Newsfeed, and then clicked the Everyone link: this time all of the documents I had already "followed" appeared listed.

Solution

• Grant the UPA service account Full Control in the UPA Connect Permissions for User Profile Service.

References

• Microsoft TechNet
• SharePoint 2013 Preview Newsfeeds showing nothing

Notes

• Note that the ULS error entries identified by Mark Priem in his blog posting are different than the ones I experienced here.  However, the symptoms were similar, the error messages both involved UPA, and the cause was found ultimately to be the same too.
• Need to update the installation manual to include provisioning the service account with UPA Full Control permission.

OWA 2013: Sorry, something went wrong

Problem

A few users report that when they click on the document hyperlink in a document folder, they experience the Sorry, something went wrong error page, instead of seeing the document displayed online in their browsers.

     

Troubleshooting

1. Verify user experience: you connect to the document hyperlink and verify the user experience - you also see the Sorry, something went wrong error page displayed.
2. Check server event logs on OWA server: you check the server's system and application event logs and do not find any event entry related to the issue.
3. Check server event logs on WFEs: you check event logs of all WFEs and do not find any event entry related to the issue.
4. Log into each WFE and then try connecting to document URL: Successfully connect to document and see it displayed on WFE1 but not on WFE2 (there are two WFE's in the system).
5. Check ULS on WFE1:  using correlation ID from error page, find series of entries having same correlation ID; one of these entries presents this message:

   System.ServiceModel.EndpointNotFoundException: There was no endpoint listening at net.pipe://localhost/SecurityTokenServiceApplication/appsts.svc that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details. ---> System.IO.PipeException: The pipe endpoint 'net.pipe://localhost/SecurityTokenServiceApplication/appsts.svc' could not be found on your local machine.

6. Check WFE1 server services: net.pipe service is not running.
7. Start net.pip service on WFE1: service started.
8. Connect to document URL from workstation and WFE1: document opens in browser.

Solution

• On each WFE, verify that the net.pipe service is running.  If it isn't, start it.

References

• AlsTechTips
• SharePoint 2013: the SharePoint item being crawled returned an error
• SharePoint 2013: There was no endpoint listening at net.pipe...

Notes

• This issue appeared after performing monthly patching and restarting servers.

SharePoint 2016: Drives are at risk of running out of free space

Problem

Performing routine check on system health, you discover the following warning in the farm's Health Reports:

Title	Drives are at risk of running out of free space.
Severity	2 - Warning
Category	Availability
Explanation	Available drive space is less than five times the value of physical memory. This is dangerous because it does not provide enough room for a full memory dump with continued operation. It also could cause problems with the Virtual Memory swap file: [failing drives]
Remedy	Examine the failing servers and delete old logs or free space on the drives. For more information about this rule, see http://go.microsoft.com/fwlink/?LinkID=142688
Failing Servers	[a farm WFE]
Failing Services	SPTimerService (SPTimerV4)
Rule Settings	View

You check system drive and find that available space is 5 x greater than configured RAM.  rrr

Solution

• This rule checks for drive space exceeding 5 x configured system memory.  It executes as all or nothing, meaning that all server drives must meet the 5 x threshold and if any one is deficient then the warning is triggered.  You can of course disable the rule.  However, another approach available to you if your farm servers are virtualized is to go ahead and configure the failing drive or drives as managed with increased disk space: while SharePoint sees a larger drive, the hypervisor only allocates necessary drive space.  Thus there is no actual increase in resource usage.

References

• TechNet SharePoint 2016
• Drives are at risk of running out of free space (SharePoint Server 2016)
• TechNet SharePoint Forums
• Drives are at risk of running out of free space.
• Disks running out of space according to SharePoint 2010