Monday, March 30, 2015

SharePoint 2013: how to install named SQL Server instance for new farm

Introduction

This posting captures the steps associated with installing a named instance of SQL Server 2013 Standard as part of building a new SharePoint Server 2013 Enterprise farm.  Installation is performed on a virtual machines hosted on Hyper-V 2012.  Prior to initiating this procedure, you must identify the SQL Server Service account that you will use to run the SQL server database engine and agent services.  For this posting, the following stipulations are assumed:
  • SQL Server Service account: DOMAIN\spSql
  • Server Drives: C:System, D:Data, E:Logs, and F:Backup
  • Database aliases have been created on all farm servers intended to host SharePoint
Budget at least an hour to perform the installation and resolve any issues that may present during the installation.  This posting also presents steps for testing the new SQL Server remote connectivity from other farm servers.

Procedure
  1. Attach SQL Server 2012 Standard ISO to the VM.
  2. Log into the the target VM using an administrator account.
  3. Navigate to the installation DVD, right-click on Setup.exe, and then choose Run as administrator.
  4. On the left, click Installation
  5. Click New SQL Server stand-alone installation or add features to an existing installation.
  6. Accept defaults until the Setup Role page appears: select SQL Server Feature Installation:
  7. Click Next.
  8. On the Feature Selection page, check these features:
    • Database Engine Services
    • Management Tools - Basic
    • Management Tools - Complete
  9. Click Next.
  10. On the Instance Configuration page, select Named instance, and then enter the name of the new instance.  After enter the name, and moving to a new field (eg, hitting the TAB key), the Instance ID: field will automatically be updated to match what you entered.
  11. Click Next.
  12. On the Disk Space Requirements page, click Next again.
  13. On the Server Configuration page:
    1. For the SQL Server Agent and Database Engine services, enter the SQL Server Service account for both.
    2. Set the Startup Type to Automatic for all services (include the Browser service).
  14. Click Next.
  15. On the Database Engine Configuration page:
    1. Set Authentication Mode to Windows authentication mode.
    2. Click the Add Current User button to add your administrator account.
    3. Select the Data Directories tab, and then configure drive locations as indicated in the Introduction to this posting.
  16. Click Next.  
  17. On the Installation Configuration Rules page, click Next.  At this point, you may experience a common error:
    If so, exit this installation; follow this procedure to resolve; then restart the installation.
  18. On the Ready to Install page, click Install.
  19. On the Complete page, click Close.
Testing
  1. Log into a farm server intended to host SharePoint.
  2. Go: Control Panel > Administrative Tools > ODBC Data Sources (32-bit)
  3. On the User DSN tab, click the Add button.
  4. Scroll down to the bottom, select SQL Server, and then click Finish.
  5. Enter any name, and then click the down arrow.
  6. Enter or select the server name
  7. Click Next.
  8. Accept all defaults and click Next.  At this point, if you experience the following error:
    check to make sure that the account you are currently logged in as has been added to the logins for the SQL Server instance and that the account is not locked or has an expired password.
  9. Advancement to the next page is all that's necessary to verify successful remote access:
References

Thursday, March 26, 2015

SQL Server 2012: How to configure it for access within the SharePoint 2013 Farm

Introduction

SQL Server 2012 must be properly configured to accept remote connections from SharePoint 2013 servers in the SharePoint farm.  This includes:
  • Enabling remote connections, 
  • Enabling the TCP/IP protocol, 
  • Setting the desired port, 
  • Ensuring the browser service is started and
  • Configuring firewall exceptions
Steps for configuring each of these are presented in this posting.  This posting assumes static ports and that standard ports 1433 and 1434 will be used for communication.  Once you have completed all configuration steps, steps are provided for quickly and conveniently testing access to the database server from another server (such as a prospective farm server).  This posting is an updated version of an earlier posting, modified to to focus on SQL Server 2012 Standard and to account for domain GPO control over Windows Firewall settings.

Enabling Remote Connections
  1. Launch SQL Server Management Studio (SSMS) as administrator.
  2. In Object Explorer, right-click the server name and select Properties.
  3. In Select a page, select Connections.
  4. Ensure that Allow remote connections to this server is enabled:
Enable TCP/IP Protocol
  1. Launch SQL Server Configuration Manager
  2. At left, expand SQL Server Network Configuration.
  3. Select Protocols for [NameOfYourSQLServerInstance].
  4. Ensure that the Status of the TCP/IP protocol is Enabled.
Start SQL Server Browser Service
  1. Launch SQL Server Configuration Manager.
  2. At left, select SQL Server Services.
  3. Ensure that the SQL Server Browser service state is Started and that its Start Mode is set to Automatic:
Set Desired Port
  1. Launch SQL Server Configuration Manager.
  2. At left, expand SQL Server Network Configuration.
  3. Select Protocols for [NameOfYourSQLServerInstance].
  4. At right, right-click on TCP/IP, and then select Properties.
  5. Scroll to the bottom of the list.
  6. Ensure that TCP Dynamic Ports is empty and that TCP Port is set to 1433.
  7. Restart the database engine.
Configure Firewall Exceptions
NOTE: these steps assume that Windows Firewall is controlled via GPO and thus firewall settings cannot be set directly through Windows Firewall but must be made via modification of the Local Group Policy.
  1. Go to Control Panel > Administrative Tools.
  2. Launch Local Security Policy.
  3. Expand: Security Settings > Windows Firewall with Advanced Security - Local Group Policy Object > Inbound Rules.
  4. Configure Inbound Rule for TCP 1433
    1. At right, right-click and then select New Rule...
    2. Select Port, and then click Next.
    3. Select TCP.
    4. Select Specific local ports, enter 1433, and then click Next.
    5. Select Allow the connection, and then click Next.
    6. Enable Domain, Private and Public and then click Next.
    7. Enter a unique name, and then click Finish.
  5. Configure Inbound Rule for UDP 1434
    1. Repeat steps 4.1 through 4.7, above, this time selecting UDP and entering 1434.
  6. Configure Program Exception for sqlbrowser.exe
    1. At right, right-click and then select New Rule...
    2. Select Program, and then click Next.
    3. Select This program path, and then click the Browse button.
    4. Navigate to: C:\Program Files (x86)\Microsoft SQL Server\90\Shared.
    5. Select sqlbrowser.exe, and then click Open
      .
    6. Click Next.
    7. Select Allow the connection, and then click Next.
    8. Enable DomainPrivate and Public and then click Next.
    9. Enter a unique name, and then click Finish.
  7. Configure Program Exception for sqlservr.exe
    1. At right, right-click and then select New Rule...
    2. Select Program, and then click Next.
    3. Select This program path, and then click the Browse button.
    4. Navigate to: C:\Program Files\Microsoft SQL Server\MSSQL11.CSSP\MSSQL\Binn.
    5. Select sqlservr.exe, and then click Open.
    6. Click Next.
    7. Select Allow the connection, and then click Next.
    8. Enable DomainPrivate and Public and then click Next.
    9. Enter a unique name, and then click Finish.
  8. Restart the machine hosting SQL Server.
Test Access
  1. Logout of the SQL Server
  2. Log into a server intended to be used for hosting SharePoint.
  3. Go to Control Panel > Administrative Tools.
  4. Launch ODBC Data Sources (32-bit).
  5. Select the User DSN tab.
  6. Click the Add button.
  7. Scroll down, select SQL Server, and then click Finish.
  8. Enter a name, enter the name of the database server machine (or the alias if you have configured that), and then click Next.
  9. Select With Windows NT Authentication using the network login ID, enable Connect to SQL Server to obtain..., and then click Next.
    1. If connectivity was established, the next page of the wizard is presented.
    2. Otherwise, and error message will be displayed:
      .
  10. If testing was successful, you are now ready to proceed with installing SharePoint Server 2013. Otherwise, review the configuration steps presented above and verify that you have executed each of them correctly.  If this still fails, disable the firewall on the SQL Server machine and try testing connectivity again.  If disabling the firewall enables access, the issue involves firewall configuration.  If so, check with your sys admins as to how domain GPO may affect firewall configuration and even override local changes.
References
  • Thanks to Andrzej Gołębiowski at Cx3 for the original posting that presents this topic so well.

SQL Server 2012: Error while enabling Windows feature : NetFx3, Error Code : -2146498298

Problem

You are installing a new instance of SQL Server 2012 onto Windows Server 2012.  During the installation, you see the following error:

Solution

  • Install the .NET Framework 3.5 feature.  No need to install HTTP or non-HTTP activation along with this.
References
  • You'll need the Windows Server 2012 disk or ISO.
  • Routine installation will fail.  You'll need to Specify An Alternate Source Path to enable the installation to succeed.  You'll see this setting during the feature installation.  It's near the bottom of the panel.  Alternatively, you can also use a DOS shell command (see reference).
  • After doing this and re-running the installation, you may see this error appearing:
    An error occurred creating the configuration section handler for userSettings/Microsoft.SqlServer.Configuration.LandingPage.Properties.Settings...
    If you do, no worries.  First, close the installation panel.  Then, navigate to:
    C:\Users\[YourUsername]\AppData\Local\ Microsoft_Corporation
    and delete the folder there that starts with LandingPage.exe_.  Then re-run the installation.

Wednesday, March 25, 2015

Office Web Apps 2013: The HTTP service located at... is unavailable.

Problem

You install updates to the (single) Office Web Apps Server 2013.  You then log into and execute Get-OfficeWebAppsMachine on this server and see the following error:

Get-OfficeWebAppsMachine : The HTTP service located at 
http://[yourOWAserver]/farmstatemanager/FarmStateManager.svc
is unavailable.  This could be because the service is too busy or because 
no endpoint was found listening at the specified address. Please ensure 
that the address is correct and try accessing the service again later.
At line:1 char:1
+ Get-OfficeWebAppsMachine
+ ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Get-OfficeWebAppsMachine], 
ServerTooBusyException
    + FullyQualifiedErrorId : System.ServiceModel.ServerTooBusyException,
Microsoft.Office.Web.Apps.Administration.GetM   achineCommand

You check Services and find that the Office Web Apps and World Wide Web Publishing services are in the Running state. You then check IIS and find that all application pools and sites have Started status.

Solution

  • In IIS, stop and start the HTTP80 site.
Notes
  • Don't yet understand why, but this worked.

Monday, March 23, 2015

SharePoint 2013 Tip: How to Export or Import a Site Using PowerShell

Introduction

This posting reviews how to export an entire site using PowerShell.  In either case, use an elevated SharePoint Management Shell and DO NOT append a "/" to the URL.

How to Export a Site

Export-SPWeb -Identity "http://[URLofSiteToBeExported]" -Path "ToLocationToSaveExportTo"

How to Import a Site

Import-SPWeb -Identity "http://[URLofSiteToBeOverWritten]" -Path "ToLocationOfExportFile"

References
Notes
  • The site you are importing to must be of the same site template as the site being imported.
  • Importing a site effectively overwrites the site (of the same name) that is already there.

Saturday, March 21, 2015

SharePoint 2013: Failed to load this workflow

Problem

The following are some common problems encountered when migrating workflows from one 2013 farm to another and their solutions.

Failed to load this workflow

You see the following error appear after clicking on a 2013 workflow in SharePoint Designer 2013:
If you simply click OK, and then attempt to open the workflow again in edit mode, by clicking on it in the Workflows listing, you may see the following error:
If instead of clicking on it in the Workflows listing, you only select it, and then click the Edit Workflow button on the ribbon, you may see this error:

Solution Option 1

The first and easiest solution to this problem is to restart SharePoint Designer, as directed in the error prompt.  From my experience, this works in most cases.

Solution Option 2

If this doesn't work, the next thing to try is: 1) closing Designer, 2) clearing your application caches:

%APPDATA%\Microsoft\Web Server Extensions\Cache
%APPDATA%\Microsoft\SharePoint Designer\ProxyAssemblyCache
%USERPROFILE%\AppData\Local\Microsoft\WebsiteCache

and then restarting designer.  From my experience, this, works in some cases.

Solution Option 3

A reader of this blog (see comment below) noted another possibility: disconnected stages.  If your 2013 workflow does not connect all the stages, it may trigger an error.  This issue is discussed in this Microsoft Hotfix: Description of the SharePoint Designer 2013 hotfix package (Spd-x-none.msp): April 8, 2014.  Thanks to the reader who brought this to attention.

Solution Option 4

From my experience, if these solutions do not resolve the problem, the cause of the problem may not be infrastructure related at all but instead may involve workflow design.

For example, one 2013 workflow designer had mistakenly implemented a type mismatch in her design (a variable of one type was being pushed into a variable of another type).  This resulted in various error events logged to the Application log of the server on which Workflow Manager 1.0 service was running.  Over a month of work was spent attempting to trace what was thought to be an infrastructure issue; a Microsoft Support request was also initiated on this.  Microsoft Support enabled us to eventually trace the issue to the workflow itself and verify it had nothing to do with infrastructure.

References

Friday, March 20, 2015

SharePoint 2013: How to set the hyperlink list column values from Workflow

Tip

Workflows can be used to set the values of both Picture and Hyperlink list column-types. The Picture column-type has just a single value, URL, that you can set.  The Hyperlink column-type, however, has two values: URL and Description.  So, if you create a workflow tied to a list having this column type, and then you try to set both these values, well..., you don't see either of these fields show up in Designer!  You just see the name of the column.  So what do you do?  How do you update both values with a single workflow Set statement?

Simple: if you want to set both the URL and the Description values of this column-type, you must enter them both into the value area, but separated by a comma ",".  That's it.  It really is that simple.  Let's examine this more closely.

Review

Here's a table that has been configured with Choice and Hyperlink column types.
The Choice column has been configured with three values: Nature, Science and Space.
And here's a workflow created on this list.  The workflow is triggered on change of the MyChoice column. Setting the column to a new value triggers the workflow to set the MyLink column, as determined by the If-branches:
If you look closely, you can see that the MyLink column is being set to two values separated by the comma:
The first value must be the URL; the second, the Description.  Let's create a new item, and then change it to see the workflow in action.
Opening up this list item in edit mode, we can see that both the URL and Description properties were in fact set:


References
  • Many thanks to Jonathan Adams with Serious SharePoint for posting the simple solution to setting hyperlink column types from workflow.  His posting was for SharePoint 2010, but is still applicable to SharePoint 2013.
  • Note that the column that shows the status of a specific workflow does not become available until you actually publish the workflow.  Thus, if you want to add the workflow status column to a list view, you have to wait until you publish the workflow before you can do this.

Friday, March 13, 2015

SharePoint 2013: Configuration Failed, SPUpdatedConcurrencyException was thrown

Problem

You installed the SharePoint Server 2013 March 2015 CU. After rebooting, you launch the SharePoint Products Configuration Wizard. The wizard completes steps 9 through 10, but then presents a failure message on the last step:
Reviewing the upgrade log file, you see this error:
An exception of type Microsoft.SharePoint.PostSetupConfiguration.PostSetupConfigurationTaskException was thrown.
Additional exception information: Failed to upgrade SharePoint Products.
You then open an elevated command shell using the SharePoint Setup User Administrator account and execute: PSConfig.exe -cmd upgrade -inplace b2b -force -wait. You see the usual messages display and eventually the message displays 100%. However, it appears to remain stuck on displaying 100% event after half an hour. Looking in the upgrade log again, you find this message:
An exception of type Microsoft.SharePoint.Administration.SPUpdatedConcurrencyException was thrown. Additional exception information: An update conflict has occurred, and you must re-try this action. The object SPUpgradeSession Name=Upgrade-20150312-141528-847 was updated by [farm service account], in the PSCONFIG (7904) process, on machine [farm server]. View the tracing log for more information about the conflict.
The issue involves a concurrency issue, just like the PowerShell error message indicates. What this means is that multiple upgrade processes were attempting to engage the same configuration table concurrently and a race condition occurred. There are several different solutions here that can be implemented. These are presented in the next section.

Solutions

These solution paths are listed in order of increasing complexity and in the suggested order in which to engage them.  Note: if you experience an apparent "Stuck on 10%, 20%... or even 100%" issue, see the Troubleshooting section below.  That's a different issue then the solutions listed below will resolve.
  1. Re-execute PSCONFIG, but ensuring that you do it on only one server at a time, each time waiting for it to complete before executing it on the next one.
  2. Re-executing the PSCONFIG command-line tool:
    Psconfig.exe -cmd upgrade -inplace b2b -wait -force
    Try it two or more times. If the same error occurs each time, move on to the next solution option.
  3. If executing the command-line tool in the SharePoint Management Shell resulted in the error, try executing it in an elevated DOS command shell.
  4. Re-execute the PSCONFIG command-line tool without the -force parameter, like so;
    Psconfig.exe -cmd upgrade -inplace b2b -wait
  5. Re-execute the PSCONFIG command-line tool and include the ApplicationContent and InstallFeatures commands, like so:
    PSConfig.exe -cmd upgrade -inplace b2b -force -cmd applicationcontent -install -cmd installfeatures
  6. Clear the SharePoint Configuration Cache, and then re-execute the command-line tool (either in and elevated SharePoint Management Shell or DOS command shell).
  7. Re-execute PSCONFIG but specifying the affinity (see reference 15 for details).
How to Clear Upgrade Progress Flag and Configuration Cache
  1. Clear the upgrade in progress flag
    1. Open an elevated command shell.
    2. Navigate to: c:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\BIN\.
    3. Execute the following: stsadm -o getproperty -propertyname command-line-upgrade-running. If this returns <Property Exist="Yes" Value="no" />, go directly to step 2). Otherwise, continue to the next step.
    4. Execute the following: stsadm –o setproperty –propertyname command-line-upgrade-running –propertyvalue no.
  2. Clear the configuration cache
    1. Open the Services.msc control panel, and then look for the SharePoint Timer Service.
    2. Stop this service.
    3. Open Windows Explorer, and then navigate to the cache folder at: C:\ProgramData\Microsoft\SharePoint\Config.
    4. Look for the most recent cache folder, and then open it.
    5. Delete all files in this folder EXCEPT cache.ini.
    6. Open the cache.ini file in a text editor, and then randomly modify the number, say, to "1".
    7. Save the cache.ini file.
    8. Start the SharePoint Timer Service.
    9. Wait for the configuration cache to fill up again. About 1700 items. Once it stabilizes, proceed to the next step
  3. Perform a new upgrade
    1. Open an elevated command prompt as Administrator.
    2. Run the following command: Psconfig.exe -cmd upgrade -inplace b2b -wait -force. NOTE: The following services will be stopped after executing this command:
      1. SharePoint Timer Service
      2. SharePoint Server Search 15
      3. SharePoint Search Host Controller
      4. SharePoint Administration
      5. World Wide Web Publishing Service
      They will be automatically restarted again at the end of the upgrade process.

Troubleshooting

    1. Stuck on 100%. You may experience an apparent "stuck on 100%" issue, even after performing the steps above. Wait before repeating the process or engaging in more troubleshooting. I have experienced delays of up to 90 minutes, between seeing "100.00%" and seeing the message that the upgrade was complete. To check if this is the case, open the Upgrade log file, and then search for the phrase "End processing children of [SPFarm Name=[farm configuration database name] for upgrade" followed by "Exiting upgrade for [SPFarm Name=[farm configuration database name]]. If these are the last two statements currently in the Upgrade file, it is likely that the upgrade is progressing fine and simply requires more time to complete. This has been my experience to date. For example, once, while upgrading two WFEs and one APP, I observed that in each case, the upgrade seemed to get stuck just after presenting 100%. But when I looked into the Upgrade file in each server, I found the above statements at the end of the file; and in every case, the process eventually completed successfully.
    2. Stuck on 10%.  You may also experience an apparent "stuck on 10%" issue.  I have experienced this message lasting nearly two hours, and at the conclusion the message still states "10%" but no followed by "Successfully upgraded SharePoint Products.  The screen shot below shows the proof.  
      From experience, my suggestion is that if you see it apparently stuck on 10%, 20% or even 100%, don't panic: just let it run its course. And yes this may take two hours or more.  Just let it run.
    3. Feature (Id = [...]) is referenced in database [...], but isn't installed on the current farm. If you experience this error after executing PSCONFIG in an elevated SharePoint Management Shell, open an elevated command shell and execute this command:
      PSConfig.exe -cmd upgrade -inplace b2b -force -cmd applicationcontent -install -cmd installfeatures

References

Notes

  • One variation of the above is where the upgrade remained stuck on 10% or some other percentage. For example, I experienced the issue noted above after running the SharePoint Productions Configuration wizard and seeing it complete successfully, but then seeing the message Product / patch installation or server upgrade required appear in the Central Administration Problems report. I then executed the psconfig command, but this remained stuck at 10%. To resolve this, I performed the solution steps identified above.
  • UPDATE (5/13/15): Another variation: after restarting a server (after installing Windows updates), opened elevated shell and ran psconfig. During step 3 of 4, the SPUpdatedConcurrencyException error was presented; and this occurred without even reaching the percent "%" phase of this stage. While beginning to implement the steps above, I noticed that a number of services were stopped, including SharePoint Administration and Timer. I started all stopped SharePoint services, then opened an elevated command shell, and ran psconfig. This time, the process ran through 100%.
  • UPDATE (09/13/16): I needed to execute the command-line tool in order to resolve a database upgrade warning (I mounted content databases from one farm to another, and the one farm was at a patch level one month behind).  I ran into the SPUpdatedConcurrencyException error again while executing the PSCONFIG command-line tool.  Re-executing the command-line tool didn't solve it, and neither did switching shells (PS to DOS). Adding the ApplicationContent and InstallFeatures commands did the trick.  Even still, it seemed to get stuck on 10%.  After waiting half an hour, I locked the server and went home.  The next day, it showed completed successfully, but... still displaying 10%.  Note that in this case, all that was being updated were several content databases - the SP binaries of the target farm had already been updated and upgraded.
    NOTE: be sure to check SharePoint server services after running PSCONFIG.  I've experienced PSCONFIG completing successfully, but leaving critical services stopped, such as the World Wide Web Publishing Service, SharePoint Search Host Controller, SharePoint Server Search 15 and SharePoint Timer Services.  Also, if you executed it first on the app server, it automatically tries to run on the other SharePoint servers to, and when it fails, it fails on all of them.  Thus, if you finally get it to run successfully on the app server, check the other SP servers to see whether it was run again on these.  if it wasn't, you'll need to run PSCONFIG on these as well.

Monday, March 9, 2015

SharePoint 2013: How to rebuild the User Profile Service Application

Introduction

This posting consolidates notes with regard to rebuilding the User Profile Service Application. A rebuild is sometimes indicated when experiencing failures in the service application, failures in profile synchronization, or other UPA issues.  This procedure focuses on UPA installation and assumes that My Site remains available and is not removed.  You will need to keep at hand the service account (and pwd) used specifically to read user profiles from Active Directory.  The connection type used in this rebuild is SharePoint Profile Synchronization.

Procedure
  1. Prepare for rebuild
    1. Review UPA concepts
    2. Identify the SharePoint Setup User Administrator account and its password
    3. Identify the profile synchronization account and its password.
    4. Verify that it has been granted Replicate Directory Changes permission (needed when using SharePoint Profile Synchronization).
    5. Verify that the NETWORK SERVICE account has read permission to the SharePoint root folder ("15.0" for 2013).
    6. Verify that there are no active synchronization connections.
      • CA > Application Management > Manage service applications > User Profile Service > Profile Synchronization Status
      • CA > Application Management > Manage service applications > User Profile Service > Synchronization > configure Synchronization Connections
  2. Remove the User Profile Service Application
  3. Stop any in-progress User Profile synchronizations
  4. Restart the SharePoint Timer Service
  5. Install a new UPA instance
  6. Start the UPA instance
  7. Configure a new synchronization connection
  8. Launch a new synchronization
Additional References
  • Whatever account you use to perform this procedure, it must be the same account as that used to originally install SharePoint.  Microsoft indicates that this should be the SharePoint Setup User Administrator account
  • User Profile Synchronization Service stuck on starting: I found that even after removing the User Profile service application that the User Profile Synchronization Service remained stuck on Starting.  Even restarting the server did not change this. After further exploration, I was able to change it into the Stopped state by performing the following steps:
    1. Get-SPServiceInstance: this returned a listing of service instances on all SharePoint servers.  You can clean up the output by using: Get-SPServiceInstance | Where-Object {$_.Server -Like "*yourserver*"} | Select-Object TypeName, ID | Sort-Object TypeName
    2. Searched through this listing for User Profile Synchronization Service, and then noted its Id (not the UserProfileApplicationGuid).
    3. Stop-SPServiceInstance -Identity <theID>: checking in CA > Manage Services on Server, this changed the status to Stopping.
    4. Restarted SharePoint Timer Service: checked Manage Services on Server again, and after a few seconds, the status changed to Stopped.

Saturday, March 7, 2015

SharePoint 2013: Detection of product '{90150000-104C-0000-1000-0000000FF1CE}', feature 'PeopleILM', component '{1C12B6E6-898C-4D58-9774-AAAFBDFE273C}' failed.

Problem

You see the following warning appear in the Application event log of the farm server on which the User Profile Application is running:
Log Name:      Application
Source:        MsiInstaller
Date:          date/time]
Event ID:      1004
Task Category: None
Level:         Warning
Keywords:      Classic
User:          NETWORK SERVICE
Computer:      [farm server]
Description:
Detection of product '{90150000-104C-0000-1000-0000000FF1CE}', feature 
'PeopleILM', component '{1C12B6E6-898C-4D58-9774-AAAFBDFE273C}' failed.  
The resource 'C:\Program Files\Microsoft Office
Servers\15.0\Service\Microsoft.ResourceManagement.Service.exe' does 
not exist.
Event Xml:
...
This warning is accompanied by two other warnings, all of which occur simultaneously:
Log Name:      Application
Source:        MsiInstaller
Date:          [date/time]
Event ID:      1001
Task Category: None
Level:         Warning
Keywords:      Classic
User:          NETWORK SERVICE
Computer:      [farm server]
Description:
Detection of product '{90150000-104C-0000-1000-0000000FF1CE}', feature 
'PeopleILM' failed during request for component 
'{1681AE41-ADA8-4B70-BC11-98A5A4EDD046}'
Event Xml:
...
and
Log Name:      Application
Source:        MsiInstaller
Date:          [date/time]
Event ID:      1015
Task Category: None
Level:         Warning
Keywords:      Classic
User:          NETWORK SERVICE
Computer:      [farm server]
Description:
Failed to connect to server. Error: 0x80070005
Event Xml:
...
These warnings appear whenever an incremental or full profile synchronization runs.  You are able to reproduce these warnings by manually initiating a profile synchronization.

Solution of this problem is trival and involves adding the NETWORK SERVICE account to the local WSS_WPG group.  This group is automatically granted Read permission to C:\Program Files\Microsoft Office Servers\15.0, which contains the Service folder identified in the 1004 event message.

Solution
  1. Log into the farm server hosting the User Profile Application.
  2. Launch the Computer Management applet.
  3. In Computer Management, go Computer Management > System Tools > Local Users and Groups > Groups.
  4. Scroll down to the bottom and then double-click the WSS_WPG group.
  5. Add the NETWORK SERVICE account.
  6. Test by manually launching a new profile synchronization and then again checking the the server's Application event log.
References
  • I discovered the solution to this problem while troubleshooting a previous related issue that had caused the entire UPA to fail on our development farm.  This time, when I saw similar warnings appear again, this time on the production farm, I suspected that a similar solution would be viable, and it was.
  • UPDATE (3/10/15): checked again after a few days: issue not appearing in event logs.

SharePoint: PowerShell script for automating daily routine server checks

Introduction

On a daily basis, I perform a set of checks on customer SharePoint farm servers using a checklist. Items in this checklist include:

  • Farm problems and solutions report
  • Server event logs
  • Status of key farm server services
  • Status of IIS application pools and sites
  • Search health reports
  • Status of AppFabric
  • Status of 2013 Workflow services
To speed up these routine checks, I've partially automated them in a PowerShell script, which reports on key server services, IIS sites and IIS application pools.  It is run from any elevated PowerShell on any farm server and presents its results in the shell. Necessary modules, snap-ins and features are automatically verified and installed if necessary to execute the script commands.  The script has been published to TechNet and is a framework in progress for further development.
Script
Notes
  • None yet.

Monday, March 2, 2015

SharePoint 2013: UserProfileImportJob Generic failure

Problem

You are unable to get the User Profile Service application to run successfully.  You rebuild the service application several times in an attempt to resolve the issue.  The service application can be completely rebuilt without issues, and you can create a new synchronization connection without issue.  However, the moment that you start a new synchronization, the service fails, no user profiles are imported and you see the following set of message appear every minute (per the timer job schedule):
Log Name:      Application
Source:        Microsoft-SharePoint Products-SharePoint Foundation
Date:          [date/time]
Event ID:      6398
Task Category: Timer
Level:         Critical
Keywords:      
User:          [farm service account]
Computer:      [farm app server]
Description:
The Execute method of job definition 
Microsoft.Office.Server.UserProfiles.UserProfileImportJob 
(ID 3c97e43b-0b13-405a-9bf3-3ff0277ad969) threw an exception. 
More information is included below.

Generic failure 
Event Xml:
...

Log Name:      Application
Source:        MsiInstaller
Date:          [date/time]
Event ID:      1004
Task Category: None
Level:         Warning
Keywords:      Classic
User:          NETWORK SERVICE
Computer:      [farm app server]
Description:
Detection of product '{90150000-104C-0000-1000-0000000FF1CE}', 
feature 'PeopleILM', component '{1AE472A9-E94A-41DC-9E98-F89A2821658F}' 
failed.  The resource 
'D:\Program Files\Microsoft Office Servers\15.0\Tools\makecert.exe' 
does not exist.
Event Xml:
...

Log Name:      Application
Source:        MsiInstaller
Date:          [date/time]
Event ID:      1001
Task Category: None
Level:         Warning
Keywords:      Classic
User:          NETWORK SERVICE
Computer:      [farm app server]
Description:
Detection of product '{90150000-104C-0000-1000-0000000FF1CE}', 
feature 'PeopleILM' failed during request for component 
'{1681AE41-ADA8-4B70-BC11-98A5A4EDD046}'
Event Xml:
...
You also see numerous instances of error event 6313 messages appearing in the application log:
Log Name:      Application
Source:        FIMSynchronizationService
Date:          [date/time]
Event ID:      6313
Task Category: Server
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      [farm app server]
Description:
The server encountered an unexpected error creating 
performance counters for management agent "ILMMA".
 Performance counters will not be available for this management agent.
Event Xml:
...

Log Name:      Application
Source:        FIMSynchronizationService
Date:          [date/time]
Event ID:      6313
Task Category: Server
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      [farm app server]
Description:
The server encountered an unexpected error creating 
performance counters for management agent "MOSS-UserProfile".
 Performance counters will not be available for this management agent.
Event Xml:
...

Log Name:      Application
Source:        FIMSynchronizationService
Date:          [date/time]
Event ID:      6313
Task Category: Server
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      [farm app server]
Description:
The server encountered an unexpected error creating 
performance counters for management agent "MOSSAD-OCS Sync Connection 1".
 Performance counters will not be available for this management agent.
Event Xml:
...
The problem was found to involve the ability of the NETWORK SERVICE account to access certain User Profile Service application resources.

Solution
  1. Log into the farm application server on which the User Profile Service application is running (and on which the FIM services will be running too).
  2. Open the server's Application log, and look for 1001 and 1004 event messages accompanied by a 6398 timer event message.
  3. Note down the folder identified in the 1004 Warning message.
  4. Navigate to this folder.
  5. Grant the NETWORK SERVICE account Read & Execute privilege to the folder.
  6. Monitor the server's Application log for new appearances of the 1001, 1004 and 6398 set of messages.
  7. If no new sets occur, the issue has been resolved.  Otherwise, repeat steps (3) through (6) until they no longer occur.
  8. After all instances have been resolved, FIM synchronization will proceed normally.

References
Notes
  • Thanks to Fred El for his posting that helped point to the solution of the problem.
  • The trick in resolving this issue was to not only look at Critical and Error severity event messages, but also Warning severity ones, which provided the necessary direction and information for actually resolving the problem.
  • The User Profile Service application was running fine previously.  The cause for the change - the permissions for the resource folders - is still being investigated.
  • This problem was first detected when the following error appeared in the app server' Application log right after the usual  6 AM morning timer recycle:
    Log Name:      Application
    Source:        Microsoft-SharePoint Products-SharePoint Foundation
    Date:          [date/time]
    Event ID:      6398
    Task Category: Timer
    Level:         Critical
    Keywords:      
    User:          [farm service account]
    Computer:      [farm app server]
    Description:
    The Execute method of job definition 
    Microsoft.SharePoint.Administration.SPTimerRecycleJobDefinition 
    (ID 966ec95d-d4de-4b54-9cdf-63cb9d2e3623) threw an exception. 
    More information is included below.
    
    The timer service was not recycled because the following jobs 
    were still running: User Profile Service - User Profile 
    Incremental Synchronization
    
    The synchronization task was found to be stuck. Checking the server Services applet, the FIM synchronization service was found to repeatedly stop and start. The Application log also showed these three error events occurring repeatedly:
    Log Name:      Application
    Source:        FIMSynchronizationService
    Date:          [date/time]
    Event ID:      6313
    Task Category: Server
    Level:         Error
    Keywords:      Classic
    User:          N/A
    Computer:      [farm app server]
    Description:
    The server encountered an unexpected error creating performance 
    counters for management agent "ILMMA".
     Performance counters will not be available for this management agent.
    
    Log Name:      Application
    Source:        FIMSynchronizationService
    Date:          [date/time]
    Event ID:      6313
    Task Category: Server
    Level:         Error
    Keywords:      Classic
    User:          N/A
    Computer:      [farm app server]
    Description:
    The server encountered an unexpected error creating performance 
    counters for management agent "MOSS-UserProfile".
     Performance counters will not be available for this management agent.
    
    Log Name:      Application
    Source:        FIMSynchronizationService
    Date:          [date/time]
    Event ID:      6313
    Task Category: Server
    Level:         Error
    Keywords:      Classic
    User:          N/A
    Computer:      [farm app server]
    Description:
    The server encountered an unexpected error creating performance 
    counters for management agent "MOSSAD-nos".
     Performance counters will not be available for this management agent.
    
    Rebuilding the connection then resulted in this event appearing every second in the Application log:
    Log Name:      Application
    Source:        Microsoft-SharePoint Products-SharePoint Foundation
    Date:          [date/time]
    Event ID:      6398
    Task Category: Timer
    Level:         Critical
    Keywords:      
    User:          [farm service account]
    Computer:      [farm app server]
    Description:
    The Execute method of job definition 
    Microsoft.Office.Server.UserProfiles.UserProfileImportJob 
    (ID deabe75d-3701-43c1-9968-920febb91024) threw an exception. More information is included below.
    
    Operation is not valid due to the current state of the object.
    
    Rebuilding the entire service application, creating a new connection and then starting a new synchronization results in this message:
    Log Name:      Application
    Source:        Microsoft-SharePoint Products-SharePoint Foundation
    Date:          [date/time]
    Event ID:      6398
    Task Category: Timer
    Level:         Critical
    Keywords:      
    User:          [farm service account]
    Computer:      [farm app server]
    Description:
    The Execute method of job definition 
    Microsoft.Office.Server.UserProfiles.UserProfileImportJob 
    (ID 3c97e43b-0b13-405a-9bf3-3ff0277ad969) threw an exception. 
    More information is included below.
    
    Generic failure 
    Event Xml:
    ...
    
  • Synchronization Service Manager: tool useful for troubleshooting the SharePoint User Profile Service Application.  You'll find it at this folder location:
    \Program Files\Microsoft Office Servers\15.0\Synchronization Service\UIShell
    Look for miisclient.exe.  It enables you to view the real-time status of UPA processes, among many other things.  I don't use it to manage FIM; just to view status information.  For example, I have used to troubleshoot a UPA that failed to start.  As I tried different troubleshooting steps, I monitored Synchronization Service Manager, watching in real-time to see how far UPA started, before stopping.  It also provides helpful other information, such as confirmation of the Domain Controller that it is connecting to.  If you configured UPA using Auto Detect for DC, you can now determine exactly which DC it is connecting to.  You can also get confirmation of the service account that is running UPA (eg, the farm service account), and the number.