SharePoint 2013: Event 1000: Faulting application name: Microsoft.Workflow.ServiceHost.exe

Problem

After installing the September 2014 PU to your SharePoint 2013 farm, users began to report that their 2013 workflows are performing sluggishly or not at all.  No users reported any adverse impacts to 2010 workflows.  Checking the Workflow Manager 1.0 services, found that all of the services were running except for the Workflow Manager Backend service, which repeatedly stopped and then restarted. Checking the Windows Server Application log, found these error events recurring frequently: 1000, 1026 and 7031 about every 50 seconds or so.  Checking the Microsoft Workflow Operational log, found a regular cycle of error and critical events: 358, 359, 19, 10 and 358 also occurring every minute or so.  This article captures the information and data examined and the efforts undertaken to resolve this issue.

UPDATE (12/03/14): This error was originally  believed to be due to a patch installation.  These errors occurred again six weeks later, this time without there having been any patches, updates or other software installed.  The errors were eventually traced to a workflow design issue.  Removing the running workflow instances resolved the errors.
 
Troubleshooting

1. Checked Workflow Manager services and related services:
• Service Bus Gateway
• Service Bus Message Broker
• Windows Fabric Host Service
• Workflow Manager Backend - repeatedly stopped and restarted; starting it (when stopped) did not resolve issue.
2. Checked Windows Server Application log:
• Saw events 1000, 1026 and 7031
3. Checked Microsoft Workflow logs
• Saw events 358, 359, 19 and 10
4. Checked Workflow Management Site through IIS:
• Site was started.  Restarting did not resolve issue.
5. Checked Workflow Management Application Pool
• Pool was started.  Restarting it did not resolve issue.
6. Checked local Administrators group:
• Added workflow service account to local Administrators group and then rebooted server, but this did not resolve issue.
7. Executed script per reference 2, but this did not resolve issue.
8. Installed full September 2014 CU, but this did not resolve issue.
9. Performed full uninstallation and then a fresh installation of entire Workflow Manager 1.0 system.
• Issue resolved
10. UPDATE (12/03/14)
11. Issue recurred six weeks later. 
12. More thoroughly reviewed error events occuring in Workflow Operational log
• Noted that the error messages discussed workflow instances and that two workflow instance IDs were repeatedly presented in these error messages.
13. Checked Workflows Settings page for all lists having 2013 workflows attached (4).
• Found one 2013 workflow having two running instances.
14. Removed these instances, and then checked server event logs:
• No more error messages - stopped the moment the instances were removed.
15. Workflow designer reviewed workflow, based upon error message text and input from SharePoint administrator
1. Workflow designer discovered an issue in the workflow.
16. New Solution
1. Revise workflow

Solution

1. Review workflow
2. Revise workflow

References

1. Workflow Manager 1.0 (SharePoint 2013): The process was terminated due to an unhandled exception
2. Update Workflow in SharePoint Server 2013
3. Install & Configure Workflow Manager 1.0
4. Configuring The Service Bus Workflow Farm
5. Workflow Manager 1.0-SharePoint 2013
6. New Workflow Manager 1.0 Getting Started Tutorial
7. Updates for SharePoint 2013 Workflow
8. Installing Workflow Manager 1.0
9. SharePoint 2013: Workflow Manager Installation and Configuration
10. Least Privilege Configuration for Workflow Manager with SharePoint 2013
11. Export SharePoint Designer 2013 Workflow XAML
12. Understanding how to package and deploy workflow in SharePoint 2013
13. SharePoint 2013: Workflow Manager Installation and configuration issues…
14. Installing Workflow Manager Cumulative Update 1
15. Understanding SharePoint 2013 Workflow Backup
16. Configuring Workflow Manager 1.0
17. Workflow Manager Farms for SharePoint 2013 Part Three: Switching an existing farm to use Domain CA issued certificates
18. Monitoring Workflow Manager 1.0
19. Creating a New Farm
20. UPDATE
21. Dispatcher failfast is causing the Workflow Manager backend to terminate
22. SharePoint 2013 workflow fundamentals
23. SharePoint 2013: how to restore Workflow Manager 1.0 workflows in the event of a disaster

Notes

• 2013 Workflow Topology:
• Single App Server: hosts Workflow Manager 1.0, Service Bus 1.0 and Windows Fabric Host
• Two WFEs: host Workflow Manager 1.0 Client
• Note that performing a full un-installation, followed by a new installation, completely wipes out any 2013 workflows that may already have been created.   While performing a restore / recovery operation is theoretically possible, and some admins appear to have done this successfully, you don't have to perform a full Workflow 2013 restore operation in order to recovery the 2013 workflows, which are actually stored in the farm content databases.  Instead, you can recover them by restoring the content databases.
   
  UPDATE (12/3/14): opened Microsoft Support ticket on original issue. Microsoft SharePoint Admin support specialist indicated that the 2013 workflows are actually stored in the 2013 content databases; and that these will become available again on successful re-installation of Workflow Manager 1.0 and re-integration with SharePoint 2013. This was verified.

Events

• Windows Server Application Log
• Error Event 1000:
  Faulting application name: Microsoft.Workflow.ServiceHost.exe, version: 1.0.20922.0, time stamp: 0x505e1b24 Faulting module name: KERNELBASE.dll, version: 6.2.9200.16864, time stamp: 0x531d34d8 Exception code: 0xe0434352 Fault offset: 0x0000000000047b8c Faulting process id: 0x4018 Faulting application start time: 0x01cfe8ac9a5561b0 Faulting application path: C:\Program Files\Workflow Manager\1.0\Workflow\Artifacts\Microsoft.Workflow.ServiceHost.exe Faulting module path: C:\Windows\system32\KERNELBASE.dll
   
• Error Event 1026:
  Application: Microsoft.Workflow.ServiceHost.exe Framework Version: v4.0.30319 Description: The process was terminated due to an unhandled exception. Exception Info: Microsoft.Workflow.Common.FatalException Stack: at Microsoft.Workflow.Common.Fx+<>c__DisplayClass2.<failfast>b__0() at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean) at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean) at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) at System.Threading.ThreadHelper.ThreadStart()
   
• Error Event 7031:
  The Workflow Manager Backend service terminated unexpectedly. It has done this 6121 time(s). The following corrective action will be taken in 30000 milliseconds: Restart the service
   
• Microsoft-Workflow Operational Log
• Error Event 358:
  Dispatcher encountered an unexpected exception: System.Runtime.CallbackException: The call to OnRequestAbort on the WorkflowInstance '35616a0a-26da-4765-b92d-3f6eb5727508' threw an exception. This is considered fatal. See inner exception for details. ---> System.AggregateException: Should be processing if notifying about persistable idle. ---> System.NullReferenceException: Object reference not set to an instance of an object. at System.Activities.Runtime.ActivityExecutor.CancelRootActivity() at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.ScheduleCancel() at Microsoft.Activities.Hosting.HostedWorkflowInstance.Cancel() at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() --- End of inner exception stack trace --- at Microsoft.Activities.Hosting.HostedWorkflowInstance.NotifyPaused(Exception abortReason) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.AbortInstance(Exception reason, Boolean isWorkflowThread) at System.Activities.Runtime.ActivityExecutor.AbortWorkflowInstance(Exception reason) --- End of inner exception stack trace --- at System.Activities.Runtime.ActivityExecutor.AbortWorkflowInstance(Exception reason) at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.WorkflowSession.TryStartSubroutine(Guid subroutineId, Guid rootActivityId, Dictionary`2 inArguments, List`1 executionProperties) at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.Cancel() at Microsoft.Activities.Hosting.WorkflowSession.OnUnhandledException(HostedWorkflowInstance throwingInstance, Exception unhandledException, Boolean isRootFaulted) at System.Activities.Runtime.ActivityExecutor.NotifyUnhandledException(Exception exception, ActivityInstance source) at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.WorkflowSession.TryStartSubroutine(Guid subroutineId, Guid rootActivityId, Dictionary`2 inArguments, List`1 executionProperties) at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.ResumeBookmark(Guid subroutineId, Bookmark bookmark, Object value, Action`1 sessionIdleCallback) at Microsoft.Activities.Hosting.WorkflowSession.ResumeInternal(BookmarkWaiter bookmarkWaiter, Object value, DateTime resumptionTime, Action`1 sessionIdleCallback) at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.ProcessWaiter() at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.Isolate(AsyncCompletion callback, IAsyncResult result) at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.ExceptionHandlingFrame(IAsyncResult result)
   
• Critical Event 359:
  Dispatcher failfast is causing the Workflow Manager backend to terminate. Exception: System.AggregateException: The dispatcher failure rate has reached 11%. InnerException is the last failure; see traces for other failures. ---> System.Runtime.CallbackException: The call to OnRequestAbort on the WorkflowInstance '35616a0a-26da-4765-b92d-3f6eb5727508' threw an exception. This is considered fatal. See inner exception for details. ---> System.AggregateException: Should be processing if notifying about persistable idle. ---> System.NullReferenceException: Object reference not set to an instance of an object. at System.Activities.Runtime.ActivityExecutor.CancelRootActivity() at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.ScheduleCancel() at Microsoft.Activities.Hosting.HostedWorkflowInstance.Cancel() at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() --- End of inner exception stack trace --- at Microsoft.Activities.Hosting.HostedWorkflowInstance.NotifyPaused(Exception abortReason) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.AbortInstance(Exception reason, Boolean isWorkflowThread) at System.Activities.Runtime.ActivityExecutor.AbortWorkflowInstance(Exception reason) --- End of inner exception stack trace --- at System.Activities.Runtime.ActivityExecutor.AbortWorkflowInstance(Exception reason) at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.WorkflowSession.TryStartSubroutine(Guid subroutineId, Guid rootActivityId, Dictionary`2 inArguments, List`1 executionProperties) at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.Cancel() at Microsoft.Activities.Hosting.WorkflowSession.OnUnhandledException(HostedWorkflowInstance throwingInstance, Exception unhandledException, Boolean isRootFaulted) at System.Activities.Runtime.ActivityExecutor.NotifyUnhandledException(Exception exception, ActivityInstance source) at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.WorkflowSession.TryStartSubroutine(Guid subroutineId, Guid rootActivityId, Dictionary`2 inArguments, List`1 executionProperties) at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.ResumeBookmark(Guid subroutineId, Bookmark bookmark, Object value, Action`1 sessionIdleCallback) at Microsoft.Activities.Hosting.WorkflowSession.ResumeInternal(BookmarkWaiter bookmarkWaiter, Object value, DateTime resumptionTime, Action`1 sessionIdleCallback) at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.ProcessWaiter() at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.Isolate(AsyncCompletion callback, IAsyncResult result) at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.ExceptionHandlingFrame(IAsyncResult result) --- End of inner exception stack trace --- ---> (Inner Exception #0) System.Runtime.CallbackException: The call to OnRequestAbort on the WorkflowInstance '35616a0a-26da-4765-b92d-3f6eb5727508' threw an exception. This is considered fatal. See inner exception for details. ---> System.AggregateException: Should be processing if notifying about persistable idle. ---> System.NullReferenceException: Object reference not set to an instance of an object. at System.Activities.Runtime.ActivityExecutor.CancelRootActivity() at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.ScheduleCancel() at Microsoft.Activities.Hosting.HostedWorkflowInstance.Cancel() at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() --- End of inner exception stack trace --- at Microsoft.Activities.Hosting.HostedWorkflowInstance.NotifyPaused(Exception abortReason) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.AbortInstance(Exception reason, Boolean isWorkflowThread) at System.Activities.Runtime.ActivityExecutor.AbortWorkflowInstance(Exception reason) --- End of inner exception stack trace --- at System.Activities.Runtime.ActivityExecutor.AbortWorkflowInstance(Exception reason) at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.WorkflowSession.TryStartSubroutine(Guid subroutineId, Guid rootActivityId, Dictionary`2 inArguments, List`1 executionProperties) at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.Cancel() at Microsoft.Activities.Hosting.WorkflowSession.OnUnhandledException(HostedWorkflowInstance throwingInstance, Exception unhandledException, Boolean isRootFaulted) at System.Activities.Runtime.ActivityExecutor.NotifyUnhandledException(Exception exception, ActivityInstance source) at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.WorkflowSession.TryStartSubroutine(Guid subroutineId, Guid rootActivityId, Dictionary`2 inArguments, List`1 executionProperties) at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.ResumeBookmark(Guid subroutineId, Bookmark bookmark, Object value, Action`1 sessionIdleCallback) at Microsoft.Activities.Hosting.WorkflowSession.ResumeInternal(BookmarkWaiter bookmarkWaiter, Object value, DateTime resumptionTime, Action`1 sessionIdleCallback) at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.ProcessWaiter() at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.Isolate(AsyncCompletion callback, IAsyncResult result) at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.ExceptionHandlingFrame(IAsyncResult result)<--- data-blogger-escaped-li="">
   
• Error Event 19:
  Application terminating due to FailFast. Message : An unrecoverable error occurred. For diagnostic purposes, this English message is associated with the failure: 'The dispatcher failure rate has reached 11%. InnerException is the last failure; see traces for other failures.'., Exception: System.AggregateException: The dispatcher failure rate has reached 11%. InnerException is the last failure; see traces for other failures. ---> System.Runtime.CallbackException: The call to OnRequestAbort on the WorkflowInstance '35616a0a-26da-4765-b92d-3f6eb5727508' threw an exception. This is considered fatal. See inner exception for details. ---> System.AggregateException: Should be processing if notifying about persistable idle. ---> System.NullReferenceException: Object reference not set to an instance of an object. at System.Activities.Runtime.ActivityExecutor.CancelRootActivity() at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.ScheduleCancel() at Microsoft.Activities.Hosting.HostedWorkflowInstance.Cancel() at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() --- End of inner exception stack trace --- at Microsoft.Activities.Hosting.HostedWorkflowInstance.NotifyPaused(Exception abortReason) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.AbortInstance(Exception reason, Boolean isWorkflowThread) at System.Activities.Runtime.ActivityExecutor.AbortWorkflowInstance(Exception reason) --- End of inner exception stack trace --- at System.Activities.Runtime.ActivityExecutor.AbortWorkflowInstance(Exception reason) at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.WorkflowSession.TryStartSubroutine(Guid subroutineId, Guid rootActivityId, Dictionary`2 inArguments, List`1 executionProperties) at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.Cancel() at Microsoft.Activities.Hosting.WorkflowSession.OnUnhandledException(HostedWorkflowInstance throwingInstance, Exception unhandledException, Boolean isRootFaulted) at System.Activities.Runtime.ActivityExecutor.NotifyUnhandledException(Exception exception, ActivityInstance source) at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.WorkflowSession.TryStartSubroutine(Guid subroutineId, Guid rootActivityId, Dictionary`2 inArguments, List`1 executionProperties) at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.ResumeBookmark(Guid subroutineId, Bookmark bookmark, Object value, Action`1 sessionIdleCallback) at Microsoft.Activities.Hosting.WorkflowSession.ResumeInternal(BookmarkWaiter bookmarkWaiter, Object value, DateTime resumptionTime, Action`1 sessionIdleCallback) at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.ProcessWaiter() at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.Isolate(AsyncCompletion callback, IAsyncResult result) at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.ExceptionHandlingFrame(IAsyncResult result) --- End of inner exception stack trace --- ---> (Inner Exception #0) System.Runtime.CallbackException: The call to OnRequestAbort on the WorkflowInstance '35616a0a-26da-4765-b92d-3f6eb5727508' threw an exception. This is considered fatal. See inner exception for details. ---> System.AggregateException: Should be processing if notifying about persistable idle. ---> System.NullReferenceException: Object reference not set to an instance of an object. at System.Activities.Runtime.ActivityExecutor.CancelRootActivity() at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.ScheduleCancel() at Microsoft.Activities.Hosting.HostedWorkflowInstance.Cancel() at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() --- End of inner exception stack trace --- at Microsoft.Activities.Hosting.HostedWorkflowInstance.NotifyPaused(Exception abortReason) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.AbortInstance(Exception reason, Boolean isWorkflowThread) at System.Activities.Runtime.ActivityExecutor.AbortWorkflowInstance(Exception reason) --- End of inner exception stack trace --- at System.Activities.Runtime.ActivityExecutor.AbortWorkflowInstance(Exception reason) at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.WorkflowSession.TryStartSubroutine(Guid subroutineId, Guid rootActivityId, Dictionary`2 inArguments, List`1 executionProperties) at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.Cancel() at Microsoft.Activities.Hosting.WorkflowSession.OnUnhandledException(HostedWorkflowInstance throwingInstance, Exception unhandledException, Boolean isRootFaulted) at System.Activities.Runtime.ActivityExecutor.NotifyUnhandledException(Exception exception, ActivityInstance source) at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.WorkflowSession.TryStartSubroutine(Guid subroutineId, Guid rootActivityId, Dictionary`2 inArguments, List`1 executionProperties) at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.ResumeBookmark(Guid subroutineId, Bookmark bookmark, Object value, Action`1 sessionIdleCallback) at Microsoft.Activities.Hosting.WorkflowSession.ResumeInternal(BookmarkWaiter bookmarkWaiter, Object value, DateTime resumptionTime, Action`1 sessionIdleCallback) at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.ProcessWaiter() at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.Isolate(AsyncCompletion callback, IAsyncResult result) at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.ExceptionHandlingFrame(IAsyncResult result)<--- data-blogger-escaped-.="" data-blogger-escaped-li="">
   
• Critical Event 10:
  Unhandled exception in AppDomain: Microsoft.Workflow.Common.FatalException: An unrecoverable error occurred. For diagnostic purposes, this English message is associated with the failure: 'The dispatcher failure rate has reached 11%. InnerException is the last failure; see traces for other failures.'. ---> System.AggregateException: The dispatcher failure rate has reached 11%. InnerException is the last failure; see traces for other failures. ---> System.Runtime.CallbackException: The call to OnRequestAbort on the WorkflowInstance '35616a0a-26da-4765-b92d-3f6eb5727508' threw an exception. This is considered fatal. See inner exception for details. ---> System.AggregateException: Should be processing if notifying about persistable idle. ---> System.NullReferenceException: Object reference not set to an instance of an object. at System.Activities.Runtime.ActivityExecutor.CancelRootActivity() at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.ScheduleCancel() at Microsoft.Activities.Hosting.HostedWorkflowInstance.Cancel() at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() --- End of inner exception stack trace --- at Microsoft.Activities.Hosting.HostedWorkflowInstance.NotifyPaused(Exception abortReason) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.AbortInstance(Exception reason, Boolean isWorkflowThread) at System.Activities.Runtime.ActivityExecutor.AbortWorkflowInstance(Exception reason) --- End of inner exception stack trace --- at System.Activities.Runtime.ActivityExecutor.AbortWorkflowInstance(Exception reason) at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.WorkflowSession.TryStartSubroutine(Guid subroutineId, Guid rootActivityId, Dictionary`2 inArguments, List`1 executionProperties) at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.Cancel() at Microsoft.Activities.Hosting.WorkflowSession.OnUnhandledException(HostedWorkflowInstance throwingInstance, Exception unhandledException, Boolean isRootFaulted) at System.Activities.Runtime.ActivityExecutor.NotifyUnhandledException(Exception exception, ActivityInstance source) at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.TryPropagateCompletion(HostedWorkflowInstance instance) at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.WorkflowSession.TryStartSubroutine(Guid subroutineId, Guid rootActivityId, Dictionary`2 inArguments, List`1 executionProperties) at Microsoft.Activities.Hosting.WorkflowSession.TryDispatchNextSubroutineWorkItem() at Microsoft.Activities.Hosting.WorkflowSession.OnPaused(HostedWorkflowInstance pausedInstance) at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.OnNotifyPaused() at System.Activities.Runtime.ActivityExecutor.OnSchedulerIdle() at System.Activities.Runtime.Scheduler.OnScheduledWork(Object state) at System.Runtime.Fx.SendOrPostThunk.UnhandledExceptionFrame(Object state) at System.Activities.Hosting.WorkflowInstance.WorkflowInstanceControl.Run() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ReleaseLock() at Microsoft.Activities.Hosting.HostedWorkflowInstance.ResumeBookmark(Bookmark bookmark, Object value) at Microsoft.Activities.Hosting.WorkflowSession.ResumeBookmark(Guid subroutineId, Bookmark bookmark, Object value, Action`1 sessionIdleCallback) at Microsoft.Activities.Hosting.WorkflowSession.ResumeInternal(BookmarkWaiter bookmarkWaiter, Object value, DateTime resumptionTime, Action`1 sessionIdleCallback) at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.ProcessWaiter() at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.Isolate(AsyncCompletion callback, IAsyncResult result) at Microsoft.Activities.Dispatcher.DispatchLoopInstanceAsyncResult.ExceptionHandlingFrame(IAsyncResult result) --- End of inner exception stack trace --- --- End of inner exception stack trace --- at Microsoft.Workflow.Common.Fx.<>c__DisplayClass2.b__0() at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart().