SharePoint 2010 Workflow: An unexpected error has occurred

Problem

You are developing existing workflows on your SharePoint 2010 farm development site.  You delete an existing workflow, while working in SharePoint Designer 2010.  You then test another workflow. After selecting Workflows from the list item dropdown menu, your browser is navigated to the standard helpful informative SharePoint 2010 error mesage:

An unexpected error has occurred

This error occurs no matter what list item workflow history you try to look at.  In fact, you cannot get to the list's workflow page at all.  Neither can you view workflow history or settings.  Troubleshooting steps follow.

Troubleshooting

1. Restored the deleted workflow from the Recycle Bin, and then tested: still experienced error message.
2. Noted the correlation ID presented in the error message.
3. Launched ULSViewer, and then connected it to live feed.
4. Searched for the correlation ID and found this event:

   
   System.ArgumentException: Column '[SomeColumnName]' does not exist. It may 
     have been deleted by another user.  /Path/To/List   
    at Microsoft.SharePoint.SPFieldCollection.GetFieldByInternalName(String strName, Boolean bThrowException)    
    at Microsoft.SharePoint.Workflow.SPWorkflow.GetIStatusAsText(Int32 iStatus)    
    at Microsoft.SharePoint.Workflow.SPWorkflow.GetIStatusAsHtml(Int32 iStatus)    
    at Microsoft.SharePoint.Workflow.SPWorkflow.GetIStatusAsHtml()    
    at Microsoft.SharePoint.ApplicationPages.WorkflowPage.StrStatus(SPWorkflow wf)    
    at ASP._layouts_workflow_aspx.__Render__control11(HtmlTextWriter __w, Control parameterContainer)    
    at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)    
    at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)    
    at System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer)    
    at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)    
    at System.Web.UI.HtmlControls.HtmlForm.RenderChildren(HtmlTextWriter writer)    
    at System.Web.UI.HtmlControls.HtmlForm.Render(HtmlTextWriter output)    
    at System.Web.UI.HtmlControls.HtmlForm.RenderControl(HtmlTextWriter writer)    
    at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)    
    at System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer)    
    at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)    
    at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)    
    at Microsoft.SharePoint.WebControls.UnsecuredLayoutsPageBase.RenderChildren(HtmlTextWriter writer)    
    at System.Web.UI.Page.Render(HtmlTextWriter writer)    
    at Microsoft.SharePoint.WebControls.UnsecuredLayoutsPageBase.Render(HtmlTextWriter writer)    
    at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

5. Noted the SomeColumnName in the event.  This column was completely unfamiliar.  Noted that the column type was not indicated.
6. With ULSViewer up, tested the error for repeatability, by attempted several times to connect to a list item's workflow history page: error was reproducable and correlated with events appearing in the trace log, viewed in ULSViewer.
7. In SharePoint Designer, checked Association Columns, but did not find SomeColumnName.  Also checked Local Variables, but also not successful.
8. Attempted to view workflow information on an archived copy of the list co-located in the same site.  This was successful.
9. Then, using SharePoint Designer, alternatively compared list column listings between the two versions: discovered a column in the archived list missing from the current one.  However, the displayed column name bore no resemblance to that shown in the ULS event message.
10. Then, connected to the site as Administrator and added a new field, SomeColumnName, to the list, as single line string.  Tested again: unsuccessful.  However, the error message correlated with the error message changed:

    
    System.ArgumentException: Column '[SomeNewColumnName]' does not exist. It may have been deleted by another user.  /Path/To/List
     at Microsoft.SharePoint.SPFieldCollection.GetFieldByInternalName(String strName, Boolean bThrowException)    
     at Microsoft.SharePoint.Workflow.SPWorkflow.GetIStatusAsText(Int32 iStatus)    
     at Microsoft.SharePoint.Workflow.SPWorkflow.GetIStatusAsHtml(Int32 iStatus)    
     at Microsoft.SharePoint.Workflow.SPWorkflow.GetIStatusAsHtml()    
     at Microsoft.SharePoint.ApplicationPages.WorkflowPage.StrStatus(SPWorkflow wf)    
     at ASP._layouts_workflow_aspx.__Render__control11(HtmlTextWriter __w, Control parameterContainer)    
     at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)    
     at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)    
     at System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer)    
     at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)    
     at System.Web.UI.HtmlControls.HtmlForm.RenderChildren(HtmlTextWriter writer)    
     at System.Web.UI.HtmlControls.HtmlForm.Render(HtmlTextWriter output)    
     at System.Web.UI.HtmlControls.HtmlForm.RenderControl(HtmlTextWriter writer)    
     at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)    
     at System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer)    
     at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)    
     at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)    
     at Microsoft.SharePoint.WebControls.UnsecuredLayoutsPageBase.RenderChildren(HtmlTextWriter writer)    
     at System.Web.UI.Page.Render(HtmlTextWriter writer)    
     at Microsoft.SharePoint.WebControls.UnsecuredLayoutsPageBase.Render(HtmlTextWriter writer)    
     at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

11. Changed column type to Choice, populated some choices, and repeated test: still unsuccessful.  Same event message appearing in ULS.
12. Removed all workflows from the site and then again tested:  still experienced error message.
1. Conclusion: issue remains tied to list.
13. Exported list from production farm, using CA, and then imported to development farm site using PowerShell.  Connected to site through SharePoint Designer: the workflows were not exported.  Testing unnecessary.
1. Conclusion: workflows are not stored at the list level.
14. Exported site (not site collection) from production farm, using PowerShell, and then restored site to dev farm using PowerShell. Opened browser, navigated to list, checked workflow history: this time able to view without error, but no history was listed. Navigated to list workflows and found none.
1. Conclusion: workflows are not stored at the site level.
15. Stopped IIS on both WFEs. In CA,"detached" content databases from web application (this is a SharePoint detach). Restored backup copies of DBs (in SQL Svr); and then mapped appropriate accounts to restored DBs (in SQL Svr). In CA, "attached" content databases to web application. Started IIS on both WFEs.  Navigated to list and checked workflows: no error, workflow history existed, workflows worked.
1. Conclusion: workflows are stored at the site collection level.

Review

This was my first entry into developing workflows and troubleshooting them.  Further research subsequent to implementing this solution identified a possible workaround not requiring database restore methods (reference 4).

Lessons Learned

Recovery from a poorly handled workflow deletion requires a site collection restore.

References

1. SharePoint 2010 List workflow status – unexpected error
2. SharePoint 2010: Manager
3. Troubleshoot workflow errors (SharePoint Foundation 2010)
4. How to migrate SharePoint 2010 List based workflows
5. SharePoint Designer Workflow Overview
6. How to Delete/Remove a Workflow from SharePoint Designer
7. HOWTO: Remove a Workflow from a List or Library