Skip to content

Faulted Task from processing does not shut down processor #122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jsmithtx opened this issue Dec 28, 2018 · 3 comments
Closed

Faulted Task from processing does not shut down processor #122

jsmithtx opened this issue Dec 28, 2018 · 3 comments

Comments

@jsmithtx
Copy link

jsmithtx commented Dec 28, 2018

Using version 2.2.5 of the library, an unhandled exception that escapes from document processing does not shut down the processor. In tracing through the logic, it appears that the PartitionSupervisor does throw the exception (and sets the close reason as ObserverError, line 66-70),
catch (Exception) when (processorTask.IsFaulted)
{
closeReason = ChangeFeedObserverCloseReason.ObserverError;
throw;
}
But the parent PartitionController eats the exception by simply warning that processing fails (line 132-135),
catch (Exception e)
{
Logger.WarnException("partition {0}: processing failed", e, lease.PartitionId);
}
And the caller of this method isn't awaiting the method (line 71)
this.ProcessPartition(supervisor, lease).LogException();

This seems like it might be a bug; for our use of the library, we must have at least once processing and previously we were raising exceptions to "blow the stream", but with the new version of the library, this is no longer happening. What is the recommendation to ensure we stop the change feed when there is an exception? Thanks!

@jsmithtx
Copy link
Author

Any update on this issue?

@ealsur
Copy link
Member

ealsur commented Jan 28, 2019

@jiffypopjr If the Observer throws an error, it would shutdown the Observer, not the Processor. The Processor would eventually spin-up another Observer to continue from the last known checkpoint (Lease Continuation).

@jsmithtx
Copy link
Author

Ok, we run in service fabric, so we've used the SF capability to blow the process which causes SF to immediately start a new process (which in theory, picks it up again). However, #131 is blocking this from happening.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants