Spectre

You can't talk about workflow in WebGUI without talking about Spectre.

S.P.E.C.T.R.E. is the Supervisor of Perplexing Event-handling Contraptions for Triggering Relentless Executions. It triggers WebGUI's workflow and scheduling functions.

(Yes James Bond fans, it was named for you.)

Spectre is the heart of the workflow circulatory system. It is the component that keeps everything moving. It runs scheduled tasks. It tells workers when to run. It keeps track of what needs to run and when.

Spectre::Admin

Spectre Admin doesn't do a lot. Its function is to act as a controller for the other Spectre components. So it's the thing that manages the IP/port connections; It handles startup and shutdown; And it handles tests. 

Spectre::Workflow

When Spectre::Workflow starts up it loads all workflow instances from each site in the system. Each workflow instance consists of a relationship to a workflow description (the workflow class or schema), object data that will be reconstituted to be used in the currently running activity, and a processing priority determined by a settable priority (low, medium, high) and the date the workflow was instantiated.

After loading the workflow instances into the queue, Spectre::Workflow starts handing workflow instances off to a configurable number of workers, and configured intervals. The workers are actually just WebGUI instances. Therefore WebGUI is actually doing most of the work, Spectre is just overseeing operations. Each worker will pull workflow instance activity (aka task) from the top of the list and execute it and return a status code to Spectre::Workflow indicating what happened. If there were no more activities to run in this workflow, it will return a status of "done" and delete the workflow from the queue.

The statuses are as follows:

complete: The worker executed an activity successfully. This signals Spectre::Workflow that this workflow is ready to execute it's next activity.

error: Something bad happened while trying to execute an activity. This signals Spectre::Workflow that it should not request this again for a while to see if things clear up. If it gets an error 5 times in a row, then it will stop trying to execute this workflow, until Spectre is restarted.

waiting: The current activity is waiting on external input. This could mean we're waiting for some time to pass, for a user to answer a question, or who knows what. But regardless, Spectre::Workflow will not execute this workflow again for a short period of time while we wait for the external input to come in.

disabled: The workflow is disabled. Therefore Spectre::Workflow should remove this from its queue.

undefined: The workflow that Spectre::Workflow wanted to run doesn't exist. Therefore it should remove the workflow from its queue.

done: The workflow has completed all its activities. Therefore Spectre::Workflow can remove this workflow from its queue. 

Spectre::Cron

Upon startup Spectre::Cron loads all the schedules from each site into POE's session heap to form a uniform list of schedules. Each schedule consists of a crontab syntax descriptor and a workflow to execute among other things.

After loading the list of schedules into memory the process goes into a monitoring loop. Once per minute Spectre::Cron compares each schedule with the current time. We use DateTime::Cron::Simple to do this. (In benchmarks it's able to compare 500 schedules in under 2 seconds on relatively low end hardware.) If the schedule matches the current time then the workflow is instantiated, where Spectre::Workflow takes over.

If a user changes a schedule while Spectre::Cron is running then WebGUI will notify Spectre::Cron of this change, by deleting the existing schedule, and then sending over the new schedule information. This allows new schedules to be added, and existing schedules to be deleted and changed without restarting the scheduler.