Reference

Run progress

Package Trax.Effect·Namespace Trax.Effect.Attributes·Source TraxRunProgressAttribute.cs, RunProgress.cs

[TraxRunProgress] lets the caller who started a run of a train follow it: where it stands, each step as it runs, and how it ended. The caller reads it once with query { runProgress(runId) } and follows it with subscription { runProgress(runId) }. Nobody else can: another caller, an operator and an anonymous one all get NOT_FOUND. The operations view keeps onJunctionEvent for that.

Signature

[AttributeUsage(AttributeTargets.Class, AllowMultiple = false, Inherited = true)]
public sealed class TraxRunProgressAttribute : Attribute
{
    public bool Answers { get; init; }
}

The attribute ships in Trax.Effect, so a train carries it without depending on Trax.Api.GraphQL. The fields exist only on a host where at least one train carries it. The train need not be exposed: a scheduled or machine-invoked train is followed the same way, by whoever its runs record as their starter.

Example

[TraxMutation]
[TraxAuthorize]
[TraxRunProgress]
public class ImportOrdersTrain : ServiceTrain<ImportOrdersInput, ImportOrdersOutput>, IImportOrdersTrain
{
    protected override Task<Either<Exception, ImportOrdersOutput>> Junctions() =>
        Chain<FetchOrders>().Chain<SaveOrders>().Resolve();
}

Who started a run

A run records its starter as a hash, never the id itself: SHA-256("p\0" + principalId) for a caller, with the qualified principal id, and SHA-256("u\0" + userKey) for a machine instance's owner (schema migration 082):

A run with no starter cannot be followed. The id to pass is the run's externalId, which the mutation answers in both modes.

The first read

query {
  runProgress(runId: "a3f1...") {
    runId
    train
    state
    failure
    steps { position name state startedAt endedAt }
    problem { code message }
  }
}

The query needs an authenticated caller. A missing id, another caller's run, a run nobody started and a run of a train without the attribute answer the same NOT_FOUND, through the same query, so the answer says nothing about whether a run exists.

The stream

subscription {
  runProgress(runId: "a3f1...") {
    sequence
    state
    step { position name state }
    failure
    problem { code message }
  }
}

The first event is the state the run was in when the subscription started, so a client that subscribes after the run ended still learns how it ended. After that each event is either a step or a change of state. The stream completes after the run's last state, with the run's last steps sent before it.

A step is sent when the run takes it, and written for the query on a queue that can run behind, so the stream can send a step before the query returns it. A client that subscribes while a run is going can therefore miss a step in both: its read was answered before the step was written, and its stream started after the step was sent. Read the run again once the first event arrives and again after the stream completes, and keep each step the stream sent over the read's copy of it. useRunProgress does both.

A run followed while still queued is read again every few seconds until it is dispatched. One cancelled before dispatch publishes no state of its own, so the stream sends CANCELLED and completes; one whose entry is gone with no run recorded answers NOT_FOUND and completes. A read that fails is tried again, waiting twice as long each time up to 30 seconds; after ten failures in a row the stream sends INTERNAL_ERROR and completes: subscribe again to keep following the run.

A failed run of a manifest stays open after its last state until the scheduler queues its retry, then sends one more event with retriedAs and completes. It looks for the retry after one second, then waits twice as long each time, up to 30 seconds between looks. It completes without one when the manifest is dead-lettered or disabled, or after 30 minutes. useRunProgress follows the retry on its own.

sequence counts up by one per event. A skipped number means events were lost on the way, from a full buffer or a step that could not be published in time: read the run again with the query. A refused subscription answers one event with a problem and completes:

An anonymous subscription is refused with TRAX_AUTHORIZATION, as other subscriptions are.

What a step shows

A step carries the same fields as onJunctionEvent's, with the host detail withheld: no input, output or failure message, no decider type, and a failure's exception type only when it is a TrainException. Without Answers, a step's answer and confidence are null, and a step on a decision track is not named.

The train's events are published to this host's subscriptions because of the attribute. A [TraxBroadcast] subscriber still never sees a train that does not carry [TraxBroadcast] itself.

Checking the starter from a train

A train that reports on a run, such as a decision journal, answers only the caller who started it with IRunStarters (Trax.Mediator.Services.RunStarters), which AddMediator() registers:

public interface IRunStarters
{
    Task<bool> StartedByCurrentCallerAsync(string runId, CancellationToken ct = default);
    Task<long?> LatestExecutionStartedByCurrentCallerAsync(string runId, CancellationToken ct = default);
}

It compares the run's recorded starter with the current caller, as runProgress does, and answers false alike for a missing id, another caller's run and a run nobody started. A train that then reads the execution's own rows takes the id LatestExecutionStartedByCurrentCallerAsync gives rather than the latest under the run id, which another caller's execution recorded under the same id could be; it is null in every case the first answers false.

Persisted operations

A host that requires persisted operations serves runProgress through generated documents, registered like every train's and machine's. trax client writes a typed hook for them.