The Itinera bookRust edition

Part 2: Getting started, Chapter 12

Trying again

What you want to do

A carrier that won't collect a parcel today may collect it on another try, a booking system that failed may work a moment later, and a bank that declines a card may accept it on a second try, so you want the shipping and the charge tried once more before an order is given up. But when the bank's network fails while charging, the bank may already have taken the money, so you want that charge never tried again.

The code

The bank and the carrier now say when something went wrong on their side: each returns a Result, whose error is a type of the program's own, NetworkDown for the bank and BookingFailed for the carrier.

Rust, from the program ch12_trying_again
/// A payment client: it charges an order's amount to a card, and gives back a receipt, or nothing
/// when the card has expired, or the error of a network that failed.
trait Payments: Send + Sync {
    fn charge(&self, order: &str, card: &str, amount: i64) -> Result<Option<String>, NetworkDown>;
}

/// A carrier: it books a parcel of this weight in kg and gives back its tracking number, or the
/// error of a booking system that failed, then says whether it collects the parcel.
trait Carrier: Send + Sync {
    fn book(&self, order: &str, weight: i64) -> Result<String, BookingFailed>;
    fn collects(&self, weight: i64) -> bool;
}

Each error is a struct of its own, whose message says what failed:

Rust, from the program ch12_trying_again
/// The error of the bank's network when it fails.
#[derive(Debug)]
struct NetworkDown;

impl fmt::Display for NetworkDown {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        f.write_str("the payment network is down")
    }
}

impl std::error::Error for NetworkDown {}

/// The error of the carrier's booking system when it fails.
#[derive(Debug)]
struct BookingFailed;

impl fmt::Display for BookingFailed {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        f.write_str("the booking system failed")
    }
}

impl std::error::Error for BookingFailed {}

The faked bank's network fails on any amount over 100, and the faked carrier's booking system on any parcel over 100 kg. The charge charges the order's card, and fails with card-expired when the bank declines it. The ? after payments.charge(...) lets the bank's error escape the step.

Rust, from the program ch12_trying_again
/// Charges an order's price to its card through the payment client, and gives back its receipt.
#[derive(Step)]
struct Charge {
    #[input]
    order: String,
    #[input]
    price: i64,
    #[input]
    card: String,
}

#[step]
impl Charge {
    fn run(
        self,
        #[service] payments: &dyn Payments,
        #[contributor] mut contributor: Contributor<'_>,
    ) -> Result<Outcome, Error> {
        if self.price <= 0 {
            return Self::fail("nothing-to-charge");
        }
        let Some(receipt) = payments.charge(&self.order, &self.card, self.price)? else {
            return Self::fail("card-expired");
        };
        contributor.contribute("receipt", receipt);
        Self::succeed()
    }
}

The shipping is chapter 9's, with the same ? after carrier.book(...), and a failure, too-heavy, when the carrier won't collect the parcel.

Rust, from the program ch12_trying_again
/// Ships an order through the carrier, with its receipt in the parcel, gives back its tracking
/// number, and says when the parcel has left. A parcel the carrier won't collect may be taken on
/// another attempt.
#[derive(Step)]
struct Ship {
    #[input]
    order: String,
    #[input]
    weight: i64,
    #[input]
    receipt: String,
}

#[step]
impl Ship {
    fn run(
        self,
        #[service] carrier: &dyn Carrier,
        #[contributor] mut contributor: Contributor<'_>,
        #[reporter] mut reporter: StepReporter<'_>,
    ) -> Result<Outcome, Error> {
        let tracking = carrier.book(&self.order, self.weight)?;
        contributor.contribute("tracking", tracking);
        if !carrier.collects(self.weight) {
            return Self::fail("too-heavy");
        }
        reporter.info(format!("the parcel leaves with receipt {}", self.receipt))?;
        Self::succeed()
    }
}

The workflow gives both steps a retry budget of 1, with retry_budget = 1 on their #[with_step(...)] lines, and turns off retries for the charge's errors with no_retry_on_abnormal_termination:

Rust, from the program ch12_trying_again
/// The shop's workflow: it charges, then ships, and each may be tried once more, except a charge
/// that ended in an error.
#[workflow(name = "orders", reporters = [Log, Desk])]
#[with_step(
    Charge,
    name = "charge",
    retry_budget = 1,
    no_retry_on_abnormal_termination
)]
#[with_step(Ship, name = "ship", retry_budget = 1)]
struct Orders;

The bank's provider now gives the bank the current year; the ID generator and the carrier's provider are chapter 11's:

Rust, from the program ch12_trying_again
/// The bank, as the payment client of the journey.
#[provider(key = "payments")]
fn bank(&self) -> Result<Provided<dyn Payments>, Error> {
    Ok(provide!(Bank { this_year: 2026 }))
}

The log and the desk are chapter 11's, and the log now also names the error that ended an attempt, and why a step was given up:

Rust, from the program ch12_trying_again
/// What the log says after an event's kind, for the kinds it says more about: the step the event
/// is about, and what it names.
fn detail(body: &EventBody) -> Option<String> {
    match body {
        EventBody::JourneyStarted { initial_keys, .. } => Some(initial_keys.join(", ")),
        EventBody::AttemptStarted { step, .. } => {
            Some(format!("{}, attempt {}", step.step, step.attempt))
        }
        EventBody::StepSucceeded { step, .. } | EventBody::StepRetrying { step, .. } => {
            Some(step.step.to_string())
        }
        EventBody::ServiceProvided {
            step,
            key,
            provider,
            implementation,
            ..
        } => Some(format!(
            "{} gets {key} from {provider}, a {implementation}",
            step.step
        )),
        EventBody::StepFailed { step, reason, .. } => {
            Some(format!("{}: {}", step.step, reason.code()))
        }
        EventBody::StepAbnormalTermination { step, message, .. } => {
            Some(format!("{}: {message}", step.step))
        }
        EventBody::StepGivenUp { step, cause, .. } => Some(format!("{}: {cause}", step.step)),
        EventBody::StepInfo { step, message, .. } => Some(format!("{}: {message}", step.step)),
        EventBody::ContributionCommitted { key, .. } => Some(key.clone()),
        EventBody::JourneyFailed { step, .. } => Some(step.to_string()),
        _ => None,
    }
}

Each order's data now holds its card, and the receipt read back may be missing:

Rust, from the program ch12_trying_again
/// What the shop knows about an order when its journey starts.
#[derive(InitialData)]
struct NewOrder {
    order: String,
    price: i64,
    weight: i64,
    card: String,
}

/// What the shop reads back once the journey has ended.
#[derive(Output)]
struct Order {
    receipt: Option<String>,
    tracking: Option<String>,
}

The shop runs Ann's and Gil's orders, and three from new customers: Jo, whose card expired in 2024, Cat, who spends 120, and Dan, whose parcel weighs 150 kg.

Rust, from the program ch12_trying_again
fn main() -> Result<(), Box<dyn std::error::Error>> {
    // The order, the customer, the price, the parcel's weight in kg and the card.
    let orders = [
        ("A-1", "Ann", 40, 2, "ann-2028"),
        ("A-7", "Gil", 45, 60, "gil-2027"),
        ("A-8", "Jo", 30, 1, "jo-2024"),
        ("A-3", "Cat", 120, 3, "cat-2027"),
        ("A-4", "Dan", 50, 150, "dan-2028"),
    ];
    for (order, customer, price, weight, card) in orders {
        println!("{customer}'s order:");
        let instance = Orders::builder()
            .initial_data(NewOrder {
                order: order.to_owned(),
                price,
                weight,
                card: card.to_owned(),
            })
            .create()?;
        let result = local_executor_for!(Orders).run(instance)?;
        let data: Order = result.output()?;
        let receipt = data.receipt.as_deref().unwrap_or("none");
        let tracking = data.tracking.as_deref().unwrap_or("none");
        println!(
            "the journey {}, with receipt {receipt} and tracking {tracking}",
            result.status.kind(),
        );
    }
    Ok(())
}

It prints:

What ch12_trying_again prints, recorded in ch12_trying_again.txt
Ann's order:
log orders A-1 #1 journey_started card, order, price, weight
log orders A-1 #2 attempt_started charge, attempt 1
log orders A-1 #3 service_provided charge gets payments from bank, a ch12_trying_again::Bank
the bank charges 40 for order A-1
log orders A-1 #4 step_succeeded charge
log orders A-1 #5 contribution_committed receipt
log orders A-1 #6 attempt_started ship, attempt 1
log orders A-1 #7 service_provided ship gets carrier from main_carrier, a ch12_trying_again::MainCarrier
the carrier books order A-1
log orders A-1 #8 step_info ship: the parcel leaves with receipt R-A-1
log orders A-1 #9 step_succeeded ship
log orders A-1 #10 contribution_committed tracking
log orders A-1 #11 journey_succeeded
the journey succeeded, with receipt R-A-1 and tracking T-A-1
Gil's order:
log orders A-7 #1 journey_started card, order, price, weight
log orders A-7 #2 attempt_started charge, attempt 1
log orders A-7 #3 service_provided charge gets payments from bank, a ch12_trying_again::Bank
the bank charges 45 for order A-7
log orders A-7 #4 step_succeeded charge
log orders A-7 #5 contribution_committed receipt
log orders A-7 #6 attempt_started ship, attempt 1
log orders A-7 #7 service_provided ship gets carrier from main_carrier, a ch12_trying_again::MainCarrier
the carrier books order A-7
log orders A-7 #8 step_failed ship: too-heavy
log orders A-7 #9 step_retrying ship
log orders A-7 #10 attempt_started ship, attempt 2
the carrier books order A-7
log orders A-7 #11 step_failed ship: too-heavy
log orders A-7 #12 step_given_up ship: retries exhausted
log orders A-7 #13 journey_failed ship
desk: order A-7 failed at ship
the journey failed, with receipt R-A-7 and tracking none
Jo's order:
log orders A-8 #1 journey_started card, order, price, weight
log orders A-8 #2 attempt_started charge, attempt 1
log orders A-8 #3 service_provided charge gets payments from bank, a ch12_trying_again::Bank
the bank declines card jo-2024: it has expired
log orders A-8 #4 step_failed charge: card-expired
log orders A-8 #5 step_retrying charge
log orders A-8 #6 attempt_started charge, attempt 2
the bank declines card jo-2024: it has expired
log orders A-8 #7 step_failed charge: card-expired
log orders A-8 #8 step_given_up charge: retries exhausted
log orders A-8 #9 journey_failed charge
desk: order A-8 failed at charge
the journey failed, with receipt none and tracking none
Cat's order:
log orders A-3 #1 journey_started card, order, price, weight
log orders A-3 #2 attempt_started charge, attempt 1
log orders A-3 #3 service_provided charge gets payments from bank, a ch12_trying_again::Bank
the bank's network fails while charging order A-3
log orders A-3 #4 step_abnormal_termination charge: the payment network is down
log orders A-3 #5 step_given_up charge: abnormal termination
log orders A-3 #6 journey_failed charge
desk: order A-3 failed at charge
the journey failed, with receipt none and tracking none
Dan's order:
log orders A-4 #1 journey_started card, order, price, weight
log orders A-4 #2 attempt_started charge, attempt 1
log orders A-4 #3 service_provided charge gets payments from bank, a ch12_trying_again::Bank
the bank charges 50 for order A-4
log orders A-4 #4 step_succeeded charge
log orders A-4 #5 contribution_committed receipt
log orders A-4 #6 attempt_started ship, attempt 1
log orders A-4 #7 service_provided ship gets carrier from main_carrier, a ch12_trying_again::MainCarrier
the carrier's booking system fails on order A-4
log orders A-4 #8 step_abnormal_termination ship: the booking system failed
log orders A-4 #9 step_retrying ship
log orders A-4 #10 attempt_started ship, attempt 2
the carrier's booking system fails on order A-4
log orders A-4 #11 step_abnormal_termination ship: the booking system failed
log orders A-4 #12 step_given_up ship: retries exhausted
log orders A-4 #13 journey_failed ship
desk: order A-4 failed at ship
the journey failed, with receipt R-A-4 and tracking none

How it works

  1. An attempt is one execution of a step. Attempts count from 1, and every attempt after the first is a retry, so in Rust the number is a NonZeroU32, held by step.attempt in every event about a step. Each attempt builds the step afresh from the data bag, and what a failed attempt contributed is never committed, as chapter 6 showed, so Gil's journey ends with no tracking number.
  2. An error that escapes is an abnormal termination. A step's run returns Result<Outcome, Error>. A failure, such as card-expired, is an outcome the step chose. An Err is an error the step didn't handle, and itinera::error::Error converts from any error type that implements std::error::Error, Send and Sync, so ? is all it takes. It ends the attempt as a failure of another kind, an abnormal termination, recorded as step_abnormal_termination with the error's message. It never aborts the journey.
  3. The retry budget alone decides whether a step is tried again. It is the number of retries the workflow allows a step, declared on the step's line, and 0 unless the line says otherwise. While a retry is left, a step that failed is tried again, recorded as step_retrying: Gil's shipping and Jo's charge each get attempt 2. A step reports what happened; the workflow decides whether it is tried again.
  4. An abnormal termination spends the same budget. Dan's booking failed with an error, so the shipping was retried just as Gil's was, and failed again with the same parcel.
  5. The workflow can turn that off for a step. no_retry_on_abnormal_termination on the charge's line says an error must never be retried there, since the bank may have taken the money before its network failed. Cat's charge ended in an error, and was given up at once with the cause abnormal termination, though a retry was left. Jo's failure, which is not an error, was retried.
  6. When the budget is spent, the step is given up. Each of Gil's, Jo's and Dan's steps failed again with no retry left, and was given up with the cause retries exhausted. Each decision is an event, after the outcome it answers, and the executor never waits between attempts.
  7. A panic is outside the model. Itinera never catches one: a panic in a step stops the journey with no more events and no result. Return an error or an outcome for anything an order can bring.

Now look at Gil's second attempt in the log. There is no service_provided before it: the main carrier was made once for the journey, as chapter 7 showed, so the second attempt was lent the same carrier, which refused the same parcel again. The shop has a backup carrier, which collects parcels up to 80 kg, but no attempt is ever offered it.

Outcomes

We have shown how the workflow decides, step by step, whether a step that failed is tried again.

The usual way With Itinera
A retry is often a loop around a call, written where the call is, so how often something is tried, and which errors are safe to try again, is decided in many places. Every step's retry budget, and whether its errors are retried, is declared on its line in the workflow.
Whether a failure was retried, and why it was finally given up, is usually known only to the loop that did it. Every retry and every give-up is an event the executor records, with its cause, after the failure it answers.