The Itinera bookRust edition

Part 2: Getting started, Chapter 14

Repairing, then trying once more

What you want to do

Jo's card has expired, so charging it again is pointless, but Jo's bank knows the card that replaced it. When the charge fails, you want the shop to ask the bank for the new card, then charge once more, without the charge knowing anything about renewing cards.

The code

The payment client can now also say which card replaced one. The faked bank says that an expired card was replaced by its holder's card that expires in three years, and prints it. Its error, NetworkDown, is chapter 12's.

Rust, from the program ch14_repairing_then_trying_once_more
/// A payment client: it charges an order's amount to a card, and gives back a receipt, or nothing
/// when the card has expired, or the error of a network that failed; and it says which card
/// replaced one, if any did.
trait Payments: Send + Sync {
    fn charge(&self, order: &str, card: &str, amount: i64) -> Result<Option<String>, NetworkDown>;
    fn replacement(&self, card: &str) -> Option<String>;
}

The repair is a rescue step: a struct deriving RescueStep, whose run, in an impl block marked #[rescue_step], returns a RescueOutcome. It asks for the order's card as an input, and for the reason the charge failed with a field marked #[reason]. It asks the bank for the card's replacement, and contributes it as the order's new card.

Rust, from the program ch14_repairing_then_trying_once_more
/// Repairs a charge that failed because its card had expired: it asks the bank for the card that
/// replaced it, and gives it back as the order's card.
#[derive(RescueStep)]
struct RenewCard {
    #[input]
    card: String,
    #[reason]
    reason: Reason,
}

#[rescue_step]
impl RenewCard {
    fn run(
        self,
        #[service] payments: &dyn Payments,
        #[contributor] mut contributor: Contributor<'_>,
    ) -> Result<RescueOutcome, Error> {
        if self.reason.code() != "card-expired" {
            return Self::fail("not-repairable");
        }
        let Some(card) = payments.replacement(&self.card) else {
            return Self::fail("not-replaced");
        };
        contributor.contribute("card", card);
        Self::succeed()
    }
}

The workflow names it on the charge's line, with rescue = RenewCard, in place of the charge's retry budget and no_retry_on_abnormal_termination: a step with a rescue step has no retry budget, and is rescued after a failed attempt whatever that setting says.

Rust, from the program ch14_repairing_then_trying_once_more
/// The shop's workflow: it charges, repairing a charge that fails, then ships, and shipping may be
/// tried once more.
#[workflow(name = "orders", reporters = [Log, Desk])]
#[with_step(Charge, name = "charge", rescue = RenewCard)]
#[with_step(Ship, name = "ship", retry_budget = 1)]
struct Orders;

The charge and the shipping are chapter 12's, and the desk is chapter 11's.

The log is chapter 12's, and now also says what the two new decisions are about, and names the data a contribution replaced:

Rust, from the program ch14_repairing_then_trying_once_more
EventBody::StepRescuing {
    step,
    cause,
    rescue,
    ..
} => Some(format!("{}: {cause}, rescued by {rescue}", step.step)),
EventBody::StepRescued { step, rescue, .. } => Some(format!("{} by {rescue}", step.step)),
EventBody::ContributionCommitted { key, .. } | EventBody::DataOverwritten { key, .. } => {
    Some(key.clone())
}

The bank's provider is chapter 12's, and the carriers' provider, one carrier for each attempt, chapter 13's. The shop runs Ann's, Gil's and Jo's orders again, and prints:

What ch14_repairing_then_trying_once_more prints, recorded in ch14_repairing_then_trying_once_more.txt
Ann's order:
log orders A-1 #1 journey_started card, order, price, weight
log orders A-1 #2 attempt_started charge, attempt 1
log orders A-1 #3 service_provided charge gets payments from bank, a ch14_repairing_then_trying_once_more::Bank
the bank charges 40 for order A-1
log orders A-1 #4 step_succeeded charge
log orders A-1 #5 contribution_committed receipt
log orders A-1 #6 attempt_started ship, attempt 1
log orders A-1 #7 service_provided ship gets carrier from carriers, a ch14_repairing_then_trying_once_more::MainCarrier
the main carrier books order A-1
log orders A-1 #8 step_info ship: the parcel leaves with receipt R-A-1
log orders A-1 #9 step_succeeded ship
log orders A-1 #10 contribution_committed tracking
log orders A-1 #11 journey_succeeded
the journey succeeded, with receipt R-A-1 and tracking T-A-1
Gil's order:
log orders A-7 #1 journey_started card, order, price, weight
log orders A-7 #2 attempt_started charge, attempt 1
log orders A-7 #3 service_provided charge gets payments from bank, a ch14_repairing_then_trying_once_more::Bank
the bank charges 45 for order A-7
log orders A-7 #4 step_succeeded charge
log orders A-7 #5 contribution_committed receipt
log orders A-7 #6 attempt_started ship, attempt 1
log orders A-7 #7 service_provided ship gets carrier from carriers, a ch14_repairing_then_trying_once_more::MainCarrier
the main carrier books order A-7
log orders A-7 #8 step_failed ship: too-heavy
log orders A-7 #9 step_retrying ship
log orders A-7 #10 attempt_started ship, attempt 2
log orders A-7 #11 service_provided ship gets carrier from carriers, a ch14_repairing_then_trying_once_more::BackupCarrier
the backup carrier books order A-7
log orders A-7 #12 step_info ship: the parcel leaves with receipt R-A-7
log orders A-7 #13 step_succeeded ship
log orders A-7 #14 contribution_committed tracking
log orders A-7 #15 journey_succeeded
the journey succeeded, with receipt R-A-7 and tracking B-A-7
Jo's order:
log orders A-8 #1 journey_started card, order, price, weight
log orders A-8 #2 attempt_started charge, attempt 1
log orders A-8 #3 service_provided charge gets payments from bank, a ch14_repairing_then_trying_once_more::Bank
the bank declines card jo-2024: it has expired
log orders A-8 #4 step_failed charge: card-expired
log orders A-8 #5 step_rescuing charge: failure, rescued by charge_rescue
log orders A-8 #6 attempt_started charge_rescue, attempt 1
the bank says card jo-2024 was replaced by jo-2029
log orders A-8 #7 step_succeeded charge_rescue
log orders A-8 #8 contribution_committed card
log orders A-8 #9 data_overwritten card
log orders A-8 #10 step_rescued charge by charge_rescue
log orders A-8 #11 attempt_started charge, attempt 2
the bank charges 30 for order A-8
log orders A-8 #12 step_succeeded charge
log orders A-8 #13 contribution_committed receipt
log orders A-8 #14 attempt_started ship, attempt 1
log orders A-8 #15 service_provided ship gets carrier from carriers, a ch14_repairing_then_trying_once_more::MainCarrier
the main carrier books order A-8
log orders A-8 #16 step_info ship: the parcel leaves with receipt R-A-8
log orders A-8 #17 step_succeeded ship
log orders A-8 #18 contribution_committed tracking
log orders A-8 #19 journey_succeeded
the journey succeeded, with receipt R-A-8 and tracking T-A-8

How it works

  1. A rescue step repairs what made a step fail. When the charge's attempt fails, the executor doesn't give the charge up: it decides to rescue it, recorded as step_rescuing, with the cause of the failure, here failure, and the rescue step's name. The rescue step is named after the step it rescues, charge_rescue, and runs next. A step never repairs its own failure: its rescue step does, and the workflow decides which rescue step a step has.
  2. It knows why the step failed. Besides inputs it asks for as any step does, such as the card, a rescue step can ask for what the failed attempt reported: its reason, with #[reason]. RenewCard reads it to check that the card expired, the one failure it knows how to repair, before it asks the bank.
  3. It is a step otherwise. It runs once, with its own attempt, asks for services and contributes data, and says how it went with Self::succeed() or Self::fail(...), as any step does. The bank's provider serves every step that asks for payments, the rescue step included, with the bank already made for Jo's journey, which is why the rescue step's attempt has no service_provided of its own.
  4. Its contributions are committed only when it succeeds. The rescue step succeeded, so its card was committed, replacing the card Jo's order started with. The executor records that as data_overwritten, after the commit, naming the key and never the value: the data bag now holds jo-2029, and only the bank's printed line still names the old card.
  5. Then the rescued step runs once more. Once the rescue step has succeeded, the executor records step_rescued, and starts the charge's attempt 2, built afresh from the data bag, so it reads the new card, and the bank charges it. A step with a rescue step has no retry budget, and can't be given one: running it once more is part of the rescue, not a retry.
  6. Only a step that fails is rescued. Ann's and Gil's charges succeed, so their journeys never run charge_rescue, and their logs are the same as in chapter 13, but for the program's name.

Jo's order now succeeds, and so does every order the shop has. But the shop's accountants want a line in their audit for every charge that succeeded, with what it took, and nothing in the shop writes one. The charge could, but then the charge would be about auditing as well as charging.

Outcomes

We have shown how a rescue step repairs a step that failed, so that the step can run once more.

The usual way With Itinera
Repairing a failure, such as renewing an expired card, is usually written inside the code that failed, in a branch of its error handling. The repair is a step of its own, named on the failing step's line in the workflow, and the step it repairs never knows about it.
After a repair, it is often hard to tell from the code which call ran again and with which data. The log shows the rescue, what it replaced in the data bag, and the rescued step's next attempt, in order.