๐ Diagram for Lesson 8: Activity Retry + Timeout
Visualizing Resilient Activity Execution
Retry patterns and timeout strategies for resilient activity execution
Activity Retry Flow with Exponential Backoff
sequenceDiagram
participant W as Workflow
participant TS as Temporal Server
participant A as Activity
participant ES as External Service
Note over W,ES: Resilient Activity Execution with Retries
W->>TS: Schedule Activity with RetryOptions
Note over TS: StartToCloseTimeout: 2min
MaxAttempts: 3
InitialInterval: 1s TS->>A: Execute Activity (Attempt 1) A->>ES: Call External Service ES-->>A: Timeout/Failure A-->>TS: Activity Failed Note over TS: Wait 1s (InitialInterval) TS->>A: Execute Activity (Attempt 2) A->>ES: Call External Service ES-->>A: Timeout/Failure A-->>TS: Activity Failed Note over TS: Wait 2s (Backoff x2) TS->>A: Execute Activity (Attempt 3) A->>ES: Call External Service ES->>A: Success Response A->>TS: Activity Completed TS->>W: Activity Result
MaxAttempts: 3
InitialInterval: 1s TS->>A: Execute Activity (Attempt 1) A->>ES: Call External Service ES-->>A: Timeout/Failure A-->>TS: Activity Failed Note over TS: Wait 1s (InitialInterval) TS->>A: Execute Activity (Attempt 2) A->>ES: Call External Service ES-->>A: Timeout/Failure A-->>TS: Activity Failed Note over TS: Wait 2s (Backoff x2) TS->>A: Execute Activity (Attempt 3) A->>ES: Call External Service ES->>A: Success Response A->>TS: Activity Completed TS->>W: Activity Result
Retry Strategy by Activity Type
graph TD
A[Activity Type] --> B{Operation Category}
B -->|Critical| C[Payment Processing]
B -->|Standard| D[Validation]
B -->|Long-Running| E[File Processing]
C --> F[MaxAttempts: 5
LongTimeout
Conservative Backoff] D --> G[MaxAttempts: 3
MediumTimeout
Standard Backoff] E --> H[MaxAttempts: 10
Heartbeat Required
Aggressive Backoff] style C fill:#ffebee style D fill:#e8f5e8 style E fill:#e3f2fd style F fill:#ffcdd2 style G fill:#c8e6c9 style H fill:#bbdefb
LongTimeout
Conservative Backoff] D --> G[MaxAttempts: 3
MediumTimeout
Standard Backoff] E --> H[MaxAttempts: 10
Heartbeat Required
Aggressive Backoff] style C fill:#ffebee style D fill:#e8f5e8 style E fill:#e3f2fd style F fill:#ffcdd2 style G fill:#c8e6c9 style H fill:#bbdefb
๐ก Key Insights from the Diagram
Retry Flow Characteristics:
- โ Automatic retry handling by Temporal Server
- โ Exponential backoff prevents service overload
- โ Configurable max attempts prevents infinite retries
- โ Success on final attempt shows resilience in action
Strategy Differentiation:
- ๐ด Critical activities (payments) โ Conservative, fewer attempts
- ๐ข Standard activities (validation) โ Balanced approach
- ๐ต Long-running activities (file processing) โ More attempts, heartbeats
Timeout and Heartbeat Strategy
Timeout Hierarchy:
- ScheduleToStartTimeout โ Maximum queue wait time
- StartToCloseTimeout โ Single execution time limit
- ScheduleToCloseTimeout โ Total time including all retries
- HeartbeatTimeout โ Progress reporting frequency
Benefits:
- Prevents stuck workflows through proper timeout configuration
- Enables progress monitoring through heartbeat reporting
- Balances resilience with resource efficiency
๐ Production Resilience
This diagram demonstrates production-ready patterns for:
- โ Service failure recovery through smart retry policies
- โ Resource protection via timeout boundaries
- โ System stability through exponential backoff
- โ Operational visibility via heartbeat monitoring
Building fault-tolerant distributed systems! ๐