Tornado-to-Asyncio Conversion: Critical Analysis

Code References

• Old (Tornado): https://github.com/Nextdoor/kingpin/tree/c2c791ee1acb1e1b1dab1feb64e45585f2f2ddce
• New (asyncio): https://github.com/Nextdoor/kingpin/tree/ede78930777ea0c38092ce0648cb3bb07cc3a3bd
• Full diff: c2c791e...ede7893

Scope

This document is a line-by-line, pattern-by-pattern comparison of the old Tornado-based codebase (commit c2c791ee) against the current asyncio-based codebase (main branch, ede78930, v8.1.0). The analysis focuses on correctness, behavioral regressions, and subtle semantic differences that may not be caught by existing tests.

Executive Summary

The conversion is broadly correct in the happy path but contains several behavioral differences, some of which change failure semantics in ways that could matter in production deployments. The most critical findings are:

1. group.Async task scheduling is fundamentally different -- tasks start immediately in Tornado, but are deferred in asyncio until the event loop yields. This changes timing and interleaving behavior.
2. TaskGroup changes error semantics -- The IAM actors switched from Tornado's yield [list] (which surfaces the first exception) to asyncio.TaskGroup (which wraps exceptions in ExceptionGroup). The BaseActor.execute() handler for ExceptionGroup only partially compensates.
3. The @dry decorator lost its gen.coroutine wrapper -- In Tornado, the @dry decorator called gen.coroutine(f) on the wrapped function, meaning the wrapped function did NOT need to be a coroutine. In asyncio, the wrapper calls await f(), which requires f to be async def. This works only because all call sites were also converted.
4. The @timer decorator has a stacking order change -- Tornado: @gen.coroutine @timer (timer wraps coroutine). Asyncio: @timer async def execute (timer wraps async). The Tornado timer internally called gen.coroutine(f), so the function it wrapped did NOT need to be a coroutine. This is subtly different.
5. HTTP client completely replaced -- tornado.httpclient.AsyncHTTPClient was replaced with synchronous urllib.request.urlopen running in a ThreadPoolExecutor. This changes connection pooling, redirect behavior, timeout handling, and error types.
6. GenericHTTP._execute dry-run has a fire-and-forget coroutine bug in Tornado that was fixed -- The old code had raise gen.Return(self._execute_dry()) which returned the coroutine object without awaiting it. The new code correctly uses return await self._execute_dry().

Prioritized Work Plan

🔴 = fix now   🟠 = fix soon   🟡 = fix when convenient   ✅ = verified correct

Detailed Findings

1. Core Framework: actors/base.py — Timeout Mechanism

#672

2. Core Framework: actors/base.py — ExceptionGroup Handler in execute()

#673

3. actors/group.py — Async Task Scheduling Semantics

#674

4. actors/utils.py — @dry and @timer Decorator Conversions

#675

5. kingpin/utils.py — Deprecated get_event_loop() Usage

#676

6. actors/misc.py — HTTP Calls Have No Timeout

#677

7. actors/aws/base.py — AWS Base Actor api_call() Conversion

#678

8. actors/aws/api_call_queue.py — Queue and Consumer Task Conversion

#679

9. HTTPBaseActor — HTTP Client Replacement (Tornado AsyncHTTPClient → urllib)

#680

10. actors/aws/cloudformation.py — CloudFormation Actor Conversion

#681

11. actors/aws/iam.py — TaskGroup Cancels Remaining Tasks on Failure (HIGH RISK)

#682

12. actors/aws/s3.py — S3 Bucket Actor Conversion

#683

13. bin/deploy.py — CLI Entry Point Shutdown Semantics

#684

Summary of Risks

HIGH Risk

MEDIUM Risk

LOW Risk

No Risk (Correct / Improvements)