Robert Roskam
Engineer Manager at Pantheon
Async Programming
with an emphasis on web frameworks
about me
Organizer
@raiderrobert
twitter | github | medium
where I've contribued
Concurrency
vs
Parallelism
Introduction to Concurrency in Programming Languages
by Sottile, Mattson, Rasmussen
Web Framework
Async
What is the Problem?
Google 1999
Google 2009
Bigger payloads per page
More requests per page
More time spent on them
By More People
Exponential demand
How have we "normally" solved this problem?
Traditional
"Upgrade the Machine" Architecture
1 VM with 1 Core, $10/mo
(Prices are from Linode)
$10/core
1 VM with 2 Cores, $20/mo
$10/core
1 VM with 4 Cores, $40/mo
$10/core
1 VM with 6 Cores, $80/mo
$15/core
1 VM with 8 Cores, $160
$20/core
1 VM with 20 Cores, $480
$24/core
| Core | Price/mo | Price per Core |
|---|---|---|
| 16 | $320 | $20 |
| 20 | $480 | $24 |
| 24 | $640 | $27 |
| 32 | $960 | $30 |
Traditional
"More Boxes" Architecture
1 VM with 4 Cores, $40/mo
$10/core
2 VMs with 4 Cores, each $40/mo
8 Cores for $120/mo
$10/core
DB VM
App VM
3 VMs with 4 Cores, each $40/mo
12 Cores for $120/mo
$10/core
DB VM
App VMs
4 VMs with 4 Cores, each $40/mo
16 Core for $160/mo
$10/core
DB VM
App VMs
Seems Legit?
Fancy
"More Boxes" Architecture
Super Fancy
"More Boxes" Architecture
Fundamentally
All of these are clever variations of....
Throw more hardware at it
Can we do more with the same?
Is our problem actually CPU-bound?
Then why is our solution more CPUs?
Context Switching is suuuuuper expensive
(for a CPU)
Add vs Context Switch
Minimum Cost Difference
10000x
*Depending on # of cores on system, working set size, # of other processes, and virtualization
Add vs Context Switch
Attempt to Humanize
| Add | Context Switch | |
|---|---|---|
| CPU | 1ns | 10 µs to 1 ms |
| Person | 1s | 2.7 hrs to 115.7 days |
Let's Count
Context Switches!
1 VM with 1 Core for 1 Request
1
2
- Web to App
- App to Web
A
Web Server
App Server
1 VM with 2 Cores for 1 Request
B
A
B
A
Web Server
App Server
"add 2 plus 2" API Endpoint
| 1 Core | 2 Cores | |
|---|---|---|
| CPU | 20 µs to 2 ms | 0.1 µs to 1µ |
| Person | 5.4 hours to 231.4 days | 1.6 min to 16 min |
Farm-to-Table
Restaurant Analogy
How can we do better?
Apache vs Nginx
threads vs workers
chart from dreamhost
chart from dreamhost
Nginx solved it. All done, right?
Web Framework
Async
Sync
Classic HTTP request-response cycle:
- Request comes in
- Response goes out
Async
WS, SSE, MQTT, GRPC, etc:
1) Negotiate a connection
2) Request comes in OR
3) Response goes out
4) Repeat (2) or (3) maybe
5) Close connection
async
user threads
coroutines
greenthreads
fibers
Not blocking the CPU while waiting on IO
Network
User Input
Disk*
Preemptive
Scheduler in control.
Acts like modern OS threads/processes and will switch "contexts" per a strategy.
Cooperative
Task is in control.
Can potentially block and not "context" switch to another task.
multi-tasking
Preemptive
Erlang/Elixir
Cooperative
Nodejs/JS
Python asyncio
PHP (swoole)
Ruby (nio4r)
Kotlin coroutines
Java greenthreads
Go
Haskell
Rust
C# Async
My utterly imperfect but best attempt to classify
But what about X language's implementation?
Let's get back to the framework part of this talk
Any Framework in
- Elixir
- Nodejs
- Go
- Haskell
Some Frameworks in
- Python
- PHP
- Rust
- Kotlin
- Java
- C#
Python
- Starlette
- Tornado
- Japronto
- Sanic
PHP
- workerman
- swoft
- amp
Rust
- actix-web
- thruster
- hyper
Ruby :(
Kotlin
- Ktor
Haskell
- Yesod
Elixir
- Phoenix
Nodejs
- Koa
- Fastify
- Express
Go
- gin
- echo
- fasthttp
Why isn't everyone doing this?
- Have to re-think how you write code
- Newer tech so tooling and ecosystem support lacking
Pay Off
https://www.techempower.com/benchmarks/#section=data-r19&hw=cl&test=json&f=zhb207-zik0zj-zik0zj-zijzen-zhxji7-zdk8an-hra0hr-zik0zj-zik0zj-zih7un-73
Async Programming - GVL Tech Nov 2020
By Robert Roskam
