index | ~dongdigua

Performance Issue in Erlang ssh output handling (fixed in OTP 28)

$Id: erlang27_ssh_perf.org,v 26.6 2026/05/25 13:52:52 dongdigua Exp $

1. Background

I wrote a simple elixir script for rickrolling via ssh.
Last summer, I deployed it in a podman container on my Pi,
this semester, I want to deploy it in my school network to rickroll my friends.

It's a R2S with vendor alpine version 3.23.3, the client's output just don't seem right,
it kept flickering! R2S do have a weaker CPU than Pi, but the difference is just to big.

2. Profiling

My friend recommended me to profile it (he's a nerd who profile his own frontend framework by creating 100_000 windows).
So, I did according to https://www.louiscb.com/blog/2023/05/20/profiling-elixir.html and https://www.erlang.org/doc/apps/erts/beamasm#linux-perf-support
(make sure to run perf script on the same machine with /tmp/jit-<pid>.dump)

erlang27_ssh_flame.svg

The hotspot is… lists:reverse/1 ?

3. conv_buf

https://github.com/erlang/otp/blob/61633c62611720cc27f92026d075a1f2862ce522/lib/ssh/src/ssh_cli.erl#L521
ssh_cli:conv_buf/4 is calling lists:reverse/1 when it starts with \n \r \r\n or empty.
which is a lot in my case (25 frame/sec, 32 line/frame)

IO.write("hello")
  │
  ▼
io:request/2  →  {io_request, From, ReplyAs, {put_chars, unicode, Chars}}
  │                          (message to group leader)
  ▼
group.erl: putc_request/4  →  {put_chars_sync, unicode, Binary, Reply}
  │                          (message to driver = ssh_cli process)
  ▼
ssh_cli:handle_msg({Group, Req}, State)     [line 305]
  │
  ▼
ssh_cli:io_request({put_chars_sync,...})     [line 488]
  │
  ▼
ssh_cli:io_request({put_chars, unicode,...}) [line 433]
  │
  ▼
ssh_cli:put_chars(Chars, Buf, Tty)          [line 542]
  │
  ▼
ssh_cli:conv_buf(Chars, Buf, [], Tty)       [line 521] ← HERE

4. Solution

AI recommended me to implement my own cli using ssh_server_channel,
when the agent was working on it, I looked at the master branch, conv_buf is gone!
It's replaced by prim_tty in OTP 28, while alpine 3.23.3 defaults to erlang27-dev.
Upgrade to erlang28-dev fixed the problem, now I can smoothly rickroll my friends.

dongdigua CC BY-NC-SA 禁止转载到私域(公众号,非自己托管的博客等)

Email me to add comment

Proudly made with Emacs Org mode

Date: 2026-03-07 Sat 00:00 Size: 5.2K (≈ 0.7912 mg CO2e)