A terminal case of Linux

Has this ever occurred to you?

You need to take a look at a JSON file in your terminal, so that you pipe it into
jq so you may take a look at it with colours and
stuff.

…oh hey cool bear. No warm-up immediately huh.

Positive, high quality, okay, I will learn the darn man web page for jq… okay it takes
a “filter” after which some information. And the filter we wish is.. . which, simply
like information, means “the present factor”:

There! Now you might have fairly colours! However say your JSON file is definitely fairly
giant, and it does not slot in your terminal window, so that you need to use a pager,
possibly one thing like less.

Effectively, exhibiting a much less invocation is definitely fairly annoying, so as a substitute we’ll
simply use cat as a substitute.

Sure, bear, once more. I understand how you’re feeling about cats, in reality you are closer to
dogs,
however please bea… please let me proceed.

Now the beautiful colours are all gone!

In fact, we are able to power jq to output colours anyway, with -C (brief for
--color-output):

However one thing is afoot.

And it isn’t simply jq! ls even begins separating objects with newlines:

We will “repair” it with --color=at all times, however yeah. There’s undoubtedly some
detection happening there too.

Additionally, if we pipe ls --color=at all times to much less, we see unusual markings!

If we need to see shade in much less, we have to use the -R (brief for
--RAW-CONTROL-CHARS — sure, actually).

In actual fact… let me attempt one thing:

AhAH! We will save colours to a file and print them later – and our good friend
xxd the hex dumper exhibits us that the colours are
actually a part of the output.

To this point we have been operating instructions from
zsh, as a result of that is the shell I dislike
the least proper now.

However what if we execute instructions from one other program? Like, a Rust program?

Let’s attempt doing that:

$ cargo new terminus
     Created binary (software) `terminus` package deal
$ cd terminus/

// in `terminus/src/important.rs`
use std::{error::Error, course of::Command};

fn important() -> Consequence<(), Field<dyn Error>> {
    let out =  String::from_utf8(s.stdout))??
    ;
    println!("{}", out);

    Okay(())
}

What, no fancy crates immediately?

Now, if we run this, we are able to see…

No colours.

That is simply cargo’s output — we might suppress it with --quiet (or -q for
brief) .

Ahhh, the output of ls does not have colours. And it ought to have like,
blue for src and goal, given that they are directories.

So! ls is aware of its output is being redirected someplace, and it does not print
colours. Even once we execute it from a Rust program.

However how? We might take a look at the supply code for ls. That might be enjoyable — I’ve
by no means accomplished that!

Fortunately, there is a GitHub mirror for
coreutils, so it isn’t too exhausting to
discover.

In ls.c, within the the decode_switches perform, we are able to discover a swap inside
of a whereas(true) loop, that appears to course of command-line arguments.

Right here for instance, it allows “human-friendly output”:

C code

// in `coreutils/src/ls.c`
        case 'h':
          file_human_output_opts = human_output_opts =
            human_autoscale | human_SI | human_base_1024;
          file_output_block_size = output_block_size = 1;
          break;

And.. here is the --color swap:

// in `coreutils/src/ls.c`
       case COLOR_OPTION:
           (i == when_if_tty && stdout_isatty ()));
            break;
          

Ah! print_with_color is about to a truthy worth if:

• --color=at all times is handed (we knew that), or
• --color=auto is handed and stdout_isatty() returns true

And here is the code for stdout_isatty():

// in `coreutils/src/ls.c`

/* Return true if commonplace output is a tty, caching the end result.  */

static bool
stdout_isatty (void)

Mhh, isatty, we are able to in all probability name that from Rust, proper?

Looks like a libc perform… ah yep here is a man page:

DESCRIPTION

These features function on file descriptors for terminal sort units.

The isatty() perform determines if the file descriptor fd refers to a
legitimate terminal sort system.

The ttyname() perform will get the associated system identify of a file descriptor
for which isatty() is true.

The ttyname() perform returns the identify saved in a static buffer which
can be overwritten on subsequent calls. The ttyname_r() perform takes
a buffer and size as arguments to keep away from this downside.

Okay okay, how will we name libc features… uhh nicely we all know the Rust commonplace
library depends on libc for a bunch of issues, so absolutely we already hyperlink towards
it…

$ ldd ./goal/debug/terminus
        linux-vdso.so.1 (0x00007ffd7cf69000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f4e15280000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f4e1525d000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f4e15257000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4e15065000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f4e15302000)

Sure we do! That is libc.so.6 within the checklist, which ought to have isatty…

$ nm -D /lib/x86_64-linux-gnu/libc.so.6 | grep -E 'isatty|ttyname'
00000000001324d0 T __ttyname_r_chk
0000000000112cc0 W isatty
0000000000112580 T ttyname
0000000000112900 W ttyname_r

Sure! Though, uh, isatty is a weak symbol.

Anyway. If we outline isatty in an extern "C" block, we ought to be capable of
name it.

// in `terminus/src/important.rs`

use std::{error::Error, os::uncooked::c_int};

extern "C" {
    fn isatty(fd: c_int) -> c_int;
}

const STDOUT: c_int = 1;

fn important() -> Consequence<(), Field<dyn Error>> {
    let stdout_is_tty = unsafe { isatty(STDOUT) } == 1;
    dbg!(stdout_is_tty);
    Okay(())
}

$ cargo construct --quiet
$ ./goal/debug/terminus
[src/main.rs:11] stdout_is_tty = true
$ ./goal/debug/terminus | cat
[src/main.rs:11] stdout_is_tty = false

So that is how they do it!

Okay so there’s two methods we are able to go from right here: we are able to go lower-level, or we are able to
go higher-level.

Let’s first dig down: how does isatty even work. We give it a file descriptor,
actually simply “the integer worth one”, and it tells us if it is a terminal or
not.

However libc is simply, you realize, a giant flaming ball of C code. There is not any purpose for
it to have any further powers — there’s applications that do not use libc in any respect, and
they’re nonetheless capable of inform whether or not or not a file descriptor is a terminal.

Which is to say, libc isn’t accountable for file descriptors. The kernel is.

And we all know a technique userland purposes (like ours) can discuss to the Linux
kernel is by performing a syscall.

So… is it making a syscall? Let’s ask our good friend
strace the… ostrich? (No, for actual,
click on that hyperlink).

We’ll change our important perform to appear to be this:

// in `terminus/src/important.rs`

fn important() -> Consequence<(), Field<dyn Error>> {
    println!("calling isatty...");
    let stdout_is_tty = unsafe { isatty(STDOUT) } == 1;
    println!("calling isatty... accomplished!");
    dbg!(stdout_is_tty);
    Okay(())
}

…simply so it is simpler to pinpoint the second we name isatty:

$ cargo construct --quiet
$ strace -o /tmp/strace.log -- ./goal/debug/terminus
calling isatty...
calling isatty... accomplished!
[src/main.rs:13] stdout_is_tty = true

$ cat /tmp/strace.log | grep 'calling isatty... accomplished' -B 2
write(1, "calling isatty...n", 18)     = 18
ioctl(1, TCGETS, {B38400 opost isig icanon echo ...}) = 0
write(1, "calling isatty... accomplished!n", 24) = 24

$ grep 'calling isatty... accomplished' -B 2 /tmp/strace.log
write(1, "calling isatty...n", 18)     = 18
ioctl(1, TCGETS, {B38400 opost isig icanon echo ...}) = 0
write(1, "calling isatty... accomplished!n", 24) = 24

The unique output is a lot so I used a well-placed grep to whittle it down
to one thing cheap.

Okay, so it is making a syscall. Our println! and dbg! macros find yourself being
write syscalls, which take three arguments: the file descriptor, the information, and
the size.

And it is doing a syscall to ioctl! Additionally passing file descriptor 1.

Which returns… 0. That is kernel for great success!

Nevertheless, if we redirect terminus‘s output:

$ strace -o /tmp/strace.log -- ./goal/debug/terminus > /dev/null
[src/main.rs:13] stdout_is_tty = false

$ grep 'calling isatty... accomplished' -B 2 /tmp/strace.log
write(1, "calling isatty...n", 18)     = 18
ioctl(1, TCGETS, 0x7ffe0d358c30)        = -1 ENOTTY (Inappropriate ioctl for system)
write(1, "calling isatty... accomplished!n", 24) = 24

Then it returns -1, which strace helpfully interprets to ENOTTY, ie. “error
not a TTY”.

So which means… we are able to make that syscall ourselves!

We’ll, nevertheless, want the unstable asm
feature.

No downside! We will swap to the nightly channel:

# in `terminus/rust-toolchain.toml`
[toolchain]
channel = "nightly-2021-09-23"

For those who’re utilizing rustup to handle Rust variations, now, all
cargo instructions within the terminus/ folder ought to use that nightly model. Neat!

// in `terminus/src/important.rs`

#![feature(asm)]

use std::error::Error;

fn important() -> Consequence<(), Field<dyn Error>> {
    // on Linux x86_64, every little thing is an `u64`.
    const STDOUT: u64 = 1;

    // present in linux/supply/embody/uapi/asm-generic/ioctls.h
    const TCGETS: u64 = 0x5401;

    // let's go forward and guess that no matter "TCGETS" is getting
    // does not take greater than 32KiB.
    const GENEROUS_BUFFER_SIZE: usize = 32 * 1024;
    let mut mysterious_buffer = vec![0u8; GENEROUS_BUFFER_SIZE];

    // okay, right here we gooo
    let ret = unsafe { ioctl(STDOUT, TCGETS, mysterious_buffer.as_mut_ptr()) };

    // phew, we made it.
    dbg!(ret);
    Okay(())
}

unsafe fn ioctl(fd: u64, cmd: u64, arg: *mut u8) -> i64 {
    let syscall_number: u64 = 16;
    let ret: u64;
    asm!(
        "syscall",
        inout("rax") syscall_number => ret,
        in("rdi") fd,
        in("rsi") cmd,
        in("rdx") arg,
        // these aren't used, however they could be clobbered by
        // the syscall, so we have to let LLVM know.
        lateout("rcx") _, lateout("r11") _,
        choices(nostack)
    );
    // errors are unfavourable, so that is really an i64
    ret as i64
}

Let’s give it a shot!

$ cargo run --quiet
[src/main.rs:21] ret = 0

$ cargo run --quiet | cat
[src/main.rs:21] ret = -25

And error 25 is…

// in `linux/supply/embody/uapi/asm-generic/errno-base.h`
#outline	ENOTTY 25	/* Not a typewriter */

Hurray!

$ nm ./goal/debug/terminus | grep "U " | grep GLIBC | wc -l
56

And thus concludes our trip down into the decrease degree of abstractions.

Now we all know!

Let’s return to a spot the place we do not really must make our personal syscalls,
lets? There is a couple others I might prefer to check out.

So let’s swap again to secure:

$ rm rust-toolchain.toml

After which simply use the libc crate!

$ cargo add libc
    Updating 'https://github.com/rust-lang/crates.io-index' index
      Including libc v0.2.102 to dependencies

// in `terminus/src/important.rs`
use libc::{isatty, STDOUT_FILENO};

fn important() {
    let stdout_is_tty = unsafe { isatty(STDOUT_FILENO) };
    dbg!(stdout_is_tty);
}

$ cargo run -q
[src/main.rs:5] stdout_is_tty = 1

$ cargo run -q | cat
[src/main.rs:5] stdout_is_tty = 0

There, avenue cred be damned.

Since we’re on Linux, it is mountains of C all the way in which down anyway.

Now, the place have been we? Ah proper! The person web page for isatty additionally talked about
ttyname, which appeared fascinating, as a result of, between you and me, I nonetheless do not
have the faintest thought what a “TTY” really is.

Honest sufficient, let’s attempt it.

// in `terminus/src/important.rs`

use libc::{isatty, ttyname, STDOUT_FILENO};
use std::{error::Error, ffi::CStr};

fn important() -> Consequence<(), Field<dyn Error>> {
    let stdout_is_tty = unsafe { isatty(STDOUT_FILENO) } == 1;
    if stdout_is_tty {
        let tty_name = unsafe { ttyname(STDOUT_FILENO) };
        assert!(!tty_name.is_null());
        let tty_name = unsafe { CStr::from_ptr(tty_name) };
        let tty_name = tty_name.to_str()?;
        println!("stdout is a TTY: {}", tty_name);
    } else {
        println!("stdout isn't a TTY");
    }

    Okay(())
}

$ cargo run -q
stdout is a TTY: /dev/pts/11

$ cargo run -q | cat
stdout isn't a TTY

Ahah! A TTY is… a file?

What can we do with that file? Can we write to it?

Oh my. Sure. Sure we are able to.

(I did not sort within the higher pane — it was printed after I ran echo within the decrease
pane.)

Can we learn from it?

Uhhhhhhh sorta kinda sure. It appears prefer it’s a free-for-all: whoever reads first
will get the.. worm. If I sort actually slowly “cat” wins the race.

That appears prefer it is sensible, however let’s test:

Proper! They actually every have their very own terminals. What number of terminals am I
even on proper now? Assuming they pop out and in of /dev/pts, we ought to be
capable of simply checklist them like so:

$ ls -lhA /dev/pts
complete 0
crw--w---- 1 root tty  136,  0 Sep 23 11:57 0
crw--w---- 1 amos tty  136,  1 Sep 23 11:57 1
crw--w---- 1 amos tty  136, 10 Sep 24 18:23 10
crw--w---- 1 amos tty  136, 11 Sep 24 18:23 11
crw--w---- 1 amos tty  136, 12 Sep 24 18:15 12
crw--w---- 1 amos tty  136, 13 Sep 24 18:23 13
crw--w---- 1 amos tty  136,  2 Sep 24 18:23 2
crw--w---- 1 amos tty  136,  3 Sep 24 12:46 3
crw--w---- 1 amos tty  136,  4 Sep 24 18:22 4
crw--w---- 1 amos tty  136,  5 Sep 24 13:09 5
crw--w---- 1 amos tty  136,  6 Sep 24 12:50 6
crw--w---- 1 amos tty  136,  7 Sep 24 12:47 7
crw--w---- 1 amos tty  136,  8 Sep 24 13:00 8
crw--w---- 1 amos tty  136,  9 Sep 24 18:22 9
c--------- 1 root root   5,  2 Sep 23 11:57 ptmx

Ah! Fourteen (counting from zero). Some even date from yesterday. How do we all know
who they belong to?

Let’s attempt it for the present TTY:

$ cargo run -q
stdout is a TTY: /dev/pts/13

$ lsof /dev/pts/13
COMMAND     PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
zsh       10614 amos    0u   CHR 136,13      0t0   16 /dev/pts/13
zsh       10614 amos    1u   CHR 136,13      0t0   16 /dev/pts/13
zsh       10614 amos    2u   CHR 136,13      0t0   16 /dev/pts/13
zsh       10614 amos   10u   CHR 136,13      0t0   16 /dev/pts/13
zsh       10661 amos   15u   CHR 136,13      0t0   16 /dev/pts/13
zsh       10661 amos   16u   CHR 136,13      0t0   16 /dev/pts/13
zsh       10661 amos   17u   CHR 136,13      0t0   16 /dev/pts/13
zsh       10667 amos   15u   CHR 136,13      0t0   16 /dev/pts/13
zsh       10667 amos   16u   CHR 136,13      0t0   16 /dev/pts/13
zsh       10667 amos   17u   CHR 136,13      0t0   16 /dev/pts/13
zsh       10669 amos   15u   CHR 136,13      0t0   16 /dev/pts/13
zsh       10669 amos   16u   CHR 136,13      0t0   16 /dev/pts/13
zsh       10669 amos   17u   CHR 136,13      0t0   16 /dev/pts/13
gitstatus 10670 amos   15u   CHR 136,13      0t0   16 /dev/pts/13
gitstatus 10670 amos   16u   CHR 136,13      0t0   16 /dev/pts/13
gitstatus 10670 amos   17u   CHR 136,13      0t0   16 /dev/pts/13
lsof      14867 amos    0u   CHR 136,13      0t0   16 /dev/pts/13
lsof      14867 amos    1u   CHR 136,13      0t0   16 /dev/pts/13
lsof      14867 amos    2u   CHR 136,13      0t0   16 /dev/pts/13

Ha! That is numerous processes. I suppose all of them simply coordinate collectively to make
that fancy immediate occur. (It is
powerlevel10k, by the way in which).

Oh and lsof even discovered itself!

$ strace -o strace.log -- lsof /dev/pts/10 &> /dev/null

$ grep -E 'openat.*/proc' strace.log | head
openat(AT_FDCWD, "/proc/filesystems", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/proc/mounts", O_RDONLY) = 3
openat(AT_FDCWD, "/proc/16057/fdinfo/3", O_RDONLY) = 6
openat(AT_FDCWD, "/proc/locks", O_RDONLY) = 3
openat(AT_FDCWD, "/proc", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
openat(AT_FDCWD, "/proc/1/stat", O_RDONLY) = 6
openat(AT_FDCWD, "/proc/1/maps", O_RDONLY) = -1 EACCES (Permission denied)
openat(AT_FDCWD, "/proc/1/fd", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = -1 EACCES (Permission denied)
openat(AT_FDCWD, "/proc/45/stat", O_RDONLY) = 6
openat(AT_FDCWD, "/proc/45/maps", O_RDONLY) = -1 EACCES (Permission denied)

Oh. Uh oh.

$ grep -E 'openat.*/proc' strace.log | wc -l
1267

Oh. Ooooooh okay. It is simply going by
procfs, in search of all open file
descriptors of all processes.

Effectively then.

Okay. OKAY. That is all nicely and good. I really feel like we’re making tangible progress
right here, however there’s one thing that puzzles me.

Aaaaaanyway.

Sure, bear. Sure, we’re solely holding half of it.

As a result of as you’ll have observed, once we rudely echoed one thing into
/dev/pts/N, it confirmed up within the different pane, however zsh did not attempt to execute it.

Effectively, let’s assume for a second.

After we sort right into a terminal, who will get the enter?

Flawed! It completely isn’t zsh. First, it is my Logitech K120 keyboard, which I
preserve making an attempt to interchange however is the one factor that has labored for me for the previous
decade, that is proper mechanical keyboard nerds, your resident deep dive author
is utilizing second-rate workplace provides.

After which, nicely, one thing one thing USB-A, a hub, USB-C, presumably some Intel
piece of {hardware} inside my MacBookPro, the motive force for the USB host controller,
ultimately the Home windows kernel, after which… the enter chain? Who is aware of what number of
layers that is acquired — however ultimately we get a “key occasion”, no matter type it takes.

Then it goes by Windows
Terminal which, actually if
you are still utilizing cmd.exe nicely — what are you ready to change over?

In fact that is as a result of I’ve a bizarre
WSL2 setup
happening. I attempted utilizing Linux as my important OS, I really did.

However my level is: it isn’t zsh that will get the enter. It is Home windows Terminal. If I
was on Linux, it would be alacritty or
xfce4-terminal or one thing.

Flawed! As a result of that might not be a TTY. If zsh’s commonplace enter was merely a
pipe that Home windows Terminal wrote to (let’s overlook there’s two totally different OSes
in motion for a minute), isatty would undoubtedly return false.

And apart from… watch this:

And now this:

Effectively, how do you do this with a pipe, huh bear? HOW INDEED?

THAT’S RIGHT. With an ioctl syscall. Which we did above.

However the ioctls we need to carry out do not work until the file descriptor is a TTY.

Then we’re again to sq. one, sure.

And you realize what’s worse? All that data, just like the terminal’s measurement and
whatnot, it is out-of-band!

Yeah! It isn’t like colours that are like “ooh here is a humorous byte after which
ASCII numbers and punctuation I suppose, 420 shipit” — these are kernel operations
on file descriptors, they are not within the stream in any respect.

When a terminal is resized? Effectively HERE COMES THE FUN BIT! It isn’t through ioctls!
By no means! As a result of these are a “pull” interface, you simply ask for the information when
you want it — and you do not need to be continuously requesting it.

(It is presumably “pricey”, in any other case ls would not memoize/cache it. On the very
least it crosses the kernel boundary, as a result of it is a syscall).

So you know the way you realize? {That a} terminal is resized?

WELL YOU GET A SIGNAL

THAT’S RIGHT. A SIGNAL. Identical to when a course of is killed or stopped.

So between in-band data (like ANSI escape
codes) and out-of-band
data (like alerts and ioctls), GOOD LUCK WRITING A TERMINAL EMULATOR.

“EVERYTHING IS A FILE” MY LILY-WHITE BUTT.

WELL THEY HAD TO FIND OUT ONE DAY.

Okay, okay, let’s take a breather. We really needn’t care about alerts,
fortunately. We’ll simply by no means ever resize our terminals. That is high quality. It is high quality.
It is okay. We’re okay.

I do not actually assume we have to care about ioctls both, if all we need to do
is simply faux we’re a terminal – we needn’t get the terminal data, our
little one course of does… similar to zsh requests the terminal data set by Home windows
Terminal.

Okay. Positive. We’ll attempt.

Let’s take a look at that perform nearer… gonna be paraphrasing that man web page to
modernize it somewhat bit.

The perform openpty() makes an attempt to acquire the subsequent out there pseudo-
terminal from the system (see pty(4)).

Okay…

If it efficiently finds one, it subsequently adjustments the possession of the secondary
system to the true UID of the present course of, the group membership to the group
“tty” (if such a bunch exists within the system), the entry permissions for studying
and writing by the proprietor, and for writing by the group, and invalidates any
present use of the road by calling revoke(2).

Okay, certain, permissions, why not.

If the argument identify isn’t NULL, openpty() copies the pathname of the
secondary pty to this space. The caller is answerable for allocating the
required house on this array.

Oh HELL no, we’re not letting libc write previous the tip of a buffer. Not immediately.
That’ll be null, thanks very a lot.

If the arguments termp or winp are usually not NULL, openpty() initializes the
termios and window measurement settings from the constructions these arguments
level to, respectively.

Ah! These can be null.

Upon return, the open file descriptors for the first and secondary aspect of
the pty are returned within the places pointed to by aprimary and asecondary,
respectively.

Very nicely, let’s examine what occurs then.

// in `terminus/src/important.rs`

use libc::{isatty, ttyname};
use std::{error::Error, ffi::CStr};

fn important() -> Consequence<(), Field<dyn Error>> {
    let mut primary_fd: i32 = 0;
    let mut secondary_fd: i32 = 0;
    println!("Opening pty...");
    unsafe {
        let ret = libc::openpty(
            &mut primary_fd,
            &mut secondary_fd,
            std::ptr::null_mut(),
            std::ptr::null(),
            std::ptr::null(),
        );
        if ret != 0 {
            panic!("Did not openpty!");
        }
    };
    dbg!(primary_fd, secondary_fd);

    let is_tty = unsafe { isatty(secondary_fd) } == 1;
    if is_tty {
        let tty_name = unsafe { ttyname(secondary_fd) };
        assert!(!tty_name.is_null());
        let tty_name = unsafe { CStr::from_ptr(tty_name) };
        let tty_name = tty_name.to_str()?;
        println!("secondary is a TTY: {}", tty_name);
    } else {
        println!("secondary isn't a TTY");
    }

    Okay(())
}

$ cargo run --quiet
Opening pty...
[src/main.rs:20] primary_fd = 3
[src/main.rs:20] secondary_fd = 4
secondary is a TTY: /dev/pts/9

Sure.

YES.

WE HAVE A PSEUDO-TERMINAL. AHHHHH

Sure sure how will we get one other course of to make use of mentioned pseudo-terminal. Effectively, you’d
assume you might simply move the file descriptor as stdin/stdout/stderr, proper?

Flawed!

As a result of see, Linux processes have, like, “periods”, and people periods have
“leaders” and in addition they’ve a “controlling terminal” and there is undoubtedly a
name to “allocate a brand new session” however the way in which you alter the “controlling
terminal” is far a lot funnier.

Let’s take a look at the code for login_tty, which does each:

// in `glibc/login/login_tty.c`

int
__login_tty (int fd)
{
    __setsid();
#ifdef TIOCSCTTY
    if (__ioctl(fd, TIOCSCTTY, NULL) == -1)
        return (-1);
#else
    {
      /* This would possibly work.  */
      char *fdname = ttyname (fd);
      int newfd;
      if (fdname)
        {
          if (fd != 0)
        _close (0);
          if (fd != 1)
        __close (1);
          if (fd != 2)
        __close (2);
          newfd = __open64 (fdname, O_RDWR);
          __close (newfd);
        }
    }
#endif
    whereas (__dup2(fd, 0) == -1 && errno == EBUSY)
      ;
    whereas (__dup2(fd, 1) == -1 && errno == EBUSY)
      ;
    whereas (__dup2(fd, 2) == -1 && errno == EBUSY)
      ;
    if (fd > 2)
        __close(fd);
    return (0);
}

I… I do not know the place to start out. On platforms that do not help the
TIOCSCTTY ioctl, it closes commonplace enter, output and error and opens
the secondary aspect of the TTY.

As a result of that makes it the controlling terminal. Due to course.

After which uhh these whereas loops look horrifying to me however I suppose they’re
extraordinarily commonplace *nix stuff and I am actually simply exhibiting my lack of expertise
there.

Regardless: yuck.

However you realize… so long as it really works…

The actual query is… when will we do this? When will we name login_tty?

After we executed a program from Rust, we did one thing like this:

        Command::new("/bin/ls")
            .arg("--color=auto")
            .output()
            .map(|s| String::from_utf8(s.stdout))??

However that good, high-level API is hiding the horrible reality.

A large number of cursed issues occur when spawning a course of on Linux.

Effectively, until you utilize posix_spawn. That is
the nice one.

However the conventional means, which we’ll don’t have any alternative however to make use of right here, has two
necessary steps.

First, we fork. This creates an “precise” copy of the calling course of, besides
for like, twenty various things (like having a special PID (course of ID), a
totally different PPID (guardian course of ID), and many others.)

What’s enjoyable about fork is that we’re splitting the space-time continuum —
technically, it returns twice. As soon as within the guardian course of, and as soon as within the
little one course of.

Within the little one course of, it returns 0, and that is how you realize it is the little one
course of as a result of, nicely, because it’s an “precise” copy, they’re nonetheless executing the
identical code at this level. So that you examine towards 0, and issues diverge from
there.

And often, at that time, you need to transfer on to exec, which asks the kernel
to violently change the present course of (the kid course of we simply original
from the ribs of Adam the reminiscence house of the calling course of) with
whichever ELF file we mentioned to make use of.

Which is the way in which you usually run applications. Until you really have too much
time on your hands.

So let’s assume! We’ve got login_tty, which makes the calling course of the chief
of a brand new session, and units its controlling terminal to whichever file descriptor
we handed.

Effectively. If we name it earlier than fork it’ll mess with our guardian course of,
which is sort of undoubtedly not what we wish. We do not need to grow to be the chief
of our session. We do not need to change our personal controlling terminal.

We solely need, say, ls to have as controlling terminal, the pseudo-terminal we
simply created (with openpty).

If we name it after exec, nicely… nicely we will not name something after exec.

Any code from the guardian course of, that was “copied into” (actually, simply mapped)
the kid course of, stops present the second we name exec.

exec by no means returns, very similar to honey badger don’t
care.

So actually, that solely leaves us one alternative.. we should execute login_tty between
fork and exec.

Which implies we will not use posix_spawn.

So we’ll simply look into our Rust code the place we are able to do this and…

Rust code

        Command::new("/bin/ls")
            .arg("--color=auto")
            .output()
            .map(|s| String::from_utf8(s.stdout))??

I do not see fork. Or exec. Or posix_spawn.

Ahhh sure, they’re nicely hidden, in Horrible Fact Land, also called the Rust
standard library.

See, there is a posix_spawn fn there, and…

Rust code

// in `library/std/src/sys/unix/process_unix.rs`

    fn posix_spawn(
        &mut self,
        stdio: &ChildPipes,
        envp: Choice<&CStringArray>,
    ) -> io::Consequence<Choice<Course of>> {
        use crate::mem::MaybeUninit;
        use crate::sys::{self, cvt_nz};

        if self.get_gid().is_some()
            || self.get_uid().is_some()
            || (self.env_saw_path() && !self.program_is_path())
            || !self.get_closures().is_empty()
            || self.get_groups().is_some()
            || self.get_create_pidfd()
        {
            return Okay(None);
        }

        // ...
    }

…and the very first thing it does is test if we’re doing one thing funky! And if we
are, it does not really use posix_spawn.

Funky issues like, I do not know, setting a special UID (person ID) or GID (group
ID), or… hey, what’s that about closures?

furiously going by std So if that is set right here then.. THERE! I acquired it!

What? Command::pre_exec? Good job bear! Now we are able to fina-

Wait, it is unsafe. Why is it unsafe.

It is uns- ah. Sure. Effectively, let’s evaluation the docs.

Notes and Security

This closure can be run within the context of the kid course of after a fork.
This primarily implies that any modifications made to reminiscence on behalf of this
closure will not be seen to the guardian course of. That is usually a really
constrained setting the place regular operations like malloc, accessing
setting variables by std::env or buying a mutex are usually not assured
to work (as a consequence of different threads maybe nonetheless operating when the fork was run).

Ooooh boy. Okay so we cannot allocate anyth-

This additionally implies that all assets equivalent to file descriptors and memory-mapped
areas acquired duplicated. It’s your accountability to be sure that the closure
doesn’t violate library invariants by making invalid use of those duplicates.

Ah uh

Panicking within the closure is secure provided that all of the format arguments for the
panic message might be safely formatted; it’s because though Command calls
std::panic::always_abort earlier than calling the pre_exec hook, panic will nonetheless attempt
to format the panic message.

Effectively I-

When this closure is run, facets such because the stdio file descriptors and
working listing have efficiently been modified, so output to those places
might not seem the place meant.

Okay, okay I acquired it — stuff will get actual bizarre in there.

We’ll simply login_tty and get out of right here as quick as we are able to.

Leeeeeeeeeeeet’s go:

Rust code

// in `terminus/src/important.rs`

use std::{error::Error, os::unix::prelude::CommandExt, course of::Command};

fn openpty() -> (i32, i32) {
    let mut primary_fd: i32 = -1;
    let mut secondary_fd: i32 = -1;
    unsafe {
        let ret = libc::openpty(
            &mut primary_fd,
            &mut secondary_fd,
            std::ptr::null_mut(),
            std::ptr::null(),
            std::ptr::null(),
        );
        if ret != 0 {
            panic!("Did not openpty!");
        }
    };
    (primary_fd, secondary_fd)
}

fn important() -> Consequence<(), Field<dyn Error>> {
    let (primary_fd, secondary_fd) = openpty();
    dbg!(primary_fd, secondary_fd);

    let mut cmd = Command::new("/bin/ls");
    cmd.arg("--color=auto");
    unsafe {
        cmd.pre_exec(transfer || {
            if libc::login_tty(secondary_fd) != 0 {
                panic!("could not set the controlling terminal or one thing");
            }
            Okay(())
        })
    };
    let output = cmd.output().map(|out| String::from_utf8(out.stdout))??;
    println!("{}", output);

    Okay(())
}

Shell session

$ cargo run --quiet
Opening pty...
[src/main.rs:19] primary_fd = 3
[src/main.rs:19] secondary_fd = 4

Oh that is.

That is maybe somewhat too quiet.

What the heck occurred right here…

Wait wait wait, we’re calling output().

How do you assume output() works?

Effectively… it in all probability has to redirect stdout and stderr to pip-

To pipes, sure exactly. And then it runs the pre-exec closures,
certainly one of which calls login_tty, which..

Oh yeahhh! That might undoubtedly override no matter output() is doing.

However then.. then how will we get the output from /bin/ls?

If it is holding the secondary… and we’re holding the first… we have to..
learn from it? Doubtlessly possibly?

Okay high quality let’s attempt:

Rust code

use std::{
    error::Error,
    fs::File,
    io::Learn,
    os::unix::prelude::{CommandExt, FromRawFd},
    course of::Command,
};

// omitted: fn openpty()

fn important() -> Consequence<(), Field<dyn Error>> {
    let (primary_fd, secondary_fd) = openpty();
    dbg!(primary_fd, secondary_fd);

    let mut cmd = Command::new("/bin/ls");
    cmd.arg("--color=auto");
    unsafe {
        cmd.pre_exec(transfer || {
            if libc::login_tty(secondary_fd) != 0 {
                panic!("could not set the controlling terminal or one thing");
            }
            Okay(())
        })
    };
    let mut little one = cmd.spawn()?;

    println!("Opening main...");
    let mut main = unsafe { File::from_raw_fd(primary_fd) };
    println!("Studying from main...");
    let mut buffer = String::new();
    main.read_to_string(&mut buffer)?;
    println!("All accomplished!");
    println!("{}", buffer);

    little one.wait()?;

    Okay(())
}

Shell session

$ cargo run --quiet
[src/main.rs:29] primary_fd = 3
[src/main.rs:29] secondary_fd = 4
Opening main...
Studying from main...
^C

Okay that simply will get caught endlessly. Mh.

Uhhh possibly the terminal… stays open?? What does strace say?

Shell session

$ cargo construct --quiet
$ strace ./goal/debug/terminus
(lower)
write(1, "Opening main...n", 19Opening main...
)    = 19
write(1, "Studying from main...n", 24Reading from main...
) = 24
learn(3, "Cargo.lock  Cargo.toml  33[0m33[01", 32) = 32
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=31500, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
read(3, ";34msrc33[0m  strace.log  33[01;34", 32) = 32
read(3, "mtarget33[0mrn", 64)        = 13
read(3, ^C0x56265b70c23d, 51)             = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
strace: Process 31499 detached

Ohhh. OHHH. Yeah! It does! And read_to_string reads until EOF, which never
happens.

As noted by many, openpty returns an open file descriptor to the primary and
the secondary. We passed a copy of the secondary to the child process but didn’t
close the parent process’s copy of it.

If we did close the parent process’s copy, then we would see EOF from the
primary as soon as the child process exits, and everything past this point in
this article would be unnecessary.

However, amos has a use case where he wants to re-use the terminal for spawning
several processes in sequence, yet needs to tell apart the output from each
individual process, so it makes sense for him. In most other cases though, you
can just close the parent’s secondary fd and be done with it!

So we need to…

..read from the primary only until the child process exits?

Yeah, that!

Rust code

fn main() -> Result<(), Box<dyn Error>> {
    let (primary_fd, secondary_fd) = openpty();
    dbg!(primary_fd, secondary_fd);

    let mut cmd = Command::new("/bin/ls");
    cmd.arg("--color=auto");
    unsafe {
        cmd.pre_exec(move || {
            if libc::login_tty(secondary_fd) != 0 {
                panic!("couldn't set the controlling terminal or something");
            }
            Ok(())
        })
    };
    let mut child = cmd.spawn()?;

    let mut primary = unsafe { File::from_raw_fd(primary_fd) };

    let mut out = vec![];
    let mut buffer = vec![0u8; 1024];

    loop {
        let n = main.learn(&mut buffer)?;
        out.extend_from_slice(&buffer[..n]);
        println!("Learn {} bytes...", n);

        if little one.try_wait()?.is_some() {
            break;
        }
    }
    println!("Youngster exited!");
    println!("{}", String::from_utf8(out)?);

    Okay(())
}

And, lo and behold:

Shell session

$ cargo run --quiet
[src/main.rs:29] primary_fd = 3
[src/main.rs:29] secondary_fd = 4
Learn 77 bytes...
Youngster exited!
Cargo.lock  Cargo.toml  src  strace.log  goal

We’ve got colours.

Oh! In fact.

And lo and behold, we’ve got colours:

We even have: a race situation.

If the kid exits between our name to try_wait and the subsequent learn, we’ll be
caught endlessly making an attempt to learn.

How might we presumably resolve this?

Okay so, it will not be fairly, but it surely will not race.

These each sound like negatives.

We’ll even run one thing fancier than ls, like, cargo test thrice in a
row, sleeping 1 second between every.

Rust code

// in `terminus/src/important.rs`


fn important() -> Consequence<(), Field<dyn Error>> {
    let (primary_fd, secondary_fd) = openpty();
    dbg!(primary_fd, secondary_fd);

    let mut cmd = Command::new("/bin/bash");
    cmd.arg("-c")
        .arg("for i in $(seq 1 3); do cargo test; sleep 1; accomplished");

    unsafe {
        cmd.pre_exec(transfer || {
            if libc::login_tty(secondary_fd) != 0 {
                panic!("could not set the controlling terminal or one thing");
            }
            Okay(())
        })
    };
    let mut little one = cmd.spawn()?;

    enum Msg {
        Output(Vec<u8>),
        Exit,
    }

    let (tx, rx) = std::sync::mpsc::channel();

    let read_tx = tx.clone();
    std::thread::spawn(transfer || {
        let mut main = unsafe { File::from_raw_fd(primary_fd) };
        let mut buffer = vec![0u8; 1024];

        loop {
            let n = main.learn(&mut buffer).unwrap();
            println!("Learn {} bytes...", n);
            let slice = &buffer[..n];
            read_tx.ship(Msg::Output(slice.to_vec())).unwrap();
        }
    });

    std::thread::spawn(transfer || {
        little one.wait().unwrap();
        tx.ship(Msg::Exit).unwrap();
    });

    let mut out = vec![];

    loop {
        let msg = rx.recv()?;
        match msg {
            Msg::Output(buffer) => out.extend_from_slice(&buffer[..]),
            Msg::Exit => break,
        }
    }

    println!("Youngster exited!");
    println!("{}", String::from_utf8(out)?);

    Okay(())
}

Mhhhh. If solely there was some Rust characteristic… that allows you to take care of issues
like these…

Oh expensive. You actually assume we must always?

Just one method to discover out…

What time is it? It is tokio time!

Channels be damned, we’re bringing an entire darn executor with us. As a result of what
is an executor? A depressing pile of secrets and techniques pursuits and duties.

And we’re curiosityed in figuring out when the kid exits, and in addition once we can
learn from our pseudo-terminal main.

Have you learnt what you are doing?

So, uhhh let’s go:

Shell session

$ cargo add tokio@1.12.0 --features full
    Updating 'https://github.com/rust-lang/crates.io-index' index
      Including tokio v1.12.0 to dependencies with options: ["full"]

Rust code

// in `terminus/src/important.rs`

use std::{error::Error, os::unix::prelude::FromRawFd};
use tokio::{fs::File, io::AsyncReadExt, course of::Command};

fn openpty() -> (i32, i32) {
    let mut primary_fd: i32 = -1;
    let mut secondary_fd: i32 = -1;
    unsafe {
        let ret = libc::openpty(
            &mut primary_fd,
            &mut secondary_fd,
            std::ptr::null_mut(),
            std::ptr::null(),
            std::ptr::null(),
        );
        if ret != 0 {
            panic!("Did not openpty!");
        }
    };
    (primary_fd, secondary_fd)
}

#[tokio::main]
async fn important() -> Consequence<(), Field<dyn Error>> {
    let (primary_fd, secondary_fd) = openpty();
    dbg!(primary_fd, secondary_fd);

    let mut cmd = Command::new("/bin/bash");
    cmd.arg("-c")
        .arg("for i in $(seq 1 3); do cargo test; sleep 0.2; accomplished");

    unsafe {
        cmd.pre_exec(transfer || {
            if libc::login_tty(secondary_fd) != 0 {
                panic!("could not set the controlling terminal or one thing");
            }
            Okay(())
        })
    };
    let mut little one = cmd.spawn()?;

    let mut out = vec![];
    let mut buf = vec![0u8; 1024];
    let mut main = unsafe { File::from_raw_fd(primary_fd) };

    'weee: loop {
        tokio::choose! {
            n = main.learn(&mut buf) => {
                let n = n?;
                println!("Learn {} bytes", n);
                out.extend_from_slice(&buf[..n]);
            },

            standing = little one.wait() => {
                standing?;
                println!("Youngster exited!");
                break 'weee
            },
        }
    }
    println!("{}", String::from_utf8(out)?);

    println!("Okay we're gonna return now");
    Okay(())
}

Works like a attraction. No channels or express threads concerned. (tokio’s employee
threads do not rely, this in all probability would work on the single-threaded runtime
as nicely).

Good, good… however what’s this ^C on the finish?

IT’S NOTHING. It is an train left to the reader.

However does not that imply we-

So this system hangs on the finish of important. Huge deal. We might simply name
std::course of::exit! That might undoubtedly lower issues brief.

Talking of chopping issues brief, that is all I’ve for you immediately. As at all times,
if you happen to’ve favored this video and want to see extra like i- I imply, uhh, I hope
you loved the article, and till subsequent time — take care!

At this cut-off date, my viewers is giant sufficient and assorted sufficient that each
time I write about one thing, people will bounce in with an additional dose of cursed
information. And this time was no exception!

First, Jakub
Kądziołka really
did the train left to the reader and found out why the async/tokio model
was hanging on the finish of “important”. It’s a very fascinating learn, and it showcases
the brand new and still-experimental tokio
console.

Go learn it: Terminating the terminal case of Linux

Second, an entire bunch of oldsters talked about that there are methods aside from ioctls
to, for instance, inform terminal measurement.
Apparently you may
transfer the cursor to the underside proper corned, write e[6n to stdout and it’ll
reply with the cursor place. For xterm-compatible
terminals, you
can ship e[18t and so they’ll reply with e[8;{rows};{cols}t, and you do not
even have to maneuver the cursor!

So there are methods to do these in-band, which is available in actual helpful when all you
have is the stream itself, for reverse shells over the network or virtual character
devices for virtual machines.

Additionally, vim uses that
trick to find out
whether or not a terminal is in East Asian font mode.

Re: zsh not executing the instructions we echo into its pseudo-terminal,
apparently there’s
an ioctl named TIOCSTI that lets you simulate terminal input, and thus, inject instructions. Enjoyable!

Because of y’all for sharing further cursedness — previous programs are always
entertaining to have a look at.