Unix Area Sockets vs Loopback TCP Sockets

Two speaking processes on a single machine have just a few choices. They’ll use common TCP sockets, UDP sockets, unix area sockets, or shared reminiscence. A current mission I used to be engaged on used Node.js with two speaking processes on the identical machine. I needed to know easy methods to cut back the CPU utilization of the machine, so I ran just a few experiments to match the effectivity between unix area sockets and TCP sockets utilizing the loopback interface. This put up covers my experiments and check outcomes.

First off, is a disclaimer. This check is just not exhaustive. Each shopper and server are written in Node.js and may solely be as environment friendly because the Node.js runtime.

All code on this put up is on the market at: github.com/nicmcd/uds_vs_tcp

Server Software

I created a easy Node.js server utility that may very well be linked to through TCP socket or Unix area socket. It merely echos all acquired messages. Right here is the code:

var assert = require('assert');
assert(course of.argv.size == 4, 'node server.js <tcp port> <area socket path>');

var web = require('web');

var tcpPort = parseInt(course of.argv[2]);
assert(!isNaN(tcpPort), 'unhealthy TCP port');
console.log('TCP port: ' + tcpPort);

var udsPath = course of.argv[3];
console.log('UDS path: ' + udsPath);

perform createServer(title, portPath) {
    var server = web.createServer(perform(socket) {
        console.log(title + ' server linked');
        socket.on('finish', perform() {
            console.log(title + ' server disconnected');
        });
        socket.write('begin sending now!');
        socket.pipe(socket);
    });
    server.hear(portPath, perform() {
        console.log(title + ' server listening on ' + portPath);
    });
}

var tcpServer = createServer('TCP', tcpPort);
var udsServer = createServer('UDS', udsPath);

Consumer Software

The shopper utility enhances the server utility. It connects to the server through TCP or Unix area sockets. It sends a bunch of randomly generated packets and measures the time it takes to complete. When full, it prints the time and exits. Right here is the code:

var assert = require('assert');
assert(course of.argv.size == 5, 'node shopper.js <port or path> <packet dimension> <packet depend>');

var web = require('web');
var crypto = require('crypto');

if (isNaN(parseInt(course of.argv[2])) == false)
    var choices = {port: parseInt(course of.argv[2])};
else
    var choices = {path: course of.argv[2]};
console.log('choices: ' + JSON.stringify(choices));

var packetSize = parseInt(course of.argv[3]);
assert(!isNaN(packetSize), 'unhealthy packet dimension');
console.log('packet dimension: ' + packetSize);

var packetCount = parseInt(course of.argv[4]);
assert(!isNaN(packetCount), 'unhealthy packet depend');
console.log('packet depend: ' + packetCount);

var shopper = web.join(choices, perform() {
    console.log('shopper linked');
});

var printedFirst = false;
var packet = crypto.randomBytes(packetSize).toString('base64').substring(0,packetSize);
var currPacketCount = 0;
var startTime;
var endTime;
var delta;
shopper.on('knowledge', perform(knowledge) {
    if (printedFirst == false) {
        console.log('shopper acquired: ' + knowledge);
        printedFirst = true;
    }
    else {
        currPacketCount += 1;
        if (knowledge.size != packetSize)
            console.log('bizarre packet dimension: ' + knowledge.size);
        //console.log('shopper acquired a packet: ' + currPacketCount);
    }

    if (currPacketCount < packetCount) {
        if (currPacketCount == 0) {
            startTime = course of.hrtime();
        }
        shopper.write(packet);
    } else {
        shopper.finish();
        endTime = course of.hrtime(startTime);
        delta = (endTime[0] * 1e9 + endTime[1]) / 1e6;
        console.log('millis: ' + delta);
    }
});

Operating a Single Take a look at

First begin the server utility with:

node server.js 5555 /tmp/uds

This begins the server utilizing TCP port 5555 and Unix area socket /tmp/uds.

Now we will run the shopper utility to get some statistics. Let’s first attempt the TCP socket. Run the shopper with:

node shopper.js 5555 1000 100000

This runs the shopper utility utilizing TCP port 5555 and sends 100,000 packets all sized 1000 bytes. This tooks 8006 milliseconds on my machine. We are able to now attempt working with the Unix area socket with:

node shopper.js /tmp/uds 1000 100000

This runs the shopper the identical as earlier than besides it makes use of the /tmp/uds Unix area socket as a substitute of the TCP socket. On my machine this took 3570 milliseconds to run. These two runs present that for 1k byte packets, Unix area sockets are about 2-3x extra environment friendly than TCP sockets.
At this level you could be utterly satisfied that Unix area sockets are higher and also you’ll use them every time you possibly can. That’s too simple. Let’s run the shopper utility an entire bunch of instances and graph the outcomes.
I just lately posted a couple of python package I created for working many duties and aggregating the info. I believed this socket comparability would make instance.

Operating the Full Take a look at

As talked about, working the complete check makes use of the Taskrun Python package deal (accessible at github.com/nicmcd/taskrun). The script I rapidly hacked collectively to run the shopper utility and parse the outcomes is as follows:

import taskrun
import os

POWER = 15
RUNS = 10
PACKETS_PER_RUN = 100000

supervisor = taskrun.Job.Supervisor(
    numProcs = 1,
    showCommands = True,
    runTasks = True,
    showProgress = True)

DIR = "sims"
mkdir = supervisor.task_new('dir', 'rm -rI ' + DIR + '; mkdir ' + DIR)

def makeName(stype, dimension, run):
    return stype + '_size' + str(dimension) + '_run' + str(run)

def makeCommand(port_or_path, dimension, title):
    return 'node shopper.js ' + port_or_path + ' ' + str(dimension) + ' ' + str(PACKETS_PER_RUN) + 
        ' | grep millis | awk '{printf "%s, ", $2}' > ' + os.path.be a part of(DIR, title)

barrier1 = supervisor.task_new('barrier1', 'sleep 0')
for exp in vary(0, POWER):
    dimension = pow(2, exp)
    for run in vary(0, RUNS):
        # Unix area socket check
        title = makeName('uds', dimension, run)
        process = supervisor.task_new(title, makeCommand('/tmp/uds', dimension, title))
        process.dependency_is(mkdir)
        barrier1.dependency_is(process)

        # TCP socket check
        title = makeName('tcp', dimension, run)
        process = supervisor.task_new(title, makeCommand('5555', dimension, title))
        process.dependency_is(mkdir)
        barrier1.dependency_is(process)

# create CSV header
filename = os.path.be a part of(DIR, 'uds_vs_tcp.csv')
header="NAME, "
for run in vary(0, RUNS):
    header += 'RUN ' + str(run) + ', '
hdr_task = supervisor.task_new('CSV header', 'echo '' + header + '' > ' + filename)
hdr_task.dependency_is(barrier1)

# UDS to CSV
cmd = ''
for exp in vary(0,POWER):
    dimension = pow(2, exp)
    cmd += 'echo -n 'UDS Measurement ' + str(dimension) + ', ' >> ' + filename + '; '
    for run in vary(0, RUNS):
        title = makeName('uds', dimension, run)
        cmd += 'cat ' + os.path.be a part of(DIR, title) + ' >> ' + filename + '; '
    cmd += 'echo '' >> ' + filename + '; '
uds_task = supervisor.task_new('UDS to CSV', cmd)
uds_task.dependency_is(hdr_task)

# TCP to CSV
cmd = ''
for exp in vary(0,POWER):
    dimension = pow(2, exp)
    cmd += 'echo -n 'TCP Measurement ' + str(dimension) + ', ' >> ' + filename + '; '
    for run in vary(0, RUNS):
        title = makeName('tcp', dimension, run)
        cmd += 'cat ' + os.path.be a part of(DIR, title) + ' >> ' + filename + '; '
    cmd += 'echo '' >> ' + filename + '; '
tcp_task = supervisor.task_new('TCP to CSV', cmd)
tcp_task.dependency_is(uds_task)

supervisor.run_request_is()

Admittedly, this isn’t the prettiest code to take a look at, however it will get the job accomplished. For each Unix area socket and TCP socket, it runs the shopper utility for all packet sizes which might be an influence of two from 1 to 16384. Every setup is run 10 instances. Every check result’s written to its personal file. After all of the exams have been run, the taskrun script creates a CSV file utilizing all of the check outcomes. The CSV file can then be imported right into a spreadsheet utility for evaluation.

Outcomes

I ran this on an Intel E5-2620 v2 processor with 16GB of RAM. I imported the CSV into Excel, averaged the ten outcomes of every setup, then graphed the outcomes. This primary graph exhibits the execution time in comparison with packet dimension on a logarithmic graph.

See Also

AOL Pretends to be the Web

The outcomes proven listed here are pretty predicable. The Unix area sockets are all the time extra environment friendly and the effectivity profit is within the 2-3x vary. After noticing some bizarre ups and down within the graph, I made a decision to generate a graph with the execution instances normalized to the TCP execution time.

I’m not precisely positive why the effectivity of Unix area sockets varies because it does in comparison with TCP sockets, however it’s all the time higher. That is just because Unix area sockets don’t traverse the working system’s community stack. The kernel merely copies the info from the shopper’s utility into the file buffer within the server’s utility.