Making a Linux residence server sleep on idle and wake on demand — the straightforward approach

It started with what appeared like a last mundane contact to my home server setup for hosting Time Machine backups: I wished it to mechanically sleep when idle and get up once more when wanted. You realize, sleep on idle — hasn’t Home windows had that in-built since like Home windows 98? How onerous might or not it’s to configure on a contemporary Ubuntu set up?

To be honest, I wished extra than simply sleep on idle, I additionally wished wake on request — and that second bit seems to be the onerous half. There have been a bunch of lifeless ends, however I caught out it to search out one thing that “simply works” with out the necessity to manually activate the server for each backup. Be part of me on the total journey additional down, or reduce to the chase with the setup directions under.

End result:

• Server mechanically suspends to RAM when idle
• Server mechanically wakes when wanted by something else on the community, together with SSH, Time Machine backups, and so forth.

You may want:

• An always-on Linux machine on the identical community as your server, e.g. a Raspberry Pi
• A community interface machine to your server that helps wake-on-LAN with unicast packets

On the server:

• Allow wake-on-LAN with unicast packets (not simply magic packets), make it persistent

Copycopy code to clipboard
sudo ethtool -s eno1 wol ug
sudo tee /and so forth/networkd-dispatcher/configuring.d/wol << EOF
#!/usr/bin/env bash
ethtool -s eno1 wol ug || true
EOF
sudo chmod 755 /and so forth/networkd-dispatcher/configuring.d/wol

Copycopy code to clipboard
tee /residence/ubuntu/auto-sleep.sh << EOF
#!/bin/bash
logged_in_count=$(who | wc -l)
# We anticipate 2 traces of output from `lsof -i:548` at idle: one for output headers, one other for the 
# server listening for connections. Greater than 2 traces signifies inbound connection(s).
afp_connection_count=$(lsof -i:548 | wc -l)
if [[ $logged_in_count < 1 && $afp_connection_count < 3 ]]; then
  systemctl droop
else
  echo "Not suspending, logged in customers: $logged_in_count, connection depend: $afp_connection_count"
fi
EOF
chmod +x /residence/ubuntu/auto-sleep.sh
sudo crontab -e
*/10 * * * * /residence/ubuntu/auto-sleep.sh | logger -t autosuspend

Copycopy code to clipboard
sudo nano /and so forth/default/grub
sudo update-grub
sudo reboot

Copycopy code to clipboard
sudo tee /and so forth/systemd/system/netatalk-sleep.service << EOF
[Unit]
Description=Netatalk sleep hook
Earlier than=sleep.goal
StopWhenUnneeded=sure
[Service]
Kind=oneshot
RemainAfterExit=sure
ExecStart=-/usr/bin/systemctl cease netatalk
ExecStop=-/usr/bin/systemctl begin netatalk
[Install]
WantedBy=sleep.goal
EOF
sudo systemctl daemon-reload
sudo systemctl allow netatalk-sleep.service

• Set up ARP Stand-in: an excellent easy Ruby script that runs as a system service and responds to ARP requests on behalf of one other machine. Configure it to reply on behalf of the sleeping server.
• Non-compulsory: Configure Avahi to promote community companies on behalf of the server when it is sleeping.

Copycopy code to clipboard
sudo apt set up avahi-daemon
sudo tee /and so forth/systemd/system/avahi-publish.service << EOF
[Unit]
Description=Publish customized Avahi data
After=community.goal avahi-daemon.service
Requires=avahi-daemon.service
[Service]
ExecStart=/usr/bin/avahi-publish -s homeserver _afpovertcp._tcp 548 -H homeserver.native
[Install]
WantedBy=multi-user.goal
EOF
sudo systemctl daemon-reload
sudo systemctl allow avahi-publish.service --now
systemctl standing avahi-publish.service

Many of the guides I discovered for sleep-on-idle, like this one, have been for Ubuntu Desktop — sleep-on-idle does not appear to be one thing that is generally finished with Ubuntu Server. I got here throughout a number of instruments that seemed promising, probably the most notable being circadian. On the whole, although, there did not appear to be a typical/best-practice strategy to do it, so I made a decision I might roll it myself the only approach I might.

Figuring out idle/busy state

I requested myself what server exercise would represent being busy, and landed on two issues:

• Logged in SSH classes
• In-progress Time Machine backups

Selecting corresponding metrics was fairly easy:

• Rely of logged in customers, utilizing who
• Rely of connections on the AFP port (548), utilizing lsof (I am utilizing AFP for Time Machine community shares)

For each metrics, I famous the values first at idle, after which once more when the server was busy.

Mechanically suspending to RAM

To maintain issues easy, I opted for a cron job that triggers a bash script — take a look at the final version shared above. Up to now it is labored effective; if I ever have to account for extra metrics in detecting idle state, I will think about using a extra refined choice like circadian.

Waking on request

With sleep-on-idle out of the best way, I moved on to determining how the server would wake on demand.

Might the machine be configured to mechanically wake upon receiving a community request? I knew Wake-on-LAN supported waking a pc up utilizing a specifically crafted “magic packet”, and it was easy to get this working. The query was if an everyday, non-“magic packet” might someway do the identical factor.

Wake on PHY?

Some on-line looking yielded a superuser discussion that seemed significantly promising. It pointed to the man page for ethtool, the Linux utility used to configure community {hardware}. It shared ethtool’s full wake-on-LAN configuration choices:

Copycopy code to clipboard
wol p|u|m|b|a|g|s|f|d...
      Units Wake-on-LAN choices.  Not all units assist
      this.  The argument to this selection is a string of
      characters specifying which choices to allow.
      p   Wake on PHY exercise
      u   Wake on unicast messages
      m   Wake on multicast messages
      b   Wake on broadcast messages
      a   Wake on ARP
      g   Wake on MagicPacket™
      s   Allow SecureOn™ password for MagicPacket™
      f   Wake on filter(s)
      d   Disable (wake on nothing).  This feature
          clears all earlier choices.

It pointed specifically to the Wake on PHY exercise choice, which appeared excellent for this use-case. It appeared to imply that any packet despatched to the community interface’s MAC tackle would wake it. I enabled the flag utilizing ethtool, manually put the machine to sleep, then tried logging again in utilizing SSH and sending pings. No cube: the machine remained asleep regardless of a number of makes an attempt. A lot for that ????

Breakthrough: wake on unicast

None of ethtool‘s different wake-on-LAN choices appeared related, however some extra looking pointed to the Wake on unicast messages as one other choice to strive. I enabled the flag utilizing ethtool, manually put the machine to sleep, then tried logging again in utilizing SSH. Bingo! This time, the machine wakened. ???? With that, I figured I used to be finished.

Not so quick — there have been two issues:

1. Typically, the server would get up with none community exercise that I knew of
2. Some time period after the server went to sleep, it will change into unattainable to wake it once more utilizing community exercise aside from a magic packet

A more in-depth have a look at the identical superuser discussion above revealed precisely the explanation for the second downside: shortly after going to sleep, the machine was successfully disappearing from the community as a result of it was not responding to ARP requests.

ARP

So the cached ARP entry for different machines on the community was expiring, which means that they’d no strategy to resolve the server’s IP tackle to its MAC tackle. In different phrases, an try to ping my server at 192.168.1.2 was failing to even ship a packet to the server, as a result of the server’s MAC tackle wasn’t identified. And not using a packet being despatched, there was no approach that server was going to get up.

Static ARP?

My first response: let’s manually create ARP cache entries on every community shopper. That is certainly potential on macOS utilizing:

Copycopy code to clipboard
sudo arp -s [IP address] [MAC address]

Nevertheless it additionally did not meet the aim of getting issues “simply work”: I used to be not keen on creating static ARP cache entries on every machine that may be accessing the server. On to different choices.

ARP protocol offload?

Some extra looking revealed one thing fascinating: this downside had already been solved way back within the Home windows world.

It was referred to as ARP protocol offload, and it goes like this:

• The community {hardware} is able to responding to ARP requests independently of the CPU
• Earlier than going to sleep, the OS configures the community {hardware} to answer ARP requests
• Whereas sleeping, the community {hardware} responds to ARP requests by itself, with out waking the remainder of the machine to make use of the CPU

Voila, this was precisely what I wanted. I even seemed on the datasheet for my network hardware, which lists ARP Offload as a function on the entrance web page.

The one downside? No Linux assist. I searched the far reaches of the web, then lastly dug into the Linux driver source code to search out that ARP offload is not supported by the Linux driver. This was after I briefly contemplated making an attempt to patch the motive force so as to add ARP offload… earlier than reminding myself that efficiently patching Linux driver code is much past what I might hope to realize in a bit free-time challenge like this one. (Although possibly someday…)

Different options utilizing magic packets

Some extra looking led me to another intelligent and elaborate options involving magic packets. The essential thought was to automate sending magic packets. One answer (wake-on-arp) listens for ARP requests to a specified host to set off sending a magic packet to that host. Another solution implements an online interface and Dwelling Assistant integration to allow triggering a magic packet from a smartphone internet browser. These are spectacular, however I wished one thing easier that did not require manually waking up the server.

I thought-about a number of different choices, however deserted them as a result of they felt too complicated and susceptible to breaking:

• Writing a script to ship a magic packet after which instantly set off a Time Machine backup utilizing tmutil. The script would have to be manually put in and scheduled to run periodically on every Mac.
• Utilizing HAProxy to proxy all related community visitors by way of the Raspberry Pi and utilizing a hook to ship a magic packet to the server on exercise.

Breakthrough: ARP Stand-in

What I used to be trying did not appear a lot totally different from the static IP mapping that is routinely configured on residence routers, besides that it was for DHCP as a substitute of ARP. Was there no strategy to make my router do the identical factor for ARP?

Some extra digging into the ARP protocol revealed that ARP decision does not even require a selected, authoritative host to reply requests — some other community machine can reply to ARP requests. In different phrases, my router did not have to be the one resolving ARP requests, it might be something. Now how might I simply arrange one thing to reply on behalf of the sleeping server?

This is what I used to be making an attempt to do:

I believed it have to be potential to implement as a Linux community configuration, however the closest factor I discovered was Proxy ARP, which achieved a special aim. So I went one stage deeper, to community programming.

Now, find out how to go about listening for ARP request packets? That is apparently possible to do using a raw socket, however I additionally knew that tcpdump and Wireshark have been able to utilizing filters to seize solely packets of a given sort. That led me to look into libpcap, the library that powers each of these instruments. I discovered that utilizing libpcap had a transparent benefit over a uncooked socket: libpcap implements very efficient filtering directly in the kernel, whereas a uncooked socket would require handbook packet filtering in consumer house, which is much less performant.

Aiming to maintain issues easy, I made a decision to strive writing the answer in Ruby, which led me to the pcaprub Ruby bindings for libpcap. From there, I simply wanted to determine what filter to make use of with libpcap. Some analysis and trial/error yielded this filter:

Copycopy code to clipboard
arp and arp[6:2] == 1 and arp[24:4] == [IP address converted to hex]

For instance, utilizing a goal IP tackle of 192.168.1.2:

Copycopy code to clipboard
arp and arp[6:2] == 1 and arp[24:4] == 0xc0a80102

Let’s break this down, utilizing the ARP packet structure definition for byte offets and lengths:

• arp — ARP packets
• arp[6:2] == 1 — ARP request packets. [6:2] means “the two bytes discovered at byte offset 6”.
• arp[24:4] == [IP address converted to hex] — ARP packets with the specified goal tackle. [24:4] means “the 4 bytes discovered at byte offset 24”.

The remaining is fairly easy and the whole solution comes out to solely ~50 traces of Ruby code. In brief, arp_standin is a daemon that does the next:

• Begins up, taking these configuration choices:
  • IP and MAC tackle of the machine it is standing in for (the “goal”)
  • Community interface to function on
• Listens for ARP requests for the goal’s IP tackle
• On detecting an ARP request for the goal’s IP tackle, responds with the goal’s MAC tackle

Because the server’s IP → MAC tackle mapping is outlined statically by way of the arp_standin daemon’s configuration, it does not matter if the Raspberry Pi’s ARP cache entry for the server is expired.

Try the hyperlink under to put in it or discover the supply code additional:

arp_standin repository on GitHub

ARP is utilized in IPv4 and is changed by Neighbor Discovery Protocol (NDP) in IPv6. I haven’t got any want for IPv6 proper now, so I disabled IPv6 completely on the server utilizing the steps shown above. It ought to be potential so as to add assist for Neighbor Discovery to the ARP-Standin service as a future enhancement.

With the brand new service operating on my Raspberry Pi, I used Wireshark to substantiate that ARP requests being despatched to the server have been triggering responses from the ARP Stand-in. It labored ???? — issues have been trying promising.

Getting all of it working

The massive items have been in place:

• the server went to sleep after changing into idle
• the server might get up from unicast packets
• different machines might resolve the server’s MAC tackle utilizing ARP, lengthy after it went to sleep

With the ARP Stand-in operating, I turned on the server and ran a backup from my pc. When the backup was completed, the server went to sleep mechanically. However there was an issue: the server was waking up instantly after going to sleep.

Undesirable wake-ups

Very first thing I checked was the Linux system logs, however these did not show too useful, since they did not present what community packet truly triggered the wakeup. Wireshark/tcpdump have been no assist right here both, as a result of they would not be operating when the pc was sleeping. That is after I thought to make use of port mirroring: capturing packets from an middleman machine between the server and the remainder of the community. After a short, unsuccessful try to repurpose an additional router operating OpenWRT, a seek for the least costly community swap with port mirroring assist yielded the TP-Link TL-SG105E for ~$30.

With the swap linked and port mirroring enabled, I began capturing with Wireshark and the culprits instantly turned clear:

1. My Mac, which was configured to make use of the server as a Time Machine backup host utilizing AFP, was sending AFP packets to the server after it had gone to sleep
2. My Netgear R7000, appearing as a wi-fi entry level, was sending frequent, unsolicited NetBIOS NBTSTAT queries to the server

Eliminating AFP packets

I had a hunch about why the Mac was sending these packets:

• The Mac mounted the AFP share to carry out a Time Machine backup
• The Time Machine backup completed, however the share remained mounted
• The Mac was checking on the standing of the share periodically, as could be finished usually for a mounted community share

I additionally had a corresponding hunch that the answer could be to verify the share acquired unmounted earlier than the server went to sleep, in order that the Mac would not ping the server for its standing afterwards. I figured that shutting down the AFP service would set off unmounting of shares on all its shoppers, attaining the aim. Now I simply wanted to make sure the service would shut down when the server was going to sleep, then begin once more when it woke again up.

Fortuitously, systemd helps precisely that, and comparatively simply — I outlined a devoted systemd service to hook into sleep/wake occasions (check out the configuration shared above). A Wireshark seize confirmed that it did the trick.

Eliminating NetBIOS packets

This one proved to be tougher, as a result of the packets have been unsolicited — they appeared random and unrelated to any exercise being finished by the server. I believed they is likely to be associated to Samba companies operating on the server, however the packets persevered even after I fully eliminated Samba from the server.

Why was my community router sending NetBIOS requests, anyway? Seems that Netgear routers have a function referred to as ReadySHARE for sharing USB units over the community utilizing the SMB protocol. Presumably, the router firmware makes use of Samba behind the scenes, which makes use of NetBIOS queries to construct and preserve its personal illustration of NetBIOS hosts on the community. Simple — flip off ReadySHARE, proper? Nope, there isn’t any approach to do this in Netgear’s inventory firmware ????.

That led me to make the leap and flash the router with open-source FreshTomato firmware. I am glad I did, as a result of the firmware is a lot better than the inventory one anyway, and it instantly stopped the undesirable NetBIOS packets.

Time Machine not triggering wake-up

I used to be getting shut now: the server remained asleep, and I might reliably wake it up by logging in with SSH, even lengthy after it went to sleep.

This was nice, however one factor wasn’t working: when beginning a backup on my Mac, Time Machine would present a loading state indefinitely with Connecting to backup disk... and ultimately quit. Was the server failing to get up from packets the Mac was sending, or was the Mac not sending packets in any respect?

A port-mirrored Wireshark seize answered that query: the Mac wasn’t sending any packets to the server, even lengthy after it began to say Connecting to backup disk.... Digging into the macOS Time Machine logs with:

Copycopy code to clipboard
log present --style syslog --predicate 'senderImagePath comprises[cd] "TimeMachine"' --info

A number of entries made it clear:

Copycopy code to clipboard
(TimeMachine) [com.apple.TimeMachine:Mounting] Trying to mount 'afp://[email protected]_afpovertcp._tcp.native./tm_mbp'
...
(TimeMachine) [com.apple.TimeMachine:General] Didn't resolve CFNetServiceRef with identify = homeserver sort = _afpovertcp._tcp. area = native.

The Mac was utilizing mDNS (a.k.a. Bonjour, Zeroconf) to resolve the backup server’s IP tackle utilizing its hostname. The server was asleep and due to this fact not responding to the requests, so the Mac was failing to resolve its IP tackle. This defined why the Mac wasn’t sending any packets to the server, leaving it asleep.

mDNS stand-in

I already had an ARP stand-in service, now I wanted my Raspberry Pi to additionally reply to mDNS queries for the server whereas it slept. I knew that Avahi was one of many most important mDNS implementations for Linux. I first tried these instructions utilizing .service information to configure my Raspberry Pi to answer mDNS queries on behalf of the server. I used the next on the Mac to test the outcome:

Copycopy code to clipboard
dns-sd -L homeserver _afpovertcp._tcp native

For some motive, that method simply did not work; Avahi did not reply on behalf of the server. I experimented as a substitute with avahi-publish (man page), which (to my nice shock) labored straight away utilizing the next:

Copycopy code to clipboard
avahi-publish -s homeserver _afpovertcp._tcp 548 -H homeserver.native

With that, I simply wanted to create a systemd service definition that may mechanically run the avahi-publish command on boot (check out the configuration shared above).

???? End

With all of the wrinkles ironed out, every part has been working effectively now for over a month. I hope you have loved following alongside and that this method works for you too.