How a kernel developer made my styluses work once more on newer kernels!

2023-11-12 17:26:31

🎉 🎉 🎉 Whooohooo! It really works once more!

Each styluses of my XPPen Artist 24 Professional and XPPen Artist 16 Professional Gen2 styluses are actually usable once more on newer Linux kernels.

This blog-post is a follow-up submit of “How a kernel update broke my stylus… Need help!” printed 10 days in the past. Please learn it if you wish to know extra about the issue I had with the stylus.

After this primary weblog submit (and because of your feedback and steerage, particularly efi@chitter.xyz) I used to be in a position to email my bug to consultants within the space.

Then Jiri Kosina republished my electronic mail to the mailing-list. A giant due to them, to Illia Ostapyshyn for the dialogue, and to Benjamin Tissoires for creating an answer.

In case you have the identical subject with the same machine, you will must compile:
https://gitlab.freedesktop.org/libevdev/udev-hid-bpf/

This answer continues to be W.I.P. and I nonetheless have some homework to ship extra information about my tablets after this weblog submit, however in total I am already utilizing a more recent kernel (Linux workstation 6.5.10-200.fc38.x86_64) and I haven’t got the issue with the eraser mode on the highest button of my XPPen Artist 24 Professional and XPPen Artist 16 Professional Gen2 styluses. The buttons are additionally now completely customisable through xsetwacom CLI device. Yay! That is why I wished to share this blog-post as quickly as doable.

On the mailing listing Benjamin wrote me an in depth reply about the entire story. It’s extremely fascinating and I made a decision to repeat and paste it right here. Thanks once more Benjamin! 👍

Benjamin’s detailed reply:

(Authentic electronic mail here.)

Hello David,

Here’s a little little bit of historical past of why you have been encountering this bug:

First, HID is a fairly outdated protocol and has the good thing about being “plug
and play” [0] [1].

However plug and play typically means for a {hardware} maker: “let’s do trial
and error till Home windows appears to behave in an accurate manner”.

In another circumstances, Microsoft places extra restrictions on the HID half
(Home windows 7 enforced touchscreens to comply with a particular sample, and
then Home windows 8 did it for touchpads). They usually additionally typically present
a take a look at suite that {hardware} makers must go to be “licensed”.
They must go the take a look at suite through the use of the Home windows supplied generic
driver, however Home windows additionally permits them to create a digital HID machine
and let a customized driver create that digital HID machine. Which suggests,
we typically have units behaving badly however working completely advantageous on
Home windows as a result of there are bits lacking within the machine itself which are
fastened by including an additional software program layer. Sigh.

However I digress and we have to return to the pen bits, and AFAIK, there
is not any such take a look at suite and certification.

So mainly, {hardware} makers comply with the empiric rule of “if Home windows
is pleased, I’m too”.

To take action, they’ve to make use of a number of occasions from HID [2] (quoting them):

Tip Change → A change situated on the tip of a stylus, indicating
contact of the stylus with a floor. A pen-based system or system
extension would use this change to allow the enter of handwriting or
gesture information. The system sometimes maps Tip Change to a main button
in a non-pen context.
In Vary → Signifies {that a} transducer is situated inside the
area the place digitizing is feasible. In Vary is a bit amount
Barrel Change → A non-tip button situated on the barrel of a
stylus. Its operate is usually mapped to a system secondary button
or to a Shift key modifier that adjustments the Tip Change operate from
main button to secondary button.
Secondary Barrel Change → A second non-tip button situated on the
barrel of a stylus farther from the tip than the Barrel Change. Its
operate is usually mapped to a system secondary or tertiary button.
Eraser → This management is used for erasing objects. Following the
metaphor of a pencil, it’s sometimes situated reverse the writing finish
of a stylus. It could be a bit change or a stress amount.
Invert → A bit that signifies that the at present sensed place
originates from the tip of a stylus reverse the tip.

I am certain that by studying these, all people ought to be capable to
instantly know learn how to write a Pen HID machine, and the way the
interactions between the occasions needs to be. 🙂 (If you’re, please
contact me ASAP, we’ve loads of kernel work to do).

So for years the state of pen units within the Linux kernel was 2 fold:

Wacom shipped an in-kernel driver for their very own units, that they
examined and that outlined the de-facto “customary” in Linux
the remainder was attempting to catch up by luck or with the assistance of tasks
like DiGiMend, by counting on the generic HID processing of the Linux
kernel

Nevertheless, they have been no specification for a way the occasions ought to come:
mainly within the hid generic enter processing every occasion was mapped to
a single enter keycode and we had conditions have been we’d get each
BTN_TOOL_PEN and BTN_TOOL_ERASER on the similar time (as a result of the In Vary and the Eraser bits have been despatched by the machine on the similar
time). Which suggests “hey, the person is interacting with a pen with each
the tail and the tip on the similar time. Good luck with that!”

This led to a fancy state of affairs the place userspace (libinput largely) had
to do guesses on what’s the intent of the person. However the issue is
that if you find yourself in userspace, you do not know the entire occasions of
the machine on the similar time, so it was messy to cope with. Once more,
Wacom units have been unaffected as a result of they managed the entire
stack: a kernel driver to repair/interpret their units and a userspace
element, xf86-drv-wacom, within the X.org world.

As soon as, as you talked about in your weblog, Microsoft determined to make use of the
second barrel button because the “rubber” mode. The rationale was sensible:
folks appreciated the rubber functionality on the styluses, however they wished to
have a separate button on the tail finish of the styluses. And I suppose
that on the time, on condition that no different {hardware} distributors have been able to
having no-battery styluses however Wacom (IP safety and capabilities
of the {hardware} IIRC), you continue to needed to put the battery someplace. And
that tail finish is handy for storing such a battery. However that makes
it tougher to have an eraser finish as a result of you’ll want to hyperlink each ends of
the pen on the identical IC, with a battery within the center that’s roughly
the identical measurement as your pen’s barrel. So having simply 2 wires for the
battery lets you have a separate bluetooth button on one finish, and
the conventional stylus on the opposite finish, and preserve the width of the pen
cheap.

In order that selection of utilizing the second button as an eraser was made, and
the {hardware} makers adopted: on the XP-Pen Artist Professional 24, the machine
stories Tip Change, Barrel Change, Eraser, In Vary.
Which is “right” based on the HID Utilization Desk [2], however which
does not adhere to the advice Microsoft is doing [3]: the
machine ought to report an additional Invert when the pen is in vary with
the intent to erase…

However you’ll be able to see that XP-Pen tried to stick to it in some methods as a result of
should you look fastidiously on the occasions coming from the machine with
hid-recorder [4], you will discover that if you find yourself in vary of the
sensor and press this button, you will get an additional “In Vary = 0” occasion
to inform that you simply went out of proximity of the sensor.

In kernel 5.18, with commit 87562fcd1342 (“HID: enter: take away the necessity
for HID_QUIRK_INVERT”), I attempted to take away these a number of device states
to have a simple state supplied by the kernel that userspace
can cope with simply. Nevertheless, given that there have been no regression
checks on the time for generic tablets, I wrote some primarily based on
Microsoft’s suggestion [3] and in addition examined on a Huion machine I
have regionally. And it was working advantageous. However I did not have the units
that weren’t sending Invert, which defined why it was bogus on
these units.

This was “fastened” in kernel 6.6 with commit 276e14e6c399 (“HID: enter:
Assist units sending Eraser with out Invert”). Placing quotes round
“fastened” as a result of I am nonetheless not pleased about this patch.

However the level is, from kernel 5.18, the Pen processing within the kernel
grew to become a state machine, which implies we cannot have exterior actors
tampering with it.

Why utilizing the ioctl EVIOCSKEYCODE is unhealthy to remap Eraser to
BTN_STYLUS2 (by means of instruments like evmap):

Being able to do one thing doesn’t suggest it is the correct factor
to do. And in that case, that is undoubtedly fallacious as a result of it’s important to
name the ioctl after the kernel presents the machine to userspace.
Which suggests userspace (and the kernel) already made assumptions on the
machine itself. There’s a excessive likelihood libinput (or the Wacom driver)
opens the machine earlier than evmap, and that it’s contemplating that the
machine does not have BTN_STYLUS2. So sending that occasion would break
userspace.

And in our case right here, the kernel expects some state between the enter
layer and its inner HID state, and remapping that HID occasion to
one thing else confuses it.

There’s one other facet impact of this: normally finish customers configuring
their units with such instruments don’t report again their configuration
to the kernel neighborhood. In some circumstances that is legitimate (that is my
choice and my selection), however in different circumstances it is not (there’s a bug
right here and I am papering over it).

So, what might be accomplished?

Principally 2 choices:

write a kernel patch to repair that drawback as soon as and for all
use the model new HID-BPF[5] functionality launched in kernel v6.3
and ship me again the BPF program so I can ultimately combine the
supply within the kernel tree itself and repair that drawback as soon as and for all
as properly

For 1., you want:

to have the ability to dig into the kernel code
to have the ability to write a patch with the proper kernel customary (with a
regression take a look at in instruments/testing/selftests/hid, please)
to have the ability to compile your personal kernel and take a look at it
to have the ability to submit your contribution by electronic mail (I can recommend utilizing
b4 for that, very good device)
to have the ability to take critiques into consideration, and study git rebase -i
to submit v2, v3, and doubtlessly v10 or extra in some advanced circumstances
to attend for the patch to be built-in into Linus’ tree
to attend for Greg to backport your patch right into a steady kernel tree
to attend on your distribution to choose up the steady tree along with your patch

That is comparatively simple, no? 🙂

OTOTH, we’ve 2.: HID-BPF [5]

In a short time, eBPF [6] is a state machine contained in the kernel that
permits person house to incorporate a verified code path within the kernel to
tweak its conduct. And I tailored this for HID so you’ll be able to:

change the report descriptor of the machine: this
disconnects/reconnects the machine, that means the kernel works on the brand new
report descriptor and is according to its state
change the occasion circulate of the machine: to repair the spurious out-of-prox
occasion for instance
extra to return

What’s fascinating in BPF (or eBPF), is that these days, libbpf
implements one thing named CORE (Compile As soon as Run In every single place). Which
signifies that if I compile regionally an eBPF program on my machine with my
improvement kernel, so long as I solely use features out there from
kernel v6.3 as an illustration, the identical compilation output (that adjustments
the occasion circulate of your HID machine) will work on any kernel from v6.3
except there are some later API breakages[7].

Which suggests, anyone can modify the occasion circulate of an HID machine, put
the file within the filesystem, and have the machine nonetheless fastened even when
they improve their kernel.

In the long term, I intend to incorporate these HID-BPF fixes within the kernel
tree to centralize them, but additionally to have the ability to routinely load
them from the kernel when these units seem.

Which suggests, for the reporter of such a bug you:

can now depend on another person to write down the code, compile it and
present the compilation consequence [10]
simply put that consequence within the filesystem to have the machine examined and stuck

Behind the scenes, that different educated particular person can take the heavy
activity of submitting the kernel patch for you, however on condition that the code
has been examined, it is manner simpler to do (and to ultimately re-test).

At present, the “let’s combine that bpf program within the kernel” isn’t
fully accomplished, so we use udev-hid-bpf[8][9] to offer it a soar begin.

And that is precisely what occurred in your case David. Which is why I am
so pleased (additionally as a result of I fastened the instruments from an creator I like and
already had the books at residence :-P):

You need your machine to be fastened now, however going by means of a daily
kernel patch means months earlier than it is fastened in your distribution.
However with HID-BPF, I fastened it now, and you’ll safely improve the
kernel, as a result of even when I do adjustments within the kernel, the HID-BPF repair
will nonetheless convert the machine into one thing legitimate from the HID level
of view, and it has a regression take a look at now. When your machine will likely be
fastened sooner or later within the kernel, there’s a excessive likelihood the probe
operate of the HID-BPF program will say that it is not the proper
machine, and so this system won’t even load and depend on the fastened
kernel solely. Transparently for you, with out you having to alter your
filesystem.

On my facet, what’s left to be accomplished:

First, I want to repair the tablets not sending the Invert utilization.
Commit 276e14e6c399 (“HID: enter: Assist units sending Eraser
with out Invert”) is IMO not adequate, and we’d as properly merely
say that if there isn’t a Invert utilization, we will convert the Eraser
utilization into Secondary Barrel Change
then I want to repair the XP-Pen Artist Professional 16 gen 2 from the kernel
too, by changing the Eraser utilization with Secondary Barrel Change.
Ideally I might simply dump the HID-BPF program within the kernel, however this
isn’t out there but, so I am going to most likely write a small kernel driver
utilizing the identical code path because the HID-BPF program.
then Peter and I want to write down a extra generic HID-BPF program to
convert “eraser mode buttons” into Secondary Barrel Change,
mainly unwinding what the {hardware} does. This may solely occur when
libinput will be capable to do the alternative transformation so we do not
regress. However we will depend on libwacom to inform us if this pen has a tail
finish eraser or not, after which have userspace select if they need the
button to be a button, or an eraser mode.