BATMAN and the LinuxKPI: Running Linux drivers on FreeBSD 🦇

Aymeric Wibo

A little bit about myself

  • CS student at UCLouvain, in Belgium 🇧🇪
  • Part-time at Bnewable (energy storage solutions startup) ⚡
  • Interested in computer graphics (and BSD).
  • GSoC 2023 student.

Dog (Bubbles)

🔭 What is the focus of this talk?

  • The LinuxKPI & the state of affairs w.r.t. porting Linux drivers to FreeBSD.
  • Will end with a case study on porting BATMAN to FreeBSD (my GSoC project).

🤔 What is the LinuxKPI?

  • Kinda just a bunch of C headers which map Linux kernel functions (the KPI) to FreeBSD ones.
  • Internal kernel functions, not syscalls, that's (part of) the Linuxulator's job.
  • Headers rooted in sys/compat/linuxkpi/common/include.

Example function: get_random_u32_below

In sys/compat/linuxkpi/common/include/linux/random.h:

static inline u32
get_random_u32_below(u32 ceil)
{

	return (arc4random_uniform(ceil));
}

get_random_u32_below is the Linux function, arc4random_uniform is FreeBSD's equivalent.

Let's port a simple Linux driver!!

Original Linux code:

#include <linux/module.h>

static int __init demo_init(void) {
	pr_info("Hello from this demo module!\n");
	return 0;
}

module_init(demo_init);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Aymeric Wibo");
MODULE_DESCRIPTION("Super simple linux driver");
MODULE_VERSION("0.0.0");

Let's port a simple Linux driver!

#include <linux/kernel.h>
#include <linux/module.h>

static int __init demo_init(void) {
	pr_info("Hello from this demo module!\n");
	return 0;
}

module_init(demo_init);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Aymeric Wibo");
MODULE_DESCRIPTION("Super simple linux driver");

#if defined(__linux__)
MODULE_VERSION("0.0.0");
#endif

#if defined(__FreeBSD__)
MODULE_VERSION(demo, 0);
MODULE_DEPEND(demo, linuxkpi, 1, 1, 1);
#endif

🛠️ Makefile...

.PATH: .

KMOD=	demo
SRCS=	demo.c

SRCS+=		${LINUXKPI_GENSRCS}
CFLAGS+=	${LINUXKPI_INCLUDES}

.include <bsd.kmod.mk>

🧪 Testing

$ kldload ./demo.ko
$ dmesg | tail -n 1
Hello from this demo module!

It works!

What can LinuxKPI do for us today?

  • More than print hello world!
  • Full network drivers (e.g. iwlwifi(4)).
  • Full graphics drivers (e.g. graphics/drm-kmod with i915, radeon, amd).

✨ Adding new stuff

Update to the demo module

#include <linux/kernel.h>

static int __init demo_init(void) {
	hypothetical_function("Hello from this demo module!\n");
	return 0;
}

😲 Doesn't build!

demo.c:4:2: error: call to undeclared function 'hypothetical_function'; ISO C99
 and later do not support implicit function declarations [-Wimplicit-function-d
eclaration]
    4 |         hypothetical_function("Hello from this demo module!\n");
      |         ^
1 error generated.
*** Error code 1

🌳 Add stub

In linux/kernel.h:

static inline void
hypothetical_function(const char *str)
{

	pr_debug("TODO: %s\n", __func__);
}

What does this do on Linux?

$ modprobe ./demo.ko
$ dmesg | tail -n 1
!eludom omed siht morf olleH

What does this do on FreeBSD?

$ kldload ./demo.ko
$ dmesg | tail -n 1
TODO: hypothetical_function

Does this matter?

Yes? Implement observed behaviour 👀

static inline void
hypothetical_function(const char *str)
{

	ssize_t const len = strlen(str);
	for (ssize_t i = len - 2; i >= 0; i++)
		pr_info("%c", str[i]);
	pr_info("\n");
}

⚠️ Can't be isomorphic to GPL code!

What about now?

$ kldload ./demo.ko
$ dmesg | tail -n 1
!eludom omed siht morf olleH

It works!

🦇 BATMAN: A case study

📚 A little bit of background...

What's a BATMAN?

  • Very annoying to Google.
  • Better Approach to Mobile Ad-hoc Networking.
  • Essentially OSPF/OLSR, but optimized for big wireless meshes (i.e. constantly changing topology/link quality, &c).

What is it used for?

  • Driving force is Freifunk.
  • Community wireless mesh networks.

Quick history of BATMAN 📖

  • Freifunk were feeling limited by OLSR - topology changes are expensive!
  • Initially, userspace batmand.
  • Then, batman-adv kernel module (BATMAN IV & V).
  • BMX6/7 → offshoot.

Kitti's hog-nosed bat

How does BATMAN work? 🦾

  • Only cares about local changes in topology.
  • Layer 2 routing protocol → everything above gets wrapped.

How does BATMAN work? 🦾

Example Tee Cee Pee packet:

Frame 1077: 90 bytes on wire (720 bits), 90 bytes captured (720 bits) on interface bridge1, id 0
Ethernet II, Src: NetApp_af:d2:91 (00:a0:98:af:d2:91), Dst: NetApp_fd:37:f5 (00:a0:98:fd:37:f5)
    Destination: NetApp_fd:37:f5 (00:a0:98:fd:37:f5)
    Source: NetApp_af:d2:91 (00:a0:98:af:d2:91)
    Type: Unknown (0x4305)
B.A.T.M.A.N. Unicast, Dst: NetApp_91:93:b7 (00:a0:98:91:93:b7)
    Packet Type: BATADV_UNICAST (64)
    Version: 15
    Time to Live: 50
    TT Version: 2
    Destination: NetApp_91:93:b7 (00:a0:98:91:93:b7)
Ethernet II, Src: 12:2f:75:58:d2:ad (12:2f:75:58:d2:ad), Dst: ea:3c:ec:a5:b8:f1 (ea:3c:ec:a5:b8:f1)
    Destination: ea:3c:ec:a5:b8:f1 (ea:3c:ec:a5:b8:f1)
    Source: 12:2f:75:58:d2:ad (12:2f:75:58:d2:ad)
    Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.0.0.3, Dst: 10.0.0.1
Transmission Control Protocol, Src Port: 420, Dst Port: 58684, Seq: 1, Ack: 7, Len: 0

How does BATMAN work? 🦾

  • Provided by kernel module batman-adv on Linux (batman_adv on FreeBSD).
  • So-called "hard" vs "soft" interfaces (e.g. vtnet0 vs batadv0).
  • Sends out "echolocation" packets (ELP) to discover neighbours (new in BATMAN V) (hard).
  • Sends out OGM(2) packets to discover best routes (soft).

How to set up a BATMAN network on FreeBSD? 😈

$ kldload batman_adv
$ ifconfig vtnet0 mtu 1532 up
$ ifconfig batadv create ra BATMAN_V
batadv0
$ ifconfig vtnet0 master batadv0
$ ifconfig batadv0 inet 10.0.0.1/24

MTU of 1532 to give room to BATMAN header.

How to set up a BATMAN network through the Linuxulator? 🐧

Exactly the same way as on Linux!

$ ip link set up mtu 1532 dev eth0
$ ip link add name bat0 type batadv
$ ip link set dev eth0 master bat0
$ ip link set up dev bat0
$ ip addr add 10.0.0.1/24 dev bat0

Quick demo of a BATMAN 🚀

What are the major things that had to be done to get batman-adv ported?

  • batadv cloner with ra parameter (only on Netlink, no ioctl).
  • No modification of other settings (yet, need Generic Netlink) - only defaults.
  • Add concept of setting the "master" interface to FreeBSD and FreeBSD's Netlink.
  • Thanks melifaro@!

What had to be done to get batman-adv ported: Linuxulator

  • Just the conversion from batX (Linux) to batadvX (FreeBSD) → ifname_bsd_to_linux_ifp.

What had to be done to get batman-adv ported: Changes to batman-adv

  • Goal: keep this source code as similar as possible to Linux!
  • Not always possible though 😢
  • Guard namespace collisions, e.g.:
#if defined(__FreeBSD__)
	soft_iface = linux_dev_get_by_index(net, ifindex);
#else
	soft_iface = dev_get_by_index(net, ifindex);
#endif

What had to be done to get batman-adv ported: Changes to batman-adv

  • Biggest modification to batman-adv needed by far is to add the cloner for the soft interface:
static struct if_clone_addreq_v2 batadv_ifc_addreq = {
	.version = 2, /* For netlink callbacks. */
	.flags = IFC_F_AUTOUNIT,
	.match_f = batadv_softif_ifc_match,
	.create_f = batadv_softif_ifc_create,
	.destroy_f = batadv_softif_ifc_destroy,
	.create_nl_f = batadv_softif_ifc_create_nl,
	.modify_nl_f = batadv_softif_ifc_modify_nl,
	.dump_nl_f = batadv_softif_ifc_dump_nl,
};

What had to be done to get batman-adv ported: Changes to batman-adv

  • On Linux, this is struct rtnl_link_ops batadv_link_ops, but would require deep changes to/integration with FreeBSD network code.
  • Sometimes supporting something with the LinuxKPI is not a worthwhile tradeoff!

What had to be done to get batman-adv ported: LinuxKPI (struct net_device)

  • Linux's equivalent to struct ifnet.
  • Made it the same as struct ifnet.
  • Fields in common aliased, other Linux stuff bolted on at the end.
  • Means struct net_device can be passed around as struct ifnet (and vice-versa).
struct net_device *
linuxkpi_alloc_netdev_ifp(size_t priv_len, u_char type,
    void(*setup_func)(struct net_device *))
{
	struct net_device *ndev;
	if_t ifp;

	ndev = malloc(sizeof(*ndev) + priv_len, M_NETDEV, M_NOWAIT);
	if (ndev == NULL)
		return (ndev);
	ifp = (if_t)ndev;

	linuxkpi_init_dummy_netdev(ndev);
	if_fill_domain(ifp, type, IF_NODOM);
	ndev->has_ifp = true;

	memset(ndev->drv_priv, 0, priv_len);
	setup_func(ndev);

	return (ndev);
}

What had to be done to get batman-adv ported: LinuxKPI (struct net_device)

  • linuxkpi_dev_queue_xmit (equivalent to ifp->if_output).
  • linuxkpi_netif_rx (equivalent to ifp->if_input).
  • The struct net_device is just cast to struct ifnet to get the ifp.

What had to be done to get batman-adv ported: LinuxKPI (struct sk_buff)

  • Linux's equivalent to mbuf.
  • Way more complex and has a ton of random functions (skb_*).
  • Not currently backed by mbuf - data is being copied back and forth (don't sue me 😢).
  • Big function is linuxkpi_skb_from_mbuf, the other way is easier.

Many more things!

That you'll have the pleasure of discovering though my upcoming reviews :)

🔮 The future?

  • batman_adv in ports!
  • Wi-Fi support!! (Only sys/net/if_ethersubr.c interfaces at the moment).
  • Backing sk_buff's with mbuf's.
  • Upstreaming LinuxKPI changes.
  • batctl?

🚨 Call to action 🚨

  • This was actually not too bad, even though I was intimidated by the LinuxKPI at first.
  • If you have a Linux driver you want to port, it could be worth trying!
  • Important for FreeBSD's continued relevancy in certain domains.

That's all folks 🐰

Questions?

Reach out to me 📫

Louvain-li-Nux? Or will I get lynched for that.

And also BSD, enough to come all the way over to BSDCan at least

Explain what GSoC is briefly on this slide. Program by Google, where you get a stipend to work on an open-source project (e.g. FreeBSD).

Please do *not* hesitate to interrupt me if I'm too quiet, I have a tendency to speak too quietly!

I think it's best to do it this way round cuz LinuxKPI is definitely more important than BATMAN.

I think specifying the license is important cuz otherwise you taint your kernel.

Graphics kernel-side drivers.

Even though the LinuxKPI can already do all this stuff, there are still plenty of functions, structs, and defines Linux kernel modules use that the LinuxKPI doesn't provide for at the moment.

Mention that functions aren't necessarily needed to be implemented by understanding what the code does. Maybe it does something relevant on Linux but that wouldn't be relevant on FreeBSD.

This code might be buggy so please no one copy/paste this.

Talk about structs?

And without using the system's routing tables.

I assume most people know what a routing algorithm does, but...

This is for explaining what a routing algorithm is/does.

![bg width:70%](img/app-light.png) ![bg width:70%](img/app-node-light.png) ![bg width:70%](img/app-info-light.png)

TODO: This should probably go into my thoughts list - a much better way of marketing is showing things through a story. I mean, that's nothing new, stories are good. But this is a great example as to why.

Before I go into the more technical details...

batman-adv is by far the most used, at least in Freifunk

Didn't really know where to put this

While we're on the topic of fun facts...

"I don't wanna get too technical on the BATMAN-side of things, but since I know this'll interest some of you, I'm going to briefly explain how BATMAN works".

hard interfaces are what actually interact with the outside world

soft interfaces are "virtual" interfaces which sit on top of one or more hard interfaces

hard interfaces are added to the batman soft interface

go back to the napkin here

I.e. reading patchnotes :)

Cloner is jargon for "type of interface".

batman-adv does everything though Netlink, thank Alex, would've been much much harder without preliminary Netlink support.

Reason for it being batadv on FreeBSD is that FreeBSD interface names start with their cloner name.

Goal: because it's easier to maintain and pull in updates and can possibly be upstreamed to BATMAN later.

Aliased fields is easy stuff like MTU.

This is a little more involved than just wrappers in headers. But for the most part it is really just wrappers in headers.

if_t is a typedef for struct ifnet.

if_alloc_domain had to be split with if_fill_domain cuz linuxkpi_alloc_netdev_ifp has to allocate it itself.

(For last two functions) These things take in skbuff's, which is the next section.

To be honest skb kind of does everything and the kitchen sink.

Can't add that to base because of GPL. Did ask the batman guys about dual-licensing with BSD but that was a no

I don't think Wi-Fi is hard, it's just that there are some places where I'm dealing only with Ethernet headers which I guess are probably different for Wi-Fi but ultimately similar.

Wanted to do this after EuroBSDCon, but got caught up with school.

Was honestly a larger scope than expected, but fundamentally nothing was particularly "hard".

People kind of expect FreeBSD to work on their laptop's Wi-Fi chip, and if it doesn't, they're quite likely to look elsewhere

And I know there are FreeBSD purists who will say that "FreeBSD is a server OS", but if we're honest with ourselves it's through desktops that people discover operating systems, not through servers.

I am more knowledgable about the LinuxKPI side of things than the BATMAN side of things, so I can't answer all questions!!

If you wanna have a beer with me I'm almost always in the mood for a beer.

That's supposed to be a w not a double v, I know the font makes it hard to read.

You can also find me on Linkedin if you type in my name