Linux driver model
Intro
When one want to get into driver development on linux it's pretty overwhelming. You do not know where to start and the src code is so big, additionally official docs lack drawings and can be pretty hard to grasp. So what to do?
All you need is small starting point wich i'm gonna provide to you in this artcile. We will go through the driver model on linux and how does it work. Once we clean this up various pieces of driver development should be much easier to you.
Without further a do, let's start.
Driver model
We need to introduce few components when it comes to driver model:
- A device: this is piece of hardware or some internal CPU IP block. It represents raw electronics.
- A driver: driver allows kernel to use a device. It tells kernel how to talk with the device.
- A bus: goal of the bus is to hold devices and drivers and match them together.
Ok, we have a device, a driver and a bus. Bus role is to match a device and a driver into a pair. Let’s consider some diagrams.
First we start with empty bus:
Then devices are initialized:
Then drivers are initialized and bus try to match drivers with devices:
The basic flow looks like this:
- device shows up (or you register it) on some bus
- driver is registered on that bus
- bus checks if they fit (match(dev, drv))
- if yes, kernel runs driver’s probe() to initialize hardware and finish the binding
Once driver and device are matched, bus runs driver’s probe(). The goal of probe() is to establish whether this particular piece of hardware, in its actual state, can be handled by the driver. If no errors are reported by probe(), they stay bound.
You should have pretty good overview of what is happening. Before we go further, let’s look into some code.
Device
struct device is located in include/linux/device.h:
struct device {
struct kobject kobj;
struct device *parent;
struct device_private *p;
const char *init_name; /* initial name of the device */
const struct device_type *type;
const struct bus_type *bus; /* type of bus device is on */
struct device_driver *driver; /* which driver has allocated this
device */
void *platform_data; /* Platform specific data, device
core doesn't touch it */
void *driver_data; /* Driver data, set and get with
dev_set_drvdata/dev_get_drvdata */
struct mutex mutex; /* mutex to synchronize calls to
* its driver.
*/
...
We are interested in only two fields from this struct, a bus to which device is assigned:
const struct bus_type *bus; /* type of bus device is on */
and a driver which is assigned to the device:
struct device_driver *driver; /* which driver has allocated this
Driver
struct driver is located in include/linux/device/driver.h:
struct device_driver {
const char *name;
const struct bus_type *bus;
struct module *owner;
const char *mod_name; /* used for built-in modules */
bool suppress_bind_attrs; /* disables bind/unbind via sysfs */
enum probe_type probe_type;
const struct of_device_id *of_match_table;
const struct acpi_device_id *acpi_match_table;
int (*probe) (struct device *dev);
void (*sync_state)(struct device *dev);
int (*remove) (struct device *dev);
void (*shutdown) (struct device *dev);
int (*suspend) (struct device *dev, pm_message_t state);
int (*resume) (struct device *dev);
const struct attribute_group **groups;
const struct attribute_group **dev_groups;
const struct dev_pm_ops *pm;
void (*coredump) (struct device *dev);
struct driver_private *p;
};
We are interested in only two fields from this struct, a bus to which driver is assigned:
const struct bus_type *bus;
And a struct driver_private keeps the list of devices currently bound to this driver (struct klist klist_devices). Since a driver represent a global instance it can handle multiple devices:
struct driver_private {
struct kobject kobj;
struct klist klist_devices;
struct klist_node knode_bus;
struct module_kobject *mkobj;
struct device_driver *driver;
};
Bus
struct bus_type is located in include/linux/device/bus.h:
struct bus_type {
const char *name;
const char *dev_name;
const struct attribute_group **bus_groups;
const struct attribute_group **dev_groups;
const struct attribute_group **drv_groups;
int (*match)(struct device *dev, struct device_driver *drv);
int (*uevent)(const struct device *dev, struct kobj_uevent_env *env);
int (*probe)(struct device *dev);
void (*sync_state)(struct device *dev);
void (*remove)(struct device *dev);
...
We are interested in hidden part of bus_type which is not up there. You see when you execute bus_add_device or similiar function indirectly via one of X_register functions, bus looks up it's hidden part called struct subsys_private and it contain devices_kset which stores devices objects:
int bus_add_device(struct device *dev)
{
struct subsys_private *sp = bus_to_subsys(dev->bus);
int error;
if (!sp) {
/*
* This is a normal operation for many devices that do not
* have a bus assigned to them, just say that all went
* well.
*/
return 0;
}
...
klist_add_tail(&dev->p->knode_bus, &sp->klist_devices);
return 0;
}
Similiarly for driver:
int bus_add_driver(struct device_driver *drv)
{
struct subsys_private *sp = bus_to_subsys(drv->bus);
struct driver_private *priv;
int error = 0;
if (!sp)
return -EINVAL;
...
klist_add_tail(&priv->knode_bus, &sp->klist_drivers);
...
return 0;
}
Example
To understand better how bus, device and driver cooperate together, let’s build a minimal example. We’ll make a three small LKMs that defines one bus, one device, and one driver, and then watch how the bus matches device and driver. Let's start by creating a file called my_device.c:
/*
* my_device
****************************************************************
* Brief Description:
* One of the basic blocks of driver model, the device.
*/
#include <linux/device.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
MODULE_AUTHOR("Jakub Buczynski");
MODULE_DESCRIPTION("A dummy device");
MODULE_LICENSE("Dual MIT/GPL");
extern struct bus_type my_bus;
static void my_device_release(struct device *dev) {
pr_info("%s\n", __func__);
}
struct device my_device = {
.init_name = "my_device",
.release = my_device_release,
.bus = &my_bus,
};
static int __init my_device_module_init(void) {
pr_info("%s\n", __func__);
/* Registering a device means it appear in:
/sys/devices
*/
int ret = device_register(&my_device);
if (ret < 0) {
pr_err("my_device: could not register my_device\n");
put_device(&my_device);
return ret;
}
return 0;
}
static void __exit my_device_module_exit(void) {
pr_info("%s\n", __func__);
device_unregister(&my_device);
}
module_init(my_device_module_init);
module_exit(my_device_module_exit);
next create a file called my_bus.c:
/*
* my_bus
****************************************************************
* Brief Description:
* Goal of a bus is to match device and a driver into the pair.
*/
#include <linux/device.h>
#include <linux/device/driver.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/of.h>
MODULE_AUTHOR("Jakub Buczynski");
MODULE_DESCRIPTION("");
MODULE_LICENSE("Dual MIT/GPL");
/*
* @match: Called, perhaps multiple times, whenever a new device or driver
* is added for this bus. It should return a positive value if the
* given device can be handled by the given driver and zero
* otherwise. It may also return error code if determining that
* the driver supports the device is not possible. In case of
* -EPROBE_DEFER it will queue the device for deferred probing.
*/
static int my_bus_match(struct device *dev, struct device_driver *drv) {
dev_err(dev, "my_bus_match\n");
return 1; // we always match
}
static int my_bus_uevent(const struct device *dev,
struct kobj_uevent_env *env) {
dev_err(dev, "my_bus_uevent\n");
add_uevent_var(env, "MODALIAS=MY BUS IS AWESOME");
return 0;
}
struct bus_type my_bus = {
.name = "my_bus",
.match = my_bus_match,
.uevent = my_bus_uevent,
};
static int __init my_bus_module_init(void) {
pr_info("Inserted: %s\n", __func__);
/* Registering a bus means it appear in:
/sys/bus
*/
int ret = bus_register(&my_bus);
if (ret) {
pr_err("my_bus: could not register bus for my_bus\n");
}
return 0;
}
static void __exit my_bus_module_exit(void) {
pr_info("Removed: %s\n", __func__);
bus_unregister(&my_bus);
}
module_init(my_bus_module_init);
module_exit(my_bus_module_exit);
EXPORT_SYMBOL_GPL(my_bus);
next create a file called my_driver.c:
/*
* my_driver
****************************************************************
* Brief Description:
* Goal of a driver is to handle a device.
*/
#include <linux/device.h>
#include <linux/device/driver.h>
#include <linux/init.h>
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
MODULE_AUTHOR("Jakub Buczynski");
MODULE_DESCRIPTION("");
MODULE_LICENSE("Dual MIT/GPL");
extern struct bus_type my_bus;
struct my_driver_device_data {
int number;
};
/*
probe holds the driver-specific logic to bind the driver to a given device.
That includes verifying that the device is present, that it’s a version the
driver can handle, that driver data structures can be allocated and
initialized, and that any hardware can be initialized. Drivers often store a
pointer to their state with dev_set_drvdata(). When the driver has
successfully bound itself to that device, then probe() returns zero and the
driver model code will finish its part of binding the driver to that device.
A driver’s probe() may return a negative errno value to indicate that the
driver did not bind to this device, in which case it should have released all
resources it allocated.
*/
static int my_driver_probe(struct device *dev) {
struct my_driver_device_data *dev_data =
kmalloc(sizeof(struct my_driver_device_data), GFP_KERNEL);
if (!dev_data) {
pr_err("Unable to malloc %s\n", __func__);
return -ENOMEM;
}
dev_data->number = 99;
dev_set_drvdata(dev, dev_data);
pr_info("%s: %d\n", __func__, dev_data->number);
return 0;
}
/*
remove is called to unbind a driver from a device. This may be called if a
device is physically removed from the system, if the driver module is being
unloaded, during a reboot sequence, or in other cases.
*/
static int my_driver_remove(struct device *dev) {
struct my_driver_device_data *dev_data = dev_get_drvdata(dev);
pr_info("%s: %d\n", __func__, dev_data->number);
kfree(dev_data);
return 0;
}
static struct device_driver my_driver = {
.name = "my_driver",
.bus = &my_bus,
.probe = my_driver_probe,
.remove = my_driver_remove,
};
static int __init my_driver_module_init(void) {
pr_info("Inserted: %s\n", __func__);
/* Registering a driver means it appear in:
/sys/bus/my_bus/drivers
*/
int ret = driver_register(&my_driver);
if (ret) {
pr_err("my_driver: could not register my_driver\n");
}
return 0;
}
static void __exit my_driver_module_exit(void) {
pr_info("Removed: %s\n", __func__);
driver_unregister(&my_driver);
}
module_init(my_driver_module_init);
module_exit(my_driver_module_exit);
And a Makefile:
ifneq ($(KERNELRELEASE),)
# kbuild part of makefile
obj-m := my_bus.o
obj-m += my_device.o
obj-m += my_driver.o
else
# normal makefile
KDIR ?= /lib/modules/`uname -r`/build
PWD := $(shell pwd)
ccflags-y += -DDEBUG
all:
make -C $(KDIR) M=$(PWD) modules
install:
make -C $(KDIR) M=$(PWD) modules_install
clean:
make -C $(KDIR) M=$(PWD) clean
endif
Once you build modules we can proceed to showing in action how device, driver and bus work together:
# insmod my_bus.ko
[ 6556.210576] Inserted: my_bus_module_init
# ls /sys/bus/my_bus/
devices drivers_autoprobe uevent
drivers drivers_probe
# ls /sys/bus/my_bus/devices/
# ls /sys/bus/my_bus/drivers/
Registering a bus creates a directory in /sys/bus/. The directory representing bus contain directories for representation od devices and drivers assigned to a bus.
When we register a device it will appear in /sys/bus/my_bus/devices/ directory:
# insmod my_device.ko
[ 6708.988359] my_device_module_init
[ 6708.990472] device: 'my_device': device_add
[ 6708.994540] my_bus my_device: my_bus_uevent
# ls /sys/bus/my_bus/devices/
my_device
If we add a driver to the bus kernel behave similiarly:
# insmod my_driver.ko
[ 6815.955324] Inserted: my_driver_module_init
[ 6815.958154] my_bus my_device: my_bus_match
[ 6815.962457] my_driver_probe: 99
[ 6815.965375] my_driver my_device: my_bus_uevent
# ls /sys/bus/my_bus/drivers
my_driver
Inspecting /sys/bus/my_bus/devices/my_device and /sys/bus/my_bus/drivers/my_driver directories let's leave as an exercise.
Driver model continued
Bus represents the binding between a device and a driver. There are two common ways the bus works's:
- discoverable bus: the OS doesn’t need to be told that the device exists. The bus can discover it by itself.
- non-discoverable bus: the OS must be told that a device is present and how it should look.
Think about USB it is the easiest discoverable bus example. When you plug in a pendrive, the USB controller detects a new thing on the wire, reads its descriptors (vendor ID, product ID, class, endpoints, etc.), and the kernel creates a usb_device for it. Then the USB bus tries to find a driver that matches the IDs or class. You didn’t describe anything in advance but the hardware basically introduces itself.
On many ARM systems the CPU doesn’t have a standard way to enumerate on‑board peripherals. The SoC might have I2C controllers, SPI controllers, GPIO blocks, UARTs, or custom FPGA logic, but nothing “announces” them. So the kernel needs a description from firmware. The common solution is Device Tree. The kernel reads the DT at boot, and for every node it creates a platform device with resources (register ranges, IRQs, clocks, resets, DMA, pinctrl, etc.). Only then can a platform driver match on the node’s compatible string and run probe(). Example can be GPIO or an I2C sensor soldered on your board. Without DT the kernel cannot guess which pins are connected, what address the sensor uses or which interrupts it triggers.
To represent non discoverable bus we have struct platform_driver and struct platform_device, we can use them to read each node of device tree and bind device to driver by compatibile field. It's api can be find in include/linux/platform_device.sh. Platform implements it's own device struct platform_device which embeds struct device:
struct platform_device {
const char *name;
int id;
bool id_auto;
struct device dev;
...
};
Creating upper layer on the basis showed in the example before. Next platform creates a bus, struct bus_type platform_bus_type:
struct bus_type platform_bus_type = {
.name = "platform",
.dev_groups = platform_dev_groups,
.match = platform_match,
.uevent = platform_uevent,
.probe = platform_probe,
.remove = platform_remove,
.shutdown = platform_shutdown,
.dma_configure = platform_dma_configure,
.dma_cleanup = platform_dma_cleanup,
.pm = &platform_dev_pm_ops,
};
And it creates also a driver (struct platform_driver embeds struct driver like with struct device) which can be used with platform bus and with platform device to register new driver to platform bus:
struct platform_driver {
int (*probe)(struct platform_device *);
int (*remove)(struct platform_device *);
void (*remove_new)(struct platform_device *);
void (*shutdown)(struct platform_device *);
int (*suspend)(struct platform_device *, pm_message_t state);
int (*resume)(struct platform_device *);
struct device_driver driver;
const struct platform_device_id *id_table;
bool prevent_deferred_probe;
bool driver_managed_dma;
};
Other buses like usb ( search for struct usb_bus, struct usb_device, struct usb_driver ) or pcie ( search for struct pci_bus, struct pci_dev, struct pci_driver ) behave similiarly as platform bus, they declare it's own driver, it's own bus and it's own device to create a layer on top of the basic driver model, just the way they handle match function differ a lot.
Conclusions
At this point you should have a solid picture of the Linux driver model. Everything starts from the same three blocks: device, driver, and bus. The bus matches them, and probe() is where real binding and hardware init happens.
When you work on a real driver, first ask yourself two simple questions:
- is my hardware discoverable by itself (like USB/PCIe), or do I need to describe it (like platform/DT)?
- which bus already exists for this kind of device, and what device/driver wrappers does it provide?
Once you answer that, the rest is just using the right bus API and filling in your driver logic.