Linux Kernel Programming–Memory Allocation

By roman10 On July 29, 2011 · Leave a Comment

Memory allocation in Linux kernel is different from the user space counterpart. The following facts are noteworthy,

Kernel memory is not pageable.
Kernel memory allocation mistakes can cause system oops (system crash) easily.
Kernel memory has limited hard stack size limit.

There’re two ways to allocate memory space for a kernel process, statically from the stack or dynamically from the heap.

Static Memory Allocation

The static memory allocation is normally used you know how much memory space you’ll need. For example,

#define BUF_LEN    2048
char buf[BUF_LEN];

However, the kernel stack size is fixed and limited (the limit is architecture dependent, but normally it’s only tens of kilobytes). Therefore people seldom request big chunk of memory in the stack. The better way is to allocate the memory dynamically from heap.

Dynamic Memory Allocation

There’re two functions available to allocate memory from heap in Linux kernel process,

1. vmalloc

The vmalloc function is defined in /lib/modules/$(uname -r)/build/include/linux/vmalloc.h as below,

void *vmalloc(unsigned long size);

It’s Linux kernel’s version of malloc() function, which is used in user space. Like malloc, the function allocates virtually contiguous memory that may or may not physically contiguous.

To free the memory space allocated by vmalloc, one simply call vfree(), which is defined as,

void vfree(const void *addr);

2. kmalloc

The most commonly used memory allocation function in kernel is kmalloc, which is defined in /lib/modules/$(uname -r)/build/include/linux/slab.h as below,

void * kmalloc(size_t size, int flags);

kmalloc allocates a region of physically contiguous (also virtually contiguous) memory and return the pointer to the allocated memory. It returns NULL when the operation fails.

The behavior of kmalloc is dependent on the second parameter flags. Here only the two most popular flags are introduced,

GFP_KERNEL: this flag indicates a normal kernel memory allocation. The kernel might block the requesting code, free up enough memory and then continue the allocation. It cannot be used in places where sleep is not allowed.

GFP_ATOMIC: this flag indicates the kmalloc function is atomic operation. Atomic operation means the function is performed entirely or not performed at all. It cannot block at the middle of execution. As the kernel cannot jump out of the allocation and free up memory to satisfy the function request, this function has a higher chance of failure with this flag passed in.

There’re several other flags, for example, GFP_DMA for allocation of memory space capable of undergoing Direct Memory Access. For more information, one can refer the links provided in the references section.

To free up the memory allocated by kmalloc, one can use kfree defined as below,

void kfree(const void *objp);

3. vmalloc VS kmalloc

vmalloc allocates virtually contiguous memory space (not necessarily physically contiguous), while kmalloc allocates physically contiguous memory (also virtually contiguous). Most of the memory allocations in Linux kernel are done using kmalloc, due to the following reasons:

On many architectures, hardware devices don’t understand virtual address. Therefore, their device drivers can only allocate memory using kmalloc.
kmalloc has better performance in most cases because physically contiguous memory region is more efficient than virtually contiguous memory. The reason behind this is not covered here, interested readers can search for Linux memory management articles.

But when a large chunk of memory is needed, vmalloc is used often as it doesn’t require physically contiguous memory and the kernel can satisfy the request with much less effort than using kmalloc.

Sample Program

Below is a sample program illustrate usage of different memory allocation methods,

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/vmalloc.h>
#include <linux/slab.h>
 
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("hello");
MODULE_AUTHOR("Liu Feipeng/roman10");
 
struct Person {
    char name[30];
    unsigned int weight;
    unsigned char gender;
    struct list_head list; /* kernel's list structure */
};
 
struct Person personList;
 
int init_module() {
    struct Person *aNewPerson, *aPerson;
 
    printk(KERN_INFO "initialize kernel module: test_mem\n");
    INIT_LIST_HEAD(&personList.list);    //or LIST_HEAD(mylist); 
 
    printk(KERN_INFO "allocate memory using kmalloc with GFP_KERNEL for first node\n");
    aNewPerson = kmalloc(sizeof(*aNewPerson), GFP_KERNEL);
    strcpy(aNewPerson->name, "roman1");
    aNewPerson->weight = 130;
    aNewPerson->gender = 1;
    INIT_LIST_HEAD(&aNewPerson->list);
    list_add_tail(&(aNewPerson->list), &(personList.list));
    
    printk(KERN_INFO "allocate memory using kmalloc with GFP_ATOMIC for 2nd node\n");
    aNewPerson = kmalloc(sizeof(*aNewPerson), GFP_ATOMIC);
    strcpy(aNewPerson->name, "roman2");
    aNewPerson->weight = 130*2;
    aNewPerson->gender = 1;
    INIT_LIST_HEAD(&aNewPerson->list);
    list_add_tail(&(aNewPerson->list), &(personList.list)); 
 
        printk(KERN_INFO "allocate memory using vmalloc for 3rd node\n");
    aNewPerson = vmalloc(sizeof(*aNewPerson));
    strcpy(aNewPerson->name, "roman3");
    aNewPerson->weight = 130*3;
    aNewPerson->gender = 1;
    INIT_LIST_HEAD(&aNewPerson->list);
    list_add_tail(&(aNewPerson->list), &(personList.list));    
    
    printk(KERN_INFO "traversing the list using list_for_each_entry()\n");
    list_for_each_entry(aPerson, &personList.list, list) {
        //access the member from aPerson
        printk(KERN_INFO "Person: %s; weight: %d; gender: %s\n", aPerson->name, aPerson->weight, aPerson->gender==0?"Female":"Male");
    }
    printk(KERN_INFO "\n");
 
    return 0;
}
 
void cleanup_module() {
    struct Person *aPerson;
    printk(KERN_INFO "kernel module unloaded: test_mem\n");
    aPerson = list_first_entry(&personList.list, struct Person, list);
    printk(KERN_INFO "freeing node %s\n", aPerson->name);
    list_del(&aPerson->list);
    kfree(aPerson);
    aPerson = list_first_entry(&personList.list, struct Person, list);
    printk(KERN_INFO "freeing node %s\n", aPerson->name);
    list_del(&aPerson->list);
    kfree(aPerson);
    aPerson = list_first_entry(&personList.list, struct Person, list);
    printk(KERN_INFO "freeing node %s\n", aPerson->name);
    list_del(&aPerson->list);
    vfree(aPerson);
}

To compile the program, use the makefile below. It’s suggested that you copy paste the following text into a file and save it as Makefile, as the spaces matter.

obj-m += test_mem.o
all:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
clean:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

Once the kernel module program is built, one can load the kernel module and unload the kernel module using the following two commands respectively.

sudo insmod test_mem.ko

sudo rmmod test_mem.ko

One can check the output from the file /var/log/messages,

Jul 30 00:16:12 roman10-laptop kernel: [44607.251090] initialize kernel module: test_mem
Jul 30 00:16:12 roman10-laptop kernel: [44607.251103] allocate memory using kmalloc with GFP_KERNEL for first node
Jul 30 00:16:12 roman10-laptop kernel: [44607.251114] allocate memory using kmalloc with GFP_ATOMIC for 2nd node
Jul 30 00:16:12 roman10-laptop kernel: [44607.251124] allocate memory using vmalloc for 3rd node
Jul 30 00:16:12 roman10-laptop kernel: [44607.251155] traversing the list using list_for_each_entry()
Jul 30 00:16:12 roman10-laptop kernel: [44607.251168] Person: roman1; weight: 130; gender: Male
Jul 30 00:16:12 roman10-laptop kernel: [44607.251178] Person: roman2; weight: 260; gender: Male
Jul 30 00:16:12 roman10-laptop kernel: [44607.251189] Person: roman3; weight: 390; gender: Male
Jul 30 00:16:12 roman10-laptop kernel: [44607.251273]
Jul 30 00:21:10 roman10-laptop kernel: [44905.918735] kernel module unloaded: test_mem
Jul 30 00:21:10 roman10-laptop kernel: [44905.918748] freeing node roman1
Jul 30 00:21:10 roman10-laptop kernel: [44905.918756] freeing node roman2
Jul 30 00:21:10 roman10-laptop kernel: [44905.918763] freeing node roman3