File IO
Introduction
There are various kinds of communication that an operating system (OS) needs to perform on a daily basis: the OS must talk to the disk whenever you use a filesystem, and it must talk to the network card whenever you use a TCP/UDP socket. All these communications require different protocols and involve very different hardware, so how does Linux handle them?
Files babe, files.
Besides syscalls, the primary medium of communication between operating-system components is a file, more precisely file descriptor.
In this article we will talk about files on Linux and how you can influence them from C. So let’s start :)
Files, pipelines and sockets
When you are learning Bash, working with files, pipelines, or sockets seems to require different symbols and commands, so it’s easy to think they are very different. Well, yes and no: in the OS they are all abstracted as files. Not “files on disk” because a file does not need to be stored on a drive; a file can live entirely in memory and still be presented to user space as if it were on disk. This uniform abstraction covers files, sockets, pipelines, devices, and more.
One example of such an in-memory filesystem is /sys/kernel. You can find there various files that represent kernel-environment configuration. For example, to check whether you have little-endian or big-endian byte order in the CPU, read /sys/kernel/cpu_byteorder. This file is not stored on any drive.
Even though /sys/kernel/cpu_byteorder is an in-memory file, we can display it like any other stored on a drive file:
✦ ❯ cat /sys/kernel/cpu_byteorder
little
Pipelines also live only in memory. Once you create a pipeline, the OS allocates memory for it and handles its operations as if it were a file:
✦ ❯ mkfifo mypipeline
✦ ❯ bash -c 'watch "cat /sys/kernel/cpu_byteorder 1>> mypipeline" &'
Now we can read it like any other file:
✦ ❯ cat mypipeline
little
little
little
little
little
little
little
little
...
Sockets are similar: you can create a socket and then write to or read from it:
✦ ❯ socat -d -d UNIX-LISTEN:/tmp/mysock,fork EXEC:/bin/cat &
✦ ❯ echo "little" | socat - UNIX-CONNECT:/tmp/mysock
little
A unified set of behaviours among files, pipelines, and sockets is possible because underneath they all use file descriptors. Let’s explore what file descriptors are and how they can be used.
File descriptor
File descriptors are more of an OS concept than a C standard concept, so before we start talking about them we need to establish two layers:
- C layer:
FILEobject—this is the stream representation. - OS layer:
struct fileobject—this is the file-descriptor representation.
When interacting with a file descriptor from C, we use a FILE object, which internally stores a number that identifies the file descriptor in the OS layer:
int _fileno;
Once the numeric file descriptor is passed to the OS (for example, to the read function), the kernel uses struct file to find the appropriate filesystem driver and disk driver and do its magic.
Using file descriptors is pretty simple and requires following a pattern:
- Create a file descriptor.
- Operate on the file descriptor.
- Close the file descriptor.
As long as you follow this simple pattern you should be all right.
As a bonus, here is the stream representation in glibc:
typedef struct _IO_FILE FILE;
...
struct _IO_FILE
{
int _flags; /* High-order word is _IO_MAGIC; rest is flags. */
/* The following pointers correspond to the C++ streambuf protocol. */
char *_IO_read_ptr; /* Current read pointer */
char *_IO_read_end; /* End of get area. */
char *_IO_read_base; /* Start of putback+get area. */
char *_IO_write_base; /* Start of put area. */
char *_IO_write_ptr; /* Current put pointer. */
char *_IO_write_end; /* End of put area. */
char *_IO_buf_base; /* Start of reserve area. */
char *_IO_buf_end; /* End of reserve area. */
/* The following fields are used to support backing up and undo. */
char *_IO_save_base; /* Pointer to start of non-current get area. */
char *_IO_backup_base; /* Pointer to first valid character of backup area */
char *_IO_save_end; /* Pointer to end of non-current get area. */
struct _IO_marker *_markers;
struct _IO_FILE *_chain;
int _fileno;
int _flags2;
__off_t _old_offset; /* This used to be _offset but it's too small. */
/* 1+column number of pbase(); 0 is unknown. */
unsigned short _cur_column;
signed char _vtable_offset;
char _shortbuf[1];
_IO_lock_t *_lock;
#ifdef _IO_USE_OLD_IO_FILE
};
Open a file
Opening a file requires knowing two things: its location in the filesystem and the mode in which we will communicate with it. The modes are:
| File-access mode | Meaning | Action if file exists | Action if file does not exist |
|---|---|---|---|
"r" |
read | read from start | fail |
"w" |
write | destroy contents | create |
"a" |
append | write to end | create |
"r+" |
read/write | read from start | fail |
"w+" |
read/write | destroy contents | create |
"a+" |
read/write | write to end | create |
Once we know the location and mode, we can use fopen to create a file descriptor. Example:
#include <bits/types.h>
#include <stdio.h>
#include <unistd.h>
int main(void) {
FILE *fp = fopen("/tmp/file.c", "r");
printf("fp->_fileno=%d\n", fp->_fileno);
char buffer[10] = {0};
read(fp->_fileno, buffer, sizeof(buffer) - 1);
puts(buffer);
return 0;
}
output:
fp->_fileno=3
aksjdhkla
fopen returns a FILE object which (on Linux) is just a wrapper around the numeric file descriptor. Once this descriptor is passed to the OS, the OS knows how to translate it into struct file.
As we can read from the openat man page:
✦ ❯ man openat
...
The return value of open() is a file descriptor, a small, nonnega‐
tive integer that is an index to an entry in the process's table of
open file descriptors. The file descriptor is used in subsequent
system calls (read(2), write(2), lseek(2), fcntl(2), etc.) to refer
to the open file.
Just to be sure, let’s examine our program with strace to check if it actually calls openat:
✦ ❯ strace ./a.out
execve("./a.out", ["./a.out"], 0x7ffc610395d0 /* 64 vars */) = 0
...
openat(AT_FDCWD, "/tmp/file.c", O_RDONLY) = 3
newfstatat(1, "", {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0x2), ...}, AT_EMPTY_PATH) = 0
write(1, "fp->_fileno=3\n", 14fp->_fileno=3
) = 14
read(3, "aksjdhkla", 9) = 9
write(1, "aksjdhkla\n", 10aksjdhkla
) = 10
exit_group(0) = ?
+++ exited with 0 +++
Now let’s check what openat actually does:
SYSCALL_DEFINE4(openat, int, dfd, const char __user *, filename, int, flags,
umode_t, mode)
{
if (force_o_largefile())
flags |= O_LARGEFILE;
return do_sys_open(dfd, filename, flags, mode);
}
long do_sys_open(int dfd, const char __user *filename, int flags, umode_t mode)
{
struct open_how how = build_open_how(flags, mode);
return do_sys_openat2(dfd, filename, &how);
}
static long do_sys_openat2(int dfd, const char __user *filename,
struct open_how *how)
{
struct open_flags op;
int fd = build_open_flags(how, &op);
struct filename *tmp;
if (fd)
return fd;
tmp = getname(filename);
if (IS_ERR(tmp))
return PTR_ERR(tmp);
fd = get_unused_fd_flags(how->flags);
if (fd >= 0) {
struct file *f = do_filp_open(dfd, tmp, &op);
if (IS_ERR(f)) {
put_unused_fd(fd);
fd = PTR_ERR(f);
} else {
fd_install(fd, f);
}
}
putname(tmp);
return fd;
}
As you can see, the kernel creates struct file here:
struct file *f = do_filp_open(dfd, tmp, &op);
Here is the struct:
struct file {
union {
struct llist_node f_llist;
struct rcu_head f_rcuhead;
unsigned int f_iocb_flags;
};
/*
* Protects f_ep, f_flags.
* Must not be taken from IRQ context.
*/
spinlock_t f_lock;
fmode_t f_mode;
atomic_long_t f_count;
struct mutex f_pos_lock;
loff_t f_pos;
unsigned int f_flags;
struct fown_struct f_owner;
const struct cred *f_cred;
struct file_ra_state f_ra;
struct path f_path;
struct inode *f_inode; /* cached value */
const struct file_operations *f_op;
u64 f_version;
#ifdef CONFIG_SECURITY
void *f_security;
#endif
/* needed for tty driver, and maybe others */
void *private_data;
#ifdef CONFIG_EPOLL
/* Used by fs/eventpoll.c to link all the hooks to this file */
struct hlist_head *f_ep;
#endif /* #ifdef CONFIG_EPOLL */
struct address_space *f_mapping;
errseq_t f_wb_err;
errseq_t f_sb_err; /* for syncfs */
}
Usage looks like this:
Userspace | Kernel space
---------------------------
FILE > int > struct file
--------------------------
Stream | File descriptor
So, we have established a basic understanding of how creating a file descriptor is implemented on Linux.
Let’s move to the next operation, reading from a file.
Read a file
Once a file is opened, we are ready to read its contents:
#include <stdio.h>
int main(void) {
FILE *fp = fopen("/tmp/file.c", "r");
char buffer[10] = {0};
fread(buffer, sizeof(char), sizeof(buffer) - 1, fp);
puts(buffer);
return 0;
}
Let’s examine our program with strace to understand how it interacts with the OS:
✦ ❯ strace ./a.out
...
openat(AT_FDCWD, "/tmp/file.c", O_RDONLY) = 3
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=18, ...}, AT_EMPTY_PATH) = 0
read(3, "aksjdhklajsdkhjad\n", 4096) = 18
newfstatat(1, "", {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0x2), ...}, AT_EMPTY_PATH) = 0
write(1, "aksjdhkla\n", 10aksjdhkla
) = 10
lseek(3, -9, SEEK_CUR) = 9
exit_group(0) = ?
+++ exited with 0 +++
We can see that our code uses the familiar openat function, but now it also invokes read.
From the read technical manual:
✦ ❯ man read
...
read() attempts to read up to count bytes from file descriptor fd
into the buffer starting at buf.
So the translation from the integer descriptor to struct file happened again, as we can confirm from the read syscall source:
SYSCALL_DEFINE3(read, unsigned int, fd, char __user *, buf, size_t, count)
{
return ksys_read(fd, buf, count);
}
ssize_t ksys_read(unsigned int fd, char __user *buf, size_t count)
{
struct fd f = fdget_pos(fd);
ssize_t ret = -EBADF;
if (f.file) {
loff_t pos, *ppos = file_ppos(f.file);
if (ppos) {
pos = *ppos;
ppos = &pos;
}
ret = vfs_read(f.file, buf, count, ppos);
if (ret >= 0 && ppos)
f.file->f_pos = pos;
fdput_pos(f);
}
return ret;
}
As we clearly see, read also translates the numeric file descriptor into struct file:
struct fd {
struct file *file;
unsigned int flags;
};
Close a file
Once a file descriptor is no longer needed, we should close it, which frees resources allocated by the OS for struct file:
#include <stdio.h>
#define FILE_NAME "signals.txt"
int main(void) {
// Open file
FILE *fp = fopen(FILE_NAME, "rb");
if (!fp) {
perror(FILE_NAME);
return 1;
}
// Read n bytes from file
char buffer[8];
int res = fread(buffer, sizeof(char), sizeof(buffer), fp);
if (res < sizeof(buffer) && !feof(fp) && ferror(fp)) {
perror(FILE_NAME);
fclose(fp);
return 2;
}
printf("%.*s\n", res, buffer);
// Close file
fclose(fp);
return 0;
}
strace output:
✦ ❯ strace ./a.out
...
openat(AT_FDCWD, "/tmp/file.c", O_RDONLY) = 3
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=477, ...}, AT_EMPTY_PATH) = 0
read(3, "#include <stdio.h>\n\n#define FILE"..., 4096) = 477
newfstatat(1, "", {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0x2), ...}, AT_EMPTY_PATH) = 0
write(1, "#include\n", 9#include
) = 9
close(3) = 0
exit_group(0) = ?
+++ exited with 0 +++
From the close manual:
✦ ❯ man close
If fd is the last file descriptor referring to the underlying open
file description (see open(2)), the resources associated with the
open file description are freed; if the file descriptor was the
last reference to a file which has been removed using unlink(2),
the file is deleted.
Read formatted file
Suppose you want to parse space-separated data. We call such data formatted, and fscanf lets us parse it easily.
First create main.c:
#include <stdio.h>
#define FILE_NAME "/tmp/signals.txt"
int main(void) {
// Open file
FILE *fp = fopen(FILE_NAME, "rb");
if (!fp) {
perror(FILE_NAME);
return 1;
}
// Read n bytes from file
// Read fmt line from file
while (!feof(fp) && !ferror(fp)) {
char line[5] = {0};
int res = fscanf(fp, "%*d %4s %*s\n", line);
if (res != 1) { // we need to check if `line` was populated
// 1 stands for number of populated args.
break;
} else {
printf("Signal: %s\n", line);
}
}
// Close file
fclose(fp);
return 0;
}
Then create /tmp/signals.txt:
1 HUP Hangup
2 INT Interrupt
3 QUIT Quit
4 ILL Illegal_instruction
5 TRAP Trace_trap
6 ABRT Abort
7 EMT EMT_trap
8 FPE Floating-point_exception
Compile and run:
✦ ❯ ./a.out
Signal: HUP
Signal: INT
Signal: QUIT
Signal: ILL
Signal: TRAP
Signal: ABRT
Signal: EMT
Signal: FPE
fscanf provides a simple yet fast way to parse whitespace-separated data.
Write to a file
Writing is very similar to reading; the main differences are the mode and the syscall:
#include <stdio.h>
#define FILE_NAME "/tmp/signals.txt"
int main(void) {
// Open file
FILE *fp = fopen(FILE_NAME, "a");
if (!fp) {
perror(FILE_NAME);
return 1;
}
// Write line to file
int res = fwrite("9 MYC My_custom_signal\n", 1,
sizeof("9 MYC My_custom_signal\n"), fp);
if (res < 0) {
perror(FILE_NAME);
fclose(fp);
return 2;
}
// Close file
fclose(fp);
return 0;
}
We changed the mode from "r" to "a" and the function from fread to fwrite.
Write to formatted file
Writing formatted data is analogous to reading it:
#include <stdio.h>
#define FILE_NAME "/tmp/signals.txt"
int main(void) {
// Open file
FILE *fp = fopen(FILE_NAME, "a");
if (!fp) {
perror(FILE_NAME);
return 1;
}
// Write fmt line to file
int res = fprintf(fp, "%d %s %s\n", 9, "MYS", "My_custom_signal");
if (res < 0) {
perror(FILE_NAME);
fclose(fp);
return 2;
}
// Close file
fclose(fp);
return 0;
}
Again, we changed the mode from "r" to "a" and the function from fscanf to fprintf.
Conclusions
Files, streams, file descriptors, sockets, pipelines—it can all be a bit overwhelming, so let’s summarise:
- The C standard specifies the
FILEstream object. - On Linux (glibc),
FILEcontains an integer file descriptor. - That integer is translated in kernel space into
struct file. - Whenever we issue a syscall that involves a file descriptor, the kernel really operates on
struct file.
That’s why we say that on Linux everything is a file.