I haven’t really done much C in my professional career. There were a couple of times that I had to look into some C code, but not really create any kind of system in C. Most of my interaction with the C programming language dates back to university times.
I always liked C though. For its simplicity as a language, for its small surface as a language (i.e. not bloated with tons of features) and finally for the perfect, artistic, abstraction it provides on top of the hardware, which is so close to the metal but at the same time hiding the complexities and providing just about the right amount of abstractions.
For all the above and thanks to the lockdown and the plenty of me-time that I now have, I decided to revise C and try to put some of its features in practice by gradually building a very simple application.
Revising Using a Book
I didn’t want to follow any tutorials or knowledge in Google, hence I picked up a book in order to revise. Even though there are a couple of great books for C out there, for me it was a no brainer to actually revise one of the best books in my library The C Programming Language by Brian W. Kernighan and Dennis M. Ritchie.
This book is written back in 1988, but personally I find it to be one of the best technical books I ever read. I read the entire book in about 2 weeks, and in parallel I was trying to do some of the plenty exercise that each chapter has. By doing so, I refreshed some of my knowledge in C and got a fresh overview of its features as a language.
Deciding On A Simple Application
After finishing the book I decided it would be nice to put some of the concepts in action and build something small, trying to use some of the features I just revised. I wanted something that wouldn’t take me much time (as it would mainly be a playground, rather than a side project), but at the same time also give me some knowledge.
I was always interested and fairly familiar with network servers. I have an understanding of the underlyling kernel space functions that take place in socket programming, but I have always been using abstractions on top of this, mainly through Java frameworks. So I decided to build something around that concept, and as I wanted to keep it fairly simple and because I hadn’t really use this feature in the past, I concluded building a client/server applications using Unix Domain Sockets makes sense and most likely would fullfill most of my requirements.
Simplifying it a lot, a Unix Domain Socket is like any other socket (i.e. internet socket), but it can only be used for inter-process communication on the host it is opened, as it is actually backed by the filesystem.
Rules Of Engagement
In order to actually build this all by myself (I could just google an example and get the solution in 5 minutes), I decided to impose some very basic rules:
- I could use the book I read as a reference
- I could use man7.org as a reference for system calls
- I could not use any other internet resource that provided a ready baked solution
- The purpose of this application was not to be cross-platform. I was mainly interested making it work in my Mac and effectively in BSD like systems
The Code
I decided to split the code into different source (.c) files:
af_unix_sockets.c: The file containing themain()method and basic logic for parsing the command line arguments in order to start a client or a serveraf_unix_sockets_common.h: A header file containing common definitions and the prototypes for the different methods, that client or server implements and also the defininion of a simple typeAFUnixAddressstoring a file descriptor and the actual socket addressaf_unix_sockets_common.c: A source file containing some common methodsaf_unix_sockets_server.c: The server implementation, to be called by the main method inaf_unix_sockets.caf_unix_sockets_client.c: The client implementation, to be called by the main method inaf_unix_sockets.c
The Header File
As described above af_unix_sockets_common.h is a header file defining the prototypes of various functions (which I view as the public interface) to be implemented by parts of the system and to be called by other parts.
Additionally, the header defines a type, which I mainly created for Part II of this post, encapsulating the file descriptor of an opened unix domain socket and also its address.
#define TRUE 1
#define FALSE 0
#define CLIENT "client"
#define SERVER "server"
typedef struct af_unix_address {
int fd;
struct sockaddr_un *address;
} AFUnixAddress;
AFUnixAddress * open_af_unix_socket(char *);
void cleanup(int fd, char *path);
void server(char *path);
void client(char *path);
The Common File
I wanted to have a common file, just for the shake of it, to be able to export some common functionality that is shared between the server and the client.
#include "stdlib.h"
#include "stdio.h"
#include "string.h"
#include "stddef.h"
#include <sys/socket.h>
#include <sys/un.h>
#include <errno.h>
#include "af_unix_sockets_common.h"
/*
* @return AFUnixAddress type, containing the address and the file descriptor for the opened unix domain socket
*/
AFUnixAddress *open_af_unix_socket(char *path) {
int fd = socket(AF_UNIX, SOCK_STREAM, 0);
if(fd == -1) {
fprintf(stderr, "Failed to open AF_UNIX socket. ErrorNo=%d\n", errno);
cleanup(fd, path);
exit(errno);
}
AFUnixAddress *af_unix_socket = (AFUnixAddress *)malloc(sizeof(AFUnixAddress));
af_unix_socket->address = (struct sockaddr_un *)malloc(sizeof(struct sockaddr_un));
af_unix_socket->fd = fd;
af_unix_socket->address->sun_family = AF_UNIX;
strcpy(af_unix_socket->address->sun_path, path);
return af_unix_socket;
}
/*
* Clean up an opened file descriptor opened for
*/
void cleanup(int fd, char *path) {
close(fd);
remove(path);
if(fd != -1) {
fprintf(stderr, "Failed to successfully close socket. ErrorNo=%d\n", errno);
}
}
The common file is very simple, containing just two methods, one that opens a Unix Domain Socket for the specified path that was passed in and returning an AFUnixAddress which is a type defined in the af_unix_sockets_common.h file, containing the actual socket’s address and the file descriptor corresponding to that socket.
The file descriptor is needed for later use, to invoke system calls on it. Finally, worth mentioning that the Unix Domain Socket opened is a SOCK_STREAM one, which based on the documentation follows TCP semantics, as opposed to SOCK_DGRAM which follows UDP semantics.
The Client
The client’s behavior is defined in its own file and is pretty simplistic. A path is passed in specifying the filepath for the Unix Domain Socket. Then a socket is opened for that path and an invocation to #connect() method, passing in the file descriptor associated with the socket, forces the connection to be established.
Finally, the client reads from stdin and writes that to the socket invoking the write() method.
#include "stdlib.h"
#include "stdio.h"
#include "string.h"
#include "stddef.h"
#include "unistd.h"
#include <sys/socket.h>
#include <sys/types.h>
#include <sys/un.h>
#include "sys/syscall.h"
#include <errno.h>
#include "af_unix_sockets_common.h"
/*
* AF_UNIX socket client, obtains an ```AFUnixAddress``` by opening the socket to the specified ```path``` and then invoking ```#connect()``` on the socket's file descriptor.
* Finally, the client reads input from ```stdin``` and writes that to the socket.
*/
void client(char *path) {
fprintf(stdout, "Starting AF_UNIX client on Path=%s\n", path);
AFUnixAddress *domainSocketAddress = open_af_unix_socket(path);
printf("AF_UNIX client socket on Path=%s opened with fd=%d\n", domainSocketAddress->address->sun_path, domainSocketAddress->fd);
int isConnected = connect(domainSocketAddress->fd, (struct sockaddr *)domainSocketAddress->address, sizeof(struct sockaddr));
if(isConnected == -1) {
fprintf(stderr, "Failed to connect to Path=%s. ErrorNo=%s\n", domainSocketAddress->address->sun_path, strerror(errno));
cleanup(domainSocketAddress->fd, domainSocketAddress->address->sun_path);
exit(errno);
}
char line[1024];
while(fgets(line, 1024, stdin) != NULL) {
int size = 0;
for(;line[size] != '\0'; size++){
}
if(size == 0) {
break;
}
int bytes = write(domainSocketAddress->fd, line, size);
}
cleanup(domainSocketAddress->fd, domainSocketAddress->address->sun_path);
exit(0);
}
The Server
The server follows the same pattern with that of the client. The only difference is the system calls involved in order to bind to the opened socket and start listening. More specifically, after the socket is created
a call to bind() connects the file descriptor with the address of the socket. Then a call to listen() allows the socket to wait
for incoming connections, and finally a call to accept() accepts the first enqued connection request and retrieves a file descriptor for that connection. That file descriptor can
be passed in the read() system call to read incoming bytes. Note, that we had to call accept() because we marked the domain socket as a SOCK_STREAM one,
hence effectively a connection oriented socket.
#include "stdlib.h"
#include "stdio.h"
#include "string.h"
#include "stddef.h"
#include "unistd.h"
#include <sys/socket.h>
#include <sys/types.h>
#include <sys/un.h>
#include "sys/syscall.h"
#include <errno.h>
#include "af_unix_sockets_common.h"
/*
* Open a ```AF_UNIX``` socket on the ```path``` specified. ```#bind()``` to that address, ```#listen()``` for incoming connections and ```#accept()```. Finally, wait for input from the socket and print
* that to the ```stdout```. When one connection is closed, wait for the next one.
*/
void server(char *path) {
printf("Starting AF_UNIX server on Path=%s\n", path);
AFUnixAddress *domainSocketAddress = open_af_unix_socket(path);
int hasBind = bind(domainSocketAddress->fd, (struct sockaddr *)domainSocketAddress->address, sizeof(struct sockaddr));
if(hasBind == -1){
fprintf(stderr, "Failed to bind AF_UNIX socket on Path=%s. ErrorNo=%d\n", path, errno);
cleanup(domainSocketAddress->fd, path);
exit(errno);
}
int isListening = listen(domainSocketAddress->fd, 10);
if(isListening == -1) {
fprintf(stderr, "Failed to listen to AF_UNIX socket on Path=%s. ErrorNo=%d\n", path, errno);
cleanup(domainSocketAddress->fd, path);
exit(errno);
}
fprintf(stdout, "Start accepting connections on Path=%s\n", path);
while(TRUE) {
int connFd = accept(domainSocketAddress->fd, NULL, NULL);
if(connFd == -1) {
fprintf(stderr, "Error while accepting connection. Error=%s, ErrorNo=%d\n", strerror(errno), errno);
cleanup(domainSocketAddress->fd, path);
exit(errno);
}
char buf[BUFSIZ];
while(TRUE){
int bytes = read(connFd, buf, BUFSIZ);
if(bytes <= 0) {
fprintf(stdout, "Connection closed\n");
break;
}
write(1, buf, bytes);
}
}
cleanup(domainSocketAddress->fd, path);
}
The Main Method
The file that contains the program’s main method, reads the command line arguments, does some rudimentary parsing and based on what was passed creates an AF_UNIX socket server or client.
#include "stdlib.h"
#include "stdio.h"
#include "string.h"
#include "stddef.h"
#include "unistd.h"
#include <sys/socket.h>
#include <sys/types.h>
#include <sys/un.h>
#include "sys/syscall.h"
#include <errno.h>
#include "af_unix_sockets_common.h"
const static char *USAGE = "Usage af_unix_socket --type [server|client] --path [path]\n";
int main(int argc, char **argv){
if(argc != 5) {
fprintf(stderr, "%s", USAGE);
exit(-1);
}
char *type = NULL;
char *path = NULL;
char *nextParam;
int idx=1;
int expectFlag = TRUE;
while(idx < 5) {
if(strstr(&(*argv[idx]), "--") != NULL) {
if(strcmp("--type", &(*argv[idx])) == 0) {
nextParam = (char *)malloc(32);
type = nextParam;
} else if(strcmp("--path", &(*argv[idx])) == 0) {
nextParam = (char *)malloc(32);
path = nextParam;
} else {
fprintf(stderr, "%s", USAGE);
exit(-1);
}
expectFlag = FALSE;
} else {
if(expectFlag) {
fprintf(stderr, "Expected flag for positional argument %d. %s", idx, USAGE);
exit(-1);
}
size_t paramSize = sizeof(&(*argv[idx]));
memcpy(nextParam, &(*argv[idx]), paramSize);
}
++idx;
}
if(type == NULL || path == NULL) {
fprintf(stderr, "%s", USAGE);
exit(-1);
}
fprintf(stdout, "Initializing AF_UNIX for Type=%s, Path=%s\n", type, path);
if(strcmp(type, SERVER) == 0) {
if(access(path, F_OK) != -1) {
fprintf(stdout, "File=%s already exists. Deleting file to be used by AF_UNIX server\n", path);
if(remove(path) != 0) {
fprintf(stderr, "Failed to remove existing File=%s. File cannot be used for AF_UNIX server\n", path);
exit(-1);
}
}
server(path);
} else if(strcmp(type, CLIENT) == 0) {
client(path);
} else {
fprintf(stderr, "Unknown Type=%s\n", type);
}
return 0;
}
Compiling And Running The Program
As now we have all the building blocks, we can actually compile, link and run the program. Based on the operating system someone is running they will need a compatible compiler. Some standard choices are GCC or Clang.
I have personally used Clang as it is the standard on a MacOS system.
Compiling and linking the different files together can plainly be done by:
clang af_unix_sockets_common.c af_unix_sockets.c af_unix_sockets_client.c af_unix_sockets_server.c -o af_unix_sockets
Running the server:
af_unix_sockets --type server --path /tmp/af_unix_socket_example
Running the client:
af_unix_sockets --type client --path /tmp/af_unix_socket_example
Why Part I
This post is named Part I. The main reason behind this is that this program only accepts one connection at a time. A more scalable way of writing this program is by using a selector mechanism (i.e. select(), epoll(), kqueue()) which will allow for connection multiplexing and concurrent handling of those connections.
This will be described in a Part II of this blog and hopefully it will not need many alterations on the code above.