Unit 1 · Chapter 4 · Book pp. 95–120

Elementary TCP Sockets

The calls themselves. Everything in Chapter 1 was a preview of this: the six functions that build a connection, plus the four that let one server handle many clients at once. Any "write a C program" question in the paper is assembled from exactly these.

Figure 4.1Socket functions for elementary TCP client/server

Learn the shape of this figure — a "write a C program" question is asking you to reproduce one side of it. The server's column is four calls before it can do any work; the client's is two.

The `socket` function

§4.2 · p.95

#include <sys/socket.h>

int socket(int family, int type, int protocol);
              Returns: non-negative descriptor if OK, -1 on error

`family`	Meaning
`AF_INET`	IPv4 protocols
`AF_INET6`	IPv6 protocols
`AF_LOCAL`	Unix domain protocols
`AF_ROUTE`	Routing sockets
`AF_KEY`	Key socket

`type`	Meaning
`SOCK_STREAM`	Stream socket — a reliable, ordered byte stream
`SOCK_DGRAM`	Datagram socket
`SOCK_SEQPACKET`	Sequenced packet socket — reliable, ordered, and message boundaries preserved
`SOCK_RAW`	Raw socket — bypass the transport layer

What you want	family	type	protocol
TCP over IPv4	`AF_INET`	`SOCK_STREAM`	`0` or `IPPROTO_TCP`
UDP over IPv4	`AF_INET`	`SOCK_DGRAM`	`0` or `IPPROTO_UDP`
SCTP over IPv4	`AF_INET`	`SOCK_STREAM` or `SOCK_SEQPACKET`	`IPPROTO_SCTP`
TCP over IPv6	`AF_INET6`	`SOCK_STREAM`	`0` or `IPPROTO_TCP`

A 0 for protocol means "you choose" — and for AF_INET + SOCK_STREAM there is only one sensible answer, TCP. SCTP is the exception: it shares SOCK_STREAM with TCP, so you must name IPPROTO_SCTP explicitly.

AF_ is address family, PF_ is protocol family. The original intent was that one protocol family might support several address families, and socket would take a PF_ value while a socket address structure would hold an AF_ one. That never happened: every system defines PF_INET to equal AF_INET. The book uses AF_ throughout, and so should you. §4.2 · p.98

The `connect` function

§4.3 · p.99

#include <sys/socket.h>

int connect(int sockfd, const struct sockaddr *servaddr,
            socklen_t addrlen);
                                Returns: 0 if OK, -1 on error

The client does not have to call bind first — the kernel chooses an ephemeral port, and a source IP address, as part of connecting. connect initiates TCP's three-way handshake and returns only when the connection is established or an error occurs. There are three error cases worth knowing:

What happens	Error	Meaning
No response to the SYN at all	`ETIMEDOUT`	4.4BSD sends a SYN, waits 6 seconds, sends another, waits 24 more, gives up after 75 seconds. Usually a host that is down or unreachable and silently dropping.
An RST comes back	`ECONNREFUSED`	A hard error. The host is up but nothing is listening on that port. Returned immediately — no retry.
An ICMP destination-unreachable	`EHOSTUNREACH` or `ENETUNREACH`	A soft error. The kernel keeps retrying for a while first, in case the routing problem is transient.

When is an RST generated?: Three conditions. (1) A SYN arrives for a port with no listening server. (2) TCP wants to abort an existing connection. (3) TCP receives a segment for a connection that does not exist.

If connect fails, the socket is in an unspecified state. You must close the descriptor and call socket again — you cannot simply retry connect on the same descriptor.

The `bind` function

§4.4 · p.101

#include <sys/socket.h>

int bind(int sockfd, const struct sockaddr *myaddr,
         socklen_t addrlen);
                                Returns: 0 if OK, -1 on error

bind assigns a local protocol address to a socket — for IPv4 and IPv6, a 32- or 128-bit address plus a 16-bit port. You may specify a port, an address, both, or neither:

Address	Port	Result
Wildcard (`INADDR_ANY`)	`0`	Kernel chooses both the IP address and the port.
Wildcard	Non-zero	Kernel chooses the IP address; you choose the port. This is what a normal server does.
Local IP	`0`	You choose the IP address; the kernel chooses the port.
Local IP	Non-zero	You choose both.

The wildcard address: INADDR_ANY, whose value is 0. It tells the kernel to accept a connection destined for any local interface. For IPv6 the equivalent is the in6addr_any variable, because a 128-bit constant cannot be represented as an initialiser — so you write serv.sin6_addr = in6addr_any; rather than an assignment of a numeric constant.

servaddr.sin_addr.s_addr = htonl(INADDR_ANY);   /* wildcard */

Yes, htonl is applied to a value of 0, where it makes no difference. Do it anyway. It documents that the field is network-ordered, and it means that changing the constant later does not introduce a bug.

EACCES — binding a port in the reserved range (below 1024) without superuser privilege. EADDRINUSE — the address is already in use, which is very often a TIME_WAIT socket from a previous run of the same server. See SO_REUSEADDR.

The `listen` function and the two queues

§4.5 · p.104

#include <sys/socket.h>

int listen(int sockfd, int backlog);
                                Returns: 0 if OK, -1 on error

listen is called only by a TCP server, and it does exactly two things:

A socket created by socket is assumed to be active — a client socket that will issue a connect. listen converts an unconnected socket into a passive socket, telling the kernel it should accept incoming connection requests directed to it. In terms of the state transition diagram, listen moves the socket from CLOSED to LISTEN.
Its second argument specifies the maximum number of connections the kernel should queue for this socket.

It must be called after socket and bind, and before accept.

The two queues

To understand backlog you have to know that for a given listening socket the kernel maintains two queues, not one. This is a frequently-asked question.

Figure 4.7The two queues maintained by TCP for a listening socket · §4.5 · p.105

Two queues, one backlog. Historically the sum of both was capped at backlog; modern systems generally apply the limit to the completed queue and use a separate mechanism for the incomplete one.

Incomplete connection queue: One entry for each SYN that has arrived from a client for which the server is awaiting completion of the three-way handshake. These sockets are in the SYN_RCVD state.
Completed connection queue: One entry for each client with whom the three-way handshake has completed. These sockets are in the ESTABLISHED state.

The life of one connection through the queues:

A SYN arrives. TCP creates a new entry on the incomplete queue and responds with the second segment of the handshake — its SYN plus an ACK of the client's SYN. The parameters from the listening socket are copied to the new connection. The connection creation mechanism is completely automatic; the server process is not involved.
The entry stays on the incomplete queue until the third segment arrives, or until it times out. Berkeley-derived implementations use a timeout of 75 seconds for incomplete entries.
If the handshake completes normally, the entry moves from the incomplete queue to the end of the completed queue.
When the process calls accept, the first entry on the completed queue is returned. If that queue is empty, the process is put to sleep until an entry appears.

Because a half-finished handshake and a finished one are not the same thing. A finished one is a real connection with a real client on the other end, ready to be handed to the application. An unfinished one may never finish — the client may have vanished, or may never have existed. Keeping them apart means a flood of unanswered SYNs fills only the incomplete queue, and legitimate completed connections still get through to accept.

backlog has never been the number of clients a server can handle — it is the number of connections the kernel may queue before accept collects them. A busy server with a backlog of 5 can serve thousands of clients; it just cannot have more than a handful waiting to be accepted at any one instant. The book's unp.h defines LISTENQ as 1024.

The `accept` function

§4.6 · p.109

#include <sys/socket.h>

int accept(int sockfd, struct sockaddr *cliaddr,
           socklen_t *addrlen);
       Returns: non-negative descriptor if OK, -1 on error

accept returns the next completed connection from the completed queue. Two things about it are examinable.

Two descriptors, two jobs: sockfd going in is the listening socket; the return value is a brand-new connected socket. The kernel creates one connected socket per client accepted, and it is closed when the server has finished with that client. The listening socket normally stays open for the lifetime of the server.
addrlen is a value-result argument: Going in it says how big your buffer is; coming out it says how much the kernel stored. See §3.3. If you are not interested in the client's identity, pass NULL for both cliaddr and addrlen — as the daytime server in Chapter 1 does.

Q4(a)(ii) [10] asked for a Daytime server that prints the client's IP address and port. That is precisely this: stop passing NULL, pass a real sockaddr_in and a socklen_t, then convert with inet_ntop and ntohs. Worked in full in the solved paper.

Concurrent servers: `fork` and `close`

§4.7 · p.111

#include <unistd.h>

pid_t fork(void);
   Returns: 0 in child, process ID of child in parent, -1 on error

fork is called once and returns twice — once in the parent and once in the child, because after the call there are two processes running the same code. The return value is how each one works out which it is. The child gets 0 (it can always find its parent with getppid); the parent gets the child's process ID (it has no other way to learn it, and may have many children).

Descriptors are shared: All descriptors open in the parent before the fork are shared with the child afterwards. This is exactly what makes a concurrent server possible: the child inherits connfd and can talk to the client without any handover.
The reference count: After the fork, both parent and child hold listenfd and connfd, so each descriptor has a reference count of 2. close only decrements the count; the FIN is sent when it reaches zero.

The canonical shape of a concurrent server:

The `close` function

§4.9 · p.117

#include <unistd.h>

int close(int sockfd);
                                Returns: 0 if OK, -1 on error

The default action of close on a TCP socket is to mark it as closed and return to the process immediately. The descriptor can no longer be used, but TCP will still try to send any data already queued, and then go through the normal termination sequence. If you need close to wait for that, see SO_LINGER.

In a concurrent server, close on a shared descriptor decrements the reference count. The FIN is sent only when the count reaches zero. This single fact explains both why the parent must close connfd and why the child closing listenfd is harmless.

If you want to send a FIN without waiting for the count to drop, that is what shutdown is for — see §6.6.

The `exec` family

fork makes a copy of the current program; exec replaces the current program with a different one. There are six variants, distinguished by three questions: is the file found via PATH (p), is the argument list a list or a vector (l or v), and is the environment passed explicitly (e)?

Function	Filename or pathname	Argument list	Environment
execl	pathname	list	inherited
execv	pathname	vector	inherited
execle	pathname	list	passed
execve	pathname	vector	passed
execlp	filename	list	inherited
execvp	filename	vector	inherited

exec returns to the caller only if an error occurs — on success the calling program no longer exists, so there is nothing to return to. Descriptors normally stay open across an exec, unless the FD_CLOEXEC flag is set on them; this is exactly how inetd hands a connected socket to a server it launches.

getsockname and getpeername

§4.10 · p.118

#include <sys/socket.h>

int getsockname(int sockfd, struct sockaddr *localaddr,
                socklen_t *addrlen);

int getpeername(int sockfd, struct sockaddr *peeraddr,
                socklen_t *addrlen);
                                Both return: 0 if OK, -1 on error

getsockname: Returns the local protocol address associated with a socket.
getpeername: Returns the foreign protocol address associated with a socket.

Note that the final argument of both is a value-result argument — these are two of the four functions in that list. And the word "name" is misleading: they return a protocol address, which for IPv4 and IPv6 is an IP address plus a port number. They have nothing to do with domain names.

The book gives five reasons these functions are required:

After connect succeeds in a TCP client that did not call bind, getsockname returns the local IP address and local port the kernel assigned.
After calling bind with a port number of 0 — telling the kernel to choose — getsockname returns the port that was actually assigned.
getsockname can be called simply to obtain the address family of a socket.
In a server that binds the wildcard address, once a connection is established the server can call getsockname to find which local IP address the connection actually landed on. The descriptor in this call must be the connected socket, not the listening socket.
When a server is execed by the process that called accept, the only way it can learn the client's identity is getpeername — which is exactly what happens when inetd forks and execs a TCP server.

In the concurrent server, why must the parent close connfd, and what actually breaks if it does not?

Two separate failures, and a good answer names both.

Resource exhaustion. The parent gains one descriptor per client and never releases it, so eventually it runs out of descriptors and accept starts failing.

The connection never terminates. A FIN is sent only when a descriptor's reference count reaches zero. If the parent keeps its copy open, the count stays at 1 even after the child closes and exits — so no FIN is sent, and the client waits forever for a close that will never come.

A client calls connect and gets ECONNREFUSED instantly, but a different address hangs for 75 seconds then gives ETIMEDOUT. What is different?

ECONNREFUSED means an RST came back: the host is up and reachable, and it actively said "nothing is listening on that port". There is nothing to retry, so the error is returned immediately. This is a hard error.

ETIMEDOUT means nothing at all came back. The host may be down, unreachable, or dropping packets silently, so the kernel retransmits the SYN — 4.4BSD sends one, waits 6 seconds, sends another, waits 24 more — before giving up at 75 seconds. Silence is ambiguous, so TCP is patient with it.

Elementary TCP Sockets

The socket function

The connect function

The bind function

The listen function and the two queues