Laboratory 2. TCP and UDP Sockets in ESP-IDF

Goals

Introduction

In the previous laboratory, we studied how to develop simple client/server systems using Python, both for TCP and for UDP. In this lab, we will study and develop network components (TCP and UDP clients and servers) that can execute on the ESP32 leveraging the facilities offered by ESP-IDF. Also, we will demonstrate that it is possible to interact clients and servers executing on the virtual machine (programmed via Python) and on the board (using the C sockets API).

The C sockets API

Funtions for byte ordering

As TCP/IP is a universal standard, and it allows for communicating across virtually any platform and architecture, it is necessary to get a byte ordering method so that big-endian and little-endian machines can communicate in a transparent and correct way. To accomplish this requirement, routines are usually provided to reorder and adapt byte ordering. In platforms in which data are already correctly ordered, these functions do not present any special functionality, but anyway, its usage is necessary so that the communication among pairs is correct.

Typical functions for data reordering are: htons, htonl, ntohs y ntohl. Their name explains their semantics: host to network (short) host to network (long), network to host (short) and network to host (long), converting datatypes short and long from the format used in network transmissions (network) to a host representation. Hence, when we send binary data over the network, it will need to be transformed using hton* and upon receiving it, using ntoh*.

Data structures

Before studyin the sockets API, it is necessary to show the goal of a set of data structures used in all of them. The most important is sockaddr_in, defined as follows:

struct sockaddr_in
{
    short          sin_family;
    u_short        sin_port;
    struct in_addr sin_addr;
    char           sin_zero[8];
};

The structure in_addr used in sockaddr_in is defined as:

struct in_addr
{
    u_long s_addr;
};

This one consists on a field of type unsigned long int that contains the IP address associated with the socket.

The structure sockaddr_in contains two important fields:

Basic API

socket()

int socket(int family, int type, int protocol);

bind()

int bind(int sockfd, const struct sockaddr *addr, socklen_t addrlen);

listen()

int listen(int sockfd, int backlog);

accept()

int accept(int sockfd, struct sockaddr *addr, socklen_t *addrlen);

connect()

int connect(int sockfd, const struct sockaddr *addr, socklen_t addrlen);

send()

ssize_t send(int sockfd, const void *buf, size_t len, int flags);

recv()/recvfrom()

ssize_t recv(int sockfd, void *buf, size_t len, int flags);

ssize_t recvfrom(int sockfd, void *buf, size_t len, int flags,
                 struct sockaddr *src_addr, socklen_t *addrlen);

close()

int close(int fd);

Examples

In the following, we propose a number of complete examples that illustrate the use of the sockets API in C for the development of client/server systems. For each one, check that, effectively, the use and sequence of application of each call follows the directives in the figure:

flow

Task 1.1

Compile (gcc example.c -o example.x) and execute (./example.x) each pair of codes and check its correct functionality. Study carefully the use of each routine and how the previous directives are followed.

Example: a TCP client

#include <arpa/inet.h>
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <unistd.h>
#include <netinet/in.h>


int main() {
        const int server_port = 9000;

        struct sockaddr_in server_address;
        memset(&server_address, 0, sizeof(server_address));
        server_address.sin_family = AF_INET;

        server_address.sin_addr.s_addr = inet_addr("127.0.0.1");
        server_address.sin_port = htons(server_port);

        int sock;
        if ((sock = socket(PF_INET, SOCK_STREAM, 0)) < 0) {
                printf("Error in socket\n");
                return 1;
        }

        if (connect(sock, (struct sockaddr*)&server_address,
                    sizeof(server_address)) < 0) {
                printf("Error in connect\n");
                return 1;
        }

        const char* data_to_send = "Hello, NP2!!";
        send(sock, data_to_send, strlen(data_to_send), 0);

        int n = 0;
        int len = 0, maxlen = 100;
        char buffer[maxlen];
        char* pbuffer = buffer;

        while ((n = recv(sock, pbuffer, maxlen, 0)) > 0) {
                pbuffer += n;
                maxlen -= n;
                len += n;

                buffer[len] = '\0';
                printf("Received: '%s'\n", buffer);
        }

        close(sock);
        return 0;
}

Example: a TCP server

#include <arpa/inet.h>
#include <netinet/in.h>
#include <stdbool.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>

int main(int argc, char *argv[]) {
        int SERVER_PORT = 9000;

        struct sockaddr_in server_address;
        memset(&server_address, 0, sizeof(server_address));
        server_address.sin_family = AF_INET;

        server_address.sin_port = htons(SERVER_PORT);

        server_address.sin_addr.s_addr = htonl(INADDR_ANY);

        int listen_sock;
        if ((listen_sock = socket(PF_INET, SOCK_STREAM, 0)) < 0) {
                printf("Error in socket\n");
                return 1;
        }

        if ((bind(listen_sock, (struct sockaddr *)&server_address,
                  sizeof(server_address))) < 0) {
                printf("Error in bind\n");
                return 1;
        }

        int wait_size = 16;  

        if (listen(listen_sock, wait_size) < 0) {
                printf("Error in listen\n");
                return 1;
        }

        struct sockaddr_in client_address;
        int client_address_len = 0;

        while (true) {
                int sock;
                if ((sock =
                         accept(listen_sock, (struct sockaddr *)&client_address,
                                &client_address_len)) < 0) {
                        printf("Error in accept\n");
                        return 1;
                }

                int n = 0;
                int len = 0, maxlen = 100;
                char buffer[maxlen];
                char *pbuffer = buffer;

                printf("Cliente conectado con IP: %s\n",
                       inet_ntoa(client_address.sin_addr));

                while ((n = recv(sock, pbuffer, maxlen, 0)) > 0) {
                        pbuffer += n;
                        maxlen -= n;
                        len += n;

                        printf("Received: '%s'\n", buffer);

                        send(sock, buffer, len, 0);
                }

                close(sock);
        }

        close(listen_sock);
        return 0;
}

Task 1.2

Reproduce the logic of the previous client/server echo system using UDP.

Message construction

In order to send messages that encapsulate different types of data in one invocation, you can define a message as follows:

typedef struct {
  int x;
  int y;
} message;

Giving value to each field and sending the structure offering the address of the structure:

message.x = x; message.y = y;
send( socketfd, &message, sizeof( message ), 0 );

Task 1.3

Modify the UDP client to encapsulate and send a structure with different fields (for example, two integers), that will be received by a Python server following the directives of Lab 1. In this case, do not use fields of floating point type (we will see how to do it in the future). The goal of the Task is to demonstrate that a client programmed in C and a server programmed in Python can communicate transparently. Hence, it is not expected from you to develop a complex system.

Client/server systems on the ESP32

The reason behind the previous exercises lies on the fact that the TCP/IP stack implemented in ESP-IDF (Lightweight TCP/IP (lwIP)) implements almost at 100% that API. Hence, the basic firmware structure for a client/server and its API remains unmodified.

In this last section, we will work with two basic examples of implementation of client/server systems TCP and UDP on the ESP32, with the goal of studying its functionality, check its interoperability and perform modifications to adapt them to a hypothetical IoT application.

UDP client/server on the ESP32

In this part, you will work with two examples provided within the examples collection from ESP-IDF. Hence, copy in your workspace (out of the main ESP-IDF tree) both examples:

General structure

Observe the codes (udp_server.c for the server, and udp_client.c for the client). Check that both the basic structure of both components and the invocations to the sockets API match with those seen for the echo system programmed in C.

Regarding the main task (function app_main) observe that it performs a series of invoations to configuration APIs of some subsystems from FreeRTOS, mainly:

// Initializes the NVS (Non-volatile storage) by default.
ESP_ERROR_CHECK(nvs_flash_init());
// Initializes the ESP-NETIF infrastructure.
ESP_ERROR_CHECK(esp_netif_init());
// Creates the main default event loop.
ESP_ERROR_CHECK(esp_event_loop_create_default());

/* This funtion configures WiFi or Ethernet, as selected via menunconfig.
*/
ESP_ERROR_CHECK(example_connect());

xTaskCreate(udp_server_task, "udp_server", 4096, NULL, 5, NULL);

Deployment. Option 1

In this case, you will deploy a client on an ESP32 and a server in the other. Obviously, both ESP32s must be part of the same wireless network, so they will be connected to the same access point (at home or at class, you can use a mobile phone for that). Configure the following point in the infrastructure:

At this point, you can boot the client and you should be communicating two ESP32 nodes via UDP.

Deployment. Option 2

If you only have one node, or just want to test other way of communication between a PC node and an ESP32, you can use any of the system tools:

Note

Take into account that your PC (that is, the virtual machine) and the ESP32 must be part of the same network. To accomplish it, stop your virtual machine and add a new network interface of type bridge connected to the WiFi interface of your PC. Proceeding this way, you will have an interface with IP within the network, granted directly by your access point.

nc -ul -p 3333

nc -u IP_REMOTE 3333

In the scripts folder of the examples folder, you can find small client/server UDP Pythhon exmaples that you can also use.

TCP client/server on the ESP32

The deployment of the client and server in their TCP version is equivalent to UDP.

nc -l IP -p 3333

nc IP 3333

Again, you can find TCP Python scripts to use on the scripts folder.

Task

Experiment with the examples provide in ESP-IDF (client/server TCP and UDP) and execute them on the ESP32.

Deliverable task

At this point, you will have a set of codes that implement client/server systems both in a host (using Python and/or C) and on the ESP32 (using C and ESP-IDF), and you should have checked their correct functioning.

Specifically, you should have developed:

  • A client/server system developed for Lab1, written in Python and implementing a basic application-level protocol proposed by you.

  • Basic C code for the implementation of a client/server echo system, with codes given in this Lab.

  • Basic C/ESP-IDF codes to implement client/servers echo on the ESP32.

As a deliverable task, you need to adapt your deliverable of Lab 1 so that both client and server can work on the host (using Python or C) and on the ESP32. You will deliver the developed codes and a short report with screen capture and explanations that demonstrate the correctness of the system.