What is etcd?

Etcd is a key value store that is intended for distributed applications. Its job is to safely store critical data. It is used in Kubernetes, for example, and offers many advantages.

Etcd is a hierarchical and distributed key-value store (key-value database ). It was written by the Core OS team in Go. It is now managed by the Cloud Native Computing Foundation and is open source .

Etcd was developed, which has been implemented in Kubernetes since 2014 , to provide secure storage space for critical data in distributed applications. The name is made up of “/ etc”, the directory for configuration files in GNU / Linux operating systems, and “d” for distributed.

Distributed applications and etcd

Distributed applications have long since become the standard. The triumphs of the clouds, smartphones and the Internet of Things have contributed to this. It is becoming increasingly rare that the front and back end run on the same physical machine. However, some problems arise in the cloud area in particular: Parts of networks can fail. Data transfer is expensive and finite.

The development and assignment of ever smaller components of applications is also tedious. After all, the user has to know where the individual elements are. Modifiable information must therefore be stored in a safe place that is not affected by these problems. This is where etcd comes in.

How etcd works

There are three relevant concepts for managing storage in application clusters: leaders, elections, and terms. The cluster holds an election and appoints a leader for a specific period of time. This handles all storage requests that require the consent of the cluster.

The leader is the representative of the whole. The inquiries are about changes to saved data. Inquiries that do not require consent (e.g. reading requests) can be answered independently by all members of the cluster. If the leader “dies” (goes down) or no longer responds, the remaining nodes in the cluster will re-elect.

The individual nodes each have their own “stopwatch” that determines how long they wait before calling for a new election and determining themselves as candidates. These periods of time differ from node to node. The background is so that nodes can step in as leaders as quickly as possible if the previous one fails or a whole series of nodes are no longer available.

The system is insecure in that the leader can make changes to stored information on his own. This is where etcd intervenes. The key-value store forces the leader to question all nodes with every change. The raft algorithm is used for this. Only if the majority votes in favor of the modification, it may be carried out.

This is particularly important for distributed applications: The individual nodes of the cluster can block changes that impair their functionality. This ensures the stability of the application. In addition, the number of subsequent problems is minimized.

Advantages of etcd

• easy to use thanks to a REST and JSON based interface
• secure through SSL certificate authentication
• Reliable
• fast (up to 10,000 writes per second)

Recommendations for implementation

Since etcd writes data to drives, it makes sense to work with SSDs. The number of nodes in a cluster should be odd in order to always be able to establish a majority. For performance reasons, it is also advisable not to work with clusters that have more than seven nodes.