What is meant by persistent storage for program objects?

In object technology, a persistent object is one that continues to exist after the program that created it has been unloaded. An object's class and current state must be saved for use in subsequent sessions. See object-oriented programming.

Persistent storage (aka non-volatile storage) is any storage device that retains data after power to the device is turned off. Examples of non-volatile storage used to persist data are flash memory, hard disk, tape or optical media. Persistent storage also comes in the form of file, block or object storage. Because persistence is assumed, this property of storage systems is rarely mentioned.

Persistent Storage and Containers

Awareness of storage persistence arise recently due to the rapid adoption of Docker containers for developing, building and deploying applications. Although initially assumed to be stateless, it became obvious there are valuable use cases (e.g. databases), where data must persist beyond the lifecycle of a container. This realization gave rise to different concepts of persistent storage for containers (e.g. bind mounts, data containers and docker volumes) for running stateful applications.

How Persistent Storage Works on Kubernetes and Docker Swarm

Today, persistent storage for container options have greatly expanded. The use of volume plugins enable cloud native storage, as well as legacy software defined and enterprise storage, to provide data persistence for containerized applications.

A volume plugin integrates container orchestration systems, such as Kubernetes and Docker Swarm, with external storage systems through an API. This API enables persistent volumes to be natively provisioned and then attached to an application running in a container or pod. Since the persistent volume survives termination of the container or pod, stateful data written by the application persists beyond the lifetime of the container.

In particular, the industry has coalesced around the Container Storage Interface (CSI) specifications to unify the storage interface for container orchestration systems.

Kubernetes

For Kubernetes, the CSI volume plugin is a step away from the original in-tree plugin to a system that is more extensible. Using CSI, third-party storage providers can write and deploy plugins exposing new storage systems in Kubernetes without having to touch the core Kubernetes code. The adoption of CSI also adds dynamic provisioning of volumes to Kubernetes.

Users manage storage in Kubernetes through three cluster resources:

• PersistentVolumeClaim (PVC), a request for storage
• PersistentVolume (PV), storage with lifecycle independent of any pod
• StorageClass, enables delivering storage with different classes of service

To dynamically provision persistent volumes, pods make requests for storage through a PVC. The definition file (usually in yaml format) for the PVC references a StorageClass. When deployed this passes a request, to a volume plugin corresponding to the StorageClass, to provision a PV with a specified set of attributes. Upon creation, the PV is bound to the pod that requested the PVC.

Docker Swarm

Docker integrates with external storage through volume plugins. While volumes can be completely managed by Docker, external storage provides greater resilience and data services options. The volume plugin API enables third party storage providers to write plugins to support stateful applications.

Persistent volumes for stateful apps can be pre-provisioned using `docker volume create`, or dynamically with `docker run` and `docker service create` commands. Volumes can also be spec into Docker compose or stack files and provisioned with `docker stack deploy`.

The volume driver parameter in the Docker CLI commands identifies which plugin is passed the call to create the volume. After creation, the volume is attached to the node where the container is started, and passed to the container through a propagated mount managed by Docker. For examples of how to specify the volume plugin and volume attributes, follow this link.

Storidge

Storidge supports stateful apps with volume plugins for both Kubernetes and Docker Swarm. The persistent volumes are dynamically provisioned from a storage abstraction layer managed by the Storidge CIO software.

Attributes to consider: 

Automation

Degree of automation so deployment is not IT bound

Performance Guarantees

Ability to isolate and provide guaranteed performance for apps

Data locality

How does data locality work to ensure consistent app performance?

Developer centric

How well does it work with developer workflows?

Multi-cloud

Does it work the same way on core, edge and cloud?

Storage-as-a-Service

Is storage delivered as a cloud like service?

Different Options for Persistent Storage

The availability of volume plugins greatly expand persistent storage choices for containerized apps. These choices include newer cloud native storage implementations that were purpose built for orchestrated systems, as well as, legacy software defined storage and enterprise systems.

These persistent storage options present a breath of platform, cost, maturity, and integration options that enterprises should consider as they transition towards containerized applications, and microservices architecture.

In a future blog, we will compare the capabilities and attributes of different storage technologies, and how well they fit in a modern application environment that is orchestrated, scalable and dynamic.

What is persistent storage of objects in Python?

What are 3 types of persistent storage?

What is a persistent store?

What is persistence in programming?