Cloud Backup

NDMP: A Brief History, Architecture, and Common Topologies

December 16, 2021

Topics: Cloud Volumes ONTAP Elementary7 minute read

What Is NDMP?

Network Data Management Protocol (NDMP) is an open network protocol for controlling communications between your main storage and secondary storage systems. It is commonly used for NAS backup and recovery operations in heterogeneous network environments.

NDMP provides a shared architecture for backing up network file servers, making it possible to run backups on file servers running on multiple platforms, using one agent and centralized management software.

NDMP separates the control path from the data path, which reduces the demand on network resources and allows you to implement local backups and leverage remote systems for disaster recovery. NDMP also creates a standard way for file servers in heterogeneous networks to communicate with network-attached tape devices.

In this article, you will learn:

A Brief History of NDMP

More than 20 years ago, NetApp and Intelliguard w  orked together to solve a frustrating problem for NAS users—the inability to reliably protect data.

Previously, organizations would back up NAS storage by installing a NAS share on the backup server, and moving the backup data to a locally attached tape device or network-attached tape library. This was challenging, due to the difficulty of managing multiple devices, reduced performance caused by the need to read all files through the POSIX interface, and bottlenecks resulting from the transfer of data through a single mount point.

NDMP was proposed by NetApp and Intelliguard in 1995 to address these issues and make it easier to protect data on NAS devices. The protocol stipulated that the control path be separated from the data path:

  • Control traffic is transferred from the backup application to the NAS platform over the IP network
  • Data traffic is transferred from the NAS platform directly to storage media over SCSI or Storage Area Network (SAN)

NDMP also defines a mechanism that allows backup applications to initiate and manage backup jobs running on multiple NAS devices. Each NAS device then prepares the files for protection and copies the files directly to a locally attached storage device or a networked storage device.

The benefits of NDMP are:

  • Offloads backup traffic to locally attached devices or devices attached to a fiber channel, avoiding data transfer bottlenecks on networks that had very limited bandwidth at the time of NDMP’s release.
  • Standardized backup agents, so data management vendors no longer need to create special agents or unique device drivers for each solution offered by a different NAS vendor. All NAS manufacturers can build interfaces that comply with the open NDMP standard.
  • Enables centralized data management, so that a single instance of the backup application can initiate and manage data protection for multiple NAS devices.

NDMP Architecture

NDMP specifies functional interfaces that can be used for heterogeneous data flows. Vendors can leverage NDMP to provide common interfaces, to achieve a common architecture. These interfaces allow data to flow from file systems to any backup device—regardless of the specific backup software used, file and control metadata can be consistently transferred to and from backup storage.

As a network protocol, NDMP defines communications between servers and backup software. A set of specified interfaces are used to define communications in the form of External Data Representation (XDR) encoded messages, exchanged over bidirectional TCP/IP connections.

NDMP architecture uses a client/server model, with the backup software acting as the client. The NDMP server is a host, maintaining virtual state machines for each connection with the backup software.

Server vendors can implement NDMP by incorporating the following interfaces into their operating system code:

  • Configure—enables backup software to discover the NDMP server configuration
  • Connect—enables authentication of a client and sets the protocol version to be used
  • SCSI—enables control of an SCSI device (i.e a jukebox) at a low level
  • Data—handles the backup data format, specifying how the backup software can initiate backup and restore operations
  • Tape—enables programmatic control over a tape drive

The client, or backup software, uses the following NDMP interfaces:

  • Notify—notifies the client if the NDMP server needs attention
  • File History—enables the NDMP server to add entries to the backup software’s file history, which is important for file retrieval later on
  • Logging—provides messages that the operator uses to monitor the progress of backups or diagnose issues

NDMP Backup Topologies

There are several topologies for tape services supported by NDMP: 

  • Storage system to local tape—this is the most basic configuration. The backup application backs up the data from your primary storage system to a physically attached tape. There is an NDMP control connection that crosses the network boundary. An NDMP local configuration manages the data connection between the data storage and the local tape service.
  • Storage system to a tape attached to a different storage system—the backup application can back up stored data to a tape library—a media changer with multiple tape drives—attached to a different storage system. This is known as an NDMP three-way configuration (storage system-to-storage system). A TCP or TCP/IPv6 network connection links the data to the tape service.
  • Storage system to a network-attached tape library—an NDMP-enabled tape library offers a three-way configuration, with the library attached directly to a TCP/IP network, allowing it to communicate with your storage system and backup application through the internal NDMP server.
  • Storage system to data server to tape—enables the backup of data from your storage system to a tape library attached to a backup application or to a different data server system.
  • Data server to storage system to tape—enables the backup of data from the server to a tape library that is attached directly to your storage system.

NDMP Challenges

A certainty about NAS is that in today’s climate data, Volumes have increased exponentially. Previously, the main focus was standalone storage servers, but today there is petabyte-level, scale-out NAS.

Traditionally, you couldn't put agents onto a NAS system as they were traded as appliances alongside proprietary operating systems (including ONTAP). Files systems in a NAS system can increase to a large size and if backups must be reinstated, this may require rebuilding file systems. This can take a long time if the file systems have become very large.  

NDMP made things simple for a period of time and dealt with certain issues regarding traffic problems with direct backup - currently, however, a number of issues persist. NDMP still doesn’t support interoperability from one NAS vendor to another, so you can’t move information to a different file system. This leaves you locked-in with one NAS vendor.

NDMP is mainly suited to file information and is not a good fit for database applications. It is limited to a finite number of incremental backups, beyond which you need to undertake a full backup. This is due to the fact that NDMP was created for use with tape and incrementals are restricted because they are an involved process.    

NetApp Cloud Backup: A Modern NDMP Alternative

NetApp Cloud Backup is a backup and restore service for NetApp Cloud Volumes deployments and on-premises ONTAP clusters. Integrated into NetApp Cloud Manager, Cloud Backup is easily enabled, automated, and scaled allowing you to keep your data safe and compliant, overcoming traditional industry challenges.

Leveraging NetApp’s SnapMirror Cloud replication technology, backups are transferred and stored in a highly durable cloud-based object storage. Backups are automatically generated and stored in an object store within your cloud account, independent of volume Snapshot copies used for near-term recovery or cloning, so that you can effortlessly restore data anytime and to anywhere you need it.

By preserving storage efficiencies and performing block level incremental updates forever, Cloud Backup guarantees minimal data footprint to transfer, leading to optimal bandwidth consumption, reduced performance impact on production and meeting SLA.

The highest level of security as backup copies are stored in your own object storage. As well, data is end-to-end encrypted with AES-256-bit encryption at-rest and with TLS 1.2 HTTPS connections when in-flight.

Get Started with NetApp Cloud Backup

New call-to-action
Denisse Soker, Cloud Backup

Cloud Backup

-