Distributed File Systems

Objectives

The objective is to provide knowledge of the architecture and design of distributed file systems and the design considerations and trade-offs that have led to them. In Section 12.1 `basic’ distributed file systems are situated in the context of the wide range of distributed data management techniques that are now available. These include distributed shared object systems and various replicated data architectures, which will be covered in later chapters.

This chapter covers the functionality, the APIs and the underlying implementations of basic distributed file systems, emphasising their similarities by the description of a generic file service architecture in Section 12.2. The NFS and AFS systems are then covered in detail. They are both widely used in practice and their designs represent interesting and contrasting tradeoffs.

By studying the two best-known examples and going on the cover the main recent advances in file service design, students should gain an understanding of the impact of requirements for scalability, fault-tolerance and high performance on the design of practical distributed services and of the impact of technological changes such as the advent of high-performance networks on system design.

After studying this chapter, students should have sufficient understanding of distributed file service design and implementation to design and construct a practical file service.

Points to emphasize

The key role of file servers means that failure transparency is important and that the performance of file servers under load must be good.

In our description of the generic architecture in Section 12.2 we introduce the following ideas:

File service components

The role of each of the components – flat file service, directory service and client module; the division of responsibilities between the components results in an exposed interface – the flat file service – that is hidden in conventional operating systems. The useful separation of concerns, openness and modularity achieved by the division of responsibilities.

Design issues

The need for unique file identifiers; file attributes; the benefits of statelessness.

Interfaces

The absence of an Open operation; the idempotent nature of all of the interfaces; comparison of the flat file service with Unix system calls; method for interpreting a multi-part pathname.

File groups

The significance of file groups is that they provide a basis for the distribution of groups of files between different servers and their replication.

The two systems studied in this chapter provide examples of the fact that designs for services can differ radically even when the application programming interface is almost identical. It is for this reason that the design of distributed systems remains an important and interesting activity.

Meaning of one-copy file semantics and the problems that this introduces for caching and replication.

NFS

Remote mounting – what it is, and what it means for location transparency; statelessness of the NFS interface – benefits and drawbacks; software components and architecture – why it is implemented in the kernel; VFS – its role; path name translation; mount service; caching – implications of updates and cache consistency; performance. See also the discussion of recent developments based on NFS in Section 12.5.

AFS

Whole-file serving and caching – compare with ftp; observations of UNIX file usage – especially average and maximum file sizes and locality of reference to files; software structure – roles of Vice and Venus, user-level processes; Vice as a flat file service, with directory structure implemented in Venus – FID structure; implementation of file system calls in AFS.

Callback promises – comparison of AFS-2 with AFS-1; Vice service interface; update semantics – for AFS-1 and AFS-2, and semantics while file is open, potential lost updates; Unix kernel changes; read-only replicas; performance.

Possible difficulties

There may be a difficulty in situating basic file services in relation to distributed object systems and other data management systems. This is best explained by discussing application programming styles and other application requirements, such as synchronous data sharing and disconnected operation.

Another difficulty that may be encountered is in appreciating the very heavy loads that file servers are designed to support and their impact on design.

Teaching hints

The service interfaces defined in this chapter are quite straightforward. They are amongst the first service interfaces that the student will have encountered; we recommend a review of these file service interfaces, comparing them with the Unix file system primitives (see Figure 12.4) and emphasizing their support for the design goal of stateless servers.

The generic model presented in this chapter is sufficiently detailed for it to be used as the basis for a coursework exercise on the implementation of a file service.