The State of NFSv4
Spencer Shepler
Sun Microsystems
6 August 2001
State of NFSv4
¢ Review of current state model
¢ Raise issue with Posix clients and current
state model
¢ Propose solution
W h e re does the State begin?
¢ Hierarchy of NFSv4 state begins with
association between a single client and
server
¢ This hiearchy is important because the
LEASE period and recovery are directly
tied to it
¢ NFSv4 supports many definitions of client
¢ A CLIENTID is shorthand for a client
definition
CLIENTID – may represent these entities at the NFS client
Traditional Unix NFS Client
(M-Processes / N-Threads)
(Y-Users/Principals)
User Level Client
(1-Processes / N-Threads)
Individual Thread
Creation of CLIENTID
¢ W ith SETCLIENTID, client chooses an
opaque identifier and a verifier
¢ Server assigns a CLIENTID
¢ Client confirms use with
SETCLIENTID_CONFIRM
¢ Server uses RPC authentication to verify
request (saves principal for future
reference)
CLIENTID creation
(Client)
SETCLIENTID
(Server)
<identifier>
<verifier>
<CLIENTID>
<verifier>
SETCLIENTID
Identifier used before?
If so, verifier different?
If so, mark for state release
On retransmit, return
previous response
SETCLIENTID_CONFIRM
SETCLIENTID_CONFIRM
<verifier>
On confirm request,
release all previous
client state
On retransmit, do nothing
and return previous
response
What do we have?
(Server)
(Client)
CLIENTID
Lease
Lockowner
¢ Next level of state associates client's
entity of file locking with the protocol's
operations
¢ For Posix clients, this is the process id*
¢ Other clients will choose the appropriate
association for lockowner
¢ OPEN operation is where the lockowner
is introduced to the server
¢ Use of lockowner / sequence-id pair is
Lockowner association
(Client)
OPEN
(Server)
<CLIENTID>
<lockowner>
<sequence-id>
<stateid>
<verifier>
OPEN
First time lockowner used?
Request confirm
On retransmit, return
previous response
OPEN_CONFIRM
OPEN_CONFIRM
<verifier>
<sequence-id+1>
On confirm, change stateid
to reflect change in shared
<stateid'> state
On retransmit, do nothing
and return previous
response
What do we have?
(Server)
(Client)
CLIENTID
Lockowner
Lockowner
Lease
Lockowner
Stateid
¢ Server generates it
¢ Unique for entire server
¢ Stateid incorporates
" Shorthand reference to lockowner
" Shared file locking/open state between
client/server
" Marker to detect server reboot
¢ Stateid changes at each of the following
operations:
LOCK, LOCKU, CLOSE, OPEN_CONFIRM,
OPEN_DOWNGRADE
Sequence-id
¢ Initiated by client
¢ Associated with lockowner
¢ Allows client to order requests (OPEN)
Stateid changes
(Client)
<stateid>
<STATE_OP> <sequence-id>
(Server)
Stateid (previous server)?
<stateid ' > Valid stateid?
Valid sequence id?
<stateid ' >
<STATE_OP> <sequence-id + 1>
Stateid (previous server)?
Valid stateid?
<stateid '' > Valid sequence id?
<STATE_OP>
On retransmit, return
previous response
What do we have?
(Server)
(Client)
CLIENTID
Lockowner
Lockowner
Stateid
Stateid'
Stateid''
Stateid'''
...
Stateid+n
Lease
Lockowner
Delegation
¢ Servers allows the client to manage state
¢ Delegation is done per-file
¢ Call-back path must be present
¢ A delegation-stateid is used as shorthand
reference
¢ If delegation provided, client manages
state for all of its lockowners
What do we have?
(Server)
(Client)
CLIENTID
Lease
Delegated file
Lockowner
Lockowner
Lockowner
Stateid
Stateid'
Stateid''
Stateid'''
...
Stateid+n
W h a t's Broken?
¢ Association of lockowner is made at
OPEN
¢ Doesn't allow for POSIX fork()/file locking
¢ Child isn't guaranteed OPEN will succeed
Posix fork(), shared file table entries, and file locking
Proc A
Proc A'
Conflict for file lockng
File-A
Lockowner (pid)
One Solution
¢ Add level of indirection
¢ Introduce openowner that can represent
multiple processes at a Posix/Unix client
¢ OPEN changes to accept openowner and
return openstateid
¢ LOCK_CONFIRM added to introduce
lockowner at the time of file locking
¢ For clients that don't need this, they will
have one lockowner per openowner
Posix fork(), shared file table entries, and file locking
Proc A
Proc A'
File-A
Openowner (per client?)
Lockowner (pid)
Lockowner (pid)
Openowner / Lockowner association
(Client)
OPEN
(Server)
<CLIENTID>
<openowner>
<open-seq-id>
First time openowner used?
<openstateid> Request confirm
<verifier>
<verifier>
<open-seq-id+1>
OPEN_CONFIRM
<openstateid'>
<openowner>
LOCK <lockowner>
<lock-seq-id>
<lockstateid'>
Now what do we have?
(Server)
(Client)
CLIENTID
Openowner
Lockowner
Lease
Openowner
Lockowner
Lockowner
Stateid
Stateid'
Stateid
Stateid'
...
Stateid+n
Openowner
...
Stateid+n
Sim u ltaneous OPENs
¢ Lack of a serialization point at the client
¢ Simultaneous OPENs can occur for the
same target file
¢ Hard linked file that is referenced under
two or more names as an example
¢ Provide access to the embedded
sequence-id which is contained in the
stateid
¢ Allow the client to choose which
"open" stateids
openstateid {
uint64_t
sequenceid;
opaque ident<>;
};
Openowner / Lockowner association
(Client)
OPEN("foo")
"foo" == "bar"
OPEN("bar")
(Server)
<CLIENTID>
<openowner>
<open-seq-id>
<CLIENTID>
<openowner>
<open-seq-id+1>
<openstateid>
<openstateid+1'>
Client stores openstateid
for later use
Client notes that
openstateid is "older"
for this response and
does not save
Rem a ining Issues
¢ Confirm use of sequence id for OPEN /
LOCK
¢ Detail rules of when openstateid or
lockstateid should be used
¢ Unknown...
Questions
¢ ???