Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
INTRODUCTION ..................................................................................................................................................... 3
LWIO-SRV .............................................................................................................................................................. 3
FUNCTION ....................................................................................................................................................................3
ARCHITECTURE ..............................................................................................................................................................4
Driver.....................................................................................................................................................................5
Function ............................................................................................................................................................................. 5
API ...................................................................................................................................................................................... 5
Shares....................................................................................................................................................................6
Function ............................................................................................................................................................................. 6
API ...................................................................................................................................................................................... 6
Transport .............................................................................................................................................................10
Function ........................................................................................................................................................................... 10
API .................................................................................................................................................................................... 11
Transport Handlers ........................................................................................................................................................... 12
Select handler .............................................................................................................................................................. 13
Poll handler ................................................................................................................................................................. 15
Epoll handler ............................................................................................................................................................... 16
KQueue/KEvent handler .............................................................................................................................................. 19
Protocols .............................................................................................................................................................20
Function ........................................................................................................................................................................... 20
API .................................................................................................................................................................................... 21
SMB Version 1 .................................................................................................................................................................. 22
SMB Version 2 .................................................................................................................................................................. 23
Negotiate ..................................................................................................................................................................... 26
Session setup ............................................................................................................................................................... 27
Logoff........................................................................................................................................................................... 28
Tree connect ................................................................................................................................................................ 29
Tree disconnect ........................................................................................................................................................... 30
Create .......................................................................................................................................................................... 31
Close ............................................................................................................................................................................ 32
Flush ............................................................................................................................................................................ 33
Read............................................................................................................................................................................. 34
Write............................................................................................................................................................................ 35
Lock ............................................................................................................................................................................. 36
IOCTL ........................................................................................................................................................................... 38
Cancel .......................................................................................................................................................................... 39
Echo ............................................................................................................................................................................. 40
Find .............................................................................................................................................................................. 41
Notify ........................................................................................................................................................................... 42
Get info ........................................................................................................................................................................ 44
Set info ........................................................................................................................................................................ 45
Break ........................................................................................................................................................................... 46
FEATURES ...................................................................................................................................................................47
Scalability ............................................................................................................................................................47
Zero-copy ............................................................................................................................................................47
1
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Throttling ............................................................................................................................................................47
Threading Model .................................................................................................................................................48
Asynchronous processing ....................................................................................................................................49
CONFIGURATION ..........................................................................................................................................................50
Location...............................................................................................................................................................50
Parameters..........................................................................................................................................................50
REFERENCES......................................................................................................................................................... 51
2
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Introduction
The Likewise Input Output system (LWIO) provides network storage support through SMB/CIFS on Linux
and UNIX platforms. LWIO is implemented using a plug-in architecture where various driver modules are
loaded dynamically, and can communicate among themselves using IRP (I/O request packet) structures.
This document is the technical specification for the “srv” driver module (LWIO-SRV), whose primary
responsibility it is to receive and fulfill requests based on the SMB/CIFS protocol in a secure manner.
The LWIO-SRV module is provided as a single dynamically loadable library named
libsrv.sys.<library_extension>. For instance, /opt/likewise/lib/libsrv.sys.so
LWIO-SRV
Function
The primary functions of the LWIO-SRV module can be summarized as follows.
a) Listens on port 445 for incoming CIFS requests
b) Receives messages delivered through the local inter-process communication channels
implemented using Linux/UNIX domain sockets. Local system daemons such as lsassd
(Authentication daemon) and srvsvcd (Share management RPC Interface) will use the local interprocess communication channel to register their communication pipe.
c) Maintains network connections according to the CIFS specification. This will include support for
durable handles in the case of SMB2.
d) Maintains a persistent set of shares (file system mappings) and provides access to these shares
in a secure manner using the GSSAPI (Kerberos, NTLMv1 and NTLMv2).
e) Fulfills requests conforming to the SMB-1 and SMB-2 protocols.
f)
Provides metrics on connections, sessions, trees and open file handles through the local interprocess communication channel as well as the SRVSVC RPC Channel.
3
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Architecture
Figure 1: LWIO-SRV Internal layout
4
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
The LWIO-SRV driver comprises of the following sub-systems.
1. Driver
2. Shares
3. Transport
4. Protocols
Driver
Function
1. This module contains the entry point used to load the sub-system into the LWIO service.
2. Provides the interface for share management through Device Control (IOCTL) functions.
3. Hosts a thread queue which processes SMB/CIFS traffic through the port 445. SMB/CIFS packets
are retrieved from the LWIO-SRV/Transport, processed in the LWIO-SRV/Protocols sub-system,
and any results are deposited back to the LWIO-SRV/Transport sub-system.
API
5
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Shares
Function
1. Provides an in-memory collection of shares, based on a repository. The repository can be a local
database, local file or be cluster-aware. The in-memory collection shall be guarded by a readerswriter lock.
2. Updates to shares are performed in memory first and then persisted to the repository as a single
transaction.
3. The implementation of the share repository shall be separated from the share management
interface, and these components will be bound at compile time.
4. The default share repository provided by Likewise will use the sqlite database.
API
NTSTATUS
SrvShareInit(
VOID
);
NTSTATUS
SrvShareMapIdToServiceStringW(
IN
SHARE_SERVICE
OUT PWSTR*
);
NTSTATUS
SrvShareMapIdToServiceStringA(
IN
SHARE_SERVICE
OUT PSTR*
);
NTSTATUS
SrvShareMapServiceStringToIdA(
IN
PCSTR
IN OUT SHARE_SERVICE*
);
service,
ppwszService
service,
ppszService
pszService,
pService
6
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
NTSTATUS
SrvShareMapServiceStringToIdW(
IN
PWSTR
IN OUT SHARE_SERVICE*
);
NTSTATUS
SrvShareMapFromWindowsPath(
IN
PWSTR
OUT PWSTR*
);
NTSTATUS
SrvShareMapToWindowsPath(
IN
PWSTR
OUT PWSTR*
);
NTSTATUS
SrvGetShareName(
IN
PCSTR
IN
PCSTR
IN
PWSTR
OUT PWSTR*
);
NTSTATUS
SrvGetMaximalShareAccessMask(
IN
PSRV_SHARE_INFO
IN OUT ACCESS_MASK*
);
NTSTATUS
SrvGetGuestShareAccessMask(
IN
PSRV_SHARE_INFO
IN OUT ACCESS_MASK*
);
pwszService,
pService
pwszInputPath,
ppwszPath
pwszInputPath,
ppwszPath
pszHostname,
pszDomain,
pwszPath,
ppwszSharename
pShareInfo,
pMask
pShareInfo,
pMask
NTSTATUS
SrvShareInitList(
IN OUT PLWIO_SRV_SHARE_ENTRY_LIST
);
7
pShareList
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
NTSTATUS
SrvShareFindByName(
IN
PLWIO_SRV_SHARE_ENTRY_LIST
IN
PWSTR
OUT PSRV_SHARE_INFO*
);
NTSTATUS
SrvShareAdd(
IN OUT PLWIO_SRV_SHARE_ENTRY_LIST
IN
PWSTR
IN
PWSTR
IN
PWSTR
IN
PBYTE
IN
ULONG
IN
PWSTR
);
NTSTATUS
SrvShareUpdate(
IN OUT PLWIO_SRV_SHARE_ENTRY_LIST
IN
PSRV_SHARE_INFO
);
NTSTATUS
SrvShareDelete(
IN OUT PLWIO_SRV_SHARE_ENTRY_LIST
IN
PWSTR
);
NTSTATUS
SrvShareEnum(
IN
PLWIO_SRV_SHARE_ENTRY_LIST
OUT PSRV_SHARE_INFO**
IN OUT PULONG
);
VOID
SrvShareFreeListContents(
IN OUT PLWIO_SRV_SHARE_ENTRY_LIST
);
8
pShareList,
pwszShareName,
ppShareInfo
pShareList,
pwszShareName,
pwszPath,
pwszComment,
pSecDesc,
ulSecDescLen,
pwszShareType
pShareList,
pShareInfo
pShareList,
pwszShareName
pShareList,
pppShareInfo,
pulNumEntries
pShareList
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
VOID
SrvShareFreeEntry(
IN
PSRV_SHARE_ENTRY
);
pShareEntry
VOID
SrvShareFreeInfoList(
IN
PSRV_SHARE_INFO*
IN
ULONG
);
ppInfoList,
ulNumInfos
VOID
SrvShareReleaseInfo(
IN
PSRV_SHARE_INFO
);
pShareInfo
NTSTATUS
SrvShareShutdown(
VOID
);
9
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Transport
Function
1. Listens on port 445 for incoming CIFS connections
2. Uses an appropriate scalable method to handle network traffic, depending on platform
capabilities. Likewise will provide implementations supporting epoll for Linux systems,
kqueue/kevent for FreeBSD, event ports for Solaris, poll and select.
3. This sub-system shall own the socket connection object.
4. After an SMB/CIFS packet is received, it will be associated with the connection over which it was
received. These parameters will form the SMB SRV Context which will be en-queued to be
processed.
5. Any SMB response packets generated when processing an SMB request will be submitted back
to the Transport layer to be transmitted back through the associated connection.
6. Hosts a thread queue which is responsible for network activity (listen/accept/read/write).
10
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
API
NTSTATUS
SrvTransportInit(
IN
PLWIO_PACKET_ALLOCATOR
IN
PLWIO_SRV_SHARE_ENTRY_LIST
IN
PSMB_PROD_CONS_QUEUE
);
NTSTATUS
SrvTransportGetRequest(
IN
struct timespec*
OUT PSRV_EXEC_CONTEXT*
);
hPacketAllocator,
pShareList,
pWorkQueue
pTimeSpec,
ppContext
NTSTATUS
SrvTransportSendResponse(
IN
PLWIO_SRV_CONNECTION
IN
PSMB_PACKET
);
pConnection,
pResponse
NTSTATUS
SrvTransportShutdown(
IN
VOID
);
11
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Transport Handlers
All incoming connection requests on port 445 will be handled by the transport sub-system in the srv
driver. In order to scale and being able to accept and service a large number of connections, an
appropriate transport handler is selected based on the platform on which the driver is deployed. For
instance, the select system call on most platforms is limited to a default maximum of FD_SETSIZE, which
might be set to 1024 descriptors. The poll and epoll mechanisms allow for a larger number of
descriptors. Also, select and poll do not deliver good performance when the server has accepted a large
number of connections with only a small number of them being active.
12
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Select handler
This handler depends on the select system call which is available on most modern operating systems.
The select mechanism is level triggered. In specific, if a file descriptor is notified as being active
(read/write) and as a result of this, one of the threads reads part of the data from the descriptor, an
ensuing call to select will notify the descriptor as being active, even if no more data was received on the
descriptor.
The typical work flow of a transport mechanism based on select is as follows.
(1) Declare variables of type fd_set to hold the set of descriptors to poll for reading, writing and
errors.
(2) On each iteration, clear the descriptor set using FD_CLR and set each member by using FD_SET.
(3) Call select using the file descriptor sets and a timeout value.
(4) The return value of select indicates how many file descriptors are active.
(5) Iterate through the descriptor sets and use FD_ISSET to find out if a descriptor is active. If
iterating through the read file descriptor set, it is determined that a file descriptor might be
ready for reading.
(6) If a file descriptor is set as active for reading and there is no data available for reading, it is an
indication that the peer has disconnected.
This select transport handler will utilize a listener thread and one or more traffic threads. The driver will
have a minimum configured number of traffic threads and a maximum number to which it could
possibly grow. The number of traffic threads will be managed by the listener thread.
A traffic thread functions as follows.
(a) Maintains a queue of connection objects.
(b) Maintains a producer-consumer queue of outgoing “SMBPacket” objects per connection.
(c) Performs “select” on the list of file descriptors pertaining to the current list of connection
objects.
(d) When an incoming “SMBPacket” is complete, queues it up on the work queue.
(e) Maintains a metric of work that could be queried for purposes of load balancing.
13
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
The listener thread functions as follows.
(a) Listens on port 445 and accepts incoming connections.
(b) Assigns an incoming connection to an appropriate traffic queue. The descriptor on the incoming
connection is set to be non-blocking (using socket options).
(c) Maintains the membership of traffic queues based on their maximum allowed size (FD_SETSIZE)
as well as current load.
(d) Manage (spawn, re-distribute membership and tear down) traffic queues based on
configuration (maximum number of connections accepted) and resources.
(e) The listener thread shares a UNIX pipe descriptor set with each traffic queue to notify it of
additions. The traffic thread includes the pipe descriptors in its set used for polling.
The external interface of this transport handler will function as follows.
(a) SrvTransportInit sets the packet allocator, the share list (container) and work queue.
(b) SrvTransportGetRequest retrieves the next “SMBPacket” available in the work-queue. If no
packets are currently available, the return value from the function will be STATUS_IO_TIMEOUT.
(c) SrvTransportSendResponse queues the current “SMBPacket” in the connection object. This call is
asynchronous and must return immediately after adding the current packet to the outgoing
queue.
(d) SrvTransportShutdown shuts down the listener thread first in order to stop accepting new
connections. Next, the current connections must be closed gracefully. Finally, the references to
the packet allocator, share list and work queue furnished upon initialization must be released
(and local references be reset).
14
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Poll handler
15
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Epoll handler
This handler depends on the epoll system call which is available on Linux, kernel versions 2.5.44 and
later. The primary interest in using epoll is its ability to scale to a large number of connections. Also, this
system call returns the set of active connections (descriptors) in contrast to other mechanisms (select)
where the caller would have to iterate through the entire descriptor set to cull the active connections.
The epoll mechanism can be deployed in both level and edge triggered fashions. However, the edge
triggered case is preferred by the srv driver for performance and scalability.
The typical work flow of the edge triggered use-case of epoll is as follows.
(1) Use epoll_create to get the epoll descriptor. This call gives a hint to the kernel as to how many
maximum events are being expected.
(2) Open the listener socket on port 445 and save this in a special variable to differentiate from
descriptors pertaining to remote connections.
(3) Add the listener socket to the descriptor set using epoll_ctl
(4) Enter loop and call the epoll_wait system call. This call returns with the number of active file
descriptors and an array of epoll_event structures (owned by the system).
Iterate through the epoll_event array; if the descriptor in the event matches the listener
descriptor obtained in step (1) above, accept an incoming connection and add it to list of
descriptors using epoll_ctl. Set the descriptor as being non-blocking using socket options.
Otherwise, perform read/write operations on the descriptor.
16
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
The epoll transport handler will utilize an event thread and a (configured) number of I/O threads.
The listener thread will function as follows.
(a) Maintains a global connection catalog of all connections (reference counted).
(b) Creates the epoll device by specifying a configured number of maximum events.
(c) Opens a socket on port 445 and adds this to the event queue.
(d) Processes a loop waiting on epoll_wait.
(e) Maintains a producer-consumer queue that it shares with the I/O threads. This queue will
contain the connections that have events.
(f) New connections are registered for polling using EPOLLONESHOT and EPOLL_CTL_ADD (this
includes both EPOLLIN and EPOLLOUT). Also included in the event flags will be EPOLLHUP and
EPOLLERR to poll for peer disconnects and errors respectively.
(g) Looks up connection object pertaining to the descriptor in an event, and queues it for processing
by the I/O threads.
(h) If an event has flags indicating a peer disconnect or any errors, the connection will be flagged as
being closed and the corresponding descriptor will be closed (thus removing it from the polling
queue).
The I/O thread will function as follows.
(a) Waits on the producer-consumer queue shared with the listener thread for connections.
(b) Removes a connection from the queue and performs I/O (read and write) until an EAGAIN is
received. When an EAGAIN is received, the connection is added back to the epoll device for
polling.
17
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
The external interface of this transport handler will function as follows.
(a) SrvTransportInit sets the packet allocator, the share list (container) and work queue.
(b) SrvTransportGetRequest retrieves the next “SMBPacket” available in the work-queue. If no
packets are currently available, the return value from the function will be STATUS_IO_TIMEOUT.
(c) SrvTransportSendResponse queues the current “SMBPacket” in the connection object. This call is
asynchronous and must return immediately after adding the current packet to the outgoing
queue. The connection object must be added to the queue if it is not already in queue.
(d) SrvTransportShutdown shuts down the removes the listener descriptor from the epoll device
first in order to stop accepting new connections. Next, the current connections must be closed
gracefully by iterating through the global connection catalog. Finally, the references to the
packet allocator, share list and work queue furnished upon initialization must be released (and
local references be reset).
18
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
KQueue/KEvent handler
19
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Protocols
Function
1. Process CIFS requests conforming to SMB Version 1
2. Process CIFS requests conforming to SMB Version 2
3. Access shares from the Shares sub-system as part of SMB processing.
4. Assign references to shares to Tree objects
5. Owns the SMB Session, Tree and File objects
20
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
API
NTSTATUS
SrvProtocolInit(
PSMB_PROD_CONS_QUEUE pWorkQueue,
BOOLEAN
bSupportSMB2
);
NTSTATUS
SrvProtocolExecute(
PSRV_EXEC_CONTEXT pContext
);
NTSTATUS
SrvProtocolShutdown(
VOID
);
typedef struct _SRV_EXEC_CONTEXT
{
LONG
}
refCount;
pthread_mutex_t
pthread_mutex_t*
mutex;
pMutex;
PLWIO_SRV_CONNECTION
PSMB_PACKET
pConnection;
pSmbRequest;
PSRV_PROTOCOL_EXEC_CONTEXT
PFN_SRV_PROTOCOL_FREE_EXEC_CONTEXT
pProtocolContext;
pfnFreeContext;
PSMB_PACKET
ULONG
pSmbResponse;
ulNumDuplicates;
PSMB_PACKET
pInterimResponse;
BOOLEAN
bInternal;
ULONG64
ullAsyncId;
SRV_EXEC_CONTEXT, *PSRV_EXEC_CONTEXT;
21
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
SMB Version 1
22
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
SMB Version 2
Operation codes
Operation code
Code
1
COM2_NEGOTIATE
0x00
2
COM2_SESSION_SETUP
0x01
3
COM2_LOGOFF
0x02
4
COM2_TREE_CONNECT
0x03
5
COM2_TREE_DISCONNECT
0x04
6
COM2_CREATE
0x05
7
COM2_CLOSE
0x06
8
COM2_FLUSH
0x07
9
COM2_READ
0x08
10
COM2_WRITE
0x09
11
COM2_LOCK
0x0A
12
COM2_IOCTL
0x0B
13
COM2_CANCEL
0x0C
14
COM2_ECHO
0x0D
15
COM2_FIND
0x0E
16
COM2_NOTIFY
0x0F
17
COM2_GETINFO
0x10
18
COM2_SETINFO
0x11
19
COM2_BREAK
0x12
Description
23
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Table 1: SMB2 op-codes
Header
Flag
Mask
SMB2_FLAGS_SERVER_TO_REDIR
0x00000001
SMB2_FLAGS_ASYNC_COMMAND
0x00000002
SMB2_FLAGS_RELATED_OPERATION 0x00000004
SMB2_FLAGS_SIGNED
0x00000008
SMB2_FLAGS_DFS_OPERATIONS
0x08000000
Table 2: SMB2 Header Flags
Field
Type/Width Description
Preamble
UINT8[4]
Header Length
UINT16
Epoch/Credits used
UINT16
Status
UINT32
Command
UINT16
Credits
UINT16
Flags
UINT32
Chaining offset
UINT32
Command sequence
UINT64
Process identifier
UINT32
Tree identifier
UINT32
Session identifier
UINT64
Must contain {0xFE,’S’,’M’,’B’}
Refer to Table 1: SMB2 op-codes
Refer to Table 2: SMB2 Header Flags
24
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Signature
UINT8[16]
Table 3: SMB2 Message Header
Field
Type/Width Description
Preamble
UINT8[4]
Header Length
UINT16
Epoch/Credits used
UINT16
Status
UINT32
Command
UINT16
Credits
UINT16
Flags
UINT32
Must contain {0xFE,’S’,’M’,’B’}
Refer to Table 1: SMB2 op-codes
Refer to Table 2: SMB2 Header Flags
Note: The SMB2_FLAGS_ASYNC_COMMAND and the
corresponding asynchronous identifier must be set in
an asynchronous message.
Chaining offset
UINT32
Command sequence
UINT64
Asynchronous identifier
UINT64
Session identifier
UINT64
Signature
UINT8[16]
This value is established in an interim response
Table 4: SMB2 Asynchronous Message Header
25
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Negotiate
Request
Response
26
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Session setup
Request
Response
27
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Logoff
Request
Response
28
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Tree connect
Request
Response
29
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Tree disconnect
Request
Response
30
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Create
Request
Response
31
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Close
Request
Response
32
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Flush
Request
Response
33
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Read
Request
Response
34
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Write
Request
Response
35
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Lock
Request
Flag
Mask
Description
Shared lock
0x00000001
Exclusive lock
0x00000002
Unlock
0x00000004
Fail immediately
0x00000010 If this flag is not set, the server can send an interim response
and follow up when the locking operation has been completed.
Table 5: SMB2 Lock Request Flags
Field
Size
Description
Offset
UINT64
This value will be 4 including the size of the next field.
Length
UINT64
Number of lock or unlock ranges
Flags
UINT32
Table 5: SMB2 Lock Request Flags
Reserved
UINT32
Padding
Table 6: SMB2 Lock Range Structure
36
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Field
Size
Description
Structure size
UINT16
This value will be 4 including the size of the next field.
Lock count
UINT16
Number of lock or unlock ranges
Reserved
UINT32
Padding
File identifier :
Persistent
UINT64
File identifier to be looked up.
File identifier:
Volatile
UINT64
File identifier to be looked up.
Lock range
Array of lock ranges as described in Table 6: SMB2 Lock Range
Structure. The number of ranges must match the lock count.
Table 7: SMB2 Lock Request
Response
37
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
IOCTL
Request
Response
38
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Cancel
This operation code is used to cancel asynchronous requests.
An asynchronous request is distinguished by an asynchronous header (Refer to Table 4: SMB2
Asynchronous Message Header).
Request
Field
Size
Description
Structure size
UINT16
This value will be 4 including the size of the next field.
Reserved
UINT16
Padding
The following are the properties of a Cancel request.
(1) It must not be chained.
(2) The message identifier must be zero.
Response
This request does not send a response.
39
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Echo
Request
Response
40
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Find
Request
Response
41
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Notify
Request
Flag
Mask
Description
FILE_NOTIFY_CHANGE_FILE_NAME
0x00000001
FILE_NOTIFY_CHANGE_DIR_NAME
0x00000002
FILE_NOTIFY_CHANGE_NAME
0x00000003
FILE_NOTIFY_CHANGE_ATTRIBUTES
0x00000004
FILE_NOTIFY_CHANGE_SIZE
0x00000008
FILE_NOTIFY_CHANGE_LAST_WRITE
0x00000010
FILE_NOTIFY_CHANGE_LAST_ACCESS
0x00000020
FILE_NOTIFY_CHANGE_CREATION
0x00000040
FILE_NOTIFY_CHANGE_EA
0x00000080
FILE_NOTIFY_CHANGE_SECURITY
0x00000100
FILE_NOTIFY_CHANGE_STREAM_NAME
0x00000200
FILE_NOTIFY_CHANGE_STREAM_SIZE
0x00000400
FILE_NOTIFY_CHANGE_STREAM_WRITE
0x00000800
Table 8: Change Notify Completion Filter
Flag
Mask
Description
WATCH_TREE
0x00000001
Table 9: Change Notify Flags
42
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Field
Size
Description
Structure size
UINT16
Must have the value 32 (bytes)
Flags
UINT16
Table 9: Change Notify Flags
Output buffer length
UINT32
The maximum length of the notify information to
capture.
File identifier : Persistent
UINT64
File identifier to be looked up. This must reference a
directory object.
File identifier: Volatile
UINT64
File identifier to be looked up. This must reference a
directory object.
Completion filter
UINT32
Refer to Table 8: Change Notify Completion Filter
Reserved
UINT32
Padding
Table 10: Change Notify Request Format
Response
43
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Get info
Request
Response
44
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Set info
Request
Response
45
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Break
This message (code: 0x12) is used to send messages related to opportunistic locks.
Request
Response
46
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Features
Scalability
1. The transport sub-system will use the epoll group of calls to receive network requests. This
should increase the number of maximum connections that can be accepted. If epoll is not
available on a particular system, kqueue/kevent (FreeBSD), event ports (Solaris), poll or select
will be used.
2. The number of worker threads can be adjusted to a multiple of the number of cores on the
system, to increase the rate of processing (of CIFS requests).
3. Pre-allocated SMB Packets are maintained in a free list. The size of the free list can be
configured.
Zero-copy
Certain requests receive a data payload that must be deposited to the file system. An example
of this is the WriteAndX request. In order to reduce memory allocation and copies, the srv driver
will read only the SMB Headers; the actual data will be read by the driver.
Throttling
The maximum number of connections, sessions, trees and files can be controlled through configuration.
Some customers have indicated that they would like to assign an upper limit to the size of the LWIO
service; this would serve as a throttling mechanism.
47
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Threading Model
The following threads are involved in the LWIO-SRV module.
(a) Driver Threads
In the case of synchronous processing, a Driver thread removes an SMB context from the
Transport’s inbound queue, processes the packet and queues the SMB response context back on
the Transport’s outbound queue.
In the case of asynchronous processing, a Driver thread removes an SMB context from the
Transport’s inbound queue, processes the packet and queues the intermediate
(STATUS_PENDING) response context back on the Transport’s outbound queue. When the
IOManager module returns with the real asynchronous response, it will be added to the
Transport’s outbound queue.
(b) Transport Threads
The transport system must have at least (2) threads, one to process inbound requests and the
other to process outbound responses.
(c) LwMsg Threads
Share Management tasks which are performed through Device Control (IOCTL) calls will not use
any threads from the LWIO-SRV module; they will use the thread from the LwMsg system (Interprocess communication).
Similarly, other services such as LSASS and SRVSVC will register their NamedPipe end-points with
the LWIO-SRV driver through the Inter-process communication channel, which will be processed
over an LwMsg thread.
(d) Timer thread
The srv driver will own the timer thread which will provide timeout notifications. This is
necessary when processing oplock requests.
48
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Asynchronous processing
49
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
Configuration
Location
The Likewise registry holds the configuration parameters under the key
[HKEY_THIS_MACHINE\Services\lwio\Parameters\Drivers\srv\shares]
Parameters
Parameter Name
Default Value
Maximum
Value
Minimum Value
Description
EnableSecuritySignatures
1
1
0
Allow message
signing.
RequireSecuritySignatures 1
1
0
Reject clients that
do not support
signing.
SupportSmb2
0
1
0
Whether the SMB2
protocol handler
will be engaged.
WorkerThreadCount
4
-
2
Number of worker
threads fulfilling
SMB requests.
50
Likewise Input Output (LWIO)
Technical specification: srv driver
Copyright: Likewise Software Inc.
References
 CIFS Technical Reference, Version 1.0 (Release date: 3/1/2002)
 Microsoft Networks SMB FILE SHARING PROTOCOL, Version 6.0p (Release date: 1/1/1996)
 Dan Kegel's website about the C10K Problem
 Jeff Darcy's webpage about implementing high performance servers
51