1
AceCache
Application-Controlled External-Cache Management
Pune Institute Of Computer Technology.
Pune, India.
Application-Controlled External-Cache Management
2
Q.1 ) Explain briefly the idea of your project.
The increasing speed of processors and the lack of comparable improvement in the
I/O performance makes the memory system performance a key limiting factor for memoryintensive applications. The cache management and disk I/O operations becomes a bottleneck
even though there is a large memory and faster processor support for memory intensive
applications like databases, multimedia etc. Current systems have a common cache and
generic cache-management policies for all processes, irrespective of actual requirements.
This ‘one shoe fit for all’ policy limits read-ahead to sequential read-ahead which do not
satisfy requirements of applications with complex access patterns.
The traditional file system implementations do not allow the applications to exert a
control on the caching and replacement decisions. The aim of the project is to allow
applications to control their cache and replacement strategies, as applications themselves
know best about their access patterns, thus they can handle the cache in their best interest
providing improved performance. It is beneficial to reduce elapse time, increase cache hit
ratio and reduce I/O traffic for resource-intensive and memory-bound applications which
handle gigabytes or even terabytes of data through synchronous I/O operations.
The project aims to integrate the application-controlled prefetching, file caching
through maintaining an external application-specific cache along with the application’s
projected access-patterns. Supporting application-controlled file caching and prefetching is
non-trivial because caching and prefetching need to be integrated such that kernel needs to
allocate blocks among processes appropriately. The project also aims to develop an
administrator tool to monitor the degree of performance, to prioritize applications and
resources. As most systems use Windows for bigger appeal, project is implemented on
Windows platform as a Kernel Pass-through Driver.
Q.2 ) Explain briefly, how your project works ?
Application-Controlled External-Cache Management
3
Figure.1 illustrates the proposed idea. In traditional file systems, if the userapplication requested data ( steps 1,2,3 ) is not found in the system cache, then page fault
occurs and steps 4,5,6,7,8 are executed, thus page is brought into the system cache and
returned to application through steps 9,10,11. This leads to I/O latency in the execution of the
requesting process, costing several milliseconds. Our solution facilitates applicationcontrolled read-ahead to avoid hard page-faults, cutting the path to 1,2,3,3.A,9,10,11. Thus
reducing the latency to the order of nanoseconds.
We insert a filter driver at the top of driver stack, as shown in figure 2, for controlling
application-specific external-cache and prefetching. This filter driver has a notification
routine which is called for every request passing through this driver. Administrator can
specify whether to use this external cache or normal caching. If External Cache is to be used,
then request will be processed by this filter driver otherwise just be passed to lower driver.
Functional analysis :Proposed filter driver has three modules namely, Application Controlled Manager
(ACM), Prefetch Control Manager (PCM), and Buffer Control Manager (BCM).
1.Application-Control Manager ( ACM )
This module is triggered through specific part of application code called Checkpoint.
A ‘Checkpoint’ is a hint delivered in the language of the existing I/O interface in terms of
filenames, file descriptors and byte ranges through asynchronous I/O. Checkpoint assigns
priorities as Critical or Normal to a page. The ‘Critical’ pages are always kept in applicationspecific cache. ‘Normal’ priority pages are replaced by the LRU page replacement policy.
Application writer can specify the future required critical pages through ‘Check points’.
ACM maintains a list of pages to be prefetched based on the actual requirements of
the application. This list, called ‘ACM_list’, is then submitted to BCM such that the data is
always prefetched into the application-specific cache.
2. Prefetch Control Manager ( PCM )
This module maintains a file for future access patterns of application. This file can be
updated dynamically by observing the current access pattern or sub-pattern match. Access
pattern can be one of i. sequential
ii. sequential cyclic
iii. random
iv.
random cyclic
According to type of access pattern, appropriate replacement algorithm is to used for
‘Normal’ part of external cache. PCM also maintains a list of pages to be prefetched. This
list, called ‘PCM_list’, is based on probabilistic requirements of application.
3. Buffer Control Manager ( BCM )
This module finalizes the list, called ‘BCM_list’, looking at the ‘ACM_list’ and
‘PCM_list’. It also maintains a list for the candidates pages to be replaced from the
Application-Controlled External-Cache Management
4
application-specific cache. BCM also selects replacement policy to be used, looking at
current access-pattern.
With application using cyclic access pattern, traditional cache manager would use
LRU ( Least Recently Used ) which is worst for this access pattern. However, BCM would
use MRU ( Most Recently Used ) which is more suitable for such access pattern. If access
pattern is non-cyclic, BCM would appropriately use LRU policy.
BCM assures that these pages will be replaced by required pages.
Q.3 ) Highlight specifically the problem for which the project is worked out as a solution
and your original contribution towards the solution of problem.
A] There are three major problems with current virtual memory systems
1. An application can not know the amount of physical memory it has available, and it is
not informed when significant changes are made in the amount of available memory.
2. A program can not efficiently control the contents of the physical memory allocated
to it.
3. A program can not control the read-ahead, write-back and discarding of pages within
its physical memory.
These problems are solved by ACM, by providing an application-controlled externalcache which is divided into two regions, namely ‘Critical’ and ‘Normal’. Critical region is
maintained by explicit needs of application through mechanism of ‘Checkpoints’. Facility to
application writer to provide critical need of the application, also contributes in this solution.
B] File caching and prefetching are not integrated to support each other. Existing system
carry out prefetching without considering cache-management policies. This can be harmful as
prefetching involves early block allocation. Conversely, existing cache replacement policies
are not adaptive for complex access patterns obtained by prefetching.
This problem is solved by PCM, by prefetching according to future access-pattern and
maintaining Normal region of the external-cache. Cache replacement policies are changed
appropriately by BCM according to current complex access-pattern.
C] Finally, for memory bound and resource-intensive application, hard page faults
resulting into synchronous I/O, is the root of the problems.
Prefetching is carried out asynchronously, just before the actual access. Thus reducing
synchronous I/O operations and execution time of the application. This approach can
significantly improve performance for many memory-bound applications while reducing
kernel complexity without complicating other applications or reducing their performance.
Application-Controlled External-Cache makes this idea innovative. Further facility to
application writer to specify the actual need of the application explicitly makes this approach
more interesting. Dynamic cache replacement policy used for external cache according to
access pattern is novel.
Q.4 ) Comment upon marketability of the project and its commercial application.
The project suggest a paradigm shift, i.e. part of the operating system controls is to be
given away to application program as it has best knowledge of its exact requirements which
Application-Controlled External-Cache Management
5
can be used as best optimization. This project can lead to make application more intelligent to
demand for dynamic resources required, which can lead to best resource management.
The project shows promising applications in the areas of :1. Resource-intensive applications database applications, data mining tool, web server, proxy server.
2. Network components network control tool, router
3. Multimedia applications media server, video on-demand, high density video editing, real time multimedia.
4. Scientific applications.
Q.5 ) Briefly mention the social aspects which your project aims to fulfill.
Though the problem is generic and has to be implemented with every next generation
operating system, we have implemented it on Windows to contribute to Windows community
for larger appeal.
Figure 1. Overall Block Diagram
User application initiates an I/O request
User Mode
1
11
NT I/O Manager
2
10
Cache Manager attempts to obtain
data directly from system cache
NT Cache Manager
3
9
Managed through
application-directed
read-ahead and future
access-pattern
System Cache
ACE Cache
8. A
4. B
8
Checkpoint Read
Operation
Application-Controlled External-Cache Management
Kernel Mode
8. B
Virtual Memory
Manager
6
File System
invoked via page
fault path
File system returns data to
page fault handler
5
7
File System Driver
4. A
6
Case 1 : Checkpoint Read
Path : 1 – 2 – 3 – 4. A – 6 – 8. A
File System Driver
obtains data
Disk
Case 2 : Read request , for data present in the ACE Cache
Path : 1 – 2 – 3 – 9 – 10 – 11
Case 3 : Read request, for data not present in the ACE Cache
Path : 1 – 2 – 3 – 4. B – 5 – 6 – 7 – 8. B – 9 – 10 – 11
Figure 2. System after integrating our Filter Driver
Process
User Mode
Kernel Mode
ACM
PCM
BCM
Filter Driver
File System Driver
Device Driver
ACM : Application Control Manager
PCM : Prefetch Control Manager
BCM : Buffer Control Manager
Disk
Application-Controlled External-Cache Management