NYC4SEC Meet-up Group
John Jay College-Criminal Justice
899 10th Avenue, New York, New York
Wednesday June 11th, 2014 06:30pm
Introduction to the Microsoft exFAT
File System
Robert Shullich
CPP, CISSP, CRISC, GSEC, GCFA, CEH, CHFI, CCFP-US
NYC4SEC Meet-up Group – June 11th, 2014
But First, are you D4CS?
NYC4SEC Meet-up Group – June 11th, 2014
NYC4SEC Meet-up Group – June 11th, 2014
Agenda
•
•
•
•
•
•
•
•
•
About me, the paper and the presentation
The need for a new generation of FAT
Digital Forensics Relevance
Exponents and Standards
exFAT Overview
Linux Development
Memory Cards & Flash Memory
exFAT File System Internals
Closing
NYC4SEC Meet-up Group – June 11th, 2014
About me, the paper and the presentation
About Me
About the Presentation
About the SANS Paper
A Gold Standard
Another Paper Reference
Disclaimer
EXFAT
NYC4SEC Meet-up Group – June 11th, 2014
About Me
• I have been in the IT field for 40+ Years, and in
InfoSec for over 20 Years
• I carry many IT and InfoSec certifications
• This research was originally for a class term
project towards my D4CS MS degree
• I then expanded that term paper into a practical
paper for my SANS “Gold” GCFA certification
• Links to the SANS paper and my blog are
provided at the end of this presentation
NYC4SEC Meet-up Group – June 11th, 2014
About the Presentation
What I call the exFAT Road Show
•
•
•
•
•
The New York Forensics Computer Show 4/20/2010
Techno Security and Digital Investigations 6/7/2010
SANS What Works in Forensics and IR Summit 7/8/2010
HTCIA International Training Conference & Expo 9/20/2010
The New York Forensics Computer Show 4/19/2011
http://techchannel.att.com/play-video.cfm/2011/8/16/Conference-TV-ComputerForensics-Show:-Introduction-to-exFAT
• NYC4SEC 6/11/2014
• HTCIA International Training Conference & Expo 8/26/2014
NYC4SEC Meet-up Group – June 11th, 2014
About the SANS Paper
• Consider it “exFAT – the missing manual”
• Very little published about exFAT today
• Two current forensics books mention exFAT:
• Wiley - Mastering Windows Network Forensics and Investigation
• Sybex - EnCase Computer Forensics - The Official EnCE: EnCase
Certified Examiner
“For those seeking an in-depth understanding of the exFAT file
system, you should read the SANS paper entitled “Reverse
Engineering the Microsoft Extended FAT File System (exFAT) by
Robert Shullich”
NYC4SEC Meet-up Group – June 11th, 2014
A Gold Standard
• 2005 Book considered the
authority on different file
systems
• The book’s Author
developed the open-source
TSK forensics tools (The
Sleuth Kit) & Autopsy
• This year adding exFAT to
TSK
NYC4SEC Meet-up Group – June 11th, 2014
Another Paper Reference
NYC4SEC Meet-up Group – June 11th, 2014
Disclaimer
• The released specification and implementation is Release
1.00 of exFAT
• The specification mentions additional features that were
not implemented yet, but may at a future time/ Some of
these are Windows CE holdovers
• Both may be presented today
• Some directory entries will be skipped
• Focus is Microsoft Desktop/Server implementation
• Will talk about Flash/Solid State, but high level
• For exFAT, tried to stay with the patent terminology
NYC4SEC Meet-up Group – June 11th, 2014
The need for a new generation
Legacy FAT
Why do we need a new file system?
Why do we need Faster I/O and Higher Capacity?
Hi-definition movie recording MPEG-4. H.264
EXFAT
NYC4SEC Meet-up Group – June 11th, 2014
Legacy FAT
•
•
•
•
•
•
•
FAT 8
• 1977 Bill Gates and Marc McDonald
• Floppy based
FAT 12
• 1980
FAT 16
• 1984 with release of PC/AT & MS DOS 3
FAT 16B
• 1987 Compaq DOS 3.31
FAT 16X
• 1995 PC DOS 7.0/Win 95 – LBA Addressing
FAT 32
• 1996 Windows 95 OSR2, 98, ME, MS DOS 7.1 – CHS Addressing
FAT 32X
• LBA Addressing
NYC4SEC Meet-up Group – June 11th, 2014
Why do we need a new file system?
•
•
•
•
•
•
•
•
•
•
Current Limits Exhausted (Ran Out of Bits!)
Larger volumes (>2TB) (Scale to Larger Capacity)
Larger files sizes (>4GB)
Faster I/O
• (UHS-I: 104 MB/s - UHS-II: 312MB/s)
Removable Media
Flash/Solid State Media
Flexibility
Extensibility (Difficult to add new features)
NTFS Features without the overhead
Easier to implement FS in firmware
NYC4SEC Meet-up Group – June 11th, 2014
Why do we need Faster I/O and
Higher Capacity?
http://www.cnet.com/news/what-is-4k-uhd-next-generation-resolution-explained/
NYC4SEC Meet-up Group – June 11th, 2014
Hi-def movie recording MPEG-4. H.264
2 GB
4 GB
8 GB
16 GB
32 GB
Fine mode
(13Mbps/CBR)
20 min 40 min
80 min
160 min
320 min
Normal Mode
(9Mbps/VBR)
30 min 60 min
120 min
240 min
480 min
Economy mode
(6Mbps/VBR)
45 min 60 min
180 min
360 min
720 min
NYC4SEC Meet-up Group – June 11th, 2014
Digital Forensics Relevance
Relevance to Forensics Study
What happens when you have exFAT formatted media and no
exFAT support?
Forensics Challenges in 2009
Forensics Challenges Today
EXFAT
NYC4SEC Meet-up Group – June 11th, 2014
Relevance to Forensics Study
• Digital Evidence Extraction
• Finding the evidence
• Including the hiding places
• Validation
• Completeness
• Daubert Expert Testimony
• Need to know and understand file org
• Establish Credibility
• New Media (SD Cards) will drive exFAT adoption, and
the potential for CP investigations.
• Larger Media Capacity also driving exFAT adoption
NYC4SEC Meet-up Group – June 11th, 2014
Trust but Verify
NYC4SEC Meet-up Group – June 11th, 2014
What happens when you have exFAT
formatted media and no exFAT support?
NYC4SEC Meet-up Group – June 11th, 2014
Forensics Challenges 2009
• In 2009, in regards to exFAT:
•
•
•
•
No tools (RAW)
No documentation or Training
No expertise
Evidence backlog
NYC4SEC Meet-up Group – June 11th, 2014
Forensics Challenges Today
Today
• exFAT Misunderstood
• Linux OS Support
• Tuxera drivers may help
• FUSE and No-FUSE hacks
• Most Distributions – No native support
• Mac OS Support (Nov 2010) OS/X 10.6.5+
• Implementation Deviations, No Standards
• Open Source Tools
• Commercial Tools
• Encase (6.14.3 Dec 2009)
• Encase (6.18.0.59) NIST Test March 2014
• FTK (3.2 Oct 2010)
• FTK (3.3) NIST Test April 2014
• Cross Vendor Compatibility
NYC4SEC Meet-up Group – June 11th, 2014
NIST Computer Forensics Tool Testing
• Cyber Fetch
• AAFS-2013 Conference 02/21/2013
• Deleted File Recovery Tool
Testing Results
• One Summary Item:
Support for ExFAT, ext3 &
ext4 is sometimes lacking.
NYC4SEC Meet-up Group – June 11th, 2014
Test Results for Deleted File Recovery and
Active File Listing
• 17 Basic Tests
• March 12, 2014 – Encase 6.18.0.59
• MAC differed by 9 hours
• April 3, 2014 – FTK 3.3.0.33124
• MAC differed by 4 hours
• The exFAT partition and HFS+ created on OS/X 10.6
• exFAT: ctime meta-data replaced with the time of
file deletion [I was unable to recreate]
• Vendor Tool or Apple Implementation?
• Who Validates the Test?
NYC4SEC Meet-up Group – June 11th, 2014
Who Validates the Validator?
Superman: Easy, Miss, I've got you
Lois Lane: You...you've got me, who's got you?
NYC4SEC Meet-up Group – June 11th, 2014
Exponents and Standards
Base 2 or 10?
Exponents
International System of Units (SI) Table
IEC 60027-2
Reference Standards
Endian
Microsoft Math
More Math – exFAT
WinCE
EXFAT
NYC4SEC Meet-up Group – June 11th, 2014
Base 2 or 10?
NYC4SEC Meet-up Group – June
11th, 2014
Exponents
•
•
•
•
•
•
102 = 10 times 10 = 100
103 = 10 times 10 times 10 = 1000 (1K)
22 = 2 times 2 = 4
29 = 2*2*2*2*2*2*2*2*2 = 512
210 = 2*2*2*2*2*2*2*2*2*2 = 1024 (1K)
212 = 2*2*2*2*2*2*2*2*2*2*2*2 = 4096
NYC4SEC Meet-up Group – June 11th, 2014
International System of Units (SI)
Table
• File System in
powers of 2
• Device
characteristics in
power of 10
Shorthand
Longhand
Nth
Bytes
KiB
Kibibyte
210
1024
MiB
Mebibyte
220
1024 KiB
GiB
Gibibyte
230
1024 MiB
TiB
Tebibyte
240
1024 GiB
PiB
Pebibyte
250
1024 TiB
EiB
Exbibyte
260
1024 PiB
ZiB
Zebibyte
270
1024 EiB
YiB
Yobibyte
280
1024 ZiB
NYC4SEC Meet-up Group – June 11th, 2014
IEC 60027-2
Prefixes for binary multiples
Factor
210
220
230
240
250
260
Name
kibi
mebi
gibi
tebi
pebi
exbi
Symbol
Ki
Mi
Gi
Ti
Pi
Ei
Origin
kilobinary: (210)1
megabinary: (210)2
gigabinary: (210)3
terabinary: (210)4
petabinary: (210)5
exabinary: (210)6
Derivation
kilo: (103)1
mega: (103)2
giga: (103)3
tera: (103)4
peta: (103)5
exa: (103)6
Examples and comparisons with SI prefixes
one kibibit 1 Kibit = 210 bit = 1024 bit
one kilobit 1 kbit = 103 bit = 1000 bit
one mebibyte 1 MiB = 220 B = 1 048 576 B
one megabyte 1 MB = 106 B = 1 000 000 B
one gibibyte 1 GiB = 230 B = 1 073 741 824 B
one gigabyte 1 GB = 109 B = 1 000 000 000 B
http://physics.nist.gov/cuu/Units/binary.html
NYC4SEC Meet-up Group – June 11th, 2014
How far off are we?
When we say
but mean
we're this far off
1 kilobyte
210 bytes
2.4%
1 megabyte
220 bytes
4.9%
1 gigabyte
230 bytes
7.4%
1 terabyte
240 bytes
10.0%
1 petabyte
250 bytes
12.6%
1 exabyte
260 bytes
15.3%
http://cnx.org/content/m13081/1.1/
NYC4SEC Meet-up Group – June 11th, 2014
Reference Standards
• Bits are numbered right to left
76543210
•
•
•
•
•
•
Decimal Offsets (zero based)
Little-Endian numbers
Unsigned numbers
Sectors vs. Clusters
Strings are 16 bit Unicode
Strings not Terminated
NYC4SEC Meet-up Group – June 11th, 2014
Endian
• Numbering order may vary based on processor
type, is determined by the order the data bytes
are read from the register.
• A 32 bit number is read as 4 8-bit bytes
• If I have the number 0x11 22 33 44
• Big-Endian will store it as:
0x 11 22 33 44
• Little-Endian will store it as:
0x 44 33 22 11
NYC4SEC Meet-up Group – June 11th, 2014
Microsoft Math
KB184006 Limitations of FAT32 File System
The maximum possible number of clusters on a volume using the
FAT32 file system is 268,435,445. With a maximum of 32 KB per
cluster with space for the file allocation table (FAT), this equates to a
maximum disk size of approximately 8 terabytes (TB).
512B Sectors in a 32 KB cluster = 64
228 (268,435,445) * 26 (64) * 29 (512) = 243 = 8,796,093,022,208
Size of FAT32 FS specified in BPB as sectors (32 bit number)
NYC4SEC Meet-up Group – June 11th, 2014
More Math, exFAT
KB955704
Description of the exFAT file system driver update package
• Support for volumes that are larger than 32 GB, the theoretical
maximum volume size for FAT32 in Windows XP
• The theoretical maximum volume size is 64 ZB.
• The recommended maximum volume size is 512 TB.
• Support for files that are larger than 4 GB, the theoretical maximum
file size for FAT32 in Windows XP
• The theoretical maximum file size is 64 ZB.
• The recommended maximum file size is 512 TB.
NYC4SEC Meet-up Group – June 11th, 2014
WinCE
Version
Released
End of Support
1.0
November 18, 1996
December 31, 2001
2.0
September 29, 1997
2.11
September 30, 2002
2.12
September 30, 2005
3.0
June 15, 2000
4.X
4.0
October 9, 2007
January 7, 2002
July 10, 2012
4.1
January 8, 2013
4.2
July 9, 2013
5.X
August 2004
October 14, 2014
6.0
September 2006
April 10, 2018
7.0
March 2011
April 13, 2021
2013
June 2013
October 10, 2023
NYC4SEC Meet-up Group – June 11th, 2014
Overview
Features of exFAT 1.00
4K (4096) Sector Size
Supported Cluster Sizes
Features of exFAT 1.00 (cont’d)
Future Features of exFAT
MBR Partition Limitations
Advantages of exFAT
Disadvantages of exFAT
OS Support for exFAT
Key Dates for exFAT
EXFAT
NYC4SEC Meet-up Group – June 11th, 2014
Features of exFAT 1.00
• Maximum Volume Size (Increased Capacity)
• Architectural ≈ 128 PiB (223-11 * 225)
• Implementation = 512 TiB
•
•
•
•
•
•
•
•
•
•
Sector sizes from 512 [SF] to 4096 bytes [AF]
Clusters sizes to 32MiB (225)
Subdirectories to 256MiB (Root not restricted)
Maximum files on volume ≈ 232
Maximum File Size 16 EiB-1
Built for speed, less overhead than NTFS
Catches up with some NTFS features
Template-based metadata structures
On-disk storage of file Valid Data Length (VDL)
Speeds up storage allocation processes
NYC4SEC Meet-up Group – June 11th, 2014
4K (4096) Sector Size
NYC4SEC Meet-up Group – June 11th, 2014
Supported Cluster Sizes
NYC4SEC Meet-up Group – June 11th, 2014
Features of exFAT 1.00 (cont’d)
•
•
•
•
•
•
OEM Parameters Sector for device dependent parameters
12 sector VBR, support of larger boot program
Up to 2,796,202 files per sub-subdirectory
File Names max to 255 Characters
16-Bit Unicode File Names and Volume Labels
Optimized for Flash Memory
• Device Boundary Alignment
• No FAT32 minimum cluster (65,525) restriction
• No 8.3 file name support (only LFN)
• UTC Timestamp Support
• Vista/Server 2008 SP2+, XP/Server 2003 with KB
• Native in Windows 7, 8, 8.1, Server 2008 R2, 2012
NYC4SEC Meet-up Group – June 11th, 2014
Future Features of exFAT
• TexFAT (To be released later)
Exists in Windows CE
Transaction Safe exFAT
• ACL (To be released later)
Exists in Windows CE
• Compression & Encryption Support?
Not announced, but would be easy to add
NYC4SEC Meet-up Group – June 11th, 2014
MBR Partition Limitations
• Microsoft File Systems are limited when stored in
a MBR partition
• A partition is defined by a Master Boot Record
• A MBR uses a 4 byte value for number of sectors
• LBA as 32 bit # times 512 Sector limits to 2TiB
• To get the maximum volume size, exFAT cannot
be created within a MBR partition, Need GPT
GUID Partition, or Super floppy Mode
• ExFAT on GPT works on Mac
NYC4SEC Meet-up Group – June 11th, 2014
Advantages of exFAT
• Large volume, file and directory sizes
• Handle growing capacities in media, increasing capacity
to >32 GB.
• > 1000 files in a single directory.
• Speeds up storage allocation processes.
• Breaks file size 4 GB barrier.
• Supports interoperability with future desktop OSs.
• Provides an extensible format.
• Large cluster sizes
• Metadata integrity with checksums
NYC4SEC Meet-up Group – June 11th, 2014
Disadvantages of exFAT
•
•
•
•
•
Not all Windows CE features implemented
No direct conversion to or from other FS
Cannot use CONVERT command to NTFS
No Floppy Support
Mostly a Microsoft Desktop and Server World
• No Support for Older MS systems (Pre-XP)
• Support for other devices, surfacing
• No Information Sector “Hint”
• Like all FAT – Finding Stuff is via brute force
NYC4SEC Meet-up Group – June 11th, 2014
OS Support for exFAT
• Windows XP & Server 2003
• KB955704 (requires SP2 or SP3)
• Vista & Server 2008 SP1
• Vista & Server 2008 SP2
•
(Adds UTC timestamp support)
• Windows 7/Server 2008 R2 and later:
• RTM
• Mac OS/X 10.6.5 and later
NYC4SEC Meet-up Group – June 11th, 2014
Key Dates for exFAT
•
•
•
•
•
•
•
•
•
•
•
September 2006 – Windows CE 6.0
March 2008 – Windows Vista Service Pack 1
January 2009 – Announcement at CES of SDXC specification
January 2009 – Windows XP Drivers Available
May 2009 – Windows Vista Service Pack 2
August 2009 – Tuxera Signs File System IP Agreement with
Microsoft
March 2009 – Pretec Releases first SDXC Cards
December 2009 – Microsoft (re)announces exFAT license program
for third-parties
December 2009 – SDXC laptops due soon
December 2009 – Diskinternals releases exFAT recovery utility
December 2009 – Encase support
NYC4SEC Meet-up Group – June 11th, 2014
More Key Dates for exFAT
•
•
•
•
•
•
•
•
•
December 2009 Sony, Canon & Sanyo License
January 2010 Funai License (LCD TV)
February 2010 Panasonic License
February 2010 Panasonic 64/48GB SDXC
February 2010 Sony Memory Stick XC
February 2010 SanDisk Ultra SDXC 64GB Card 3.0 Spec $350
April 26, 2010 DCF Version 2.0 (Edition 2010)
June 1st 2010 Tuxera Releases Linux & Android exFAT drivers
June 3rd 2010 Kingston Releases Class 10 SDXC 64GB Card 60 MB/s
read, 35 MB/s write.
• October 11th, 2010 FTK 3.2 with exFAT support announced
NYC4SEC Meet-up Group – June 11th, 2014
More Key Dates
•
•
•
•
•
•
•
Mar 16th 2011 Lexar Releases SDXC 128GB
May 3rd, 2011 e.solutions (Volkswagen)
Aug 8, 2012 Sharp for Android Smart Phones
Sep 18, 2012 RIM (Blackberry) Smartphones
Nov 7, 2012 Sharp, Sigma, NextoDi, Black Magic and Atomos Global
Jan 16, 2013 BMW
April 30, 2014 PS4 V1.7 update – hidden new feature: exFAT
NYC4SEC Meet-up Group – June 11th, 2014
Linux Development
FUSE Project
Samsung (No-FUSE)
EXFAT
NYC4SEC Meet-up Group – June 11th, 2014
Linux Development
• Open Source community developing FUSE
• FUSE – File System in User Space
• Samsung accidently leaks native exFAT
implementation, dubbed NO-FUSE
• Samsung source code on GitHUB with GPL
License
• Still legal issues because of patent protection
NYC4SEC Meet-up Group – June 11th, 2014
FUSE Project
NYC4SEC Meet-up Group – June 11th, 2014
Samsung (No-FUSE)
NYC4SEC Meet-up Group – June 11th, 2014
Memory Cards (Including SSD)
Applications (IOT)
exFAT Gone Wild
SD Card Association
Compact Flash
SDXC Storage Capabilities
Standard vs. Non-Standard
General Flash Notes
SD Card Notes
EXFAT
NYC4SEC Meet-up Group – June 11th, 2014
Applications (IOT)
•
•
•
•
•
•
•
•
•
•
•
•
Camera (Still, Video)
Entertainment Systems (Home, Plane, Train, & Automobiles)
GPS, Navigation Systems
Smart Phones, Audio/MP3 players
Laptop, Monitor, Printers
Handheld Computers (Tablets, Netbooks, Mobile)
Smart TVs, Home Theaters
Automatic inflight infotainment systems
Game Consoles
Medical Devices
Measuring Equipment
Other Consumer Electronics
NYC4SEC Meet-up Group – June 11th, 2014
exFAT Gone Wild
• Adoption Rate ↑
• Prevalence ↑
• Media Prices ↓
Storage Media larger than 32GB is being shipped out of the
factory door pre-formatted with the exFAT file system
NTFS, FAT32, and HFS+ are still used in some cases but to
a lesser degree
NYC4SEC Meet-up Group – June 11th, 2014
SD Card Association
•
•
•
•
New Memory Card SDXC
Consumer Appliances
Follows SDHC
Specification for 2TB
Maximum Capacity
NYC4SEC Meet-up Group – June 11th, 2014
http://anythingbutipod.com/2009/01/next-generation-sdxc-details/
NYC4SEC Meet-up Group – June 11th, 2014
Market for SD Cards to Reach
$21.3 Billion by 2018
The SD technology is employed by over 400
brands across numerous product categories
and over 8,000 models, making it the de-facto
industry standard. SD memory cards have been
able to meet the requirements of high-end
consumer devices.
http://www.storagenewsletter.com/rubriques/market-reportsresearch/global-industry-analysts-sd-cards/
NYC4SEC Meet-up Group – June 11th, 2014
Compact Flash
•
•
•
•
•
•
•
•
•
Small Market
Specification 5.0 (Feb 2010)
Specification 6.0 (Nov 2010)
48-Bit Addressing
Max Size 144PB (Up from 137GB)
UltraDMA 7 (167MBytes/s)
FAT32 won’t do (2TB Limit)
SanDisk factory preformats 256GB CF using exFAT
Not Sure Where the file system support will go, but
expect that exFAT will also become a FS of choice for
other media
NYC4SEC Meet-up Group – June 11th, 2014
SDXC Storage Capabilities
•
•
•
•
From 32GB to 2TB on a card
Exclusively exFAT File System
312 MB/s I/O Transfer (UHS-II)
Storage (examples)
4,000 RAW images (14mb file size/64GB)
136,000 fine-grade photos
100 HD movies
480 hours of HD recording
On a single 2TB SDXC card
NYC4SEC Meet-up Group – June 11th, 2014
Standard vs. Non-Standard
•
•
•
•
SDXC is supposed to be exFAT
In computer, you can format as anything
Many devices, will enforce standard
Formatting SD card with OS Format has
issues and differences
• Don’t assume FS based on card type
NYC4SEC Meet-up Group – June 11th, 2014
General Flash Notes
•
•
•
•
•
Write Endurance (Program Erase Cycles)
Write Cliff
Wear Leveling
Pages (Unit of a write)
Blocks (Unit of an erase)
NYC4SEC Meet-up Group – June 11th, 2014
SD Card Notes
•
•
•
•
•
•
•
•
•
•
SDXC Maximum set at 2TB
Two FAT Partitions within MBR
“Protected Area” and “User Area”
WinHex – Partition Offset ≠ 0
VBR differences on format/factory
AU (Allocation Unit) same as Cluster Size
Max AU = 64MiB
RU (Recording Unit) 16KB+
FAT Write Cycle {FAT1/FAT2/DIR}
exFAT Write Cycle {FAT/ABM/DIR}
NYC4SEC Meet-up Group – June 11th, 2014
File System Internals
Regions
FAT
VBR
Directories
Volume Label
Allocation Bit Map
UP Case Table
File Directory Entry Sets
EXFAT
NYC4SEC Meet-up Group – June 11th, 2014
File System Integrity
• Version Verified
• 4 Checksums
•
•
•
•
•
•
•
•
•
VBR
UP-Case Table
Directory File Set entry
Directory GUID entry
Critical Directory Entries
Other Checks and Balances
File System should NOT mount if failures
File System may mount R/O when dirty
Dirty flags in VBR, not in the FAT
NYC4SEC Meet-up Group – June 11th, 2014
Data Hide Alert!
• FAT32 max cluster 64KiB
• exFAT max cluster 32MiB
This is an increase of 512 fold
• Potential for massive slack space
NYC4SEC Meet-up Group – June 11th, 2014
Volume Space Layout
• The Main Boot Region
• Contains main VBR
• The Backup Boot Region
• Contains backup VBR
• The FAT Region
• Contains FAT Table(s)
• The Data Region (Cluster Heap)
• This is where data resides
NYC4SEC Meet-up Group – June 11th, 2014
NYC4SEC Meet-up Group – June 11th, 2014
VBR – Volume Boot Record
• Contains 12 sectors
1 sector main boot sector
• Jump Code (3 bytes)
• Must be Zero (53 bytes)
• BPB (BIOS Parameter Block)
• Boot Strap Code
8
1
1
1
sectors main extended boot sectors (MEBS)
sector OEM parms
sector reserved
sector VBR Checksum
NYC4SEC Meet-up Group – June 11th, 2014
Boot Parameter Block (BPB)
•
•
•
•
•
•
•
•
•
•
OEM Label “EXFAT ”
Volume Length (64-bit) [sector]
FAT Location & Size [sector]
Heap Location & Size [sector, cluster]
Volume Serial Number
Location of Root Directory [cluster]
Volume Flags
Sector and Cluster Sizes [2-shift]
Percent in use
File System Revision (0x0010=1.00)
NYC4SEC Meet-up Group – June 11th, 2014
Sectors & Clusters
• A 2-Shift is a power of 2
• Another name for exponent
• Sector size and sectors per cluster
• Each stored in 1 byte
• Theoretical maximum is 2255
• Sector Size Maximum 212
• Sectors per cluster is derived
• Cluster Size Maximum is 225
NYC4SEC Meet-up Group – June 11th, 2014
Executable Boot Code
• First 3 bytes of Main Boot Sector
• Jump Code
• 0xEB7690
• Offset 120 size 390
• Remainder of boot code
• Offset 510
• End signature marker
• 0xAA55 = “55AA”
• Offset 512
• Unused if defined
NYC4SEC Meet-up Group – June 11th, 2014
More Bootable Code
• Up to 8 Main Extended Boot Sectors
•
•
•
•
FAT32 had 3 sector VBR with 1 MEBS
Entire sector can be used for boot code
Last 8 bytes of sector is marker
0xAA550000 = “000055AA”
• Larger capacity for boot virus!
NYC4SEC Meet-up Group – June 11th, 2014
VBR Checksum Sector
•
•
•
•
The 12th sector of the VBR
Repeating 4 byte checksum
Checksum of previous 11 sectors
Flags and Percent excluded
• These are volatile and change often
• Boot Sector Virus & Checksum
NYC4SEC Meet-up Group – June 11th, 2014
VBR Checksum Sector
Offset
00000000
00000010
00000020
00000030
00000040
0
1
2
3
4
5
6
7
8
9
A
B
C
D
E
F
C9
C9
C9
C9
C9
D0
D0
D0
D0
D0
18
18
18
18
18
8B
8B
8B
8B
8B
C9
C9
C9
C9
C9
D0
D0
D0
D0
D0
18
18
18
18
18
8B
8B
8B
8B
8B
C9
C9
C9
C9
C9
D0
D0
D0
D0
D0
18
18
18
18
18
8B
8B
8B
8B
8B
C9
C9
C9
C9
C9
D0
D0
D0
D0
D0
18
18
18
18
18
8B
8B
8B
8B
8B
ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
D0
D0
D0
D0
18
18
18
18
8B
8B
8B
8B
ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹
Lines 00000050 through 01BF repeated
000001C0
000001D0
000001E0
000001F0
C9
C9
C9
C9
D0
D0
D0
D0
18
18
18
18
8B
8B
8B
8B
C9
C9
C9
C9
D0
D0
D0
D0
18
18
18
18
8B
8B
8B
8B
C9
C9
C9
C9
D0
D0
D0
D0
18
18
18
18
8B
8B
8B
8B
C9
C9
C9
C9
NYC4SEC Meet-up Group – June 11th, 2014
FAT – File Allocation Table
•
•
•
•
•
•
•
•
•
When it is used, same as legacy FAT
Not used when file contiguous
Never used for cluster allocation
FAT 32 has 32 bit cells, uses 28 bits (LBA-28)
exFAT has 32 bit cells, uses 32 bits (LBA-48)
• There is no 64 bit FAT
Maximum clusters is 232-11
With TexFAT – 2 FAT Tables (2 Bitmaps)
1st Addressed by pointer in VBR, 2nd Immed Follows
Size stored in VBR
NYC4SEC Meet-up Group – June 11th, 2014
Reserved Cluster Index Values
•
•
•
•
•
0x00000000 – No significant meaning
0x00000001 – Not a valid cell value
0xFFFFFFF6 – Largest Value
0xFFFFFFF7 – Bad Block
0xFFFFFFF8 – Media Descriptor
• Fixed Disk
• 0xFFFFFFF9-0xFFFFFFFE – Not Defined
• 0xFFFFFFFF – End of Cluster Chain (EOC)
NYC4SEC Meet-up Group – June 11th, 2014
NYC4SEC Meet-up Group – June 11th, 2014
FAT Table Example
Media
Allocation Bit Map
Reserved
UP-Case Table
Root Directory
Offset
0000
0010
0020
0040
0060
0080
00A0
00C0
00E0
0100
0
1
2
3
4
5
6
7
8
F8
FF
00
00
00
00
00
00
00
00
FF
FF
00
00
00
00
00
00
00
00
FF
FF
00
00
00
00
00
00
00
00
FF
FF
00
00
00
00
00
00
00
00
FF
00
00
00
00
00
00
00
00
00
FF
00
00
00
00
00
00
00
00
00
FF
00
00
00
00
00
00
00
00
00
FF
00
00
00
00
00
00
00
00
00
FF
00
00
00
00
00
00
00
00
00
9 10 11 12 13 14 15
FF
00
00
00
00
00
00
00
00
00
FF
00
00
00
00
00
00
00
00
00
FF
00
00
00
00
00
00
00
00
00
FF
00
00
00
00
00
00
00
00
00
FF
00
00
00
00
00
00
00
00
00
FF
00
00
00
00
00
00
00
00
00
FF
00
00
00
00
00
00
00
00
00
NYC4SEC Meet-up Group – June 11th, 2014
Allocation Bitmap
• Keeps track of cluster allocation status
• Zero – Free Cluster
• One – Allocated Cluster
• 1 Byte = Tracking of 8 Clusters
• Bit Zero – Byte Zero = Cluster 2
• Cluster 0 & Cluster 1 are not defined
• Addressed by Directory Entry
• With TexFAT – 2 of these (FAT Pairing)
NYC4SEC Meet-up Group – June 11th, 2014
Legacy FAT vs. exFAT Chains
• When deleting a file in a legacy FAT FS
the cells are wiped out
• When deleting a file in the exFAT FS the
cells are not touched, regardless whether
there is data in the cell
• If a file is fragmented, and is deleted,
then the FAT may be still have the chain
intact
*Some
exFAT implementations might do it the legacy way
NYC4SEC Meet-up Group – June 11th, 2014
Data Hide Alert!
• The Allocation Bitmap and the UP-Case
Table are stored as files, and provide
hiding space in the metadata
• These files are static, typically won’t
move, and have slack space.
• Nothing prevents someone from moving
these files elsewhere in the cluster heap,
and actually making them larger
NYC4SEC Meet-up Group – June 11th, 2014
NYC4SEC Meet-up Group – June 11th, 2014
Directories in exFAT
• Root (VBR Pointer)
• Contains certain critical entries
• Almost unlimited in size
• Subdirectory (by File Entry)
• Contains file sets
• 256MiB Max size
• No physical “.” or “..” entries
• Uses 16 Bit Unicode for strings
• Every Entry 32 bytes in size
• Entry 0x00 is end of directory
• Has capabilities for user entries
NYC4SEC Meet-up Group – June 11th, 2014
Data Hide Alert!
• Manipulation of the Allocation Bitmap, and
creation of user directory entries provides
the capability of hiding file within the file
system
• It may also be possible to hide data within
the directory metadata itself
NYC4SEC Meet-up Group – June 11th, 2014
Entry Type
Type Field
Offset (Bits)
Size (Bits)
In Use
7
1
Category
6
1
Importance
5
1
Code
0
5
NYC4SEC Meet-up Group – June 11th, 2014
Entry Type
• In Use:
• 0 – Not in Use, 1- In Use
• Category:
• 0 – Primary, 1 – Secondary
• Importance:
• 0 – Critical, 1 – Benign
• Code: Identifies the entry
NYC4SEC Meet-up Group – June 11th, 2014
Volume Label Directory Entry
•
•
•
•
•
•
0x83 or 0x03 Entry
Primary Entry
Only resident in Root Directory
Contains the Volume Label
16 bit Unicode
0x03 means no volume label (Blank Label)
NYC4SEC Meet-up Group – June 11th, 2014
Volume Label Directory Entry
Offset
00000000
00000010
0
1
2
3
4
5
6
7
83 0A 65 00 78 00 46 00
32 00 38 00 4B 00 00 00
8
9
A
B
C
D
E
F
41 00 54 00 2D 00 31 00
00 00 00 00 00 00 00 00
ƒ.e.x.F.A.T.-.1.
2.8.K...........
Type
Volume Name Length (10)
Volume Label (exFAT-128K)
NYC4SEC Meet-up Group – June 11th, 2014
Allocation Bitmap Directory
Entry
•
•
•
•
0x81 Entry
Primary Entry
Only resident in Root Directory
Points to the Allocation Bitmap
• If TexFAT, then 2 of these
• Flag bits says which FAT/Bitmap
• Cluster Address of Bitmap
• Size of Bitmap
• NO flag for INVALID FAT
NYC4SEC Meet-up Group – June 11th, 2014
Allocation Bitmap Directory
Entry
Offset
0000
0010
Type
0
1
2
3
4
5
6
7
81 00 00 00 00 00 00 00
00 00 00 00
8
9
A
B
C
D
E
F
00 00 00 00 00 00 00 00
Cluster Address (Cluster 2)
Size (63 bytes)
NYC4SEC Meet-up Group – June 11th, 2014
UP-Case Table Directory Entry
•
•
•
•
•
•
0x82 Entry
Primary Entry
Only resident in Root Directory
File names are case insensitive
Used to fold file name
Table has a checksum (32 bits)
NYC4SEC Meet-up Group – June 11th, 2014
UP-Case Table Directory Entry
Offset
0000
0010
Type
0
1
2
3
4
5
6
7
82 00 00 00 0D D3 19 E6
00 00 00 00
8
9
A
B
C
D
E
F
00 00 00 00 00 00 00 00
Cluster Address (3)
Table Checksum
Length (0x16CC = 5,836)
NYC4SEC Meet-up Group – June 11th, 2014
File Directory Entry Set
•
•
•
•
Used to define a file
May have 3 to 19 entries, or more
1 Primary, many Secondary
Is considered an array
• Must be in order
• Must be contiguous (no gaps)
• Entire Set has Checksum
NYC4SEC Meet-up Group – June 11th, 2014
File Directory Entry
• 0x85 or 0x05 Entry
• Primary Entry
• Set Checksum (16 bits)
• Not modified on file delete
• Secondary Count
• # Secondary entries that follow
• File Attributes
• Timestamps
NYC4SEC Meet-up Group – June 11th, 2014
Timestamps & Time Zones
• 3 Timestamps (MAC)
• 32 bit DOS Date/Time
• Local Machine Time
• 10ms Offset (MC)
• TZ Offset (MAC)
• 15 minute increments
• 7 bit signed number
• ±16 hours
• Present with UTC support
NYC4SEC Meet-up Group – June 11th, 2014
Timestamp Accuracy
•
•
•
•
•
FAT32 – Last Access – Date only
exFAT – Last Access – Date/Time
All DOS DATE/TIME Double Seconds
10ms adds 0-1990 ms to time
10ms only for Create/Modify
NYC4SEC Meet-up Group – June 11th, 2014
Timestamps
Timestamp
CreationTime
EXFAT
Stored in UTC if available, else in
local time
LastAccessTime
10 millisecond granularity
Stored in UTC if available, else in
local time
ChangeTime
LastWriteTime
2 second granularity
Not Supported
Stored in UTC if available, else in
local time
10 millisecond granularity
NYC4SEC Meet-up Group – June 11th, 2014
Timestamp Reliability
• Timestamps appear to be updated when
the file is created or modified.
• Last Accessed Timestamp appear to be
updated when file is created or modified.
• Last Accessed Timestamp appear NOT
modified on file read.
• Forensics Implication on MAC time
analysis
NYC4SEC Meet-up Group – June 11th, 2014
File Attributes
Attribute
Offset
Size
Mask
Reserved2
6
10
Archive
5
1
0x20
Directory
4
1
0x10
Reserved1
3
1
System
2
1
0x04
Hidden
1
1
0x02
Read-Only
0
1
0x01
NYC4SEC Meet-up Group – June 11th, 2014
File Directory Entry
Type
# Secondary Entries
Set Checksum (0x92D4)
Attributes (0x0020 = Archive)
Offset
0000
0010
0
1
2
3
4
5
6
7
85 04 D4 92 20 00 00 00
A8 00 EC EC
Accessed
8
A
B
C
D
E
F
F1 62 BA 3A
EC 00 00 00 00 00 00 00
Modified 10ms
Create 10ms
9
Create
Modified
TZ Offset CMA EC = GMT-5
NYC4SEC Meet-up Group – June 11th, 2014
Formatted File Directory Entry
Root Entry Type Read is:
Checksum:
Calculated Checksum is:
Secondary Count
File Attributes:
Create Timestamp:
Last Modified Timestamp:
Last Accessed Timestamp:
10 ms Offset Create
10 ms Offset Modified
Time Zone Create
Time Zone Modified
Time Zone Last Accessed
85 Directory Entry Record
92D4
92D4 Size Directory Set (bytes): 160
004
0020 Archive
3B866244 12/06/2009 12:18:08
3ABA62F1 05/26/2009 12:23:34
3B866244 12/06/2009 12:18:08
A8 168
00
0
EC 236 Value of tz is: GMT -05:00
EC 236 Value of tz is: GMT -05:00
EC 236 Value of tz is: GMT -05:00
NYC4SEC Meet-up Group – June 11th, 2014
Stream Extension Directory
Entry
•
•
•
•
•
•
•
0xC0 or 0x40 Entry
Secondary Entry
Length of Name
Length of File (2 of them)
Cluster address of first data block
Name Search Hash value
Secondary Flag
• FAT Invalid
• Allocation Possible
NYC4SEC Meet-up Group – June 11th, 2014
Stream Extension Directory
Entry
Flags (Alloc Possible/Fat Invalid)
Entry
Length of File Name (0x28= 40)
Name Hash (0x3CAD)
Offset
0000
0010
0
1
2
3
C0 03 00 28
00 00 00 00
4
5
6
7
00 00
8
9
A
B
C
D
E
F
00 00 00 00
00 00 00 00
Cluster (5)
Data Length 0x011d461f = 18,695,711
NYC4SEC Meet-up Group – June 11th, 2014
Parameters for Samples
Bytes Per Sector: 2 to the 09 power is: 512
Sectors Per Cluster: 2 to the 08 power is: 256
Bytes per Cluster: 131072 (128K)
NYC4SEC Meet-up Group – June 11th, 2014
Formatted Stream Extension
Root Entry Type Read is: C0 Directory Entry Record,
Stream Extension
Secondary Flags: 03
Flag Bit 0: Allocation Possible
Flag Bit 1: FAT Chain Invalid
Length of UniCode Filename is: 40
Name Hash Value is:
AD3C
Stream Extension First Cluster
5
Cluster
5 is Allocated
Stream Extension Data Length
18695711 Bytes
Slack:
83487 Clusters Used:
143
Stream Extension Valid Data Length
18695711 Bytes
Slack:
83487 Clusters Used:
143
NYC4SEC Meet-up Group – June 11th, 2014
File Name Extension Directory Entry
• 0xC1 or 0x41 Entry
• Secondary Entry
• Secondary Flags
• Allocation not possible
• FAT Invalid
• 15 Characters (30 bytes) of Name
• Name in 16 Bit Unicode
• In order (FAT32 LFN was reversed)
• Up to 17 max, total 255 character
NYC4SEC Meet-up Group – June 11th, 2014
File Name Extension Directory Entry
Offset
0
1
2
3
4
5
6
7
8
9
A
B
C
D
E
F
0000
C1 00 62 00 75 00 73 00
Á.b.u.s.i.n.e.s.
0010
73 00 5F 00 6F 00 66 00
s._.o.f._.s.e.c.
69 00 6E 00 65 00 73 00
0000
C1 00 75 00 72 00 69 00
Á.u.r.i.t.y._._.
0010
62 00 75 00 73 00 2D 00
b.u.s.-.1.0.5.-.
74 00 79 00 5F 00 5F 00
0000
C1 00 33 00 32 00 6B 00
Á.3.2.k.b.p.s...
0010
6D 00 70 00 33 00 00 00
m.p.3...........
62 00 70 00 73 00 2E 00
5F 00 73 00 65 00 63 00
31 00 30 00 35 00 2D 00
00 00 00 00 00 00 00 00
File Name = business_of_security__bus-105-32kbps.mp3
NYC4SEC Meet-up Group – June 11th, 2014
Significance of “not in use” flag
• 0x05, 0x40 & 0x41 Entries
• “Not in use” may mean deleted files
• May also be reallocated rename
• Set Checksum not changed when entries
marked “not in use”
NYC4SEC Meet-up Group – June 11th, 2014
Closing
Problems Observed
Summary
Q&A
Contact Information
References
EXFAT
NYC4SEC Meet-up Group – June 11th, 2014
Problems Observed
•
•
•
•
•
•
•
•
•
•
•
Looking at Forum Posts
Google Dork on “exFAT”
People getting thrown into exFAT and Lost
Conversion between exFAT & Fat32/NTFS, How-to
Corruption between Windows and Mac
Should File Defragmentation be done?
Repartitioning
Timestamp differences, and incompatibilities
Vendor cross compatibility
Chkdsk not cleaning disk
Users want large files (>4GB) not Large Volumes
NYC4SEC Meet-up Group – June 11th, 2014
Summary
•
•
•
•
•
•
exFAT is still a relatively new FS
Need for exFAT support in forensics tools ↑
Inconsistent Implementations of exFAT
Compatibility across OS needed
Tools & Utilities Need Improvement
Need to Tool Up
NYC4SEC Meet-up Group – June 11th, 2014
Q&A
NYC4SEC Meet-up Group – June 11th, 2014
Contact Information
• E-mail:
•
•
•
•
rshullic@earthlink.net
exFAT@mindspring.com
Blog: rshullic.wordpress.com
Blog: shullich.blogspot.com
Linkedin: www.linkedin.com/in/RobertShullich
Twitter: rshullic
Credit Cookie
NYC4SEC Meet-up Group – June 11th, 2014
NTFS 232-1 Clusters
Cluster size
512 bytes
1024 bytes
2048 bytes
4096 bytes
8192 bytes
16384 bytes
32768 bytes
65536 bytes
NTFS Max Size
2,199,023,255,040 (2TB)
4,398,046,510,080 (4TB)
8,796,093,020,160 (8TB)
17,592,186,040,320 (16TB) (Default)
35,184,372,080,640 (32TB)
70,368,744,161,280 (64TB)
140,737,488,322,560 (128TB)
281,474,976,654,120 (256TB) (Maximum)
NYC4SEC Meet-up Group – June 11th, 2014
ReFS
Resilient File System
Coming to a Windows System soon
http://blogs.msdn.com/b/b8/archive/2012/0
1/16/building-the-next-generation-filesystem-for-windows-refs.aspx
NYC4SEC Meet-up Group – June 11th, 2014
References
Sans Reading Room:
http://www.sans.org/reading_room/whitepapers/forensics/rss/reverse_eng
ineering_the_microsoft_exfat_file_system_33274
SANS Summit ExFAT Presentation:
exFAT (Extended FAT) File System – Revealed & Dissected
Jeff Hamm & Robert Shullich, July 2010
https://digital-forensics.sans.org/summit-archives/2010/10-exfat-ham.pdf
NYC4SEC Meet-up Group – June 11th, 2014
References
Microsoft Patent US8583708, “Extensible File System”
Retrieved June 9, 2014 from
https://www.google.com/patents/us8583708
Microsoft Patent US8321439, “Quick Filename Lookup
Using Name Hash”. Retrieved 06/09/2014 from
https://www.google.com/patents/US8321439
NYC4SEC Meet-up Group – June 11th, 2014
References
Microsoft Patent US8606830, “Contiguous file
allocation in an extensible file system” retrieved
06/09/2014 from
http://www.google.com/patents/US8606830
Microsoft Patent US8024383, “Fat directory structure
for use in transaction safe file System” retrieved
06/09/2014 from
https://www.google.com/patents/US8024383
NYC4SEC Meet-up Group – June 11th, 2014
References
ExFAT overview
http://ntfs.com/exfat-overview.htm
Data Recovery Concept: Extended File System
(exFAT)
http://www.active-undelete.com/xfat_overview.htm
CIPA Standard DC-009-2010 (DCF)
http://www.cipa.jp/std/documents/e/DC-009-2010_E.pdf
CIPA Standard DC-008-2012 (Exif)
http://www.cipa.jp/std/documents/e/DC-008-2012_E.pdf
Comparison of File Systems
http://en.wikipedia.org/wiki/Comparison_of_file_systems
NYC4SEC Meet-up Group – June 11th, 2014
References
The Extended FAT file system - Differentiating with
FAT32 file system - Keshava Munegowda , Venkatraman
S, Dr. G T Raju
http://events.linuxfoundation.org/images/stories/pdf/lceu11_mune
gowda_s.pdf
File System Functionality Comparison
http://msdn.microsoft.com/enus/library/windows/desktop/ee681827(v=vs.85).aspx
NYC4SEC Meet-up Group – June 11th, 2014
Resume
http://jjcweb.jjay.cuny.edu/d4cs/faculty/Shu
llich Robert.pdf
NYC4SEC Meet-up Group – June 11th, 2014