Forensics
Thursday, March 30, 2023
2:43 PM
Thumbnail ==> Last 2 lectures
Chapter 1:
Chapter 3:
-Locard's exchange principle: The perpetrator of a crime will bring something into the crime
scene and leave with something from it.
-Examples of technologies in detecting crimes/finding evidence: Automatic License Plate
Recognition (ALPR), Shadow Tracing, Facial Recognition, Big Data Forensics.
- Computer Forensics: A branch of forensic science pertaining to legal evidence found in
computers and digital storage mediums. (Extracting digital evidence from suspect systems while
maintaining its legal worth). It's Goal is to explain the current state of a digital artifact.
-Digital Investigation: An investigation where a digital device is involved in an incident or crime;
was used to commit a crime, was used to violate a policy or the law, the nature of the
investigation requires a variety of technology during an investigation. It also develops and tests
hypotheses that answer questions about digital events.
-Forensics: Use of science or technology in the investigation and establishment of facts or
evidence in a court of law.
-Digital investigation: a process where we develop and test hypotheses that answer questions
about digital events.
-Digital forensics investigation: A more restricted form digital investigation which involves using
legal requirements for submitting evidence into a court of law.
-Digital evidence: Information that is stored or transmitted in a machine-readable form, and
maintains enough integrity and legitimacy to be used in a court of law.
-Hash function: Used to maintain integrity of evidence.
-Chain of Custody: Evidence must be strictly secured during its life cycle (Must be proved to
challenge any claims stating that the evidence was tampered with).
Chapter 2:
-Data Acquisition: A process that encompasses all the procedures involved in gathering digital
evidence including cloning and copying evidence from any electronic source. Types: Static
(Powered-off system, no data alteration), Live acquisition(System is running, may alter data,
best against encrypted disk)
-Digital Evidence Storage Formats: Raw format (dd), Proprietary (Expert Witness
Format, .E01, .E02), Advanced Forensics Format (Compressed, designed with extensibility)
-Data Acquisition Methods: Disk to Image File, Disk to Disk, Logical (Acquire certain files),
Sparse (Similar to logical but it also collects fragments of unallocated data)
-Acquisition Layers: Application (Logical/Sparse), File System, Partition/Volume, Hard Disk.
-Write Blockers: Doesn't allow data to be written, Doesn't allow the disk to be mounted with
write-access, Read commands only, HW or SW.
-Order of Volatility: Collect the data that will perish first before anything else (Registers - ns,
Main Memory - ns, Network state - ms, Running processes - sec, Floppies, HDD, backups years, CD-ROMS, printouts - 10s of years.)
-Digital Forensic Imaging: Process and tools used in copying a physical storage device for
conducting investigations and gathering evidence. Every bit of data is copied.
Challenges of Data Acquisition: Password protected devices, Data Encryption, Finding Relevant
Data, Lack of standardization, Lack of training.
Chapter 4:
-MBR: Only supports 4 primary partition table entries, 32-bit values to indicate the starting
offset and length of a partition = 2.19TB Max disk space.
GPT (GUID Partition Table): 128 primary partitions supported, 64 bit values to indicate starting
offset and length of a partition = 9.4 zetta bytes (9.4*10^21 bytes), Redundancy by writing the
GPT header and partition table at the beginning of the disk and also at the end of the disk.
CRC32 checksum for the header and one for partition table to verify contents.
-First 512 byte in DOS style partition system = MBR / LBA = 0 CHS = 001
-Master Boot Code, used at boot time, examines partition table, identifies system partitions and
finds partition start location, loads copy of partition boot sector into memory, transfers
execution
-Code area ==> first 440 bytes.
-Optional Disk signature == 4 bytes
-Usually Nulls ==> 2 bytes
-Table of primary partitions ==> starts at 0x01BE or 446 decimal, 64 bytes (4 partitions)
-MBR signature ==> last 2 bytes.
-Partition Table entries contain: start CHS, Ending CHS, Starting LBA, Number of sectors in
partition, Type of partition, Flags.
- Partition types:
-A file system is a method for storing a file in an efficient way that allows fast access and retrieval of
that file.
-Disk allocation Methods:
a- Contiguous allocation: Contiguous blocks, table contains file name (Descriptor), starting block, and
size. Bad for increasing file size, because the following blocks might be occupied and it must be
contiguous.
b- Linked Allocation: Each block contains disk address of next file block, table contains file name
(Descriptor), and starting block. Problem with that is the slow file read for large files, and it does not
support random or direct access. USED IN FAT FILE SYSTEM.
3
-The surface is divided into circular rings called tracks, these tracks are
further divided into sectors. A Cylinder is a set of tracks with the same
radius. The head is what is used to read the data from either sides of the tracks.
-CHS is what is used to identify a sector, it is being replaced with LBA (Logical
Block Address). Each LBA defines a 512 bytes sector into the device's storage space. {0, n-1}; n =
max available sectors. N.B: CHS DOES NOT WORK FOR SSD, ONLY LBA DOES.
-Volume is a logical assembly of one or more partitions.
-A partition is a collection of addressable sectors on storage devices.
-Low-level formatting ==> Making the hard disk
-High-level, logical formatting, or just formatting ==> Partitioning the hard disk
-A partition is a collection of addressable sectors in a disk (Some partitions can be hidden, like
the first one that contains system files). Partition systems depend on the OS and not the hard
disk interface.
-The LBA starts at 0 at the first sector of the disk, which means that if there are multiple disks
the LBA is relative to each one. This is also known as the physical address of a sector.
-The volume address (or Logical disk volume address) is the address of a sector relative to the
start of its disk volume.
-If we are talking about one hard disk here, the physical address of a sector is the same as the
logical disk volume address.
-We can merge multiple physical disks, the result is a
-Logical partition volume address is the address of a sector relative to the start of the partition.
-If a sector does not belong to a partition, it does not have a logical partition volume address
-File systems can use data storage devices or they might provide access to data on a file server
(SMB, NFS, etc..), or they may be virtual and exist only as an access method for virtual data
(procfs). Logical storage unit (file)
-Modern OSes can support multiple File Systems, and there are 2 possible interfaces (textual
and graphical).
2
FF
1
c- Indexed Allocation: Each block contains the file descriptor and all the blocks used in the file.
Problem with that is the redundancy in the allocation space. And no way to know size of the index in
advance.
-Each disk is split into partitions that are formatted with a file system.
-Essential data for finding a file: File name, size, and address
-No Essential data for finding a file: Access time, and security permissions.
-Data in file systems can be divided into five categories: File system data, File Content, Metadata, File
Name data, Application data.
-File system data: general info like the location of data structures, data unit sizes. Like a map for the
FS.
-Content: The data in units (Clusters or Sectors)
-Metadata: Data that describes files, such as Location of file content, size of file, creation time,
modification time, Access control info. Examples include: FAT directory entries, NTFS Master File Table
(MFT), and Ext3 inode structures.
-File name data: name of file.
Application: Data that provides special features, such as user quota info, file system journals.
-The information obtained from the file system info category:
--general layout information
--or the version of file system,
--or the application that created the file system
--or the creation date, and the file system label.
--First sector of the file system, and data structure is standard
New Section 1 Page 1
Type
Description
0x00
Empty
0x01
DOS 12-bit FAT
0x07
Windows NT NTFS
0x82
Linux swap
0x083
Linux native partition
-The extended partition can serve as ANOTHER PARTITION TABLE (lab-1). They can only be
created on basic MBR disks.
-Unlike primary partitions, you do not format an extended partition with a file system and then
assign a drive letter to it. Instead, you create one or more logical drives within the extended
partition.
- It can be nested (Last partition = extended table and last partition of extended table = another,
and so on) BUT, the primary Extended partition will layout the start and end of ALL EXTENDED
PARTITIONS
Most removable media (e.g., USB flash drive) also have partitions, and use the same structures
that hard disks use. However, by default, they only have one partition in the fourth slot.
By default, Windows will not allow you to multi-partition removable media.
What if we want multi-partition removable media and a bootable USB device?
In order for this to happen, we need to flip the Removable Media Bit (RMB) on the USB Device.
-The information obtained from the file system info category:
--general layout information
--or the version of file system,
--or the application that created the file system
--or the creation date, and the file system label.
--First sector of the file system, and data structure is standard
-Consistency Check: The size of the file system = the size of the volume in which it is located?
-There could be volume slack, where the partition volume is larger than the allocated space for the file
system.
-The information obtained from the content category:
--Deleted files
--Low level string searches
Most removable media (e.g., USB flash drive) also have partitions, and use the same structures
that hard disks use. However, by default, they only have one partition in the fourth slot.
By default, Windows will not allow you to multi-partition removable media.
What if we want multi-partition removable media and a bootable USB device?
In order for this to happen, we need to flip the Removable Media Bit (RMB) on the USB Device.
This will tell Windows that the flash drive is a fixed disk and not removable media. For example,
Lexar has a tool for this purpose.
Chapter 5:
-FAT file systems were developed by Bill Gates and Marc McDonald in 1976/1977 and it was used in
MS-DOS and Windows (Till Windows Me). It is still used in mobile devices and removable storage
devices.
-Sectors can have physical and logical addresses, physical is relative to drive start and logical is relative -FAT filesystem uses Linked allocation
-FAT filesystem consists of 3 regions:
to volume start.
--Reserved area: file system category
-Allocation strategies: First available, next available, best fit.
--FAT area: primary and backup FAT structures
--Data area: clusters used for storing file and directory content.
-Bad sector: cannot be written on or read from due to physical damage or inconsistency in the parity
check bits, cannot be rPepaired, but can be marked as unusable.
-Analysis Techniques:
--Data Unit Viewing: See the content of a data unit in a file system.
-- Logical File System-Level Searching: Search for a string
--Data Unit Allocation Status: Extract unallocated data units, by examining the allocation bitmap
-Orphan Data Units: Allocated data units without a corresponding metadata structure. Every data unit
-FAT Partition Boot sector:
should have one and only one metadata entry.
--Information on how to access the volume (Bytes/Sector, Sectors/Cluster, ….) , MBR uses it to load
-Bad sectors are not the only check that can be made, as bad sector does not necessarily mean
kernel files.
damaged. Another check is made.
-File Allocation Tables:
--2 copies (Primary and secondary or backup), Stores information about each cluster.
-If all unallocated space contains random values or zeroes, then there is a decent chance a wiping tool
was used.
-Common methods for data wiping:
--US DoD 5220.22-M:
--Peter Gutmann's method: 35 times of overwriting with random characters
--Random overwrite algorithm:
-Three perspectives to data deletion:
--Forensic investigator: Evidence showing the existence of a wiping tool
--Data privacy protection: Securely delete or remove the confidential data upon decommission of the
storage device
-The Root Directory is the file in the top of the files' hierarchy, an example for that is the C:\ in
--Anti-forensics: Criminals using anti-forensic methods to cover up their wrongdoings by deleting
windows and \ in linux.
evidence.
-Data included in file metadata:
--File location, size, timestamps (creation, modification) access control info or permissions on a file.
==> istat tool
-Slack space: size of a file is not a multiple of a data unit size. (File is 100 bytes, and the cluster size is
2048, then there is 1948 bytes of slack space). It is considered allocated data and it may contain
hidden info.
-Booting instructions: BIOS instructions ==> Disk Sector 0 (MBR) Instructions ==> Partition Sector 0
Instructions** ==> Windows OS
(** Resides on Disk 0).
Data structure of a partition Boot Sector: (Cannot know the max root dir folders for FAT32)
The rest is the data area.
-Root Directory has many entries, each entry represents a file or a subfolder
--Name, attribute, create time, last access date, modified date/time, starting cluster number, file size.
-Metadata-based file recovery scenarios:
FILENAME:
--File 1 - MD entry 100 allocated data unit 1000, then it deleted it. Then File 2 - MD entry 200 allocated -8.3 convention, 8 characters (first byte then the remainder)+ 3 extension
the same data unit 1000, which should we recover (The most recent, it will be shown in the data unit)
--File 1 - MD entry 100 allocated data unit 1000, then it deleted it. Then File 2 - MD entry 200 allocated Flags for directory entry:
the same data unit 1000, which should we recover, then File 3 allocated MD entry 300 and data unit 0x01: Read only, 0x02: Hidden, 0x04: Sys file, 0x08: Vol label, 0x0f: Long fname, 0x10: Dir, 0x20: archiv
1000, then it got deleted and file 4 allocated MD 300 and data unit 2000. Which file to recover
between File1 and File2 (NONE, because there is no way to tell which is the most recent since File 3
If a file has a long name, it will have a LFN entry that will precede the normal entry:
overwritten them)
-- Allocation order varies among different Oses
Windows create short names for LFNs by using the first 6 chars appended with '~' Filena~.txt
-The NIST National Software Reference Library (NSRL) contains hashes of files that are found in
operating systems and software distributions. (Good hashes ==> Ignore database, Bad hashes ==>
Alert database).
-In a Fat Filesystem --> Cluster 2 is always the first one (Min number of clusters):
--In FAT12/16 Filesystem, Cluster 2 follows the root directory, and in a FAT32 filesystem, cluster 2 is
the first data unit of the data area.
-MAC times ==> Modification, Access, Creation
-File System Journals: A file system in which the hard disk maintains data integrity in the event of a
system crash or if the system is otherwise halted abnormally.
-Data carving: Carving out deleted files based on known file extension headers and footers (JPEG, PDF,
TXT, …)
-Sector address of cluster C = (C - 2) * (# of sectors per cluster) + (sector address of cluster 2)
===>THE DISK SPACE BEFORE CLUSTER 2 HAVE NO CLUSTER NUMBER ASSOCIATED TO THEM
-Cluster C = ( (S - sector address of cluster 2) / (# of sectors per cluster) ) + 2
New Section 1 Page 2
Fat structure size = 12 bits for FAT12
16 bits for FAT16
32 bits for FAT32
Chapter 5 cont:
Cluster 2 for readme.txt ==> cluster 2 starts at 0400 0000, points to cluster 4 then points to 5
and so on till 11
Note: Cluster bytes and file size is read in little endian.
Also, the file size is 0 if directory
Search methods: File signature (hash), Keyword search (grep, srch_str)
WHEN A FILE IS DELETED: Its first character in the FAT DATA is replaced by 0xE5 and FAT
Entries are wiped. The file size and start cluster are still there, which makes the file
restoration possible.
Chapter 6
-Data structures recovering syntax: Layout, Size, Offset
-//
//
//
semantics: Context Meaning
-2 Options for RE, Static analysis or dynamic
-Differential forensic analysis compares two different digital forensic images (or, more
generally, any pair of digital artifacts) and reports the differences between them.
Chapter 7
-NTFS offers what FAT does not: Reliability, Security, Scalability/Performance
--Reliability: Journaling, where changes are logged before they are taking effect, helping recover
from crashes.
--Security: Better file permissions, can be set on file or folder level
--Scalability: Larger file size (FAT only supports 4 GB, while NTFS supports 16 exabytes)
--Performance: NTFS supports compression and encryption.
-NTFS was designed to improve Read/Write, Search, and data recovery for large hard disks
B-Tree Checks:
-FAT is slower in retrieval because it searches through the chain of allocation units/directory entries
A B-tree of order n (the maximum number of children for each node) consists of
a root which has at least 2 children (if not a leaf), and all other nodes (except the root) have
at least children ceil(n/2) (ceil(n/2)- 1 keys) . Every node has at most n children (n-1 keys (or
data values)) .
ceil(n/2)<=k<=n
Where k represents the number of children
A B-tree is kept balanced by requiring that all leaf nodes are at the same depth.
Ordering: The keys (values) in the data items in all nodes are in increasing order.
-In NTFS, the first information on the volume is the Partition Boot Sector which starts at Sector 0
and can be up to 16 sectors long.
-The first file is a Master File Table (MFT) which holds info about all files and folders in the volume
-Layout of NTFS Volume:
MFT clusters can be non-continuous
-Windows creates MFT as small as possible and only expands it when there are more entries
-To decrease the possibility of fragmentation, NTFS uses the MFT Zone (12.5% of the file system
allocated to MFT)
-PBT: Contains starting location of the MFT, cluster size (1-128 sectors, commonly 8 sectors), size of
each MFT entry (usually 1024)
-MFT: Relational DB table in which attributes of each file or directory is represented by an entry
-$BITMAP has a $DATA attribute that has one bit for every cluster in the file system
-The first MFT entry (entry 0) points to the MFT itself describing its size and location, the next one
(entry 1) points to MFTMirr or the backup copy of the first MFT entry.
-Each cluster address starts with 0
-Sector address = Cluster * Sectors/Cluster
-Clusters are allocated to files by a best fit algorithm
-MFT entry always starts with signature attribute ("FILE" if okay, "BAAD" if there is an error).
-In NTFS There are some File System Metadata Files (also known as System Files), which are used by
NTFS to store metadata and implement the file system. They are critical for the system to work.
-There are 2 kinds of attributes:
--Resident: Inside MFT entry
--Non-resident: Inside FS (Cluster runs)
---Cluster runs or data runs, are defined
by their start and length in clusters
New Section 1 Page 3
Techniques to analyze NTFS
-Finding a cluster
-Allocation status of a cluster
-Extracting unallocated space
-It is possible that the filename is not
the first attribute, depends on the MFT
entries.
-We can know the start of the attributes and their lengths from the MFT (Can know each one)
-NTFS uses indexes to lookup faster (Sorted filenames inside directories) Balanced tree structure
--B-tree: left is smaller and right is greater, more than 2 paths diverge from a single node
New Section 1 Page 4