%
% JFFS3 design issues.
%
% Copyright (C) 2005 Artem B. Bityuckiy, dedekind@infradead.org
%
% $Id: JFFS3design.tex,v 1.6 2005/06/16 07:18:30 dedekind Exp $
%

\documentclass[12pt,a4paper,oneside,titlepage]{article}
\usepackage{hyperref}
\usepackage{html}
\usepackage{amssymb}
\usepackage{longtable}

\ifx \pdfoutput \undefined
\usepackage{graphicx}
\else
\pdfpagewidth=210mm
\pdfpageheight=297mm
\usepackage[pdftex]{graphicx}
\fi

% Set pages layout.
% A4 paper has 210mm x 297mm
% TeX automatically makes 25.4mm left and top indents
\oddsidemargin=0mm  % 25.4mm by default
\textwidth=159.2mm  % 25.4mm right indent
\topmargin=0mm      % 25.4mm by default
\textheight=241.6mm % 30mm bottom indent
\headheight=0mm
\headsep=0mm

% Define TODO command
\newcommand{\TODO}[1]{[{\textbf{TODO}: #1}]\marginpar{\large \textbf{?!}}}

\begin{document}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
%  TITLE PAGE
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\title{JFFS3 design issues}
%\author{Artem B. Bityuckiy}
\begin{titlepage}
\vspace*{5cm}
\begin{center}
\Huge{\textbf{JFFS3 design issues}}\\
\vspace{1cm}
\large{Artem B. Bityuckiy\\
dedekind@infradead.org}\\
\vspace{13cm}
\large{Version 0.2 (draft)}\\
\vspace{0.5cm}
Jun 16, 2005
\end{center}
\end{titlepage}
%\maketitle

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% ABSTRACT
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\pagestyle{empty}
\begin{abstract}
This document discusses various JFFS3 high-level design aspects.
Additionally, it defines standard JFFS3 dictionary and terms.
\end{abstract}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% TABLE OF CONTENTS
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\tableofcontents
\newpage

\pagestyle{plain}
\pagenumbering{arabic}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% JFFS3 MOTIVATION AND GOALS
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{JFFS3 motivation and goals}
JFFS2, the Journalling Flash File System version 2 [\ref{ref_JFFSdwmw2}]
is widely used in the embedded systems world. JFFS2 was originally designed for
small NOR flashes ($<$ 32MB), and NAND support was added later. The first NAND
flashes were also small enough, but grew in size very quickly and are currently
much larger then 32MB JFFS2 was originally designed for (e.g., Samsung produces 2GB NAND
flashes [\ref{ref_SamsungNANDlist}]). Unfortunately, owing to its design,
JFFS2 cannot work well on such large flash chips. The following problems
arise:

\begin{itemize}
\item the \emph{mount time} becomes too high;
\item the \emph{memory consumption} becomes enormous;
\item the \emph{access time} to large files (the \texttt{open()} call) becomes too
high.
\end{itemize}

Due to the design of JFFS2, the three above parameters (mount time, memory
consumption and the file access time) depend linearly on the size of flash and files.
To put it differently, JFFS2 \emph{does not scale}. But in spite of the scalability
problems, JFFS2 has many advantages:

\begin{itemize}
\item very economical flash usage -- data usually take as much flash
space as they actually need, without wasting much space as in case of
traditional file systems for block devices;
\item admitting of very efficient utilization of "on-flight" compression which
allows to fit a big deal of data into the flash;
\item very quick read and write operations;
\item natural unclean reboot robustness;
\item good wear-leveling.
\end{itemize}

The \textbf{goal of the JFFS3 project} is to develop a scalable flash file system
which may be used on large scale flashes. JFFS3 must meet the following
requirements:

\begin{enumerate}
\item provide fast mounting;
\item consume few memory although it isn't forbidden to consume much RAM providing
the RAM is used \emph{for caching} and may be any time freed;
\item be tolerant to unclean reboots;
\item all the JFFS3 characteristics must scale well up to 1TB flash chips.
\end{enumerate}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% JFFS3 design overview
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{JFFS3 design overview}
This section overviews the main JFFS3 design aspects without any thorough
description and is to be regarded an introduction to the design of JFFS3.

Firstly, the utter difference between JFFS3 and JFFS2 designs must be noted.
JFFS3 design is greatly impacted by the scalability issues and cannot be just an
improvement over the design of JFFS2 (which is not scalable) -- some completely
different principles must be applied to JFFS3.

The following are the most essential things we want JFFS3 to meet:

\begin{itemize}
\item The JFFS3 memory consumption must not depend on the size of the JFFS3
partition, the number of inodes in the file system, size of files and
directories and the like. Of course, JFFS3 must be able to use the advantage
of the available
RAM, but only for different kinds of \emph{caches} which may be flushed back to
flash any time in case of memory pressure.

\item All the JFFS3 characteristics ought to vary not faster then $log(S)$,
where $S$ is the size of
JFFS3 partition. JFFS2-like linear dependencies are forbidden.

\item The flash wear-levelling have to always be kept in mind and shouldn't
suffer.

\item The fact that flash devices may be not very reliable - bits may flip, bad
blocks may appear, etc should be taken into account.

\item It is obligatory to make JFFS3 be very tolerant to unclean reboots,
just like JFFS2.
\end{itemize}

To meet the above requirements, the following design decisions were made.

\begin{itemize}
\item JFFS3 is \emph{block-oriented}, much like traditional file systems for hard
disk drives (on the contrary to JFFS2, which treats flash eraseblocks as contiguous
space).

\item JFFS3 orients to \emph{large-scale NAND flashes}. Other flash types may be
supported too, but they should emulate NAND flash - e.g., JFFS3 assumes there is
an OOB area on each NAND page (sector), etc.

\item There has to exist a \emph{superblock} containing the sufficient amount of
information to perform quick JFFS3 mount. As flash chips don't admit of
rewriting of the same place several times without the erasure of the whole
eraseblock and because of the the wear-levelling
issues, some complex superblock management schemes are compulsory.

\item Since the compression is very difficult to implement in the block-based
approach, it was decided not to support it.

\end{itemize}

The way how JFFS3 stores the filesystem data is similar to the
approach of \emph{\mbox{ReiserFS}} file system [\ref{ref_Reiser4}].
All the file system
objects (inodes, file data blocks, directory entries, extended attributes, etc)
are kept in one large $B^+$-tree. Effectively, the whole JFFS3 file system may be
regarded as one large $B^+$-tree, excluding some superblock-related things,
which don't belong to the tree. Please, refer to [\ref{ref_Reiser4}] if you are
interested in the merits of such a tree-based approach.

The tree consists of \emph{nodes}, which may occupy either one sector
(\emph{branch nodes}) or one block (\emph{leaf nodes})
(please, refer to the definitions chapter \ref{ref_SectDefinitions}
for the definitions of \emph{sector}, \emph{block}, \emph{branch node}, etc).
All the objects stored in the tree are actually kept in the tree's leaf nodes,
while the non-leaf nodes contain only (\emph{key}, \emph{link}) pairs (see the
definition of $B^+$-tree in any book devoted to computer algorithms).

Each file system object has an associated key which provides fast object search.
Different objects has different keys, but keys may occasionally collide. In
that case, all the colliding objects are kept together and JFFS3 selects the
right one by means of the simple linear search.

When files are created, changed or deleted, the corresponding leaf nodes of the
tree are changed. When a node is changed, JFFS3 writes the node update to
another place on flash, not to the same place where it was before. 
To keep the tree consistent, the upper
node, which points to the leaf, have to also be changed (the link update), and
so forth up to the root node. I.e., the more levels the tree has,
the more writes any single node update requires. The same story with Garbage
Collector - if it moves any tree node, the whole path up to the root must be
updated.

With the tree-based approach, mount essentially means locating the root node
of the tree. The root node is referred from the JFFS3 superblock. The superblock
is found relatively quickly (see section \ref{ref_SectionSB}), so the file
system mount is also fast.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% SUPERBLOCK MANAGEMENT
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Superblock management} \label{ref_SectionSB}
The superblock (SB) is the data structure which contains information about the
whole JFFS3 file system and its layout. When the file system is being mounted,
the superblock should be found and read.

In case of traditional file systems the
superblock usually resides at a fixed position of the disk and may be quickly
found.
Conversely, due to the "out-of-place write" flash property we
cannot guarantee the fixed position of the JFFS3 superblock. Things are getting
even more complex because of the need to provide gentle flash wear levelling -- we
cannot just reserve several erasable blocks for the SB unless we've guaranteed
that these SB~eraseblocks are not worn out before the other
eraseblocks.

It is important to note that we're using somewhat confusing terminology
(see section~\ref{ref_SectDefinitions} for all the definitions in the document):

\begin{itemize}
\item \textbf{erasable block, eraseblock} -- the minimal erasable unit of the
flash chip; eraseblock consists of sectors;

\item \textbf{sector} -- the smallest writable unit of the \emph{flash chip},
i.e. the NAND page in case of NAND; the
minimal size of the sector is assumed to be 512KB;

\item \textbf{block} -- the smallest writable unit of the \emph{file system};
blocks are 4KB in size, like the size of the RAM page in most architectures;
blocks may consist of multiple sectors.
\end{itemize}

The reason why we chose this terminology is to be consistent with the
terminology used in case of ordinary file system systems and the Linux block I/O
layer. Different flash chips use different terms and we decided to select more
settled terminology.

Thus, we have the following two requirements that ought to be met:
\begin{itemize}
\item JFFS3 must be able to quickly find the superblock;
\item the superblock managing techniques must not spoil the overall flash wear
levelling.
\end{itemize}

JFFS3 reserves the first two good erasable blocks of the JFFS3 partition for the
superblock managing. We call these two eraseblocks \textbf{anchor eraseblocks}
or \textbf{anchor area}.
Note, the superblock takes only \emph{one sector} in JFFS3.

Suppose we are storing the JFFS3 superblock in the anchor area. 
The superblock updates are written sequentially to adjacent
sectors 1,2,3,... of the anchor area.
When one of the anchor eraseblocks becomes full, the updates are
written to the other anchor eraseblock and the first one is erased. Thus, the
unclean reboot tolerance is ensured. There may be many copies of the superblock
in the anchor area, but only one copy is valid. Assigning version
numbers to the copies we may find the valid SB. Since the copies are written
sequentially, we may even use the binary search algorithm.

It stands no reason that the above scheme is bad because the anchor area
will be worn out much earlier then the rest of flash. To prevent this 
we introduce \textbf{SB~eraseblocks} where we consequently write our
superblock updates. Anchor
eraseblocks will contain references to the SB~eraseblock.
When the SB~eraseblock
becomes full, JFFS3 picks another SB~eraseblock, writes the superblock update to the new
SB~eraseblock, and updates the reference to the SB~eraseblock in the anchor area.
Note, that the SB~eraseblocks fit the main wear-levelling scheme,
just like any other eraseblocks.

%
% The figure with the SB management scheme
%
\begin{figure}[h]
\begin{center}
\includegraphics[width=159mm,height=120mm]{sb_mgmt}
\end{center}
\caption{The superblock management scheme with one level of SB~eraseblocks.}
\label{ref_FigureSBMgmt}
\end{figure}

Figure \ref{ref_FigureSBMgmt} illustrates the SB management scheme with one
level of the SB~eraseblock. 

\begin{enumerate}
\item[a).] Eraseblocks number 1 and 2 are reserved for the anchor area.
The first sector of the first anchor eraseblock refers the SB~eraseblock.
The first sector of the SB~eraseblock (number 5) contains the superblock.

\item[b).] The superblock was updated and was written to the second sector of
the SB~eraseblock. The first sector of
the SB~eraseblock becomes dirty and doesn't contain valid data any longer.
The anchor eraseblocks remain unchanged. 

\item[c).] The superblock was updated several times and was written to the
adjacent sectors of the SB~eraseblock. The last sector of the SB~eraseblock is
occupied by the superblock, the other sectors contain dirt. The anchor eraseblocks
were not changed so far.

\item[d).] The SB~eraseblock had no more free sectors. JFFS3 selected another
SB~eraseblock, taking into account the wear-levelling. At that point the Garbage
Collector may have been invoked if there were no free eraseblocks. The SB~eraseblock
reference in the anchor block was updated, and at the moment the second sector of the anchor
block is occupied by the new SB~eraseblock reference while the first sector contains
dirt. The new SB~eraseblock is now at the flash eraseblock number 79. The
old SB~eraseblock may now be erased and re-used for different purposes.

\item[e).] The superblock was updated many times and the position of the
SB~eraseblock was changed many times. The current SB~eraseblock number is 3. The
first anchor eraseblock is full and its last sector refers the current
SB~eraseblock.

\item[f).] The superblock was updated many times. JFFS3 started using the
second anchor eraseblock, and the first anchor eraseblock was erased. The
reason why JFFS3 makes use of two anchor eraseblocks is to guarantee the
unclean reboot tolerance.
\end{enumerate}

Obviously, with the SB~eraseblock approach the anchor area is erased $N$ times
less then if there would be no SB~eraseblocks ($N$ is the number of sectors
per eraseblock).

If the flash chip is larger then a certain size, the proposed scheme with one
level SB~eraseblocks does not guarantee that the anchor eraseblocks are not worn out
earlier then the other eraseblocks. But we may introduce the
second level of SB~eraseblocks. Then the anchor eraseblock will refer the
SB~eraseblock L2, which, in turn, will refer the SB~eraseblock where the
superblock is stored. If the two-level scheme is not enough, the three-level
scheme may be used and so
forth. Thus, by means of adding more SB~eraseblock levels we may guarantee the
appropriate anchor area erase rate. Note, that the SB~eraseblocks of all
the levels obey the global JFFS3 wear-levelling rules, so we don't need to
worry about the wearing of those blocks.

Note, we number the SB eraseblock levels starting from the SB eraseblocks,
referred from the anchor area, i.e., the anchor area refers the SB eraseblock
level $n$, which refers the SB eraseblock level $n-1$ and so on. The superblock
is found in the SB eraseblock -- last level of the chain.

Let's calculate the number of SB~eraseblock levels assuming that:
\begin{itemize}
\item the live-cycle of one eraseblock is $L$, i.e., there are $L$ guaranteed
erases for each eraseblock (typically $\sim 10^5$ for NAND flashes);
\item the total number of the flash eraseblocks is $M$;
\item the number of sectors per eraseblock is $N$;
\item the file system data is updated with the constant frequency $R$ sectors
per second;
\item the anchor area is updated with the constant rate $R_A$ sectors per
second;
\item the \textbf{data area} is updated with the constant rate $R_D$ sectors per second;
\item the period of time of the total anchor area wear is $T_A$;
\item the period of time of the total data area wear is $T_D$;
\item the number of used SB~eraseblock layers is $n$.
\end{itemize}

In our calculation we assume the worst case scenario when any file system data
update requires the superblock update.

Obviously, what we want is to ensure that the anchor area is worn out not
earlier then the data area, i.e. the following inequality is true:

\begin{equation}
\frac{T_A}{T_D} \geqslant 1
\label{ref_EquationSBIneq}
\end{equation}

The formulas for $T_A$ and $T_D$ are

$$
T_A = \frac{2L}{R_A}, 
T_D = \frac{(M-2) \cdot L}{R_D}
$$
and hence,
\begin{equation}
\frac{T_A}{T_D} = 2 \cdot \frac{R_D}{(M-2) \cdot R_A}
\label{ref_Equation_TA_and_TD}
\end{equation}

If $n = 0$, i.e., there is no SB~eraseblocks and the superblock is stored in
the anchor area, then taking into account (\ref{ref_Equation_TA_and_TD})
and that in this case $R_A = R_D = R$, we have 

$$
\frac{T_A}{T_D} = 2 \cdot \frac{1}{(M-2)}
$$

If $n = 1$. i.e., the level one SB~eraseblocks are used, the anchor area will
be written $N$ times less frequently  as the data area will be written $2$ times
more frequently, because each file system data update will require (a) the
superblock update and (b) the SB reference update in the SB~eraseblock.
Therefore, $R_A = R/N$ and $R_D = 2R$ and from (\ref{ref_Equation_TA_and_TD})
we have

$$
\frac{T_A}{T_D} = 2 \cdot \frac{2N}{M-2}
$$

When $n = 2$, i.e., two levels of SB~eraseblocks are used, the anchor area will
be written $N^2$ times less frequently, while the data area will be written
$2+1/N$ times more frequently (one additional superblock update on each file
system update and one L2 SB~eraseblock is updated once per N L1 SB~eraseblock
updates). Therefore, $R_A=R/N^2$ and $R_D = (2 + 1/N) \cdot R$ and from
\ref{ref_Equation_TA_and_TD}

$$
\frac{T_A}{T_D} = 2 \cdot \frac{2N^2+N}{M-2}
$$

For $n = 3$, analogously,

$$
\frac{T_A}{T_D} = 2 \cdot \frac{2N^3 + N^2 + N}{M-2}
$$
and for $n = 0,1,2,\ldots$

$$
\frac{T_A}{T_D} = 2 \cdot \frac{2N^n + N^{n-1} + \ldots + N}{M-2}
\label{ref_Equation_TA_div_TD}
$$

Consequently, from (\ref{ref_EquationSBIneq}) we have the following inequality
to be satisfied

$$
2 \cdot \frac{2N^n + N^{n-1} + \ldots + N}{M-2} \geqslant 1
$$
or, neglecting the minor components,

\begin{equation}
\frac{4N^n}{M} \geqslant 1
\label{ref_EquationSBIneq1}
\end{equation}

Table \ref{ref_TableNANDLevels} describes the number of required SB eraseblock
levels for different NAND flash sizes.

\begin{center}
\begin{table}[h]
\begin{tabular}{llllll}
Flash type & Flash size & Sector size & $M$ & $N$ & Levels needed ($n$)\\
\hline
Toshiba TC58DVM92A1FT   & 64MB  & 16KB  & 4096  & 32 & 2\\
Toshiba TH58NVG1S3AFT05 & 512MB & 128KB & 4096  & 64 & 2\\
ST Micro NAND08G-B      & 1GB   & 128KB & 8192  & 64 & 2\\
Samsung K9K1G08X0B      & 2GB   & 128KB & 16384 & 64 & 2\\
\end{tabular}
\caption{The number of required SB eraseblock levels for different types of
existing NAND flashes.} \label{ref_TableNANDLevels}
\end{table}
\end{center}

Note, providing that $N=64$, three SB~eraseblock levels are enough to guarantee
the acceptable anchor area wear leveling up to 128GB flash and four
SB~eraseblock levels -- up to 8TB (the inequality \ref{ref_EquationSBIneq1}).

To find the superblock during the mount JFFS3 finds the valid reference in the
anchor eraseblocks, then finds the valid reference in the SB eraseblock level $n$,
$n-1$, etc, and finally finds the valid superblock sector in the SB eraseblock.
Since JFFS3 assigns versions to the sectors of the anchor eraseblocks and SB
eraseblocks and the version is increased by one every time the sector is
updated, the binary search algorithm may be used to find the valid sector.

The valid SB eraseblock level $n$ reference in the anchor area may be found after
$log_2(2N)+2$ steps, the reference to the level $n-1$ SB eraseblock -- after
$log_2(N)+2$ steps, and so on. Thus, to find the superblock, JFFS3 must read

\begin{equation}
S = 2(n+1) + log_2(2N) + n \cdot log_2(N)
\end{equation}
flash sectors.

Table \ref{ref_TableNANDTimes} contains the approximate SB search time for
different existing NAND flashes.

\begin{center}
\begin{table}[h]
\begin{tabular}{lllllll}
Flash type & Flash size & $N$ & $n$ & Sector read time & $S$ & SB find time\\
\hline
Toshiba TC58DVM92A1FT & 64MB & 32 & 2 & $\sim$50$\mu$s  & 22 & $\sim$1.1ms\\
ST Micro NAND08G-B    & 1GB  & 64 & 2 & $\sim$130$\mu$s & 25 & $\sim$3.3ms\\
Samsung K9K1G08X0B    & 2GB  & 64 & 2 & $\sim$70$\mu$s  & 25 & $\sim$1.6ms\\
\end{tabular}
\caption{The superblock search time for different NAND flashes} \label{ref_TableNANDTimes}
\end{table}
\end{center}

For larger flash chips which would utilize the level 3 SB management scheme (no
such flashes exist at the moment) the SB search time would be 4.3ms providing
the flash characteristics are the same as the ST Micro's one (see table
\ref{ref_TableNANDTimes}).

Note, the above time doesn't contain the ECC/CRC checking overhead as well as any
other CPU overhead.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% DEFINITIONS
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Definitions}\label{ref_SectDefinitions}
\begin{enumerate}
\item \textbf{Anchor erasable block, anchor area}
-- the first two \emph{good}
erasable blocks of the JFFS3 partition which are reserved for the superblock management.

\item \textbf{Erasable block, eraseblock} -- the minimal erasable unit of the
flash chip.
Although flash manuals usually call erasable blocks just "blocks" or
"sectors", we prefer to be consistent with traditional Linux file system and
block IO layer terminology and reserve the "block" and "sector" terms for
other purposes.

\item \textbf{Block} -- several adjancent sectors form a 4KiB block.
Block size is equvalent to
the size of the RAM page in most architectures.

\item \textbf{Data area} -- all the JFFS3 partition excluding the anchor area.

\item \textbf{Sector} -- the smallest writable unit of the \emph{flash chip},
i.e. the NAND page in case of NAND. Multiple sectors compose a block. The
minimal size of the sector is assumed to be 512KB.

\item \textbf{SB, superblock} -- the special data structure which describes the whole
JFFS3 filesystem. The superblock may reside anywhere within the JFFS3 partition
(except of the anchor eraseblocks). When JFFS3 is being mounted, the
superblock must be found. JFFS3 superblock takes one sector.

\item \textbf{SB eraseblock} -- The SB eraseblock is the flash eraseblock that
contains the JFFS3 superblock. The JFFS3 superblock management scheme implies
several levels of SB eraseblocks - typically 2-4 levels. For example, in the
3-level scheme, the anchor eraseblock refers the SB eraseblock level 3, which
refers the SB eraseblock level 2, which refers the SB eraseblock containing the
superblock.

\item \textbf{Node} -- the tree pile, i.e., the tree consists of nodes. The
root of the tree is the \textbf{root node}, the leafs of the tree are
\textbf{leaf nodes}. Non-leaf nodes are called \textbf{branch nodes}.
This terminology is very similar to the terminology used in
Reiser4 [\ref{ref_Reiser4}]. 
\end{enumerate}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% ABBREVIATIONS
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Abbreviations}
\begin{enumerate}
\item \textbf{ECC} - Error Correction Code
\item \textbf{CRC} - Cyclic Redundancy Check
\item \textbf{JFFS2} - Journalling Flash File System version 2
\item \textbf{JFFS3} - Journalling Flash File System version 3
\item \textbf{MTD} - Memory Technology Devices
\item \textbf{RAM} - Random Access Memory
\item \textbf{SB} - Super Block
\item \textbf{VFS} - Virtual File System
\end{enumerate}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% CREDITS
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Credits}
The following are the people I am grateful for help (alphabetical order):

\begin{itemize}
\item \textbf{David Woodhouse} \texttt{<dwmw2@infradead.org>} -- the author of JFFS2,
answered a great deal of my questions about MTD, JFFS2, JFFS3 design approaches,
English (e.g., "how to express this correctly in English"), etc.

\item \textbf{Joern Engel} \texttt{<joern@wohnheim.fh-wedel.de>} -- discussed a
lot of JFFS3 design aspects with me. Many ideas described in this document were
suggested by Joern.

\item \textbf{Thomas Gleixner} \texttt{<tglx@linutronix.de>} -- 
proposed many interesting ideas which I'm exploiting in the JFFS3 design as
well as answered many question, especially concerning flash hardware and
low-level flash software.

\item \textbf{Victor V. Vengerov} \texttt{<vvv@oktetlabs.ru>} -- My colleague
from OKTET~Labs who spent much time discussing the JFFS3 design approaches with
me and suggested many interesting ideas. He also reviewed my writings.
\end{itemize}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% REFERENCES
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{References}
\begin{enumerate}
\item \raggedright \label{ref_JFFSdwmw2}
JFFS : The Journalling Flash File System,\\
\url{http://sources.redhat.com/jffs2/jffs2-html/}

\item \raggedright \label{ref_SamsungNANDlist} 
Samsung Flash memory products,\\
\url{http://www.samsung.com/Products/Semiconductor/Flash/index.htm}

\item \raggedright \label{ref_Reiser4},
Reiser4 File System, \url{http://www.namesys.com/}

\end{enumerate}

\end{document}

