Aric's Notes: file system

ubi and ubifs应用手记

1.配置ubi and ubifs
in .config
  CONFIG_MTD_UBI=y
    CONFIG_UBIFS_FS=y
    CONFIG_CRYPTO_ALGAPI=y CONFIG_CRYPTO_DEFLATE=y CONFIG_CRYPTO_LZO=y CONFIG_CRC16=y CONFIG_LZO_COMPRESS=y CONFIG_LZO_DECOMPRESS=y
注意：如果配置成模块(=m)，则可以手动加载

2.manual attach/detach ubi to mtd
./ubiattach /dev/ubi_ctrl -m mtdnumber
./ubidetaach /dev/ubi_ctrl -m mtdnumber

3.manual create ubi volume
./ubimkvol /dev/ubi_device_number -s size -N name
like:
./ubimkvol /dev/ubi0 -s 300MiB -N ubifs1
4.mount ubifs volume
mount -t ubifs ubi0:ubifs1 /tmp/ubifs1

5.ubi node and ubi_ctrl node
#cat /sys/class/misc/ubi_ctrl/dev
10:63
加入/dev下没有ubi_ctrl,则我们可以sudo mknod ubi_ctrl c 10 63创建一个端点
#./ubiattach /dev/ubi_ctrl -m 6
#cat /sys/class/ubi/ubi0/dev
252:0
当我们attach ubi0 to mtd6后，如果/dev下没有ubi0,则创建一个，sudo mknod ubi0 c 252 0

6.我们可以手动create volume,然后手动mount ubifs,也可以在PC上创建ubi.img（创建好volume,volume写有数据)烧录进mtd device
How to generate ubi image and write to mtd device
./mkfs.ubifs -r a205_rootdisk -m 4096 -e 516096 -c 40 -o ubifs.img
./ubinize -o ubi.img -m 4096 -p 512KiB ubinize.cfg
./ubiformat -q /dev/mtd5 -f ubi.img

-m minimum I/O unit size
-e maximum logical erase block count
-c maximum logical erase block count
-x compression type - "lzo", "favor_lzo", "zlib" or "none" (default: "lzo")
-p size of the physical eraseblock of the flash this UBI image is created for in bytes,

注意：在PC上ubuntu使用mkfs.ubifs and ubinize,则我们要用普通的gcc来编译它们，同时在ubuntu上装上lzo库：sudo apt-get install liblzo2-dev
附录：
1).ubinize.cfg
[ubifs]
mode=ubi
image=ubifs.img
vol_id=0
vol_size=500MiB
vol_type=dynamic                     //if vol_type=static, then ubi volume is read only
vol_name=ubifs0
vol_flags=autoresize
这样这样当./ubiattach /dev/ubi_ctrl -m n后，就可以mount -t ubifs ubi0:ubifs0 /tmp

2)如果是想mount crafms image,只要
./ubinize -o ubi.img -m 4096 -p 512KiB ubinize.cfg
./ubiformat -q /dev/mtd5 -f ubi.img
ubinize.cfg
[ubifs]
mode=ubi
image=cramfs.img
vol_id=0
vol_size=500MiB
vol_type=dynamic
vol_name=cramfs
vol_flags=autoresize
这样当./ubiattach /dev/ubi_ctrl -m n后就可以从cat /proc/mtd中看到一个ubi volume仿真的mtd device,我们只要mount这个mtd设备对应的mtdblock就可以了（如mount -t cramfs /dev/mtdblock10 /tmp),注意，既然是烧录了cramfs到ubi volume，则我们只能以cramfs方式mount这个volume,不能再以ubifs方式（mount -t ubifs ubi0:cramfs /tmp）mount这个volume.但如果我们用./ubiupdate /dev/ubi0_0 -t wipe out擦干净这个volume后，我们是可以用ubifs方式mount这个volume,但mount起来这个volume，进入mount的目录，是什么内容也没有的。
3)三个volume的ubinize.cfg（注意[]中名字不能一样，vol_id不能一样，vol_name不能一样，另vol_flags=auto_resize只能使用在一个volume上）
[ubifs1]
mode=ubi
image=ubifs.img
vol_id=0
vol_size=20MiB
vol_type=dynamic
vol_name=ubifs0
vol_alignment=1

[cramfs1]
mode=ubi
image=smallroot.cramfs
vol_id=1
vol_size=20MiB
vol_type=dynamic
vol_name=cramfs
vol_alignment=1
[cramfs2]
mode=ubi
image=qtroot.cramfs
vol_id=2
vol_size=50MiB
vol_type=dynamic
vol_name=cramfs2
vol_alignment=1
vol_flags=autoresize
这样当使用./ubiformat写入ubi.img后，则./ubiattach后，我们可以知道多了三个假的mtd device.
第一个可以用mount -t ubifs ubi0:ubifs0 /tmp/ubifs1
第二个可以用mount -t cramfs /dev/mtdblockn /tmp/cramfs1
第二个可以用mount -t cramfs /dev/mtdblockm /tmp/cramfs2

7. How to disable compression?
UBIFS compression may be disabled for whole file system during the image creation time using the "-x none" mkfs.ubifs option. However, if UBIFS compression is enabled, it may be disabled for individual files by cleaning the inode compression flag:
$ chattr -c /mnt/ubifs/file
in shell, or

ioctl(fd, FS_IOC_GETFLAGS, &flags);

flags &= ~FS_COMPR_FL;

ioctl(fd, FS_IOC_SETFLAGS, &flags);
in C programs. Similarly, if compression is disabled by default, you may enable if for individual inodes by setting the compression flag. Note, the code which uses the compression flag works fine on other Linux file-systems, because the flag is just ignored in this case.
It might be a good idea to disable compression for say, mp3 or jpeg files which would anyway not compress and UBIFS would just waste CPU time trying to compress them. The compression may also be disabled if one wants faster file I/O, because UBIFS would not need to compress or decompress the data on reads and write. However, I/O speed may actually become slower if compression is disabled. Indeed, in case of a very fast CPU and very slow flash compressed writes are faster, but this is usually not true for embedded systems.

8.mount cramfs on ubi volume
ubi volume is fake mtd device.
# cat /proc/mtd
dev:    size   erasesize name
mtd0: 180000 00080000 "Bootloader"
mtd1: 400000 00080000 "Kernel 0"
mtd2: 400000 00080000 "Kernel 1"
mtd3: 80000 00080000 "Boot up screen"
mtd4: a00000 00080000 "Rescue file system"
mtd5: 1400000 00080000 "Root file system"
mtd6: 3e800000 00080000 "Data area1"
mtd7: 3e800000 00080000 "Data area2"
mtd8: 6a400000 00080000 "Data area3"
mtd9: 16380000 00080000 "reserve"

#./ubiattach /dev/ubi_ctrl -m 6
#./ubimkvol /dev/ubi0 -s 300MiB -N ubifs1
# cat /proc/mtd
dev:    size   erasesize name
mtd0: 180000 00080000 "Bootloader"
mtd1: 400000 00080000 "Kernel 0"
mtd2: 400000 00080000 "Kernel 1"
mtd3: 80000 00080000 "Boot up screen"
mtd4: a00000 00080000 "Rescue file system"
mtd5: 1400000 00080000 "Root file system"
mtd6: 3e800000 00080000 "Data area1"
mtd7: 3e800000 00080000 "Data area2"
mtd8: 6a400000 00080000 "Data area3"
mtd9: 16380000 00080000 "reserve"
mtd10: 12c3c000 0007e000 "ubifs1"
# cp cramfs.img /dev/mtdblock10
# mount -t cramfs /dev/mtdblock6 /tmp

After create fake mtd device(ubi volume), mount jffs2
#mount -t jffs2 /dev/mtdblock10 /mnt
9.ubiupdatevol /dev/ubi0_0 -t   //wipe out volume
ubiupdatevol /dev/ubi0_0 fs.img //write image to volume

./ubiupdatevol /dev/ubi0_0 ubifs.img          //之后we can mount ubifs: mount -t ubifs ubi0:ubifs0 /tmp来挂载这个ubifs
./ubiupdatevol /dev/ubi0_1 smallroot.cramfs //之后我们就可以mount -t cramfs /dev/mtdblockn /tmp来挂载这个cramfs

10.挂载vfat
1)制作vfat.img（在PC上制作）
$ dd if=/dev/zero of=vfat.img bs=1M count=20
#losetup /dev/loop0 vfat.img
#mkfs.vfat /dev/loop0
注意：这有一个warnning,但不用理会：Loop device does not match a floppy size, using default hd params
#mount -t vfat /dev/loop0 vfat_mount_point
往vfat_mount_point目录写东西，或copy东西到这目录
#umount vfat_mount_point
#losetup -d /dev/loop0
2)用ubinize打包成ubi.img，然后用ubiformat写入mtd devie。方法二是用ubiupdatevol先wipe out volume，然后用ubiupdatevol将vfat.img
写入volume.
但注意：因为emulate mtd device是不支持写操作的，所以我mount -t /dev/mtdblockn，这个mtdblockn是一个ubi volume emuluate的mtd device，
所以mount的vfat只可以读，写是无法保存的。（测试中写是能完成，ls也能看到，但sync后重启unit,重新mount可以看到写的数据是没有保存如vfat的）

11.ubifs(read/write/attach/mount)speed,
Wrtie speed -------------------speed=1.66M/s
# time dd if=/dev/zero of=/tmp/ubifs1/zero100M bs=1M count=100;time sync
100+0 records in
100+0 records out
real    0m 59.13s
user    0m 0.00s
sys     0m 4.62s
real    0m 1.11s
user    0m 0.00s
sys     0m 0.74s
Read speed-----------------------speed=2.27M/s
# time cp ubifs1/zero100M /dev/null;time sync
s3c-nand: 1 bit(s) error detected, corrected successfully
s3c-nand: 1 bit(s) error detected, corrected successfully
s3c-nand: 1 bit(s) error detected, corrected successfully
real    0m 44.06s
user    0m 0.14s
sys     0m 42.67s
real    0m 0.06s
user    0m 0.00s
sys     0m 0.01s

12.配置ubifs as rootfs
in .config:
CONFIG_CMDLINE="console=ttySAC0 ubi.mtd=5 root=ubi0:rootfs rootfstype=ubifs"

then if we had wrote root fs ubi image to mtd5, then we can boot up with ubi root fs.

13.遇到的rw filesystem change to read only filesystem
# ./ubiattach /dev/ubi_ctrl -m 6
UBI: attaching mtd6 to ubi0
UBI: physical eraseblock size:   524288 bytes (512 KiB)
UBI: logical eraseblock size:    516096 bytes
UBI: smallest flash I/O unit:    4096
UBI: VID header offset:          4096 (aligned 4096)
UBI: data offset:                8192
PEB 0 is bad
PEB 32 is bad
UBI: attached mtd6 to ubi0
UBI: MTD device name:            "Data area1"
UBI: MTD device size:            1000 MiB
UBI: number of good PEBs:        1998
UBI: number of bad PEBs:         2
UBI: max. allowed volumes:       128
UBI: wear-leveling threshold:    4096
UBI: number of internal volumes: 1
UBI: number of user volumes:     3
UBI: available PEBs:             0
UBI: total number of reserved PEBs: 1998
UBI: number of PEBs reserved for bad PEB handling: 19
UBI: max/mean erase counter: 7/0
UBI: background thread "ubi_bgt0d" started, PID 940
./ubiupdatevol /dev/ubi0_0 -t
UBI error: ubi_io_write: error -5 while writing 4096 bytes to PEB 21:4096, written 0 bytes
UBI warning: ubi_eba_write_leb: failed to write VID header to LEB 2147479551:0, PEB 21
UBI: try another PEB
UBI error: ubi_io_write: error -5 while writing 4096 bytes to PEB 12:4096, written 0 bytes
UBI warning: ubi_eba_write_leb: failed to write VID header to LEB 2147479551:0, PEB 12
UBI: try another PEB
UBI error: ubi_io_write: error -5 while writing 4096 bytes to PEB 1999:4096, written 0 bytes
UBI warning: ubi_eba_write_leb: failed to write VID header to LEB 2147479551:0, PEB 1999
UBI: try another PEB
UBI error: ubi_io_write: error -5 while writing 24576 bytes to PEB 1998:8192, written 0 bytes
UBI warning: ubi_eba_write_leb: failed to write 24576 bytes at offset 0 of LEB 2147479551:0, PEB 1998
UBI warning: ubi_ro_mode: switch to read-only mode
UBI error: ubi_io_write: read-only mode
UBI error: erase_worker: failed to erase PEB 7, error -30
UBI error: do_work: work failed with error code -30
UBI error: ubi_thread: ubi_bgt0d: work failed with error code -30
ubiupdatevol: error!: cannot truncate volume "/dev/ubi0_0"
              error 30 (Read-only file system)
UBI warning: vol_cdev_release: update of volume 0 not finished, volume is damaged
遇到此问题后，再flash_eraseall /dev/mt6或/dev/mtd7后，再写image to /dev/mtd7 or /dev/mtd6都出错
./ubiformat -q /dev/mtd6 -f ubi.img.20M_none
libmtd: error!: cannot write 32768 bytes to mtd6 (eraseblock 1, offset 0)
        error 5 (Input/output error)
ubiformat: error!: cannot write eraseblock 1
           error 5 (Input/output error)

遇到此问题后，我在uboot下执行
NAND erase: device 0 offset 0x600000, size 0x400000
ret:0 erase.addr:600000
Erasing at 0x600000 -- 12% complete.ret:0 erase.addr:680000
Erasing at 0x680000 -- 25% complete.ret:0 erase.addr:700000
Erasing at 0x700000 -- 37% complete.ret:0 erase.addr:780000
Erasing at 0x780000 -- 50% complete.ret:0 erase.addr:800000
Erasing at 0x800000 -- 62% complete.ret:0 erase.addr:880000
Erasing at 0x880000 -- 75% complete.ret:0 erase.addr:900000
Erasing at 0x900000 -- 87% complete.ret:0 erase.addr:980000
Erasing at 0x980000 -- 100% complete.
OK
SMDK2450 # nand write c0000000 600000 300000
NAND write: device 0 offset 0x600000, size 0x300000
0 bytes written: ERROR

最后，我重新用笔尖fine tine nand flash的引脚(特别是/WE),终于救回了这片flash

结论：好像是此flash已经损坏，或者是引脚接触不好----因为erase是ok的，而且nand read读也是ok的,导致无法写入PEB，会使得
UBIFS变为只读。

14.mkfs.ubifs -c issus (注意 fat directory is empty)
dannylo@fs1:~/cram2fs_tools$ ./mkfs.ubifs -r fat -m 4096 -e 516096 -c 10 -o test.img
Error: too low max. count of LEBs, minimum is 17
dannylo@fs1:~/cram2fs_tools$ ./mkfs.ubifs -r fat -m 4096 -e 516096 -c 19 -o test.img
Error: too many log LEBs, maximum is 2
dannylo@fs1:~/cram2fs_tools$ ./mkfs.ubifs -r fat -m 4096 -e 516096 -c 21 -o test.img
Error: too many log LEBs, maximum is 4
dannylo@fs1:~/cram2fs_tools$ ./mkfs.ubifs -r fat -m 4096 -e 516096 -c 22 -o test.img
结论：即最小的ubifs.img为11M.注意对要烧录进volume的cramfs or cram2fs format的image的size好像没有要求。

15.ubi face bad block
我们在uboot中用nand markbad 试过mark bad block(一块在mtd device的首块，一块在中间，一块在最后），测试表面ubi在attach时的scan能认出
bad block nand skip bad block.同时用ubiformat烧写ubi.img时也会自动跳过bad block.

Original link

Initramfs is the new way of doing stuff before the root partition is mounted. This is useful if you need to do something special to get your root partition visible to the kernel. For example, if it is an lvm partition, encrypted or on USB.
You can also stay in the initramfs. This can be very useful for debugging, or if you just want a very fast booting system.

Creating the folder structure

Before we get to the fun bit, you'll need to create a few folders.
Alternatively you can use this tarball, which also contains the /bin/busybox binary and the /init script.
The empty mdev.conf is created so that newer versions of busybox would not complain.

$ mkdir -p work/initramfs/{bin,sbin,etc,proc,sys,newroot}
$ cd work
$ touch initramfs/etc/mdev.conf

Getting BusyBox

BusyBox combines most common UNIX utilities into a single binary. If it's compiled statically, it is usually the only binary file you will need in the initramfs image. Instead you could ldd the dynamicly linked BusyBox binary, and copy all the necessary libraries to /lib in the initramfs, but it is a lot easier to go with a static binary.
BusyBox can be compiled using glibc or uClibc. Both work just the same, but uClibc produces quite a lot smaller binaries.

$ wget http://jootamam.net/initramfs-files/busybox-1.10.1-static.bz2 -O - | bunzip2 > initramfs/bin/busybox
$ chmod +x initramfs/bin/busybox
$ ln -s busybox initramfs/bin/sh

To use your own BusyBox, but make sure it's static. You can use file /bin/{bb,busybox} | grep static to check if your current system has a static version of busybox

/init

If an initramfs is built into the kernel or passed to it, the kernel will try to execute /init in the initramfs. Usually /init is a shell script that will find and mount the root partition, then switch_root onto the root partiton and execute /sbin/init.

$ touch initramfs/init
$ chmod +x initramfs/init

Then edit initramfs/init

#!/bin/sh

#Mount things needed by this script
mount -t proc proc /proc
mount -t sysfs sysfs /sys

#Disable kernel messages from popping onto the screen
echo 0 > /proc/sys/kernel/printk

#Clear the screen
clear

#Create all the symlinks to /bin/busybox
busybox --install -s

#Create device nodes
mknod /dev/null c 1 3
mknod /dev/tty c 5 0
mdev -s

#Function for parsing command line options with "=" in them
# get_opt("init=/sbin/init") will return "/sbin/init"
get_opt() {
echo "$@" | cut -d "=" -f 2
}

#Defaults
init="/sbin/init"
root="/dev/hda1"

#Process command line options
for i in $(cat /proc/cmdline); do
case $i in
root\=*)
root=$(get_opt $i)
;;
init\=*)
init=$(get_opt $i)
;;
esac
done

#Mount the root device
mount "${root}" /newroot

#Check if $init exists and is executable
if [[ -x "/newroot/${init}" ]] ; then
#Unmount all other mounts so that the ram used by
#the initramfs can be cleared after switch_root
umount /sys /proc

#Switch to the new root and execute init
exec switch_root /newroot "${init}"
fi

#This will only be run if the exec above failed
echo "Failed to switch_root, dropping to a shell"
exec sh

This /init script doesnt actually do anything that you couldn't do without an initramfs, but it can very easily be extended.

Creating the .cpio and .igz

When passing an initramfs to the kernel using your bootloader (the initrd option in Grub), the kernel is expecting a gzipped cpio archive (.igz). When building the initramfs image into the kernel, one of the easiest ways is to create a cpio archive (.cpio).
We will be creating both files.

$ cd initramfs
$ find . | cpio -H newc -o > ../initramfs.cpio
$ cd ..
$ cat initramfs.cpio | gzip > initramfs.igz

You might want to put these commands in a shell script file so you can easily re-create the initramfs images when you make changes to the content of work/initramfs/

Kernel configuration

The only thing you will need in the kernel is CONFIG_BLK_DEV_INITRD=y
If you want to build your initramfs image into the kernel. This way you won't need to pass an initrd option to your bootloader, but when you update the initramfs image, you will also need to rebuild the kernel. If you do want to do this, give the path to the initramfs.cpio file in Initramfs source file(s).

General Setup --->
[*] Initial RAM filesystem and RAM disk (initramfs/initrd) support
()    Initramfs source file(s)

Grub configuration

To pass the initramfs.igz to the kernel using Grub you will need the initrd option in menu.lst (grub.conf for Gentoo users).

title Gentoo
kernel /vmlinuz
initrd /initramfs.igz

As we have set default root partition and init in our work/initramfs/init, we do not need to specify them on the kernel line. You still can specify those options if you want to or need to by appending root=/dev/hda1 init=/sbin/init.

Aric's Notes

2009-12-09

ubi and ubifs应用手记(quote)

2009-12-04

Loop mount JFFS2 image(quote)

Loop mount JFFS2 image

How to create a initramfs(quote)