z4mod – updated

So I had to change quite a bit, and instead of one script I am now handling 3:

  • z4build – extracts initramfs from a given zImage, patches the init scripts to allow ext3 filesystem to be added to the rfs mount command (ie, each rfs mount command will be prefixed with a prior mount using ext3 to check first if this partition can be mounted, otherwise, it passes along to the rfs mount) and regenerates a patched zImage.
  • updater-script – converts the partitions from rfs to ext3, and flashes zImage when done
  • z4mod – the actual converting script used by updater-script


  • Im presenting the scripts as-is, so you can add your own remarks as we continue.

    I am aware of the fact that ext4 is better suited, but for the testings ext3 should suffice, since it simplifies my work.

    The change to the initramfs init scripts is minor. consider the following mount command:

        mount rfs /dev/block/stl11 /cache nosuid nodev check=no

    it will be replaced with:

        mount ext3 /dev/block/stl11 /cache noatime nodiratime || mount rfs /dev/block/stl11 /cache nosuid nodev check=no

    Thats it. no bogus hacks, scripts, patches, etc.
    Ideally, z4mod could be applied to many roms, as long as it supports the targeted filesystem.

    The rest of the partitions will be added to the code soon – this is only a proof-of-concept.

    [UPDATE:]
    I’ve updated the scripts in this post, and I will continue doing so until I decide how to proceed with the project (github, etc).
    The z4build now supports all RFS partitions, soon the updater-script will too.

    [UPDATE 2:]
    I’ve updated the scripts. It doesn’t mean you can test them, it means we are one step closer.
    The /cache (which i use for testings) mounts perfectly fine automatically with ext2.
    The /data is being formatted, converted to ext2, the device reboots, and it shows as RFS again. Thats the last bug. Once we squash it, I guess we can call this Alpha πŸ™‚

    [UPDATE 3:]
    This is the last update before official Alpha release. The code has been redesigned and recoded, keeping KISS in mind. This project is without a doubt a community project – I simply put to commands what great XDA developers posted and explained in words on the forums. So kudos to us all πŸ™‚

    THIS HAS NOT BEEN TESTED YET! 100% IT WILL CRASH!



    I am posting this only for reference, so you can see where z4mod is headed.



    WARNING: DO NOT USE THIS YET.


    z4build:

    #!/bin/bash
    ###############################################################################
    #
    # z4build by Elia Yehuda, aka z4ziggy, (c) 2010
    # part of the z4mod project - a ROM mod without RFS.
    #
    # extracts initramfs from a given zImage, patch it to allow ext2/3/4 mounts,
    # and repack it.
    #
    # Released under the GPLv2
    #
    # many thanks goto various coders & Android hackers out there who made this
    # possible: supercurio, Unhelpful, dkcldark, RyanZA, XDA & modaco forums.
    #
    # WARNING:
    # FOR YOUR OWN SAFETY, IF YOU CAN'T FOLLOW THE SCRIPT, AVOID USING IT.
    # USE AT YOUR OWN RISK! NO WARRANTIES WHAT SO EVER!
    #
    # zImage extraction script copied from here:
    # http://forum.xda-developers.com/wiki/index.php?title=Extract_initramfs_from_zImage
    #
    # kernel_repacker taken from here:
    # http://forum.xda-developers.com/showthread.php?t=789712
    #
    # TODO:
    # - As RyanZA suggested, maybe patch and change each 'mount' to 'z4mount'
    #   and use a dedicated shell script. It would give us better control over the
    #   whole init/mount process. This will also fix the two-mounts issue, but it
    #   also adds dependency on busybox binary to be added to the initramfs.
    # - Add a simpler kernel_repacker
    # - Fix the kernel unpacker. It's useless for custom kernels.
    # - Release final version (1.0) only after all TODOs are completed
    #
    # Changelog:
    # - 0.1 initial release
    #
    # - 0.2 recoded to fit update.zip method
    #
    # - 0.3 added support for all RFS parititons
    #
    # - 0.4 minor cosmetic changes and cleanup
    #       fixed the grep path
    #       fixed check if zImage exists
    #
    # - 0.5 fixed the sed command (replaced '||' with '\n')
    #
    # - 0.6 fixed the sed again
    #       changed the mount parameters
    #
    # - 0.7 added patcher to the binary 'init' (thanks to Unhelpful)
    #
    # - 0.8 'auto' option doesn't work for mount, reverted to $FORMAT, so either
    #       use a mount wrapper (z4mount) to get 'auto' or forget it.
    #       fixed another sed bug ([^#])
    #
    #
    ###############################################################################
    
    ###############################################################################
    #
    # SETTINGS: You must provide valid path to kernel_repacker below
    #
    ###############################################################################
    # set -x
    
    # where to find kernel_repack/editor.sh
    KERNEL_REPACKER=/home/ziggy/code/android/kernel_repacker/editor.sh
    # which format would you like to change your filesystem to:
    # make sure it is supported by the kernel!
    FORMAT=ext2
    
    # No need to change anything below this
    
    ###############################################################################
    #
    # general functions
    #
    ###############################################################################
    
    C_H1="\033[1;37m"        # highlight text 1
    C_ERR="\033[1;31m"
    C_CLEAR="\033[1;0m"
    
    # helper functions:
    
    printhl() {
    	printf "${C_H1}${1}${C_CLEAR} \n"
    }
    
    printerr() {
    	printf "${C_ERR}${1}${C_CLEAR} \n"
    }
    
    exit_error() {
    	printerr "$1"
    	rm -rf /tmp/z4mod
    	exit 1
    }
    ###############################################################################
    #
    # checking parameters and initalize stuff
    #
    ###############################################################################
    
    # Making sure we have everything
    if [ -z "$1" ] || [ ! -f $1 ]; then
    	exit_error "[E] Can't find kernel: $1"
    fi
    if [ ! -f "${KERNEL_REPACKER}" ]; then
    	exit_error "[E] Can't find kernel_repacker"
    fi
    
    printhl "\n[I] z4build begins, going to try and patch $1"
    
    # We can start working
    rm -rf /tmp/z4mod
    mkdir -p /tmp/z4mod/initramfs
    
    ###############################################################################
    #
    # extract the initramfs.img from zImage
    #
    ###############################################################################
    
    zImage=$1
    #=======================================================
    # find start of gziped kernel object in the zImage file:
    #=======================================================
    pos=`grep -P -a -b --only-matching $'\x1F\x8B\x08' $zImage | cut -f 1 -d :`
    printhl "[I] Extracting kernel image from $zImage (start = $pos)"
    dd if=$zImage bs=1 skip=$pos | gunzip > /tmp/kernel.img
    #===========================================================================
    # find start and end of the "cpio" initramfs image inside the kernel object:
    # ASCII cpio header starts with '070701'
    # The end of the cpio archive is marked with an empty file named TRAILER!!!
    #===========================================================================
    start=`grep -a -b --only-matching '070701' /tmp/kernel.img | head -1 | cut -f 1 -d :`
    end=`grep -a -b --only-matching 'TRAILER!!!' /tmp/kernel.img | head -1 | cut -f 1 -d :`
    end=$((end + 10))
    count=$((end - start))
    if (($count < 0)); then
    	exit_error "[E] Couldn't match start/end of the initramfs image."
    fi
    printhl "[I] Extracting initramfs image from /tmp/kernel.img (start = $start, end = $end)"
    dd if=/tmp/kernel.img bs=1 skip=$start count=$count > /tmp/z4mod/initramfs.img
    
    ###############################################################################
    #
    # extract initramfs.img, patch the binary init, patch the scripts and add 
    # additional mount, repack initramfs, and repack a new zImage
    #
    ###############################################################################
    
    printhl "[I] Extracting initramfs compressed image"
    (cd /tmp/z4mod/initramfs/; cpio -i --no-absolute-filenames < /tmp/z4mod/initramfs.img)
    
    # patch the init binary
    PATCHED="false"
    for INIT_FILE in "init" "sbin/init" "bin/init"; do
    	ELF_SIGNATURE=`file -b /tmp/z4mod/initramfs/${INIT_FILE}`
    	if [ "${ELF_SIGNATURE:0:3}" == "ELF" ]; then
    		printhl "[I] Patching ${INIT_FILE} binary"
    		sed -i 's/mmcblk0\x00/\x00mcblk0\x00/g' /tmp/z4mod/initramfs/${INIT_FILE}
    		sed -i 's/mmcblk0p2\x00/\x00mcblk0p2\x00/g' /tmp/z4mod/initramfs/${INIT_FILE}
    		PATCHED="true"
    		break
    	fi
    done
    if [ "${PATCHED}" == "false" ]; then
    	exit_error "[E] Couldn't find a valid init executable"
    fi
    
    # loop on all known rfs partitions and add our $FORMAT mount prior to the RFS mount
    #for PART in "stl9 /system" "mmcblk0p2 /data" "stl10 /dbdata" "stl11 /cache" "stl3 /efs"; do
    for PART in "stl11 /cache" "mmcblk0p2 /data"; do
    	# Making sure we are not re-patching the initramfs scripts
    	if [ "`grep "mount ${FORMAT} /dev/block/${PART}" /tmp/z4mod/initramfs/*.rc`" ]; then
    		printhl "[I] Skipping ${FORMAT} support for ${PART} - ${FORMAT} is already supported"
    	else
    		printhl "[I] Adding ${FORMAT} support for ${PART}"
    		sed -i "s|[^#]mount rfs /dev/block/${PART}*|# Added by z4mod\nmount ${FORMAT} /dev/block/${PART} nosuid nodev noatime nodiratime errors=continue\nmount rfs /dev/block/${PART}|g" /tmp/z4mod/initramfs/*.rc
    	fi
    done
    
    #echo paused
    #read key
    
    printhl "[I] Saving patched initramfs.img"
    (cd /tmp/z4mod/initramfs/; find . | cpio -H newc -o > /tmp/z4mod/initramfs.img)
    printhl "[I] Repacking zImage"
    OLD_PATH=`pwd`
    pushd `dirname ${KERNEL_REPACKER}`
    rm -f new_zImage
    ${KERNEL_REPACKER} ${OLD_PATH}/zImage /tmp/z4mod/initramfs.img
    popd
    if [ ! -f `dirname ${KERNEL_REPACKER}`/new_zImage ]; then
    	exit_error "[E] Failed building new zImage"
    fi
    printhl "[I] Moving new kernel to current dir"
    mv `dirname ${KERNEL_REPACKER}`/new_zImage zImage
    
    rm -rf /tmp/z4mod
    printhl "[I] Done."
    

    updater-script:

    # z4mod updater-script
    
    # TODO: assert fsck.ext2 exists!
    
    ui_print("");
    ui_print("--[ z4mod Convert, by Elia Yehuda");
    ui_print("--[ (c) 2010, GPLv2");
    ui_print("");
    ui_print("--[ Starting convert process");
    ui_print("--[ This WILL take a while...");
    ui_print("");
    
    show_progress(0.1, 0);
    
    # we must have /sdcard mounted to host compressed files
    mount("vfat", "/dev/block/mmcblk0p1", "/sdcard");
    assert( is_mounted("/sdcard"));
    #delete("/sdcard/z4mod.log");
    
    
    ui_print("--[ Extracting /system");
    package_extract_dir("system", "/system/");
    
    # make sure busybox/sh can run
    #set_perm_recursive(0, 2000, 0755, 0755, "/system/xbin");
    #run_program("/system/xbin/busybox", "--install", "-s", "/system/xbin/");
    set_perm(0, 0, 04755, "/system/xbin/busybox");
    symlink("/system/xbin/busybox", "/system/xbin/sh");
    
    show_progress(0.1, 0);
    
    #
    # Start converting partitions
    #
    
    package_extract_file("z4mod", "z4mod");
    set_perm(0, 0, 04755, "z4mod");
    
    # Converting /cache for testings only - should be removed
    ui_print("--[ Converting /cache");
    show_progress(0.3, 5);
    # TODO: consider removing assert - don't want to exit on error...
    assert( run_program("z4mod", "cache", "stl11"));
    #
    
    ui_print("--[ Converting /data");
    show_progress(0.3, 120);
    # TODO: consider removing assert - don't want to exit on error...
    assert( run_program("z4mod", "data", "mmcblk0p2"));
    
    #
    # Finished converting partitions, now we reboot
    #
    
    # Flashing the kernel for my testings only, needs to be removed.
    ui_print("--[ Flashing kernel, will reboot when done...");
    package_extract_file("redbend_ua", "redbend_ua");
    package_extract_file("zImage", "zImage");
    set_perm(0, 0, 0755, "redbend_ua");
    show_progress(0.1, 5);
    run_program("redbend_ua", "restore", "zImage", "/dev/block/bml7");
    #
    
    ui_print("--[ Done. rebooting now...");
    show_progress(0.2, 0);
    run_program("reboot", "now");
    

    z4mod:

    #!/system/xbin/sh
    #
    # z4mod, by Elia Yehuda, (c) 2010, GPLv2
    # backup a partition, format, and restore
    #
    # TODO: Better backup - dump rfs partition as image, 
    #       mount as loop to restore data, dump using redbend_ua/dd for recovery.
    #       It adds a bit of complexity to the code (loop), but _much_ more safe.
    #
    # TODO: *cough* Check for free space *cough*
    set -x
    (
    export PATH=/:/system/xbin/:$PATH
    
    halt()
    {
    	umount /${partition}
    	if [ "${partition}" == "data" ]; then
    		umount /dbdata
    		busybox rm -f /sdcard/mmcblk0.mbr
    	fi
    	exit 1
    }
    
    busybox cat <<EOF
    ###############################################################################
    #
    # starting z4mod, logging into z4mod.log
    # `busybox date`
    #
    ###############################################################################
    EOF
    
    [ `busybox which parted` ] || exit 1
    [ `busybox which mkfs.ext2` ] && format=mkfs.ext2
    [ `busybox which mkfs.ext3` ] && format=mkfs.ext3
    [ `busybox which mkfs.ext4` ] && format=mkfs.ext4
    [ "${format}" == "" ] && exit 1
    [ `busybox which fsck.${format/*mkfs./}` ] || exit 1
    
    # TODO: Test mkfs first, check if our running kernel supports this fs!
    #       The user must pre-install a kernel supporting this fs!
    
    #dd if=/dev/zero of=/sdcard/tmp.img bs=1024 count=1K
    #${format} -F /sdcard/tmp.img
    #mkdir /sdcard/z4mnt
    #ext_type=${format/*mkfs./}
    #mount -t $ext_type -o loop /sdcard/tmp.img /sdcard/z4mnt
    #fs_is_supported=$?
    #umount /sdcard/z4mnt && rm -r /sdcard/z4mnt
    #rm -f /sdcard/tmp.img
    
    partition="$1"
    devblock="$2"
    
    # /system takes special handling before we can unmount it
    # TODO: NOT TESTED YET!
    if [ "${partition}" == "system" ]; then
    	busybox killall adbd
    	busybox killall recovery
    fi
    
    # we unmount and remount target - exit on failure
    busybox umount /${partition}
    busybox mount /dev/block/${devblock} /${partition} || exit 1
    
    # remove old backup if any
    busybox rm -f /sdcard/${partition}.tar.bz2
    
    # we need to add /dbdata contents to the backup
    if [ "${partition}" == "data" ]; then
    	busybox mkdir /dbdata
    	mount -t rfs -o nosuid,nodev,check=no /dev/block/stl10 /dbdata || halt
    	#busybox mount -t rfs -o nosuid,nodev,check=no /dev/block/stl10 /dbdata || halt
    	# backup target - exit on failure
    	busybox tar zcvf /sdcard/${partition}.tar.gz /${partition} /dbdata || halt
    else
    	# backup target - exit on failure
    	busybox tar zcvf /sdcard/${partition}.tar.gz /${partition} || halt
    fi
    
    # umount - exit on failure, no harm is done
    busybox umount /${partition} || halt
    
    # we have to recreate the partition as ext2/3/4 in the partition table
    if [ "${partition}" == "data" ]; then
    	# backup the current partition table
    	busybox dd if=/dev/block/mmcblk0 of=/sdcard/mmcblk0.mbr bs=512 count=1
    	# grab the current (RFS) start/end partition layout
    	start_end=`parted -s /dev/block/mmcblk0 unit B print quit | busybox grep '^ 2' | busybox awk '{print $2,$3}'`
    	# extract the ext type from the mkfs.???? name so we know what ext type are we dealing with
    	ext_type=${format/*mkfs./}
    	# remove the 2nd partition, and recreate a primary partition of type ext2/3/4 	
    	parted -s /dev/block/mmcblk0 rm 2 unit B mkpartfs primary ${ext_type} ${start_end} quit
    	# need to bail out on error, but parted always returns error...
    #	if [ $? != 0 ]; then
    #		# restore the partition table
    #		busybox dd if=/sdcard/mmcblk0.mbr of=/dev/block/mmcblk0 bs=512 count=1
    #		exit 1
    #	fi
    fi
    
    # Format partition with optimized settings (TODO: restore on error)
    # TODO: check why '-E lazy_itable_init=1 -O sparse_super,uninit_bg' doesnt work (with ext2 testings)
    ${format} -L ${partition} -b 4096 -m 0 -F /dev/block/${devblock}
    
    # tune the filesystem - check every 100 mounts or 30 days
    tune2fs -c 100 -i 30d -m 0 /dev/block/${devblock}
    
    # mount the new filesystem - exit on failure (TODO: restore on error)
    busybox mount /dev/block/${devblock} /${partition} || halt
    
    busybox tar zxvf /sdcard/${partition}.tar.gz -C /
    ret=$?
    [ $ret == 0 ] && rm /sdcard/${partition}.tar.gz
    
    # umount
    busybox umount /dev/block/${devblock}
    if [ "${partition}" == "data" ]; then
    	umount /dbdata
    	busybox rm -f /sdcard/mmcblk0.mbr
    fi
    
    exit $ret
    
    # everything is logged into /sdcard/z4mod.log
    ) >> /sdcard/z4mod.log 2>&1
    
    Advertisements

    15 Comments

    1. I tried to manually implement the ext2 convert and mount for the /data partition, the kernel works and data partition was converted and mounted under ext2, however, after the mobile restarted, the phone failed to start and seems the /dev/block/mmcblk0p2 is still in rfs format and mounted as rfs, why is the partition still in rfs format after restarted?

      1. hmm, i dont think the partition is still in RFS – what I think is that the init script TRIES to mount it as RFS. To confirm this, try to manually mount the partition. If it mounts as ext2, then you’ll have to patch the init scripts. If it won’t mount as ext2, do report back.

    2. i have reply your post on the xda(id: haoning), it works now after patching init, but there is still some manual work to change the partition via partedstatic tool, only using mkfs.ext2 does not work after rebooted, maybe you need to integrate this into your script. Anyway, thanks for the great work.

    3. actually, i just converted the data partition and got 1500, will try to convert all except system to see if it is improved much. πŸ™‚

      1. I doubt other partitions would enhance the speed more, maybe just by a fraction – they are all small-sized partitions. Maybe /dbdata might give a tiny boost, and /system will probably shorten startup speed, although if you’ll check z4mod script you’ll notice /system takes greater care when formatting.

    4. just tried to convert other rfs partition other than system(could not umount system partition under recovery mode), it works but seems the score does not improve at all

    5. after checked, the system becomes unstable and many application becomes fc (e.g. camera fc, nothing found in album…), seems there is problem when using ext2 for /dbdata…

    6. and also for /dev/block/stl3, whichi is used for radio, after refreshing the official ROM, the phone function is totally disabled as it is not able to be mounted as rfs, someone who can tell me how to conver this partition back to rfs format?

    7. It looks like a very important script – the kernel repacker – is missing. would you be so kind as to share it with us?

      # where to find kernel_repack/editor.sh
      KERNEL_REPACKER=/home/ryanza/kernel_repacker/editor.sh

    8. Ram :
      It looks like a very important script – the kernel repacker – is missing. would you be so kind as to share it with us?
      # where to find kernel_repack/editor.sh
      KERNEL_REPACKER=/home/ryanza/kernel_repacker/editor.sh

      Nevermind, I found it on XDA

    Leave a Reply

    Fill in your details below or click an icon to log in:

    WordPress.com Logo

    You are commenting using your WordPress.com account. Log Out / Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out / Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out / Change )

    Google+ photo

    You are commenting using your Google+ account. Log Out / Change )

    Connecting to %s