Fixed! ROBOCOPY Usage on Linux EXT2 / EXT 3 File Systems

For some time, I have been a fan of the robust copy feature that has been available for several Windows releases including Windows XP, Windows 2003, and of course Windows Vista. In fact, I changed from using programs like SecureCopy or built previously built in utilities like cp (copy) or xcopy.

In fact my primary to secondary disk backup method for some time has been to use the following command syntax to mirror directories to the backup drives:

ROBOCOPY "C:\My Music" "B:\Backup\My Music" /MIR

With the above syntax, it creates an exact mirrored copy of the data in the directory "C:\My Music" and copies (purging deleted files and directories) the entire tree to "B:\Backup\My Music". This works like a charm on NTFS files systems.

Now enter into the picture my new NAS device which is running a Linux based EXT2 / EXT3 file system and we start to see some problems. I first noticed the symptoms I saw is that when I was testing out the backup copies to the new location on that Linux based network attached storage device. Specifically speaking, I would see the identical file being copied over and over again.

Thinking that it most likely had to do with either an archive bit or similar type backup issue, I first tried adding the /M switch to the command line to forcibly set the archive bit with every file.

That new syntax looked like: ROBOCOPY "C:\My Music" "B:\Backup\My Music" /MIR /M

However, that was not much of a help -- so my attention turned to the Linux EXT2 / EXT3 file system structure. Some research by entering "robocopy ext2" into Google revealed some information buried ironically within a NetGear message board. The posts revealed the following information about inodes that I was not too familiar with:

The standard ext2/ext3 inode has 3 time fields, last inode change, last file data change and deletion time. These timestamp fields have a resolution of 1 second. The only time field that maps from NTFS to ext2/ext3 is the file data modification time.

That revealed the issue -- the NTFS timestamp was more precise than the resolution for the EXT2 / EXT3 inode file systems. Therefore, a different option had to be tweaked with ROBOCOPY to allow for a less precise timestamp measurement.

Fortunately, out of the box there is a /FFT switch for ROBOCOPY which specifically accomplishes this task by assuming FAT File Times which translates to a more forgiving 2-second granularity. This is more than sufficient to account for the one-second granularity that the EXT2 / EXT3 file systems support.

So, I modified my backup command one more time by adding '/FFT' to the end of the command:
ROBOCOPY "C:\My Music" "B:\Backup\My Music" /MIR /FFT

I gave the command a couple of tests and bingo, that solved the problem of the same files copy over and over again. If you are encountering a similar issue, then most likely the /FFT switch may resolve issues that you have copying files between NTFS and other non-NTFS systems including Linux and others.

Good luck!

‎ Read more of this post!

How to Restore Backups to Your Motorola V3 RAZR Phone

I have had a few request for this guide on how to restore your Motorola RAZR V3 phone from a backup. It is a much needed guide, and one that it took me a little while to put together. Why is it needed? As we all know, when you modify your RAZR phone, it is part science -- part experiment. Sometimes, with just a simple mistake or incorrect tweak, you can disable your phone and make render it useless.

In fact, I accidently did it just over a month ago -- disabled my phone when I was messing with a seem on my RAZR.

The reality is that you should always 1) have a good backup of you Motorola phone (guide here) and 2) know how to restore it if you are going to mess with the settings and files on it. If you do not, you are taking a big risk in ending up with a bricked phone, and possibly paying for the carrier to reload software on it.

So, without further warning, here is the basic how-to guide on restoring your phone from a backup. This guide assumes that you have drivers installed to connect your RAZR V3 to your computer. If you do not, you can go here to find out more.

1. If you do not already, get a copy of Flash&Backup, a great shareware software that you can use to create and restore backups as well as flash your phone. The initial download is good for 30 days, and there are older, less functional versions out there that are freeware.

2. Choose your phone (Motorola V3) from the drop down list and locate the backup file you created previously. For this example, I selected a SBF (Single Binary File) that contained my backup information that I just re-created today. Make sure that all of the files are selected by clicking on the "Select all" checkbox. This tool supports more than just the RAZR V3, for a complete list, please see the bottom of this post.

3. Put you phone into the bootloader mode. If you do not know how to, go here to see the guide.

4. Your phone will show connected to the Flash&Backup program. Now click on the write data button to start the restoration of your backup file.

5. The tool immediately deletes all the data on your phone before starting the restore process.

6. The restore process takes approximately 3 minutes to complete to restore all of the files and settings to your Motorola V3.

7. The restore process completes, and automatically restarts your phone.

8. Your phone is now restored to the previous state and you are free to use it as you needed. Congratulations, you did it.

This program and guide is also compatible with the following phones:
Motorola C380, Motorola C381P, Motorola C390, Motorola C650, Motorola E1 ROKR, Motorola E1000, Motorola E1070, Motorola E375, Motorola E398, Motorola E770v, Motorola K1 KRZR, Motorola K3, Motorola L2, Motorola L6, Motorola L7 SLVR, Motorola L7e, Motorola U6 PEBL, Motorola V180, Motorola V186, Motorola V188, Motorola V220, Motorola V235, Motorola V3, Motorola V300, Motorola V360, Motorola V3i, Motorola V3r, Motorola V3re, Motorola V3t, Motorola V3x, Motorola V3xx, Motorola V400, Motorola V500, Motorola V547, Motorola V6 MAXX, Motorola V600, Motorola V620, Motorola V635, Motorola V980, Motorola Z3 RIZR

‎ Read more of this post!

It's Time to Backup Your Digital Photos!

With the prevalence of Digital Photos, we end up with many more photographs than we had back in the film processing days. That adds up to a ton extra snapshots – to make sure you get that ‘perfect’ picture – comes a larger problem, a lot more photos taking up a lot more hard drive space.

Of course, the more photos you add to your computer, the more risk there is that you lose everything if you hard drive or your computer crashes. Not an ideal situation, to say that least.

It is the right time of year to think about how you care for your online photos. Having just entered into 2008, you need to think about how you are protecting yourself from that next hard drive crash.

The way I look at it, you have a couple of options.
1. Subscribe to an online service that allows you to upload your photos to them. Among many service out there: Google’s Picasa, Yahoo’s Flickr, and Shutterfly – all popular services. While Google (one of the few times) trails in this area, for $24.95/year allows unlimited uploads, storage, and a 10MB file limit. With a 10.1MP camera, the JPEG images produce end up in the 4-5MB range so that should be plenty. Picasa and Shutterfly may also reduce the size of your original image degrading the quality – although Shutterfly does allow unlimited picture storage today.
2. Your other option is to put in a process so that you protect yourself against the loss of such pictures.

I could write for the next 10 pages about the pros and cons of the two choices, but will attempt to sum it up in a couple of sentences.

For the online option the pros are someone else is worrying about storing your pictures, and you can get to them from anywhere you are at, often for a small feed. The cons are that you’re at the mercy of the services if they change their terms, go out of business or have a major disaster – remember online photo storage has only been around for a short period of time – and you must have a high-speed upload connection – cable modems and DSL may not be quick enough for high quality pictures. Another big con is that if you shoot in RAW mode, even Flickr converts it to a JPG, so if retaining the original quality of the photo is important, this may not be an appropriate option.

For the do-it-yourself option, you have to commit to whatever processes you put in place to make sure you are consistent. The buck stops with you, but you can make it more fool and economy proof.

Thus far – I have chosen the do-it-yourself methodology although I have recently considered an upgrade to Flickr Pro at the $24.95 rate. For me, it is both a daily backup (automatic) and an annual archive process. The rest of this post will cover the process I have put in place.

First though, the situation:
For the year 2007 that just ended, I took over 11GB worth of photos from both our Canon PowerShot G7 and the Powershot SD800IS. As I repeat the process, the size of the pictures I took in 2007 was nearly triple the size of the photos in 2006, and 2006 was triple the size of 2005. See a progression -- does this mean 40GB for 2008?

The process I have put in place consists of the following...

First, I have an extra hard drive in my computer that is just for backup purposes. Every night at 10:00PM I have a backup job using a basic ROBOCOPY backup job to copy all my pictures and other important things to the backup drive. That insures if my primary hard drive goes bad, then I still have all the valuable data and pictures backed up!

Secondly, at the beginning of every year – and today is that day – I create and Archive folder on my hard drive and move all of the photos from the previous year into that folder. Once I create that location, I spend a couple of hours removing pictures I do not want to keep and then prepare to move them to media. From there, I create the appropriate number of DVDs via my DVD burner with Roxio Creator DE – this year it was 3 DVDs. Then I take those DVDs, put them in my fire safe and consider that to be relatively safe. In the next couple of years I plan to instead move the DVDs to a safety deposit box at a secure location like a bank.

Just set an annual reminder in Outlook to make sure you do not forget. And with that, in the case of most disasters – all of our photos are protected in multiple ways.

Have you backed up your pictures recently? If not, now is a great time to start!

‎ Read more of this post!

Using ROBOCOPY to Backup Files instead of XCOPY or COPY

With my new 500GB hard drive, I have the need to backup or synchronize the important files on my computer. The copy, is for the simple purpose if the primary drive on the operating system goes down. I have had this happen before and it is very painful to recover the data from it. (Post coming soon on this)

I am not a fan of the shadow copy or backup utility provided in Windows Vista as it keeps more of a history of changes on the file system then simply perserving the current content of a disk system. In the past I had used Secure Copy 3.X by ScriptLogic to accomplish this. That version however does not work on Windows Vista 64-bit (x64), neither does their latest version 4.X based on my installation test yesterday. Based on the trouble of locating a tool, I started thinking about the age-old XCOPY utility and accidently stumbled upon ROBOCOPY which is included by default in the Windows Vista Operating system as a utility.

Those of us familiar with the copy, cp, or xcopy commands were not so robust and often choken when you needed to copy hundreds or thousands of files. Not to mention all the switches you needed to learn to make sure you did not clobber, included directories and so forth. That's where ROBOCOPY comes in.

With most resource kits including Windows NT, Windows 2000, and Windows 2003, a new version or upgrade of XCOPY.EXE or the traditional COPY.EXE was made available in the resource kit called ROBOCOPY. (ROBOCOPY.EXE). ROBOCOPY is supposed to stand for Windows Robust File Copy and is available via the Windows Command Line via a couple of simple commands.

The best part? With a simple command and just a single switch, you can synchronize directories on your computer with little or no effort. What does the command look like? Here is an example:

ROBOCOPY "C:\My Music" "B:\Backup\My Music" /MIR

The MIR switch stands for "Mirror a complete directory tree. That means that it not only copies the new files but deletes any files that are no longer there to manage a synchronized backup of a directory. And it is fast. For instance for the system to read and detect the 4600+ music files on my machine and complete the synchronization (already completed before) was less than a second worth of time.

With just a few simple commands added to a .bat or Batch file that you can make inside of notepad and save as a .bat file, you can create a pretty effective backup job that will synchronize you files. In my case, I setup the batch job inside of the scheduled task manager to run each evening to keep everything backed up properly.

One thing to watch out for, if you are copied directories where files such as NTUSER.DAT or other files might be locked and are relatively unimportant, use the the /R:0 switch to set the access denied retires to zero. That will keep your job moving quickly!

Best of luck with creating a quick and easy backup job on your personal or professional computer!

Here is the list of commands when just typing in "ROBOCOPY" into the command line:
-------------------------------------------------------------------------------
ROBOCOPY :: Robust File Copy for Windows

-------------------------------------------------------------------------------

Started : Sun Aug 19 17:34:35 2007

Simple Usage :: ROBOCOPY source destination /MIR

source :: Source Directory (drive:\path or \\server\share\path).
destination :: Destination Dir (drive:\path or \\server\share\path).
/MIR :: Mirror a complete directory tree.

For more usage information run ROBOCOPY /?


**** /MIR can DELETE files as well as copy them !

Here is the full list of commands when using help:
C:\Users\Ken>robocopy /?

-------------------------------------------------------------------------------
ROBOCOPY :: Robust File Copy for Windows

-------------------------------------------------------------------------------

Started : Sun Aug 19 17:11:55 2007

Usage :: ROBOCOPY source destination [file [file]...] [options]

source :: Source Directory (drive:\path or \\server\share\path).
destination :: Destination Dir (drive:\path or \\server\share\path).
file :: File(s) to copy (names/wildcards: default is "*.*").

::
:: Copy options :
::
/S :: copy Subdirectories, but not empty ones.
/E :: copy subdirectories, including Empty ones.
/LEV:n :: only copy the top n LEVels of the source directory tree.

/Z :: copy files in restartable mode.
/B :: copy files in Backup mode.
/ZB :: use restartable mode; if access denied use Backup mode.
/EFSRAW :: copy all encrypted files in EFS RAW mode.

/COPY:copyflag[s] :: what to COPY for files (default is /COPY:DAT).
(copyflags : D=Data, A=Attributes, T=Timestamps).
(S=Security=NTFS ACLs, O=Owner info, U=aUditing info).

/DCOPY:T :: COPY Directory Timestamps.

/SEC :: copy files with SECurity (equivalent to /COPY:DATS).
/COPYALL :: COPY ALL file info (equivalent to /COPY:DATSOU).
/NOCOPY :: COPY NO file info (useful with /PURGE).

/SECFIX :: FIX file SECurity on all files, even skipped files.
/TIMFIX :: FIX file TIMes on all files, even skipped files.

/PURGE :: delete dest files/dirs that no longer exist in source.
/MIR :: MIRror a directory tree (equivalent to /E plus /PURGE).

/MOV :: MOVe files (delete from source after copying).
/MOVE :: MOVE files AND dirs (delete from source after copying).

/A+:[RASHCNET] :: add the given Attributes to copied files.
/A-:[RASHCNET] :: remove the given Attributes from copied files.

/CREATE :: CREATE directory tree and zero-length files only.
/FAT :: create destination files using 8.3 FAT file names only.
/256 :: turn off very long path (> 256 characters) support.

/MON:n :: MONitor source; run again when more than n changes seen.
/MOT:m :: MOnitor source; run again in m minutes Time, if changed.

/RH:hhmm-hhmm :: Run Hours - times when new copies may be started.
/PF :: check run hours on a Per File (not per pass) basis.

/IPG:n :: Inter-Packet Gap (ms), to free bandwidth on slow lines.

::
:: File Selection Options :
::
/A :: copy only files with the Archive attribute set.
/M :: copy only files with the Archive attribute and reset it.
/IA:[RASHCNETO] :: Include only files with any of the given Attributes set.
/XA:[RASHCNETO] :: eXclude files with any of the given Attributes set.

/XF file [file]... :: eXclude Files matching given names/paths/wildcards.
/XD dirs [dirs]... :: eXclude Directories matching given names/paths.

/XC :: eXclude Changed files.
/XN :: eXclude Newer files.
/XO :: eXclude Older files.
/XX :: eXclude eXtra files and directories.
/XL :: eXclude Lonely files and directories.
/IS :: Include Same files.
/IT :: Include Tweaked files.

/MAX:n :: MAXimum file size - exclude files bigger than n bytes.
/MIN:n :: MINimum file size - exclude files smaller than n bytes.

/MAXAGE:n :: MAXimum file AGE - exclude files older than n days/date.
/MINAGE:n :: MINimum file AGE - exclude files newer than n days/date.
/MAXLAD:n :: MAXimum Last Access Date - exclude files unused since n.
/MINLAD:n :: MINimum Last Access Date - exclude files used since n.
(If n < 1900 then n = n days, else n = YYYYMMDD date).

/XJ :: eXclude Junction points. (normally included by default).

/FFT :: assume FAT File Times (2-second granularity).
/DST :: compensate for one-hour DST time differences.

/XJD :: eXclude Junction points for Directories.
/XJF :: eXclude Junction points for Files.

::
:: Retry Options :
::
/R:n :: number of Retries on failed copies: default 1 million.
/W:n :: Wait time between retries: default is 30 seconds.

/REG :: Save /R:n and /W:n in the Registry as default settings.

/TBD :: wait for sharenames To Be Defined (retry error 67).

::
:: Logging Options :
::
/L :: List only - don't copy, timestamp or delete any files.
/X :: report all eXtra files, not just those selected.
/V :: produce Verbose output, showing skipped files.
/TS :: include source file Time Stamps in the output.
/FP :: include Full Pathname of files in the output.
/BYTES :: Print sizes as bytes.

/NS :: No Size - don't log file sizes.
/NC :: No Class - don't log file classes.
/NFL :: No File List - don't log file names.
/NDL :: No Directory List - don't log directory names.

/NP :: No Progress - don't display % copied.
/ETA :: show Estimated Time of Arrival of copied files.

/LOG:file :: output status to LOG file (overwrite existing log).
/LOG+:file :: output status to LOG file (append to existing log).

/UNILOG:file :: output status to LOG file as UNICODE (overwrite existing
log).
/UNILOG+:file :: output status to LOG file as UNICODE (append to existing
log).

/TEE :: output to console window, as well as the log file.

/NJH :: No Job Header.
/NJS :: No Job Summary.

/UNICODE :: output status as UNICODE.

::
:: Job Options :
::
/JOB:jobname :: take parameters from the named JOB file.
/SAVE:jobname :: SAVE parameters to the named job file
/QUIT :: QUIT after processing command line (to view parameters).

/NOSD :: NO Source Directory is specified.
/NODD :: NO Destination Directory is specified.
/IF :: Include the following Files.

C:\Users\Ken>robocopy

-------------------------------------------------------------------------------
ROBOCOPY :: Robust File Copy for Windows

-------------------------------------------------------------------------------

Started : Sun Aug 19 17:12:02 2007

Simple Usage :: ROBOCOPY source destination /MIR

source :: Source Directory (drive:\path or \\server\share\path).
destination :: Destination Dir (drive:\path or \\server\share\path).
/MIR :: Mirror a complete directory tree.

For more usage information run ROBOCOPY /?


**** /MIR can DELETE files as well as copy them !

Trackbacks that led to this post:
using robocopy
robocopy efs raw
robocopy monitor problems
robocopy usage
robocopy x64
using robocopy for backup
access denied batch file shared directory xcopy
robocopy backup
robocopy copy ntuser.dat
robocopy shadow backup
robocopy shares migrate
robocopy tutorial
xcopy ""access denied"" fix
xcopy svchost access denied
+""using robocopy"" +""windows vista""
after using robocopy files are read only
append blackberry command
backup robocopy xcopy
command xcopy /exclude usage
copy encrypted files with robocopy raw mode
copy instead of backup
copy raw $efs robocopy
difference between robocopy, xcopy and secure copy
efs how do i robocopy
exclude junction points
filecopy efs raw
how do i reset robocopy
instead of xcopy
max file size for xcopy
robocopy
robocopy ""backup mode"" locked
robocopy ""job options"" example
robocopy + /xo wont work
robocopy + mir + examples
robocopy /ipg recommended
robocopy /mir : mirror a complete directory tree.
robocopy 60gb directory
robocopy and dst
robocopy backup files in use
robocopy backup ntuser.dat
robocopy backup options
robocopy batch file that excludes old files
robocopy command
robocopy complete directory move
robocopy copy files minage maxage
robocopy create directory tree only example
robocopy dcopy gui
robocopy directory timestamp
robocopy does not work on x64

‎ Read more of this post!