Looking for a new windows computer backup system – a rant

I’ve used Retrospect since their first days in the Mac business. Spent a few hundred dollars a few times too many on upgrades. About the time CDs were becoming available, Retrospect was realizing there was no more money to be made, so they cut development to the bone and there were many years that CD backup never worked properly. This continued into DVD backups. Sorta works, but often not. And then they had one really bad release that made everyone furious, and they lost their way.

Well now I have 2TB in video file archives and Retrospect is getting wonky. Too many coasters (Could be my software, of course, but that is the problem innit?) Damn thing fails to do an incremental backup to DVD. I have an old x2 pioneer dvd, and I even bought an external usb dvd to see if that could fix the problem.

Is it time for something new? Yes. Oh dear, this is going to suck…

The thing about backup systems is that it only sounds easy. The truth is, behind “easy” is a rats-nest of potential problems. You have to be a really good programmer and you have to be good at hardware issues, and that is rare. For example, say you backup your files from “A” to “B”. All is good if it works. But what if your computer crashes while copying these files, and when you reboot you discover that “A” is corrupted and “B” is corrupted? You’re screwed. The problem with the free stuff and the poorly written stuff is that you’re at risk for precisely this kind of thing.

This is my current backup need:

1) I have one primary computer (windows xp) which contains all the important stuff. This has to be backed up every night. It is an old (year 2000) hand-built system and is flaky, so I may be forced to rebuild this computer at any time. I cannot lose the data. The computer has a 600GB raid 0 hard drive. I have a backup drive with 120 GB, and a second backup drive with 700GB.

2) I have other laptop computers which I don’t really care about. As long as I can rebuild them in case of a meltdown, that is good enough. Would be nice to back them up if it was easy and free, but not otherwise.

3) I do a lot of video editing. I have a folder called “Video Editing” where all my final work is saved. When that folder gets very large, I save it all to DVD, then delete the contents and start over. I have 2 TB of data in that folder that I have archived to DVD. I need to be able to search and retrieve anything that was ever there.

4) If my house burns down tonight, the really important files need to survive. Currently I backup these files to DVD every month or so and take the DVDs to work, but it is a pita and I forget. The latest backup software lets me backup to “the cloud”, so I could store my critical files on the web automatically and be sure they are safe. Now those files need to be secure, i.e., no one else can read them.

I do not have a “network storage system” or NAS. All computers have their own hard drives, and sharing is done via “shared folders”. NAS are terrific but expensive, and I am cheap.

Three weeks ago, I had one too many frustrations with Retrospect screwing up DVD archiving, so I decided to look around. I saw plenty of reviews, but all of them were BS. “Fluffy” tech writers who check stuff off feature lists and don’t get it. I read good reviews on software that I knew was junk within 10 seconds.

I don’t have to time to put together my own feature list, but I do have time to write long meandering posts. So that is what I’ll do. If you want to know my recommendations, skip to the end. Or read the next sentence: Truth is, Retrospect is still the best, provided your computer is new enough to avoid the hardware frustrations. And a tentative recommendation for NovaBackup.

Terminology

System Restore: The ability to rebuild your computer after a catastrophic crash. It is one thing to recover files, but all that installed software, all those custom settings … without it you could easily spend a week trying and failing to get back to where you were before. An advanced variant of this is System Restore to different hardware, where one recovers to a different computer.

Session You don’t backup something once. Usually you back everything up once, then back it up incrementally many more times. Each of these times is called a Session.

Full vs. Incremental backup: Say you backup everything on Monday. On Tuesday you want  to backup again. You could do everything all over again (full), or you could just backup the things that changed since Monday (incremental) It is a given that you might want to recover back to either Monday or Tuesday.

Network vs. Local backup: Are you backing up one computer or a network of computers?

Offsite backup: If the house burns down, can you recover? Some software vendors offer cloud storage where they let you upload to the internet for a monthly fee. Some software has FTP-storage, where you upload to an ftp-server. This is terrific because I have a website with an ftp-server that gives me unlimited storage.

3rd party recoverable: Do you need the backup software installed in order to recover the data? Sounds simple, but I have been archiving my videos for over 12 yrs. If the computer goes bankrupt and I move to a new operating system (where the software no longer works), I’m screwed.

File filters When you backup a hard drive or folder, you may want to include or exclude certain files. For example, who needs the temporary “tmp” files? So you can add a filter that says “never store files with names matching the pattern “*.tmp”. This is practically a must-have feature, but some apps don’t support it.

Validate After writing the backup, read it back and verify that everything was written correctly.

I begin the search

The first thing is to find a good list of options. I google searched for “windows backup software”. This gave me a good list from wikipedia and some horrible review sites. I looked at everything. Several tools are more applicable to large networks and really expensive stuff, which I won’t cover. I’m just going to bag on all the other options.

Retrospect

I bought Retrospect when it first came out for the Mac around 1990. It was rock-solid and far ahead of it’s peers. This was first-rate engineering. However, when CD burners appeared, support was very poor, and this continued into the DVD world. I suspect all the good engineers are gone and they only have support personnel left. Retrospect does not have FTP-storage, and stores everything is a custom format. Also it is relatively expensive at $119.

GFI Backup Freeware

This is a product that has been turned into freeware. Meaning the company could not make enough money on it, so they are giving it away. Meaning that they will soon stop supporting it. Meaning you can stop right here.

Now I did try it and the experience was bizarre. From my original notes…. “While burning DVDs, it seemed to be working nicely but I don’t really know what it was doing. Poor visual progress feedback. Makes a compressed copy somewhere else then burns it. I don’t like that it can’t do it on the fly and I don’t like that I don’t know where the file is and I don’t like that it is really slow (compressing 6GB takes over 1 hr before it even decides to burn to DVD?). I think … I really don’t know. Somehow it seemed to archive 6GB to a 4.3GB DVD, which is impossible. Then I added another 6GB and it ejected the DVD, which I visually inspected to see that it did not completely fill the DVD. And it completely confuses Windows. When I do a “properties” inspect on the mounted DVD, it says I have used 986MB with 0 available. And this on a 6GB backup! I probably can figure this out, but too much under the covers so far, so let us move on.”

 Norton Ghost

Hell, I couldn’t get FTP setup properly. Also, ghost backs up systems, not individual folders.

Paragon

Moving on to paragon software. At least they charge $40 so we can trust them, right? Inspection of the manual looks positive. But damn, what an ugly UI. Simple crappy options for the noob user, an “advanced” tab with more crappy options. Your conventional “wizard” interface for the boobs that is unexplained, and a conventional expert interface that has all the options that the expert needs and no help figuring out where they are.

I tried to select a folder and back it up to DVD, but it would not let me. Apparently this is not allowed? DVD backup only for disk or partition backups? Are you kidding me? Huh, this is a must-have feature and should be obviously supported. Hmm, lookup the website for support, but there is no help. Try and submit a question, but their website is so buggy that it puts me into an infinite loop. I have to specify the particular product, but the particular product does not show in their dropdown. Doesn’t matter because no matter what I select, it is refused and sends me back to the same place.. Leave a nasty message on their facebook page and see if they respond. Seriously, this is exhibit A for terribly designed and unhelpful interface. Separate issue from whether or not it works properly under the covers, but how can I find out? P.S. The help buttons don’t actually do anything. Wow. Moving on…

Note: I did eventually get a response from my support question. You cannot write a folder to DVD. Just a bizarre omission.

 Backup4All

Initially I was really encouraged. This is one of the best designed apps. (You can tell the UI programmers were good.) It would have been a clear winner, except it has no system restore. And then I was testing some backups to DVD and it locked up the DVD during an incremental backup. After reboot, the error was not recoverable. Imagine crashing on your 100th DVD, then not being able to tell the system that your 100th DVD was bad and can we continue with #101? The tool could not recover. It thought it had backed up files it hadn’t. Fatal flaw.

Acronis True Image

Extremely buggy.. I created some files filters, and they disappeared when I saved and reopened the app. It ejected a DVD I was writing to, thought for about 5 min, then gave me an error. I eventually discovered that if you ran the backup project (called an “archive”) immediately after creating it, then it saved correctly. However, if you saved it and then ran it, it failed to save properly..

The default is not to validate after storage. This is really dumb.

When testing recovery of a DVD archive, it failed and gave me a weird error message (something like “Error 31. See website”). When I looked up the website, the error code said “The DVD must be in the drive before starting the restore”. Are you kidding me?

The problems went on and on. I could not believe what a PISA this app was, since it is so popular (almost 24,000 likes on their FB page) and is often highly recommended. Then again, many of the posts in FB go something like this: “Anyone that uses this product needs their heads examined. It is, by far, the worst product ever to hit the net.”

Genie9

I started with high hopes, but got into trouble fast. First, I discovered that system restore only works in full backup mode and copies the User Documents and Program Files folders only. You can add folders, but you have to do it manually every time and there is no incremental backup with it. This is really dumb. I asked a question of their support and was told that this was the way it should work and any other way would not work. Which was odd, as I have 20 yrs experience to the contrary.

During that backup test, I had 60GB available on my target hard drive, but needed to backup 61GB. Genie told me it was probably compress to 40GB, but even then refused to continue with the backup. Jeez software; try and fail, but don’t refuse to try.

During DVD backup, if you start the project without a blank  DVD inserted in the drive, the tool simply fails silently with any kind of warning or error message. It got into trouble when it ejected the DVD then windows immediately complained and I had to reinsert it. More than a few times.

The real big problem was that it does not come with “packet writing” support for DVD. This is the ability to write half a DVD during one session, then fill up the other half during the next session. So when I first tried this, it tried to format the (non-formattable, already burned) previous DVD, causing a ruined backup. Windows 7 and up is supposed to come with this built-in, but XP requires some free software add-on. Free but not reliable, unfortunately, and it can screw up other tools that use your DVD. I tried for a couple of days to fix all this, but I eventually gave up. I hereby reject any backup software which does not handle this internally. You should too.

NTI Backup Now

No file filtering on system backup. Backup the entire hard drive or nothing at all. No thanks.

Bitser

Simple free tool using the 7zip utility. Nice, but it cannot write to DVD. Not a real solution.

Comodo

Comodo give you 5GB free backup on the web. However, it’s DVD backup failed when I tried to store files in zip format and only worked in it’s native format. Ok, but then it only allowed me to write a single backup. I could not append a backup in a second session. Next…

Crashplan

Now this is really interesting. A free java tool that lets computers back themselves up to other computers in a network. This might be a way to backup multiple computers. Very cool, and I will be exploring this in the future. However, crashplan does not do system restore and it cannot backup to DVD.

 GRBackPro

My first thought was “how ugly is this?” This tool also suffers from needing 3rd-party packet-writing software, so I did not bother any further.

7-Backup

Sorry. No trial version, no try.

Macrium Reflect

The free version cannot backup folders, so I could not test it. The DVD burning needs packet-writing software. You cannot do an image backup of the hard drive, but only select a subset of folders.

Shadow Protect

Image backup and system restore only

SyncBackPro

Finalizes the DVD after the first session. Cannot do append. Tries to overwrite the DVD on the second session. Restores only the last backup. OMG!

TotalRecovery

Cannot write to DVD.

Ultrabac

Ultrabac specializes in server software, so I had high hopes they had competent engineering. The setup was annoying as the UI was very clunky, and demanded I supply a User Account with a non-blank password. Most annoying.

There was no file-filtering capability. The UI layout was extremely unfriendly. For example, it was nearly impossible to figure out how to change a project’s backup properties. The tool’s awareness of full vs. incremental backup seemed to consist of a dropdown where one selected “all files” or “Modified”. I can handle unfriendly UIs but this was downright hostile.

And then I did a 2nd incremental backup to DVD and discovered there was no “Find files” search capability as part of the restore. Far far worse, I had to look at each session individually. Imagine having to look through 300 sessions to find the one where you stored your file? And then I noticed that the help files were different from my own. In the help files, the “Database” icon was activated, and you could click that to do multi-session backup (I think; the writeup was pretty poor so I wasn’t positive). But then following that link, I discovered I needed to install MYSql or SqlServer-Express to have this capability. Granted this is free software, but really! I actually started down this part, but got into trouble on the configuration (something I’m actually pretty experienced with) and eventually said the hell with it after an hr. Sorry, Ultrabac, but you’re too high maintenance.

Novastor / Novabackup

Novabackup appears to have all the features and is only $45. The initial look and feel is promising, and it’s feature set is practically complete. My review is confused. When I reviewed this two weeks ago, I initially liked it and then I had a fatal problem in the backup: Some files were missing, even though the logs said everything worked fine. So I made a small writeup and moved on. A few hrs later I accidentally lost my writeup (never trust a wordview draft apparently), and so I cannot remember exactly the circumstances. However, more recently I tried again and it is working fine. About the only missing feature I’d really like is a file-search capability in the restore. With 2TB of video, sometimes I need to file a file and all I have is a fragment of a name.

So provisionally, I am going to give Novabackup another chance. It has an FTP capability. The system restore looks good (although untested; but see below). The DVD archive seems ok, except for the one problem I cannot reproduce.

Back to Retrospect

If it isn’t obvious, I was really depressed at how bad all this software is (Novabackup on provisional probation). Seemed like each tool had some brain-dead missing feature or just sloppily coded something. Nothing as good as I had with Retrospect. So I decided to see if I could eliminate my DVD woes by using a different computer. I have a dell laptop with windows XP and a newer internal dvd sitting idle. Perhaps this will work?

Well, the good news is that it seems to. Sometimes problems only show up after a while, but I have had a full day’s worth of incremental DVD archiving and it all worked solidly. So perhaps I don’t need any new backup software after all. As of now, my primary archiving to DVD has been moved to the more reliable computer and continues to use Retrospect. However, I plan to buy the $45 Novabackup and continue to test it as an eventually successor.

UPDATE: Back to Novabackup

As I said above, backup to dvd seems to be going smoothly now. I have to see how it works when I experience a crash in the middle of an operation before I can give final approval. But moving on to restore, it is confusing. The restore button shows you a “restore files by device” window. But who the hell wants that? I want to restore a backup, not a device! And devices are hardware and hardware dies. What if I write to one DVD and want to restore to a different DVD. Will it still work?

Anyway, expanding the device I happened to store my backup on, I see every backup session. I don’t want that though, as I’ll have hundreds of sessions eventually.

After a while I noticed a “To Time Mode”, and this shows you some more options, i.e., “Restore Files By Time”. At least I can restore a folder from a device to a certain point in time. But what if I archived that folder to multiple devices? Again, I want my MTV and I want my “Restore Files By Backup”. Who would want anything else? I don’t get it.

Worse, I finally ran into a bug. I had written three sessions to DVD. Came time to restore and the time slider showed only two. Simple bug their code and probably quite easy to fix. I have a support ticket with the report and we shall see. More later.

 

UPDATE: 9/16/2012

Well the Novabackup tech support response went into my “bulk” folder so I missed it until now. The response may work. Problem is that the tryout deadline has passed so I cannot run any more tests. More later…

How to export data from a SQLServer database into a sql file

This just cost me an hr or two and was a very frustrating search, so without further ado:

You are working with SQLServer. You need to export the data in 1 or more (all) tables. You don’t want to use the backup database approach. You don’t want to use text files. Rather, you want to use SQL insert statements.

Forget ExportSQLCE  link here A very nice tool, but it only works on sql compact database files.

Forget all those shitty commerical apps that want you to pay $60 or more to do this.

Forget the SQLServer documentation. ARE YOU KIDDING ME!? No, unfortunately not. A bunch of useless nonsense about using BCP.

The solution is to use the “Generate Scripts” menu option in SQL Server Management Studio and READ THE VERY FINE PRINT as documented HERE

http://stackoverflow.com/questions/2321052/get-script-of-sql-server-data

That teeny tiny option in the Advanced button. That is where it is.

RESTful applications (good idea) and interface contacts (required good idea)

On first and second impression, RESTful concepts do not appear to be consistent with application “contracts”. In fact, they appear to do away with them. Abandon the awful soaps and wsdls!

But “contracts” matter. At the very least, documenting the data structures. You won’t get very far if neither the supplier and receiver have a “good” way of knowing what they’re exchanging.

Sure wsdls suck, but don’t pretend you don’t need something else in their place.

3 months into the question (which I should have answered by now, since I’m developing a RESTful web application), I found the first article today that gave me a deep “aha”.

Definitely not the final answer, and really only a sketched out idea, but it immediately goes deeper than anything else I’ve seen.

http://www.infoq.com/articles/subbu-allamaraju-rest